Behavioral and Neural Evidence of Incentive Bias for Immediate Rewards Relative to Preference-Matched Delayed Rewards

Abstract

Several theories of self-control [including intertemporal bargaining (Ainslie, 1992) and self-signaling (Bodner and Prelec, 2001)] imply that intertemporal decisions can be more farsighted than would be predicted by the incentive associated with rewards outside a decision context. We examined this hypothesis using behavior and functional neuroimaging. First, subjects expressed preferences between amounts of money delayed by 4 months and smaller amounts available that day. This allowed us to establish “indifference pairs” individualized to each participant: immediate and delayed amounts that were equally preferred. Participants subsequently performed a reaction time functional magnetic resonance imaging task (Knutson et al., 2001a) that provided them with distinct opportunities to win each of the rewards that comprised the indifference pairs. Anatomical region of interest analysis as well as whole-brain analysis indicated greater response recruited by the immediate rewards (relative to the preference-matched delayed rewards) in regions previously implicated as sensitive to incentive value using the same task (including bilateral putamen, bilateral anterior insula, and midbrain). Reaction time to the target was also faster during the immediate relative to delayed reward trials (p < 0.01), and individual differences in reaction time between immediate versus delayed reward trials correlated with variance in magnetic resonance signal in those clusters that responded preferentially to immediate rewards (r = 0.33, p < 0.05). These findings indicate a discrepancy in incentive associated with the immediate versus the preference-matched delayed rewards. This discrepancy may mark the contribution of self-control processes that are recruited during decision-making but that are absent when rewards are individually anticipated.

Introduction

Relative to other species, human exhibit extraordinary willingness to forgo smaller sooner rewards (SSs) to obtain larger later ones (LLs). This may be related to distinct mechanisms of self-control that are engaged during decision-making. One account links self-control to a hypothesized recursive feedback loop between one's present choices and their anticipated future behavior (Ainslie, 1992), an idea that has been formalized in economic modeling (Bodner and Prelec, 2001; Bénabou and Tirole, 2004) but that hypothesizes no neurophysiological substrate. Another account equates self-control with the intentional suppression of goal-inappropriate prepotent responses, which is hypothesized to depend on inhibitory pathways between the lateral prefrontal cortex and the basal ganglia (Barkley, 1997; Jentsch and Taylor, 1999; Goldstein and Volkow, 2002). Still another hypothesis suggests that self-control involves the alteration of value signals that results from effortful processing of long-term contingencies and that is primarily dependent on the dorsolateral prefrontal cortex (Hare et al., 2009).

Common to these accounts is the notion that self-control involves distinct processes engaged during decision-making that, in the case of intertemporal choice, may attenuate the tendency to discount delayed rewards. Concretely, if self-control processes specific to decision-making affect choice, then it follows that the individual might choose, for example, $50 delayed by 4 months over an immediate $40, despite it being the case that, when the two expectancies are evaluated individually outside of a decision context, the valuation of the immediate $40 is the higher of the two alternatives. If this is in fact the case, it entails an important challenge that must be met by any neuroeconomic model of intertemporal choice.

There are at least three ways that incentive may be measured in a non-choice context. First, individuals can be asked to introspect on the motivation associated with single rewards. Second, a behavioral correlate of incentive [e.g., speed on a reaction time (RT) task] can be measured while single rewards are pursued. Third neural correlates of incentive can be measured during the pursuit of single rewards. In the present study, we included each of these approaches. We were particularly interested in testing the hypothesis that, when immediate and delayed rewards are encountered in isolation, behavioral and neural markers would indicate greater incentive for the immediate, even when those alternatives are matched for value as inferred from preferences. Such a demonstration would be consistent with conceptions of self-control as involving processes that occur during decision-making and that result in reduced delay discounting. Moreover, if a discrepancy between choice and non-choice incentive value were quantifiable, it could provide a starting point for operationally defining the extent to which self-control affects choices for specific individuals and specific decisions.

Materials and Methods

Subjects.

Forty-three healthy volunteers participated in the study. All subjects gave written informed consent, and the experiment was approved by the Institutional Review Board of the University of Southern California. Before enrolling, volunteers were screened for physical and neurological disorders (using self-report questionnaires) and for current Axis I psychiatric disorders, including substance abuse and dependence (as assessed by the Mini International Psychiatric Interview). Of the 43 subjects, three were excluded from analysis because of failure to reach stability criteria in the adaptive delay discounting task (described below). In addition, one subject was excluded subsequent to scanning because of history of stroke (not reported during initial screening), one subject was excluded because of a neurological abnormality observed (severe ventricular enlargement), and another subject was excluded because of an operational error during scanning. Among the 37 subjects included in analyses, 19 were female. Ages ranged from 21 to 44 years (mean, 32.1 ± 6.9 years).

Subjects were informed that they could win bonuses up to $160 during the course of their participation and that these bonuses would be awarded in the form of Visa gift cards that they would receive at the end of the session. They were further instructed that some available bonus earnings would be delayed, and, if they won delayed bonus earnings, the Visa gift cards would not register that money as available until the specified date. Finally, subjects were informed that, if they won delayed earnings and lost the Visa gift cards before the specified date, they would be provided with a replacement card.

All subjects first completed a computerized version of the Monetary-Choice Questionnaire developed by Kirby et al. (1999). Subjects were presented with a fixed set of 27 choices between smaller immediate rewards (ranging from $11 to $80) and larger delayed rewards (ranging in amount from $20 to $85 and in delay from 7 to 186 d). These responses were used to derive an initial estimate of the subject's level of delay discounting using the following hyperbolic discount function:

in which V is value, A is amount, D is delay in days, and k is the fit parameter that quantifies level of discounting, with k = 0 indicating no delay discounting and higher values of k indicating steeper discounting (for details of the estimation procedure used, see Monterosso et al., 2007). This model makes the simplifying assumption that value scales linearly with amount. In the present context in which participants choose between sooner smaller and later larger rewards, unmodeled concavity in the actual association between amount and value results in inflation of best-fit values for the k parameters (Pine et al., 2009).

The individual estimate of discounting from the Monetary-Choice Questionnaire was used as the starting value in a second computer-administered delay discounting choice task, this one using adaptive questioning to gain precision in determining indifference pairs of rewards. On each trial, subjects were presented with a choice between an LL and an SS. Participants were informed that one trial would be selected from this procedure and that they would receive the alternative they selected on that trial. The delay of the LL was always 120 d, and the SS was always zero delay. For half the trials, the LL was $28 ± $7 (“low”) and for half the trials, the LL was $53 ± $7 (“high”). The magnitude of the SS was initially generated by computing what would be an amount of equal value to the LL based on the fit parameter for the subject (k value) derived from responses on the Monetary-Choice Questionnaire, as modeled using Equation 1. On each trial, if the participant chose the SS alternative, then the k parameter associated with that reward was adjusted upward a quarter step on a log₁₀ scale, and, consequently, the SS on the next trial in which that LL appeared was lower. Conversely, if the participant chose the LL, then the k parameter associated with that reward was adjusted downward (again, a quarter step on a log₁₀ scale), resulting in a higher SS on the next trial in which that LL appeared. This adjusting procedure continued until subjects reached stability for both reward pairs, with stability operationalized as a window of eight trials in which k values did not deviate by more than two steps. Participants who did not reach this criterion after 8 min (n = 3) were excluded from the study. The final indifference pairs were then generated using the geometric mean of the k values for the eight trials during which stabilization was achieved (separately for the low and high LL amounts). In this way, two indifference pairs were established for additional investigation during functional magnetic resonance imaging (fMRI). For analyses that required a single discount parameter estimate for each participant, we used the geometric mean of these two k parameter estimates.

Monetary incentive delay task.

In the next step of the study, participants performed a variant of the Monetary Incentive Delay (MID) task (Knutson et al., 2001a). Before the scan, participants were trained to associate each of the two SSs and two LLs comprising the derived indifference pairs with each of four colored shapes (LLs were always $28 in 4 months and $53 in 4 months, and SSs were individualized to the participant's performance.) The pairing of the colored shapes with rewards was counterbalanced across participants. Subjects completed a computerized memory training program in which the four pairs of colored squares and corresponding “prizes” briefly appeared on the screen, one at a time, and were instructed to memorize the pairs. Next, subjects completed a memory test, during which the colored squares flashed on the screen, and subjects reported the corresponding prizes. Subjects were asked to provide the prize in the appropriate format (e.g., amount followed by delay or vice versa depending on counterbalanced order) and received feedback about their responses. After satisfactory completion of the memory test, subjects completed a practice version of the task, similar to the fMRI version.

Each trial of the MID task began with the appearance of a colored shape that indicated which of the four rewards was available on that trial. After an anticipation period of between 4 and 4.5 s subsequent to presentation of the available reward stimulus, either a target appeared (the character “+”) or the words “no target” appeared (50% of the time). If the target appeared, the participant was required to respond as quickly as possible with a button press to win on the trial. Participants were instructed that their likelihood of winning would be greater if they responded faster, although in reality, outcome was predetermined to optimize orthogonality between anticipation and outcome periods (with the exception of RTs >500 ms, which were always scored as too slow, to avoid suspicion). To optimize power, an exponential distributed intertrial interval with mean of 2 s was used. Unlike the standard MID task, the colored shape remained on the screen until the target (or “no target” message) appeared. The critical epoch for analyses was the 4–4.5 s during which the participant was cued to the possible reward and was readying to try to obtain it. Following previous work with the task (Knutson et al., 2001a), we refer to this throughout as the “anticipation period.” In each run of the task, each of the four targets was presented 16 times. Each participant completed two runs of the task. Participants were instructed that one target trial would be selected from each run, and any money that they won on these trials would be paid, again using Visa cards, with credit activated at the specified date. If their response was not sufficiently fast on the target trial selected, then they did not win a bonus for that run of the task.

fMRI acquisition.

fMRI data were collected using 3T Siemens MAGNETOM Tim/Trio scanner with a standard birdcage head-coil in the Dana and David Dornsife Cognitive Neuroscience Imaging Center at University of Southern California. For each participant, sagittal images (256 × 256 × 176) with 1 × 1 × 1 mm³ resolution were obtained by a T1-weighted three-dimensional magnetization prepared rapid gradient echo (MPRAGE) sequence [inversion time, 900 ms; repetition time (TR), 1950 ms; echo time (TE), 2.26 ms; flip angle, 90°]. Functional scanning used echo planar imaging sequence (TR, 2000 ms; TE, 30 ms; flip angle, 90°; field of view, 192; in-plane resolution, 64 × 64) with prospective acquisition correction (PACE), which helps reduce head motion during data acquisition. Thirty-two axial slices were used to cover the whole cerebral cortex with no gap, and the slices were positioned along anterior commissure–posterior commissure plane.

fMRI analysis.

fMRI data processing was conducted using FEAT (fMRI Expert Analysis Tool) version 5.98, part of the Oxford University Centre for Functional MRI of the Brain (FMRIB) Software Library (www.fmrib.ox.ac.uk/fsl). The first four volumes before the task were automatically discarded by the scanner for T1 equilibrium. For preprocessing, the head movement that was not captured by PACE was corrected in three dimensions by MCFLIRT (Motion Correction using FMRIB's Linear Image Registration Tool) (Jenkinson et al., 2002). Six motion parameters were added into the general linear model (GLM) to explain variance in signal related to head motion. Data were temporally filtered by a high-pass filter with 100 s cutoff and spatially smoothed by a Gaussian kernel of full-width at half-maximum of 5 mm. The preprocessed data were then submitted to a GLM that was used to analyze the contributions of experimental factors to blood oxygen level-dependent responses. All within-subject statistical analyses were performed in native image space, and then the statistical maps were transformed into standard space before high-level (group) analysis. The transformation into standard space was performed in two steps: echo planar images were first aligned to the participant's own MPRAGE structural scan, and then the image was normalized into standard space [Montreal Neurological Institute (MNI)] using affine transformation (Jenkinson and Smith, 2001).

Our primary analyses targeted brain signal changes during the anticipation period. Because this period ended at the onset of the target stimulus or the onset of the stimulus indicating no target, all trials could be used in analysis of the anticipation period, regardless of the trial outcome, resulting in good statistical power. The subsequent variation relating to the target and the outcome of the trial allowed us to better isolate anticipation period effects by reducing covariation with effects related to subsequent events. We performed two variants of this analysis. In the first variant of this analysis, there were 13 events modeled: four events during the anticipation period including high–immediate, low–immediate, high–delay, and low–delay, and nine events during the feedback period including high–immediate–win/loss, low–immediate–win/loss, high–delay–win/loss, low–delay–win/loss, and no target. Each event was convolved with double-gamma hemodynamic response function, and temporal derivatives were added as a covariate of no interest to improve statistical sensitivity. Null events were not modeled. In the second variant of this analysis, the anticipation epoch was modeled using three parameters: value (high or low), discount fraction (the participant-specific denominator in Eq. 1), and amount (the undiscounted monetary amount available). Both of the latter variables were orthogonalized to value, and amount was additionally orthogonalized to discount fraction. It is important to note that the value parameter is inferred from preference data. If there is a shift in value in the non-choice context that is a function of delay (as hypothesized), regions tracking this shift would be associated with the discount fraction rather than value parameter.

For both above models, a cross-run high-level analysis was performed using a fixed effects model by forcing the random effects variance to zero in FLAME (FMRIB's Local Analysis of Mixed Effects) (Beckmann et al., 2003). Results were input to group-level analysis using FLAME stage 1 (Beckmann et al., 2003; Woolrich et al., 2004; Woolrich, 2008).

Region of interest analyses.

Region of interest (ROI) analyses were performed. Regions were selected based on previous findings associated with the MID task (Knutson et al., 2001a, 2005). Four of the selected ROIs (bilateral putamen, thalamus, caudate, and nucleus accumbens) were defined based on the automated segmentation tool FIRST (FMRIB's Integrated Registration and Segmentation Tool), which is specifically designed to classify subcortical structures (Patenaude et al., 2007). Because this tool is not applicable to the additional ROIs selected (the left and right anterior insula, the midbrain, and the supplementary motor area), we adopted an alternative strategy for these regions, drawing 6 mm spheres around the peek coordinates within the region, reported in a previous study that included a contrast isolating sensitivity to reward magnitude (Knutson et al., 2005) (coordinates converted into MNI space.) To examine whether reward magnitude and immediacy affected MR signal change in these regions during the anticipated period, extracted β values for each of the four rewards were subjected to repeated measure analysis with magnitude (high vs low rewards), delay (immediate vs 4 months delay), and brain region included as within-subject variables. To examine the possibility that particular regions might show more sensitivity to delay and others might show more sensitivity to magnitude, we repeated the analysis, with magnitude and delay recoded as two levels of a within-subject variable (“dimension”), each of which in turn included two levels (high and low, and immediate and delayed). An interaction between dimension and region would provide evidence of differential sensitivity to magnitude and delay.

Whole-brain analyses.

The primary whole-brain analysis was based on the above model in which the anticipation epoch was modeled with four separate events corresponding to the four rewards. In this, we compared preference-matched immediate and delayed rewards (immediate–delayed and delayed–immediate), thresholded using cluster detection statistics, with a height threshold of Z > 2.3 and a cluster probability of p < 0.05 corrected for search space (Worsley, 2001). Contrasts were also performed isolating sensitivity to reward magnitude (high vs low and low vs high) across the entire brain. Exploratory conjunction analyses were performed examining the overlap between high–low contrasts (this time without correction for multiple comparisons) and both immediate–delay and delay–immediate contrasts (without search space correction). Because visualization related to this analysis was not corrected for multiple comparisons, we do not use it to support hypothesis testing but include it because the observed pattern with this less conservative thresholding is, we think, informative.

A parametric analysis was also performed, as described above, in which value (high or low), discount fraction (the participant-specific denominator in Eq. 1), and amount (the actual monetary amount available) were used to model change in signal during anticipation. In this analysis, we were particularly interested in whether the discount fraction predicted response during anticipation after variance related to value was modeled.

Subjective ratings.

Subsequent to the completion of the task, subjects were instructed to rank the four rewards according to the following instructions: “Rank the prizes in terms of how each made you feel at the moment you were going for them during the game. If there are any that are exactly tied, you can give them the same rank . … This is a little different than asking you which one you would choose. Don't worry about which prize you would choose if you compared them, just think about how each made you feel at the moment you were going for the prizes.”

Post-MID choice task.

To test for the presence of systematic drift in discounting, a subset of participants (n = 15) completed a choice task after the MID task. These participants were presented with 24 choice trials in which alternatives were value matched based on the discount parameter derived previously in the adaptive choice procedure and 24 choice trials that were mismatched based on the same previously derived discount parameter. These mismatched trials were generated by creating indifference pairs based on a k fit parameter estimate (Eq. 1) that was one log unit larger (50% of trials) or one log unit smaller (50% of trials) than the participant's actual fit parameter estimate.

Results

Intertemporal choice task

Across participants, the immediate reward amounts that formed indifference pairs with $28 delayed by 4 months ranged from $3 to $25, with a median of $13 (corresponding to k = 0.010) (Eq. 1). Across participants, the reward amounts that formed indifference pairs with $53 delayed by 4 months ranged between $6 and $52, with a median of $28 (corresponding to k = 0.007).

Behavioral results for the monetary incentive delay task

Median reaction time data for each subject on the MID task were modeled using repeated-measures ANOVA with magnitude and delay included as within-subject independent variables. Magnitude was coded as high for both $53 delayed by 4 months and for the immediate amount that was equally preferred to it (individualized to the participant). Magnitude was coded as low for the $28 delayed by 4 months and for the immediate amount that was equally preferred to it (also individualized to the participant). RT was faster for the high magnitude trials than the low magnitude (F_(1,36) = 4.38, p < 0.05) and faster for the immediate reward trials than for the (preference-matched) delayed reward trials (F_(1,36) = 9.1, p < 0.01) (Fig. 1).

Figure 1.

Mean and SE of individual median reaction time by condition on target trials (computed as distance from the overall median for each participant). Based on repeated-measures ANOVA, RTs were significantly faster for both the high versus low pair and for the immediate versus delayed rewards.

Neuroimaging ROI analyses

β values were extracted during anticipation for 12 anatomically defined ROIs for each of the four rewards (high–immediate, high–delayed, low–immediate, and low–delayed). These values were subjected to a repeated-measure analysis, with magnitude, delay, and region included as within-subject variables. Significant main effects were observed for amount (F_(1,36) = 6.46, p < 0.05), delay (F_(1,36) = 4.32, p < 0.05), and region (F_(11,26) = 14.7, p < 0.001). When magnitude and delay were recoded as two levels of the variable dimension (as described above), no interactions were observed between dimension and region. Difference scores highlighting the immediacy effect (β values for the two immediate rewards minus β values for the two delayed rewards) and the magnitude effect (β values for the two high rewards minus that for the two low rewards) are presented for each ROI in Table 1 (supplemental Fig. 1, available at www.jneurosci.org as supplemental material). As can be seen, all difference scores were positive (indicating higher β values for immediate relative to delayed rewards and higher β values for high magnitude rewards relative to low magnitude). In 6 of the 12 anatomical ROIs, difference scores were greater than zero at least at a trend level (two-tailed, α = 0.1) for both amount and delay. Two additional regions reached this threshold only for amount, and two more regions reached this amount only for delay. In general, difference scores across different regions were highly correlated for both delay–immediate and high–low (Chronbach's α = 0.95 and 0.92, respectively).

Table 1.

Difference scores highlighting the immediacy effect (β values for the two immediate rewards minus β values for the two delayed rewards) and the magnitude effect (β values for the two high rewards minus that for the two low rewards)

	Magnitude effect(high–low)		Delay effect(immediate–delay)
	β values	p value	β values	p value
Left caudate	4.97 ± 22.06	0.18	4.51 ± 21.64	0.21
Right caudate	7.48 ± 20.35	0.03	4.71 ± 20.11	0.16
Left putamen	4.43 ± 14.48	0.07	7.77 ± 19.92	0.02
Right putamen	5.19 ± 14.23	0.03	6.26 ± 18.4	0.04
Left thalamus	6.13 ± 16.9	0.03	5.80 ± 19.9	0.08
Right thalamus	7.74 ± 16.79	0.008	6.15 ± 20.52	0.08
Left nucleus accumbens	4.61 ± 25.29	0.28	4.27 ± 30.43	0.40
Right nucleus accumbens	7.05 ± 23.23	0.07	1.62 ± 27.32	0.72
Left insula, 6 mm sphereat −30, 20, 2	6.53 ± 23.96	0.11	9.21 ± 23.83	0.02
Right insula, 6 mm sphereat 30, 19, 10	4.34 ± 19.36	0.18	5.66 ± 16.58	0.04
Midbrain, 6 mm sphere at0, −18, −13	7.96 ± 19.03	0.02	8.29 ± 26.05	0.05
Supplementary motor area,6 mm sphere at 0, −24, 50	10.49 ± 31.00	0.047	14.80 ± 35.98	0.017

Neuroimaging whole-brain analyses

We also performed a whole-brain analysis comparing MR signal during anticipation of immediate rewards with signal during anticipation of the preference-matched delayed rewards. We used cluster detection statistics with the height threshold of Z > 2.3, p < 0.05 cluster-level correction for search space. As shown in Figure 2, immediate rewards recruited greater signal change than delayed rewards in left caudate, putamen (bilateral), insula (bilateral), left pallidum, supramarginal gyrus (bilateral), anterior cingulate cortex, and supplementary motor area (Table 2, Immediate–delayed). Immediate rewards recruited significantly less activation in clusters within the precuneus and occipital cortex.

Figure 2.

Contrast maps for immediate–delayed (warm colors indicate immediate > delayed, cool colors indicate delayed > immediate). All clusters based on whole-brain analysis with voxel threshold of Z = 2.3 and cluster-level correction of p < 0.05. Findings for immediate > delayed include left caudate, putamen (bilateral), insula (bilateral), left pallidum, supramarginal gyrus (bilateral; SMG), anterior cingulate cortex (ACC), and supplementary motor area (SMA). Findings for delayed > immediate include clusters within the precuneus and occipital cortex. IFG, Inferior frontal gyrus; OP, occipital pole.

Table 2.

Whole-brain analysis results

	Whole brain	x, y, z	Maximum Z
High–low	Right caudate	12, 14, 4	2.83
	Right thalamus	8, −4, 10	3.15
	Left thalamus	−12, −22, −4	3.34
	Lateral occipital cortex	−18, −84, 12	3.5
	Occipital pole	−8, −98, 10	3.37
Immediate–delayed	Right insula	44, 12, −4	4.17
	Right putamen	26, 14, −2	3.39
	Right supramarginal gyrus	64, −34, 28	3.37
	Right inferior frontal gyrus	54, 18, 4	2.86
	Left insula	−40, 16, −4	4.03
	Left putamen	−16, 8, −8	3.54
	Left caudate	−10, 12, 2	2.9
	Left pallidum	−16, 4, 2	3.0
	Left supramarginal gyrus	−60, −34, 28	3.12
	Anterior cingulate cortex	0, 22, 32	2.81
	Supplementary motor cortex	0, 2, 56	4.24
Delayed–immediate	Occipital cortex	0, −94, 10	4.07
	Precuneous cortex	0, −56, 24	3.26

We also contrasted high and low rewards in a whole-brain analysis. As shown in Figure 3, cluster-level significance was reached in the right caudate, thalamus (bilateral), lateral occipital cortex, and occipital pole (Table 2, High–low). Higher rewards did not recruit significantly less activity than lower rewards in any region. As an alternative approach, we also modeled the data during anticipation parametrically, as described above. The value and discount fraction predictor variables used in this additional model yielded similar activation maps to those associated with immediate–delay and high–low contrasts (supplemental Fig. 2A,B, available at www.jneurosci.org as supplemental material).

Figure 3.

Contrast maps for high–low rewards (red indicates high > low; no clusters were observed for low > high). Findings for high > low include right caudate, thalamus (bilateral), lateral occipital cortex (OC), and occipital pole (OP).

For exploratory purposes, a conjunction analysis was performed that identified all voxels that evidenced greater activity (Z > 2.3 voxel height threshold, without cluster correction) in both (1) high > low and (2) either immediate > delay (Fig. 4, shown in red) or delay > immediate (Fig. 4, shown in blue.) The overlap between voxels identified in high–low and immediate–delay was widespread within subcortical structures previously associated with incentive on the MID task (including brainstem, right caudate, bilateral pallidum, bilateral putamen, right thalamus, bilateral insula, and left supramarginal gyrus). There was one large cluster in the visual cortex in which delay > immediate overlapped high > low voxels. We hypothesized that this activity in visual cortex was likely related to visual attention that might be heightened for different reasons in the two contrasts. To explore this hypothesis, a functional connectivity analysis was performed in which we used activity in the cluster that overlapped both high > low and delayed > immediate contrasts as a seed to predict activity throughout the rest of the brain, contrasting connectivity with the seed in the immediate versus delayed trials. As shown in supplemental Figure 3 (available at www.jneurosci.org as supplemental material), a significant differential functional connectivity effect was observed (p < 0.05, cluster corrected for search space) in regions overlapping with those previously associated with incentive on the task (putamen, anterior insula, thalamus, and pallidum). Specifically, there was significantly greater association with the occipital cortex seed region when rewards were immediate relative to when rewards were delayed.

Figure 4.

Conjunction whole-brain maps for high–low and immediate vs delayed (red indicates immediate–delayed, blue indicates delayed–immediate; p < 0.05, uncorrected for each). R, Right; L, left; OP, occipital pole; OC, occipital cortex; SMG, supramarginal gyrus.

Post hoc correlational analysis indicated no correlation between individual variance in delay discounting as measured by the choice procedure and either variance in the effect of immediacy on RT (r₍₃₇₎ = −0.19, p = 0.26) or immediacy on MR signal in regions of interest (r₍₃₇₎ = −0.10, p = 0.56). The immediacy effect on RT was significantly correlated with the effect of immediacy on MR signal in a priori ROIs (r₍₃₇₎ = 0.35, p < 0.05).

β values in the occipital cortex for delayed > immediate were not correlated with discounting (r₍₃₇₎ = −0.14, p = 0.40) or the immediacy effect on RT (r₍₃₇₎ = −0.08, p = 0.62) but were inversely correlated with β values within clusters identified in the immediate–delayed contrast (r₍₃₇₎ = −0.49, p = 0.002).

Subjective ranking questionnaire

The mean ranking for the larger immediate reward was 1.56 ± 0.84, for the larger delayed reward was 1.58 ± 0.77, for the smaller immediate reward was 3.14 ± 0.90, and for the smaller delayed reward was 3.28 ± 0.78. When subjected to a repeated-measures ANOVA, magnitude (high vs low) was a highly significant predictor of ranking (F_(1,35) = 214.0, p < 0.001), and delay was not a significant predictor of ranking (F_(1,35) = 0.16, p = 0.69). Although there was not an indication of a group-level effect of immediacy on ranking for the preference-matched pairs, we examined the relation between individual differences in ranking based on immediacy and individual variance in the effect of immediacy on RT and MR signal change during anticipation. To do this, we computed a difference score for the mean ranking of delayed rewards minus the ranking of immediate rewards. Although this index of the subjective effect of immediacy (given matched preference) did not predict variance in the effect of immediacy on MR signal difference (r₍₃₆₎ = 0.27, p = 0.11), the association with the effect of immediacy on RT was significant and in the anticipated direction (r₍₃₆₎ = 0.36, p < 0.05).

Post-fMRI delay discounting choice reassessment

In the post-MID task reassessment of delay discounting that was administered to a subset of participants (n = 15), in trials generated to be at the participant's indifference point, the mean percentage choice of the SS was 50.3 ± 12.7%. Among trials generated to be mismatched in value, participants chose the option generated to be of higher value (based on their individual estimated discount function) on 93.5 ± 8.5% of trials.

Discussion

We compared behavior and brain response during anticipation of long delayed (4 months) and smaller, equally preferred immediate monetary rewards (indifference pairs). The data unequivocally indicate observable differences between the conditions; when anticipating immediate rewards relative to delayed rewards, responses to target stimuli were faster and neural activity was greater in a network of regions previously implicated in incentive during the task. This was especially evident in the superior portion of the anterior insula (contiguous with frontal operculum) and putamen, in which findings were bilateral and evident in anatomical ROI and whole-brain analyses. In addition to its established association with feeling states (Damasio et al., 2000), the anterior insula is implicated in executive function tasks (Wager et al., 2004) and may be associated with affective signals accompanying mental effort (Wager and Barrett, 2004). A similar locus of activation was reported using the MID task in contrasts identifying sensitivity to the presence (Knutson et al., 2001b) and magnitude (Knutson et al., 2005) of reward. The putamen is implicated in reinforcement learning (Packard and Knowlton, 2002) and especially in preparation of motor responses (Alexander and Crutcher, 1990). Using the same task, activity in the putamen has also been repeatedly reported during anticipation of rewards (Knutson et al., 2001a) and appears to be preferentially recruited during positive incentive (Knutson et al., 2005). In addition, evidence of differential activity during immediate relative to delayed reward trials was observed in most regions implicated previously as sensitive to reward during the MID task, including the brainstem, pallidum, caudate, supplementary motor area, supermarginal gyrus, and anterior cingulate cortex. The fact that participants who completed a choice task subsequent to the MID task preferred the SS on 50.3% of trials designed to be at the participant's indifference point indicates that findings were not the product of drift toward greater discounting during the experiment.

Although these data demonstrate that there was something different during anticipation of preference-matched immediate versus delayed rewards, several reasonable interpretations warrant consideration. One possible basis is that subjects, on average, valued the immediate rewards more than the delayed rewards, despite their having been matched based on revealed preference. This is consistent with conceptions of self-control that posit distinct mechanisms engaged during decision-making that generally shift preference toward greater future orientation. Accordingly, the more immediate reward within a decision-based indifference pair would be, on average, of higher value than the more delayed reward if each was encountered in a context in which self-control was not operative, as arguably is the case in the MID task. Before returning to this interpretation, we consider two alternative accounts.

Alternatively, it could be hypothesized that the differences in MR signal result from differences in the representation of subcomponents of expected value; regions more active in the immediate reward condition might be specifically sensitive to the dimension of immediacy rather than a difference in overall incentive value. To maintain this as an explanation of observed findings, differential activation should be absent in regions sensitive to overall value. The lack of differences observed in the nucleus accumbens and ventromedial prefrontal cortex could be viewed as supportive. However, in our repeated measures analysis of signal in ROIs implicated in incentive in previous work with the MID task, we found no evidence of specificity in sensitivity to reward magnitude versus reward immediacy. That is, we observed no statistical evidence of divergence within the examined network in sensitivity to the two dimensions. Furthermore, differential activation was observed in regions that are important to the execution of the experimental task but that are not plausibly substrates representing subcomponents of expected value (e.g., the supplementary motor area). If some of the observed findings based on the immediate–delay imaging contrast are related to subcomponents that contribute to value but that imply no overall divergence in value across the conditions, then an additional explanation would have to be provided to explain activation differences in regions that are unlikely related to subcomponents of expected value, as well as to the observed difference in reaction times to the targets. The correlation between the immediacy effect across brain regions (Chronbach's α = 0.95) and between fMRI data and reaction time (r = 0.35, p < 0.05) make this prospect less convincing. It may be important to note that the MID task uses a small set (here four) of highly familiar rewards; it is possible that neural recruitment associated with value differs here from situations in which novel rewards are encountered.

A second alternative account is that observed findings reflect a conditioned response, whereby stimuli signaling immediacy potentiate a motor response (rather than discrepant valuation). Perhaps because of semantic overlap or learning history, cues of immediacy could prime action, and cues of delay could prime inaction. The widespread overlap between voxels sensitive to amount and delay (Fig. 4) and the absence of any statistical evidence of an interaction involving these predictor variables fails to lend support. Furthermore, although visual inspection of the thresholded images invites conjecture that the imbalance in incentive observed between immediate and delayed rewards has some particular association with the potentiation of action, signal change data across ROIs (Table 1) (supplemental Fig. 1, available at www.jneurosci.org as supplemental material) does not suggest regional specificity in sensitivity to the two orthogonal independent variables of magnitude and immediacy. It is also worth noting that subjects that reported more favorable subjective rankings for immediate rewards tended to demonstrate greater reaction time superiority during immediate reward trials (r = 0.36, p < 0.05). This association would not be predicted based on the conditioned response interpretation of our findings.

Finally, it could reasonably be suggested that the primary findings might not be related to choice per se but rather to the mere juxtaposition of multiple alternatives. Perhaps having another option for consideration, regardless of whether it has to be chosen against, increases focus on amount rather than on delay. With respect to this possibility, we note that the four rewards we used in the MID task were presented continually in close temporal proximity and so did naturally form a frame for comparison. Also, we know of no a priori basis for supposing that juxtaposition would differentially shift attention to the amount dimension over the immediacy dimension. Nevertheless, the possibility that concurrent juxtaposition shifts valuation toward amount and away from immediacy cannot be ruled out (for evidence that choice itself is a context that can change the value of options, in their case, enduringly, see Sharot et al., 2009).

On balance, we believe the data are best explained as the result of on-average higher incentive value for the immediate rewards relative to the equally preferred delayed rewards. However, one aspect of the data appears inconsistent at first blush. Differential activity was observed in a cluster in the visual cortex for high relative to low and also delayed relative to immediate reward trials. The results of our functional connectivity provide a clue. Functional connectivity to this occipital cortex cluster (plausibly related to heightened visual attention) was significantly diminished when rewards were delayed in a network of regions implicated in incentive during the task (putamen, anterior insula, thalamus, and pallidum). Although high relative to low rewards might differentially recruit visual attention because of differential incentive value (David et al., 2005), the reduced connectivity to basal ganglia activity during delayed rewards suggests heightened visual attention during the delay condition related to something other than value. So, although the basis for greater activation in this region for delayed relative to immediate rewards remains unclear, connectivity results suggest that it is not based on heightened incentive value. It should be noted that, unlike the standard MID task, the incentive stimulus was visually displayed in the present study throughout the anticipation period.

Interpretation of findings

Our findings are consistent with theories that posit that self-control includes engagement of processes during decision-making (as opposed to during valuation more generally). On this interpretation, the heightened incentive for the immediate rewards relative to the preference-matched delayed rewards reflects removal of the typically moderating influence that self-control processes had on the tendency to devalue delayed rewards (or the tendency to exhibit concavity in the association between amount and value, which in this context would similarly result in greater preference for sooner smaller over later larger). This suggests that two factors contribute to response to delay during intertemporal choices. The first factor is the direct effect delay has on incentive value, which we expect is itself complexly determined, and subject to framing effects (e.g., whether the delay is expressed as a waiting period or by specifying the day the reward will be received, as in the study by Read et al., 2005). The second factor consists of self-control processes engaged during decision-making, which tend to push preference toward later, larger rewards. Bringing this dissociation under quantitative analysis may be illuminating. For example, to the extent that farsighted choices are based on self-control processes engaged during explicit decision-making (second of the above factors), the individual may be more vulnerable to short-sightedness in the presence of anything that selectively undermines higher cognitive functions, such as fatigue or distraction. Alternatively, some framing manipulations could selectively influence the direct incentive value of rewards (first of the above factors) but have influence over high-level decision-making if the decision-making process explicitly ignores the frame. Although the present work does not investigate the mechanisms underlying self-control, by examining valuation outside of a decision context, these data provide support for the conceptions of self-control as entailing one or more processes that are engaged during decision-making and that generally result in less shortsightedness in behavior than would be predicted by the individual incentive value of immediate and delayed rewards.