Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 May 1.
Published in final edited form as: J Neurosci Psychol Econ. 2010 May 1;3(1):15–26. doi: 10.1037/a0017252

Now or later? Striatum and insula activation to immediate versus delayed rewards

Marc Wittmann 1, Kathryn L Lovero 2, Scott D Lane 3, Martin P Paulus 4
PMCID: PMC2907916  NIHMSID: NIHMS149695  PMID: 20657814

Abstract

Neuroimaging studies on delay discounting tasks that use reward delays ranging from minutes to days have implicated the insula and striatum in the processing of inter-temporal decisions. This study aimed at assessing whether these brain regions would also be involved in decision-making when subjects have to wait through the delays within the range of seconds. Employing functional magnetic resonance imaging (fMRI) in thirteen healthy volunteers, we repeatedly presented monetaryoptions with delays that differed within the range of multiple seconds. Using a region of interest approach, we found significant activation in the bilateral anterior insula and striatum when subjects chose either the immediate (smaller) or delayed (larger) option. In particular, insular activation was observed after the response and the delay, when the outcome of the immediate or the delayed choice was shown. Significantly greater activation was observed in the ventroanterior striatum while subjects chose the immediate, as opposed to the delayed, options, and also after receiving the outcome of waiting through the longer delay option. The evidence presented here indicates that both the ventral striatum and the insula are involved in the processing of choosing delay options as well as the consequences of choices with delays in the seconds’ range.

Keywords: delay discounting, decision making, time perception, reward, fMRI

Time perception is a crucial factor in the evaluation of choices and outcomes. We are often faced with decisions between options bearing immediate and delayed consequences, such as choosing an immediate financial loss in order to gain future retirement security. These inter-temporal decisions can be assessed experimentally by using paradigms that assess the degree of “delay discounting” (Berns et al., 2007; Prelec & Loewenstein, 1997; Bickel et al., 1999; Lane et al., 2003). Delay discounting refers to the phenomenon where a reward available sooner is valued more than a reward available at a later time. In other words, time delays devalue future rewards.

Most delay discounting paradigms require participants to make choices between smaller, more immediate rewards, and larger rewards occurring at a later time. For example, in tasks using either hypothetical or real rewards, subjects have to choose between $10 today versus $20 in two weeks. The degree of preference for the later option is considered to be a function of reward magnitude and temporal delay, where an increase in the delay of the larger reward will typically lead to a decrease in preference for that option. Over time, this discounting pattern has been repeatedly described to fit a hyperbolic function (Ainslie, 1975; Lane et al., 2003; Madden et al., 2003), and more recently, it has been described as a ‘quasi-hyperbolic’ function (Laibson, 1997; McClure et al., 2004). It has been argued that the slope of the discounting curve is attributed to a probabilistic consideration where rewards are considered to be less likely to occur if they are delayed, and thus are discounted strongly (Rachlin et al., 1991). The hyperbolic nature of the slope has also been attributed to the concept that subjects estimate the duration of time in a non-linear fashion (Takahashi, 2005; Zauberman et al., 2009; Wittmann & Paulus, 2009). In addition, the use of hypothetical versus real rewards across tasks also influences choice behavior and the shape of the discounting slope (Lane et al., 2003). In order to maximize total reward outcome, subjects are flexible to adapt their delay discounting behavior to the specific parameters and constraints of the task (Schweighofer et al., 2006).

In recent years, functional magnetic resonance imaging (fMRI) studies have begun to examine the neural correlates of delay discounting in humans. Studies using two-option delayed reward tasks have shown that monetary rewards separated by a delay of two weeks (McClure et al., 2004), and liquid rewards, in thirsty volunteers, separated by several minutes (McClure et al., 2007) lead to similar brain activation clusters. Activation in limbic structures was greater for two choices involving an immediate versus a delayed reward than for two choices where both rewards were delayed. In contrast, lateral prefrontal cortex and posterior parietal cortex were more generally activated irrespective of delay. Since activation of the limbic system was associated with choosing immediate rewards and fronto-parietal cortex regions were associated with choosing delayed rewards, the authors discuss their findings in terms of a dual valuation system with distinct brain regions – the former implicated in a strong orientation to the present moment, and the latter involved in foresight and self-control.

Similar results were found in an fMRI study by Tanaka and colleagues using contingent monetary rewards (Tanaka et al., 2004). In two different task conditions lasting on the order of several minutes, subjects learned to choose between actions leading to small immediate rewards versus actions leading to no rewards. In the same study, subjects also learned to choose options that delivered small immediate losses but yielded a net positive gain at the end of the task. When subjects learned to collect immediate rewards over no rewards, regions of the prefrontal-cortico-limbic network, such as the lateral orbitofrontal cortex and the ventral striatum, were activated. In contrast, when subjects learned to delay monetary gratification instead of immediately cashing in small sums of money, dorsolateral prefrontal and inferior parietal cortices were activated. However, a more recent study (Kable & Glimcher, 2007) challenges the separation of functional systems suggested in the McClure et al. (2004; 2007) and Tanaka et al. (2004) studies. In this study (Kable & Glimcher, 2007), the paralimbic structures of the medial prefrontal cortex, the posterior cingulate, and the ventral striatum were shown to be involved together in the representation of the subjective value of rewards irrespective of whether they could be received immediately or only after a delay.

The Tanaka et al. study (2004) has provided the intriguing suggestion that two brain structures, the insular cortex and the striatum, code the selection of options for immediate versus delayed gratification. Ventroanterior regions of these two brain regions were activated for the immediate choices, whereas dorso-posterior regions were activated when subjects learned to choose delayed rewards. These results suggest that separate pathways within the cortico-striatal circuitry may be involved in processing more immediate versus more delayed time scales of reward prediction. Two recent studies using inter-temporal decision tasks with hypothetical rewards complement such findings that the striatum plays an important role in delay discounting. The first study found ventral striatal activity that correlated with subjects’ preferences for smaller-immediate over larger-delayed monetary rewards in the range of days to years (Hariri et al., 2006). The second study, which varied similar time ranges, found that the participants’ preferences for immediate vs. delayed rewards resulted in different discounting slopes for delays up to one year and for delays longer than one year (Wittmann et al., 2007b). Contrasting activations for these shorter and longer delays has revealed that the neural activation associated with shorter delays was increased in the striatum, specifically the head of the left caudate nucleus and putamen.

Here, we investigated whether inter-temporal decisions in the range of seconds would lead to similar brain activations found in delay discounting tasks with reward delays of longer intervals. There is accumulating evidence that different temporal processing mechanisms operate on separate time scales (Wittmann, 2009). Inter-temporal choices, and their underlying processes, could also depend on the reward delay subjects have to consider when making their decisions. Two very recent studies have varied the wait time before a reward was delivered by using an interval of several second, and having a maximum delay of 13.5 seconds (Luhmann et al., 2008; Gregorios-Pippas et al., 2009). In the present event-related fMRI study, we employed a delay-discounting task where reward delays varying in the range of two to several seconds (4, 16, 32, and 64 s) had to be experienced, meaning participants had to actually wait through the delays via an on-screen countdown, before they could press a button to see how the outcome of their choice affected the running total reward that would be received at the end of the scanning session. We predominantly analyzed brain activation in two specific regions of interest that have been shown to be implicated in the perception of time as well as inter-temporal decision-making – the striatum and the insular cortex. The striatum has repeatedly been implicated as critical for the estimation of time intervals in the range of seconds (Wittmann et al., 2007a; Hinton & Meck, 2004). Likewise, the insular cortex has strong involvement in the perception of time (Craig, 2009; Wittmann, 2009). Since these two brain regions are also differentially involved in reward prediction in the range of minutes (with real rewards) and years (hypothetical rewards) (Tanaka et al., 2004; Wittmann et al., 2007b), we wanted to find out whether the striatum and the insular cortex would also be activated for inter-temporal decisions within the range of seconds with delays that are actually experienced.

Materials and Method

Subjects

Thirteen right-handed, healthy volunteers were interviewed to participate in the study (6 females; mean age 30.2 years, S.D. ± 8.2 years; average education level 15.7 years, S.D. ± 1.2 years). All participants denied a history of drug or alcohol dependence or regular use of prescription medications other than oral contraceptives. Additionally, subjects gave written informed consent, as approved by the University of California, San Diego Human Research Protection Program.

Behavioral Paradigm

During the fMRI run, subjects completed a delay discounting task (see Figure 1), based on a paradigm previously utilized by Lane et al. (2003). Throughout the task, subjects were repeatedly presented with the option of a nearly immediate reward or a delayed reward and were instructed to choose one of the two by pressing a corresponding button on a response box. The immediate option consisted of a reward value of $0.01, $0.06, $0.12, $0.18, or $0.24 available after a constant 2 second waiting period. The delayed option had a constant reward value of $0.24, but the waiting period varied between 2, 4, 16, 32, and 64 seconds. After the selection of an option, subjects waited through the chosen delay period and then were able to press the same button on the response box to see how the outcome of their choice had affected their total reward. The running total of the subject’s earnings was shown at all times in the upper center portion of the display screen. With this design we were able to assess which brain regions are active when choosing the immediate or the delayed option as well as when viewing the accumulating reward at the end of each trial. Subjects were informed that they would receive the final sum at the end of the scanning session.

Figure 1.

Figure 1

Open in a new tab

Subjects were repeatedly presented with a nearly immediate (right side) and a delayed reward option (left side) and instructed to choose one of the two by pressing a corresponding button on a response box. The immediate option consisted of a reward value of either $0.01, $0.06, $0.12, $0.18, or $0.24 available after a 2 second waiting period. The delayed option had a constant reward value of $0.24, but the waiting period varied between 4, 16, 32, and 64 seconds. A running total of the subjects earning was shown at all times in the upper center portion of the display screen.

It is important to note that although there were smaller and the longer delay choices within a given trial, they both carried the same delay to actual reward consumption – after the scan when the subject received a check. Nevertheless, we consider our task to involve similar decisions and collections of rewards as situations where a video game player or gambler makes decisions and experiences delays while accumulating points for redemption at the end of a session. At the end of each trial, subjects receive the consequences of their initial choice and this consequence is hypothesized to be the rewarding component of the task.

The delay periods were presented to subjects with each of the five varying monetary reward options ($0.01, $0.06, $0.12, $0.18, or $0.24) for one delay period (defining one block), before moving on to the next delay period and cycling through the monetary rewards again. The reward values were presented in ascending order within blocks. The inter-trial interval lasted 5 seconds. After cycling through all five delay period blocks, subjects began again at the 2 second delay and repeated the same presentation order until they completed the second round. The order of presentation can be found in the supplemental information (Table S1). For the first five trials the smaller immediate and the larger delay option consisted of a 2 s wait time each. The choice on these trials is obvious since subjects choose the larger reward in all cases. This condition is a control to ensure subjects are attending to the stimuli and choosing rationally. Because the amount of immediate choices affects the number of trials one can complete in a fixed amount of time (8 min 32 s), subjects completed a varying amount of delay period cycles. In order to prevent subjects from devising any reward maximization strategy, they were not instructed about the duration of the run.

fMRI-Data Acquisition

The blood oxygen level dependent (BOLD) fMRI data were collected during the task using a Signa EXCITE 3.0 Tesla-GE scanner (T2*-weighted echo planar imaging (EPI) scans, TR = 2000ms, TE = 32 ms, FOV = 220 × 220 mm, 64 × 64 matrix, thirty 2.6 mm axial slices with a 1.4 mm gap, 512 whole-brain acquisitions). For anatomical reference, a high-resolution T1-weighted image (spoiled gradient recalled (SPGR), TI = 450, TR =8ms, TE = 3ms, FOV = 250 mm, flip angle = 12°, 176 sagitally acquired slices 1×0.97×0.97 mm3 voxels) was obtained during the same scanning session.

fMRI Protocol Analysis Pathway

All image processing and analysis of this event-related fMRI study was done with the Analysis of Functional Neuroimages Software (AFNI) package (Cox, 1996). For preprocessing, EPI images were interpolated to correct for three-dimensional motion, time-corrected for non-simultaneous slice acquisition, and normalized to Talairach coordinates (Talairach & Tournoux, 1988). The event-related time series data for each individual was then analyzed using a multiple regression model. Five regressors of interest were created to measure the neural substrates contributing to each element of the task: (1) the initial five choices, used not for specific analysis contrasts but to account for the early trials in which the delay times were equal at 2 seconds for both immediate and delayed options, (2) the immediate option, when the subject chose the immediate reward, (3) the delayed option, when the subject chose the delayed reward, (4) the immediate collection, when the subject collected the reward for the immediate option, and (5) the delayed collection, when the subject collected the reward for the delayed option. Additionally, three movement-related nuisance regressors were used to account for residual motion (in the roll, pitch, and yaw directions). A regressor was included for filtering out activation attributable to noise. To achieve this, white matter was segmented and the time series was extracted and entered as a regressor to the deconvolution analysis.

The regressors of interest were convolved with a modified gamma variate function to account for the delay and dispersion relating presumed neural activation to hemodynamic changes measured by the BOLD response (Boynton et al., 1996). The AFNI program 3dDeconvolve calculated the estimated voxel-wise response amplitude, and a Gaussian filter with FWHM 6 mm was applied to voxel-wise percent signal change data to account for individual anatomical variations. Moreover, statistical analysis using planned two-tailed t-tests were performed on a series of a priori determined contrasts, which included: (1) the difference between choosing the immediate and delayed option vs. baseline, respectively, (2) the difference between choosing the delayed versus the immediate reward option, (3) the difference between collecting (experiencing) the immediate and delayed reward vs. baseline, respectively, and (4) the difference between collecting delayed versus immediate rewards.

Statistical Analyses

Voxel-wise percent signal change data for the whole brain were entered into a t-test, using the AFNI program 3dTtest, to examine activation during the various conditions of the delay discounting task. A Gaussian filter with full-width half-maximum (FWHM) of 4 mm was applied to the voxel-wise percent signal change data to account for individual variations of the anatomical landmarks. A threshold adjustment method based on Monte Carlo simulations was calculated using the AFNI program AlphaSim in order to guard against identifying areas of false positive activation. Based on these simulations, it was determined that a voxel-wise a-priori probability of 0.05 would result in a corrected cluster-wise activation probability of 0.05 if a minimum volume of 704 μL ml and a connectivity radius of 4.0 mm was considered. Additionally, a priori regions of interest (ROI), defined by the Talairach demon atlas (Lancaster et al., 2000), in bilateral cingulate, striatum, and insular cortex were used as anatomical masks in an ROI based analysis. Within these constrained search regions, a voxel-wise a priori probability of p < 0.05 resulted in a corrected cluster-wise activation probability of p < 0.05 when using a minimum volume of 256 μL, with a cluster connectivity radius of 4.0 mm for an insular, striatal, and cingulate cortex cluster.

Moreover, voxel-wise correlational analyses were conducted between behavioral and fMRI data. Specifically, subjects’ number of immediate choices (number of trials in which the immediate option was chosen divided by the total number of trials) and the average reaction time for a choice was correlated with brain activation (whole brain) to quantify behavioral attributes. Using the statistical package R (Ihaka & Gentleman, 1996), clusters of significant correlation were accepted with a cluster-wise activation probability p < 0.05 and a minimum volume of 704 μL. The percent signal change found in these areas was extracted and used for subsequent scatter plots.

Results

Behavioral Analyses

Traditionally, studies of delay discounting have examined choice patterns by fitting the data to the hyperbolic function V = A / (1 + kD). V represents the subjective value of the reward, or “indifference” point at which the subject prefers the immediate small and delayed large rewards equally. The value A represents the value of the larger reward, D represents the delay to the larger reward, and k represents the rate at which the function declines as the delay increases. This hyperbolic equation was fitted to individual subject data, explained 94% of the variance on average, and resulted in an inter-individual average k value of 0.106 (S.D. = 0.071, range 0.003 – 0.197). Figure 2 shows the hyperbolic function derived by fitting the mean indifference point of all 13 subjects across each delay value. Individual subject data for number trials completed, percent immediate choices, average delay period, k-value, and R-squared value can be found in the supplemental information (Table S2). On average, subjects chose the more immediate options (short delay and smaller reward) on 43.3% (± 17%) of all trials (range = 14.3% – 64.3%). The average decision time to the task was 1.75 seconds (S.D. = 0.32s).

Figure 2.

Figure 2

Open in a new tab

The hyperbolic function derived by fitting the mean indifference point of all subjects across each delay value (2s, 4s, 16s, 32s, and 64s). Indifference points represent the mid-point at which the subjects switched from choosing the delayed reward to the immediate reward.

fMRI Analyses

Immediate and Delayed Options vs. Baseline

The whole brain analysis (p < 0.05, corrected; V > 704 μL) revealed that several brain areas were activated during both the immediate and the delayed choices (see Tables S3 and S4 in the supplemental information). Specifically, bilateral insula, striatum, inferior frontal/middle frontal gyri, inferior/superior parietal lobule, pre/postcentral gyri, precuneus, and cuneus were more active when individuals decided between the immediate or delayed option relative to the baseline (referring to intervals not defined by a specific regressor). Moreover, there was significant deactivation in the left cingulate gyrus during option selection condition relative to baseline. ROI based analysis (p < 0.05, corrected; V > 256 μL) also showed significant activation in bilateral insula and striatum during both immediate and delayed choice conditions (see Figure 3).

Figure 3.

Figure 3

Open in a new tab

ROI based analysis: bilateral caudate and anterior insula exhibited significantly greater activation (p < 0.05, corrected; V > 256 μL) during both immediate (A) and delayed (B) decisions (selecting an option) as compared to baseline.

Immediate versus Delayed Options

When immediate and delayed options were contrasted in the ROI-based analysis (p < 0.05, corrected; V > 256 μL), a distinction in striatal processing was found between the two conditions. In bilateral ventroanterior striatum, there was significantly greater activation in response to choosing the immediate option as compared to choosing the delayed option (see Figure 4A) (see Table S7 in the supplemental information for whole brain activation sites).

Figure 4.

Figure 4

Open in a new tab

ROI based analysis (p < 0.05, corrected; V > 256 μL): A) Ventroanterior striatum exhibited significantly greater activation when subjects chose the immediate option than when they chose the delayed option (contrast: delayed vs. immediate option). B) The ventroanterior striatum displayed significantly greater activation when collecting the delayed as opposed to the immediate reward (contrast: delayed vs. immediate collection).

Immediate and Delayed Collections vs. Baseline

Relative to baseline, bilateral insula, superior temporal gyri, and inferior parietal lobule were more active during the immediate and delayed collection conditions as compared to baseline (whole brain analysis: p < 0.05, corrected; V > 704 μL). Regions of significant deactivation were found in bilateral medial prefrontal cortex and left precentral gyrus for both collection conditions (see Tables S5 and S6 in the supplemental information). Targeted region of interest analyses (p < 0.05, corrected; V > 256 μL) showed that bilateral insular cortex was active during both immediate and delayed reward collection (versus baseline), which is consistent with the whole brain analysis. In addition, the bilateral striatum was found to be significantly activated during the delayed reward collection.

Delayed versus Immediate Collection

A similar disparity in ventroanterior striatal activation was seen when comparing the immediate and delayed reward collection. However, as opposed to the option conditions, in this case the delayed reward collection resulted in significantly greater striatal activation as compared to the immediate reward collection (see Figure 4B) (see Table S8 in the supplemental information for whole brain activation sites).

Behavior and Brain Activation

A voxel-wise correlation analysis on the whole brain was carried out to determine whether the degree of activation during each of the task conditions was related to subject choices. We found that the degree of activation when choosing immediate options in a region within bilateral right inferior frontal gyrus (x, y, z = 38, 28, −3) was significantly (p < 0.05, corrected; V > 704μL) correlated with the percent immediate rewards chosen during the task (see Figure 5A), i.e. the stronger the activation the more frequently the subject selected the immediate option. Conversely, activation in a different region encompassing the superior temporal gyrus and the right inferior frontal gyrus when choosing delayed rewards (x, y, z = 38, 3, minus;14) was negatively correlated (p < 0.05, corrected; V > 704μL) with the percent of immediate options chosen (see Figure 5B), i.e. greater activation in this area was associated with a lower frequency of selecting the immediate option.

Figure 5.

Figure 5

Open in a new tab

Whole brain analysis (p < 0.05, corrected; V > 704μL): (A) the percent of immediate options chosen by the subject was positively correlated with activity in the right inferior frontal gyrus during selection of the immediate option. (B) a negative correlation was found between the percent of immediate options chosen by the subject and activity in the right posterior temporal gyrus and the inferior frontal gyrus during selection of the delayed option.

Discussion

This investigation yielded four main findings. First, bilateral anterior insulae and striatum displayed significant activation during the choice of both the immediate and delayed options. Second, anterior and posterior insula were activated during the collection of both immediate and delayed reward. Third, there was greater activation in ventroanterior striatum when subjects chose the immediate as opposed to the delayed option. Fourth, ventroanterior striatum exhibited greater activation when subjects collected the delayed larger reward relative to the immediate smaller reward.

Ventral Striatum Activation to Inter-temporal Choice and Outcome

The ability to select between potential actions that lead to advantageous or disadvantageous outcomes is essential for maximizing goal-directed activities. The neural process and computations that underlie our decisions are manifold. There is considerable evidence that the striatum is a critical component in the processing of reward, including reward expectations, reward choices, and actual reward evaluations leading to goal-directed behavior (Hassani et al., 2001; Schultz et al., 2000; Cardinal et al., 2004). Several fMRI studies have demonstrated ventral striatum activity during the anticipation of conditions associated with rewards versus those without rewards (Knutson et al., 2001; Tanaka et al., 2004). Our findings of ventral striatum activation in both decision conditions, the immediate and the delayed reward (Fig. 3), further evidences the general notion that the ventral striatum, among other regions, tracks the perceived value of a potential reward (Kable & Glimcher, 2007; Plichta et al., 2009; Gregorios-Pippas et al., 2009). Moreover, the specific ventral striatum activation for the immediate choices (Figure 4) complements the notion that more ventral parts of the striatum code for the selection of short-term reward options versus delayed options (Tanaka et al., 2004).

Perhaps our most interesting finding in our data is that the ventral striatum has greater activation when collecting the delayed larger reward as compared to the more immediate, smaller, reward (Figure 5). While the ventral striatum has been shown to be involved in the decision stage of selecting immediate rewards, it seems that it is also implicated in the reward collection stage, especially for receiving a greater reward after a longer delay. In addition, the ventral striatum has been shown to increase in activity after receiving a reward or immediately before a reward (Kable & Glimcher, 2007; Breiter et al., 2001). Therefore, the ventral striatum is activated in accordance with the decision-processing stage – during action selection when one chooses the sooner reward, and reward collection when one actually collects the larger reward. Thus, the ventral striatum is active when people either desire something immediately or when they receive a greater reward for which they delayed gratification.

Insular Cortex Activation and Decision Making

Insular regions are critical components of the decision-making neural network, integrating visceral sensations and emotional states to modulate decisions (Craig, 2002). In accordance with Damasio’s ‘somatic marker hypothesis’(Damasio, 1994), the insular cortex integrates interoceptive information as a basis for judging reward values and, thus has an involvement in action selection. For example, the right anterior insula is related to risk-taking decisions when subjects select a “risky” response as compared to a “safe” response (Paulus et al., 2003), also showing activated in fMRI studies examining simple preference judgments (Paulus & Frank, 2003), and when performing a task to receive monetary rewards (Knutson et al., 2000). Results of our delay discounting task, which employed delays in the range of seconds, supports findings from several fMRI studies showing that the insular cortex is strongly involved in the selection process between two reward delays (McClure et al., 2007; Monterosso et al., 2007; Tanaka et al., 2004; Wittmann et al., 2007b). In the present study, we found pronounced bilateral anterior insula activation for both immediate and delayed choices (Figure 3). However, contrary to an earlier finding with inter-temporal choice tasks employing a different set of time scales (Tanaka et al., 2004; Wittmann et al., 2007b), we did not find differences in insular cortex activation depending on whether participants choose immediate or delayed monetary incentives in range of seconds.

Inferior Frontal Correlation to Decision Making

Several fMRI studies have revealed a strong relationship between decision processes and frontal and parietal areas of the brain, both of which contribute to dissociable sub-processes of selecting among competing response options (Zysset et al., 2002; Paulus et al., 2004). Greater relative activity in dorsolateral and ventrolateral prefrontal cortices (as well as the inferior parietal cortex) can be detected in fMRI delay discounting studies when subjects choose between temporally delayed options (Tanaka et al., 2004; McClure et al., 2007; McClure et al., 2004; Monterosso et al., 2007). These studies support the notion that areas involved in executive control are important for being able to delay rewards as opposed to giving in to selecting smaller but sooner rewards. Since the right inferior frontal gyrus is known to be specifically important for inhibitory control of behavior (Aron et al., 2003), this region could facilitate the control of premature or impulsive actions, such as selecting an immediate reward when a larger reward is available at some later time point. Our results show that two adjacent regions of the right inferior frontal cortex (Figure 5) are related to choices for immediate and for delayed rewards. A greater percent signal change in the two regions is associated with whether the subject was choosing immediate or delayed rewards, meaning that one region in the inferior frontal cortex was active during short-term choices (Figure 5A) whereas another region in close spatial proximity was active during selection of delayed rewards (Figure 5B; note, that as depicted in the Figure the superior temporal gyrus is also strongly implicated).

It may be argued that the subject’s choice for an immediate reward can be regarded as rational, since the delay to receiving a reward introduces the possibility that events may interfere during the delay and thus decrease the likelihood that the reward will remain available (Takahashi, 2009). In the present task, when participants favored the delayed option, and consequently had to wait longer they used up more time idling and therefore had fewer selections to make in the total task time. Thus, the additional right-sided inferior frontal region correlating in activity with the selection of immediate responses can be interpreted as deriving from a rational trade-off between time and money. Subjects had to take into account (1) that choosing more delayed rewards could lead to a greater overall monetary gain but (2) that the response for the more immediate reward was related to less waiting time and thus more choices.

The time vs. money trade-off is an important but unavoidable limitation within our fMRI design. We employed a procedure in which the critical variable of wait time before reward reception was dynamically varied based on the subject’s choices within the total duration of the experimental run – which, again, had a fixed length. However, this arrangement may allow subjects to devise reward-optimizing strategies (Schweighofer et al., 2006), thereby making interpretation more difficult. Two such strategies are possible. First, if considering a fixed length of the run, subjects could have attempted to maximize overall reward, thus complicating the effect of delay within a temporal discounting context. However, in the present case subjects had no idea how long the run would last. Therefore, it would not have been possible to accurately calculate a strategy that maximized earnings. Second, for each trial subjects could have calculated the relative payoff for each choice as reward density, or cents per second (i.e. 12 cents in 2 s for the more immediate reward equals 24 cents in 4 seconds for the more delayed reward). Across all subjects, 75% (range: 41% – 93%) of all choices were for the option with the larger reward density (cents per second), but less than half of all switches (47.5%) occurred at the optimal point within a block (e.g., the point of switching from taking the delayed option to choosing the immediate option, which would have maximized reward amount in that block). Thus, although optimizing strategies may have occurred on individual decisions, the obtained behavior patterns do not indicate the existence of a predominant global strategy. In addition, it should be noted that using optimizing strategies is not logically incompatible with temporal discounting processes. When assessing delayed and immediate options (real or hypothetical), there is little evidence to suggest that individuals do not calculate rough estimates of time vs. reward trade-offs. Additionally, regardless of the subjects’ choice strategies, delays and collection of reward were necessarily experienced on every trial, and immediate versus delayed options were clearly distinguished by unique brain activation patterns.

The neuroscience of temporal discounting is still in its infancy. Since the duration of time until a reward is to be received is viewed as a cost to be weighed against the benefits of the reward, the anticipation of duration is an essential component when deciding which option to take (Zauberman et al., 2009; Wittmann & Paulus, 2009; Gregorios-Pippas et al., 2009). In fact, the striatum, the insular cortex as well as inferior frontal regions – our three main regions of interest in this seconds range delay discounting study – are areas of the brain that are strongly involved in the perception of duration and the timing of behavior, also within the seconds range (Wittmann et al., 2008; Pouthas et al., 2005; Stevens et al., 2007; Wittmann, 2009). Since time is an important dimension when individuals make decisions, future neuroimaging studies that combine both delay discounting and measures of time perception will help reveal the relationship between the subjective experience of time and inter-temporal decision making (Gregorios-Pippas et al., 2009). Moreover, an extensive body of knowledge exists concerning animal studies with delay discounting intervals spanning several seconds (Cardinal, 2006; Monterosso & Ainslie, 1999), that demonstrates how discounting of a reward happens within delays of a few seconds. However, newer evidence on the neural basis of inter-temporal choice in humans is starting to accumulate, enabling researchers to draw comparisons between human and animal studies – the former employing neuroimaging techniques and the latter using invasive methods.

This study extends research into delay discounting by using a task that varied in the multiple seconds range, as well as required subjects to wait through the chosen delay period before collecting a reward. In summary, our results suggest the participation of three key brain structures – the ventral striatum, the insular cortex, and the dorsal frontal cortex – in deciding between options characterized by a trade off between reward magnitude and reward delay. The data indicates that the ventral striatum processes both the choice and the reception of rewards, potentially by integrating body signals via interoceptive information processed in the insular cortex, for judging the reward values. In addition, the inferior frontal cortex might be involved in action selection and inhibition depending on the reward evaluation mediated by striatal-insular processes.

Supplementary Material

1

Acknowledgments

This work was supported by grants from the National Institute on Drug Abuse (R01DA016663, R01DA018307, R01DA015392, 1R03DA020687-01A1), the National Institute on Alcohol Abuse and Alcoholism (grant R01AA016965), and the Kavli Institute for Brain and Mind (07-33).

We thank David Deriso for his critical and helpful remarks on the manuscript.

Footnotes

Publisher's Disclaimer: The following manuscript is the final accepted manuscript. It has not been subjected to the final copyediting, fact-checking, and proofreading required for formal publication. It is not the definitive, publisher-authenticated version. The American Psychological Association and its Council of Editors disclaim any responsibility or liabilities for errors or omissions of this manuscript version, any version derived from this manuscript by NIH, or other third parties. The published version is available at www.apa.org/journals/npe

Contributor Information

Marc Wittmann, Department of Psychiatry, University of California San Diego, Veterans Affairs San Diego Healthcare System, San Diego.

Kathryn L. Lovero, Department of Psychiatry, University of California San Diego

Scott D. Lane, Psychiatry and Behavioral Sciences, Graduate School of Biomedical Science, University of Texas Health Science Center–Houston

Martin P. Paulus, Department of Psychiatry, University of California San Diego, Veterans Affairs San Diego Healthcare System, San Diego

Reference List

  1. Ainslie G. Specious reward - behavioral theory of impulsiveness and impulse control. Psychological Bulletin. 1975;82:463–496. doi: 10.1037/h0076860. [DOI] [PubMed] [Google Scholar]
  2. Aron AR, Fletcher PC, Bullmore T, Sahakian BJ, Robbins TW. Stop-signal inhibition disrupted by damage to right inferior frontal gyrus in humans. Nature Neuroscience. 2003;6:115–116. doi: 10.1038/nn1003. [DOI] [PubMed] [Google Scholar]
  3. Berns GS, Laibson D, Loewenstein G. Intertemporal choice - toward an integrative framework. Trends in Cognitive Sciences. 2007;11:482–488. doi: 10.1016/j.tics.2007.08.011. [DOI] [PubMed] [Google Scholar]
  4. Bickel WK, Odum AL, Madden GJ. Impulsivity and cigarette smoking: delay discounting in current, never, and ex-smokers. Psychopharmacology. 1999;146:447–454. doi: 10.1007/pl00005490. [DOI] [PubMed] [Google Scholar]
  5. Boynton GM, Engel SA, Glover GH, Heeger DJ. Linear systems analysis of functional magnetic resonance imaging in human V1. Journal of Neuroscience. 1996;16:4207–4221. doi: 10.1523/JNEUROSCI.16-13-04207.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron. 2001;30:619–639. doi: 10.1016/s0896-6273(01)00303-8. [DOI] [PubMed] [Google Scholar]
  7. Cardinal RN. Neural systems implicated in delayed and probabilistic reinforcement. Neural Networks. 2006;19:1277–1301. doi: 10.1016/j.neunet.2006.03.004. [DOI] [PubMed] [Google Scholar]
  8. Cardinal RN, Winstanley CA, Robbins TW, Everitt BJ. Limbic corticostriatal systems and delayed reinforcement. Annals of the New York Academy of Science. 2004;1021:33–50. doi: 10.1196/annals.1308.004. [DOI] [PubMed] [Google Scholar]
  9. Cox RW. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research. 1996;29:162–173. doi: 10.1006/cbmr.1996.0014. [DOI] [PubMed] [Google Scholar]
  10. Craig AD. How do you feel? Interoception: the sense of the physiological condition of the body. Nature Reviews Neuroscience. 2002;3:655–666. doi: 10.1038/nrn894. [DOI] [PubMed] [Google Scholar]
  11. Craig AD. How do you feel - now? The anterior insula and human awareness. Nature Reviews Neuroscience. 2009;10:59–70. doi: 10.1038/nrn2555. [DOI] [PubMed] [Google Scholar]
  12. Damasio AR. Descartes’ error: Emotion, reason, and the human brain. New York: Grosset-Putnam; 1994. [Google Scholar]
  13. Gregorios-Pippas L, Tobler PN, Schultz W. Short-term temporal discounting of reward value in human ventral striatum. Journal of Neurophysiology. 2009;101:1507–1523. doi: 10.1152/jn.90730.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Hariri AR, Brown SM, Williamson DE, Flory JD, de Wit H, Manuck SB. Preference for immediate over delayed rewards is associated with magnitude of ventral striatal activity. Journal of Neuroscience. 2006;26:13213–13217. doi: 10.1523/JNEUROSCI.3446-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Hassani OK, Cromwell HC, Schultz W. Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. Journal of Neurophysiology. 2001;85:2477–2489. doi: 10.1152/jn.2001.85.6.2477. [DOI] [PubMed] [Google Scholar]
  16. Hinton SC, Meck WH. Frontal-striatal circuitry activated by human peak-interval timing in the supra-seconds range. Brain Research Cognitive Brain Research. 2004;21:171–182. doi: 10.1016/j.cogbrainres.2004.08.005. [DOI] [PubMed] [Google Scholar]
  17. Ihaka R, Gentleman R. R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics. 1996;5:299–314. [Google Scholar]
  18. Kable JW, Glimcher PW. The neural correlates of subjective value during intertemporal choice. Nature Neuroscience. 2007;10:1625–1633. doi: 10.1038/nn2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Knutson B, Fong GW, Adams CM, Varner JL, Hommer D. Dissociation of reward anticipation and outcome with event-related fMRI. NeuroReport. 2001;12:3683–3687. doi: 10.1097/00001756-200112040-00016. [DOI] [PubMed] [Google Scholar]
  20. Knutson B, Westdorp A, Kaiser E, Hommer D. FMRI visualization of brain activity during a monetary incentive delay task. Neuroimage. 2000;12:20–27. doi: 10.1006/nimg.2000.0593. [DOI] [PubMed] [Google Scholar]
  21. Laibson D. Golden eggs and hyperbolic discounting. Quarterly Journal of Economics. 1997;112:443–477. [Google Scholar]
  22. Lancaster JL, Woldorff MG, Parsons LM, Liotti M, Freitas CS, Rainey L, et al. Automated Talairach atlas labels for functional brain mapping. Human Brain Mapping. 2000;10:120–131. doi: 10.1002/1097-0193(200007)10:3&#x0003c;120::AID-HBM30&#x0003e;3.0.CO;2-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lane SD, Cherek DR, Pietras CJ, Tcheremissine OV. Measurement of delay discounting using trial-by-trial consequences. Behavioural Processes. 2003;64:287–303. doi: 10.1016/s0376-6357(03)00143-8. [DOI] [PubMed] [Google Scholar]
  24. Luhmann CC, Chun MM, Yi DJ, Lee D, Wang XJ. Neural dissociation of delay and uncertainty in intertemporal choice. Journal of Neuroscience. 2008;28:14459–14466. doi: 10.1523/JNEUROSCI.5058-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Madden GJ, Begotka AM, Raiff BR, Kastern LL. Delay discounting of real and hypothetical rewards. Experimental and Clinical Psychopharmacology. 2003;11:139–145. doi: 10.1037/1064-1297.11.2.139. [DOI] [PubMed] [Google Scholar]
  26. McClure SM, Ericson KM, Laibson DI, Loewenstein G, Cohen JD. Time discounting for primary rewards. Journal of Neuroscience. 2007;27:5796–5804. doi: 10.1523/JNEUROSCI.4246-06.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. McClure SM, Laibson DI, Loewenstein G, Cohen JD. Separate neural systems value immediate and delayed monetary rewards. Science. 2004;306:503–507. doi: 10.1126/science.1100907. [DOI] [PubMed] [Google Scholar]
  28. Monterosso J, Ainslie G. Beyond discounting: possible experimental models of impulse control. Psychopharmacology. 1999;146:339–347. doi: 10.1007/pl00005480. [DOI] [PubMed] [Google Scholar]
  29. Monterosso JR, Ainslie G, Xu J, Cordova X, Domier CP, London ED. Frontoparietal cortical activity of methamphetamine-dependent and comparison subjects performing a delay discounting task. Human Brain Mapping. 2007;28:383–393. doi: 10.1002/hbm.20281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Paulus MP, Feinstein JS, Tapert SF, Liu TT. Trend detection via temporal difference model predicts inferior prefrontal cortex activation during acquisition of advantageous action selection. Neuroimage. 2004;21:733–743. doi: 10.1016/j.neuroimage.2003.09.060. [DOI] [PubMed] [Google Scholar]
  31. Paulus MP, Frank LR. Ventromedial prefrontal cortex activation is critical for preference judgments. NeuroReport. 2003;14:1311–1315. doi: 10.1097/01.wnr.0000078543.07662.02. [DOI] [PubMed] [Google Scholar]
  32. Paulus MP, Rogalsky C, Simmons A, Feinstein JS, Stein MB. Increased activation in the right insula during risk-taking decision making is related to harm avoidance and neuroticism. Neuroimage. 2003;19:1439–1448. doi: 10.1016/s1053-8119(03)00251-9. [DOI] [PubMed] [Google Scholar]
  33. Plichta MM, Vasic N, Wolf RC, Lesch KP, Brummer D, Jacob C, et al. Neural hyporesponsiveness and hyperresponsiveness during immediate and delayed reward processing in adult attention-deficit/hyperactivity disorder. Biological Psychiatry. 2009;65:7–14. doi: 10.1016/j.biopsych.2008.07.008. [DOI] [PubMed] [Google Scholar]
  34. Pouthas V, George N, Poline JB, Pfeuty M, Vandemoorteele PF, Hugueville L, et al. Neural network involved in time perception: an fMRI study comparing long and short interval estimation. Human Brain Mapping. 2005;25:433–441. doi: 10.1002/hbm.20126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Prelec D, Loewenstein G. Beyond time discounting. Marketing Letters. 1997;8:97–108. [Google Scholar]
  36. Rachlin H, Raineri A, Cross D. Subjective probability and delay. Journal of the Experimental Analysis of Behavior. 1991;55:233. doi: 10.1901/jeab.1991.55-233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Schultz W, Tremblay L, Hollerman JR. Reward processing in primate orbitofrontal cortex and basal ganglia. Cerebral Cortex. 2000;10:272–283. doi: 10.1093/cercor/10.3.272. [DOI] [PubMed] [Google Scholar]
  38. Schweighofer N, Shishida K, Han CE, Okamoto Y, Tanaka SC, Yamawaki S, et al. Humans can adopt optimal discounting strategy under real-time constraints. PLoS Computational Biology. 2006;2:e152. doi: 10.1371/journal.pcbi.0020152. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Stevens MC, Kiehl KA, Pearlson G, Calhoun VD. Functional neural circuits for mental timekeeping. Human Brain Mapping. 2007;28:394–408. doi: 10.1002/hbm.20285. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Takahashi T. Loss of self-control in intertemporal choice may be attributable to logarithmic time-perception. Medical Hypotheses. 2005;65:691–693. doi: 10.1016/j.mehy.2005.04.040. [DOI] [PubMed] [Google Scholar]
  41. Takahashi T. Theoretical frameworks for neuroeconomics of intertemporal choice. Journal of Neuroscience, Psychology, and Economics. 2010 in press. [Google Scholar]
  42. Talairach J, Tournoux P. Co-planar stereotaxic atlas of the human brain: a 3-dimensional proportional system, an approach to cerebral imaging. Stuttgart; New York New York: G. Thieme; Thieme Medical Publishers; 1988. [Google Scholar]
  43. Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, Yamawaki S. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neuroscience. 2004;7:887–893. doi: 10.1038/nn1279. [DOI] [PubMed] [Google Scholar]
  44. Wittmann M. The inner experience of time. Philosophical Transactions of the Royal Society. 2009;364:1955–1967. doi: 10.1098/rstb.2009.0003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Wittmann M, Leland DS, Churan J, Paulus MP. Impaired time perception and motor timing in stimulant-dependent subjects. Drug and Alcohol Dependence. 2007a;90:183–192. doi: 10.1016/j.drugalcdep.2007.03.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Wittmann M, Leland DS, Paulus MP. Time and decision making: differential contribution of the posterior insular cortex and the striatum during a delay discounting task. Experimental Brain Research. 2007b;179:643–653. doi: 10.1007/s00221-006-0822-y. [DOI] [PubMed] [Google Scholar]
  47. Wittmann M, Paulus MP. Temporal horizons in decision making. Journal of Neuroscience, Psychology, and Economics. 2009;2:1–11. [Google Scholar]
  48. Wittmann M, Simmons AN, Aron JL, Paulus MP. Accumulation of neural activity in the posterior insula encodes the passage of yime. Nature Precedings; 2008. http://hdl.handle.net/10101/npre.2008.2062.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Zauberman G, Kyu Kim B, Malkoc SA, Bettman JR. Discounting time and time discounting: subjective time perception and intertemporal preferences. Journal of Marketing Research. 2009 in press. [Google Scholar]
  50. Zysset S, Huber O, Ferstl E, von Cramon DY. The anterior frontomedian cortex and evaluative judgment: An fMRI study. Neuroimage. 2002;15:983–991. doi: 10.1006/nimg.2001.1008. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS149695-supplement-1.doc (115.5KB, doc)

RESOURCES