Neural systems underlying individual differences in intertemporal decision making

Abstract

Excessively choosing immediate over larger future rewards, or delay discounting (DD), associates with multiple clinical conditions. Individual differences in DD likely depend on variations in the activation of and functional interactions between networks, representing possible endophenotypes for associated disorders, including alcohol use disorders (AUDs). Numerous fMRI studies have probed the neural bases of DD, but investigations of large-scale networks remains scant. We addressed this gap by testing whether activation within large-scale networks during “Now/Later” decision-making predicts individual differences in DD. To do so, we scanned 95 social drinkers (18–40 years; 50 females) using fMRI during hypothetical choices between small monetary amounts available “today” or larger amounts available later. We identified neural networks engaged during Now/Later choice using independent component analysis (ICA) and tested the relationship between component activation and degree of DD. The activity of two components during Now/Later choice correlated with individual DD rates: a temporal lobe network positively correlated with DD, while a frontoparietal-striatal network negatively correlated with DD. Activation differences between these networks predicted individual differences in DD and their negative correlation during Now/Later choice suggests functional competition. A generalized psychophysiological interactions (gPPI) analysis confirmed a decrease in their functional connectivity during decision-making. The functional connectivity of these two networks negatively correlates with alcohol-related harm, potentially implicating these networks in AUDs. These findings provide novel insight into the neural underpinnings of individual differences in impulsive decision making with potential implications for addiction and related disorders in which impulsivity is a defining feature.

INTRODUCTION

Humans and other animals show a preference for immediate versus future rewards (Mazur, 1987; Rachlin, 2000). The behavioral economic principle of reward devaluation as a function of time, commonly referred to as “delay discounting” (DD), has been extensively studied in the laboratory using intertemporal choice tasks in which subjects decide between smaller, sooner rewards, or larger, later rewards. In such tasks, individuals vary considerably in their reward preference, demonstrating the highly subjective nature of intertemporal decision-making. A greater bias for sooner rewards, widely accepted as a form of impulsivity, is associated with addictive disorders (Becker & Murphy, 1988; W. K. Bickel, Odum, & Madden, 1999; Dixon, Marley, & Jacobs, 2003; Kris N. Kirby & Petry, 2004; K. N. Kirby, Petry, & Bickel, 1999; J. MacKillop et al., 2011; John R. Monterosso et al., 2007; Reynolds, 2006), attention-deficit hyperactivity disorder (Barkley, Edwards, Laneri, Fletcher, & Metevia, 2001; Paloyelis, Asherson, & Kuntsi, 2009; Sonuga-Barke, Sergeant, Nigg, & Willcutt, 2008; Wilson et al., 2015), and eating disorders (Davis, Patte, Curtis, & Reid, 2010; Weller, Cook, Avsar, & Cox, 2008). Therefore, an improved understanding of the neural processes underlying individual differences in DD could provide novel insight into disorders of impulsivity and identify potential biomarkers and treatment targets.

The component processes supporting intertemporal decision-making have been examined by a multitude of noteworthy studies. For example, McClure and colleages (McClure, Ericson, Laibson, Loewenstein, & Cohen, 2007; Samuel M. McClure, David I. Laibson, George Loewenstein, & Jonathan D. Cohen, 2004) described two competing brain systems – a cognitive control system and impulsive system – whose relative activity has been construed to predict choice behavior on a trial-by-trial basis. Other investigators posit a single valuation network that represents the subjective value of reward choices (Kable & Glimcher, 2007; J. R. Monterosso & Luo, 2010). Still others emphasize the role of several neural networks that each contribute to different aspects of DD behavior, including cognitive control, valuation, prospection, and emotion processes (Jan Peters & Büchel, 2011; van den Bos & McClure, 2013). The variations in activation and functional interactions between these various systems are proposed to underlie individual differences in DD behavior (Boettiger et al., 2007; Jan Peters & Büchel, 2011). Crucially, the identification of neural systems that predict the degree of DD across individuals could offer potential intermediate phenotypes of disorders of impulsivity.

Previous work suggests that individual differences in impulsive decision-making correlates negatively with variations in fMRI activation in prefrontal and anterior cingulate cortical regions proposed to support executive control processes (Ballard & Knutson, 2009; Boettiger et al., 2007; J. Peters & Buchel, 2010; Stanger et al., 2013) and positively with the ventral striatum and medial temporal lobe structures proposed to underlie reward motivational processes (Boettiger et al., 2007; Hariri et al., 2006; Stanger et al., 2013). Despite the emphasis on neural systems and “networks” of brain regions in published literature, most previous studies have focused on standard univariate voxel-wise or region-of-interest based analytical approaches, and multivariate investigations of large-scale neural networks contributing to individual differences in DD in healthy adults remain lacking. Based on the existing literature, we hypothesized that individual differences in DD would be predicted by at least two sets of brain regions: greater activation of a network of prefrontal brain regions implicated in cognitive control would associate with less impulsive decision-making whereas greater activation of a network primarily consisting of subcortical regions implicated in reward motivational processes would associate with more impulsive decision-making.

To test this hypothesis, we scanned ninety-five healthy adults using fMRI while they performed a DD task in which they made numerous choices between smaller amounts of hypothetical money “today,” or larger amounts available later. We employed independent component analysis (ICA) to identify large-scale neural networks engaged during intertemporal choice and tested the relationship between component activation and individual differences in choice behavior. We identified two neural systems in which activity during intertemporal choices significantly correlated with the degree of DD across subjects: a medial and lateral temporal lobe network positively correlated with immediate reward bias, whereas a frontal-parietal-striatal network correlated negatively with immediate reward bias, consistent with our hypothesis. Furthermore, negative correlations in task-related functional connectivity between these two systems suggest a competing functional relationship. Overall, our results suggest that enhanced recruitment of a temporal lobe network in lieu of a functionally-competitive frontal-striatal-parietal network is associated with disadvantageous decision-making. Such perturbations in the activity and connectivity of neural networks supporting individual differences in intertemporal decision-making may contribute to pathological behaviors, such as substance misuse.

MATERIALS AND METHODS

Subjects

Healthy adult (ages 18–40 years; mean=25.9) participants (n=95; 50 females) were recruited from the University of North Carolina, Chapel Hill (UNC) campus and surrounding communities. The participant sample was 62% white, 15% black, 11% Asian, 4% Hispanic, and 8% mixed or other race. Participants had completed an average of 16.3 ± 2.5 years of education. Social status as a proxy for socioeconomic status was characterized by the Barratt Simplified Measure of Social Status (BSMSS; Barratt 2006), and the sample average was 50.0 ± 12.4 (range: 8–66) out of a possible range of 8–66. Exclusion criteria included use of psychoactive drug or medication use (excluding moderate alcohol or caffeine intake), a neurological or psychiatric diagnosis, including history of treatment for a substance use disorder or a lifetime history of alcohol or other drug dependence based on a structured clinical interview using DSM-IV criteria (Sheehan et al., 1998). All participants were native English speakers, right-handed, and had at least a high school education (or equivalent). Participants were screened for drug or alcohol use on the day of the scan via breathalyzer test and urine drug screen. Each participant provided written, informed consent as approved by the UNC Office of Human Research Ethics.

Alcohol use data

Although this study included only healthy control participants, the primary motivation for this investigation was to identify potential intermediate phenotypes of AUDs. Therefore, each participant completed the Alcohol Use and Disorders Identification Test (AUDIT) (Saunders, Aasland, Babor, de la Fuente, & Grant, 1993) to provide measures of alcohol consumption, harmful alcohol use, and alcohol dependence symptoms. While no participants met criteria for an alcohol use disorder, all participants reported having consumed alcohol at least once, and alcohol consumption levels ranged from low to heavy, based on AUDIT consumption subscale scores.

Delay discounting task

Participants performed a DD task designed for use in the MRI scanner (Boettiger, Kelley, Mitchell, D’Esposito, & Fields, 2009; Boettiger et al., 2007), and described in detail previously (Altamirano, Fields, D’Esposito, & Boettiger, 2011; Kelm & Boettiger, 2013; Smith & Boettiger, 2012; Smith, Sierra, Oppler, & Boettiger, 2014; Smith, Steel, Parrish, Kelm, & Boettiger, 2015). Participants were given task instructions, completed a short practice, and then completed six blocks of 42 trials each (~8 minutes in length). In each trial, participants were asked to select one of two presented options, a smaller, immediate monetary reward (“Now”), or a larger, delayed reward (“Later”). Selections were indicated by pressing one of two buttons on an MRI-compatible response box. Each trial began with an instruction cue, followed by a screen presenting the Now and Later options. Later options were hypothetical monetary rewards of $2, $5, $10, $20, or $100 at one of five future delays (1 week, 2 weeks, 1 month, 3 months, or 6 months), while the Now option was reduced by 30%, 15%, 10%, or 5% from the Later amount and was available “Today”. Later option delays and amounts, as well as the Now option “discount” and left/right position were randomized across trials. Each choice trial was preceded by an instruction cue presented for 4.4 seconds to indicate one of four choice trial types: “WANT,” “DON’T WANT,” “SOONER,” or “LARGER.” For Want trials, subjects chose their preferred option. In the Don’t Want trials, subjects selected the option that they did not prefer. Sooner and Larger trials represented objective choice control (CON) trials in which subjects indicated which option was available sooner in time, or which reward was larger, respectively. In addition to the chosen option, we also collected the response time for each response. Accuracy in CON trials verifies adherence to task instructions, and comparison of response time (RT) between the CON, WANT and DON’T WANT trials indicates whether additional cognitive processes are being engaged in the WANT and DON’T WANT conditions, relative to the simple objective comparisons required in CON trials. The options appeared on the screen for 4.4 seconds and were followed by a jittered inter-trial interval of 4.4–8.8 seconds; the instruction cue remained on the screen during this time. During “null” trials, the instruction cue was presented but no choice options were presented. WANT trials comprised 50% of the choice trials whereas the other 50% were divided equally among DON’T WANT, SOONER, and LARGER trials, with trial type pseudorandomly ordered across trials.

Task behavioral measures

As the primary measure of impulsive decision making, an impulsive choice ratio (ICR, Mitchell et al., 2005) was calculated as the proportion of Now choices made in the WANT condition. Higher ICR values indicate more impulsive decision making. Although the delay discounting paradigm used in this study was not designed to optimally estimate model-based discounting rates, we present secondary analyses for which the degree of impulsive choice was quantified using the q-exponential discount function based on Tsallis’ statistics (Takahashi et al., 2008; Takahashi, 2009):

$$\mathit{Discounted}\mathit{Value}(\mathrm{D})=1/{[1+(1-q)k_{q}D]}^{1/(1-q)}$$ (Eqn. 1)

where D represents delay time, and k_q and q are measures of impulsivity and of inconsistency across delay times, respectively. To estimate k_q and q, we conducted nonlinear curve fitting with the Levenberg–Marquardt algorithm implemented in R (Team, 2014) with the minipack.lm package (Elzhov TV, 2015). Discounted Value was calculated as the cumulative selected/maximum dollar ratio at each delay, D. (Smith et al., 2014; Smith et al., 2016). Higher values of k_q indicate greater discounting of delayed rewards (more impulsivity) and lower values of k indicate less delayed reward discounting (less impulsivity). Based on a threshold of R²>.2, the q-exponential model failed to adequately fit the data of n=43 subjects. Therefore, for the secondary analyses that considered the k_q metric, only subjects for whom the model was deemed valid were included. All primary analyses utilized the ICR metric and included all 95 subjects.

Magnetic Resonance Imaging (MRI) data acquisition

Task fMRI datasets were acquired as T2*-weighted images (EPI) on a Siemens 3T Tim Trio MRI whole body scanner equipped with a TEM send-receive radio frequency (RF) head coil, using a 1-shot gradient-echo EPI pulse sequence to measure localized blood oxygenation level dependent (BOLD) contrast. Acquisition parameters were as follows: TR = 2000 ms, TE = 25 ms, flip angle = 50°, 35 slices tilted by 30° from the horizontal plane; FoV = 192×192 mm; voxel size=3×3×4 mm with a 0.5 mm gap, matrix = 64×64. The fMRI acquisition was preceded by 11s of dummy gradient RF pulses to achieve steady-state tissue magnetization and minimize startle-induced motion. Duration for each run acquisition was approximately 9 minutes (243 frames). Low-resolution T1-weighted coplanar images were acquired for each participant. In addition, a high-resolution magnetization prepared rapid gradient echo (MPRAGE) T1-weighted structural image was acquired from each participant for alignment and tissue segmentation purposes. The MPRAGE pulse sequence parameters were as follows: TR =2530 ms TE=2.27 ms, flip angle=9⁰, matrix=176×512, 512 slices, final resolution=1×0.5×0.5mm³. Head movement during the scanning session was restricted through the placement of padding, to minimize confounding effects on image quality. E-Prime-2 software (PST, Inc., Pittsburg, PA) synchronized the stimulus display to the fMRI acquisition and recorded participant responses via an MRI-compatible fiber-optic keypad. An LCD projector (Avotec Inc., Stuart, FL) projected visual stimuli onto a rear projection screen, which the participants viewed via a mirror mounted within the head coil.

MRI data preprocessing

The data were processed offline using Analysis of Functional Neuroimages [AFNI version 16.0.13, (Cox, 1996)] software and included the following steps: slice time correction, reorienting of oblique slices to the axial plane, realignment of images, despiking of noise time points, alignment to the subject’s T1 image, warping to a Montreal Neurological Institute (MNI) template, removal by regression of signal from white matter and cerebral spinal fluid as well as the six motion covariates, linear detrending, Gaussian smoothing with an 8 mm full-width half-maximum smoothing kernel, and scaling to percent signal change. The Artifact Detection Tools toolbox (ART; http://www.nitrc.org/projects/artifact_detect) was used to identify time points with high amounts of noise according to head motion and global signal intensity measures.

General linear modeling of task activations

Task-related activation was detected from the fMRI task design matrix produced using Statistical Parametric Mapping (SPM8) software and analyzed using a general linear modeling approach (K. J. Friston et al., 1994) conducted with restricted maximum likelihood estimation in 3dREMLfit in AFNI. Task cues were modeled as delta functions and decision-making periods were modeled as short epochs with their onset at the time the Now and Later options appeared and a duration equal to the trial-specific RT. The six fMRI runs were concatenated to form a single data set. Zero, first, and second order polynomial regressors were specified for each of the six runs to remove trends within and between runs. Six head motion parameters as well as covariates denoting outlier time points based on the ART analysis were also included in the GLM. Beta estimates from WANT trials were contrasted with CON trials to isolate activity specific to making subjective intertemporal reward choices.

Voxel-wise relationships with behavior

Voxel-wise statistical analyses utilized a permutation-based method that is robust against departures from statistical assumptions required for parametric tests, provides stringent protection against false positives (α = 0.05), and provides comparable or improved Type II error rates over other methods (Huang et al., 2015). The main effect of task was examined using the WANT>CON contrast maps of all 95 subjects calculated from estimations of brain activation using 3dDeconvolve in AFNI; the analysis covaried for age and sex. The neural correlates of Individual differences in impulsive choice were similarly calculated in a voxel-wise permutation test of the relationship between ICR and WANT>CON contrast values, covarying for age and sex. These voxel-wise analyses provided results that enable direct comparisons with the existing literature and against which the findings of the network-level analysis could be validated and interpreted.

ICA approach

ICA is a data-driven method for data reduction that has been adopted for use with fMRI data to derive spatially-independent brain networks of functionally connected voxels (Calhoun, Adali, Pearlson, & Pekar, 2001). Data reduction for group fMRI data is often a multi-step process to reduce individual subject four dimensional (spatial and temporal) data into three dimensional principal components, and then an ICA is performed on concatenated principal components to provide a group result. Components resolved in this fashion from resting-state data are purported to represent intrinsic connectivity networks; however, this approach carries several limitations for task fMRI data. Since components derived from task-related fMRI time series comprise multiple brain states (i.e. passive fixation, task cues, control trials, experimental trials, etc.), the interpretation of components derived from such data is less straightforward. Although these components can be related back to specific task conditions using back reconstruction methods, general linear modeling, and statistical contrasts (Stanger et al., 2013), the spatial organization of the components is influenced by all constituents of the task regardless of their relevance for the specific task condition or psychological construct of interest.

For this study, we sought to examine the neural systems underlying intertemporal reward decision-making. As such, the WANT>CON contrast maps derived from the voxel-wise GLM analysis (Fig. 1A) were entered into an ICA to identify networks of brain regions specific to this decision making process that covary across individuals (Fig. 2A). Thus, rather than using a multistep process to identify group components from fMRI time series, we derived components in a single step using the contrast maps from all 95 subjects. The ICA was conducted with the GIFT group ICA toolbox (v3.0a; (Calhoun et al., 2001)) using the Infomax algorithm following variance normalization. We ran 20 ICASSO iterations in a series of analyses solving for 25, 20, 15, or 10 components and determined that solving for 10 components yielded the tightest clustering of results across iterations, suggesting good stability. Thus, we conducted our analysis using 10 independent components. We visually inspected the 10 component spatial maps to identify those representing obvious artifacts, resulting in the removal of three components: two for which peak values were around the outside edges of the brain and one that was focused in the ventricles.

Figure 1.

Schematic diagram of independent component analysis (ICA) approach. Voxel-wise maps of the WANT>CON contrast from 95 subjects (A) were entered into an ICA solving for 10 independent components (B). (C) Component maps were then regressed on each subject’s WANT>CON contrast maps to obtain estimates of the relationship of each component to the contrast for each subject; these subject-level contrast estimates were entered into group level analyses represented by Table 2 and Figures 3 and 4.

Figure 2.

Voxel-wise statistical relationships. A) Results of the voxel-wise test of the main effect of task (WANT>CON contrast) corrected for multiple comparisons (α = 0.05) using permutation testing. B) The voxel-wise relationship between individual impulsive choice ratio (ICR) and activation during subjective intertemporal decision-making (WANT>CON contrast), unthresholded for visualization purposes. C) The voxel-wise relationship between ICR and activation during subjective intertemporal decision-making, as in panel B, but corrected for multiple comparisons (α = 0.05) using permutation testing. W: Wald statistic.

Next we calculated the degree of task-related activation of each of the remaining seven components across subjects. The relationship of each component to intertemporal reward decision-making was tested by regressing component maps on each subject’s WANT>CON contrast maps (Fig. 1C) as an alternative to estimating this relationship with GLM analyses of component time series (James, Tripathi, & Kilts, 2014). The resulting contrast estimates were mean and variance normalized by converting them to z-scores.

Network activation relationships with behavior

A group-level statistical measure of WANT>CON component activation was calculated by one-sample t-test of contrast estimates across all 95 subjects. Additionally, the relationship between individual differences in decision making and task-related component activation was tested using Spearman’s partial correlation between ICR and contrast estimates for each component, controlling for age and sex. Results were corrected for multiple comparisons using a false discovery rate (FDR) correction (Benjamini & Yekutieli, 2001).

RESULTS

Behavioral data

We quantified individual DD tendency as the ratio of Now options selected in the WANT condition, relative to all choices made in the WANT condition (Mitchell, Fields, D’Esposito, & Boettiger, 2005), which we refer to as the impulsive choice ratio (ICR). Participants in this sample demonstrated the full range of ICR values (i.e., 0 to 1) with a mean value of 0.62 (Q1=0.35; Q2=0.71; Q3=0.92). As a DD index, ICR has the advantage of very strong internal reliability (Smith et al., 2015; Smith et al., 2016), coupled with the fact that ICR avoids the assumptions of model-based metrics. In contrast, indices derived from temporal discounting models, such as temporal discount rates (“k”), are influenced by the assumptions of the particular model employed, and some participants’ data may not conform to these assumptions. This DD task includes objective choice control (CON) trials that allow us to verify participants’ adherence to task instructions in two ways. First, we verified high accuracy in CON trials (mean: 96.1 ± 5.4%). Second, we compared RTs between the WANT and CON conditions, which indicates whether additional cognitive processes are engaged in the WANT condition, relative to the simple objective comparison needed in CON trials; mean WANT-CON RT difference: 588 ± 358 ms. In this study, n=8 participants were excluded from all analyses due to insufficient CON trial accuracy and/or to equivalent RTs in the WANT and CON trials, yielding the final sample of 95 participants.

ICA of activity during subjective intertemporal choice

To identify large-scale networks engaged during subjective intertemporal choice, we first generated whole brain contrasts of activity associated with choice evaluation in the WANT and CON trials (WANT>CON contrast) for each participant (Fig. 1A). We next tested the main effect of task on these contrast maps (Fig. 2A), which closely matched findings reported by previous studies (Warren K. Bickel, Pitcock, Yi, & Angtuaco, 2009; John R. Monterosso et al., 2007). Statistical maps depicting the voxel-wise relationship between ICR and activity during subjective intertemporal decision-making (WANT>CON contrast) are shown both unthresholded (Fig. 2B), and thresholded with a cluster-level correction for multiple comparisons (Fig. 2C). As shown in Figure 2C and detailed in Table 1, ICR positively correlated with enhanced activity during subjective choice in a multitude of regions, including clusters in the medial temporal lobe (i.e., amygdala, hippocampus, parahippocampal gyrus), superior frontal gyrus, retrosplenial cortex, and cerebellum. In contrast, ICR negatively correlated with activity during subjective intertemporal decision-making in the dorsal anterior cingulate cortex, right lateral frontal and parietal cortices and caudate tail.

Table 1.

Voxel-wise correlates of the impulsive choice ratio (ICR)

ICR Correlation	Label	X	Y	Z	# Voxels
positive	left superior frontal gyrus	−23	31	42	70
	left retrosplenial cortex	−9	−47	7	35
	left amygdala	−12	−12	−15	29
	right parahippocampal gyrus	30	−33	−11	24
	left hippocampus	−30	−33	−11	24
	left parahippocampal gyrus	−19	−36	−15	20
	right retrosplenial cortex	12	−43	3	17
	left middle cingulate gyrus	−9	−40	35	12
	right superior frontal gyrus	16	41	45	9
	left putamen	−16	3	−8	7
	left superior occipital gyrus	−37	−75	28	5
	left cerebellum	−9	−43	−46	3
	left cerebellum	−12	−75	−43	2
	right superior frontal gyrus	16	34	35	2
	left superior temporal gyrus	−43	−36	6	1
	left superior frontal gyrus	−9	62	17	1
	right cerebellum	42	−63	−39	1
negative	right precentral gyrus	40	−5	35	32
	right supramarginal gyrus	47	−43	35	5
	left dorsal anterior cingulate cortex	12	3	38	2
	left caudate tail	−19	−29	17	1
	right caudate tail	19	−19	20	1

The ICA identified seven physiologically relevant networks that covaried across subjects during intertemporal decision-making (Fig. 3A). Three obviously artifactual components, numbered 1, 2, and 7, were excluded from further analyses (see Methods). The statistical association of each component with the WANT>CON contrast based on one sample t-test is reported in Table 2. Three networks demonstrated significant activation during subjective choice (Components 3, 4 and 8; Fig. 3, red bounding box), and three others showed significant deactivation during subjective choice (Components 5, 6 and 10; Fig. 3, blue bounding box). Moreover, of the seven components, two significantly correlated with ICR (Figs. 3–4). Component 3, a frontal-striatal-parietal network (Fig. 3, third row), negatively correlated with ICR (ρ=−0.28, p=0.006), as shown in Figure 4A. Conversely, Component 9, a network incorporating the amygdala, hippocampus, parahippocampal gyrus, posterior insula, and superior temporal gyrus (Fig. 3, bottom row) positively correlated with ICR (ρ=0.29, p=0.005; Figure 4B). While ICR has advantages as a DD metric, as noted above, it is a rather blunt measure. In contrast, the q-exponential discount function can distinctly parameterize both Now bias (impulsivity; k_q) and the inconsistency (q) in such Now bias across delay times in intertemporal choice tasks (Smith et al., 2014; Smith et al., 2016; Takahashi, 2009; Takahashi, Oono, & Radford, 2008). Consistent with our prior findings (Smith et al., 2014; Smith et al., 2016), ICR and k_q values were very highly correlated (ρ=0.92, p<0.001), and the relationships between k_q and components 3 and 9 were qualitatively similar to the relationships between ICR and these components, despite reduced statistical power (Component 3: ρ=−0.26, p=0.067; Component 9: ρ=0.32, p=0.024). In addition to their relationship with ICR, the WANT>CON contrast estimates for components 3 and 9 demonstrated a negative correlation with each other (r=−0.39, p<0.001; Figure 4C) based on Pearson partial correlation controlling for age and sex. Furthermore, the difference in activity of these two neural systems (Component 3 minus Component 9) explained more variance in ICR than did the activity of either system alone (ρ=−0.34, p<0.001; Figure 4D), suggesting their relative activity predicts individual differences in impulsive choice.

Figure 3.

Spatial maps of the seven physiologically relevant networks identified from independent component analysis (ICA) sorted by their relationship to intertemporal decision-making. Numbers at left represent the component number; note that components 1, 2, and 7, were excluded as noise. Components significantly activated during intertemporal decision-making are surrounded by a red bounding box; significantly deactivated components are surrounded by a blue bounding box. Components significantly correlated with the impulsive choice ratio (ICR) are indicated with an asterisk (*). Components are displayed with a threshold of |z|>1.

Table 2.

Relationship of independent components to intertemporal decision-making

Task Activation	Component #	t value	p value
positive	4	t=7.70	<0.001
	8	t=6.65	<0.001
	3	t=4.55	<0.001
negative	10	t=−10.65	<0.001
	6	t=−9.73	<0.001
	5	t=−6.33	<0.001
none	9	t=−0.66	0.51

Results from one-sample t-tests of the WANT>CON contrast representing decision-making-related activation of independent components. The reported positive and negative activations survived an FDR correction for multiple comparisons (p<0.05). Exact p-values reported, except where p<0.001.

Figure 4.

Scatter plots depicting the relationships between task-related component activation and intertemporal choice. A) Scatter plot of the relationship between the impulsive choice ratio (ICR) and Want>Control (WANT>CON) contrast estimates for Component 3. B) Scatter plot of the relationship between ICR and WANT>CON contrast estimates for Component 9. C) Scatter plot of the relationship between WANT>CON contrast estimates for Components 3 and 9. D) Scatter plot of the relationship between ICR and the difference in WANT>CON contrast estimates for Components 3 and 9. Least squares fit lines are plotted for visualization purposes. Correlation coefficients represent partial correlations controlling for age and sex.

Psychophysiological interactions (PPI) analysis

To explore whether these two networks operate in a competing manner during intertemporal reward choice decision-making, we examined the task-specific functional connectivity of Component 3 and Component 9 using a generalized psychophysiological interactions analysis [gPPI, (K. Friston et al., 1997; McLaren, Ries, Xu, & Johnson, 2012)]. As opposed to examining the correlation of component contrast estimates across subjects as presented in Figure 4B, gPPI is a within-subject analysis that examines their correlation across the task. Component time series were estimated by regressing component spatial maps on each subject’s whole brain fMRI time series data using on the spatial-temporal regression method implemented in GIFT. We developed a general linear model to estimate the task-related modulation of functional connectivity between the networks. Notably, a separate regressor is included for each task condition. A simplified schematic of the gPPI procedure is depicted in Figure 5. Specifically, we tested the dependence of Component 3’s time series on the following predictors: 1) Component 9’s time series, 2) the task design matrix, 3) the interaction of the Component 9’s time series and each task condition, and 4) a set of nuisance regressors (i.e. six motion parameters, linear and quadratic trends). Likewise, the corresponding model in which Component 9 was similarly predicted by Component 3 was also tested. These models were estimated using the glmfit function in Matlab R3013a and the β coefficients for the interaction term corresponding to the WANT condition were contrasted with that of the CON (SOONER and LARGER) trials. We took the average WANT>CON contrast value for the two models as an estimate of the task-dependent connectivity between Components 3 and 9. The significance of intertemporal decision-making-related connectivity was calculated with one-sample t-tests of gPPI contrast estimates across all 95 subjects, indicating a significant negative relationship between Components 3 and 9 (t=−3.13, p=0.002).

Figure 5.

A visual representation of the generalized psychophysiological interactions analysis (gPPI) employed to investigate the subjective decision-making-related modulation of connectivity between Component 3 and Component 9. The dependence of one time series (i.e., TS1) was predicted by the second time series (i.e., TS2), the task conditions, the interaction of the task design and TS2, and covariates of no interest (not pictured). The task conditions modeled included subjective decisions (i.e., WANT), objective (control; CON) decisions (i.e., SOONER, LARGER), as well as DON’T WANT decisions and corresponding cues for each decision type (not pictured). Similarly, the interaction between TS2 and each task condition was modeled. The interaction beta estimates corresponding to WANT, SOONER, and LARGER trials were used to estimate a WANT>CON contrast. Component 3 and Component 9 were tested as dependent variables in two separate models.

As noted in the Introduction, an excessive tendency to discount delayed rewards (e.g. high ICR) is associated with alcohol and other substance use disorders. While this was a healthy, non-clinical sample, accumulating evidence suggests that excessive delay discounting may be a pre-existing intermediate phenotype for alcohol use disorders (Anokhin, Grant, Mulligan, & Heath, 2014; Dougherty et al., 2014; James MacKillop, 2013; Smith et al., 2015). Therefore, we probed whether connectivity between these brain networks was associated with aspects of subclinical alcohol use. Partial correlation analyses controlling for age and sex revealed a significant relationship between gPPI contrast estimates and the AUDIT subscale measuring harmful alcohol use (ρ=−0.26, p=0.013), but not the subscales measuring the quantity of alcohol consumption or alcohol dependence symptoms (p>0.05). The gPPI contrast estimates were unrelated to ICR (ρ=0.01, p=0.93), however, indicating that although activity levels of each of these networks during decision making predicts impulsive choices, the connectivity between them does not.

DISCUSSION

This study utilized a novel approach to identify neural networks associated with individual differences in immediate reward selection bias in healthy adults. An ICA-based approach revealed two components of activation during intertemporal choice that associated with ICR: a network of brain regions including the medial temporal lobe, insula and superior temporal gyrus was associated with greater impulsivity, whereas a network encompassing striatal, frontal and parietal brain regions was associated with less impulsive choice behavior. Activity within these two networks was inversely correlated during decision-making, and task-dependent decreases in connectivity between these networks indicate that these brain regions functionally compete during intertemporal choice.

Consilience with existing DD neuroimaging literature

The independent component (#3) whose activity during intertemporal decision-making was negatively correlated with ICR (Fig. 4A) incorporated many brain regions previously proposed to comprise a cognitive control system central to favoring selection of delayed rewards. For example, McClure and colleagues (2004) reported enhanced activity in a constellation of areas with remarkable spatial similarities with Component 3, including the bilateral posterior parietal cortex, right dorsolateral and ventrolateral prefrontal cortex, and right inferior frontal cortex/anterior insula, when subjects opted for larger, later rewards (S. M. McClure, D. I. Laibson, G. Loewenstein, & J. D. Cohen, 2004). Several studies have also identified similar activity related to DD in brain regions corresponding to a right-lateralized frontal-parietal network (Warren K. Bickel et al., 2009; Boettiger et al., 2007; John R. Monterosso et al., 2007; Stanger et al., 2013; Xu, Liang, Wang, Li, & Jiang, 2009), whereas others emphasize the role of the left lateral prefrontal cortex in overriding impulsive choices (Figner et al., 2010; Hare, Camerer, & Rangel, 2009). Notably, Component 3 also incorporates substantial activity in the dorsal and ventral striatum, regions implicated in reward-based decision making processes (Balleine, Delgado, & Hikosaka, 2007; Kable & Glimcher, 2007). The association of less activity within this component with more impulsive choices supports the contention that impulsive choice behavior is related to diminished influence of executive control mechanisms, which override the tangible value of immediate rewards to enable the choice of less tangible but larger delayed rewards (Rick & Loewenstein, 2008). The lateral frontal, anterior cingulate and parietal regions of the Component 3 network (Fig. 3) correspond closely with a previously described working memory network (Zurowski et al., 2002). Notably, working memory training has been shown to decrease DD rates (W. K. Bickel, Yi, Landes, Hill, & Baxter, 2011) suggesting possible shared neural substrates for intertemporal choice and working memory processes. Cognitive control may enable prospective processes to “find” and value delayed rewards (Benoit, Gilbert, & Burgess, 2011; Kurth-Nelson, Bickel, & Redish, 2012; J. Peters & Buchel, 2010; Rick & Loewenstein, 2008). The recent demonstration that a network of lateral prefrontal, dorsomedial prefrontal, and parietal brain regions convert subjective value information from value-encoding regions into actual choices is consistent with this idea (Rodriguez, Turner, Van Zandt, & McClure, 2015). Building on these theories, we propose that engagement of this network may reflect a strategy of utilizing approximate mathematical computations to evaluate choices based on delay interval and objective value of reward options (Arsalidou & Taylor, 2011; Fehr, Code, & Herrmann, 2007).

In contrast to Component 3, described above, Component 9 incorporated activations in the amygdala, hippocampus, parahippocampal gyrus, posterior insula and superior temporal gyrus (Fig. 3). This set of brain regions has been implicated in such processes as memory, emotion, visceral responses, and reward motivation (Gottfried, O’Doherty, & Dolan, 2003). The association between greater activity within this network and greater DD suggests that reliance on affective processes in decision-making biases choices towards the immediate reward (Antoine Bechara, 2005; Gupta, Koscik, Bechara, & Tranel, 2011). In other words, activation of this component may reflect a decision-making strategy in which visceral responses to reward choices guide behavior, consistent with Component 9’s incorporation of posterior insula and amygdala activations (A. Bechara, Damasio, & Damasio, 2003; Craig, 2002). The role of the medial temporal lobe regions in intertemporal choice has alternatively been proposed as supporting episodic prospection (Benoit et al., 2011; J. Peters & Buchel, 2010; Winstanley, Theobald, Cardinal, & Robbins, 2004). Our finding that greater activity in Component 9 was associated with more impulsive decision-making seems in conflict with this hypothesis; however, imagining future outcomes may be more closely related to the functional connectivity between prefrontal and medial temporal lobe regions (Benoit et al., 2011; J. Peters & Buchel, 2010).

Competing networks engaged during subjective choice

The relative balance of activity between a putative fronto-striatal-parietal “control” network and a putative medial temporal and insula “impulsive” network predicted individual differences in intertemporal choice behavior. Furthermore, these networks demonstrated inversely related activity across participants and decreased connectivity across decision-making trials. These relationships suggests that these networks may operate competitively in influencing choice behavior. The relative activation of these anticorrelated networks may not determine choice behavior per se, but rather indicate individual differences in neural strategies engaged to make intertemporal reward decisions. The functional relevance of the anticorrelated nature of these systems was further highlighted by the negative relationship between gPPI estimates and a measure of alcohol-related harm (i.e. negative consequences of drinking). This relationship suggests that subclinical hazardous alcohol use is associated with the reliance on a single neural system while making decisions, which may reflect a deficit in the integration of information across systems.

Other networks engaged during intertemporal choice

Along with Component 3, the activity of which predicted individual differences in DD, Components 4 and 8 were also significantly active across individuals during intertemporal choice. Component 4 has a high spatial correspondence to the well-studied “default-mode network” (Raichle et al., 2001), with activations focused in the posterior cingulate cortex; this network is involved in prospection, and activation in these regions are proposed to represent temporal delays (Luhmann, Chun, Yi, Lee, & Wang, 2008). In contrast, Component 8 contains medial prefrontal and anterior insula activations consistent with the proposed “salience network” (Seeley et al., 2007); however, its focus in the medial prefrontal cortex is consistent with its potential involvement in subjective reward valuation in which the relative values of the reward choices are integrated in a single system (Kable & Glimcher, 2007; J. R. Monterosso & Luo, 2010; van den Bos & McClure, 2013; Wang et al., 2014). It is noteworthy that neither of these components demonstrated a relationship with individual differences in ICR, indicating that these networks are activated regardless of whether subjects tend to select immediate or delayed rewards more frequently.

Intertemporal choice networks and substance use disorders

As elevated DD is linked extensively with substance use disorders (Becker & Murphy, 1988; W. K. Bickel et al., 1999; Dixon et al., 2003; Kris N. Kirby & Petry, 2004; K. N. Kirby et al., 1999; J. MacKillop et al., 2011; John R. Monterosso et al., 2007; Reynolds, 2006), alterations in neural networks predicting individual differences in intertemporal decision-making represent potential biomarkers of alcohol and other substance use disorders. Neurocognitive impairments associated with addictive disorders are related to functional deficits in frontal cortical regions consistent with parts of Component 3 in the current study (Bolla et al., 2003; Hester & Garavan, 2004; Hoffman et al., 2008; Lundqvist, 2010). Drug-dependent individuals also demonstrate altered striatal responses to reward (Kalivas & Volkow, 2005; Kreek & Koob, 1998; Volkow et al., 2010), further implicating the network represented by Component 3 in addiction-related processes. In addition, drug craving elicited by stress or drug cues is associated with increased activity in limbic and paralimbic regions included in Component 9 (Garavan et al., 2000; Kilts et al., 2001; Potenza et al., 2012). Moreover, impulsive behavior associated with drug addiction has been attributed to impairment in executive control over impulsive processes related to motivation for immediate rewards (Antoine Bechara, 2005; Jan Peters & Büchel, 2011), suggesting that interactions between executive and motivational systems may be particularly disrupted in addictive disorders. Furthermore, DD processes are more directly implicated in drug use behavior by data demonstrating that heightened DD among addicts is associated with poorer treatment outcomes (Stanger et al., 2011; Washio et al., 2011). DD among drug-dependent individuals can be reduced by working memory training (W. K. Bickel et al., 2011), suggesting that improving frontal-parietal network functioning and increasing cognitive control has potential as a therapeutic strategy for improving substance use outcomes (Boettiger et al., 2009; Leeman, Bogart, Fucito, & Boettiger, 2014). However, the extent to which changes in these networks influence addiction processes remains to be tested.

Limitations

We acknowledge several limitations of the present study. First, the ICR values for seven subjects fell at either extreme (i.e., 0 or 1), suggesting that the choices offered in the task were not sufficiently challenging for all subjects. Furthermore, the truncated distribution may have limited our power to detect significant correlations with ICR. Additionally, because the difficulty of choice options was not controlled across individuals, some individual differences in neural activity during choice could be driven by differences in the recruitment of decision-making processes, rather than differences in subjective valuation of immediate versus delayed rewards. Finally, this study included both males and females, and sex steroids differences have been linked to differences in DD behavior (Bobova, Finn, Rickert, & Lucas, 2009; Peper et al., 2013; Smith et al., 2014). Although our analyses controlled for sex, and we identified no significant sex effects in our analyses, future studies should consider how the neural correlates of impulsive choice may differ between males and females.