Transforming brain signals related to value evaluation and self‐control into behavioral choices

Abstract

The processes involved in value evaluation and self‐control are critical when making behavioral choices. However, the evidence linking these two types of processes to behavioral choices in intertemporal decision‐making remains elusive. As the ventromedial prefrontal cortex (vmPFC), striatum, and dorsolateral prefrontal cortex (dlPFC) have been associated with these two processes, we focused on these three regions. We employed functional magnetic resonance imaging during a delayed discounting task (DDT) using a relatively large sample size, three independent samples. We evaluated how much information about a specific choice could be decoded from local patterns in each brain area using multivoxel pattern analysis (MVPA). To investigate the relationship between the dlPFC and vmPFC/striatum regions, we performed a psychophysiological interaction (PPI) analysis. In Experiment I, we found that the vmPFC and dlPFC, but not the striatum, could determine choices in healthy participants. Furthermore, we found that the dlPFC showed significant functional connectivity with the vmPFC, but not the striatum, when making decisions. These results could be replicated in Experiment II with an independent sample of healthy participants. In Experiment III, the choice‐decoding accuracy in the vmPFC and dlPFC was lower in patients with addiction (smokers and participants with Internet gaming disorder) than in healthy participants, and decoding accuracy in the dlPFC was related to impulsivity in addicts. Taken together, our findings may provide neural evidence supporting the hypothesis that value evaluation and self‐control processes both guide the intertemporal choices, and might provide potential neural targets for the diagnosis and treatment of impulsivity‐related brain disorders.

1. INTRODUCTION

Human beings often make decisions while considering future events that are delayed in time relative to when the choice is made, this is called intertemporal decision‐making (Green & Myerson, 2004). For example, deciding to make a large purchase or not can affect a person's consumption patterns for months into the future, or deciding to take drugs may put a person at risk of long‐term health problems in individuals with a drug addiction. Intolerance of long‐term rewards, when persistently expressed, is considered maladaptive and symptomatic of several brain disorders, including addiction, attention‐deficit hyperactivity disorder, and affective disorders (Dalley & Robbins, 2017).

Two processes are involved in determining the behavioral choices during intertemporal decision‐making (Figner et al., 2010; Kable & Glimcher, 2007; McClure, Laibson, Loewenstein, & Cohen, 2004): (a) the value evaluation process and (b) the self‐control process. For the value evaluation process, the magnitude of the reward and the delay of options are evaluated and integrated into a single measure. For instance, Kable and Glimcher (2009) and Hare, Hakimi, and Rangel (2014) reported that the ventral medial prefrontal cortex (vmPFC) as well as the striatum assessed various attributes of options and integrated them into a single net value for an option as a whole. Human neuroimaging studies using an intertemporal decision‐making task have shown that blood‐oxygen‐level‐dependent (BOLD) signals in the vmPFC and striatum correlate with the subjective values (Kable & Glimcher, 2007). More recent meta‐analyses have also shown that the vmPFC and striatum are implicated in value representation (Bartra, McGuire, & Kable, 2013; Clithero & Rangel, 2014). An immediate smaller reward might be valued as greater than a larger reward that is delayed, but the delayed option may still be chosen because of intervening self‐control processes (Figner et al., 2010); the dorsolateral prefrontal cortex (dlPFC) is considered to be important in this process (Figner et al., 2010; McClure et al., 2004; van den Bos, Rodriguez, Schweitzer, & McClure, 2015). Furthermore, Hare, Camerer, and Rangel (2009) and Peters and D'Esposito (2016) reported that the dlPFC exerted control over behavior by modulating vmPFC value representations.

In the present study, using multivoxel pattern analysis (MVPA), we sought to test how much information about a specific choice during intertemporal decision‐making could be decoded from local patterns in the brain regions implicated in the value evaluation, as well as self‐control processes. This may provide neural evidence to support the hypothesis that both value evaluation and self‐control processes are critical cognitive components of making behavioral choices during intertemporal decision‐making.

MVPA takes into account the spatial patterns of voxel activity to examine what information is expressed jointly (Cohen et al., 2017). It is an appropriate method for decoding cognitive processes and has been widely used in recent functional magnetic resonance imaging (fMRI) studies (Chavez, Heatherton, & Wagner, 2016; Koizumi et al., 2016), including those on decision‐making (Domenech, Redoute, Koechlin, & Dreher, 2017; Huang, Soon, Mullette‐Gillman, & Hsieh, 2014; Pogoda, Holzer, Mormann, & Weber, 2016; Vickery, Chun, & Lee, 2011; Wang et al., 2014). Neural activity patterns in the frontal lobe can be used to decode subjective choices during a reward‐learning task in humans (Hampton & O'Doherty, 2007) as well as in macaques (Rich & Wallis, 2016).

Steep delay discounting, as a correlate of high impulsivity, has been implicated in addictive behaviors (Amlung, Vedelago, Acker, Balodis, & MacKillop, 2017). Specifically, smokers show higher impulsivity than healthy controls (Barlow, McKee, Reeves, Galea, & Stuckler, 2016). Although Luo, Ainslie, Giragosian, and Monterosso (2011) and Kobiella et al. (2014) have shown that different brain activity in smokers as compared to healthy controls when choosing long‐term reward versus early reward, these two studies did not well control 2 hr of smoking cessation (Hukkanen, Jacob, & Benowitz, 2005). Nicotine may effect on the findings. Participants with Internet‐gaming disorder were also found to present steeper discounting than recreational Internet game users or healthy individuals (Wang et al., 2016). In their study, participants with Internet‐gaming disorder showed altered brain activities in the anterior cingulate cortex and parahippocampal gyrus compared to recreational Internet‐gaming users, as well as in the inferior frontal gyrus as compared to healthy controls, in trials choosing between long‐term reward versus immediate reward. Therefore, previous studies did not implicate self‐control–related and value‐evaluation–related brain regions in impulsive choices in participants with Internet‐gaming disorder. Although the great potential of MVPA in decision‐making studies has been revealed, previous studies (e.g., Luo et al. (2011) and Kobiella et al. (2014)) did not use MVPA to relate brain activity patterns to impulsive choices in smokers, which involve abnormal decision‐making. In the present study, we also sought to decode choices from brain activity in addictive disorders.

Three regions of interest (ROIs), including the vmPFC and striatum, implicated in value evaluation processes, and the left dlPFC, implicated in the self‐control process, were specified according to previous studies (Figner et al., 2010; Hare et al., 2009; Kable & Glimcher, 2007; Li et al., 2013; McClure et al., 2004). Using MVPA, we evaluated the amount of information about specific choices that could be decoded from local patterns in each ROI. If two or all brain regions contained information about specific choices, it would be reasonable to test whether brain networks are built for making decisions. We assessed this via PPI analysis. However, as the replication of decoding and functional connectivity results may be poor for an independent data set (Poldrack et al., 2017), we tested its reliability in Experiment II to provide validation for the results of Experiment I, using independent data sets with different age, education, and location, acquired on different MRI scanners. Finally, we performed Experiment III to test whether the decoding accuracy was impaired in patients with mental disorders involving abnormal impulsivity, for example, individuals with addictions.

2. MATERIALS AND METHODS

2.1. Experiment I

2.1.1. Participants

Forty‐six cognitively healthy male participants were recruited in the present study [age: mean, 23.8 years; SD, 1.7 years; range, 20–27 years; education: mean, 17.0 years; SD, 1.8 years; range, 12–21 years]. To avoid intense emotions effect on the decision‐making (Al Omari, Razeq, & Fooladi, 2016; Lempert & Phelps, 2016), females were not recruited in the present study. All the participants had normal or corrected‐to‐normal vision, and reported no history of neurological or psychiatric disorders, or the use of addictive drugs. The study procedures were performed by a trained PhD student. All participants had negligible head motion (translation <2 mm and rotation <2°) during fMRI scanning.

This study was approved by the Human Research Ethics Committee of the University of Science and Technology of China. The methods and procedures used in this study were carried out in accordance with the approved guidelines. Written informed consent was obtained from all participants before the study, and adhered to the tenets of the Declaration of Helsinki.

2.1.2. Task paradigm

We used a DDT as in our previous study (see Figure 1a in Li et al., 2013). The first screen presented the immediate option, which always offered a reward of a magnitude of 50 Chinese Yuan at a delay of 0 days, followed by the delayed option. Delayed options were combined with two sets of monetary magnitudes, and two sets of time delays, yielding two runs with 42 trials each (see Supporting information Methods for more details). The order of the two runs was counterbalanced across participants.

Figure 1.

Beta values for subjective value and difficulty level in Experiment I. (a) The mean BOLD signal of the ventromedial prefrontal cortex (vmPFC) and striatum was significantly correlated with the subjective value. The mean BOLD signal of the left dorsolateral prefrontal cortex (dlPFC) was not significantly correlated with the subjective value. (b) The mean BOLD signal of the left dlPFC was significantly correlated with the difficulty level. The mean BOLD signal of the vmPFC and striatum were not significantly correlated with the difficulty level. *Significant at p < .05/3; n.s., not significant; error bars, SEM; SV, subjective value

A practice version of the DDT (ca. 5 min and without any payment) was performed before scanning, to familiarize participants with the task. To verify that discounting measures were reliable, participants were informed that they would obtain actual payment in cash on one randomly drawn trial of the task based on their choice. Participants obtained the payment at the end of the scanning if the outcome of the selected trial was an immediate gain; otherwise, payment was sent to participants with the specified delay (Li et al., 2013).

2.1.3. Behavioral analysis

The details of behavioral analysis have been described in our previous study (Li et al., 2013). Briefly, the behavioral choices were fit using a logistic function to calculate the monetary amount where there was an equal probability of selecting the immediate versus the delayed option at the specific delay. Then, the monetary amount was defined as the indifference point, to calculate the discounted value (DV, fraction of immediate value) for each delay,

$$\mathrm{DV}=\frac{\text{magnitude of immediate reward}}{\text{indifference point}},$$ (1)

where the magnitude of immediate reward in the present study was always 50 Chinese Yuan. DVs were fit against the delays (D) with the hyperbolic discounting model (Green & Myerson, 2004),

$$\mathrm{DV}=\frac{1}{1+k\times D},$$ (2)

called the discounting curve, where k is an individual's discounting rate, with larger k values indicating higher impulsivity in decision‐making. Before the analyses, the discounting rate, k, was normalized by log transformation (Hariri et al., 2006).

The subjective value for each delayed option was estimated by multiplying the money magnitude of the delayed option by the fraction for that delay (Kable & Glimcher, 2007).

Here, we used “difficulty” to indicate self‐control level, as a previous study (McClure et al., 2004) suggested that impulse control may be preferentially elicited during difficult decisions. Trials near the discounting curve demand more consideration (i.e., more time to make a decision) and are considered as “hard” trials, whereas trials far from the discounting curve represent “easy” trials. Hard trials and easy trials were defined, to distinguish among the difficulty levels of decision‐making (Hoffman et al., 2008). The distance to the discounting curve (dist) for each trial was defined as ${\mathrm{DV}}^{\prime }-\widehat{\mathit{DV}}$, where

$${\mathrm{DV}}^{\prime }=\frac{\text{magnitude of immediate reward}}{\text{magnitude of delayed reward}},$$ (3)

and$\widehat{\mathit{DV}}$ was the predicted value based on a participant's discounting curve for the corresponding delay. Using the normal distribution function, we fit response time (RT) against the dist:

$$\mathrm{RT}=A^{*}\exp \left(-\frac{1}{2}{\left(\frac{\mathit{\text{dist}}}{\sigma }\right)}^2\right)+C.$$ (4)

Trials outside the region ±σ were defined as easy trials and the remaining trials included in the region were defined as hard trials. For trials with a delayed option involving either 50 Yuan or 0 days, participants could make choices by simply comparing the digits of the other information in the options, without integrating information about money magnitude and time delay that is considered to be an essential process in the DDT (Green & Myerson, 2004; Li et al., 2013). Therefore, these trials, labeled as control trials, were excluded from the hard and easy trials (see Supporting information Figure S1) and from decoding analysis.

2.1.4. fMRI data acquisition and preprocessing

All MRI data were acquired using 3‐T, 8‐channel head coil Siemens Magnetom Trio scanners (Siemens Medical Solutions, Erlangen, Germany) in the Anhui Provincial Hospital, Hefei, China. A circularly polarized head coil with foam padding was used to restrict head motion. Functional images were acquired using a T2*‐weighted echo‐planar imaging sequence (Repetition Time [TR] = 2 s, Echo Time [TE] = 30 ms, Field of view [FOV] = 240 mm) with 33 axial slices (3.7‐mm thickness, no gap), covering the whole brain. Two runs of 257 volumes were acquired. Between the two runs, there was an interval of ca. 1 min. High‐resolution T1‐weighted spin‐echo images were also collected for anatomical overlay.

The preprocessing protocol was the same as that used in our previous study (Li et al., 2013). The imaging data were processed using Analysis of Functional Neuroimages (version AFNI_2011_05_26_1457) software (Cox, 1996). The first two volumes of each run were discarded. The volumes were corrected for temporal shifts between slices, corrected for head motion, and grand‐mean scaled. We also performed linear trend removal using linear regression.

2.1.5. fMRI data analysis. Step 1: General Linear Model (GLM) 1 for subjective value and difficulty

We used a GLM to confirm whether the BOLD signals in our ROIs (i.e., the vmPFC, striatum, and dlPFC) were correlated with the subjective value or difficulty level.

The preprocessed fMRI data were spatially smoothed with a Gaussian kernel (full‐width at half‐maximum [FWHM] = 8 mm). This spatial smoothing was carried out selectively for this GLM. GLM analysis employed a multiple regression model that included one subjective value regressor for the delayed option (defined as subjective value when both money and time information of the delayed option was presented; 0 for the other epochs), and two choice regressors for hard and easy trials (defined as 1 when current trial was hard one or easy one, respectively, and 0 for the other epochs), and several noninterest regressors, including those for head motion (six motion parameters), a constant for each of the two runs, and general period effects (defined as 1 for general period effects for the epochs when immediate option was presented or when the first information of delayed option was presented or when choice was required, respectively, and 0 for the other epochs).

The map of beta values of the subjective value regressor and of the hard minus easy regressors (difficulty level) was based on the multiple regression analysis findings, and were then transformed to Talairach space (resampled voxel size: 3 × 3 × 3 mm³) according to the spatial transformation between the anatomical data and the Talairach space. To avoid “double dipping” (Kriegeskorte, Simmons, Bellgowan, & Baker, 2009), the coordinates of the ROIs (Table 1) were specified as coordinates of the voxel of interest according to our previous study (Li et al., 2013), and 5‐mm spheres surrounding the coordinates were defined as masks for the ROIs. For generalizing our results, analyses with other radius (6 mm, 8 mm, and 10 mm) were also performed (see Supporting information Methods and Table S2). As reported in our previous study (Li et al., 2013), activity in the right vmPFC and right striatum (Table 1) was significantly associated with the valuation process, which was consistent with results of other studies (Kable & Glimcher, 2007; McClure et al., 2004; Pine et al., 2009). The left dlPFC was specified as a ROI because most previous studies have implicated the left, but not right, dlPFC in self‐control (Figner et al., 2010; Hare et al., 2009). Note that the participants in the present study were independent from those in our previous study (Li et al., 2013). The average beta values for subjective value and difficulty within a mask were acquired for each ROI and for each participant.

Table 1.

ROIs defined by our previous study (Li et al., 2013)

Regions	x a	y	z
vmPFC	4.9	32.5	13.7
Striatum	10.5	10.8	8.8
Left dlPFC	−41.6	21.3	32.1

Note. dlPFC = dorsolateral prefrontal cortex; L = left; R = right; ROI = region of interest; vmPFC = ventromedial prefrontal cortex.

^aVoxel coordinates in Talairach space.

The average beta value was entered into a group‐level one‐sample t test separately for each ROI, for the subjective value as well as for the difficulty level. Bonferroni correction was used to correct for multiple comparisons (corrected p < .05/3). The Cohen's d values were calculated via G*Power 3.1 software (Faul, Erdfelder, Lang, & Buchner, 2007). We also corrected the ROI by using small volume correction (Supporting information Methods and Results for more details). We also performed whole‐brain analysis for the subjective value and difficulty, and results were shown in Supporting information Figure S2.

We tested whether the main effect of behavioral choices (i.e., choosing long‐term rewards vs. choosing short‐term rewards) reached significance in the vmPFC and dlPFC through traditional univariate methods, that is, GLM (Supporting information Methods and Results).

2.1.6. Step 2: Decoding from each ROI activity using pattern classification

We evaluated the amount of information about a specific choice that could be decoded from local patterns in each ROI. Data extraction, that is, Least Squares—Separate, was adapted from a previous study (Mumford, Turner, Ashby, & Poldrack, 2012). The preprocessed volumes were transformed to Talairach space (resampled voxel size: 3 × 3 × 3 mm³) according to the spatial transformation between the anatomical data and the Talairach space. We ran a separate GLM for each choice (i.e., both immediate and delayed choices), wherein the choice was modeled as the regressor of interest and all other choices were combined into a single regressor. Similar to the previous study (Mumford et al., 2012), the regressors only modeled the time points between 2 s and 6 s after the choices, to capture the peak of the response. The GLM also included the six motion parameters as nuisance variables. This analysis was based on unsmoothed data to allow extraction of the complete information in the spatial patterns and to maximize sensitivity (Soon, Brass, Heinze, & Haynes, 2008).

In this study, we adapted MVPA from a previous study (Soon et al., 2008). The approach examined the information in local patterns of brain activity surrounding each voxel of interest, v_i. Therefore, for each voxel v_i, we examined whether its local pattern contained spatial information that could decode the specific choice. For a given voxel v_i, we defined a spherical cluster of voxels (M_1…N) with 33 voxels (Kriegeskorte, Goebel, & Bandettini, 2006) centred on voxel v_i. For each voxel M_1…N in the fixed spherical cluster, we extracted the activation (t value for the choice trial) separately for immediate and delayed choice trials. This generated two N‐dimensional pattern vectors, namely V_{I, 1…N} and V_{D, 1…N}, representing the local response patterns in the spherical cluster in trials in which the participants chose between an immediate and a delayed option. By recognizing local patterns related to each choice, these pattern‐based decoders were trained to predict the specific choice (fivefold cross‐validation) (Soon et al., 2008). Participant's choices that were extracted from two runs were used to train and test the classifier. The fivefold cross‐validation was performed using the Statistics and Machine Learning Toolbox in MATLAB (version 8.6.0.267246; MathWorks, Natick, Massachusetts, America), yielding the average decoding accuracy of the spherical environment of the central voxel v_i. We used a radial basis kernel, and the cost parameter was 5.06 (Mumford et al., 2012). The procedure was then repeated for the next position at the voxel of interest v_j. Finally, the average decoding accuracy for each ROI was acquired for each participant (see Supporting information Methods and Tables for more details).

We investigated the amount of information that could be decoded from local patterns for each ROI. Therefore, for each ROI, the decoding accuracy was tested against chance level. To assess whether decoding accuracy was influenced by impulsivity, correlations between the discounting rate and the decoding accuracy from each ROI were tested using Pearson's correlations. Bonferroni correction was used to correct for multiple comparisons.

2.1.7. Step 3: Psychophysiological interaction

To investigate the relationship between the dlPFC and vmPFC or striatum regions, we performed a PPI analysis (McLaren, Ries, Xu, & Johnson, 2012). We sought to determine whether the dlPFC showed greater functional connectivity with the vmPFC or striatum when participants selected an option (we also tested whether the dlPFC showed greater functional connectivity with the vmPFC or striatum when choosing delayed choice than immediate choice, however, the results were not significant(Supporting information Methods and Results). In the analysis, we estimated a GLM for the dlPFC using the following protocol (Minati, Grisoli, Seth, & Critchley, 2012). First, we created a time series by extracting the mean time courses within the dlPFC. Second, we computed the interaction terms between the dlPFC and decision, which equals 1 if the participant chooses the immediate or delayed option, and 0 at baseline. Third, we estimated a PPI GLM for the dlPFC containing the following regressors: (a) decision, (b) dlPFC time series, (c) interaction, and (d) all the regressors in GLM 1 (see Step 1). We computed contrasts for the interaction regressor for the dlPFC for each participant. Note that this contrast examines functional connectivity between the dlPFC and vmPFC or striatum ROIs, which is similar to the process described by Lim et al. (2016).

The contrasts for the interaction regressor were averaged for the vmPFC and striatum, yielding two connectivity strengths for each participant. To investigate whether the dlPFC showed functional connectivity with the two ROIs when participants selected an option, the two connectivity strengths were entered in a group t test. Bonferroni correction was used to correct for multiple comparisons.

2.1.8. Step 4: ROIs acquired from one meta‐analysis about subjective value

To complement brain regions activated during DDT task, we also re‐ran all analysis with ROIs reported in the meta‐analysis about subjective value (Clithero & Rangel, 2014). The results were shown in Supporting information Table S1. Whether the prediction error or subjective goal value, can better account for striatal BOLD activity was also tested (see Supporting information Methods and Results).

2.2. Experiment II

2.2.1. Participants

Twenty‐seven cognitively healthy male participants were recruited for Experiment II. None of the participants reported any history of neurological or psychiatric disorders, or the use of addictive drugs. This procedure was performed by a trained PhD student. All participants performed the same version of the DDT used in Experiment I. Five participants were excluded due to excessive head motion (translation >2 mm or rotation >2°) during fMRI. The remaining participants (age: mean, 20.0 years; SD, 1.6 years; range, 18−24 years; Education: mean, 14.2 years; SD, 1.6 years; range, 12−17 years) were used for the final analysis.

2.2.2. Data acquisition and analysis

All the MRI data were acquired using 3‐T, 12‐channel head coil Siemens Magnetom Trio scanners (Siemens Medical Solutions, Erlangen, Germany) in the Xuanwu Hospital Capital Medical University, Beijing. Experiment II used the same scanning parameters as those used in Experiment I. The behavioral analysis, GLM analysis, and decoding analysis were performed as in Experiment I.

2.3. Experiment III

2.3.1. Participants

Participants with Internet‐gaming disorder

Nineteen male participants with Internet‐gaming disorder (age: mean, 20.5 years; SD, 3.1 years; range, 16−25 years; Education: mean, 12.3 years; SD, 2.7 years; range, 9−17 years) were recruited. Participants with Internet‐gaming disorder met the Diagnostic and Statistical Manual of Mental Disorders the 5th Edition (DSM‐5) criteria for Internet‐gaming disorder (facing at least five of the nine inclusionary criteria over a 12‐month period). None of the participants reported any history of neurological or psychiatric disorders, or the use of addictive drugs, and performed the same version of the DDT used in Experiment I. The criteria for participants with Internet‐gaming disorder were based on structured clinical interviews conducted by two psychiatrists.

Smokers

Thirteen male cigarette smokers (age: mean, 25.3 years; SD, 2.7 years; range, 22−31 years; Education: mean, 15.4 years; SD, 2.1 years; range, 11−19 years) were recruited. None of the participants reported any history of neurological or psychiatric disorders, or the use of addictive drugs (except nicotine), and performed the same version of the DDT used in Experiment I.

The number of cigarettes smoked per day for all smokers was greater than 10 for at least 1 year. This procedure was overseen by at least one author. Time period of 2 hr of smoking cessation was required after having smoked their last cigarette, as the half‐life of nicotine is about 2 hr (Hukkanen et al., 2005). During this 2‐hr period, smokers wrote informed consent and read standardized instructions, and completed several questionnaires. These procedures were performed under the supervision of one author.

This study was approved by the Human Research Ethics Committee of the University of Science and Technology of China. The methods and procedures used in this study were carried out in accordance with the approved guidelines. Written informed consent was obtained from all participants before the study, and adhered to the tenets of the Declaration of Helsinki.

2.3.2. Data acquisition

Participants with Internet‐gaming disorder

All MRI data for participants with Internet‐gaming disorder were acquired using the same scanners as those in Experiment II at the Xuanwu Hospital Capital Medical University, Beijing. We used the same scanning parameters as those used in Experiment II.

Smokers

For the smokers, all MRI data were acquired using the same scanners as those in Experiment I at the Anhui Provincial Hospital, Hefei. We used the same scanning parameters as those used in Experiment I.

2.3.3. Data analysis

Participants with Internet‐gaming disorder

The behavioral analysis, PPI, and decoding analysis followed the same procedures as those used in Experiment I. We investigated the amount of information that could be decoded from local patterns for each ROI in participants with Internet‐gaming disorder. Therefore, for each ROI, the decoding accuracy was tested against chance level. We tested whether the decoding accuracy in participants with Internet‐gaming disorder was lower than that in healthy participants, using a t test.

2.3.4. Smokers

The analyses for smokers' data were the same as those used for the data of participants with Internet‐gaming disorder.

All participants

We also performed group comparisons on the functional connectivity between the dlPFC and vmPFC or striatum and decoding accuracy for each ROI using one‐way analysis of variance (ANOVA), and results were shown in Supporting information Results. We performed group comparisons on the discounting rate in all participants (healthy participants, smokers, and individuals with Internet‐gaming disorder) using one‐way ANOVA. We also performed analysis of covariance (ANCOVA) while controlling for their age and education level. To determine whether decoding accuracy from all ROIs differed between participants with Internet‐gaming disorder and smokers, we performed two‐way repeated ANCOVA (3 ROIs × 2 Groups) while controlling for their age and education level.

We tested whether the discounting rate was related to the decoding accuracy for each ROI in patients with Internet‐gaming disorder and smokers, controlling for the variance arising from different sites. Previous studies about addiction (DeWit, Adlaf, Offord, & Ogborne, 2000; Filbey et al., 2014; Wannamethee, Camargo, Manson, Willett, & Rimm, 2003) and delay discounting task (Achterberg, Peper, van Duijvenvoorde, Mandl, & Crone, 2016; Scheres, Tontsch, Thoeny, & Sumiya, 2014) have shown that typical nonlinear relationship in human brain or behaviors. For example, Filbey et al. (2014) found that the quadratic relationship between duration of marijuana use and forceps minor's fractional anisotropy. Achterberg et al. (2016) found that rather than linear model, age‐related change in impulsivity was best described by a quadratic age‐model linear. A general linear model was used to test this relationships. The corrected Akaike information criterion (AICc) was used to determine model selection (Filbey et al., 2014). Model 1: y = bx + cz + d + ε; Model 2: y = ax² + bx + cz + d + ε, where y is discounting rate, x is decoding accuracy, z is group factor, a,  b,  c, and d are parameters, and ε is a vector of errors. The t‐test was run for b parameter if Model 1 was selected. The t‐test was only run for a parameter if Model 2 was selected, as Model 2 meant nonlinear relationship between discounting rate and accuracy.

3. RESULTS

3.1. Experiment I

The demographic data of participants in Experiment I are shown in Table 2. Group analysis showed that the mean BOLD signal in the vmPFC (t₄₅ = 3.772,  p_corrected = .001, Cohen^’s d = .556) and striatum (t₄₅ = 4.506,  p_corrected < .001,  Cohen^’s d = .664) were significantly correlated with the subjective value (Figure 1a). The mean BOLD signal in the left dlPFC (t₄₅ = 1.572,  p_corrected = .369) was not significantly correlated with the subjective value (Figure 1a). Group analysis showed that the mean BOLD signal in the left dlPFC (t₄₅ = 6.560,  p_corrected < .001,  Cohen^’s d = .967) was significantly correlated with the difficulty level (Figure 1b). The mean BOLD signals in the vmPFC (t₄₅ = 2.374,  p_corrected = .066) and striatum (t₄₅ =  − 0.815, p_corrected > .99) were not significantly correlated with the difficulty level (Figure 1b).

Table 2.

Group characteristics in Experiments I, II, and III

	Experiment I	Experiment II	Experiment III		F‐value	p‐value
	Healthy participants (n = 46)	Healthy participants (n = 22)	Participants with Internet‐gaming disorder (n = 19)	Smokers (n = 13)
Sex (% male)	100%	100%	100%	100%	—	—
Agea – d (mean ± SD)	23.8 ± 1.7	20.0 ± 1.6	20.5 ± 3.1	25.3 ± 2.7	28.56	<.001
Formal educationa – c , d , e (mean ± SD)	17.0 ± 1.8	14.2 ± 1.6	12.3 ± 2.7	15.4 ± 2.1	27.16	<.001
Discounting rateb – d , f (mean ± SD)	−4.25 ± 0.74	−4.16 ± 0.91	−3.73 ± 0.72	−3.07 ± 0.88	8.57	<.001
The number of delayed choices	11~50	10~49	11~44	4~38	—	—
The number of immediate choices	10~49	11~50	16~48	21~56	—	—

^aSignificant difference between Experiments I and II.

^bSignificant difference between Experiments I and participants with Internet gaming disorder.

^cSignificant difference between Experiments I and smokers.

Significant difference between Experiments II and smokers.

^dSignificant difference between participants with Internet‐gaming disorder and smokers.

^eSignificant difference between Experiment II and participants with Internet‐gaming disorder.

^fLog (discounting rate) transformed.

We found that the local patterns in the vmPFC (t₄₅ = 3.045, p_corrected = .012,  Cohen^’s d = .449) and the left dlPFC (t₄₅ = 3.132,  p_corrected = .009,  Cohen^’s d = .462), but not those in the striatum (t₄₅ = 0.957,  ns), contained information encoding behavioral choices (Figure 2).

Figure 2.

Decoding the choices from local patterns in all the regions of interest (ROIs) in Experiment I. Healthy participants in Experiment I. *Significant at p < .05/3; n.s., not significant; error bars, SEM

The left dlPFC showed significant functional connectivity with the vmPFC (t₄₅ = 5.686,  p_corrected < .001,  Cohen^’s d = .838), but not with the striatum (t₄₅ = 1.682,  p_corrected = .199) when participants selected an option versus baseline (Figure 3).

Figure 3.

Functional connectivity between dorsolateral prefrontal cortex (dlPFC) and other regions of interest (ROIs) in the value evaluation process when participants selected an option versus baseline during the delayed discounting task (DDT) in Experiment I. *Significant at p < .05/2; n.s., not significant; error bars, SEM

Correlations between the discounting rate and decoding accuracy for each ROI were not significant (all p_corrected > .2).

3.2. Experiment II

The demographic data of participants in Experiment II are shown in Table 2. The results of the mean BOLD signal related to subjective value (Figure 4) and difficulty (Figure 4) were consistent with those observed in Experiment I (see Supporting information Results). Furthermore, the decoding accuracy based on local patterns (Figure 5) in all the ROIs and functional connectivity (Figure 6) were consistent with those observed in Experiment I (Supporting information Results).

Figure 4.

The beta value separately for subjective value as well as difficulty level and for each ROI in Experiment II. (a) The significant regions of interest (ROIs) of subjective value regressor included ventromedial prefrontal cortex (vmPFC) and striatum. The left dorsolateral prefrontal cortex (dlPFC) was not significantly correlated with the subjective value regressor. (b) The significantly activated ROI of difficulty level was the left dlPFC. The vmPFC and striatum was not significantly correlated with difficulty level. *Significant at p < .05; n.s., not significant; error bars, SEM

Figure 5.

Decoding the choices from local pattern in all regions of interest (ROIs) in Experiment II. *Significant at p < .05; n.s., not significant; error bars, SEM

Figure 6.

Functional connectivity between dorsolateral prefrontal cortex (dlPFC) and other regions of interest (ROIs) in the value evaluation process in Experiment II. *Significant at p < .05; n.s., not significant; error bars, SEM

3.3. Experiment III

The demographic data of participants in Experiment III are shown in Table 2.

3.3.1. Participants with Internet‐gaming disorder

We found that the local patterns of all ROIs (all p_s > .2) for participants with Internet‐gaming disorder did not contain information encoding behavioral choices. We found that the vmPFC (t₈₅ = 2.153,  p = .034,  Cohen^’s d = 0.559), but not the striatum or the dlPFC (both p_s > .1), in patients with Internet‐gaming disorder showed significantly lower decoding accuracy values compared to the healthy participants (Figure 7).

Figure 7.

Group differences on decoding accuracy between Experiments I, II, and III. Compared to the healthy participants in Experiments I and II, participants with Internet‐gaming disorder and smokers in Experiment III showed significantly lower decoding accuracy from the ventromedial prefrontal cortex (vmPFC) and the left dorsolateral prefrontal cortex (dlPFC), respectively. *Significant at p < .05; n.s., not significant; error bars, SEM

3.3.2. Smokers

We found that the local patterns of all ROIs (all p_s > .2) in smokers did not contain information encoding behavioral choices. We found that the left dlPFC (t₈₅ = 3.005,  p = .004,  Cohen^’s d = 0.910), but not the striatum or the vmPFC (both p_s > .3), in smokers showed significantly lower decoding accuracy values than those of healthy participants (Figure 7).

All participants

One‐way ANOVA showed a significant group effect on the discounting rate (F_(3, 96) = 8.573,  p < .001, partial η2 = .211). The group effect on the discounting rate remained significant after controlling for their age and education level (p = .001). Post hoc analysis with the least significant difference method showed that the discounting rates in participants with Internet‐gaming disorder and smokers in Experiment III were higher than that in healthy participants in Experiment I (both p_s < .05) (Table 2). There were no significant differences in the decoding accuracy values of participants with Internet‐gaming disorder and smokers (all p_s > .36) while controlling for their age and education level.

There was a significant nonlinear relationship between the discounting rate and decoding accuracy from the left dlPFC in participants with Internet‐gaming disorder and smokers (t₂₈ = 2.234, p = .034, Figure 8 and Table 3). No significant relationships were found between discounting rate and decoding accuracy from the vmPFC or striatum (both p_s ≥ .1).

Figure 8.

Relationship between discounting rate and decoding accuracy from the left dorsolateral prefrontal cortex (dlPFC) in patients with Internet‐gaming disorder and smokers (Experiment III). There was a significant nonlinear relationship between discounting rate and decoding accuracy from the left dlPFC in patients with Internet‐gaming disorder and smokers. Nonlinear relationship suggests that both extremely good and extremely poor decoding accuracy in dlPFC is related to steeper discounting rates. This may demonstrate the complexity of addictive behaviors' effect on the impulsivity, particularly on addictive behaviors' interaction with brain. The discounting rates were normalized by log transformation

Table 3.

Associations between discounting rate and decoding accuracy in Experiment III

	Decoding accuracy	Parametric modela	a	b	c	t	Ddf	p
	vmPFC	Model 1	—	−0.002	0.666	−0.263	29	.795
Discounting rate	Striatum	Model 2	0.001	−0.157	0.460	1.992	28	.056
	Left dlPFC	Model 2	0.001	−0.118	0.557	2.234	28	.034

Note. AICc = Akaike information criterion; dlPFC = dorsolateral prefrontal cortex; vmPFC = ventromedial prefrontal cortex.

^aThe model selection was controlled by AICc.

4. DISCUSSION

This study provides neural evidence supporting the hypothesis that both value evaluation and self‐control processes are critical cognitive components for behavioral outcome, that is, participants' choices during intertemporal decision‐making. Activity in both the vmPFC and the striatum was correlated with the subjective goal value, and activity in the dlPFC was also correlated with the difficulty level. In Experiments I and II, the dlPFC and the vmPFC, but not the striatum, contained information that encoded specific behavioral choices. Furthermore, the dlPFC showed functional connectivity with the vmPFC, but not the striatum, during decision‐making. In Experiment III, as compared to healthy participants, the decoding accuracy of the choices in the vmPFC and dlPFC was decreased in individuals with addictions. Our data extended the role of these brain areas to determining behavioral choices in the context of intertemporal decision‐making and provide key insights into why patients with addictions prefer immediate reward at the risk of long‐term physical and mental health or academic performance.

4.1. Brain regions involved in value evaluation and self‐control processes guiding participant choices

We found that the vmPFC, which is related to value evaluation (Bartra et al., 2013; Clithero & Rangel, 2014), and the dlPFC, which is related to self‐control (Figner et al., 2010; Hare et al., 2009), could guide human behavioral choices during intertemporal choices by spatial patterns of activity of multiple voxels. Previous studies have also revealed that various types of decision‐making could be decoded from brain activity (Domenech et al., 2017; Huang et al., 2014; Pogoda et al., 2016; Vickery et al., 2011).

In the present study, the main effect of behavioral choices (i.e., choosing long‐term rewards vs. choosing short‐term rewards) failed to reach significance in the vmPFC and dlPFC through traditional univariate methods, that is, GLM (see Supporting information Methods and Results). This is consistent with previous studies using univariate methods in the GLM on healthy adults. Particularly, Kable and Glimcher (2007) and Monterosso et al. (2007) used whole‐brain exploratory analyses and reported no brain areas corresponding to the main effect of choice. Using ROI analyses, several other studies have shown that activity in the dlPFC, a key brain area related to self‐control processes, also does not correlate with the main effect of choice (Jimura, Chushak, Westbrook, & Braver, 2017; Monterosso et al., 2007), although shallow discounters show greater dlPFC activation than steep discounters when making difficult delayed choices (Jimura et al., 2017). Hare et al. (2014) have shown that the dlPFC showed greater activity when choosing long‐term rewards; however, such activation does not control general decision processes well, as they compared choosing long‐term rewards with baseline. Another study found that the insula might be implicated in choice making; however, this study used hypothetical rewards (Wittmann, Leland, & Paulus, 2007).

In contrast to univariate methods, MVPA allowed us to examine the spatial patterns of ensembles of voxel activities, to determine the type of information collectively represented in this way. MVPA was sensitive to this information, as neuronal populations may be distributed heterogeneously across voxels (Cohen et al., 2017). Therefore, our MVPA findings extended the role of these brain regions to encoding specific behavioral choices and may provide neural evidence supporting the hypothesis that both value evaluation and self‐control processes are critical cognitive components in making behavioral choices (Figner et al., 2010; Kable & Glimcher, 2007). The decoding accuracy in the vmPFC and dlPFC was not correlated with impulsivity in participants, suggesting that the decoding analysis can be potentially generalized to more or less impulsive healthy individuals.

We also identified functional connectivity between the left dlPFC, involved in the self‐control process, and vmPFC, involved in the value evaluation process, when making decisions. dlPFC−vmPFC coupling has been implicated in behavioral choices during intertemporal decision‐making (Hare et al., 2014; Steinbeis, Haushofer, Fehr, & Singer, 2016) and in self‐control processes in other decision‐making tasks involving self‐control. Taken together with the finding that the vmPFC and dlPFC could guide choices, this finding further indicates that the vmPFC and dlPFC form part of brain networks necessary for decision‐making.

There is an ongoing debate about the precise functional role of the dlPFC in the self‐control involved in decision‐making (Kable, 2010). The dlPFC might act in opposition to immediate choices, that is, a more influence on choice (Figner et al., 2010), or alternatively, the dlPFC might modulate the value representations elsewhere in the brain, that is, inhibitory relationship with the vmPFC (Hare et al., 2009; Lim et al., 2016). We found that the dlPFC could encode the specific choice, and that the dlPFC was positively connected with the vmPFC when selected an option, and that the dlPFC did not show a significant inhibitory relationship with the vmPFC when selected delayed choice versus immediate choice (see Supporting information Methods and Results). Based on these findings, our data support the former proposal: the dlPFC plays a role in the choice process.

4.2. Does the striatum play a role in the computation of subjective values during intertemporal decision‐making?

In the present study, we found that the activities of the vmPFC and striatum were correlated with the subjective goal value. This is consistent with results from previous fMRI studies that have identified the vmPFC as well as the striatum as playing a role in value evaluation during decision‐making (Kable & Glimcher, 2007; Li et al., 2013). Interestingly, we found that the vmPFC, but not the striatum, contained information that encoded behavioral choices and that the dlPFC was connected to vmPFC, but not to the striatum, during decision‐making. Our data suggest disassociated roles for the vmPFC and the striatum in value computation during intertemporal decision‐making.

Multiple value computations, including prediction errors and subjective goal values, are critical to making adaptive choices (Hare, O'Doherty, Camerer, Schultz, & Rangel, 2008; Schultz, Dayan, & Montague, 1997). Specifically, prediction errors measuring the deviations from previous reward predictions are used to learn the value of the conditions of the world, and are, therefore, primary contributors to making predictions. Subjective goal values, measuring the predicted reward for each action under consideration, are used to guide the actions in order to acquire the most valuable benefit when making decisions.

According to the disassociated roles of the vmPFC and striatum in the computation of subjective goal values and prediction errors in food‐based bid decision‐making (Hare et al., 2008), there are two possible explanations of striatal activation during intertemporal decision‐making. First, the striatum responds to prediction error and, therefore, its activation in intertemporal decision‐making may be due to the strong correlation between subjective value and prediction error (Hare et al., 2008). Hare et al. (2008) suggested that people track the average value of previously obtained outcomes, and are positively surprised when receiving a relatively good offer. The prediction error at the time of choice offer during intertemporal decision‐making is equal to the subjective value of the current offer minus the prediction derived from the outcome of previous offers. Second, the striatum responds to the subjective goal value during intertemporal decision‐making, as this context is different from that in food‐based bid decision‐making(Knutson, Rick, Wimmer, Prelec, & Loewenstein, 2007). If the former is correct, the vmPFC, and not the striatum, should represent behavioral choices during intertemporal decision‐making, and the vmPFC should be functionally connected with the dlPFC during decision‐making. Prediction errors primarily update reward prediction for each option. They unlikely influence the comparisons of reward prediction between options. Therefore, we inferred choices could not be decoded from striatum if it responded to prediction error. Brain regions related to prediction errors may be not connected to brain regions related to self‐control. One study showed that the striatum responds to prediction error and shows functional connectivity with the dlPFC during the feedback phase (Park et al., 2010). The prediction error during the feedback phase should be responsible for trial‐by‐trial learning, which may be different from that during the decision phase. If the latter is correct, the vmPFC and the striatum should both represent behavioral choices and show functional connectivity with the dlPFC.

As we found that the vmPFC, but not the striatum, contained information that encoded behavioral choices, and that the dlPFC showed connectivity with the vmPFC, but not the striatum, during decision‐making, our data support the former alternative. Furthermore, we performed additional analyses, and found that the prediction error, than subjective goal value, can better account for striatal activity (see Supporting information Methods and Results). The striatum may be involved in learning the value of the conditions of the world; prediction errors may then be delivered primarily to the vmPFC, which is responsible for making the predictions. The vmPFC measures the predicted reward for each action under consideration and uses these predictions to guide behavioral choices. However, whether these two brain regions are truly involved in determining the subjective goal value or prediction errors during intertemporal decision‐making should be tested directly in future studies.

4.3. Potential neural targets for the diagnosis and treatment of addiction

In Experiment III, we found that the vmPFC and the dlPFC in participants with addiction did not contain information that reflected the specific choice, and that the decoding accuracy from both the vmPFC in patients with Internet‐gaming disorder and the dlPFC in smokers were significantly lower than those in healthy participants. Consistent with previous studies showing altered brain activity in the dlPFC in smokers and participants with Internet gaming disorder (Clewett et al., 2014; Wang et al., 2017), our results highlight the important role of dlPFC in addiction from the perspective of activity patterns. Our data also support the concept that, in terms of addictions, intact vmPFC function is important during value representation (Bedi, Lindquist, & Haney, 2015; Qi et al., 2016).

We found that there was a significant relationship between the discounting rate and decoding accuracy in the left dlPFC in patients with Internet‐gaming disorder as well as in smokers. Previous studies have shown that higher impulsivity in patients with addiction is correlated with altered functional connectivity of the dlPFC (Clewett et al., 2014; Wang et al., 2016). Our data confirmed the critical role of the dlPFC in the impulsive behaviour of individuals with addiction. The dlPFC is critical for the self‐control, therefore, extremely poor decoding accuracy in dlPFC may suggest loss of self‐control over the impulsive behaviors and may bring about steeper discounting rates. On the other hand, extremely good decoding accuracy in dlPFC may suggest that the dlPFC was recruited more to provide self‐control to compensate steeper discounting rates. Consistently, recent studies have shown that the structural connectivity and functional activation in the prefrontal cortex are nonlinear related to the behavioral indices in addictive and other domains (Filbey et al., 2014; Grabell et al., 2018) and that dopamine pathway may be responsible for the nonlinear relationship (Selvaggi et al., 2018). The nonlinear relationship suggests that the complexity of addictive behaviors' effect on the impulsivity, particularly on addictive behaviors' interaction with the dlPFC. Our data provide a key insight into the preference of patients with impulse control disorders for immediate reward, even at the risk of long‐term health or other problems when faced with decisions with both long‐term and short‐term effects.

This, as well as the finding that the decoding accuracy in the dlPFC was not correlated with the impulsivity of healthy participants, suggests that the neural correlates for steep discounting in the participants with addiction might be qualitatively different from those for more impulsive healthy individuals. The altered neural patterns in patients with addiction might be a reason for their maladaptive behavior. Therefore, our results suggest that altered neural patterns in the dlPFC may be a potential biomarker that could be used to improve the diagnosis and treatment for such conditions, a possibility that should be formally tested in future studies.

5. LIMITATION AND CONCLUSIONS

Several shortcomings of the present study should be acknowledged. First, there are other potential cognitive mechanisms, such as salience, that might have confounded the signals of value evaluation and self‐control in the present study. However, not all the ROIs can encode behavioral choices. Furthermore, many previous studies have implicated the dorsal anterior cingulate cortex and anterior insula cortex, and not the vmPFC or dlPFC, in salience processing (Seeley et al., 2007). Therefore, our findings should not be confounded by salience processing. Second, only male participants were recruited in the present study; we did not focus on the gender effect on the decoding results. Previous studies have shown that intense emotions are common symptoms in females during menstruation (Al Omari et al., 2016; Bata, 2012) and emotions affect intertemporal choices (Lempert & Phelps, 2016). Therefore, we did not recruit female participants for the present study. Whether the findings can be generalized to female participants remains to be demonstrated in future investigations. Third, at participant level, choosing the immediate option may be associated with increased difficulty for less impulsive participants, and greater difficulty for more impulsive participants; therefore, the collinearity between difficulty and choice might affect the power for participants who did not fall near the middle. However, at group level, we showed an appropriate positioning of questions relative to discounting (see Supporting information Figure S1). Fourth, as the immediate option was fixed, choices might not be independent of the subjective value of the delayed option. However, we found that striatal activity was associated with the subjective value, but did not contain information that encoded behavioral choices, and we found that dlPFC activity was not associated with the subjective value, but contained information that encoded behavioral choices. Therefore, the decoding results might not be explained by the value of the delayed option.

In conclusion, our results have provided neural evidence linking both value evaluation and self‐control processes to participant choices during intertemporal decision‐making. Our data suggest that spatial patterns of brain activity may determine the behavioral choices made during intertemporal decision‐making. The findings supporting altered neural patterns in patients with addiction provide a new insight into understanding why highly impulsive patients prefer temptations at the risk of long‐term physical and mental health, poor academic performance, or other problems, when they faced with drugs or Internet games. Our findings may provide potential neural targets for the diagnosis and treatment of impulsivity‐related brain disorders.

CONFLICT OF INTEREST

The authors declare no potential conflict of interests.

AUTHOR CONTRIBUTIONS

R. Z. and X. Z. designed the research. R. Z. and Z. W. performed the research. R. Z., J. B., X. Z., Z. W., P. Z., J. R., J. L., Y. W., L. Y., and L. H. analyzed the data. R. Z. and X. Z. wrote the manuscript. J. B., Z. W., Y. W., L. Y., S. V., and L. H. polished the manuscript.

Supporting information

Appendix S1 Supporting information

Zha R, Bu J, Wei Z, et al. Transforming brain signals related to value evaluation and self‐control into behavioral choices. Hum Brain Mapp. 2019;40:1049–1061. 10.1002/hbm.24379