Influencing food choices by training: Evidence for modulation of frontoparietal control signals

Abstract

To overcome unhealthy behaviors, one must be able to make better choices. Changing food preferences is an important strategy in addressing the obesity epidemic and its accompanying public health risks. However, little is known about how food preferences can be effectively affected and what neural systems support such changes. In this study we investigated a novel extensive training paradigm where participants chose from specific pairs of palatable junk food items and were rewarded for choosing the items with lower subjective value over higher value ones. In a later probe phase, when choices were made for real consumption, participants chose the lower-valued item more often in the trained pairs compared to untrained pairs. We replicated the behavioral results in an independent sample of participants while they were scanned with fMRI. We found that as training progressed there was decreased recruitment of regions that have been previously associated with cognitive control, specifically left dorsolateral prefrontal cortex (dlPFC) and bilateral parietal cortices. Furthermore, we found that connectivity of the left dlPFC was greater with primary motor regions by the end of training for choices of lower-valued items that required exertion of self-control, suggesting a formation of a stronger stimulus-response association. These findings demonstrate that it is possible to influence food choices through training, and that this training is associated with a decreasing need for top-down frontoparietal control. The results suggest that training paradigms may be promising as the basis for interventions to influence real world food preferences.

Introduction

Changing individual food preferences is a key step to solving a broad range of challenges in public health. This problem is most obvious in the current epidemic of obesity in the United States. In the period spanning 1999 to 2008, about one third of the American population was obese and another third was overweight (Flegal, Carroll, Ogden, & Curtin, 2010), placing these individuals at high risk for a broad range of chronic medical conditions, including cardiovascular diseases, diabetes, and cancer. The ability to reduce preferences for highly palatable processed foods is essential to solving these public health problems.

Recent studies explored the brain mechanisms of self-control in the domain of food items. Hare, Camerer, & Rangel (2009) found that dieters exhibited greater activation of several regions, among them the left dorsolateral prefrontal cortex (dlPFC) when they were asked to focus on the health rather than the taste aspect of food items. The authors hypothesized that successful self-control might relate to the extent to which the dlPFC can modulate the activity of the ventromedial PFC, an area implicated in valuation of stimuli (e.g. Chib, Rangel, Shimojo, & O’Doherty, 2009; Rangel & Hare, 2010; Rushworth, Noonan, Boorman, Walton, & Behrens, 2011). In another study with healthy participants the same group (Hare, Malmaud, & Rangel, 2011) found that activity in the left dlPFC correlated with the health aspects of food items rather than their taste. These studies measured the effects of directing attention to different features of food items but did not use conditioning to induce preference changes. Tricomi, Balleine, & O’Doherty (2009) performed an extensive training procedure in humans and showed that by repeatedly choosing a certain food item in sessions spanning 3 days, participants were no longer sensitive to the value of that option after selective satiation compared to a non-satiated one. Following findings in animals (Yin, Knowlton, & Balleine, 2004), the authors focused their analysis on the dorsolateral striatum and showed an increase in its activity as training progressed and responses became more habitual. A recent study (Wunderlich, Dayan, & Dolan, 2012) corroborated these results by using an extensive training two-armed-bandit task that also showed a similar pattern of activity in the dorsolateral striatum using abstract (non-food) stimuli. However, no study attempted to influence the preference of healthy participants when choosing between two food items that initially have different values.

In the current study we assessed participants’ individual preferences of palatable junk food items (Plassmann, O’Doherty, & Rangel, 2007) and developed an extensive training paradigm to enhance choice behavior of less-preferred items over more favorable ones. We first show, behaviorally, that after extensive training, subjects are more likely to choose items that they formerly placed less value on compared to untrained items. In an independent sample we replicate this behavioral finding and examine the underlying neural substrates of extensive training. Based on the above-mentioned studies we hypothesized a two-sided process will occur during training reflecting a shift from goal-directed to more habit-like responding. On the one hand, we will observe increased activity of dorsolateral striatum with training, reflecting the increased involvement of sensorimotor striatum in habitual responding. On the other hand, there will be a decrease in activity with repeated choices of the less preferred option in the control network including the dlPFC and other regions (Dosenbach et al., 2007; 2006). We also hypothesized we will observe changes in the connectivity with dlPFC as has been reported by Hare et al. (2009, 2011), reflecting decreasing need for top-down control with practice and stronger reliance on stimulus-response associations.

Materials and Methods

Participants

A total of fifty healthy participants took part in 2 separate studies. Twenty-nine participants completed the behavioral experiment out of which data from 28 (22 female; mean age, 20.3 ± 1.5; range, 18–24. Mean Body Mass Index (BMI) = 21.6 ± 3.22) are included in the analysis reported below (one participant was excluded due to auction exclusion criteria - see below under behavioral analysis). Twenty-one right-handed participants completed the imaging version. Data from 17 participants (8 female; mean age, 22.4 ± 3.6; range, 18–30. Mean BMI = 25 ± 4.1) are reported in the imaging analyses (one participant was excluded due to auction exclusion criteria, 3 others due to task analysis exclusion criteria - see below under imaging analysis). All subjects had normal or corrected-to-normal vision, no history of psychiatric diagnoses, neurologic or metabolic illnesses, no history of eating disorders, had no food restrictions, and were not taking any medications that would interfere with the experiment. Additionally, participants who were scanned were free of any metal implants or any other contraindications for MRI. Participants were told that the goal of the experiment was to study food preferences and were asked to refrain from eating 4 hours prior to arrival to the laboratory (Plassmann et al., 2007). All participants gave informed consent and the internal review board (IRB) at the University of Texas at Austin approved the study.

Task

For the general procedure of the task see Figure 1. Participants first underwent an auction (Figure 1A), then a training task (Figure 1B), then a probe (Figure 1C) and a repeat of the auction (Figure 1D).

Figure 1.

Task procedure showing the different stages on the left panel and the different task stages in the right panel: A) Auction; B) Training (timings refer to imaging version); C) Probe (timings refer to imaging version); D) Auction repeat

Auction

First, participants took part in an auction (Becker, DeGroot, & Marschak, 1964) (Figure 1A) in which photographs of 60 appetitive junk food items (Plassmann et al., 2007) were presented. Participants were endowed with three dollars and told that they could have an opportunity to use them to buy a snack at the end of the session. During the auction, participants were presented with one item at a time on a computer screen. They placed their bid by moving the mouse cursor along an analog scale that spanned from 0 to 3 at the bottom of the screen. The auction was self-paced and the next item was presented only after the participants placed their bid. This procedure has been shown to reliably obtain a measure of willingness to pay per item (WTP) (for a full description see (Plassmann et al., 2007)). Two participants (one from each study) were excluded because they bid less than $0.25 on more than 45 items; this was done to ensure a sufficient number of highly-valued items for the pairing procedure (see below).

Training

Behavioral version

The items were divided into 30 lower value and 30 higher value items according to a median split of each individual participants’ bids (Figure 2A). Each item within the higher value and lower value splits was then ranked (H1-H30 and L1-L30) and pairs were created to ensure the largest possible gap in WTP by pairing H1 with L1, H2 with L2, etc. (Figure 2B). These 30 pairs were then divided into 3 sets of 10 pairs by selecting every third pair starting from the first, second or third pair. One of these pair sets was chosen for the training task as Train Low pairs, another was used for the probe as Untrained pairs and the last was only used for the 2nd auction. Pair set assignments were randomized across participants.

Figure 2.

Diagram of the sorting and pairing procedure

During training, participants were shown 2 items and told to choose one item on each trial and that some of the choices would earn them points that would later be converted to money (each point was worth 1 cent). Unbeknownst to the participants, the only rewarded choices were of the low value item in each pair. Feedback was deterministic, such that choosing this item was rewarded 100% of the time and the alternative choice was never rewarded.

Each trial lasted 5 seconds. At the start of each trial (Figure 1B) one of the 10 pairs was presented, one item to the right and the other to the left of a fixation cross (locations were randomized across trials). The participants had 2.5 seconds to select one of the items using the keyboard. If the participants made a selection within this time window, their choice was confirmed by highlighting the selected item for one second and then the outcome was displayed: either ”+10” or ”- - -” for one second. During the inter-trial interval a fixation cross was presented in the center of the screen for a variable amount of time until the end of the five seconds. 125 trials were presented per run. Four runs of training were completed for a total of 500 trials (50 presentations of each of the 10 pairs).

Imaging version

The pairing method for the imaging study was slightly different. Instead of using all 30 pairs, only 15 pairs from the middle portion (8–22) were used. Three sets of 5 pairs were created by selecting every third pair starting from 8, 9 and 10, respectively. The 3 sets of 5 pairs were Train Low, Train Both, which were used in the training phase and untrained, which were used during the probe phase only. Pair set assignments were randomized across participants. The additional pair type, Train Both pairs, like the Train Low pairs, contained one low value and one high value item but choice of either of these items yielded points during training. We included this pair type to serve as a high-level control in the imaging analysis. The participants were not informed of the fact that there were two pair types during training, but were told that some of their choices will earn them points (later converted to real money). In the imaging version choices were made using an MRI compatible button box. Participants had 1.5 seconds to make their choice once the stimuli were presented (one to the right and one to the left of a central fixation cross, locations randomized across trials). Upon successful choice, the chosen item was highlighted for 500 milliseconds, then the outcome (”+10” or ”- - -”) was displayed for 500 milliseconds. During the inter-trial interval, a fixation cross was presented for a jittered time drawn randomly from an exponential distribution with a mean of 3, truncating values at 1 and 12. Fifty trials (25 Train Low and 25 Train Both) were randomly presented per run for a run time of four minutes and forty-five seconds where each pair was presented five times per run. Ten runs of training were completed such that each pair was presented 50 times.

Probe

Behavioral version

Following the completion of training, participants filled in a computer-adapted version of the Barratt Impulsiveness Scale (BIS)-11 questionnaire (Patton, Stanford, & Barratt, 1995). They were then told that they would next perform a new task (Figure 1C) where they choose an item in each pair but in this case instead of earning points, a single trial would be drawn at random at the end of the session and their choice on that trial would be honored (i.e., they would receive the item that they had chosen on that trial at the end of the experiment and will stay to consume it in the lab). The pairs from the training task were presented in a random order alongside 10 new Untrained pairs (not presented during training). These pairs also contained high and low value items and were drawn from the same pair matching procedure mentioned above. The task and timing at probe were very similar to that at training; the only difference is that the outcome (points/no-points) was not displayed following the choice. Trial timing was identical to training omitting the outcome presentation time. Each pair was presented five times during probe and the left-right locations of the items on the screen were randomized across presentations.

Imaging version

In the imaging version, participants filled in the computer-adapted version of the BIS-11 (Patton et al., 1995) using the MRI-compatible button box prior to the probe phase. At probe, 3 pair types were presented: the 5 Train Low and 5 Train Both pairs from training as well as 5 Untrained pairs. Trial timings were identical to training omitting the outcome (points/no-points) presentation time. Each pair was presented five times during probe and the right-left locations of the items on the screen were randomly assigned across presentations.

Questionnaires

As mentioned above, the BIS-11 (Patton et al., 1995) questionnaire was administered between training and probe. At the end of the session, when participants remained in the lab to consume the food item they received, they were also asked to fill in the BIS/BAS (Carver & White, 1994), two questionnaires that assessed the strength of a self-reported personal habit (Ji & Wood, 2007; Verplanken & Orbell, 2003) and were also asked to describe any strategies they used to maximize the number of points during training. The imaging participants also filled out the Kirby, Petry, & Bickel (1999) temporal discounting questionnaire.

Behavioral analysis

Behavioral version

Training

We performed a repeated measures logistic regression to test the difference in the odds of choosing the low value to high value item during valid trials from run 1 compared to the following nine runs. To allow comparison across the behavioral and imaging version we divided the entire training session of 500 trials into 10 parts with 50 trials in each part (the 500 trials were presented to participants with 3 short breaks).

Probe

To test if our training was successful in influencing choices, we performed a repeated measures logistic regression to compare the odds of choosing the low value to high value items between the 2 pair types (Train Low and Untrained) during probe. We ran a repeated measures linear regression to look at differences in reaction time (RT) for choices of the low value item between pair types. We also tested for the consistency of choices of the low value items in the two pair types using repeated measures logistic regression: Trained Low and Untrained across the 5 presentations during probe.

Auctions

We calculated the change in WTP of the high and low value items separately between the first and second auction (Δ). We compared that change between the 3 pair types: Train Low (presented during training), Untrained (presented only during probe) and another set that was never presented during either training or probe, using repeated measures linear regression.

Imaging version

Training

Similar to the behavioral version we compared the odds of choosing the low value to high value item in each of the pair types for run one compared to the following nine runs to test for learning effects. We also performed a repeated measures logistic regression to compare the odds of choosing the low value to high value item in the Train Low pairs compared to odds of choosing low value to high value items in the Train Both pairs. We used repeated measures linear regression to compare RT’s during choices of the low value items between pair types across runs.

Probe

We performed a repeated measures logistic regression to compare the odds of choosing the low value to high value item between the 3 pair types (Train Low, Train Both and Untrained) during probe. We also ran repeated measures linear regression to compare RT’s during choices of low value items between the different pair types. Similar to the behavioral version we tested for the consistency of choices of the low value items in the three pair types: Train Low, Train Both and Untrained across the 5 presentations during probe.

We also examined the unique influence on choices during probe of two opposing factors: 1) the number of times the low value items were chosen during training, which represents the influence of extensive training on choice behavior and 2) the difference in WTP between the high and low value item in each pair, which represents the goal-values of the items. For this purpose we performed a repeated measures linear regression to test if the number of choices of the low value items during training predict participants’ choices at probe, while controlling for the difference in WTP between the high and low value items in each pair. We performed this for each pair type (Train Low and Train Both) separately and tested the interaction between pair types.

Auction

We calculated the change in WTP of the high and low value items separately between the first and second auction (Δ). We compared that change between the 3 pair types: Train Low (presented during training), Train both and Untrained (presented only during probe), using a repeated measures linear regression.

fMRI acquisition and analysis

Imaging data were acquired on a 3T Signa Excite MRI scanner (General Electric Medical Systems, Milwaukee, WI) with an eight channel head coil. Functional data were acquired using a T2*-weighted echo planar imaging sequence (repetition time [TR] = 2500 ms, echo time [TE] = 30 ms, flip angle [FA] = 70°, field of view [FOV] = 22 cm²). Thirty two oblique axial slices with a 3.5 mm inplane resolution were positioned 20° off the anterior commissure-posterior commissure line to reduce the frontal signal dropout (Deichmann, Gottfried, Hutton, & Turner, 2003) and spaced 3 mm with a 0.5 mm gap to achieve full brain coverage. Slices were acquired in an interleaved fashion and higher order shimming was used to reduce susceptibility artifacts. Each of the training runs consisted of 114 volumes and the probe run consisted of 158 volumes. In addition to functional data, a single 3D T1-weighted high-resolution full brain image acquired using a spoiled gradient recalled pulse sequence (TR = 5.9 ms, TE = 1.2 ms, FA = 11°, FOV = 25 cm²) was acquired for brain masking and image registration.

Raw imaging data in DICOM format were converted to NIFTI format and preprocessed through a standard preprocessing pipeline using the FSL package (Smith et al., 2004) version 5. Functional image time series were first aligned using the MCFLIRT tool to obtain six motion parameters that correspond to the x/y/z translation and rotation of the brain over time. Second, the skull was removed from the T2* images using the brain extraction tool (BET) and from the high-resolution T1 images using Freesurfer (Ségonne et al., 2004). Spatial smoothing was performed using a Gaussian kernel with a FWHM of 5 mm. Data and design matrix were high-pass filtered using a Gaussian-weighted least-squares straight line fit with a cutoff period of 100 seconds. Grand-mean intensity normalization of each run’s entire 4D dataset by a single multiplicative factor was also performed. The functional volumes for each participant and run were registered to the high-resolution T1-weighted structural volume using a boundary-based registration method (Greve & Fischl, 2009) implemented in FSL5 (BBR). The T1-weighted image was then registered to the MNI152 2mm template using a linear registration implemented in FLIRT (12 DOF). These two registration steps were concatenated to obtain a functional-to-standard space registration matrix.

Imaging analysis

Training

The general linear model (GLM) during the training phase included 5 regressors for each pair type: 1) onsets of Train Low trials when low value items were chosen, modeled with a fixed duration of 1 second; 2) onsets of Train Low trials when the low value items were chosen but with actual RT’s as duration. We included this regressor to account for specific variability due to RT differences across trials. To improve the interpretation of the first regressor, the RT regressor was orthogonalized with respect to the first regressor so inferences for the first regressor reflect the average BOLD activation during the Train Low trials; 3) onsets of Train Low trials when the low value items were chosen with a fixed duration of 1 second but parametrically modulated by the demeaned number of times the low value item in the pair was chosen during probe. This regressor was added to test whether specific choices during probe could be directly linked to brain changes during training. 4) onsets of Train Low trials when the high value items were chosen with a fixed duration of 1 second; 5) onsets of Train Low trials when the high value items were chosen but with actual RT’s as duration orthogonalized with respect to the previous regressor. The same 5 regressors were modeled for Train Both trials. A missed trials regressor was also included. We included the 6 motion regressors described above, framewise displacement (FD) and RMS intensity difference from one volume to the next (DVARS) (Power, Barnes, Snyder, Schlaggar, & Petersen, 2012) as confound regressors. We also modeled out trials with FD and DVARS that exceeded a threshold of 0.5 by adding a single time point regressor for each ”to-be-scrubbed” volume. All regressors were entered at the first level of analysis and all (but the added confound regressors) were convolved with a canonical double-gamma hemodynamic response function. The temporal derivative of each regressor was included in the model. The model was estimated separately for each participant for each run.

Our analysis was aimed at identifying brain regions that showed either increases or decreases with training. Contrasts for the mean BOLD activation for each of Train Low and Train Both choices of low value item trials vs. baseline were estimated for each of the 10 runs separately. The proportion of times that the low value items were chosen within the Train Low and Train Both trials during training was computed for each run within-subject. This proportion tracks individual learning across runs. In a second level, within-subject analysis, the linear relationship between the BOLD contrast and corresponding proportion of low value choices was computed voxelwise for Train Low and Train Both, respectively. Note that an intercept, or column of 1s, was also included in this second level model to account for the overall mean of the data within each voxel. This second level contrast then reflects the within-subject relationship between the BOLD contrast and learning for Train Low and Train Both. At the group level we averaged these values across subjects in two separate one-sample t-tests to obtain the overall learning effect within Train Low and Train Both, respectively. Additionally we used a paired t-test to directly compare the Train Low to Train Both effect. The choices of the low value items were rewarded for both pair types. However, the participants were not required to choose the low value items to obtain points in the Train Both pairs (since choices of either high or low value items were reinforced). Thus, the paired t-test isolates the process of choosing a low value item that required exertion of self-control (in Train Low pairs) while controlling for response to reward as well as motor and visual processes involved in the choice itself (in Train Both pairs).

Three participants were excluded from the imaging analysis: Two did not choose the low value item in Train Both pairs even once for two of the training runs. The third participant chose the low value items in Train Both runs at exactly the same proportion across all training runs and thus the second level design was rank deficient and not estimable since the regressor for the proportion of low choices was perfectly correlated with the intercept regressor (column of 1s).

We also studied how the BOLD activation related with the proportion of times a low value item was chosen during probe using a parametrically modulated regressor at the first level for Train Low and Train Both trials. For Train Low, this is the third regressor described above. This relationship between the BOLD and later choice during probe was compared between the 10th and first runs and was tested using paired t-tests for Train Low and Train Both, separately. This contrast shows the relationship between training of specific pairs and choices of the same pairs during probe.

PPI

To create the seed for the PPI analysis we defined a 5mm sphere around the dlPFC activation found in the training analysis (see below; MNI coordinate [−52 28 28]) and masked it by the group result. PPI regressors were created by deconvolving the seed to obtain an estimated neural signal using the deconvolution algorithm of SPM (Gitelman, Penny, Ashburner, & Friston, 2003), calculating the interaction with the task in the neural domain and then reconvolving to create the final regressor. Following the gPPI modeling procedure of McLaren, Ries, Xu, & Johnson (2012), three regressors were added to the first level design matrix described above: 1) the raw time course extracted from the seed (after registering the sphere to native space of each run of each participant); 2) A PPI regressor based on onsets of choices of low value items in Train Low pairs; 3) a similar PPI regressor to the previous regressor but for Train Both pairs. We studied the PPI between choices of low value items in Train Low and Train Both pairs within runs 1 and 10 (separately for each run) and between these runs.

Probe

We used a GLM for the probe phase which included 4 regressors for each of the three pair types: 1) onsets of Train Low trials when low value items were chosen with fixed duration of 1 second; 2) onsets of Train Low trials when low value items were chosen but with actual RT’s as duration. This regressor was orthogonalized with respect to the previous regressor; 3) onsets of Train Low trials when the high value items were chosen with fixed duration of 1 second; 4) onsets of Train Low trials when the high value items were chosen but with actual RT’s as duration, orthogonalized with respect to the previous regressor. To test whether extensive training managed to shift choices from reliance on goal-directed neural mechanisms towards more habitual ones during probe, we included 2 additional regressors to the imaging analysis design matrix: 5) onsets of Train Low trials when low value items were chosen with fixed duration of 1 second and modulation by demeaned proportion of choices of low value items during training; 6) onsets of Train Low trials when low value items were chosen with fixed duration of 1 second and modulation by the difference in WTP between the high and low items in the pair. This was added to test if the difference in WTP had an effect on choices during probe. The last 2 regressors were also added for choices of high value items. The same 8 regressors were modeled for Train Both and Untrained pair type trials (besides the last 4 regressors since the Untrained items were not presented during training). A missed trials regressor was also included. We included confound regressors similar to the ones in the training GLM.

Our analysis was aimed at identifying brain regions showing greater activation during choices of low value over high value items for the Train Low pairs. We also performed comparisons between the Train Low and Train Both pair types for trials where the low value items were chosen. Effects of brain activity greater than baseline were also computed for each of the pair types separately for trials when the low value items were chosen.

All statistical maps for all analyses reported below were corrected at the whole-brain level using a cluster-based Gaussian Random Field correction for multiple comparisons, with an uncorrected cluster-forming threshold of z = 2.3 and corrected extent threshold of p < 0.05.

Results

Behavioral Results

Training

Figure 3A and 3B show the training results for the behavioral and imaging experiments. After 15 (out of 50) repetitions of each pair, the participants learned and continued to choose the low value items for over 80% of the trials for both samples (runs 2 through 10 significantly greater than run 1 p’s < 0.01 for the behavioral study and p’s < 0.05 except for run 4 p = 0.058 for the imaging study). Participants did not choose the low value items significantly more during the subsequent nine runs for the Train Both pairs in the imaging experiment (p’s > 0.29 for run 1 compared to runs 2 through 10). In the imaging version participants chose the low value items for the Train Low pairs significantly more than for the Train Both pairs across the entire training task (p < 0.001).

Figure 3.

A) Choice of low value items during training for behavioral participants, B) Choice of low value item during training for imaging participants for Train Low and Train Both pair types separately, C) Choices of the low value item during probe for behavioral participants for Train Low and Untrained pairs, D) Choice of the low value item during probe for imaging participants for Train Low, Train Both and Untrained pairs E) Mean WTP pre- and post- training for behavioral participants for Train Low and Untrained pairs, separated by high and low value items F) Mean WTP pre- and post- training for imaging participants for Train Both, Train Low and Untrained pairs, separated by high and low value items. Error bars reflect standard error of the mean.

Eighty percent of the participants chose the high value item on the first trial. Only by the 10^th trial did they reach 50% choice of low value items. Figure 3 presents that data binned by run, which shows that by the end of run 1 they chose the Low Value at 50%, when actually prior to learning that choices of the low value items are reinforced, the participants had a very strong preference to choose the higher value items in the pairs.

There were no significant RT differences for choices of low value items between Train Low and Train Both pairs across all runs (p’s > 0.3).

Probe

The probe was performed on average 3 minutes after the end of training. During probe, participants made choices for later consumption of actual food items to test the effects of training on a preference change. Points/money were not assigned for choices during probe. Figure 3D and 3C show the results during probe for both samples: participants chose the low value item in the Train Low pairs significantly more often than the low value item in the Untrained pairs: in the behavioral study they chose the low value item on 19.7% of Train Low pair trials versus 12.3% of Untrained trials (Figure 3 C, p < 0.001). Participants in the imaging study similarly chose the low value item on 20.5% of Train Low trials versus 12.6% of Untrained pair trials (p < 0.001). In the imaging study, participants chose the low value items in Train Both pair trials 18.7% of the time (p < 0.001 compared to choice of low value items in Untrained pair trials; n.s. compared to choices of low value items in Train Low pairs).

In the analysis of persistence of choices of the low value items across the five presentations at probe we found that in the behavioral study there was a main effect of pair type (Train Low vs. Untrained p = 0.0023), no main effect of presentation number (p = 0.85) and no interaction between presentation number and pair type (p = 0.82), suggesting a consistent effect across the five presentations. In the imaging study we found a main effect of pair type (Train Low vs. Untrained p = 0.01, Train Both vs. Untrained p = 0.029 but no effect of Train Low vs. Train both p = 0.72). There was a trending effect of presentation number (p = 0.087), but no pair type by presentation number interaction (p = 0.6). Thus the effect was still relatively consistent across the presentations across pair type.

There were no RT differences between choices of low value items in the Train Low and Untrained pair trials in the behavioral study (p = 0.15). Similarly, there were no differences in RT during low value choices between Train Both and Train Low pair types in the imaging study (p = 0.2), nor between Train Both and Untrained pairs (p = 0.19) and between Train Low and Untrained pairs (p = 0.08).

Auction

The raw WTP’s of all pair types in both auctions are presented in Figure 3E for the behavioral study and Figure 3F for the imaging study. As we ensured in our pairing procedure there were no significant differences in WTP between pair types for either sample (p’s > 0.24). There were no significant differences in pre-versus post- training WTP in either study. In the behavioral study we did not find a significant difference in the change in WTP between the two auctions (before and after training) for the Train Low pairs compared to either Untrained or never-seen pairs (p’s > 0.4). In the imaging study there was also no significant difference in the change in WTP over time between pair types (Train Both vs. Untrained p = 0.26, Train Low vs. Train Both p = 0.6 and the one with the largest trend was Train Low vs. Untrained p = 0.12). We are not aware of other studies that attempted to show an effect of training on WTP of items. Careful observation of Figures 3E and 3F show a regression to the mean of the WTP of the items such that the Higher Value items were rated as less valuable and the Low Value items as more valuable in the 2^nd auction compared to the first one.

Furthermore we found that the pairs on which the participants chose the low value items had a lower WTP difference (averages $0.83 and $0.86 for the behavioral and imaging studies respectively) between the high and low value items compared to the pairs on which they chose the higher value items (averages $1.10 and $1.25 for the behavioral and imaging studies respectively). There was a main effect of choice (p’s < 0.048) but there was no main effect of pair type (p’s > 0.15). This result suggests that the training paradigm managed to influence participants’ choice behavior during probe primarily on trials when the difference between high and low-valued items was not too large. It should be noted that there was still a highly significant difference in WTP between the low and high items in the pairs where participants chose the low value items at probe even according to the 2^nd auction (p’s < 0.0001).

In the regression to identify the relative contribution of the number of times an item was chosen during training on how many times it was subsequently chosen during probe and the difference in WTP between the items in each pair, we found that the number of choices of low value items per Train Low pair during training predicted subsequent choices of low value items during probe (p=0.001). However, the difference in WTP between items in the Train Low pairs did not (p=0.14). This relationship was not significant for choices of the low value items in Train Both pairs for either factor. There was no significant interaction between choices of the low value items during training and probe between pair types.

Questionnaires

We tested for the correlation between proportion of low value choices on Train Low pairs during probe (indicative of behavioral change) and BIS-11, BIS/BAS, habit strength and temporal discounting. No significant correlations in either sample were found between these measures (all p’s > 0.1 without control for multiple comparisons). In the self-report question pertaining to strategies used during training to maximize points, 18 of 28 participants in the behavioral version indicated they chose the item with the lower value. However, in the imaging version, only one participant mentioned this general rule whereas the rest said they had memorized which choices gave them points. Thus, it seems that participants in the behavioral version more easily formed a general rule. This was not the case for participants in the imaging version who formed only specific cue-reward pairings.

Imaging Results

Training

The primary analyses studied the linear relationship between BOLD activation during choices of low value items and the proportion of low value item choices in each run across the 10 training runs for Train Low and Train Both separately. For Train Low, we found that activity in bilateral dlPFC, parietal cortices and precentral gyrus had a negative relationship with learning (see Figure 4A and Table 1). A similar result was obtained for the Train Both pairs with low value choices except that there was no negative relationship between the activity in left dlPFC and learning above the correction threshold (see Figure 4B and Table 2).

Figure 4.

Imaging results showing the negative relationship with proportion of choices of low value items across training run for : A) Train Low pairs; B) Train Both pairs; C) the difference between these two pair types Train Both > Train Low shows a more restricted set of regions including bilateral parietal and only left dlPFC. Subtracting choices of low value items in Train Both pairs controls for all other trial elements which are do no require self-control since both low value and high value items were reinforced. Surface renderings were created using CARET after mapping of the group statistical maps to an average cortical surface using multifiducial mapping (Van Essen, 2005). All maps are presented at p < .05, corrected, as in the accompanying tables.

Table 1.

Results from analysis of training-related modulation of activity during choices of low value items on Train Low pairs (p < .05, corrected); regions presented here demonstrated negative relationship with the proportion of choices of the low value items on Train Low pairs across the 10 runs. For each cluster, the list shows all regions from the Harvard-Oxford atlas that contained more than 10 voxels within that cluster, along with the peak X/Y/Z location for the cluster in MNI space.

Train Low Chose Low
cluster	region	# voxels in region	Cluster size	X	Y	Z	Peak Z
1	L Superior Parietal Lobule	465	3039	12	−66	56	4.23
	R Superior Lateral Occipital Cortex	455
	LSuperior Lateral Occipital Cortex	437
	R Precuneus Cortex	373
	R Superior Parietal Lobule	353
	L Postcentral Gyrus	115
	R Postcentral Gyrus	102
	R Angular Gyrus	81
	R Posterior Supramarginal Gyrus	59
	L Posterior Supramarginal Gyrus	56
	L Anterior Supramarginal Gyrus	44
	L Precuneus Cortex	40
2	L Inferior Lateral Occipital Cortex	549	1068	−46	−74	−8	3.97
	L Occipital Fusiform Gyrus	210
	L Temporal Occipital Fusiform Cortex	92
	L Temporooccipital ITG	37
3	R Middle Frontal Gyrus	453	713	52	18	32	3.75
	R Precentral Gyrus	52
	R IFG, pars opercularis	27
	R IFG, pars triangularis	16
	R Frontal Pole	15
4	R Inferior Lateral Occipital Cortex	292	494	42	−68	−16	3.66
	R Temporooccipital ITG	98
	R Occipital Fusiform Gyrus	28
	R Temporal Occipital Fusiform Cortex	14
5	L Middle Frontal Gyrus	265	393	−44	28	28	3.87
	L Precentral Gyrus	37
	L IFG, pars opercularis	11
	L IFG, pars triangularis	10

Table 2.

Results from analysis of training-related modulation of activity during choices of low value items for Train Both pairs across the 10 training runs (p < .05, corrected); regions listed here demonstrated negative relationship with the proportion of choices of the low value items on Train Both pairs across the 10 runs. For each cluster, the list shows all regions from the Harvard-Oxford atlas that contained more than 10 active voxels within that cluster, along with the peak X/Y/Z location for the cluster in MNI space.

Train Both Chose Low
cluster	region	# voxels in region	Cluster size	X	Y	Z	Peak Z
1	R Precuneus Cortex	482	2491	8	−72	10	4.17
	R Intracalcarine Cortex	353
	L Occipital Pole	273
	R Superior Lateral Occipital Cortex	250
	L Intracalcarine Cortex	233
	R Supracalcarine Cortex	129
	R Lingual Gyrus	112
	R Cuneal Cortex	78
	R Occipital Pole	75
	R Superior Parietal Lobule	59
	L Superior Lateral Occipital Cortex	38
	L Precuneous Cortex	30
	L Supracalcarine Cortex	25
	L Lingual Gyrus	20
	L Cuneal Cortex	15
2	L Inferior Lateral Occipital Cortex	453	805	−48	−84	−2	3.86
	L Occipital Fusiform Gyrus	130
	L Temporal Occipital Fusiform Cortex	35
	L Occipital Pole	15
	L Lingual Gyrus	10
3	Right Thalamus	112	802	0	−4	14	3.62
	Left Thalamus	107
	Left Caudate	67
	Right Caudate	60
	Right Pallidum	22
	Right Putamen	11
4	R Inferior Lateral Occipital Cortex	397	740	42	−74	−20	3.76
	R Temporooccipital ITG	103
	R Occipital Pole	65
	R Occipital Fusiform Gyrus	44
	R Temporal Occipital Fusiform Cortex	12
5	L Superior Parietal Lobule	325	726	−30	−56	48	3.56
	L Superior Lateral Occipital Cortex	152
	L Postcentral Gyrus	85
	L Posterior Supramarginal Gyrus	36
	L Anterior Supramarginal Gyrus	22
6	R Middle Frontal Gyrus	472	712	44	22	30	3.89
	R IFG, pars triangularis	51
	R Precentral Gyrus	21
	R IFG, pars opercularis	14
7	R Superior Parietal Lobule	189	470	36	−52	46	3.49
	R Angular Gyrus	83
	R Posterior Supramarginal Gyrus	59
	R Postcentral Gyrus	42
	R Superior Lateral Occipital Cortex	38
	R Anterior Supramarginal Gyrus	16

We suggest that self-control was initially required to overcome the tendency to choose the unreinforced higher valued item in favor of the reinforced choice of the lower valued item. To test for the unique neural mechanisms underlying choices of low value items in the situation where only the lower valued choice was rewarded and not both, we directly compared the slopes between BOLD and proportion of low value item choices across the 10 runs for Train Low and Train Both using a group level paired t-test. We tested which brain regions had a more positive relationship with the proportion of choices of the low value items in the Train Low pairs across training compared to the Train Both pairs; this controlled for all other processes involved in choice and receipt of reward. We found that the linear relationship between BOLD activation and proportion of choice of low value items was more positive for Train Both than Train Low in bilateral parietal regions and the left dlPFC (see Figure 4C and Table 3). Previous studies showed differences in the processing of health vs. taste of food items in dieters with different levels of self-control (Hare et al., 2009; 2011). As we did not include healthy items in our study nor did we ask participants to consume an item up to satiety (Tricomi et al., 2009) we did not have dieting as an exclusion criterion in this study. After the study, we asked participants to report if they would describe themselves as being on a diet. Four participants reported being on some form of diet (BMI ranging from 22–27). Exclusion of these participants did not change the findings.

Table 3.

Results from a whole-brain group paired t-test comparison between choices of low value items for Train Low pairs and choices of low value items for Train Both pairs and their negative relationship with proportion of choices of low value items across the 10 training runs (p < .05, corrected). For each cluster, the list shows all regions from the Harvard-Oxford atlas that contained more than 10 active voxels within that cluster, along with the peak X/Y/Z location for the cluster in MNI space.

Train Both Chose Low > Train Low Chose Low
cluster	region	# voxels in region	Cluster size	X	Y	Z	Peak Z
1	L Superior Lateral Occipital Cortex	510	1692	−28	−68	34	3.68
	L Superior Parietal Lobule	419
	L Postcentral Gyrus	199
	L Anterior Supramarginal Gyrus	156
	L Posterior Supramarginal Gyrus	90
	L Precuneus Cortex	27
2	L Middle Frontal Gyrus	263	687	−46	2	42	3.17
	L Superior Frontal Gyrus	147
	L Precentral Gyrus	131
	L IFG, pars triangularis	24
3	R Superior Lateral Occipital Cortex	214	561	34	−72	44	3.22
	R Superior Parietal Lobule	178
	R Precuneous Cortex	88
	R Postcentral Gyrus	17

No increases in BOLD activation were found as training progressed for choices of the low value items in the Train Low pairs, Train Both pairs or their difference at a whole brain corrected level. In addition, no regions survived a small volume correction of either a 10 mm sphere around the right dorsolateral putamen coordinate reported by Tricomi et al. (2009) or using the right and/or left putamen masks from the Harvard-Oxford atlas (distributed with FSL). There were also no significant differences in training activation as a function of the number of low value choices at probe for either pair type.

PPI

For the choices of the low value items in Train Low greater than Train Both pairs during run 10, we observed a difference in connectivity with the left dlPFC seed region (defined by the training analysis above). This PPI effect was found in parietal and visual regions (see Figure 5B and Table 4). We did not observe this PPI effect during run 1. When we tested for the direct comparison between run 10 and run 1 we found greater connectivity with motor regions such as the supplementary motor area and bilateral precentral gyri (see Figure 5C and Table 5). Thus, it seems that following training, the dlPFC modulated activity in perceptual, attentional and motor regions to facilitate choices of low value items in the Train Low pairs compared to Train Both pairs. When we tested for the separate PPI effects of each condition vs. baseline seed connectivity we found only significant positive PPI effects that might suggest a stronger positive PPI effect of Train Low vs. Train Both with the regions reported above. Based on previous studies we defined a 10 mm sphere around the vmPFC coordinate reported by Hare et al. (2011) to test for a PPI effect with dlPFC. There were no significant PPI effects with this region in any of the analyses reported above.

Figure 5.

PPI results showing connectivity with dlPFC seed (shown in blue) for choice of low value items in Train Low pairs and Train Both pairs in the first (run 1) and last (run 10) runs of training (p < .05, corrected).

Table 4.

Results for PPI analysis showing regions with significant PPI with the left DLPFC seed at run 10 (p < .05, corrected). For each cluster, the list shows all regions from the Harvard-Oxford atlas that contained more than 10 active voxels within that cluster, along with the peak X/Y/Z location for the cluster in MNI space.

PPI run 10
cluster	region	# voxels in region	Cluster size	X	Y	Z	Peak Z
1	L Occipital Pole	194	777	−22	−92	10	3.75
	R Lingual Gyrus	137
	L Intracalcarine Cortex	119
	R Intracalcarine Cortex	89
	L Lingual Gyrus	46
	R Supracalcarine Cortex	33
	R Occipital Pole	23
	LSuperior Lateral Occipital Cortex	15
	L Inferior Lateral Occipital Cortex	10
2	L Postcentral Gyrus	170	409	−46	−42	44	3.54
	L Anterior Supramarginal Gyrus	124
	L Posterior Supramarginal Gyrus	50
	L Precentral Gyrus	43
	L Superior Parietal Lobule	12
3	R Inferior Lateral Occipital Cortex	22	135	50	−72	−20	3.31
	R Occipital Fusiform Gyrus	11
4	R Precuneous Cortex	57	129	16	−50	8	3.26
	R Posterior Cingulate Gyrus	15
	R Lingual Gyrus	10

Table 5.

Results for PPI analysis showing regions with significant difference in PPI between run 10 and run 1 with the left DLPFC seed (p < .05, corrected). For each cluster, the list shows all regions from the Harvard-Oxford atlas that contained more than 10 active voxels within that cluster, along with the peak X/Y/Z location for the cluster in MNI space.

PPI run10 > run1
cluster	region	# voxels in region	Cluster size	X	Y	Z	Peak Z
1	L Postcentral Gyrus	220	585	−52	−18	52	3.23
	L Middle Frontal Gyrus	145
	L Precentral Gyrus	122
	L Superior Frontal Gyrus	15
2	R Postcentral Gyrus	187	306	56	−14	56	3.42
	R Precentral Gyrus	54
3	R Supplementary Motor Cortex	74	219	−4	8	48	3.11
	R Paracingulate Gyrus	39
	L Supplementary Motor Cortex	29
	L Paracingulate Gyrus	28
	R Superior Frontal Gyrus	24
4	L Central Opercular Cortex	45	188	−50	−2	12	3.3
	L Precentral Gyrus	40
	L Anterior Superior Temporal Gyrus	40
	L Posterior Superior Temporal Gyrus	10
5	R Inferior Lateral Occipital Cortex	15	145	46	−66	−20	3.41
	R Occipital Fusiform Gyrus	11

Probe

When participants chose the low value items in either Train Low or Train Both pair types (compared to baseline), we observed an increase in activity in similar regions to those that decreased their activity across training runs (see Tables 6 and 7). Regions showing an increase include visual regions, bilateral parietal regions, anterior cingulate cortex (ACC) (in both pair types) and bilateral dlPFC for Train Both pairs only (see Figure 6). Interestingly there were no dlPFC activations while choosing the low value items in the Train Low pairs, but these regions were active during choices of low value items in Train Both pairs. This is consistent with a practice-related decrease in the engagement of top-down control systems over choice. However, no regions survived the direct comparison between choices of low value items in Train Low compared to Train Both pairs. Similarly, we did not find any activity above our correction threshold for choices of low value compared to high value items in Train Low pairs. These null findings are likely due to low power resulting from the small number of participants who had choices of the low value items in both pair t ypes (n = 12) or choices of both low and high value items in the Train Low pairs (n = 15). It is also possible that we did not find differences in the direct comparisons due to the short duration of this phase; Tricomi et al. (2009) did not report any results from the probe phase due to its short duration.

Table 6.

Regions showing significant activation for choices of low value items in Train Low pairs greater than baseline during Probe. For each cluster, the list shows all regions from the Harvard-Oxford atlas that contained more than 10 active voxels within that cluster, along with the peak X/Y/Z location for the cluster in MNI space.

Probe Train Low Chose Low > baseline
cluster	region	# voxels in region	Cluster size	X	Y	Z	Peak Z
1	L Occipital Pole	2116	22508	−32	−66	−16	5.78
	R Occipital Pole	2038
	R Superior Lateral Occipital Cortex	1331
	L Superior Lateral Occipital Cortex	1188
	L Inferior Lateral Occipital Cortex	1132
	R Occipital Fusiform Gyrus	871
	L Occipital Fusiform Gyrus	843
	R Inferior Lateral Occipital Cortex	828
	R Lingual Gyrus	741
	R Intracalcarine Cortex	700
	L Lingual Gyrus	534
	R Precuneous Cortex	516
	L Intracalcarine Cortex	489
	L Superior Parietal Lobule	471
	R Temporal Occipital Fusiform Cortex	355
	L Temporal Occipital Fusiform Cortex	309
	L Precuneous Cortex	307
	R Cuneal Cortex	261
	Brain-Stem	210
	L Temporooccipital ITG	166
	R Supracalcarine Cortex	125
	L Cuneal Cortex	104
	R Temporooccipital ITG	90
	Left Hippocampus	88
	R Superior Parietal Lobule	73
	L Supracalcarine Cortex	32
	Left Thalamus	29
	L Postcentral Gyrus	25
2	R Paracingulate Gyrus	508	1786	4	20	52	4.42
	L Paracingulate Gyrus	311
	R Superior Frontal Gyrus	279
	L Superior Frontal Gyrus	186
	R Anterior Cingulate Gyrus	160
	L Anterior Cingulate Gyrus	98
3	L Insular Cortex	284	782	−28	12	6	3.94
	L Central Opercular Cortex	75
	Left Putamen	70
	L Frontal Operculum Cortex	65
	L Planum Polare	19

Table 7.

Regions showing significant activation for choices of low value items in Train Both pairs greater than baseline during Probe. For each cluster, the list shows all regions from the Harvard-Oxford atlas that contained more than 10 active voxels within that cluster, along with the peak X/Y/Z location for the cluster in MNI space.

Probe Train Both Chose Low > baseline
cluster	region	# voxels in region	Cluster size	X	Y	Z	Peak Z
1	R Occipital Pole	2227	28611	30	−74	−8	5.58
	L Occipital Pole	2084
	R Superior Lateral Occipital Cortex	1658
	R Lingual Gyrus	1226
	L Superior Lateral Occipital Cortex	1166
	L Inferior Lateral Occipital Cortex	1125
	R Inferior Lateral Occipital Cortex	1107
	R Occipital Fusiform Gyrus	879
	L Occipital Fusiform Gyrus	790
	L Lingual Gyrus	757
	R Intracalcarine Cortex	673
	Left Thalamus	595
	R Precuneous Cortex	530
	L Insular Cortex	472
	L Intracalcarine Cortex	444
	R Temporal Occipital Fusiform Cortex	442
	R Cuneal Cortex	414
	L Superior Parietal Lobule	399
	L Temporal Occipital Fusiform Cortex	348
	R Superior Parietal Lobule	346
	L Cuneal Cortex	330
	Left Putamen	318
	L Precuneous Cortex	269
	R Temporooccipital ITG	192
	Brain-Stem	186
	Right Thalamus	181
	L Frontal Operculum Cortex	157
	L Middle Frontal Gyrus	141
	Left Pallidum	138
	R Supracalcarine Cortex	133
	L Temporooccipital ITG	127
	Left Hippocampus	114
	L Central Opercular Cortex	106
	L IFG, pars opercularis	85
	L Postcentral Gyrus	81
	Right Hippocampus	65
	L IFG, pars triangularis	37
	L Planum Polare	35
	L Temporal Pole	24
	L Posterior Temporal Fusiform Cortex	22
	L Supracalcarine Cortex	22
	L Precentral Gyrus	20
	L Posterior Inferior Temporal Gyrus	20
	L Posterior Cingulate Gyrus	15
	L Posterior Parahippocampal Gyrus	14
	R Posterior Cingulate Gyrus	13
	L Frontal Orbital Cortex	11
	Left Caudate	10
2	R Paracingulate Gyrus	382	1475	2	22	50	4.31
	L Paracingulate Gyrus	298
	R Superior Frontal Gyrus	246
	L Superior Frontal Gyrus	140
	R Anterior Cingulate Gyrus	77
	L Anterior Cingulate Gyrus	74
3	Right Putamen	110	625	18	10	2	3.52
	Right Caudate	76
	R Frontal Orbital Cortex	12
4	R Insular Cortex	218	544	44	12	2	3.5
	R Frontal Operculum Cortex	146
	R Central Opercular Cortex	59
5	L Precentral Gyrus	154	319	−44	−12	50	3.32
	L Middle Frontal Gyrus	97
	L Postcentral Gyrus	36
6	R Middle Frontal Gyrus	111	232	48	10	56	3.41
	R Precentral Gyrus	42
7	R Precentral Gyrus	50	103	46	10	32	3.19
	R Middle Frontal Gyrus	30
8	R Middle Frontal Gyrus	46	85	46	20	24	2.95
	R IFG, pars opercularis	12
9	R Posterior Cingulate Gyrus	44	80	6	−28	28	2.96
	L Posterior Cingulate Gyrus	31

Figure 6.

Imaging probe results showing regions exhibiting increased activity with choices of the low value items in the two pair types compared to baseline: A) Train Low ; B) Train Both (p < .05, corrected).

Choices of low value items during probe showed a modulation by choices during training for both pair types in visual, motor and right premotor regions (Figure 7A). Further, there was a negative correlation between choices of the low value item during training and activity in the vmPFC and OFC during choices of the low value item for Train Low pairs at probe. We did not find any neural evidence at probe for greater modulation of choices of low value items during training for Train Low greater than Train Both. However, for the contrast of choices of low value items during probe for Train Both greater than Train Low pairs, we found greater activity in vmPFC and OFC (Figure 7B and Table 8). This result is consistent with a shift from goal-directed to habitual responding (and decreased reliance on goal-values) during probe, but only for the Train Low pairs. This result was obtained with only n=12 during probe (that had choices of low value items in both pair types) so should be regarded with caution.

Figure 7.

Imaging probe results showing regions exhibiting: A) the conjunction of positive modulation by choices during training for both pair types; B) the contrast of modulation by choices of low value items during probe for Train Both greater than Train Low pairs (p < .05, corrected).

Table 8.

Regions showing significant activations for the contrast of modulation by choices of low value items during probe for Train Both greater than Train Low pairs. For each cluster, the list shows all regions from the Harvard-Oxford atlas that contained more than 10 active voxels within that cluster, along with the peak X/Y/Z location for the cluster in MNI space

Probe Train Both Chose Low > baseline
cluster	region	# voxels in region	Cluster size	X	Y	Z	Peak Z
1	L Frontal Pole	202	343	−24	58	−6	3.48
	L Frontal Medial Cortex	36
2	L Inferior Temporal Gyrus, posterior division	33	216	−44	−10	−34	3.33
	L Middle Temporal Gyrus, anterior division	27
	L Temporal Pole	22
	L Inferior Temporal Gyrus, anterior division	19
	L Temporal Fusiform Cortex, posterior division	19
3	R Inferior Temporal Gyrus, posterior division	82	155	44	−14	−36	3.57
	R Temporal Fusiform Cortex, posterior division	24
	R Inferior Temporal Gyrus, anterior division	12

Discussion

The ability to influence food choices is critical to solving health-related problems currently affecting large portions of the US and world population (World Health Organization, 2013). Here we report the results of a new behavioral paradigm, which enhanced the likelihood of choosing a less-preferred food for actual consumption over a previously more-favored food. In this task, pairs of appetitive junk food items were presented during a training period of 1 hour, such that each pair contained a lower value item versus a higher value one; in the critical condition, only choices of the lower value item were reinforced with money. In a subsequent probe phase, where participants made choices for later actual consumption, they chose the previously reinforced lower value items significantly more than similar value items in untrained pairs. We replicated the behavioral results in an independent sample of healthy participants, scanned with fMRI while performing the task. We found that as extensive training progressed, activity in regions in the brain that are part of the cognitive-control network (the dlPFC and bilateral parietal cortices) had a negative linear relationship associated with choosing the lower value item. Further, we found that this pattern of activity was specific to the left dlPFC and bilateral parietal cortex only for choices of the lower value items that required exertion of self-control (while controlling for all other choice-related processes including receipt of reward).

Recent studies reported effective dietary interventions using incentives (Driver & Hensrud, 2013; Volpp et al., 2008). Our study provides a clue of mechanistic insight into the potential effectiveness of such a program. Furthermore, it might suggest that repeating the procedure we performed here could prove helpful to obtain long term effects via reduction of engagement of self-control mechanisms.

These results align with and extend current findings in the neuroeconomics literature. Hare et al. found that a similar region of left dlPFC was more active in dieters with greater self-control (2009) and also in healthy participants (2011) when focusing on the health rather than on the taste aspects of food. Our results extend those findings to a choice situation, showing that in healthy participants this same region of left dlPFC (alongside parietal regions, also reported by those studies) decreases its activity with extended training of choosing a less preferred item. With repeated choices, the self-control network was less and less necessary to choose the lower value items over the higher value ones. Figner et al. (2010) used rTMS in a temporal discounting task to show that disrupting activity of the left but not right dlPFC led to choices of smaller-shorter options over larger-later ones. The authors concluded that this region serves a role in self-control in the domain of temporal discounting. Based on Dosenbach et al. (2007), we suggest that the regions we found here to decrease their activity with extensive training are part of the fronto-parietal network that is involved in active adaptive control, in particular adjusting the exertion of top-down control in response to feedback. It should be highlighted that although the choices for the low value items in the training phase were not made for consumption, choosing them still required participants to override their initial preference for higher value items in each pair, to achieve a different goal of monetary reward and thus required exertion of self-control. It is plausible that the decrease in these regions stems from facilitation resulting from extensive training. However, we believe that the fact that there were no RT differences between pair types and between the beginning and the end of training, suggests that the neural effect we observed goes beyond a simple facilitation effect.

During the probe phase we found that activity in a similar network of self-control regions increased when participants chose low value items in each of the pairs for later consumption (see Figure 6). There was great overlap, especially in parietal regions, with brain regions that decreased activity as training progressed. Thus, this network that once decreased activity with training, now activated during choice of low value items in the absence of outcome, suggesting that the values of these items were not changed enough and that exertion of self-control was still required to choose them. It is possible that prolonged training will ”detach” the involvement of these regions when choosing low value items during probe.

We identified significant modulation of connectivity of the left dlPFC ROI between pair types, consistent with previous studies (Hare et al., 2009; 2011). During the last run there was greater connectivity for choices of low value items in the Train Low over Train Both pairs with parietal and visual regions suggesting a potential top-down process (Corbetta & Shulman, 2002; 2011). Furthermore, in the comparison between run 10 and run 1 there was greater connectivity for choices of low value items in Train Low pairs over the same choices in Train Both pairs with primary motor regions and supplementary motor area. This might be related to spillover of urges into the motor cortex (Gupta & Aron, 2011) and/or the action competition in motor cortex (Klein-Flügge & Bestmann, 2012). These results, together with the probe results might hint at ongoing changes during training that could have led to a more substantial preference change had we used a longer training session.

We had hypothesized that we will observe a shift from goal-directed to more “habitual-like” responding following extensive training. However, we did not identify any regions that increased their activity with the progression of extensive training, particularly the striatal regions predicted on the basis of the animal literature (Yin et al., 2004) and previous human fMRI studies (Tricomi et al., 2009; Wunderlich et al., 2012). There are several possible reasons why we did not replicate these previous imaging results. Most importantly, both of those studies involved longer training across several days. In addition, the Tricomi et al. (2009) study involved repeated pressing of a button to obtain a reward, rather than a choice between two options, which might have led to the putamen response due to its involvement in motor processes. In the Wunderlich et al. (2012) study participants repeated the choices across 3 days, and those choices were between 2 abstract options rather than food items. We claim that the participants in our study did not treat the items as abstract stimuli. This is apparent from their post-task reports and the fact that on 80% of the initial trials they chose the higher valued items. Thus it is possible that it requires more training to form habitual responding for items that contain inherent values. Nevertheless, the connectivity results suggest that the extensive training shifted responding more toward a stimulus-response representation over a goal-directed one. Furthermore, choices of the low valued items during training predicted lower activity in vmPFC for Train Low and not Train Both pair trials during probe (while accounting for the difference in WTP between the items in each pair) lending credence to the idea that extensive training leads to a stronger goal-directed-to-habitual shift (but only to the low value items in Train Low pairs.)

We did not find a change in valuation of the items between the two auctions. We are not aware of any other study that reported a change in bids in such an auction following a behavioral manipulation. We did observe an interesting significant regression to the mean between the two auctions. We do not have the tools in this study to conclude whether this would occur naturally without the training procedure between the auctions. It is possible that this occluded our ability to find a significant valuation difference that would have followed the choice preference change induced by training.

The Low value items in the Train Both pairs were chosen during probe slightly less frequently (but not significantly) than the low value items in the Train Low pairs. It is reasonable to assume that even the partial reinforcement of these items led to greater choice during probe compared to Untrained pairs. The self-report post-task questionnaires of the imaging version suggests that the inclusion of the Train Both pairs made it harder for the participants to form a general rule of the task and thus led to increased variance in their choices of the low value items for the Train Both pairs. This in turn might have led to increased choices of the low value items during probe. We can speculate that in a longer training paradigm these pairs would have shown a smaller effect than the Train Low pairs compared to Untrained pairs. Furthermore, the fact that participants showed a consistent effect of choices at probe across the five repetitions but did not show a strong choice preference for the low value items overall in either pair type speaks against a demand characteristic explanation of the probe results.

Our study still leaves several open questions to be addressed in future studies. First, can this enhancement of choices be applied to the case of healthy over unhealthy food items and not only within junk food snacks? We believe it is plausible given that healthy items such as fruit and vegetables usually obtain positive values, although lower than non-healthy snacks. Second, the training and probe were done on specific pairs. Therefore, one might ask if the change of value will be generalized beyond the specific pairs? The finding that the effect at probe was found on pairs with smaller (though still highly significant) WTP difference leads us to believe that our task could have been much more successful if aimed to influence preference of items with closer WTP with prolonged and/or repeated training. Furthermore, even changing choices in fixed pairs can be ecologically valid to enhance a specific choice one faces on an everyday basis: e.g. choosing carrots over chips as an evening snack. Lastly, an interesting question is how long-lasting the effect will be and how maintenance can be modulated by the nature and length of training. The finding that choices persisted during the five presentations of pairs at probe shows that at least during this short period of time the choices were consistent. Only a study involving a larger delay will show if this was consolidated into longer-term memory. One additional potential caveat for the face value of our procedure is the limited choice window of 1.5 seconds during probe, which does not apply to real world choices. That is the case for many laboratory studies but we can say that participants missed less than 1% of trials overall in the probe phase in both studies (with an average RT of less than 1 second), which suggests that they had enough time to make this decision. Tasks that include an ad libitum consumption phase at the end of an experiment allow testing the influence of laboratory tasks on real-world food consumption. However, this does not allow for testing how preferences changed on more than 2 items.

The significance of this study is twofold: first we show that an extensive training session lasting only 1 hour can shift participants’ preferences for later food consumption. Compared to untrained pairs, we managed to enhance participants’ choices of less-valued items by almost 10% via only one hour of training. As far as we know our study is the first to show an ability to influence choice preferences for food items in humans. Second, we show that preference change is associated with a decrease in activity of self-control regions previously implicated in focusing on long-term goals in decision making in the context of food health over taste (Hare et al., 2009; 2011) and/or inter-temporal discounting (Figner et al., 2010; McClure, Laibson, Loewenstein, & Cohen, 2004). This suggests that reinforced practice at making better choices may be a potential mechanism to engrain these choices and thus lead to better dietary choices in real-world settings.