Foraging under Competition: The Neural Basis of Input-Matching in Humans

Abstract

Input-matching is a key mechanism by which animals optimally distribute themselves across habitats to maximize net gains based on the changing input values of food supply rate and competition. To examine the neural systems that underlie this rule in humans, we created a continuous-input foraging task where subjects had to decide to stay or switch between two habitats presented on the left and right of the screen. The subject's decision to stay or switch was based on changing input values of reward-token supply rate and competition density. High density of competition or low-reward token rate was associated with decreased chance of winning. Therefore, subjects attempted to maximize their gains by switching to habitats that possessed low competition density and higher token rate. When it was increasingly disadvantageous to be in a habitat, we observed increased activity in brain regions that underlie preparatory motor actions, including the dorsal anterior cingulate cortex and the supplementary motor area, as well as the insula, which we speculate may be involved in the conscious urge to switch habitats. Conversely, being in an advantageous habitat is associated with activity in the reward systems, namely the striatum and medial prefrontal cortex. Moreover, amygdala and dorsal putamen activity steered interindividual preferences in competition avoidance and pursuing reward. Our results suggest that input-matching decisions are made as a net function of activity in a distributed set of neural systems. Furthermore, we speculate that switching behaviors are related to individual differences in competition avoidance and reward drive.

Introduction

A key inference of Ideal Free-Distribution Theory is that foragers geographically distribute themselves in relation to the proportion of food available and to the density of competition (Fretwell and Lucas, 1969). This distribution is expressed in the Simplest Input-Matching Rule where, if Habitat A contains more food and less competition than Habitat B, Habitat A will be the preferred location. Although some theorists argue whether the input-matching rule is an evolutionarily stable strategy (Kennedy and Gray, 1993), pioneering work from behavioral ecology has shown it to be pervasive across a variety of species from invertebrates to humans (Fretwell and Lucas, 1969; Fretwell, 1972). In explaining some of these inconsistent results, some argue that many input-matching models fail to consider abiotic variables and are indifferent to the subjective weight placed on avoiding competition and seeking reward. Humans, in particular, may either shirk competition (Balafoutas and Sutter, 2012) or be biased toward reward signals when pursuing profitable habitats. To date, no study has examined the neural substrates of continuous input matching in relation to the density of competition (C) and reward rate (R), nor how idiosyncratic preferences based on changing input weights of C and R spur habitat switching.

Using functional magnetic resonance imaging (fMRI), we scanned volunteers as they played a continuous-input foraging task (Fig. 1A). The subject's goal was to acquire as many tokens as possible by switching between two habitats to minimize C and compete for tokens with the highest supply rate (R). In both habitats, tokens were presented in four shades—light, medium, dark gray, and black—each indicating the frequency of supply rate (i.e., 1–4 Hz; Fig. 1B). In addition, red dots also appeared in both habitats and were representative of online competitors against whom the subjects believed they were competing in real time. Up to six subjects competed for tokens (Fig. 1C) and were each instructed to press a response button as soon as a token appeared in their habitat. Subjects who responded fastest won the tokens. Tokens were exchanged for money at the end of the experiment.

Figure 1.

Schematic of the continuous-input foraging task and plots depicting R and C changes over time. A, B, Plots depict frequency in hertz of R (A), which were presented in 8 s blocks, and C (B) across both habitats over time. Schedules were chosen such that R and C were decorrelated across habitats. C, The task involved the subject trying to acquire as many tokens as possible by switching between habitats to increase the likelihood of acquiring the tokens and by pressing a button as fast as possible when a token appeared. Subjects, told that speed of response was a factor in determining which competitor won the token, were encouraged to press a response button as quickly as possible after a token appeared in their habitat. Competitors were unknown to each other and had approximately the same competitive abilities.

We parametrically examined brain activity associated with input matching both as a function of when competitors entered or left the volunteer's habitat (changes in C) and of varying reward supply rate (R). We had two main predictions. We first hypothesized that, as it becomes increasingly disadvantageous to be in the current habitat (i.e., linear increases in C and decreased R), there would be increased activity in brain regions associated with preparatory motor urges including primary motor areas, anterior cingulate cortex (ACC), and the insula. Conversely, as the current habitat becomes increasingly advantageous, there would be augmented activity in basal ganglia reward systems. Second, we wanted to examine individual differences in habitat-switching behavior. That is, we wanted to learn whether subjects were more inclined to switch based on increasing C or on decreasing R.

Materials and Methods

Subjects.

We scanned 18 volunteers. All were right-handed, fluent speakers of English, and screened for psychiatric or neurological problems. Three were excluded due to expressing disbelief that they were playing against other subjects online. Thus, the final sample was 15 (8 males; age, 25.0 ± 7.3 years). When subjects arrived at the scanner facility, they were first told that they would be playing a live online game with a number of other people via the Cambridge Brain Science website (www.cambridgebrainsciences.com). The study was approved by Cambridgeshire Psychology Research Ethics Committee. All volunteers provided written informed consent and were paid for participating.

Paradigm.

The continuous-input foraging task is depicted in Figure 1. Before being placed supine in the MRI, participants were briefed about the nature of the task. Participants were then presented with a screen depicting two habitats. The participant was represented by a yellow dot residing on one of the two habitats. The two habitats also contained one or more competitors (depicted by red dots). From 1 to 6 competitors were present in each habitat at any one time and the participant believed these red dots represented real online participants also playing the task in real time. The participant's task was to acquire tokens (via a button press) that appeared in their own habitat before those tokens were acquired by the competitors in the same habitat. Each token was worth 50 points and the aim was to amass as many points as possible over the duration of the task. The tokens were represented by monochrome dots. The tokens appeared in both habitats so the participant was able to see what token was in the habitat they are not occupying and also to see how many competitors were present in that other habitat. At any one time, there was never more than a single token appearing in each of the two habitats on the screen and participants could only harvest tokens in their own habitat (i.e., where the yellow dot resided). To acquire the token once it appeared in the participant's own habitat, the participant had to press the response button faster than any of the competitors in that habitat. Successful acquisition was depicted by a green halo appearing around the yellow dot, if the participant won out, or around one of the red dots, if a competitor won out. The token then disappeared from the screen until it was refreshed. Critically, the tokens varied in the rate at which they were refreshed. These refresh rates range from 1 to 4 Hz and the refresh rates were represented by different shades of monochrome token. A light-gray token refreshed at a supply rate of 4 Hz, medium gray at 3 Hz, dark gray at 2 Hz, and black at 1 Hz. Being in a habitat that contains a token with a higher refresh rate increased the chances of token acquisition as more tokens were available to acquire over time. The key challenge for the participant therefore was to maximize token acquisition by moving to the habitat with the tokens with the highest refresh rate and the fewest competitors. Therefore, the most optimal habitat to occupy was one that contained only 1 competitor and a 1 Hz black token. To meet this challenge, participants were able to freely switch between the two habitats (using left or right key presses) at any time during the task, although there was a short time cost with each switch.

The foraging conditions were presented in 2 min epochs each separated by a 30 s black screen with text showing how many points the participant had won. Each subject completed 20 epochs totaling 46 min. Every 8 s, the shade of the monochrome token (and consequently the refresh rate) in each habitat changed. Similarly, competitors (red dots) could enter or leave either habitat every 2 s. Consequently, both the frequency of reward (R, tokens) and the degree of competition (C, competitors) in each habitat was constantly changing, requiring participants to continually weigh whether the best strategy was to stick to their current habitat (H_S) or to switch to the other habitat (H_O). Each switch incurred a 1 s time cost. In reality, R and C were predetermined as was the participant's probability of winning a token during each acquisition opportunity (although they had to press the button to win). At the end of the experiment, token points were exchanged for money.

Increase the verisimilitude of the competitors.

To increase the verisimilitude of the competitors, we asked subjects to sign up on the website www.cambridgebrainsciences.com. Once the subjects signed up, we showed them some of the experiments and a paper recently published using data from an online brain-training game (Owen et al., 2010). We informed them that an e-mail had been sent to all those who had registered with the website and that a number of these other participants would come online and play against them while they were having an fMRI scan.

Questionnaire measures.

Following the MRI scan, participants answered a number of questions concerning the task (postscan questionnaire). The rating scale was structured as follows: 1, 2: very slightly or not at all; 3, 4: a little; 5, 6: moderately; 7, 8: quite a bit; 9, 10: extremely. Examples of the questions include: “How much attention did you pay to the tokens changing?” and “How much attention did you pay to the competitors entering your patch?” The rationale for these postscan questions was to see whether subjects used explicit strategies to aid in their decisions to switch habitats.

Image acquisition.

MRI scanning was conducted at the Medical Research Council Cognition and Brain Sciences Unit on a 3-tesla Trio Tim Magnetic Resonance Imaging scanner (Siemens) by using a head coil gradient set. Whole-brain data were acquired with echoplanar T2*-weighted imaging (EPI) sensitive to BOLD signal contrast (48 sagittal slices; thickness, 3 mm; TR, 2400 ms; TE, 30 ms; flip angle, 78°; FOV, 192 mm; voxel size, 3 × 3 × 3 mm). To provide for equilibration effects, the first five volumes were discarded. T1-weighted structural images were acquired at a resolution of 1 × 1 × 1 mm.

Image preprocessing.

SPM5 software (www.fil.ion.ucl.ac.uk/spm) was used for data analysis. The EPI images were sinc interpolated in time for correction of slice timing differences and realignment to the first scan by rigid body transformations to correct for head movements. For each participant, the mean EPI was calculated and examined to guarantee that none exhibited excessive signal dropout in insula and ventral striatum. Using linear and nonlinear transformations, and smoothing with a Gaussian kernel of full-width-half-maximum 8 mm, EPI and structural images were coregistered and normalized to the T1 standard template in MNI space [Montreal Neurological Institute (MNI)—International Consortium for Brain Mapping]. Global changes were removed by high-pass temporal filtering with a cutoff of 128 s to remove low-frequency drifts in signal.

Statistical analysis.

After preprocessing, statistical analysis was performed using the general linear model. Analysis was performed to establish each participant's voxelwise activation during parametric increases [i.e., increased R (R↑) or increased C (C↑)] and decreases [i.e., decreased R (R↓) or decreased C (C↓)] in R and C in one's own habitat (self habitat, H_S) compared with the other habitat [other habitat, H_O; e.g., [H_S] (C↑ + R↓) − [H_O] (C↓ + R↑)]. For example, C↑ in the H_S signifies the parametrically increased activity corresponding with increased number of competitors (1–6) entering the habitat where the subject's yellow dot is located. Our regressor matrix included the following:

1. Amount (i.e., 1–6) of C in H_S

2. Amount of R in H_S

3. Amount of C in H_O

4. Amount of R in H_O

5. Switching habitats

6. Amount of C when the subject decided to press the button to move to the other habitat based on C present in H_S

7. Switching based on R input in H_S

8. Switching based on C input in H_O

9. Switching based on R input in H_O

10. Win token in H_S

11. Loss token in H_S

Also, six head-motion parameters defined by the realignment procedure were added to the model as regressors of no interest. Regressors 1–4 represented the parametric changes in input while the subject remained in the habitat. Regressors 5–8 represented the amount of C and R in each habitat when the subject pressed the button to switch habitats. As for timing, each change in C was defined as an event of 1 s (parametrically modulated by amount of C in each patch) with an intertrial interval of 1 s. Because of the changing frequency from 1 to 4 Hz, we examined the 8 s event when R changed frequency, with a jitter of 1–4 s between changing R tokens. Over the full run, both left and right sides of the screen shown an equal number of competitors and an equal number of reward tokens. For the switch events (i.e., when the subject pressed the button to switch habitats), we examined subject responses the 1 s after they entered the habitat (again, parametrically modulated by the amount of C and R in both habitats). Activated voxels in each experimental context were identified using an event-related statistical model representing each of the regressors in the design matrix, convolved with a canonical hemodynamic response function and mean-corrected. Multiple linear regression was then run to generate parameter estimates for each regressor at every voxel.

We used a statistical threshold of p < 0.05 corrected for multiple spatial comparisons across the whole brain, except for a priori hypothesized regions, and familywise error (FWE) correction was applied. Small volume correction (SVC) was used on several a priori regions of interest, including the amygdala, medial orbital frontal cortex, presupplementary motor area (pre-SMA), motor cortex, ventrolateral prefrontal cortex (PFC), dorsal and ventral caudate, dorsal putamen, bilateral insula, and dorsal and ventral medial PFC (vmPFC), including the ventral ACC (Daw et al., 2006; Kennedy et al., 2009; Mobbs et al., 2009; Yu et al., 2010; Noonan et al., 2012). All SVCs were thresholded at p = 0.05_FWE-SVC; only clusters involving 10 or more contiguous voxels were reported.

Model: logistic regression.

We applied logistic regression to estimate w_c and w_r, two parameters that represented the relative weights that subjects placed on the number of competitors C and the reward rate R, respectively, when making a switching decision (Table 1). Logistic regression uses a Bernoulli distribution to model the likelihood of the data. In this work, we modeled the likelihood of switching of a particular subject by the following equation:

Where t_n ∈ {0,1}, 1 means switching, and 0 means staying, n is the total number of trials, and y_n is the probability of switching in the nth trial. The probability of switching, y_n, is modeled by the following logistic sigmoid function:

This function maps the input between 0 and 1, and is often used to model the probability. In this work, v = w_r (r₂ − r₁) − w_c (c₂ − c₁) − k. The standard iterative Newton's method was applied to find the parameters, w_c and w_r, which maximize the likelihood of the switching p(t). Our implementation of the likelihood function also weighted each sample differently, because the numbers of switching trials and staying trials were different. Each sample was weighted by the reciprocal of the sample size of the belonging group in the objective function.

Table 1.

For each subject, the parameter estimates for w_c and w_r and associated log-likelihoods showing the goodness of fit (the closer to zero, the better the fit)

Subject	wr	wc	Log-likelihood
1	6.7261	5.0945	−58.82
2	3.5859	9.0281	−61.10
3	−1.4145	3.4053	−112.29
4	0.8228	5.4602	−93.41
5	2.757	0.9231	−110.95
6	1.1919	4.4164	−97.23
7	5.0594	4.3368	−69.61
8	5.6292	1.5937	−58.01
9	6.1645	6.9535	−57.74
10	6.7461	6.8012	−68.43
11	4.015	5.6779	−74.49
12	1.6376	3.2004	−132.84
13	11.5011	4.5389	−36.08
14	5.4246	4.1447	−65.32
15	4.6573	6.4267	−71.49

Results

Behavioral results

We first examined whether the likelihood of switching was based upon increasing C or decreasing R in the subject's current (“self”) habitat (H_S) compared with the “other” habitat (H_O) and the influence of individual differences in this behavior. The number of tokens acquired by participants (mean ± SD: 5226.7 ± 417.8) correlated positively with the frequency of switches based on R (Pearson's r = 0.45; p < 0.044, two-tailed), and negatively with the frequency based on C (r = −0.50; p < 0.029), showing that switching based on the optimal balance between C and R was the best strategy. To learn whether subjects used explicit strategies to aid in their decisions to switch habitats, we examined the subjective ratings of the degree of attention paid to the competitors' behavior (assessed via postscan questionnaire) were positively correlated with switches based on increasing C in the participant's own habitat (r = 0.61; p < 0.008). Subjectively rated attention to R was positively correlated with switching based on increased R in the other habitat (r = 0.46; p < 0.041).

fMRI results

Reward systems in the capture of the token

We next examined the neural systems associated with capturing reward tokens. For the contrast of win (i.e., capturing the token) minus loss of token (i.e., observing a competitor win the token) across the whole experiment, we found increased activity in classic reward areas, including the bilateral ventral striatum [−14, 8, 14; Z = 4.46; p < 0.043, whole-brain corrected for FWE (WBC_FWE)] and medial prefrontal cortex (mPFC; 2, 44, −2; Z = 3.64; p < 0.005 FWE-SVC; Table 2).

Table 2.

Winning tokens

Region	Peak MNI coordinates			Z value	p value
Ventral striatum	14	8	14	4.46	0.043a
mPFC	22	44	2	3.64	0.0005b

^aWhole-brain corrected.

^bRegions small volume corrected at p < 0.05 FWE with a 10 mm sphere.

Parametric changes in C and R in input

Residing in the disadvantageous habitat.

In accordance with the simplest of input matching models, we next examined neural activation associated with being in a disadvantageous habitat [i.e., parametrically increased C and decreased R in H_S compared with decreased C and increased R in H_O ([H_S] (C↑ + R↓) − [H_O] (C↓ + R↑)]. To make sure that this activity was not driven by the increased probability of losing, we exclusively masked this comparison with the loss–win contrast (p < 0.05, uncorrected mask). We found increased activity in the pre-SMA (0, 10, 46; Z = 2.99; p < 0.021_FWE-SVC), anterior insula (AI; 34, 24, 4; Z = 2.63; p < 0.048_FWE-SVC), and dorsal anterior cingulate cortex (dACC; 12, 24, 28; Z = 2.84; p < 0.030_FWE-SVC; Fig. 2A; Table 3). Finally, this activity was not a result of the motor responses to switching (i.e., button press).

Figure 2.

Parametric analysis of brain activation associated with changing competition (C) and reward frequency (R). A, dACC, AI, dorsal premotor cortex (PMd), and pre-SMA activity when C increased and R decreased in the subject's own habitat (H_S) compared with the other habitat (H_O). B, vmPFC, ventral caudate, and dorsal caudate when C decreased and R increased in H_S relative to H_O. This suggests that preparatory motor areas come online more when it is disadvantageous to be in the H_S while staying in H_S due to low C and high R activates the mPFC and ventral caudate.

Table 3.

Parametric changes in C + R input; residing in the increasingly disadvantageous habitat

Region	Peak MNI coordinates			Z value	p value
Pre-SMA	0	10	46	4.21	0.021a
AI	34	24	4	2.63	0.048a
dACC	12	24	28	2.84	0.030a

^aRegions small volume corrected at p < 0.05 FWE with a 10 mm sphere.

Residing in the advantageous habitat.

Conversely [i.e., [H_S] (C↓ + R↑) − [H_O] (C↑ + R↓)], we found that being in the advantageous habitat resulted in increased activity in the mPFC and ventral striatum (Fig. 2B; Table 4). Again, we exclusively masked with the win–loss contrast (p < 0.05, uncorrected mask), which resulted in significant activity in the posterior dorsal caudate (−18, −12, 20; Z = 3.47; p < 0.036_FWE-SVC) extending to the anterior caudate (−16, 12, 12; Z = 2.84; p < 0.030_FWE-SVC), ventral caudate (10, 24, −10; Z = 3.91; p < 0.003_FWE-SVC), vmPFC; (−14, 58, −10; Z = 3.91; p < 0.026_FWE-SVC), posterior cingulate cortex (PCC; −6, −48, 12; Z = 4.52; p < 0.001_FWE-SVC), and intraparietal sulcus (−36, −70, 42; Z = 5.61. p < 0.0005_FWE-SVC). Finally, to make sure this activity was not driven by the increased probability of winning, we examined the correlation between how much the subject won and activity when in the advantageous habitat [i.e., [H_S] (C↑ + R↓) − [H_O] (C↓ + R↑)]. We found no significant in the striatum or mPFC (p < 0.05, uncorrected).

Table 4.

Parametric changes in C + R input; residing in the increasingly advantageous habitat

Region	Peak MNI coordinates			Z value	p value
Posterior caudate	−18	−12	20	3.87	0.036b
vmPFC	−14	58	−10	3.91	0.026b
Anterior caudate	−16	12	12	2.84	0.030b
Ventral caudate	10	24	−10	3.91	0.003b
PCC	−6	−48	12	4.25	0.001b
Intraparietal sulcus	−36	−70	42	5.61	0.005a
Cerebellum	−20	−84	−40	5.72	0.005a
Visual cortex	24	−98	−4	5.31	0.005a

^aWhole-brain corrected.

^bRegions small volume corrected at p < 0.05 FWE with a 8 mm sphere.

Residing in the advantageous habitat based on C or R.

We next examined the condition where it was disadvantageous to stay in H_S based on C, but advantageous based on R [i.e., [H_S] (C↑ + R↑) − [H_O] (C↓ + R↓)]. Here we found that increased C density in H_S was associated with activity in the left amygdala (−18, −6, −18; Z = 3.05; p < 0.019_FWE-SVC), as well as other parts of the social brain network, including the mPFC (14, 62, −14; Z = 3.81; p < 0.0005) and PCC (−2, −42, 26; Z = 3.69; p < 0.010_FWE-SVC) (Fig. 3A; Table 5). Conversely, when it was advantageous to switch based on increased R in H_O [i.e., [H_S] (C↓ + R↓) − [H_O] (C↑ + R↑)], we observed increased activity in the basal ganglia, specifically the dorsal caudate (24, 14, 18; Z = 3.15; p < 0.026_FWE-SVC), insula (48, 4, 14; Z = 2.97; p < 0.001), and dorsolateral PFC (28, 26, 32; Z = 3.95; p < 0.0005). This suggests that changes in C are tracked by activation in the amygdala, while changes in R are tracked by the dorsal caudate (Fig. 3B; Table 6).

Figure 3.

Switches based on C or R and individual differences in habitat switching behavior. A, Amygdala activity correlating with switches based on increasing C in the subject's habitat (H_S). B, Putamen activity associated with switches based on decreasing R in H_S. C, Bars represents each subject's percentage difference between switches based on w_c − w_r. Positive values represent a weighted preference for C while negative values are indicative of a preference to switches based on R. D, Amygdala activity associated with increasing C and increasing R in H_S, again supporting the reasoning that the amygdala is more sensitive to changes in w_c − w_r. E, Conversely, putamen activity associated with w_r − w_c in H_S, suggesting that this region is sensitive to changes in R.

Table 5.

Parametric changes when it is disadvantageous to stay in the habitat based on C but not R input

Region	Peak MNI coordinates			Z value	p value
Amygdala	−18	−6	18	3.05	0.019b
vmPFC	−14	62	−14	3.81	0.0005
PCC	−2	42	26	3.69	0.010b
Parietal cortex	8	−68	56	4.29	0.0005a
Cerebellum	0	−82	−24	4.23	0.0005a

^aWhole-brain corrected.

^bRegions small volume corrected at p < 0.05 FWE with a 8 mm sphere.

Table 6.

Parametric changes when it is disadvantageous to stay in the habitat based on R but not C input

Region	Peak MNI coordinates			Z value	p value
Dorsal putamen	24	14	18	3.15	0.026a
AI	48	4	14	2.97	0.001
Dorsolateral prefrontal cortex	28	26	32	3.95	0.0005

^aRegions small volume corrected at p < 0.05 FWE with a 8 mm sphere.

Individual preferences in C and R

We tested this idea further by examining individual preferences to switch based on C or R for the input-matching contrast (i.e., [H_S] (C↑ + R↓) − [H_O] (C↓ + R↑)). In our model, we included w_c and w_r, two weight coefficients that represented the relative value that subjects placed on the number of competitors C and the reward rate R, respectively. We looked at all trials (i.e., changing R) across all the subjects and found weights that best predicted switching behavior for each subject. The value of the current habitat, V_HS, was modeled as follows: V_HS = w_rr_s − w_cc_s.

r₁ was the reward rate in the current habitat expressed as a percentage of the maximum reward rate [i.e., r₁ = R/4 where R ranged from 1 (low) to 4 (high)]. c₁ was the number of competitors in the current habitat expressed as a percentage of the maximum number of competitors [i.e., c₁ = C/6 where C ranged from 1 (low) to 6 (high)]. Similarly, the value of the other habitat was modeled as V_HO = w_rr₂ − w_cc₂ where r₂ and c₂ are the percentage reward rate and competitor numbers in the other habitat. Assuming one is a rational actor then one would predict switching if V_HO > V_HS + k, where k is some constant switching cost. We estimated the switching cost to be k = r₁/8c₁ as the probability of winning was approximately r₁/c₁ and, because trials took 8 s (trials are defined as the period the tokens switched colors) and the habitat switching took 1 s, expected reward lost during switching would be one-eighth of that amount. It is assumed that the probability of switching correlates positively with the difference between V_HO and V_HS + k. When V_HO − (V_HS + k) is large, the subject is more likely to switch. We modeled this problem by logistic regression (Bishop, 2006) and estimated the two coefficients by maximizing the likelihood of switching according to the following equation: p(switch) = f[V_HO − (V_HS + k)] = f(w_rr₂ − w_cc₂ − w_rr₁ + w_cc₁ − k) = f[w_r(r₂ − r₁) − w_c(c₂ − c₁) − k], here f(x) is the logistic function, which models the probability of switching. Using these coefficients as group-level regressors, we found that the amygdala (−26, −2, −26; Z = 2.86; p < 0.035_FWE-SVC; Fig. 3C; Table 7) positively correlated with the amount of relative weight a subject assigned to C compared with the R when switching (i.e., w_c − w_r) (Fig. 3D). Conversely, dorsolateral putamen activity (22, 0, 18; Z = 2.88; p < 0.048_FWE-SVC), dACC (14, 36, 16; Z = 3.15; p < 0.001), and ventral tegmental area (14, −20, −16; Z = 2.73; p < 0.003) correlated with the amount of relative weight a subject assigned to R compared with C (i.e., w_r − w_c) (Fig. 3E; Table 8). We finally wanted to see whether activity in amygdala and putamen were confounded with the amount of tokens acquired. We found no significant correlation with either of these regions (p > 0.05, uncorrected).

Table 7.

Parametric changes in C + R input

Region	Peak MNI coordinates			Z value	p value
Amygdala	−26	−2	−26	2.86	0.035b
Lateral parietal cortex	−38	−68	44	3.49	0.0005
Dorsal temporal lobe	−40	−22	−8	3.44	0.0005
PCC	−14	−26	40	3.22	0.001a
Middle temporal gyrus	−58	−40	−8	3.09	0.001a

^aWhole-brain corrected.

^bRegions small volume corrected at p < 0.05 FWE with a 8 mm sphere.

Correlation with switching preferences based on C − R.

Table 8.

Parametric changes in C + R input

Region	Peak MNI coordinates			Z value	p value
Dorsal caudate	22	0	18	2.88	0.048b
Ventral tegmental area	14	−20	−16	2.73	0.003b
dACC	14	36	16	3.15	0.001
Lateral visual cortex	46	−84	−10	3.76	0.0005a
Globus pallidus	−24	12	−2	3.17	0.001
Superior temporal sulcus	44	−50	6	3.12	0.001
AI	34	28	−8	3.39	0.001

^aWhole-brain corrected.

^bRegions small volume corrected at p < 0.05 FWE with a 8 mm sphere. Other coordinates are uncorrected.

Correlation with switching preferences based on R − C.

Discussion

We demonstrate that when it is increasingly disadvantageous to be in a given habitat compared with another more advantageous location, BOLD signal was increased in a basic set of neural systems involved in preparatory motor responses and possibly urges to switch location. The pre-SMA notably becomes more active during motor preparation and action selection (Romo and Schultz, 1990). For example, cells in the pre-SMA may provide a link between reward expectancy and motor execution (Scangos and Stuphorn, 2010). Likewise, the insula, which is connected to the pre-SMA in humans (Johansen-Berg et al., 2004), may also relate to errors that should drive a change in behavior (Klein et al., 2007).

Our dACC activity is also intriguing given its involvement in behavioral adjustment, reward monitoring, and encoding of cost (Cohen et al., 2007). Indeed, single-cell recordings show that firing rate in macaque dACC predicts patch leaving during a reward foraging task (Hayden et al., 2011). In humans, the dACC may be involved in the cost of foraging or the average value of the foraging environment (Kolling et al., 2012). The dACC is unlikely to act in isolation and activity in these interconnected neural systems could be associated with either the negative value associated with staying in the disadvantageous habitat or the drive to switch to the advantageous habitat (Hayden et al., 2011). Importantly, we also found activity in the AI, a region commonly coactivated with the dACC (Craig, 2009). Given that the AI is implicated in a number of operations, it is difficult to state what role it plays in the current experiment. However, the AI has been implicated in decision-making, interoception, awareness, and urges or motivation to act (Craig, 2009). Therefore, coactivation of this region with other frontal regions, including dACC, has previously been reported in other situations, for example when errors occur, that indicate that a change in behavior is needed (Klein et al., 2007, Noonan et al., 2012).

Keeping track of advantageous habitats is presumably critical in stay-or-switch decisions. We found that being in the advantageous habitat was associated with increased activity in the mPFC. Research in humans and other animals shows that the mPFC is critical to learning primary and secondary rewards (Chib et al., 2009) and lesions to vmPFCs of nonhuman primates significantly impair reward-guided selection of action (Noonan et al., 2012), which in our study is perhaps analogous to staying in the current advantageous habitat. The mPFC and ventral caudate are both also implicated in value-based exploitative decision-making (Daw et al., 2006), supporting speculation that activity in both regions would increase when exploiting an advantageous habitat. The mPFC is highly connected to the ventral caudate and dorsal caudate, regions involved in encoding reward value and activated to increasing amounts of reward (Delgado et al., 2003). Indeed, the dorsal caudate has been shown to bias animals to collect rewards and, in turn, may invigorate flexible goal-related actions (Kimchi et al., 2009).

Amygdala activity occurred both (1) when it was increasingly disadvantageous to stay in the habitat based on the increasing density of C (Fig. 3A) and (2) when making a switching decision based on the number of competitors compared with the reward rate (Fig. 3D). This supports the suggestion that the amygdala mediates complex social strategies that characterize competitive behaviors (Silk, 2007). For example, lesions to the primate amygdala have dramatic effects on social interaction and status (Kling, 1972), including socially disinhibited behavior (Amaral, 2002). In humans, the amygdala has been shown to relate to keeping track of social relationships and attention to social cues (Adolphs et al., 1998). It is important to note the amygdala is also associated with fear and anxiety (Bishop et al., 2004; Mobbs et al., 2010). However, we found no correlation between trait anxiety scores and amygdala activity, suggesting that this region is not just activating in response to general fear. Research suggests that the amygdala is implicated in social avoidance in humans (Hahn et al., 2011), in discomfort when others invade our personal space (Kennedy et al., 2009), and keeps track of social value (Adolphs, 2010), an aptitude presumably valuable for tracking competitors. Together, these studies support the idea that the amygdala plays a role in competitive avoidance.

Complementing the role played by the amygdala in switching behavior, was the increased activity in the dorsal putamen when, based on the decreasing supply rate of R, it was increasingly disadvantageous to stay in the habitat (Fig. 3B). Supporting this was the interindividual differences in the amount of relative weight a subject assigned to w_r − w_c, which again resulted in activity in the putamen (Fig. 3E). The putamen receives input from the primary sensory and motor cortices (McGeorge and Faull, 1989) and mediates the interplay between sensorimotor and motivation information (Balleine et al., 2007). The general consensus is that the putamen is also involved in habitual actions, yet may be involved in simple, possibly Pavlovian cue-controlled reward seeking (Balleine et al., 2007; Zapata, 2010). Indeed, the putamen is involved in motor readiness and the initiation of actions based on the current reward value, fitting with the notion that the putamen tracks how actions lead to correct responses (Balleine et al., 2010). The putamen is also well connected to the dorsolateral prefrontal cortex, a region involved in self-control (Hare et al., 2009) and may interact with the striatum when orienting attention toward reward (Hikosaka, 2006).

Our findings bridge the fields of social and decision neuroscience and behavioral and computational ecology to demonstrate that a basic set of neural systems drive subjects to reside in, or relocate to, profitable habitats. Moreover, we show that the decision to switch habitats is driven by interindividual preferences based on social competition or reward-weighted bias. In real ecologies, input matching is likely to be more complex whereby despotic individuals occupy quality habitats and predatory threat drives habitat switching. Here we test the simplest of input-matching models, yet we set the scene for future studies to examine a broad range of contexts that influence optimal foraging behaviors (Gallistel et al., 2001; Gallistel and Gibbon, 2001).