Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Aug 15.
Published in final edited form as: Neuroimage. 2021 May 12;237:118159. doi: 10.1016/j.neuroimage.2021.118159

Subjective value, not a gridlike code, describes neural activity in ventromedial prefrontal cortex during value-based decision-making

Sangil Lee a,1,*, Linda Q Yu a,b,1, Caryn Lerman c, Joseph W Kable a
PMCID: PMC8298105  NIHMSID: NIHMS1723753  PMID: 33991700

Abstract

Across many studies, ventromedial prefrontal cortex (vmPFC) activity has been found to correlate with subjective value during value-based decision-making. Recently, however, vmPFC has also been shown to reflect a hexagonal gridlike code during navigation through physical and conceptual space, and such gridlike codes have been proposed to enable value-based choices between novel options. Here, we first show that, in theory, a hexagonal gridlike code can in some cases mimic vmPFC activity previously attributed to subjective value, raising the possibility that the subjective value correlates previously observed in vmPFC may have actually been a misconstrued gridlike signal. We then compare the two accounts empirically, using fMRI data from a large number of subjects performing an intertemporal choice task. We find clear and unambiguous evidence that subjective value is a better description of vmPFC activity in this task than a hexagonal gridlike code. In fact, we find no significant evidence at all for a hexagonal gridlike code in vmPFC activity during intertemporal choice. This result limits the generality of gridlike modulation as description of vmPFC activity. We suggest that vmPFC may flexibly switch representational schemes so as to encode the most relevant information for the current task.

Keywords: vmPFC, Valuation, Grid cell, Hexagonal modulation

Many studies in decision neuroscience have identified a critical role for the ventromedial prefrontal cortex (vmPFC) in decision-making. Specifically, neural activity in the vmPFC correlates with the subjective value of expected or experienced outcomes across a wide variety of decision-making tasks (Bartra et al., 2013; Clithero and Rangel, 2013; Levy and Glimcher, 2012). Neural correlates of subjective value have been found in vmPFC using both fMRI in humans as well as single cell recording in non-human animals (Howard et al., 2015; Kable and Glimcher, 2007; McNamee et al., 2013; Strait et al., 2014; Yamada et al., 2018). One straightforward interpretation of these results is that vmPFC encodes the subjective value of potential outcomes, which is then used to make choices between outcomes (Kable and Glimcher, 2009).

However, recent human neuroimaging studies have found that a similar area of vmPFC also serves a different function: encoding representational maps that enable navigation through physical and conceptual spaces (Constantinescu et al., 2016; Doeller et al., 2010; Jacobs et al., 2013). Both intracortical recordings and fMRI studies of humans navigating through virtual arenas have shown that activity in vmPFC is modulated in a hexagonal manner by the direction of travel, which is a pattern characteristic of grid cells (Bao et al., 2019; Doeller et al., 2010; Jacobs et al., 2013; Nau et al., 2018). Grid cells were first discovered in entorhinal cortex (ERC) during spatial navigation and provide an efficient representation of two-dimensional space (Behrens et al., 2018; Doeller et al., 2010; Hafting et al., 2005; Whittington et al., 2020). More recently, fMRI signatures of this hexagonal gridlike code have been observed in vmPFC during navigation in a purely conceptual space (Constantinescu et al., 2016). Specifically, stimuli in that study were defined along two dimensions, and when subjects imagined a stimulus transforming through the conceptual space defined by those two dimensions, activity in vmPFC showed a similar response pattern as that observed during two-dimensional spatial navigation.

These recent results have led some to argue that these hexagonal gridlike codes could serve a general role in complex cognition, including decision-making (Bellmund et al., 2018; Bongioanni et al., 2021). Many decisions require choosing between options that differ along (at least) two dimensions – for example, gambles that differ in risk and payoff, foods that differ in health and taste, or goods that differ in quality and price. A gridlike representation of such a two-dimensional space could allow, for example, inferences to be made about the qualities of novel attribute combinations based on their proximity in this conceptual space to attribute combinations that had previously been experienced (Bellmund et al., 2018). Recently, hexagonal gridlike modulation has been observed in the vmPFC fMRI signal in macaques in response to cues signaling novel combinations of reward attributes (magnitudes and probabilities). Based on this observation, the authors argued that gridlike representations in vmPFC could underlie a mechanism for making choices between novel (rather than highly pretrained) options that is shared across species, especially as most human fMRI studies implicating the vmPFC in decision-making involve such choices (Bongioanni et al., 2021). However, there has not yet been a direct test of whether there is hexagonal grid-like modulation, in the vmPFC or elsewhere, during decision-making.

Here we put this idea to the test: does the vmPFC exhibit hexagonal gridlike modulation during decision making, thereby reflecting a conceptual map of attribute space? We first show that a hexagonal gridlike modulation signal can in some cases be highly correlated with subjective value in the kinds of experimental designs widely used to identify value signals, suggesting that previously identified value correlates could have been misconstrued gridlike modulation. Combined with recent empirical results (Bongioanni et al., 2021), this motivates returning to a large existing intertemporal choice dataset (Kable et al., 2017), to empirically test if BOLD activity in vmPFC during this task is better explained by subjective value or a hexagonal gridlike code. Across three different analyses, we show unambiguously that vmPFC activity in the intertemporal choice task is better described by a subjective value signal than by a hexagonal gridlike modulation. This finding limits the generality of hexagonal grid representations in vmPFC. Instead, vmPFC may flexibly switch representational schemes so as to encode the most pertinent information to the task at hand.

Methods

Dataset

All raw fMRI images and the behavioral responses are available online at openneuro.org (DOI: 10.18112/openneuro.ds002843.v1.0.0). All fMRI analyses were performed in FSL, which can be downloaded for free (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FSL). All analysis codes and interim files are available online at OSF (https://osf.io/78nym/).

We used the fMRI dataset from Kable et al. (2017), as its large number of subjects permits a high-powered test between a subjective value and hexagonal grid code during decisions that involve a tradeoff between two choice attributes. All participants provided informed consent and all procedures were approved by the University of Pennsylvania Institutional Review Board. Participants completed two sessions in which they performed an intertemporal choice task in the scanner 10 weeks apart. In each scan session, participants made 120 binary choices between a smaller immediate reward and a larger later reward, across 4 runs (30 choices per run). The smaller immediate reward was held constant at $20 today while the larger later reward varied in amount (A: $21 ~ $85) and delay (D: 20 days ~ 180 days) from trial to trial (Fig. 1). The specific combination of amount and delay of the larger later reward were never repeated within the session and were sampled across the entire range for each run so that there would be no difference in the range of options between runs. On average, participants saw most of the amount and delay range in the task after the first run (96.5% of amount range, 97.2% of delay range). Session 2 had the same choice options as session 1, but in a different random order. On each trial, the amount and delay of the larger later option was displayed on the screen, while the constant immediate option was not displayed. Participants used a button pad to indicate whether they would accept the larger delayed option shown on the screen or reject it in favor of the smaller immediate option, which was not shown on the screen. This design simplifies the identification of neural correlates of subjective value, as only the subjective value of the delayed option needs to be considered; since the immediate option is fixed, the sum, difference or ratio of the subjective values of the two options are all linearly related to the subjective value of the delayed option.

Fig. 1. Intertemporal choice task design.

Fig. 1.

Open in a new tab

Panel a shows an example trial in intertemporal choice task where the delayed larger option is shown on the screen (black square). The immediate smaller option, which is the same on every trial, is not shown on the screen (dotted square). Panel b shows all 120 choice trials in the task. Each choice is between an immediate $20 option (green dot) and a delayed larger monetary option (one of the blue circles).

The two hypotheses provide different predictions of the expected signal in this task (Fig. 2). Subjective values depend on how individual subjects weight the two attribute dimensions, but all subjects prefer larger magnitudes and smaller delays, leading to the highest response in one corner of the two-dimensional attribute space. For hexagonal grid modulation, the relevant conceptual navigation in this task involves “traversing” between the two options being considered. A gridlike code would lead to peaks in activity when navigating along hexagonally symmetric directions spaced sixty degrees apart, with the angle of these peak directions (the grid angle) being the one free parameter. For example, a person with a grid-angle at 0° would have activity peaks at 0° and 60° of traversing angle and troughs at 30° and 90°, while another person with a grid angle at 30° would have peaks at 30° and 90° and troughs at 0° and 60°. The distribution of potential trajectory angles in this task (between a delayed larger monetary option and immediate $20) are shown in Fig. 3a. To verify that our chosen task design does not unduly limit the possible trajectories, we also calculated the distribution of trajectory angles should one choose two random options in the task space (i.e., two of the blue circles in Fig. 1b). After excluding ‘no-brainer’ choices, in which one option is stronger than the other on both attributes, the remaining choices offer trajectory angles that are similar to what we have in the current design (Fig. 3b).

Fig. 2. Example SV signal and hexagonal gridlike signal in two attribute space.

Fig. 2.

Open in a new tab

Panel a shows an example subjective value signal when comparing two options that vary in amount and delay. In the choice task used here, an SV signal will vary depending on the subjective value of the variable delayed option. Panel b shows an example hexagonal gridlike signal when mentally traversing between two points (shown in example is a grid angle of 0°, such that the peak activity occurs when traversing at 0° and 60°; this grid angle may vary from person to person). In the choice task used here, a hexagonal gridlike signal will vary depending on the trajectory angle between the two options.

Fig. 3. Distribution of trajectory angles.

Fig. 3.

Open in a new tab

Panel a shows the distribution of trajectory angles in this task when traveling between the immediate option (green dot in Fig 1b) and the delayed options (blue circles in Fig 1b). Panel b shows the distribution of trajectory angles if two random options were to be chosen from the task space (i.e., two of the blue circles in Fig 1b). Trajectory angles between two options with a tradeoff are marked in blue while those between two options with no tradeoff (i.e., when one option dominates the other option) are marked in orange.

Of the 160 participants that completed session 1, we excluded participants with any missing runs (n = 6), too much head movement (any run out of 4 runs with > 5% of mean image displacements greater than 0.5 mm; n = 3), more than 3 missing trials per run for two or more runs (n = 2), entirely one-sided choice such that the participant always chose the immediate or the delayed option (n = 3), and one participant who expressed knowledge of their experimental condition in the original study. This resulted in the final sample size of 145 participants for session 1. Of these participants, only 114 completed session 2, from which we also excluded those with missing runs (n = 3), too much head movement (n = 2), too many missing trials (n = 2), or entirely one-sided choices (n = 5). This gave us a total of 102 participants for session 2. In total, 145 participants’ data were used for session 1 and 102 participants’ data were used when analyzing both sessions.

Participants were scanned with a Siemens 3T Trio scanner with a 32-channel head coil. T1-weighted anatomical images were acquired using an MPRAGE sequence (T1 = 1100 ms; 160 axial slices, 0.9375 × 0.9375 × 1.000 mm; 192 × 256 matrix). T2* -weighted functional images were acquired using an EPI sequence with 3mm isotropic voxels, (64 × 64 matrix, TR = 3,000ms, TE = 25 ms; tilt angle = 30°) involving 53 axial slices with 104 volumes. B0 fieldmap images were also collected for distortion correction (TR = 1270 ms, TE = 5 and 7.46 ms). The datasets were preprocessed via FSL FEAT (FMRIB fMRI Expert Analysis Tool). Functional images were skull stripped with BET (FMRIB Brain Extraction Tool), motion corrected and aligned with MCFLIRT (FMRIB Linear Image Restoration Tool with Motion Correction), spatially smoothed with a FWHM 9 mm Gaussian Kernel, and high pass filtered (104 s cutoff). Registration was performed with FNIRT with warp resolution of 20 mm (FMRIB’s Non-linear Image Registration Tool) to a 2 mm MNI standard brain template.

Could a hexagonal grid signal be mistaken for a subjective value signal?

Before any empirical analysis, we asked whether, in theory, hexagonal gridlike modulation could mimic or account for activity correlated with subjective value in this task. To do this, we simulated a subjective value signal at a given discount rate for a range of amounts and delays, and then estimated the best-fitting gridlike modulation for this signal. For a given discount rate, the subjective values of delayed monetary outcomes were calculated using the hyperbolic model:

SV of $A in D days =A1+kD (1)

where k is the individual discount rate. The amount A varied from 20 to 80 in increments of 2 (31 levels) and the delay D varied from 20 to 180 in increments of 5 (33 levels) resulting in a total of 1023 subjective values for a given k. This 1023-element vector (SV) was then regressed against two hexagonal grid modulation regressors:

Vt=β0+β1cos(6θt)+β2sin(6θt)+ϵt, θt=atan(At−20Dt) (2)

The model contains a linear combination of the sine and cosine of the trajectory angle θ with 60° periodicity. The trajectory angle θ is taken as the angle between the abscissa and the traversing line between the immediate option ($20 now) and the delayed option ($A in D days) in the two-dimensional space defined by the amount and delay attributes (e.g., Fig. 1b, 2b). The linear sum of cosine and sine function acts as a phase shift (i.e., generally: pcosθ + qsinθ = kcos (θ + ϕ)) such that we can identify the best phase (i.e., grid angle) of the cosine modulation without using a non-linear regression. After fitting the regression, we calculated the Pearson correlation between SV and the fitted signal to assess the similarity between the two. This procedure was repeated for 51 levels of k whose base-10 log ranged from −5 (i.e., k = 0.00001: very patient) to 0 (i.e., k = 1: very impatient) in 0.1 increments. For comparison, we also calculated the Pearson correlation between SV and hexagonal grid modulation signals for six grid angles uniformly sampled between 0 and 50 degrees and report these in the supplemental materials.

fMRI analysis – voxelwise GLM

In empirical fMRI data, we first performed two GLMs in session 1 data to test for activity that was correlated with subjective value or hexagonal grid signals. For the subjective value GLM, we used two regressors: an event regressor that modeled average activity of all trials, and second regressor that modeled activity modulated by subjective value. The subjective value of the delayed reward was estimated by fitting a hyperbolic discounting function to choice data using a logit choice model (A is the delayed amount, D is the delay, k is the discount rate, and β is the scaling factor):

logit(P(choice = delayed ))=β(A1+kD−20) (3)

Regions correlated with subjective value were identified by performing a sign-flipping permutation test on individual beta images of the subjective value regressor. For the gridlike modulation GLM, we used three regressors: an event regressor and two hexagonal grid angle regressors (n = 6):

cos(nθt), sin(nθt), θt=atan(At−20Dt) (4)

Regions correlated with hexagonal grid signal were identified by first transforming the F-statistic of the two hexagonal grid regressors to a z-statistic and then by performing a sign-flipping permutation test on the z-transformed F-stat images (Constantinescu et al., 2016).

fMRI analysis – model comparison

We tested whether vmPFC activity is better described by a subjective value or hexagonal gridlike signal in three different ways. First, and most straightforward, we performed a model comparison between the subjective value GLM and the hexagonal modulation GLM. We compared the Akaike Information Criterion (AIC) scores of the two GLM models in each of four ROIs: vmPFC and ventral striatum ROIs from the Bartra et al. (2013) meta-analysis of subjective value, and two spherical ROIs (radius = 2 voxels) from the peak activation coordinates reported by Constantinescu et al. (2016) in vmPFC and ERC.

Second, we assessed if 6-fold grid modulation was the best model out of all n-fold grid modulation GLMs. If there is indeed hexagonal grid modulation, a 6-fold model should explain the most variance compared to 4-fold, 5-fold, 7-fold, and 8-fold modulation models. We repeated the grid-modulation GLM analysis in the four ROIs for 4~8fold regressors: cos(nθt), sin(nθt), n = 4 ~ 8. Since all the models have the same number of parameters, we compared the 4-fold, 5-fold, 6-fold, 7-fold, and 8-fold models using the z-converted F-statistics of the grid angle regressors to assess which set of n-fold regressors explain the most variance in each ROI. Furthermore, to set up clear expectations of the resulting pattern under different hypotheses, we simulated the analysis as closely as possible by generating the BOLD signal for each subject and regressing these simulated data with the 4–8fold regressors. To simulate the results of the analysis when the signal is subjective value, we used the individual’s fitted hyperbolic subjective value as modulators of neural activity that were convolved with a double-gamma HRF with autocorrelated noise added (drawn from a multivariate Gaussian distribution with inter-TR correlation of 0.12). To simulate the results of the analysis when the signal is a hexagonal gridlike modulation, we randomly chose one unique grid angle for each subject (between 0° and 60°) and calculated each trial’s neural activation according to the alignment between the grid angle and trial angle (n = 6):

cos(n(θt−ϕ)), θt=atan(At−20Dt) (5)

The resulting hexagonal modulation was also convolved with a double-gamma HRF with autocorrelated noise added. The simulated subjective value BOLD signal and the hexagonal grid BOLD signal were then regressed against the same regressors used for the real data.

To ensure the robustness of the results, we performed the model comparison analyses above in alternatively defined ROIs and also in alternatively scaled attribute spaces. For the former, we defined spherical ROIs from peak GLM coordinates closest to the vmPFC and ERC peaks in Constantinescu et al. (2016) and to the vmPFC and VS peaks in Bartra et al. (2013). For the latter, we calculated n-fold grid modulation regressors in an attribute space where both amount and delay were min-max normalized to have equal range. The model comparison results in these alternative ROIs and alternative attribute spaces were the same as in the original ROIs and original attribute spaces and are therefore presented in the supplemental materials.

fMRI analysis – cross-session consistency analysis

Third, following the methods of previous studies, we tested the consistency of grid angles across session 1 and session 2 data. Based on the properties of grid cell representations in non-human animals, it is assumed that for a given brain region each person’s hexagonal grid is oriented at a unique angle that stays constant across time. For example, when people are navigating through a two-dimensional space, one person’s grid may have a 6-fold modulating activity that peaks when the person traverses through space at 20 + 60x (x = 0 … 5) degree angles while another person’s activity may peak at 40 + 60x (x = 0 … 5) degree angles. We test for such consistency by first calculating each individual’s unique grid angle from the first session’s data using cos(nθ) and sin(nθ) as regressors. The average coefficients in each of the four ROIs (β¯cos,β¯sin) were used to calculate that individual’s n-fold grid angle for that ROI (ϕn=atan(β¯sin/β¯cos)/n). Then, we tested if neural activity in the same ROI in session 2 was aligned with this grid angle by using the consistency regressor shown in Eq. 5. The average z-statistic of this consistency regressor within the pre-defined ROIs was used to measure the consistency effect across sessions. The key test was whether the consistency effect was the highest at 6-fold, rather than 4-, 5-, 7-, or 8-fold modulation. In addition to examining grid angle consistency, we also examined the distribution of estimated 6-fold grid angles and report this in the supplementary materials.

Again, to set up clear expectations of the grid angle consistency results under different hypotheses, we simulated the results of this grid angle consistency analysis under two conditions: when the underlying signal is subjective value, and when the underlying signal is hexagonal modulation. For subjective value, we calculated the 102 participants’ subjective values in both session 1 and session 2 separately via Eq. 3 and simulated the BOLD response by convolving the subjective value modulation with a double gamma HRF. For 6-fold modulation, we picked a random angle between 0° and 60° for each subject, which stayed the same across both sessions, and simulated a 6-fold modulation signal according to Eq. 5 (with n = 6). The resulting simulated neural activity were then convolved with a double-gamma HRF. After adding autocorrelated noise to both subjective value and grid signal, we performed the grid-angle consistency analysis in the same manner as on the real data as outlined above.

The grid angle consistency analysis is the only test that uses data from session 2. As reported in Kable et al. (2017), after session 1 participants were randomized to either a cognitive training intervention or an active control condition. As there were no differences between the two groups in brain activity or decision-making in session 2, we combine them in the analyses reported below. However, when we performed the consistency analysis in the control group only, we found the same results.

We performed the grid angle consistency analysis across sessions 1 and 2 to maximize statistical power. However, since one concern might be that the two sessions are too distant in time for any potential grid-like representations to be stable, we also performed the grid angle consistency analysis across runs within session 1 data. The within-session cross-run results were similar and are provided in the supplemental materials.

Results

Here we compare two potential coding schemes for vmPFC during decision-making, subjective value and hexagonal grid modulation, both of which are functions of the two attribute dimensions of the choice options. In intertemporal decision-making, the domain we study here, the two attribute dimensions are the amount of the money and the delay until its receipt. A subjective value code predicts that the activity elicited by an option increases as the amount of money increases and decreases as the delay to receipt increases (Fig. 2 a). The relative slopes of these changes will vary across people depending on the relative weight they place on monetary amounts and delays. On the other hand, a hexagonal gridlike code predicts that activity will vary as a function of the angle in two-dimensional attribute space between the two options that are being compared, with highest activity when this angle matches the person’s unique grid angle (Fig. 2 b). This prediction assumes that comparing two options in a choice task is akin to conceptually navigating between them in two-dimensional attribute space. In contrast to other navigation tasks, in the case of binary choice comparisons, it is unknown where conceptual navigation would start and where it would end; however, because the hexagonal grid is symmetric, travel in both directions (that is, from option 1 to option 2 or vice versa) yields equivalent predictions (e.g., 45° is the same as 225°).

It is important to note that in decision-making tasks the trajectory angles are necessarily constrained to half of the entire possible space because only comparisons between two options that involve a tradeoff between the two attributes are non-trivial. That is, most two-attribute choice problems involve a comparison between one option that is better on one attribute and another option that is better on the other attribute; in the intertemporal decision-making case, this translates to choices between greater amounts of money at longer delays versus lesser amounts of money at shorter delays. For example, a choice between $20 today and $30 in 10 days involves a tradeoff between more money and a longer delay, while a choice between $20 today and $10 in 10 days involves no tradeoffs, the immediate option dominates as it is both more money and sooner. Both decision theory and neural evidence suggest that such dominated or “no-brainer” choices, in which one option is better than the other on both attributes, can use different psychological and neural mechanisms than choices that involve tradeoffs between the attributes (Hunt et al., 2012; Kahneman and Tversky, 1979). Because two quadrants within the two-dimensional attribute space correspond to dominated choices (in the examples above, $20 today versus $30 in 10 days corresponds to a trajectory angle of 45° while $20 today versus $10 in 10 days corresponds to a trajectory angle of 135°), most two-attribute choice tasks, including the choice task we use here, only sample the other two quadrants (Fig. 2a–b).

In the intertemporal choice task we study, one of the two options is fixed, so that participants choose on every trial between an immediate option that is always $20 and a delayed option that varies from trial to trial in monetary amount and length of delay. This feature of the design simplifies testing for correlates of subjective value, as we can now test for signals modulated by the subjective value of the variable, delayed larger reward, according to that individual’s intertemporal preferences (captured by their discount rate). However, this feature does not complicate our ability to test for a hexagonal gridlike code; though it is equivalent to restricting the space of choices to one quadrant of Fig. 2a, given the symmetry of the hexagonal grid code, it does not further restrict the range of potential trajectory angles sampled in our design. As shown in Fig. 3, the distribution of trajectory angles in the current task (Fig. 3a) is similar to the distribution of trajectory angles one would get if two options are chosen at random with the constraint that there is a tradeoff between the two options (i.e., that one option is not both bigger in amount and proximal in delay; Fig. 3b).

We first show via simulation that a hexagonal grid modulation over decision attribute space could in theory account for previously observed neural correlates of subjective value in this task (Cox and Kable, 2014; Kable et al., 2017; Kable and Glimcher, 2007, 2010). We calculated the best-fitting grid angle for different subjective value landscapes generated assuming different intertemporal preferences (i.e., different discount rates for delayed rewards). Fig. 4 shows subjective value signals and their best-fitting gridlike signals at various discount rates. The maximum correlation between the two signals ranges between r = 0.5 and r = 0.7 depending on the discount rate. Note that these high correlations occur only at specific grid angles and not at others (specifically grid angles around 30–40°, Supplementary Fig. 1). Nonetheless, these simulations show that it is possible for hexagonal modulation and subjective value to be confused with each other under some conditions and help to motivate testing and comparing these two accounts in neural data.

Fig. 4. Correlation between SV signal and its most similar hexagonal modulation signal.

Fig. 4.

Open in a new tab

The top three panels show simulated subjective value signals for various delayed amounts at various discount rates and the next three panels below show their respective best fitting hexagonal grid modulations. The correlations between the two signals are provided below in dotted lines across various discount rates.

We next test for neural activity correlated with subjective value or gridlike modulation in real data. Subjects in our study participated in two imaging sessions separated by ten weeks, and we perform this test in session 1 data. Fig. 5 shows significant effects across the whole brain for both subjective value and gridlike modulation regressors. Perhaps given the statistical power in our dataset (n = 145), we observed significant effects in widespread brain regions for both analyses. The subjective value effects, which we have reported previously for this dataset (Kable et al., 2017), include peaks in the vmPFC and ventral striatum that overlap with previous meta-analyses of subjective value correlates (Bartra et al., 2013). The gridlike modulation effects include peaks in the vmPFC and entorhinal cortex (ERC) as observed previously during conceptual navigation by Constantinescu et al. (2016).

Fig. 5. Regions with significant subjective value correlation (top) and significant hexagonal gridlike modulation (bottom) in session 1.

Fig. 5.

Open in a new tab

(p < 0.05 with permutation testing with threshold free cluster enhancement). The rightmost brain shows overlays of the two ROIs from Bartra et al. (2013) on top and the two ROIs from Constantinescu et al. (2016) on bottom.

The widespread effects observed for the gridlike modulation analysis, which encompass almost the entire brain, emphasize an important point: given that the angle of grid modulation is a free parameter, including between runs, these regressors provides a good degree of flexibility to fit a diversity of response patterns. Hence, definitive evidence for a hexagonal grid modulation signal requires showing that modulation is stronger at 6-fold than at other folds and that the angle of grid modulation is consistent across time. We turn to these stronger tests as we next employed three different ways to directly compare subjective value and hexagonal gridlike codes as accounts of neural activity in this task.

First, we simply compare the AICs of the subjective value and hexagonal grid GLMs above. We found the subjective value GLM was a better descriptive model of fMRI activity in all four ROIs tested (Fig. 6). Most subjects had smaller AICs for the subjective GLM than for the hexagonal grid GLM in both the vmPFC and ventral striatum ROIs drawn from Bartra et al. (2013), as well as in both the vmPFC and ERC ROIs drawn from Constantinescu et al. (2016). We also performed this comparison across all voxels in the brain, and we did not find any voxels where the hexagonal grid GLM has significantly lower AIC.

Fig. 6. Model comparison between subjective value GLM and hexagonal grid GLM.

Fig. 6.

Open in a new tab

Each of the four panels show the histogram of individual AIC differences between the hexagonal grid GLM and the subjective value GLM such that a positive number indicates AIC difference favoring the SV model. The top two panels are from the ROIs from Bartra et al. (2013); the bottom two panels are from the hexagonal grid ROIs from Constantinescu et al. (2016). All four ROIs’ mean AIC difference was significantly different from 0 at p ≪. 001 with a one-sample t-test.

Despite this significant result in all four ROIs, one may wonder if the subjective value GLM is favored simply because AIC favors models with fewer parameters. Our next test avoids this issue, as we compare different n-fold grid modulation GLMs in the same ROIs. If the true signal is a hexagonal gridlike modulation, a 6-fold modulation model should account for more variance in the data compared to 4, 5, 7, or 8-fold modulation (Fig. 7). In contrast, if the true signal is subjective value, 4-fold modulation should account for the most variance, and variance explained should decline across 4–8 folds (Fig. 7; in other words, 4-fold modulation best mimics a subjective value signal, which in our task is always highest in one corner of the two-dimensional attribute space and lowest in the opposite corner). In none of the four ROIs did we find that 6-fold modulation was the best descriptor of the BOLD signal. In fact, in both the vmPFC and ventral striatum ROIs from Bartra et al. (2013), and in the vmPFC ROI from Constantinescu et al. (2016), we found that 4-fold modulation explains significantly more variance than the 6-fold modulation (Fig. 7). This difference was not significant in the ERC ROI from Constantinescu et al. (2016). Again, we also performed this comparison across all voxels in the brain, and we did not find any voxels where the 6-fold model explains significantly more variance than the 4-fold model.

Fig. 7. Comparison of n-fold grid modulation GLMs in session 1 data.

Fig. 7.

Open in a new tab

The top two panels show simulated results of n-fold grid modulation GLMs when the true signal is SV (left) or a hexagonal gridlike code (right). When the true signal is subjective value, we expect a descending staircase pattern, and when the true signal is a hexagonal grid, we expect a pyramid pattern. The middle two panels show the GLM analysis in SV ROIs from Bartra et al. (2013); the bottom two panels show them for hexagonal grid ROIs from Constantinescu et al. (2016). The error bars denote the standard errors of the mean. Paired t-test between 4-fold and 6-fold: * p < .05, ** p < .01, *** p < .001. All 20 bars are significantly greater than zero (uncorrected p < .001 for all).

As a third test, we take advantage of the fact that subjects participated in two sessions in our study and perform the exact grid angle consistency analysis proposed to be the critical test for gridlike responses in Constantinescu et al. (2016). This test fixes the grid angle based on the fit of 6-fold modulation GLM in the first session and tests for modulation at that grid angle for 4~8 fold-modulation in the second session. Similar to the analysis above, if the true signal is hexagonal modulation, we should see the strongest grid-angle consistency effect with 6-fold modulation. In contrast, if the true signal is subjective value, we should see the strongest grid-angle consistency effect with 4-fold modulation. This analysis of grid angle consistency again unambiguously shows that vmPFC activity is consistent with subjective value and does not exhibit consistent hexagonal gridlike modulation (Fig. 8). We found that the grid-angle consistency effect was significantly larger at 4-fold modulation than 6-fold modulation in both the vmPFC ROIs from Bartra et al. (2013) and from Constantinescu et al. (2016).

Fig. 8. Grid-angle consistency analysis in session 2 data.

Fig. 8.

Open in a new tab

The top two panels show simulated results of grid-angle consistency analysis when the true signal is subjective value (left) or a hexagonal gridlike code (right). When the true signal is subjective value, we expect a descending staircase pattern, and when the true signal is a hexagonal grid, we expect a pyramid pattern. The middle two panels show the grid-angle consistency analysis in subjective value ROIs from Bartra et al. (2013); the bottom two panels show them for hexagonal grid ROIs from Constantinescu et al. (2016). Paired t-test between 4-fold and 6-fold: ** p < .01. Only 4-fold models are significantly different from zero (uncorrected p < .05) in vmPFC ROIs. 4-fold, 5-fold, and 6-fold models are all significantly different from zero (uncorrected p < .05) in the ventral striatum ROI. No models are significantly different from zero in entorhinal cortex.

Some readers may wonder about the distribution of estimated 6-fold grid angles. Because the correlation between a gridlike modulation signal and a subjective value signal is maximized at certain grid angles (Supplementary Fig. 1), if the true signal is subjective value, the distribution of estimated 6-fold grid angles should be distinctly non-uniform, with more grid angles in the range that maximizes this correlation (depending on the discount rate, the maximum occurs between 34° and 44°). In contrast, if the true signal is hexagonal modulation, there is no reason to expect any non-uniformity. Observing a uniform distribution is less definitive, though, as grid angles would also be expected to be distributed randomly under the null hypothesis. Further, the lack of significant 6-fold grid angle consistency suggests that the estimated 6-fold grid angles should be interpreted cautiously. Nonetheless, for completeness, we also examine the distribution of estimated 6-fold grid angles, and consistent with an underlying signal of subjective value, find a distinctly non-uniform distribution of 6-fold grid angles with a peak near 40° in both the vmPFC and ventral striatum ROIs from Bartra et al. (2013) and in the vmPFC ROI from Constantinescu et al. (2016) (Supplementary Fig. 2).

Finally, we perform several additional analyses to demonstrate the robustness of these results under different sets of assumptions. First, our ROIs, defined based on past studies, could have missed the exact location of gridlike modulation in the current study. If we define subjective value and gridlike code ROIs based on the peaks of these analyses in the current study, rather than on previous studies, we observe exactly the same pattern of results (Supplementary Fig. 3). Second, our gridlike modulation analyses assume that in the scaling of two attribute dimensions one dollar is equivalent to one day. If we alternatively assume that the two dimensions are such that the range of amounts and delays sampled cover an equivalent scale, we again observe exactly the same pattern of results as above (Supplementary Figs. 4–5). Third, we examine grid angle consistency across pairs of runs within session 1 data, in order to address the concern that gridlike representations were present but not stable across the delay between our two sessions. We again observe exactly the same pattern of results (Supplementary Fig. 6).

Discussion

During decision-making, vmPFC activity has previously been shown to correlate with subjective value; recently, it has been argued that vmPFC represents a hexagonal gridlike code for the conceptual space formed by the attributes of the choice options, which enables choices between novel attribute combinations (Bongioanni et al., 2021). Here we directly compared these two accounts of vmPFC activity in a large fMRI dataset of a standard two-attribute intertemporal decision task involving two sessions. We found, unambiguously across three different tests, that the vmPFC signal during intertemporal decision making is better explained by a subjective value than a hexagonal grid-like signal. First, a simple model comparison between a GLM that assumed activity was correlated with subjective value and one that assumed activity was modulated in a hexagonal gridlike manner favored the subjective model in vmPFC. Second, across various n-fold grid modulation models, the 6-fold hexagonal modulation model was not the best model of vmPFC activity. Rather, a 4-fold modulation that could best mimic subjective value was the best fitting model. Thirdly, we found that the cross-session consistency of an individual’s grid angles was not the highest when assuming 6-fold hexagonal modulation. Instead, we again found that the 4-fold modulation model resulted in higher cross-session grid angle consistency, matching our simulation results for a subjective value signal. Thus, we found strong evidence that BOLD activity in vmPFC during intertemporal decision-making is correlated with subjective value and does not reflect a hexagonal gridlike code.

Though there are assumptions in any one of the analyses we performed, the strong convergence of results across analyses and robustness checks lends strength to these conclusions. For example, the direct model comparison requires a choice about how to penalize the hexagonal grid model for its greater number of parameters, the grid model requires an assumption about the relative scaling of the two attribute dimensions, and the cross-session grid angle consistency analysis assumes stability of this representation across ten weeks. However, across three different kinds of tests, for two different methods of defining ROIs and for two different assumptions about the scaling of attribute dimensions, and examining grid angle consistency both within- and across-sessions, we consistently find evidence for a subjective value signal and no significant evidence at all for a hexagonal gridlike code in vmPFC.

Our results constrain the implications of Constantinescu et al. (2016) and Bongioanni et al. (2021) by limiting the conditions under which a gridlike code is observed in the vmPFC. Our intertemporal choice task is different from the tasks used in those studies. Both of those studies involved extensive learning of a novel conceptual space by the participants; ours takes advantage of a spontaneous conceptual space created by the option attributes as they are presented in a choice task. Participants in Constantinescu et al. (2016) imagined the visual transformation of a stimulus along a certain trajectory in conceptual space; participants in Bongioanni et al. (2021) responded to novel cue combinations to obtain outcomes with novel combinations of reward attributes (probabilities, magnitudes); whereas we simply ask participants to make a choice. Nevertheless, all three tasks similarly involve a two-dimensional space where gridlike representations of the task structure might be expected and indeed have been proposed (Bellmund et al., 2018). Our findings suggest that the two-dimensional space defined by the option attributes that is available during decision-making does not necessarily provoke gridlike representations.

In humans, future studies of decision-making should examine a wider range of task designs to further test the generality of our conclusions. The potential correlation between subjective value and gridlike modulation is specific to the kind of experimental design used here, where one option is fixed and the other option is consistently better on one attribute and worse on the other attribute. This kind of design has been widely used in decision-making studies, but in other designs it is possible to completely orthogonalize value and gridlike modulations, which would allow for testing for distinct neural representations of each. For example, in intertemporal choice tasks, one could use a fixed reference option that is in the middle of the attribute space (e.g., $40 in 40 days), so that there can be feasible tradeoff options in two quadrants (i.e., more delay with more amount, or less delay with less amount). Alternatively, one could allow both options to vary in amount and delay, which might also lead the decision-maker to better represent the attribute space (analogous to starting navigation tasks from many different points in space). Finally, it will also be important to investigate decision-making tasks involving different reward commodities and attribute dimensions.

In such future studies, there are strong reasons to believe that our results here will generalize. Here it is important to consider the behavioral relevance of different representations. Behavior in many decision-making tasks, including the current intertemporal choice task, can be accounted for by assuming that decision-makers integrate the attributes of each option (e.g., amount and delay) into a single dimension of subjective value and choose the option that is highest in value. Subjective value is therefore directly linked to behavior, and this simpler representational scheme would nonetheless provide the most relevant information for decision-making, the relative ordering of different options in terms of a decision-maker’s preferences. In contrast, the metric relationships between different choice options in two-dimensional attribute space that gridlike representations would provide – distances between any two options, what options lie in between two others, and paths that connect the two options – are less clearly relevant for the kind of decision tasks we studied here. On this point, notably even the study arguing that gridlike representations underlie novel choice did not show evidence of such representations in the paradigm where a choice was actually required (Bongioanni et al., 2021). Our results suggest that during preference-based choice, activity in vmPFC instead tracks the most behaviorally relevant representation of the option space – subjective value.

More broadly, a strong possibility is that representations in vmPFC are flexible, that vmPFC represents the relevant cognitive map for the current task (Bernardi et al., 2018; Schuck et al., 2016; Wilson et al., 2014), and therefore the nature of the coding scheme in vmPFC may depend on the demands of the task at hand. Computational modeling suggests similar flexibility in entorhinal cortex representations, that grid cells arise because of the structure of navigational tasks, and that different task environments lead to different kinds of entorhinal representations (Whittington et al., 2020). A similar account could explain how the vmPFC, like the medial temporal lobe, plays an important role in such a wide variety of functions, from learning and decision-making to schematic memory and social cognition (Behrens et al., 2018; Fehr and Camerer, 2007; Gilboa and Marlatte, 2017; Grabenhorst and Rolls, 2011; Janowski et al., 2013; Lieberman et al., 2019; Powell and Redish, 2016; Roy et al., 2012).

Supplementary Material

1

Acknowledgments

This study was funded by grants from the National Cancer Institute R01 CA170297 (JK, CL) and R35 CA197461 (CL).

Footnotes

Declaration of Competing Interest

The authors declare no competing interests.

Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi: 10.1016/j.neuroimage.2021.118159.

References

  1. Bao X, Gjorgieva E, Shanahan LK, Howard JD, Kahnt T, Gottfried JA, 2019. Grid-like neural representations support olfactory navigation of a two-dimensional odor space. Neuron doi: 10.1016/j.neuron.2019.03.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bartra O, McGuire JT, Kable JW, 2013. The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. NeuroImage 76, 412–427. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Behrens TEJ, Muller TH, Whittington JCR, Mark S, Baram AB, Stachenfeld KL, Kurth-Nelson Z, 2018. What is a cognitive map? Organizing knowledge for flexible behavior. In. Neuron. doi: 10.1016/j.neuron.2018.10.002. [DOI] [PubMed] [Google Scholar]
  4. Bellmund JLS, Gärdenfors P, Moser EI, Doeller CF, 2018. Navigating cognition: Spatial codes for human thinking. In. Science (New York, N.Y.) doi: 10.1126/science.aat6766. [DOI] [PubMed] [Google Scholar]
  5. Bernardi S, Benna MK, Rigotti M, Munuera J, Fusi S, Salzman D, 2018. The geometry of abstraction in hippocampus and prefrontal cortex. BioRxiv doi: 10.1101/408633. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bongioanni A, Folloni D, Verhagen L, Sallet J, Klein-Flügge MC, Rushworth MFS, 2021. Activation and disruption of a neural mechanism for novel choice in monkeys. Nature 1–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Clithero JA, Rangel A, 2013. Informatic parcellation of the network involved in the computation of subjective value. Soc. Cognit. Affect. Neurosci doi: 10.1093/scan/nst106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Constantinescu AO, O’Reilly JX, Behrens TEJ, 2016. Organizing conceptual knowledge in humans with a gridlike code. Science doi: 10.1126/science.aaf0941. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cox KM, Kable JW, 2014. BOLD subjective value signals exhibit robust range adaptation. J. Neurosci 34 (49), 16533–16543. doi: 10.1523/JNEUROSCI.3927-14.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Doeller CF, Barry C, Burgess N, 2010. Evidence for grid cells in a human memory network. Nature doi: 10.1038/nature08704. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Fehr E, Camerer CF, 2007. Social neuroeconomics: the neural circuitry of social preferences. In. Trends Cognit. Sci doi: 10.1016/j.tics.2007.09.002. [DOI] [PubMed] [Google Scholar]
  12. Gilboa A, Marlatte H, 2017. Neurobiology of schemas and schema-mediated memory. In. Trends Cognit. Sci doi: 10.1016/j.tics.2017.04.013. [DOI] [PubMed] [Google Scholar]
  13. Grabenhorst F, Rolls ET, 2011. Value, pleasure and choice in the ventral prefrontal cortex. In. Trends Cognit. Sci doi: 10.1016/j.tics.2010.12.004. [DOI] [PubMed] [Google Scholar]
  14. Hafting T, Fyhn M, Molden S, Moser M-B, Moser EI, 2005. Microstructure of a spatial map in the entorhinal cortex. Nature doi: 10.1038/nature03721. [DOI] [PubMed] [Google Scholar]
  15. Howard JD, Gottfried JA, Tobler PN, Kahnt T, 2015. Identity-specific coding of future rewards in the human orbitofrontal cortex. Proc. Natl. Acad. Sci. USA doi: 10.1073/pnas.1503550112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Hunt LT, Kolling N, Soltani A, Woolrich MW, Rushworth MFS, Behrens TEJ, 2012. Mechanisms underlying cortical activity during value-guided choice. Nat. Neurosci doi: 10.1038/nn.3017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Jacobs J, Weidemann CT, Miller JF, Solway A, Burke JF, Wei XX, Suthana N, Sperling MR, Sharan AD, Fried I, Kahana MJ, 2013. Direct recordings of grid-like neuronal activity in human spatial navigation. Nat. Neurosci doi: 10.1038/nn.3466. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Janowski V, Camerer C, Rangel A, 2013. Empathic choice involves vmPFC value signals that are modulated by social processing implemented in IPL. Soc. Cognit. Affect. Neurosci doi: 10.1093/scan/nsr086. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kable JW, Caulfield MK, Falcone M, McConnell M, Bernardo L, Parthasarathi T, Cooper N, Ashare R, Audrain-McGovern J, Hornik R, Diefenbach P, Lee FJ, Lerman C, 2017. No effect of commercial cognitive training on brain activity, choice behavior, or cognitive performance. J. Neurosci 37 (31), 7390–7402. http://www.jneurosci.org/lookup/doi/10.1523/JNEUROSCI.2832-16.2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Kable JW, Glimcher PW, 2007. The neural correlates of subjective value druing intertemporal choice. Nat. Neurosci 10 (12), 1625–1633. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Kable JW, Glimcher PW, 2009. The neurobiology of decision: consensus and controversy. In. Neuron doi: 10.1016/j.neuron.2009.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kable JW, Glimcher PW, 2010. An “as soon as possible ” effect in human intertemporal decision making: Behavioral evidence and neural mechanisms. J. Neurophysiol doi: 10.1152/jn.00177.2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kahneman D, Tversky A, 1979. Kahneman & Tversky (1979) - prospect theory - an analysis of decision under risk. Econometrica doi: 10.2307/1914185. [DOI] [Google Scholar]
  24. Levy DJ, Glimcher PW, 2012. The root of all value: a neural common currency for choice. Curr. Opin. Neurobiol 22 (6), 1027–1038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lieberman MD, Straccia MA, Meyer ML, Du M, Tan KM, 2019. Social, self,(situational), and affective processes in medial prefrontal cortex (MPFC): causal, multivariate, and reverse inference evidence. Neurosci. Biobehav. Rev. [DOI] [PubMed] [Google Scholar]
  26. McNamee D, Rangel A, O’Doherty JP, 2013. Category-dependent and category- independent goal- value codes in human ventromedial prefrontal cortex. Nat. Neurosci 16 (4), 479–485. doi: 10.1038/nn.3337. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Nau M, Navarro Schröder T, Bellmund JLS, Doeller CF, 2018. Hexadirectional coding of visual space in human entorhinal cortex. Nat. Neurosci doi: 10.1038/s41593-017-0050-8. [DOI] [PubMed] [Google Scholar]
  28. Powell NJ, Redish AD, 2016. Representational changes of latent strategies in rat medial prefrontal cortex precede changes in behaviour. Nat. Commun doi: 10.1038/ncomms12830. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Roy M, Shohamy D, Wager TD, 2012. Ventromedial prefrontal-subcortical systems and the generation of affective meaning. In. Trends Cognit. Sci doi: 10.1016/j.tics.2012.01.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Schuck NW, Cai MB, Wilson RC, Niv Y, 2016. Human orbitofrontal cortex represents a cognitive map of state space. Neuron 91 (6), 1402–1412. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Strait CE, Blanchard TC, Hayden BY, 2014. Reward value comparison via mutual inhibition in ventromedial prefrontal cortex. Neuron doi: 10.1016/j.neuron.2014.04.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Whittington JCR, Muller TH, Mark S, Chen G, Barry C, Burgess N, Behrens TEJ, 2020. The Tolman-Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation. Cell 183 (5), 1249–1263. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Wilson RC, Takahashi YK, Schoenbaum G, Niv Y, 2014. Orbitofrontal cortex as a cognitive map of task space. Neuron doi: 10.1016/j.neuron.2013.11.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Yamada H, Louie K, Tymula A, Glimcher PW, 2018. Free choice shapes normalized value signals in medial orbitofrontal cortex. Nat. Commun doi: 10.1038/s41467-017-02614-w. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1723753-supplement-1.docx (994.1KB, docx)

RESOURCES