Abstract
How do humans make choices between different types of rewards? Economists have long argued on theoretical grounds that humans typically make these choices as if the values of the options they consider have been mapped to a single common scale for comparison. Neuroimaging studies in humans have recently begun to suggest the existence of a small group of specific brain sites that appear to encode the subjective values of different types of rewards on a neural common scale, almost exactly as predicted by theory. We have conducted a meta analysis using data from thirteen different functional magnetic resonance imaging studies published in recent years and we show that the principle brain area associated with this common representation is a subregion of the ventromedial prefrontal cortex (vmPFC)/orbitofrontal cortex (OFC). The data available today suggest that this common valuation path is a core system that participates in day-to-day decision making suggesting both a neurobiological foundation for standard economic theory and a tool for measuring preferences neurobiologically. Perhaps even more exciting is the possibility that our emerging understanding of the neural mechanisms for valuation and choice may provide fundamental insights into pathological choice behaviors like addiction, obesity and gambling.
Introduction
At a neurobiological level how does a thirsty monkey choose between one and two milliliters of water? How does the human brain choose between one apple and two apples? In principle this seems fairly straightforward. If we assume that more is better under these conditions then we simply need to represent and compare quantities. But what happens in the brain when we need to choose between a large amount of water and a single apple? Or a small amount of water and two apples? The options we face in these situations are different, and our answers depend both on the reward types and the quantities of each of those rewards. Just counting will not help. What we need to do is to take into consideration many different attributes of each option (like color, size, taste, health benefits, our metabolic state, etc.), assess the value of each of the attributes, and combine all of these attributes into one coherent value representation that allows comparison with any other possible option. What we need, at least in principle, is a single common currency of valuation for comparing options of many different kinds. In as much as our choices are consistent and lawful, the brain must represent the values of many different kinds of rewards on a common scale for comparison and choice.
Over the course of the past decade there have been a wealth of studies suggesting that activity in small number of brain areas encodes reward quantities during decision-making tasks. Areas like the parietal cortex appear to encode how many milliliters of juice an action will yield to a thirsty monkey. Areas like the ventral striatum and the medial prefrontal cortex appear to encode the amount of money an option will yield. Indeed, there is now broad consensus in the neuroscience of decision-making community that reward magnitude is represented in a small number of well-identified areas. Here we conduct a meta analysis using evidence from human functional magnetic resonance imaging (fMRI) studies conducted over just the past few years that suggests that one of these reward magnitude encoding areas, the ventromedial prefrontal cortex/orbital frontal cortex (vmPFC/OFC), can be thought of as representing the value of nearly all reward-types on a common scale that predicts behaviorally observed comparison and choice. Of course, this does not mean that common currency representations occur only in this area, but available fMRI evidence clearly indicates the existence of a common currency network at least in this area.
Perhaps the first common currency representation experiment was conducted while recoding from monkey parietal cortex [1,2•] and related work has also indicated that the midbrain dopamine neurons employ a common currency for reward representation in monkeys [3]. For the purposes of this review, however, we restrict ourselves to the rapidly growing human fMRI literature on this subject so as to focus our analysis on the structural features of the human brain related to this class of representation.
The idea of a common currency representation at a purely theoretical level is, of course, hardly new. The economist Paul Samuelson [4] proved almost a century ago that any decision-maker who is internally consistent in their choices behaves exactly as if they were employing a single common scale for the representation of value. Of course, the assumption that a chooser is rational (or consistent) is not a necessary condition for a common currency representation. Since Samuelson's proof, nearly all theories of decision, from expected utility theory [5] through prospect theory [6] and even modern reinforcement learning algorithms [7] have shared the notion that in order to choose, the different attributes of each option must at some point be converged, however idiosyncratically, incompletely and imperfectly, into a single value for the actual process of comparison.
fMRI and the representation of value
Over the course of the past decade there have been a huge number of studies that have related the magnitude of monetary rewards, and the idiosyncratic values subjects place on those rewards, to brain activations in humans. In a typical study of this kind subjects either receive, or choose between, monetary rewards of different sizes during a scanning session. The study then searches for correlations between either the size of the reward or the subject's subjective valuations of reward magnitudes and the BOLD signal throughout the brain. Perhaps surprisingly, these studies have yielded a very homogenous result. Essentially all of them identify the medial pre-frontal cortex, the ventral striatum and the posterior cingulate cortex as correlated with these reward magnitudes. In addition, a subset of these studies reveal correlations in the amygdala, the insula and the posterior parietal cortex (for reviews of this literature see [8–13]).
Delgado et al. [14], for example, used a magnitude evaluation task with monetary rewards to show that activity in the ventral striatum was correlated with monetary gains and losses. Rebecca Elliot et al., at the same time, showed that ventral striatal activity correlates with the magnitude of cumulative rewards [15] and Knutson showed, again at essentially the same time, that activity in this area correlates with the anticipation of reward [16]. Subsequent studies have clearly supported these early findings; monetary reward expectation [17], monetary reward receipt [18], the expected values of rewards [19] potential monetary reward magnitude and loss magnitude [20] and discounted reward value at delays ranging from minutes to months [21] are all correlated with activity in the ventral striatum – to cite just a tiny fraction of the relevant literature.
A similar story seems to hold in the medial prefrontal cortex and to a lesser degree in the posterior cingulate cortex. Activity in these areas correlates with monetary reward magnitude [16,22], the expected values of a monetary lotteries [19], the subject-specific valuations of gains and losses [20], subject-specific discounted reward value [21] and willingness to pay for primary rewards [23]. To summarize a huge literature, activity in these areas seems extremely well correlated with how good a reward outcome will be and this is true even when the notion of ‘how good’ must incorporate subject-specific subjective evaluations like the trade-offs between how long one has to wait for a reward and how large is that reward [21,24].
However, in all of these studies only a single reward type and a single task-type was used to examine the neural representation of value. While these studies clearly identified important areas that participate in value representation and choice, they can only provide circumstantial evidence for the notion of a neural common currency that represents value across reward types. The more direct evidence for this notion that serves as the focus for this review arises from studies that search specifically for a common representation of value across different choice tasks or across different reward types measured within individual subjects.
Different choice tasks within an individual
One of the first studies to use more then one behavioral task to search for the neural representation of a single reward type, in this case monetary rewards, was a 2009 study by Glascher et al. [25]. In that experiment, subjects completed two versions of a monetarily rewarded decision-making task while in an fMRI scanner. In the first version, subjects chose between two different visual stimuli that were associated with two different probabilistic monetary rewards. They hypothesized that under these conditions the visual cues would come to be associated with monetary values and it was those stimulus-based value representations that they hoped to identify. In the other version, subjects made choices between two different motor responses in the absence of visual cues, each of which was also associated with a probabilistic monetary reward. They hypothesized that under these conditions the motor actions would come to be associated with the monetary rewards and it was the neural representation of these action-values that they hoped to identify. As can be seen in Figure 1a, they found that the activity of a subregion of the medial prefrontal cortex – a region that had been identified previously in the single-task and single-reward studies mentioned above – correlated with expected future reward in both task versions.
A closely related paper by Peters and Buchel (2009) [26] searched for the brain areas that represent the subjective values of delayed monetary rewards and the subjective values of risky monetary lotteries. Their main finding was, again, that a subregion of the medial prefrontal cortex, which they referred to quite reasonably as the OFC, tracked the subjective value of both delayed and probabilistic rewards (Figure 1b, right panel). They also found that the ventral striatum showed this same pattern of activity (Figure 1b, left panel). Levy et al. (2010) [27], in a similar vein, searched for neural representations of both risky (when the probabilities are known) and ambiguous (when the probabilities are unknown) monetary lotteries. Again they found both of those representations in the medial prefrontal cortex and the ventral striatum. Basten et al. (2010) [28] even showed that when subjects must integrate information about both monetary gains (benefit) and monetary losses (cost), activity in this same medial frontal area is correlated with the integrated difference between these two properties.
From these studies, and a host of others not described directly, it seems clear that a subregion of the vmPFC/OFC appears to encode subjective monetary value signals of almost every kind. This subregion of the vmPFC/OFC represents different kinds of monetary values and it suggests that these different kinds of monetary values may be represented on common scale, irrespective of task details. But much more compelling evidence of a common currency for reward comparison would be the demonstration that, within an individual, value representations for fundamentally different reward types arise in exactly these same areas.
Multiple reward types in the same task
FitzGerald et al. (2009) [29•] were the first to conduct such a study. They searched for value-related representations of money and consumer goods like mugs, boxes of chocolate, and universal serial bus keys. Subjects had to choose between receiving (or giving up) some amount of money and receiving (or giving up) a few of these consumer goods. As Figure 2a shows, the authors found that activation in the vmPFC/OFC (and also in the PCC, and the insula – which had a negative correlation, see Table 1) was correlated with the difference between the subjective values of the two available options. Importantly, they showed that this was true for both gains and losses. Soon afterwards, Chib et al. (2009) [30] made this argument in a more fundamental way when they explored the neural representation of three different reward types using a within-subject design. They explored the value-associated representation of money, snack foods and CalTech novelty items like hats (trinkets) in single individuals. Their design was organized into two scanning sessions. In the first, subjects chose on each trial between a certain monetary gain and a probability of winning a snack food or trinket. In the second session these same kinds of choices were made, but this time between the certain win of a fixed snack food and probability of winning a trinket or a given amount of money. Once again, they found that a subregion in the vmPFC/OFC represented the subjective values of all three reward types (Figure 2b).
Table 1.
Details regarding the studies that are described In the text and the MNI coordinates of the peak voxels for each particular value-related signal. Task(s) – the task or tasks used in the study. Reward type – the reward types used in the study. Value - the regressors used in the study. Overlap area – the brain area that showed overlap activity. x,y,z - the MNI coordinates for the peak voxel. Yellow - right vmPFC/ OFC; Blue – left vmPFC/OFC; Green – Striatum; Orange – right anterior Insula. We have transformed the values of the peak voxel in Levy and Glimcher (2011) from Talariach space to MNI using matlab code – tal2icbm.m developed and validated by Jack Lancaster at the Research Imaging Center in San Antonio, Texas. The code was taken from BrainMap.org website. BDM – Becker – DeGroot – Marschak method; WTP – willingness to pay; RPE - reward prediction error; r – right; l – left; a – anterior; p - posterior; m – medial; Corr – correlation; DV – Decision variable; PPI – Psychophysiological interaction | |||||||
---|---|---|---|---|---|---|---|
Reference | Task(s) | Reward type | Value | Overlap area | xy z | ||
Peters and Buchel (2009) | Delay discounting and Risk choices | Money | Subjective value | l_VS | –8 | 4 | –8 |
r_central OFC | 26 | 18 | –16 | ||||
l_lateral parietal | –50 | –52 | 44 | ||||
l_vmPFC | –4 | 34 | –6 | ||||
Glascher et al. (2009) | Motor and Cue choices | Money | Action and Stimulus values | l_mOFC | –6 | 24 | –21 |
r_vmPFC | 6 | 30 | –9 | ||||
r_vmPFC | 9 | 27 | –12 | ||||
r_vmPFC | 3 | 54 | –3 | ||||
r_PCC | 9 | –30 | 51 | ||||
RPE for Action and Cue | l_Amg-Hippo | –27 | –18 | –18 | |||
r_Amg-Hippo | 24 | –9 | –18 | ||||
Basten et al. (2010) | Choice task - Cost and Benefit in each option Gamble task | Money | Expected Reward - Expected Cost | l_vmPFC | –4 | 60 | –6 |
dlPFC | –22 | 18 | 44 | ||||
r_a_mPFC | 12 | 50 | 8 | ||||
Lin etal. (2011) | Money and Social | Stimulus value | r_vmPFC | 6 | 27 | –15 | |
Reward at outcome | l_vmPFC | –6 | 36 | –15 | |||
Chib et al. (2009) | BDM | Money, trinkets | WTP | l_vmPFC | –3 | 42 | –6 |
and snacks | |||||||
Kim et al. (2010) | Choice task(realized each trial) | Money and Juice | Expected outcome | r_vmPFC | 6 | 27 | –15 |
Expected outcome | r_a_Insula | 33 | 24 | 0 | |||
(negative Corr) | |||||||
FitzGerald et al. (2009) | Choice task | Monay and | Difference in Values | l_vmPFC | –15 | 30 | –6 |
(no probabilities) | Incommensurable | l_PCC | –3 | –36 | 48 | ||
Difference in Values | r_Insula | 30 | 24 | 6 | |||
(negative Corr) | l_Insula | –36 | 18 | 9 | |||
Izuma et al. (2008) | Probability task (money); Reputation task (social) | Money and Social | High value vs. base line | l_Putamen | –22 | 20 | –2 |
l_Caudate | –8 | 14 | 2 | ||||
Valentin et al. (2009) | Probability task (realized each trial) | Money and Juice | RPE | l_Caudate | –9 0 6 | ||
Smith et al. (2010) | No choices - passive view | Money and | Experienced reward | a_vmPFC | 0 | 46 | –8 |
attractiveness | Social - Monetary | r_p_vmPFC | 6 | 26 | –14 | ||
of Faces | value (DV/exchange rate) | ||||||
Plassmann et al. (2010) | BDM | Appetitive and | WTP | r_vmPFC | 6 | 30 | –12 |
aversive food | r_DLPFC | 50 | 30 | 21 | |||
items | 3 studies (above 2 and Plassmann et al. 2007) | r_vmPFC | 3 | 36 | –18 | ||
r_DLPFC | 42 | 42 | 15 | ||||
Talmi et al. (2009) | Choice task - Pain and Money in each option | Money and pain | Modulation of | l_VS | –6 | 10 | –4 |
Pain on RPE | l_VS | –6 | 2 | –8 | |||
ACC | 8 | 44 | 0 | ||||
ACC | 0 | 40 | –6 | ||||
Hippocampus | 22 | –12 | –18 | ||||
Modulation of pain on RPE (PPI analysis) | r_vmPFC | 2 | 44 | –18 | |||
Levy and Glimcher (2011) | Risky choices | Money and Food | Subjective value | l_vmPFC | –13 | 40 | –14 |
r_VS | 3 | 2 | –4 |
In line with these studies, Kim et al. (2010) [31•] examined brain activity while subjects made a forced choice between visual cues associated with positive/negative amounts of money and aversive/appetitive fluids delivered orally while in the scanner. As can be seen in Figure 2c they found that a subregion of the vmPFC/OFC tracked the expectation of receiving both monetary and fluid offers. Interestingly, they also found that the right anterior insula had a negative correlation with increasing expected reward value for both money and juice (Table 1).
Talmi et al., in yet another related study, examined the interaction between monetary rewards and physical pain [32]. Subjects in that study chose between two stimuli, each associated with either a high or low probability (75% and 25%, respectively) of money and a high or low probability of pain (thus creating a 2 × 2 stimulus design). Thus when subjects faced a possible monetary gain they had to take into consideration the ‘cost’ of receiving possible pain when making their choices. What the authors found was that the cost–benefit value signals converged in an interactive manner: Activity in the insula was correlated with the behavioral impact of the pain on their choices and this insula activation was inversely correlated with activity in the vmPFC/OFC. The greater the perceived cost of the pain, the lower the activity in the vmPFC/OFC, and this effect appeared to be modulated through the level of activity in the insular region they examined.
Izuma et al. (2008) expanded the domain of reward studies of this kind when they examined the neural representation of both social and monetary rewards [33]. In their experiment, subjects engaged in a monetary task and a social reputation task. Acquiring positive reputation and gaining monetary rewards both activated the same area in the left striatum, suggesting that monetary rewards and social rewards are represented in a similar manner in the striatum. Lin et al. (2010) also examined the interaction between monetary and social values in a probabilistic choice task [34•]. On some trials, subjects had to choose between two uncertain social rewards and on other trials between two uncertain monetary rewards. Again (Figure 3a), they found that activity in a subregion of the vmPFC/OFC correlated with both monetary and social subjective values.
Transformation of value to common currency representation
These studies all suggest that the vmPFC/OFC, and perhaps the ventral striatum, represent the values of rewards of many different, and perhaps all, kinds. But in order to demonstrate that these representations exist in a single common currency appropriate for computing the trade-offs that guide choice one must also show that the activity-level in these areas is equivalent whenever subjects report that offers of two different kinds of rewards are equally desirable. There are two papers that have done that, finding that equal behavioral value equates to equal BOLD signal in the vmPFC/OFC; evidence for a neural common currency. The BOLD signal, however, is not actually a direct measure of neural activity but rather a measure of the metabolic demand, and thus only a proxy for the actual neural activity [35]. Thus while our current understanding of fMRI strongly indicate the existence of neural activity encoding value in a common currency, the final proof that neural activity encodes value on a common scale will ultimately have to be made electrophysiologically.
The first study to provide evidence for a common currency representation in the BOLD signal was by Smith et al. (2010) [36••]. In that study, male subjects performed two tasks while being brain scanned: A forced choice task in which subjects could either win or loose money while watching female faces that ranged from very attractive to very unattractive and a second task in which subjects had to decide how much money they were willing to spend to view a female face at a given level of attractiveness. This allowed them to establish an explicit exchange rate between viewing female faces and money and then to scan face/money combinations, thus establishing a common neural representation of value for both reward types. They found that a specific subregion in the anterior parts of the vmPFC/OFC tracked the subject-specific values for each of the reward types. More importantly though, as can be seen in Figure 4a, they found a subregion in the posterior part of the vmPFC/OFC that predicted the exchange rate between money and faces, established in the second task, across subjects. This is important because their data suggest that this particular area tracks the subject-specific values of faces and money in a single neural currency.
The second study that used this strategy came from our lab [37]. We had very similar results using a different task and examined the neural representation of the value of food items and money. In that study, hungry subjects made choices between certain and risky rewards of money or foods (either chocolate M&Ms or Ritz crackers) inside the fMRI scanner. Out of the scanner we also had subjects make choices between fixed monetary offers and probabilistic lotteries over foods in order to establish the exchange rate between food and money for each subject. From this paradigm we were able to identify, as have the many previous studies mentioned above, that subregions of the vmPFC/OFC and the striatum tracked the subjective values for both money and food. We then asked whether the activation levels of these subregions that tracked the values of both food and money could be used to predict the exchange rate for food and money identified behaviorally outside of the scanner. As can be seen in Figure 4b our data suggested that in the vmPFC/OFC region that represented both reward types did predict the exchange rate between money and food across our subject pool.
From these studies we can conclude a few things. First, there is compelling evidence that a subregion of the vmPFC/OFC represents the subject-specific subjective value of multiple reward types, across various tasks, and in a common neural currency appropriate for guiding choice. Second, there is some evidence – although this is much less certain – that suggests the insula may also represent subjective value on a common currency but in a negative manner and that the striatum may also represent subjective value.
Conclusions about the striatal representation are complicated by two factors. First, single unit studies in the monkey [38,39] report a robust subjective value signal in the dorsal striatum, which has rarely been observed in human fMRI. This disparity may be due to the fact that the value encoding neurons in the dorsal striatum are diffusely distributed [39,40] and thus difficult to image using fMRI. Using fMRI, however, value-related signals have been very widely observed in the ventral striatum (see references in introduction). These ventral signals are more difficult to interpret because they are often associated with learning value representations rather than value representations per se [41].
Localizing value representations across studies
When we last reviewed studies of value representation and choice [9] there was a tremendous amount of evidence suggesting that the vmPFC/OFC region played a critical role. Literally dozens of studies available at that time pointed toward this area as critical. Since that time, not only have many other single-reward studies continued to point toward that area as critical, but a host of fMRI studies have now converged on the vmPFC/OFC as the cite of a common neural currency for value representations. With all of this apparent convergence it seems important to ask whether it is really the same area that is active in these many studies by many different labs. Put another way, how strong is the evidence for an anatomically localized subregion in the human frontal cortex that tracks subjective values on a common currency for all previously studied reward-types?
In order to answer this question we conducted a novel meta-analysis for this report. Each of the 13 principle studies described in Table 1 specified the coordinates of the voxel that was most active (peak voxel) for the value-related signals they measured (mainly based on conjunction analyses between reward types or tasks). We marked these coordinates on a single brain template (using Montreal Neurological Institute coordinates). As can be seen very clearly in Figure 5 and in Table 1, the coordinates describing the peak voxels are in the vmPFC/OFC and in nearly all of these studies are strikingly similar.
Having completed this visualization, we conducted qualitative analyses to begin to identify the boundaries of this putative common-currency area. To do that, we calculated the anatomical range of these peak voxel locations and found it to be surprisingly small. For the right hemisphere it is 0–9, 26–45, ( 3)–( 18) for x,y,z MNI coordinates, respectively. For the left hemisphere it is almost a mirror image of the right hemisphere: ( 15)–0, 24–60, ( 6)–( 21) for x,y,z MNI coordinates, respectively. An additional qualitative analysis we conducted was to calculate a weighted average of each of the three axes of the peak voxels using all the studies described in Table 1, which had activations in the vmPFC/OFC subregion. Note that each peak voxel was weighted-in even if in a given study there were more the one peak voxel. We found that for the right hemisphere it is: x = 4.27; y = 35.18; z = –11.82 and for the left hemisphere it is: x = –7.29; y = 38; z = –10.57.
From these data we think that a single conclusion seems at this point relatively straightforward. There is indeed a small subregion in the vmPFC/OFC that tracks subjective value on a common currency appropriate for guiding choices between different kinds of rewards. Indeed, these data seem to suggest that the average peak voxel we have identified here can be used as a basis for constructing an unbiased ROI for further studies of reward and valuation. Because there are now ample data demonstrating that areas in the vmPFC/OFC correlate with value signals, it now seems appropriate to conclude that research can begin to advance from using whole brain analyses of fMRI data to a more focused approach reminiscent of the strategy used in electrophysiological studies. This could lead to more concrete and testable predictions using hypothesis testing, rather than the relying on whole brain analyses aimed only at the cerebral localization of value. The data suggest, in essence, that fMRI studies of value have now advanced beyond the point of whole brain analyses driven only toward cerebral localization and to a point where the high-resolution physiology of valuation can become a tractable goal.
Conclusions
Quite a few studies have now demonstrated that a sub-region of the vmPFC/OFC centered around MNI coordinates in the left and right hemispheres represent subject-specific reward value in a common neural currency, the expected subjective value of Neuroeconomic theory [4,5]. This remarkably small area in both right and left vmPFC/OFC that is activated in a way that parametrically correlates with the subjective values subjects attribute to nearly every kind of reward that has ever been studied in the scanner. The data indicate that when two disparate kinds of rewards are equally desirable to a subject, then activity in this area will be of equal magnitude for these two rewards in that individual. This is strong evidence supporting the claim that a subregion in the vmPFC/OFC tracks subjective value in a single common currency of the kind first described in the abstract by economic theory hundreds of years ago. Using the insights from the current Meta analysis combined with additional data from many other studies we have generated a diagram that is a suggested possible schema for understanding the decision-making networks of the human brain (Figure 6).
It is important to note, however, that there is no evidence to support the claim that the neural common currency of value arises only in this subregion of the vmPFC/OFC. Any common currency observed in the brain must reflect the activation of multiple brain areas. It is almost certainly the case that other local and network activations lie beneath the resolution of the techniques used in these studies. Indeed, the evidence reviewed here suggests that portions of the striatum and perhaps the insula also participate in this process.
Before concluding, however, two potential caveats need to be considered. First, it is important to note that all studies, which have examined multiple reward-types have included monetary rewards. Thus, it might be the case that the vmPFC/OFC region translates all reward types into monetary equivalents and that in the complete absence of monetary tasks other brain circuits serve a common currency encoding function. While that possibility does not diminish the importance of this common area in cultures organized around financial exchange, it without a doubt raises the need for additional studies in this area.
A second point that needs to be considered is that the vmPFC/OFC has long been associated with functions other than decision-related valuation [42–44]. Factors ranging from emotion [45–48], to social behavior [43], learning and memory [49–52] through mental disorders such as depression [53,54], posttraumatic stress disorder [55], obsessive-compulsive disorder [56–58], and psycho-pathy [59,60] have all been identified in this brain region. This general area has also been associated with theory of mind [61–63] and with the default network [64]. Given this host of functional associations how should one interpret the wealth of data linking a subregion in this area to a function as specific as encoding a common neural currency for decision-making? Of course it is the case that the vmPFC/OFC is a fairly large area and not all of these functions will be mapped to the precise subregion we identified but there is enough overlap that this problem cannot be overlooked. And thus the observed overlap of these functions raises the fundamental question of whether we can consolidate all these functions into a unified theory of what this brain area is doing.
One possibility is that this brain area is associated with many functions in different contexts and states. This possibility, although appealing, does not describe a fundamental and parsimonious principal of how the brain works. It is just restating the problem in different words. One might also hypothesize that a unifying feature of this area might be the notion that this area is representing some kind of value-related signal in each of these contexts. Presumably, what drives and directs much neural activity and subsequent behavior is value maximization in some form and it may be that this is one of several common threads relating the many findings in this brain region.
To resolve these issues more detailed anatomical measurements will be required that can map subregions to specific loci in the brain, ideally at a within-subject level. Given that higher anatomical resolution causal studies will be required to relate activation in these brain areas to specific functions. But at this stage any reconciliation would be purely speculative. The BOLD signal in a subregion of the vmPFC/OFC clearly represents the values of choice objects on a single common neural scale appropriate for guiding choice behavior. What that means for a larger functional assessment of the vmPFC/OFC area remains to be determined.
Neuroeconomic and decision making studies in the last decade have revealed some basic notions on how we make choices and how value is represented in the brain. Moreover, there have been great advancements in our understanding of how we learn and store new values in the brain and how they influence our expectations and future behavior. In this exciting time scientists from many fields are aiming to develop a unified theory of value and choice. This fast growing area of inquiry will help us not just understand some of the basic principles of how the brain works, but should also help us understand and treat pathologies of choice such as addiction, pathological gambling and obesity.
References and recommended reading
Papers of particular interest, published within the period of review, have been highlighted as:
• of special interest
•• of outstanding interest
- 1.Deaner RO, Khera AV, Platt ML. Monkeys pay per view: adaptive valuation of social images by rhesus macaques. Curr Biol. 2005;15:543–548. doi: 10.1016/j.cub.2005.01.044. [DOI] [PubMed] [Google Scholar]
- 2•.Klein JT, Deaner RO, Platt ML. Neural correlates of social target value in macaque parietal cortex. Curr Biol. 2008;18:419–424. doi: 10.1016/j.cub.2008.02.047. [The authors demonstrate, for the first time, common currency representations of value. They showed that single unit activity in parietal area LIP reflects the aggregated subjective value of expected social rewards, threats, and fluid rewards.] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Matsumoto M, Hikosaka O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature. 2009;459:837–841. doi: 10.1038/nature08028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Samuelson PA. Foundations of Economic Analysis. Harvard University Press; Cambridge: 1947. [Google Scholar]
- 5.Von Neumann J, Morgenstern O. Theory of Games and Economic Behavior. Princeton Univ. Press; 1944. [Google Scholar]
- 6.Kahneman D, Tversky A. Prospect theory – analysis of decision under risk. Econometrica. 1979;47:263–291. [Google Scholar]
- 7.Sutton RS, Barto AG. Reinforcement Learning: An Introduction. MIT Press; Cambridge, MA: 1998. [Google Scholar]
- 8.Grabenhorst F, Rolls ET. Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn Sci. 2011;15:56–67. doi: 10.1016/j.tics.2010.12.004. [DOI] [PubMed] [Google Scholar]
- 9.Kable JW, Glimcher PW. The neurobiology of decision: consensus and controversy. Neuron. 2009;63:733–745. doi: 10.1016/j.neuron.2009.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Padoa-Schioppa C. Neurobiology of economic choice: a good-based model. Annu Rev Neurosci. 2011;34:333–359. doi: 10.1146/annurev-neuro-061010-113648. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Platt ML, Huettel SA. Risky business: the neuroeconomics of decision making under uncertainty. Nat Neurosci. 2008;11:398–403. doi: 10.1038/nn2062. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Rushworth MF. Intention, choice, and the medial frontal cortex. Ann N Y Acad Sci. 2008;1124:181–207. doi: 10.1196/annals.1440.014. [DOI] [PubMed] [Google Scholar]
- 13.Wallis JD. Cross-species studies of orbitofrontal cortex and value-based decision-making. Nat Neurosci. 2011;15:13–19. doi: 10.1038/nn.2956. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol. 2000;84:3072–3077. doi: 10.1152/jn.2000.84.6.3072. [DOI] [PubMed] [Google Scholar]
- 15.Elliott R, Friston KJ, Dolan RJ. Dissociable neural responses in human reward systems. J Neurosci. 2000;20:6159–6165. doi: 10.1523/JNEUROSCI.20-16-06159.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Knutson B, Adams CM, Fong GW, Hommer D. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci. 2001;21:RC159. doi: 10.1523/JNEUROSCI.21-16-j0002.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron. 2001;30:619–639. doi: 10.1016/s0896-6273(01)00303-8. [DOI] [PubMed] [Google Scholar]
- 18.Elliott R, Newman JL, Longe OA, Deakin JF. Differential response patterns in the striatum and orbitofrontal cortex to financial reward in humans: a parametric functional magnetic resonance imaging study. J Neurosci. 2003;23:303–307. doi: 10.1523/JNEUROSCI.23-01-00303.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Knutson B, Taylor J, Kaufman M, Peterson R, Glover G. Distributed neural representation of expected value. J Neurosci. 2005;25:4806–4812. doi: 10.1523/JNEUROSCI.0642-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Tom SM, Fox CR, Trepel C, Poldrack RA. The neural basis of loss aversion in decision-making under risk. Science. 2007;315:515–518. doi: 10.1126/science.1134239. [DOI] [PubMed] [Google Scholar]
- 21.Kable JW, Glimcher PW. The neural correlates of subjective value during intertemporal choice. Nat Neurosci. 2007;10:1625–1633. doi: 10.1038/nn2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Knutson B, Fong GW, Bennett SM, Adams CM, Hommer D. A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI. Neuroimage. 2003;18:263–272. doi: 10.1016/s1053-8119(02)00057-5. [DOI] [PubMed] [Google Scholar]
- 23.Plassmann H, O'Doherty J, Rangel A. Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J Neurosci. 2007;27:9984–9988. doi: 10.1523/JNEUROSCI.2131-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.McClure SM, Laibson DI, Loewenstein G, Cohen JD. Separate neural systems value immediate and delayed monetary rewards. Science. 2004;306:503–507. doi: 10.1126/science.1100907. [DOI] [PubMed] [Google Scholar]
- 25.Glascher J, Hampton AN, O'Doherty JP. Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex. 2009;19:483–495. doi: 10.1093/cercor/bhn098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Peters J, Buchel C. Overlapping and distinct neural systems code for subjective value during intertemporal and risky decision making. J Neurosci. 2009;29:15727–15734. doi: 10.1523/JNEUROSCI.3489-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Levy I, Snell J, Nelson AJ, Rustichini A, Glimcher PW. Neural representation of subjective value under risk and ambiguity. J Neurophysiol. 2010;103:1036–1047. doi: 10.1152/jn.00853.2009. [DOI] [PubMed] [Google Scholar]
- 28.Basten U, Biele G, Heekeren HR, Fiebach CJ. How the brain integrates costs and benefits during decision making. Proc Natl Acad Sci USA. 2010;107:21767–21772. doi: 10.1073/pnas.0908104107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29•.FitzGerald TH, Seymour B, Dolan RJ. The role of human orbitofrontal cortex in value comparison for incommensurable objects. J Neurosci. 2009;29:8388–8395. doi: 10.1523/JNEUROSCI.0717-09.2009. [The authors showed that activation in the vmPFC/OFC was correlated with the difference between the subjective values of the two available goods. The goods were money and consumer goods like mugs, boxes of chocolate, and universal serial bus keys. Importantly, they showed that this was true for both gains and losses.] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Chib VS, Rangel A, Shimojo S, O'Doherty JP. Evidence for a common representation of decision values for dissimilar goods in human ventromedial prefrontal cortex. J Neurosci. 2009;29:12315–12320. doi: 10.1523/JNEUROSCI.2575-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31•.Kim H, Shimojo S, O'Doherty JP. Overlapping responses for the expectation of juice and money rewards in human ventromedial prefrontal cortex. Cereb Cortex. 2011;21:769–776. doi: 10.1093/cercor/bhq145. [The authors found that a subregion of the vmPFC/OFC tracked the expectation of receiving both monetary and fluid offers. Importantly, subjects made a forced choice between visual cues associated with positive/negative amounts of money and aversive/appetitive fluids delivered orally while in the scanner.] [DOI] [PubMed] [Google Scholar]
- 32.Talmi D, Dayan P, Kiebel SJ, Frith CD, Dolan RJ. How humans integrate the prospects of pain and reward during choice. J Neurosci. 2009;29:14617–14626. doi: 10.1523/JNEUROSCI.2026-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Izuma K, Saito DN, Sadato N. Processing of social and monetary rewards in the human striatum. Neuron. 2008;58:284–294. doi: 10.1016/j.neuron.2008.03.020. [DOI] [PubMed] [Google Scholar]
- 34•.Lin A, Adolphs R, Rangel A. Social and monetary reward learning engage overlapping neural substrates. Soc Cogn Affect Neurosci. 2012;7:274–281. doi: 10.1093/scan/nsr006. [The authors examined the interaction between monetary and social values in a probabilistic choice task. They found that activity in a subregion of the vmPFC/OFC correlated with both monetary and social subjective values.] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. [Functional magnetic resonance imaging on World Wide Web URL: http://www.loc.gov/catdir/toc/ecip0413/2004000500.html..]
- 36••.Smith DV, Hayden BY, Truong TK, Song AW, Platt ML, Huettel SA. Distinct value signals in anterior and posterior ventromedial prefrontal cortex. J Neurosci. 2010;30:2490–2495. doi: 10.1523/JNEUROSCI.3319-09.2010. [The first human study to provide evidence for a common currency representation in the brain. The authors established for each subject an explicit exchange rate between the value of viewing female faces and money and found that the activity in the posterior parts of the vmPFC/OFC correlated across subjects with the subject specific exchange rate.] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Levy DJ, Glimcher PW. Comparing apples and oranges: using reward-specific and reward-general subjective value representation in the brain. J Neurosci. 2011;31:14693–14707. doi: 10.1523/JNEUROSCI.2218-11.2011. [The authors identified for each subject outside the scanner the exchange rate between money and food. They then showed that activity in a subregion of the vmPFC/OFC correlated across subjects with this subject specific exchange rate.] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Samejima K, Ueda Y, Doya K, Kimura M. Representation of action-specific reward values in the striatum. Science. 2005;310:1337–1340. doi: 10.1126/science.1115270. [DOI] [PubMed] [Google Scholar]
- 39.Lau B, Glimcher PW. Value representations in the primate striatum during matching behavior. Neuron. 2008;58:451–463. doi: 10.1016/j.neuron.2008.02.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Kawagoe R, Takikawa Y, Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci. 1998;1:411–416. doi: 10.1038/1625. [DOI] [PubMed] [Google Scholar]
- 41.O'Doherty JP. Reward predictions and computations. In: Gottfried J, editor. Neurobiology of Sensation and Reward. CRC Press; 2011. [Google Scholar]
- 42.Schoenbaum G, Takahashi Y, Liu TL, McDannald MA. Does the orbitofrontal cortex signal value? Ann N Y Acad Sci. 2011;1239:87–99. doi: 10.1111/j.1749-6632.2011.06210.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Viskontas IV, Possin KL, Miller BL. Symptoms of frontotemporal dementia provide insights into orbitofrontal cortex function and social behavior. Ann N Y Acad Sci. 2007;1121:528–545. doi: 10.1196/annals.1401.025. [DOI] [PubMed] [Google Scholar]
- 44.Zald DH, Andreotti C. Neuropsychological assessment of the orbital and ventromedial prefrontal cortex. Neuropsychologia. 2011;48:3377–3391. doi: 10.1016/j.neuropsychologia.2010.08.012. [DOI] [PubMed] [Google Scholar]
- 45.Bechara A. The role of emotion in decision-making: evidence from neurological patients with orbitofrontal damage. Brain Cogn. 2004;55:30–40. doi: 10.1016/j.bandc.2003.04.001. [DOI] [PubMed] [Google Scholar]
- 46.Diekhof EK, Geier K, Falkai P, Gruber O. Fear is only as deep as the mind allows: a coordinate-based meta-analysis of neuroimaging studies on the regulation of negative affect. Neuroimage. 2011;58:275–285. doi: 10.1016/j.neuroimage.2011.05.073. [DOI] [PubMed] [Google Scholar]
- 47.Mitchell DG. The nexus between decision making and emotion regulation: a review of convergent neurocognitive substrates. Behav Brain Res. 2011;217:215–231. doi: 10.1016/j.bbr.2010.10.030. [DOI] [PubMed] [Google Scholar]
- 48.Sabatinelli D, Fortune EE, Li Q, Siddiqui A, Krafft C, Oliver WT, Beck S, Jeffries J. Emotional perception: meta-analyses of face and natural scene processing. Neuroimage. 2011;54:2524–2533. doi: 10.1016/j.neuroimage.2010.10.011. [DOI] [PubMed] [Google Scholar]
- 49.Corcoran KA, Quirk GJ. Recalling safety: cooperative functions of the ventromedial prefrontal cortex and the hippocampus in extinction. CNS Spectr. 2007;12:200–206. doi: 10.1017/s1092852900020915. [DOI] [PubMed] [Google Scholar]
- 50.Gilboa A. Autobiographical and episodic memory – one and the same? Evidence from prefrontal activation in neuroimaging studies. Neuropsychologia. 2004;42:1336–1349. doi: 10.1016/j.neuropsychologia.2004.02.014. [DOI] [PubMed] [Google Scholar]
- 51.Nieuwenhuis IL, Takashima A. The role of the ventromedial prefrontal cortex in memory consolidation. Behav Brain Res. 2011;218:325–334. doi: 10.1016/j.bbr.2010.12.009. [DOI] [PubMed] [Google Scholar]
- 52.Petrides M. The orbitofrontal cortex: novelty, deviation from expectation, and memory. Ann N Y Acad Sci. 2007;1121:33–53. doi: 10.1196/annals.1401.035. [DOI] [PubMed] [Google Scholar]
- 53.Lorenzetti V, Allen NB, Fornito A, Yucel M. Structural brain abnormalities in major depressive disorder: a selective review of recent MRI studies. J Affect Disord. 2009;117:1–17. doi: 10.1016/j.jad.2008.11.021. [DOI] [PubMed] [Google Scholar]
- 54.Pizzagalli DA. Frontocingulate dysfunction in depression: toward biomarkers of treatment response. Neuropsychopharmacology. 2011;36:183–206. doi: 10.1038/npp.2010.166. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Koenigs M, Grafman J. Posttraumatic stress disorder: the role of medial prefrontal cortex and amygdala. Neuroscientist. 2009;15:540–548. doi: 10.1177/1073858409333072. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Zald DH, Kim SW. Anatomy and function of the orbital frontal cortex, II: function and relevance to obsessive-compulsive disorder. J Neuropsychiatry Clin Neurosci. 1996;8:249–261. doi: 10.1176/jnp.8.3.249. [DOI] [PubMed] [Google Scholar]
- 57.Del Casale A, Kotzalidis GD, Rapinesi C, Serata D, Ambrosi E, Simonetti A, Pompili M, Ferracuti S, Tatarelli R, Girardi P. Functional neuroimaging in obsessive-compulsive disorder. Neuropsychobiology. 2011;64:61–85. doi: 10.1159/000325223. [DOI] [PubMed] [Google Scholar]
- 58.Zald DH, Kim SW. Anatomy and function of the orbital frontal cortex. I: Anatomy, neurocircuitry; and obsessive-compulsive disorder. J Neuropsychiatry Clin Neurosci. 1996;8:125–138. doi: 10.1176/jnp.8.2.125. [DOI] [PubMed] [Google Scholar]
- 59.Blair RJ. Psychopathy, frustration, and reactive aggression: the role of ventromedial prefrontal cortex. Br J Psychol. 2010;101:383–399. doi: 10.1348/000712609X418480. [DOI] [PubMed] [Google Scholar]
- 60.Blair RJ. The roles of orbital frontal cortex in the modulation of antisocial behavior. Brain Cogn. 2004;55:198–208. doi: 10.1016/S0278-2626(03)00276-8. [DOI] [PubMed] [Google Scholar]
- 61.Sebastian CL, Fontaine NM, Bird G, Blakemore SJ, Brito SA, McCrory EJ, Viding E. Neural processing associated with cognitive and affective Theory of Mind in adolescents and adults. Soc Cogn Affect Neurosci. 2012;7:53–63. doi: 10.1093/scan/nsr023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Abu-Akel A, Shamay-Tsoory S. Neuroanatomical and neurochemical bases of theory of mind. Neuropsychologia. 2011;49:2971–2984. doi: 10.1016/j.neuropsychologia.2011.07.012. [DOI] [PubMed] [Google Scholar]
- 63.Lewis PA, Rezaie R, Brown R, Roberts N, Dunbar RI. Ventromedial prefrontal volume predicts understanding of others and social network size. Neuroimage. 2011;57:1624–1629. doi: 10.1016/j.neuroimage.2011.05.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Buckner RL, Andrews-Hanna JR, Schacter DL. The brain's default network: anatomy, function, and relevance to disease. Ann N Y Acad Sci. 2008;1124:1–38. doi: 10.1196/annals.1440.011. [DOI] [PubMed] [Google Scholar]