Neurons in Orbitofrontal Cortex Encode Economic Value

Abstract

Economic choice is the behavior observed when individuals select one of multiple options available in alternative to each other. In economic choice, there is no intrinsically “correct” answer; the choice depends on subjective preferences. This behavior is traditionally the object of economic analysis¹ and is also of primary interest in psychology². However, the underlying mental processes and neuronal mechanisms are not well understood. Theories of human and animal choice have a cornerstone in the concept of value^1–3. Consider for example a monkey offered one raisin versus one piece of apple. Behavioral evidence suggests that the animal chooses by assigning values to the two options⁴. But where and how are values represented in the brain? Here we show that, during economic choice, neurons in orbitofrontal cortex^5–18 (OFC) encode the value of offered and chosen goods. Importantly, OFC neurons encode value independently of visuospatial factors and motor responses. For example, if a monkey chooses between A and B, neurons in OFC encode the value of the two goods independently of whether A is presented on the right and B on the left, or vice versa. This trait distinguishes OFC from other brain areas where value modulates activity related to sensory or motor processes^19–25. Our results have broad implications for possible psychological models of economic choice. They suggest that economic choice is at its essence choice between goods, as opposed to choice between actions. In this scheme, neurons in OFC seem a good candidate network for value assignment underlying economic choice.

In our experiments, monkeys choose between two types of juice (A and B; A preferred) offered in different amounts. For example, in the session shown in figure 1, the monkey chooses between water (juice A) and unsweetened Kool-Aid (juice B). Offer types include 1B:2A, 1B:1A, 2B:1A, 3B:1A, 4B:1A, 6B:1A, and 10B:1A, and the “forced choices” 0B:1A and 3B:0A. Behaviorally, we observe a trade-off between juice type and juice quantity. The monkey chooses A when 1B, 2B, or 3B are available in alternative, it is roughly indifferent between the two juices when offered 4B:1A, and it chooses B when 6B or 10 B are available. We interpret this pattern of choice in terms of relative value of the two juices⁴: in this case, the value of 1A is roughly equal to the value of 4B. Fitting a sigmoid provides the better estimate V(1A)=V(4.1B). Assuming a linear value function, we obtain V(A)=4.1V(B). This equation puts different quantities of juices A and B on a same value scale. On this basis, we can compute for each trial the value of the juice chosen by the monkey. Expressing values in units of V(B), the chosen value is ≈4 when the monkey chooses 1A or 4B. When the monkey chooses 2A, the chosen value is ≈8. When the monkey chooses 6B, 10B, and 3B, the chosen value is respectively equal to 6, 10, and 3. Hence, we can make specific hypotheses regarding the neuronal representation of juice values. In different sessions and with different juices, we record different behavioral choice patterns. We then analyze each cell in relation to the choice pattern recorded in the same session.

Figure 1.

Experimental design. a. Trial structure (see Methods). b. Example of behavioral choice pattern. The plot shows the percentage of trials in which the monkeys chose juice B (y-axis) for various offer types (x-axis). A sigmoid fit provides the measure of the relative value n*=4.1.

Our recordings focused on area 13 in OFC. Figure 2 illustrates the activity of one representative neuron. The cell’s activity does not depend on whether juice A is offered on the left or on the right (figure 2b). It also does not depend on whether the monkey chooses the juice on the left or the juice on the right (i.e., makes an eye movement to the left or to the right; figure 2c). However, the cell’s activity varies with the offer type. This is consistent across the neuronal population. We recorded the activity of 931 cells and we analyze their neuronal responses in seven time windows. We test the activity of each cell in each time window with a 3-way ANOVA (factors: [position of juice A] × [movement direction] × [offer type], p<0.001). Rarely do responses depend on either the spatial configuration of the offers or the motor output (<5% neurons). In contrast, the activity of 505 (54%) neurons varies significantly depending on the offer type in at least one time window. Pooling time windows, a total of 1379 responses are significantly modulated by the offer type (Supplementary Figure S2).

Figure 2.

Activity of one neuron. a. Rasters. Each line represents one trial and each small dot represents one spike. Trials, arranged by offer type, are aligned at the offer (left) and at the juice (right). The blue highlight marks the post-juice time window. b. Activity profiles shown separately for trials in which juice A is offered on the left (red) or on the right (green). The cell activity does not depend on the spatial configuration of the visual stimulus. c. Activity profiles shown separately for trials in which the monkey chooses the juice offered on the left (red) or on the right (green). The cell activity does not depend on the direction of the eye movement. d. Top: choice pattern recorded in this session (n*=1.9). Bottom: activity of the cell (±s.e.m.) recorded in the pre-offer (light gray, control) and post-juice time window (black). Note that the response does not reflect simple physical properties of the visual stimulus, such as the number of squares displayed on the monitor. For example, offer types 1B:3A and 3B:1A, which are visually identical except for the color of the squares, elicit very different activation.

The cell shown in figure 2d has a “U-shaped” response similar to that hypothesized for a neuron encoding the chosen value. For this session V(A)=1.9V(B). Accordingly, the activity of the cell is lower when the monkey chooses 1A or 2B (in units of V(B), chosen value ≈ 2), it is higher when the monkey chooses 2A or 4B (chosen value ≈ 4), and it is highest when the monkey chooses 3A or 6B (chosen value ≈ 6). A linear regression of this response on the variable chosen value provides R²=0.86. Similar U-shaped responses are frequent in OFC, and figures 3a–c illustrate three more examples. We also find other types of responses. For example, neuronal responses often reflect the value of one of the two juices alone. Figures 3d–e show two cells whose activity co-varies with the value of A offered and the value of B offered, respectively. We label these responses as related to the offer value. Other frequently observed responses vary in a binary fashion depending on the type of juice chosen by the monkey, independently of the amount (figure 3f). We interpret these responses as related to juice taste.

Figure 3.

Activity of six neurons. For each cell, the top panel shows the choice pattern, with the relative value indicated on the top left. a–c. Responses encoding the chosen value. The response in c is negatively correlated with the chosen value (high activity for low value). d–e. Responses encoding the value of juice A offered (d) and the value of juice B offered (e). We refer to these responses as related to the offer value. d. Response encoding the juice taste. Here we separate trials in which the monkey chose juice A (diamonds) or juice B (circles). The response reflects the chosen juice type independently of the amount. Responses were recorded in the post-offer (a d e, blue), pre-juice (b, cyan), and post-juice (c f, black) time windows. For each cell, the curves in light gray show the activity in the pre-offer time window. Error bars represent the s.e.m.

While many neurons appear to encode chosen value, offer value or juice taste, the relation between their activity and these three variables could be subordinate to a correlation with other behavioral variables. For example, neurons in OFC might encode the number of squares on the monitor (or variables proportional to number, such as juice quantity, or absolute luminance of the visual stimulus). To cast a wide net, we examine the linear dependence of neuronal data on 19 possible variables (Supplementary Figure S1). For example, we analyze the variables chosen number, total number, and total value. We include in this analysis 1379 responses significantly modulated by the offer type, and we regress each response separately on each variable. Collectively, the 19 variables explain 1227 (89%) neuronal responses. However, the 19 variables are often highly correlated. To identify a few variables that best describe the neuronal population, we adapt procedures for variable selection commonly used in multi-linear regression in the presence of multi-collinearity. Both the stepwise and the best-subset methods identify variables offer value, chosen value, and taste, which explain well the large majority of responses (1085/1379=79% responses explained, mean R²=0.63). A post-hoc analysis indicates that the explanatory power of these three variables is significantly higher than that of challenging alternatives. Furthermore, data from the two monkeys analyzed separately provide statistically indistinguishable results. Finally, a bi-linear regression analysis indicates that in 890/1085 (82%) cases adding a second variable or a quadratic value term does not improve the regression significantly (Supplementary Results, p.S5–S10; Figures S4–S11). We conclude that, indeed, OFC responses encode variables offer value, chosen value, and taste.

We next turn to a specific analysis of U-shaped responses (figures 2d, 3a–c). In our experiments, relative values were generally stable within any recording session. However, the relative value of any given pair of juices could vary from day to day. For example, the relative value of apple juice versus peppermint tea varied between 1.5 and 3. This variability provides a further opportunity to test the neuronal encoding of value; specifically, U-shaped responses should reflect this variability. For this analysis, we test the entire neuronal population with the regression function a₀ +a_A (# A) +a_B (# B), where (#A) and (#B) represent the amounts of juices A and B chosen by the monkey. We define a response to be U-shaped if both a_A and a_B differ from zero (p<0.01). If U-shaped responses indeed encode the value of the chosen juice, the slope ratio k*=a_A/a_B should be, for each response, equal to the relative value (n*) measured in that session. Most importantly, the slope ratio k* obtained for different responses should co-vary with n*. To test this prediction, we compute the regression k*=b₀+b₁ n* separately for every juice pair. Averaging across juice pairs, we obtain b₀=−0.13 (±0.15) and b₁=1.05 (±0.15) (±s.e.m.), consistent with the identity k*=n*. This result demonstrates that U-shaped responses do not reflect the quantity of any particular juice ingredient (e.g., sugar). Rather, they encode the value monkeys assign to the juice they choose to consume (Supplementary Results, p.S10–S12; Figures S12–S14).

With respect to the timing of neuronal activation, the average neuronal activity peaks shortly after the offer, declines during the delay, is low before and during the eye movement, and has two secondary peaks at juice delivery (Supplementary Figure S3). To appreciate how the variables offer value, chosen value, and taste are represented in OFC over time, we analyze the activity of each cell in 50-ms, non-overlapping time bins. Figure 4 shows the number of cells encoding each of the three variables at different times. Remarkably, the time profile of different variables seems to reflect the mental processes the monkey likely undertakes during a trial. Shortly after the offer, when the monkey presumably assigns values to the two juices, neurons encoding the offer value (i.e., the value of one juice or the other) are most prevalent. Also during the delay, many neurons encode the chosen value (i.e., the value of the juice the monkey will eventually consume), even though the choice is still covert (because the go signal has not been given yet). Finally, after the monkey has indicated its choice, before and after juice delivery, many neurons encode the taste of the chosen juice.

Figure 4.

Time course. We assign each neuron to one variable of the three variables if the regression slope is significantly different from zero (p<0.01), and we include all 931 neurons in the analysis. The dotted line indicates chance level (9.31).

Conceptually, responses encoding the chosen value are particularly interesting because, in addition to being independent of the visuomotor contingencies of the task, they are also independent of the specifics of the good, namely juice type and juice amount. These responses encode economic value in a rather non-specific way. Further research is necessary to establish whether this result generalizes to other kinds of goods, such as non-comestible goods²⁶. The interpretation of offer value responses is made more cautiously because, assuming linear value functions, the value of a given amount of juice is proportional to the juice quantity.

“Value” is known to modulate the activity of neurons in several sensory and motor areas^19–25. For example, neurons in the lateral intraparietal area activate when monkeys plan a saccade towards a particular location of the visual field; their response is enhanced when the eye movement is associated with higher value²⁰. On this basis, it has been proposed that parietal neurons encoding the value of all possible courses of action form a common path for decision-making, and that their activity is actually the subject of economic theory²⁷. According to this “action-based” model, economic choice is fundamentally choice between actions. The fact that neurons in OFC encode the economic value of offered and chosen goods per se, and not as modulation of visuomotor processes, suggests an alternative “good-based” model, according to which economic choice is fundamentally choice between goods. In this view, choice is made between goods, and a suitable motor action is subsequently planned and executed. Several arguments seem to favor the good-based model. From a computational perspective, a modular design separating the mental operations of [choosing] and [moving] is more parsimonious^28,29. In addition, values processed in OFC are logically sufficient for good-based choice. The action-based model would thus imply that, during economic choice, the nervous system operates in a computationally inefficient way, while undertaking all the processes needed to choose efficiently. Finally, a vast literature links choice in various domains to OFC. For example, human patients and monkeys with OFC lesions can present eating disorders and hyperorality^6–8, abnormal risk-seeking and gambling behavior^9,10, and impulsivity, altered personality and social behavior^6,11. In contrast, parietal lesions typically result in visuospatial deficits such as hemi-neglect or Balint’s syndrome³⁰. In conclusion, together with other lines of evidence, the present results support a good-based psychological model of economic choice behavior.

Methods

Each trial begins with the monkey fixating the center of a computer monitor (figure 1a). After 1.5s, two sets of squares appear on opposite sides of the fixation point (offer). The color of the squares indicates the juice type and the number of squares indicates the juice amount. For example, a monkey offered 3 blue squares versus 1 yellow square chooses between 3 drops of peppermint tea and 1 drop of grape juice. After a randomly variable delay (1–2s), two saccade targets appear near the offers (go). The monkey indicates its choice with an eye movement, and must maintain fixation on the target for an additional 0.75s before juice delivery (juice). The trial is aborted if the monkey breaks fixation before the go. The amounts of the two offered juices (0–10 drops) vary pseudo-randomly. For a given offer type, left/right positions are counterbalanced (i.e., the monkey may be offered 1A on the left and 3B on the right, or vice versa). A variety of different pairs of juices are used in different sessions (Supplementary Methods).

We analyze cell activity in the following time windows: 0.5s pre-offer (a control time window); 0.5s post-offer; late delay (0.5–1.0s after the offer); 0.5s pre-go; reaction time (RT; from go to saccade); 0.5s pre-juice; and 0.5s post-juice. For the statistical analysis, we separate for each offer type trials in which the monkey chooses juices A and B (Supplementary Methods).

Supplementary Material

Supplementary Information

Supplementary information is linked to the online version of the paper at www.nature.com/nature.