Specializations for reward-guided decision making in the primate ventral prefrontal cortex

Abstract

The estimated values of choices, and therefore decision making based on those values, is influenced by both the chance that the chosen items or goods can be obtained (availability) and their current worth (desirability) as well as the ability to link the estimated values to choices (a process sometimes called credit assignment). In primates, the prefrontal cortex (PFC) has been thought to contribute to each of these processes; however, causal relationships between particular subdivisions of the PFC and specific functions have been difficult to establish. Recent lesion-based research studies have defined the roles of two different parts of the primate PFC — the orbitofrontal cortex (OFC) and the ventral lateral frontal cortex (VLFC) — and their subdivisions in evaluating each of these factors and in mediating credit assignment during reward-based decision making.

Introduction

The primate PFC has several subdivisions and it has long been assumed that each plays a distinct role in guiding behavior. It has proven difficult, however, to determine the causal contributions of each PFC region to well defined and experimentally verified functional specializations¹. For instance, unlike the visual cortex — in which sensory information is processed in discrete modules at the physiological level — the input–output functions of the PFC have not been identified and are hotly debated^2–4. However, several recent findings point to functional specializations within the ventral sector of the PFC in macaques — a region that is comprised of the orbitofrontal cortex (OFC) and the adjacent ventral lateral frontal cortex (VLFC) — and have implications for understanding mechanisms of reward-guided learning and decision making in humans and nonhuman primates.

The OFC has been implicated in inhibitory control (also known as response inhibition) ⁵, flexible stimulus-reward learning ⁶, and value-based decision making ⁷. Additional roles for the OFC in emotion⁸, credit assignment ⁹ and in representing a cognitive map of task space¹⁰ have also been proposed. Recent reviews^11–13 have discussed the evidence surrounding several of these ideas, and so we will not repeat this discussion here. In brief, the consensus of these articles was that the idea that the OFC is necessary for inhibitory control is no longer tenable ^11–13 (although see below for additional discussion). Likewise, it is clear that neither flexible stimulus-reward learning nor emotion captures the core function of the OFC¹². Instead, evidence supports a role for the OFC in value-based decision making, especially in circumstances in which value changes over time and its representation must be updated¹⁴. Changes in the subjective value of an outcome or reward could occur for several reasons; for instance, in the context of foraging, a decrease in the value of foodstuffs follows consumption of a single resource to satiety (a mechanism that supports dietary diversity) and a change in the value of the locations of such foodstuffs accompanies seasonal variation in the availability of resources¹⁵.

Exactly how the role of the OFC in value-based decision making is instantiated is subject to debate. One proposal is that the OFC contributes by representing the value of the goods on which a decision is to be based in a common currency⁷. Another theory argues that the OFC is essential for credit assignment, the ability to link a particular outcome (or goods) with the choice that produced it. If one has knowledge of the contingency between a choice and an outcome, then one can actively work to achieve more or less of that outcome, depending on the circumstances⁹. The idea that the OFC provides a cognitive map of state space¹⁰ could also explain its role in value-based decision making; according to this view, the OFC represents previous stimuli, actions and other sensory features that occur in association with outcomes in a multidimensional array and thus supports good decision making. However, one of the most well supported theories emphasizes a role for the OFC in value-updating, suggesting that the OFC links object (or choice) representations with the current subjective value of the outcomes of those choices, as required for adaptive decision making. Recent studies have compared the contributions of the OFC to value updating with those of the adjacent VLFC ¹⁶. Both regions have been proposed to contribute to updating valuations, but their precise roles have been unclear.

Although neurophysiological studies in monkeys have demonstrated that OFC and VLFC neurons dynamically encode many aspects of anticipated rewards—including their magnitude, probability and sensory features^17–25 — and could, in theory, provide the basis for economic choices⁷, these physiological recording methods cannot determine whether the OFC or VLFC make a causal contribution to a behavior. Put another way, we know a lot about what cells in the OFC and VLFC encode and the circumstances in which these regions show activations in brain-imaging experiments, but not whether these signals are necessary for a particular behavior. The importance of the distinction between physiological correlates and causal contributions is exemplified by recent findings on the function of the parietal cortex, in which a region that had been shown to encode signals guiding perceptual decisions was shown to be unnecessary for those decisions²⁶. To determine causality, specific manipulations designed to increase or decrease the activity of neurons, or removal of neurons (i.e., lesions) in the regions in question are required.

In this Review, we describe how recent findings in nonhuman primates have clarified the roles of the OFC and VLFC in learning and decision making and discuss the implications for understanding human PFC function.We consider three related topics: the contrasting specializations of the OFC and the VLFC, the existence of functionally specialized regions within the OFC and the nature of amygdala–OFC interactions, which are crucial to many of its functions in decision making. We emphasize findings from macaques because their PFC is more like the human PFC than that of any other widely available animal model^27,28; however, we also draw on rodent research where it allows additional insight. In addition, we emphasize the results of studies that have employed manipulations designed to assess causality, such as permanent or reversible lesions.

OFC and VLFC anatomy in brief

The OFC, as its name indicates, occupies much of the cortical surface overlying the orbits (FIG. 1). On the basis of cytoarchitecture, combined with information derived from multiple histochemical and immunochemical stains, the macaque OFC has been divided into areas 11, 13 and 14, along with subdivisions of these areas ^29,30. The VLFC, also known as the inferior convexity of the frontal lobe, has been divided into areas 12, 45²⁹ and various subdivisions. Somewhat confusingly, the orbital surface includes a small part of the VLFC and the OFC extends onto the medial aspect of the hemisphere. Notably, the parcellations described here are consistent with recent work based on resting state fMRI, which has identified potentially homologous regions in macaques and humans based on their distinct patterns of correlated activity, or ‘connectivity fingerprints^31,32.

Figure 1:

a| Medial (left) and lateral (right) views of the macaque brain showing the locations of the orbitofrontal cortex (OFC), ventrolateral frontal cortex (VLFC), amygdala, mediodorsal thalamus and indicating selected connections between these regions. Dashed lines indicate structures buried deep in the brain. The OFC extends from the fundus of the lateral orbital sulcus to the rostral sulcus on the medial aspect of the hemisphere. The VLFC extends from just below the principal sulcus, laterally, to the fundus of the lateral orbital sulcus, which forms its shared boundary with the OFC. Caudally, the VLFC is bounded by the fundus of the inferior limb of the arcuate sulcus. Rostrally, both the OFC and the VLFC extend to their boundary with the frontal pole cortex^30,49. b| Medial (left) and ventral (right) views of the frontal lobe of the macaque brain showing approximate locations of the medial OFC, the lateral OFC and the VLFC. Numerals correspond to cytoarchitectonic areas of Walker²⁹. The lateral OFC can be subdivided into anterior and posterior zones, corresponding to cytoarchitectonic areas 11 and 13, respectively. Arrows indicate direct anatomical connections from the agranular orbital and insular cortex to the medial and lateral OFC.

Here we provide a selected survey of the main anatomical connections of these ventral prefrontal areas gleaned from standard anterograde and retrograde tract-tracing studies in macaques. These studies have shown that both the OFC and the VLFC have reciprocal connections with the amygdala, the mediodorsal nucleus of the thalamus (MD) and visual and multimodal regions of the temporal cortex. Specifically, the OFC and the VLFC receive projections from the lateral and basal nuclei of the amygdala^33–37, which, together with the accessory basal nucleus, comprise the basolateral amygdala. These projections are reciprocated^34,37. The OFC and the VLFC also project to the ventral striatum^38,39, where there are focal regions of termination of fibers as well as more diffuse projection fields. The overlap of focal and diffuse projection zones that arise from different frontal cortical regions (such as the OFC and the dorsolateral PFC) may permit functional interaction of otherwise separate cortical-striatal-pallidal-thalamic-cortical loops at the level of the striatum⁴⁰.

The MD, and especially its magnocellular part (MDmc), has reciprocal connections with both the OFC and the VLFC^41–43. The MDmc is unusual in being one of the few thalamic nuclei that has both ipsilateral and contralateral connections with the cortex. The contralateral projections are more pronounced for the corticothalamic projections relative to the reverse⁴⁴. In addition, some neurons in the basolateral amygdala innervate neurons in the MDmc that, in turn, project to the OFC⁴⁵. The MDmc also receives projections from the ventral pallidum, rostral globus pallidus, entorhinal cortex and perirhinal cortex (PRh)⁴³. Notably, the striatal regions giving rise to the striatal-pallidal-MD loops are the same ones that receive projections (which are thought to be focal) from the OFC and the VLFC⁴⁶. The entorhinal cortex and the PRh receive inputs from widespread regions of the sensory cortex, meaning that highly processed, multimodal information can directly influence the MDmc and thereby indirectly influence the OFC and VLFC.

Primates have a highly developed visual system and rely heavily on this sensory modality for many behaviours, including the detection and identification of resources and predators, social interactions and route selection. Visual inputs to the OFC and VLFC arrive from both the inferior temporal cortex (many from area TE, a cytoarchitectonic area identified by von Bonin and Bailey)⁴⁷, and the adjacent multimodal PRh. The VLFC receives inputs predominantly from area TE, whereas the OFC gets visual information predominantly from PRh^{35,36,48–51}. Recent findings from studies that have examined cortical coupling (i.e., regions with correlated activity as discerned from resting-state fMRI) ^31,32,52 and connectional anatomy^35,53 have confirmed these conclusions.

In addition to visual inputs, the OFC (but not the VLFC) receives prominent connections from posterior agranular orbital and insular areas, which collectively represent olfactory, gustatory and visceral sensory information⁵⁴ By contrast, the VLFC (but not the OFC) receives prominent inputs both from visual area TE and from auditory areas on the lower bank of the lateral fissure and the adjacent superior temporal gyrus^35,55. Although these differences in connectional anatomy may seem trivial, they are likely to enable the different functions of the OFC and VLFC, as we shall see below.

Specializations of the OFC and VLFC

Because outcome valuations underlie choices among objects, goods, actions, rules and strategies, they have a direct influence over behaviors of all kinds. ‘Outcome’ here refers to the consequences of a choice (typically food or fluid rewards) and ‘valuation’ refers to the neural representations of the subjective value of an outcome. Two kinds of outcome valuations are particularly important: desirability (or, equivalently, palatability) and availability (or, equivalently, probability). No matter how much potential value an outcome might have, if it cannot be obtained it has no actual value. Likewise, no matter how readily available an outcome might be, if it is worthless it is useless. As an analogy, in the 1840s California experienced a gold rush rather than a wheat rush, despite the fact that the state had both commodities: gold was highly desirable but difficult to find, whereas wheat was readily available but not nearly as desirable.

Neural recording studies have indicated that both the OFC and the VLFC are involved in outcome valuation. For example, neurons in both areas are sensitive to the desirability of the reward predicted by a stimulus^25,56–58. In addition, neurons in both regions encode the contingent associations between stimuli and rewards ^17,24,59,60. As indicated earlier, however, only causal manipulations can demonstrate whether the activity of neurons in a particular brain region is essential for a particular cognitive function. Some lesion studies in macaques have emphasized a role for the OFC in making choices that are based on the subjective value of outcomes in experiments in which states of selective satiation are used to reduce reward desirability^61,62, whereas others have highlighted a role for the OFC in linking choices to their contingent rewards, regardless of their desirability ^9,63. At the same time, lesion studies have implicated the adjacent VLFC in similar types of contingent learning^64–66 as well as in desirability valuations ⁶⁷. Thus, until recently, neither physiological nor lesion methods provided a clear case for dissociable functions of the OFC and VLFC in the context of valuation.

Findings based on aspiration lesions

An extensive body of findings from aspiration lesion studies in monkeys that began in the 1960s showed that damage to the OFC impaired performance on tasks that required changes to the animals’ natural inclinations or to trained (rewarded) responses^68–70. Later studies that employed stimulus-reward reversal learning paradigms identified the impairment resulting from OFC damage as being largely due to an inability to avoid selecting the previously rewarded object (now unrewarded) ^6,71,72, which was interpreted as “a deficit in suppressing the previously established habit” ⁶. In the minds of many investigators over the past 50 years (and continuing to the present^5,73), this finding firmly established the OFC as a region essential for inhibitory control. Several years ago, however, it became apparent that excitotoxic lesions of the OFC did not yield the same impairments, at least for reversal learning. In a series of experiments on macaque monkeys, subtotal excitotoxic lesions of the OFC failed to produce the predicted effects^74,75. This evidence helped overturn the idea that OFC had a role in inhibitory control during flexible stimulus-reward learning ¹².

These findings also led to speculation that the previous aspiration lesions of the OFC had some of their effects by causing inadvertent damage to axons passing through or near the OFC en route to terminations elsewhere⁷⁵. The uncinate fascicle, a large C-shaped fiber bundle connecting the temporal lobe and frontal lobe, fans out across the orbital surface as its fibers enter and exit the frontal lobe, and these fibers travel adjacent to the gray matter of the OFC⁷⁶. As a result, aspiration lesions might be expected to transect some of the axons traveling near or through the OFC, but fiber-sparing lesions made by injection of excitotoxins would not. To test this possibility, in a subsequent study a narrow aspiration lesion was made at the caudal edge of the OFC. This lesion reproduced the effects of complete aspiration lesions of the OFC⁷⁷, despite the fact that the bulk of the OFC remained intact. This result is consistent with the hypothesis that at least some behavioral impairments that follow aspiration lesions of the OFC are due to disruption of fibers of passage. The findings also suggested that the effects of similar OFC aspiration lesions on inhibitory control^6,71, emotion^72,78,79 and contingent stimulus–outcome learning ⁹ should be reassessed.

Contributions of OFC to contingent stimulus-outcome learning

The role of the OFC in contingent stimulus–outcome learning has profitably been examined using a 3-choice visual discrimination task with dynamic reward probabilities, known as the three-armed bandit task (FIG. 2a). In this task, the reward probabilities assigned to each stimulus change over the course of the session; to maximize reward, monkeys thus need to continually update their valuations of the stimuli. Importantly, this task taxes monkeys’ abilities to gauge the availability of outcomes predicted by a given stimulus, independent of that outcome’s desirability. This task was initially used⁹ to study the effects of aspiration lesions of the OFC, leading to the conclusion that the OFC is essential for updating valuations via contingent stimulus–outcome learning (FIG 2). However, a more recent study that evaluated the effects of more selective excitotoxic lesions of the OFC on the same task found that monkeys with OFC lesions were unimpaired (FIG. 3a)¹⁶, indicating that the OFC is not necessary for learning the contingent relationship between stimuli and outcomes. This study, and the others discussed in this section, aimed to elucidate the function of the OFC as a whole and therefore, unless otherwise specified, the term OFC here refers collectively to the lateral and medial subdivisions of the OFC (Fig. 1).

Figure 2: Selected behavioral tasks used to assess value-based decision making.

a| The 3-armed bandit task. At the start of each session, monkeys are presented with three novel images (options) on a touch screen monitor. Monkeys select one of the images by touching it on the screen. The same three images appear on each trial of the 300-trial session. The delivery of a reward for the selection of each image is predetermined based on four different probabilistic schedules, an example of which is shown in the plot at the right. In this case, over the first 150 trials, option A is the best choice; however as the monkeys approach trial 150 option B becomes the better option and remains so for the next 150 trials. To perform well, monkeys must discriminate the images, sample the outcome associated with each image by choosing it, and track the likelihood of rewards that they received for choosing each image during a test session. Because the reward probabilities assigned to each image changed over the course of the session, the monkeys needed to continually update their valuations of the images. b| A devaluation task. In one training trial, the monkeys learn to discriminate two objects, one of which—the blue cone—covers a food reward. In another training trial, the monkeys learn that a different object—a green hemisphere—is associated with a different food reward. In practice there were 60 pairs of objects and therefore a total of 60 training trials per session, 30 trials with each type of food reward. In the test phase of the task, the monkey consumes one of the two foods to satiety and must then make a choice between two previously rewarded objects. Each test comprised 30 such trials, each with a different pair of objects. This paradigm measures the ability of the monkeys to link objects with the current value of the food, which is influenced by the degree of satiety.

Figure 3: Effects of selective, excitotoxic lesions of the OFC and VLFC on availability- or desirability-based choices.

a,b| Performance on 3-armed bandit task (FIG. 2) by monkeys with OFC lesions and unoperated controls (a) and monkeys before and after VLFC lesions (b). Colored dots represent the probability of reward associated with choice of the best (high reward) option across a 300-trial session (the best option changes across the session). Colored lines and shaded areas show mean and SEM probability of choice of the high reward option by monkeys with OFC lesions, unoperated controls, monkeys before VLFC lesions and monkeys after VLFC lesions and reflect the ability to track the changing reward probabilities associated with each choice option. The scores for the OFC lesion and control groups overlap, whereas the score is lower for the VLFC lesion group. Bars to the right of the plots show group mean probability of choosing the high reward option in last 150 trials of session. c| Performance on the devaluation task (FIG. 2b) by monkeys with OFC lesions, VLFC lesions and unoperated controls. The graph indicates the proportion of the monkey’s choices that shifted to selection of the object associated with the nonsated food reward, when compared to choices in a baseline condition without satiation, and reflects the monkeys’ ability to make adaptive choices after one food is devalued by selective satiation. The higher the score, the greater the ability of monkeys to choose objects overlying the higher value (nonsated) food. Monkeys with OFC lesions make significantly fewer adaptive choices relative to monkeys in both other groups. d|. Summary of the effects of OFC and VLFC lesions: the difference in the score of lesioned groups from those of comparison groups derived from the contingent learning analysis (3-armed bandit) and the proportion shifted scores from the devaluation task are indicated. Plot highlights the double dissociation of function between two ventral prefrontal cortical areas, i.e., the selective and independent contributions of OFC and VLFC to different types of value updating. Data from Rudebeck and colleagues¹⁶. Abbreviations: Con, unoperated control monkeys; OFC lesion, monkeys with bilateral excitotoxic lesions of the orbitofrontal cortex; Before VLFC, monkeys before lesions of the ventral lateral frontal cortex; After VLFC, monkeys with bilateral excitotoxic lesions of the ventral lateral frontal cortex.

The boundaries of the aspiration and excitotoxic OFC lesions were shown to resemble each other closely^9,16,80, meaning that an anatomical difference in the extent of the intended lesions is unlikely to account for the discrepancies between these results. Instead, it appears that — as suggested by an earlier study⁷⁵ and described above — inadvertent disruption of white matter tracts running nearby or through OFC accompanies aspiration lesions and caused the impairment observed in these lesion experiments⁹. Indeed, given the anatomy of this region, in which white matter travels close to and in some cases within the deepest cortical layers⁷⁶, it is likely that even the most careful aspiration lesions will damage these tracts. Furthermore, because the geometry of the major fiber bundles in human brains resembles that of macaques^81–83, it is apparent that many of the impairments attributed to damage to the OFC in humans^84,85 need to be re-evaluated in the same way.

Contributions of VLFC to contingent stimulus-outcome learning

If, as the more recent results suggest, the loss of neurons in OFC did not cause the impairments in performance seen in the aspiration lesion studies, what did? Several lines of evidence point to the VLFC. Evidence from aspiration⁶⁴ and excitotoxic⁶⁵ lesion studies in monkeys suggest a role for the VLFC in contingent learning of stimulus–outcome associations, as does the relationship between monamine levels in the VLFC and performance in flexible stimulus–reward learning tasks⁶⁶. In addition, functional neuroimaging studies in humans^86–89 showed VLFC activation during changes in stimulus–reward associations. Finally, anatomical evidence regarding the trajectory of fibers traveling in the uncinate fascicle is consistent with the possibility that aspiration lesions of OFC could interrupt projections into and out of the VLFC⁷⁶.

To test whether neurons in the VLFC contribute to contingent stimulus-outcome learning, one study evaluated the performance of macaques with excitotoxic lesions of the VLFC on the 3-armed bandit task and found that these animals exhibited a profound deficit in their ability to track dynamic stimulus–outcome contingencies (FIG. 3b)¹⁶. Further analysis of the contribution of past choices and rewards to future choices, using an approach pioneered by Walton, Behrens and colleagues⁹, showed that monkeys with excitotoxic lesions of the OFC, like controls, were strongly influenced by recent outcomes, whereas monkeys with excitotoxic lesions of the VLFC were not. Thus, excitotoxic lesions of the VLFC produced a pattern of altered performance that was similar to that caused by aspiration lesions of the OFC⁹: an inefficiency in updating stimulus–outcome contingencies.

Contributions of OFC and VLFC to stimulus-outcome desirability valuations

Monkeys with either excitotoxic OFC lesions or excitotoxic VLFC lesions have also been tested on a reinforcer devaluation task (FIG. 2), in which their choices should be guided by the current desirability of different rewards, with reward probability remaining constant. In test sessions, monkeys with excitotoxic lesions of the VLFC, like controls, were able to update and use the current desirability of food rewards to guide their choices. By contrast, monkeys with excitotoxic lesions of the OFC made choices associated with food that had been devalued through satiation at a much higher rate (FIG. 3c)¹⁶. Similar findings on the devaluation task have come from macaques that received aspiration lesions: removal of OFC reduces the devaluation effect^61,62,90; removal of VLFC does not⁹⁰. Taken together, the findings from macaques indicate that the OFC but not VLFC is necessary for updating valuations that index desirability, whereas VLFC but not OFC is necessary for updating valuations that index availability.

Recently, this work has been extended to humans. Patients with damage to the OFC and adjacent portions of the medial PFC were tested on a devaluation task and exhibited a pattern of behavior strikingly similar to that of monkeys with excitotoxic OFC lesions⁹¹. In addition, patients with damage to the VLFC and adjacent lateral OFC, but not those with lesions involving more medial regions of the OFC and the medial PFC, were shown to be impaired in contingent learning on a version of the 3-armed bandit task ⁹². Thus, the available evidence is consistent with the idea that the human VLFC and OFC perform dissociable functions similar to those described here in macaques. Because there are multiple subdivisions of the VLFC that are likely to be homologous in macaques and humans^31,32, including rostral and caudal subdivisions of area 12/47 and area 45, we anticipate that future studies applying causal manipulations to these subdivisions will uncover additional functions within the VLFC.

Notably, previous macaque studies have shown that damage to the OFC typically does not disrupt value judgements or the ability to choose familiar objects or foods associated with a higher subjective value. For example, monkeys with OFC lesions (either aspiration or excitotoxic) that are sated on a given food will, if given a visual choice between two different food options, reliably avoid the sated food in favor of another⁶¹. Furthermore, food preferences after OFC lesions are unchanged relative to those present before surgery⁶¹. Thus damage to the OFC does not disrupt all value judgements; however, it does lead to an inconsistency in choices^75,93,94 and to a specific inability to link objects with recently altered (desirability) valuations (Fig 3). This distinction reflects the widespread encoding of value in the brain and the multiple attributes of value that may be encoded during learning.

VLFC and attention

Although it has been useful to frame the findings discussed above by considering value updating in terms of availability and desirability, there are a number of other differences between the 3-armed bandit and devaluation tasks that we have described. These include the extent to which they rely on external cues versus internal state and the extent to which they depend on information acquired over several trials to guide choice (availability) as opposed to current value (desirability). Accordingly, alternative interpretations are possible. One proposal — not incompatible with a role for the VLFC in representing availability — is a role for this region in top-down selective attention. Specifically, VLFC damage has been linked to reduced attentional selection, as evidenced by impairments in shifting between stimulus dimensions^95,96, reduced performance on tasks requiring allocation of attention to specific visual cues in monkeys^97–99 and humans¹⁰⁰ and poor implementation of vision-based rules in the absence of either discrimination or working memory impairments^90,101,102. Electrophysiological studies in macaques have revealed that the activity of VLFC neurons encodes features that serve as a target for visual search⁹⁹ and other attributes related to cued retrieval¹⁰³. Taken together, these data are consistent with the idea that the VLFC, though its interaction with the inferior temporal cortex area TE, could be implementing top-down attentional selection or retrieval, whether instructed or not, in the service of performance of a wide variety of tasks^103–105.

Take-home message

Studies employing causal manipulations in macaques, as well as lesion studies in humans, point to complementary roles for the OFC and VLFC in value-based decision making, especially in circumstances in which value is changing over time. OFC makes a selective contribution to choices based on the desirability of outcomes whereas VLFC makes a selective contribution to choices based on the availability of outcomes. The use of selective, fiber-sparing lesions in macaques has further indicated that earlier ideas pointing to an overarching role for the OFC in inhibitory control can be rejected. These findings pave the way for more fine-grained analyses of the functions of OFC, VLFC and their subregions (FIG. 1).

Specializations in the orbitofrontal cortex

The primate OFC is often treated as a single entity. However, its architectonic and connectional neuroanatomy (see above)⁴⁶, as well as recent work based on patterns of cortical coupling ^31,32, indicate that it has distinguishable parts (FIG. 1b). Several studies now point to dissociable functions not only for the medial and lateral subdivisions of the OFC, but also for its anterior and posterior sectors.

Medial versus lateral orbitofrontal cortex

The two behavioral tasks described above (FIG. 2) have helped delineate the functions of medial and lateral parts of the OFC. One study⁶³ used the 3-armed bandit task in macaques to investigate the neural basis of value comparison among alternative options. In this experiment, a high-value image (V1) and a low-value image (V3) had a fixed probability of reward delivery (0.6 and 0, respectively)⁶³. The probability of reward associated with a middle-value image (V2), although fixed within a session, changed across sessions. If the monkeys made optimal choices, the value of V2 would be irrelevant as V1 would remain the best option. However, when V2 was close to V1 in value, aspiration lesions of the medial OFC, but not the lateral OFC, disrupted their ability to make such optimal selections (FIG. 4a, b). Monkeys with lesions of the medial OFC could learn at a similar rate to controls, but their ability to choose the most valuable option was affected by the value of a close alternative. Additional tests indicated that the degree of the deficit after medial OFC lesions was also influenced by the difference in value between V2 and V3, the value of V3, and interactions among these irrelevant factors. Thus, in monkeys with medial OFC lesions, the value of alternative options inordinately affects the relative valuation of the best option.

Figure 4: Independent contributions of medial and lateral OFC to value-based decision making.

a,b|. Macaques took part in a three-arm bandit task (FIG. 2) in which the value of the best (V1) and worst (V3) choice options are stable within and across the sessions. The value of choice option V2 was stable within sessions but varied across sessions; in the example session shown the value of option V2 was close to (but less than) the value of option V1. The charts show the effects of aspiration lesions of the medial OFC (a) and the lateral OFC (b) on performance⁶³. Colored lines and shaded areas show mean and SEM probability of choice of the best value option (V1). Monkeys with lesions of the medial OFC (but not those with lesions of the lateral OFC) were poor at choosing the best option when the value of V2 was close to the value of V1, suggesting that medial OFC contributes to value comparisons. c| Effects of selective, excitotoxic lesions of the lateral or medial OFC on the devaluation task (FIG. 2). The difference score reflects the extent to which the monkeys shift their choices of objects after selective satiation relative to baseline conditions without satiation. The higher the score the greater the number of choices of objects overlying the higher value (nonsated) food. Controls show robust difference scores, indicating their sensitivity to the value of the outcome that is associated with object choices. Lateral OFC lesions result in long-term disruption of this capacity. Medial OFC lesions cause a transient impairment, as illustrated in measurements taken during an early postoperative test; however, the choices of these animals return to control values by the second postoperative test (late). Data from Rudebeck and Murray⁷⁵. d| Effects of reversible GABAergic agonist-induced inactivations of the anterior (area 11) and posterior (area 13) OFC on adaptive choices, assessed using the devaluation task. The proportion choice shifted score reflects the monkeys’ ability to make adaptive choices after one food is devalued by selective satiation. The higher the proportion shifted score, the greater the ability of monkeys to choose objects associated with the higher value (nonsated) food. Inactivation of area 13 during (but not after) selective satiation disrupted adaptive choices, reflecting an impairment in value updating. Inactivation of area 11 after (but not before) selective satiation disrupted adaptive choices, reflecting an impairment in goal selection. Symbols show scores of individual monkeys. Data from Murray et al.¹¹⁷

To further probe this phenomenon¹⁰⁶ a functional imaging study was carried out while humans performed a similar task. On some trials, subjects chose between two options in the presence of a third option that was unavailable. When the difference in value between the third (unavailable) option and the highest-value option was maximal, subjects often chose the second highest option: a suboptimal, and therefore inaccurate, choice. This choice behavior correlated with the strength of activations in the medial PFC, in a region near the medial OFC. On trials in which only two options were available for choice, activation in the medial PFC reflected the difference in value between the two options. However, in the presence of the third (unavailable) option, greater differences in value between the high-value and unavailable options correlated with smaller activations in the medial PFC, consistent with a function of this region in value comparisons. A role for the medial OFC and adjacent portions of the medial PFC in value comparison rather than in value learning is consistent with the results of other functional imaging studies^107–109 and with reports from humans with lesions within the medial OFC^92,93, suggesting that this area is critical for comparing options for choice.

Monkeys with excitotoxic lesions limited to either the medial or lateral OFC have also been tested on the devaluation task. After selective satiation, monkeys with lesions of the lateral OFC, but not those with lesions of the medial OFC, were impaired in their capacity to shift their choices of objects to avoid the devalued food (FIG. 4c)⁷⁵. This suggests that the impairment on the devaluation task that was observed after combined removal of the medial and lateral subdivisions of the OFC^61,77 (FIG. 3, see above) was due to damage to the lateral component. Monkeys with selective medial or lateral OFC lesions were also tested on a task intended to tax the ability to make ‘good’ choices based on comparing newly learned values. After being trained with objects associated with five different food items (in the same manner shown in FIG 2b ‘Training’ but now with five foods instead of two), monkeys were allowed to choose – for the first time – between objects associated with two different foods⁷⁵. Unoperated control animals and monkeys with lateral OFC lesions made choices consistent with their subjective valuation of the foods associated with an object, whereas those with medial OFC lesions often chose objects associated with less valued rewards. The latter results support the concept of a role for medial, but not lateral, OFC in comparing values of alternative options.

The distinction between the suggested roles of the medial and lateral OFC agrees with at least two other functional imaging studies in humans. In one, participants chose between images associated with pleasant food odors before and after they were sated with a food associated with one of the odors¹¹⁰. Activations in the lateral OFC tracked the current value of the options, whereas activations in the medial PFC, including medial OFC, tracked the value of the chosen outcome. A separate study assessed how valuations of individual nutrients were represented in medial and lateral OFC¹¹¹. Intriguingly, although the subjective food value of different food items was encoded in both the medial and lateral OFC, only the lateral OFC represented the attributes of individual nutrients within those foods. Effective connectivity analyses supported the conclusion that the information in the lateral OFC is integrated in the medial OFC to compute an overall value¹¹¹.

Taken together, the findings from both monkeys and humans indicate that the lateral OFC is important for learning, representing and updating specific object–outcome associations. As part of the ‘sensory network’ that was described by Price and colleages, a description derived from the pattern of anatomical connections of this region ^54,112, the lateral OFC therefore seems to house high-dimensional representations that arise from the convergence of several sensory modalities. As such, it is in a position to encode contrasts among outcomes. The medial OFC seems to be important for choices based on value comparisons. It could perform this function by transforming representations of value into a single dimension (a common currency) to facilitate comparisons among diverse options¹¹³ perhaps via attentional selection ¹¹⁴.

Given the themes of this article, it bears repeating that, although neural recordings have shown that neurons in the lateral OFC of macaques encode the probability of a reward as well as the magnitude and type of reward^20,115,116 (often in the same neurons^20,115), fiber-sparing, excitotoxic lesions that include the lateral OFC fail to affect choices based on probability (FIG. 3a and FIG 4b). Thus, although the activity of single neurons is a useful indicator of the potential functions of a given brain area, it cannot reveal whether a given brain area is essential for a given function. Instead, only causal manipulations can inform the specific contributions of a given brain region to behavior.

Anterior versus posterior orbitofrontal cortex

Although the evidence reviewed above indicates that the lateral OFC is essential for linking objects with current reward values, it does not tell us what mechanism is disrupted by lesions to this region. One possibility is that the role of the lateral OFC in the devaluation task involves registering the updated value of the food during selective satiation. This scenario would correspond to a role for the lateral OFC in value updating. Another possibility, however, is that lateral OFC links updated valuations to an option, such as an object, at the time of choice. We here suggest that this latter function be termed ‘goal selection’, because the object is the goal for action. To determine which of these scenarios best describes the function of the lateral OFC, the key question is when it makes its contribution to the devaluation effect.

To probe these possibilities, one study temporarily inactivated the lateral OFC¹¹⁷ by infusing a GABAergic agonist into the anterior lateral OFC (area 11) or posterior lateral OFC (area 13) before and after selective satiation in the devaluation task (FIG. 2). Disruption of devaluation effects by infusions before selective satiation would indicate a failure of either value updating or goal selection, whereas disruption of devaluation effects by infusions after selective satiation would indicate a selective contribution to goal selection. This study revealed that normal functioning of area 13 was necessary during, but not after, selective satiation (FIG. 4d)¹¹⁷, suggesting that the posterior part of the lateral OFC is essential for value updating but not for selecting goals based on these updated valuations. Notably, a similar pattern of effects was observed in a study of amygdala inactivation¹¹⁸. The opposite pattern of results was obtained for area 11: inactivation of area 11 before satiation had no effect on behavior, whereas inactivation after satiation disrupted devaluation effects. Thus, the anterior part of the lateral OFC appears to be essential for goal selection but not value updating¹¹⁷.

These findings suggested specialized functions for these two components of the lateral OFC. The posterior component appears to function in conjunction with the basolateral amygdala to update the valuations of expected outcomes, based on an animal’s current state: in this case, a state of selective satiation^117,118 (see also below). The anterior component appears to play a critical role in translating these valuations into goals for action. These conclusions agree with a recent brain imaging result in humans¹¹⁹ and the results from the anterior OFC bear a resemblance to the phenomenon of goal neglect, which involves an inability to translate abstract valuation knowledge into action following damage to the human PFC^120,121.

Some investigators¹²² have proposed that the subjective values guiding goal-directed choices are computed ‘on the fly’ in conditions outside the laboratory, when multiple factors may be influencing the calculation of subjective value, as this is thought to be less computationally cumbersome than having a large number of valuations (one appropriate for each individual circumstance) stored in long-term memory. Although the term value updating, as used here, might suggest that such a stored memory is located in the posterior OFC, the data do not support that idea. As shown in FIG. 4d, although the posterior OFC is essential for registering the change in value (desirability) that occurs during selective satiation, it does not need to be available to support later retrieval of desirability to support adaptive choices. This finding implies that the updated value is broadcast to several brain regions. The amygdala, anterior insular cortex, gustatory cortex and piriform cortex are regions that may well contribute to the ‘desirability’ aspect of valuation. If so, information retrieved from multiple sources in addition to the OFC might be accessed to compute ‘current desirability’ at the time of choice.

The connections of the anterior and posterior OFC subdivisions provide some insight into their respective roles. Consistent with a role for area 13 in value updating, OFC–amygdala connections (likely to be important for this function, see below) are heaviest in area 13 and sparse in area 11¹²³. Similarly, connections of the rostral VLFC (area 12r) with area 11 are stronger than those with area 13¹²⁴, which is important because this part of the VLFC is said to be part of a ‘lateral grasping network’ that guides goal-directed movements¹²⁵. Other routes for translating valuations into actions might depend on connections between the VLFC and the dorsolateral PFC and dorsal premotor cortex^58,126. Lesions of the VLFC do not affect performance on the devaluation task, perhaps because there are direct and indirect routes from the OFC and amygdala to cingulate premotor areas that may link actions to updated valuations^127,128.

Amygdala–OFC interactions

Updating desirability valuations

To understand the wider neural circuits involved in updating valuations, the effects of disconnection lesions on the devaluation task have been investigated. Disconnection lesions combine removal of a structure in one hemisphere with removal of a different (usually anatomically connected) structure in the other hemisphere. This procedure tests whether the two structures need to functionally interact in mediating a behavior. In macaques, crossed-disconnection experiments have shown that good performance on the devaluation task requires functional interactions between the amygdala and the OFC¹²⁹. In the discussion above, tasks that require choices between objects were emphasized; however, studies show that amygdala–OFC interactions are also necessary for action-based choices^130,131. These two structures also interact with at least one more structure, the MDmc, in mediating these effects¹³².

Recent work has provided some insight into the comparable mechanisms in rodents, in which there is an extensive body of research on OFC contributions to learning and decision making¹³³. One study¹³⁴ used viral methods to selectively express the inhibitory hM4Di receptor in amygdala neurons that project to the OFC in rats. When the receptors were activated, thereby silencing the amygdala inputs to the OFC, the rats were unable to update the valuations of expected outcomes. Inhibiting the reciprocal projections from the OFC to the amygdala did not have this effect. Taken together with the work in primates, these findings suggest that inputs from the amygdala to the OFC play a critical role in value updating. The role of the reciprocal projection remains unknown, but the hypothesis that it elicits OFC–amygdala interactions seems worthy of investigation.

Just as the amygdala interacts with the OFC to mediate desirability estimates, it is possible that it interacts with the VLFC to mediate availability estimates. Although surgical disconnection to test this hypothesis has not yet been carried out, bilateral lesions of the amygdala¹³⁵ and, in separate experiments, of the VLFC¹⁶, disrupt visual discrimination learning based on probabilistic outcomes.

Effects of amygdala lesions on value coding in the OFC

The idea that interactions between the amygdala and the OFC update valuations of outcome desirability predicts that removal of the amygdala should affect value encoding by neurons in the OFC. As noted above, when monkeys see stimuli that predict rewards, OFC neurons encode the values of the anticipated outcomes¹⁹. For example, if two different images predict the delivery of different volumes of a juice reward, value-encoding OFC neurons exhibit activity that reflects the relative value of the two outcomes associated with those images^19,136. In addition, many OFC neurons encode the sensory properties of rewards, such as their taste, flavor and texture^8,137 and these properties are also linked to reward value.

How is this value encoding established and maintained? Given the interaction of the amygdala with the OFC in value updating, it seems likely that these regions work together to establish and maintain value signals in the OFC. The role of the amygdala in the encoding of stimulus–outcome value by OFC neurons was tested in a combined physiological recording and lesion experiment¹³⁸. Monkeys were trained on a fixed set of stimulus–reward associations, in which different images predicted different magnitudes of fluid reward. On each trial, the monkeys chose one of two images and obtained the reward quantity associated with it. Before amygdala lesions, when monkeys viewed images on the monitor screen, the activity of many neurons in the OFC reflected the magnitude of the reward associated with individual images. Many of the same neurons also were active in relation to the magnitude of received reward. Removing amygdala inputs to the OFC significantly reduced (but did not abolish) the encoding of expected reward value in the OFC, in both the stimulus period and around the time of reward delivery (FIG. 5)¹³⁸. These findings suggest a role for the amygdala in the active maintenance of learned stimulus–outcome associations.

Figure 5: Effect of amygdala lesions on value coding in the OFC.

Monkeys performed a choice task with a set of familiar stimulus-outcome associations. Different stimuli were associated with different amounts of a juice reward. On each trial, two images (S1 and S2) were presented sequentially and then monkeys were allowed to choose one of the two stimuli. The amount of juice assigned to the chosen image (S1 or S2) was delivered a short time later. a|. Percentage of OFC neurons encoding the magnitude of the associated reward while the monkeys viewed S1 and S2. Data are averaged over the entire population of recorded neurons to illustrate changes in encoding during the period stimuli were being evaluated (0–2000 ms.). Data are aligned to the onset of the presentation of the first image (S1). A bilateral excitotoxic amygdala lesion led to a reduction in the encoding of the value of the anticipated outcome. b|. Percentage of OFC neurons encoding the magnitude of reward near the time of choice and reward delivery. Data are aligned to the onset of reward delivery. A bilateral excitotoxic amygdala lesion led to a reduction in the encoding of the value of the chosen image and of the received reward. Data from Rudebeck and colleagues¹³⁸.

In the same monkeys, OFC neuronal activity was recorded during the acquisition of new stimulus–outcome associations. Although amygdala removal had no effect on the ability of monkeys to choose familiar, high value stimuli that had been learned preoperatively, it slowed the rate at which monkeys learned to choose novel, high value stimuli. In addition, as was the case for the familiar (preoperatively learned) stimuli, amygdala removal reduced the encoding of expected reward value during the evaluation of different novel stimuli. Surprisingly, however, amygdala lesions had little effect on the encoding of value around the time of reward delivery (the period during which the value of novel stimuli had to be learned). This finding indicates that input to the OFC from the amygdala is especially important for allowing OFC neurons to encode the association between familiar stimuli and the outcomes that they predict. In addition, even though the novel stimulus-outcome associations were learned relatively quickly (monkeys chose the image predicting the largest possible reward within just a few trials), many fewer neurons in the OFC encoded stimulus–outcome value during the evaluation of novel stimuli relative to familiar stimuli ¹³⁹. This finding implies that the encoding of reward value continues to develop as time and experiences accumulate.

Conclusion

By studying the OFC and VLFC with selective lesions and inactivations, recent research has shown that these two parts of the primate frontal lobe have different and complementary functions. These findings indicate that together they enable adaptive choices based on a combined assessment of the desirability and availability of the reward outcomes that are predicted to occur as a result of a choice, with the OFC mediating desirability estimates and the VLFC specializing in availability estimates. Put another way, the lateral OFC guides decision making through assessments of what an outcome is worth at the moment, if it could be obtained, and the VLFC provides a complementary estimate of the chance that it can be obtained. Worth and chance, when combined in a coordinated way, determine the benefit that can be expected on the basis of a choice. In economic theory, expected value corresponds to the product of the desirability and probability of a potential outcome and choices biased too much toward one or the other are bound to be maladaptive.

The results reviewed here suggest that many of the functions formerly attributed to the OFC depend instead on other cortical areas. In the past, the OFC has been viewed as a center for behavioral inhibition that forestalls impulsive, compulsive, prepotent and automatic behaviors. Although some authorities still support this idea¹⁴⁰, a good deal of evidence contradicts it^{11,13,141,142}. Another current idea is that the OFC represents the value of predicted outcomes in a common neural currency⁷; however, this view appears to be at odds with studies that point to a role of the medial OFC and medial PFC in this function^{63,75,109,143–145}. Another view, which comes mainly from rodent research, is that the OFC — a region homologous to the posterior, agranular parts of the primate OFC (FIG. 1) — provides a map of task space¹⁰. This map, it is proposed, represents a state space: a record of previous stimuli, actions and other sensory features that occur in association with outcomes in a multi-dimensional array. In support of this idea, fMRI studies in humans report activations in the parts of the medial OFC that border on the medial PFC when participants must track (and attend to) unobservable task states to perform a task¹⁴⁶. However, the finding that the credit assignment function previously attributed to the OFC depends instead on the VLFC in primates undermines this idea to an extent, as it implies that task states related to different aspects of a reward would be represented in different brain areas. Indeed, we here suggest that many parts of the primate PFC—if not the entire PFC — represent a high-dimensional task space. This view emerges, in part, from the idea that different parts of the primate granular PFC emerged at distinct times during primate evolution^147,148. According to this view, specialized neural representations evolved in every PFC area for the same fundamental reason: to transcend problems and exploit opportunities encountered by specific ancestors at particular times in the distant past¹⁴⁸.

Beyond the differentiation of OFC and VLFC function, the results discussed above also point to both anterior–posterior and medial–lateral specializations within the OFC. Discussions of the OFC (and the region often identified as the ventromedial PFC, which overlaps with OFC) often treat it as as a single functional entity. Furthermore, an understanding of brain-behavior relationships in psychiatry and neurology are impeded by the fact that brain lesions in patients are rarely selective to cell bodies (gray matter) or limited to individual regions of interest^149,150. Consequently, the understanding of the intrinsic specializations of the nonhuman primate OFC, reviewed here, and the medial PFC^151–153, together with consideration of PFC homology in macaques and humans, should promote a more nuanced understanding of the potential origins of neurological and psychiatric disease. The findings discussed here suggest the lateral parts of the OFC represent predicted outcomes for choices among objects and actions and do so in terms of an updated valuation of outcome desirability based on current biological states and needs. A part of this region specializes in translating these valuations into actions. Medial parts of the OFC and perhaps other medial PFC areas represent potential outcomes in a common currency. All of these areas contribute to adaptive decision making through their specialized representations and their efferent operations, helping to increase the chance that actions worth making are chosen.

Glossary terms

Reward-guided learning

A general term that refers to any kind of learning facilitated by reward. Learning may include stimulus-outcome learning, action-outcome learning, and stimulus-response learning, among others.

Inhibitory control

The ability to inhibit choices or responses that have previously been rewarded. The concept of behavioral inhibition includes the ability to suppress default, habitual and prepotent behaviors.

Flexible stimulus-reward learning

A general term that refers to the ability to quickly make and break associative links between objects (or other cues) and rewards.

Value-based decision making

A general term that refers to the ability to make facultative choices that optimize subjective value.

Credit assignment

The ability to learn that a particular outcome (typically food or fluid) was produced by a particular choice.

Cognitive map

A neural representation of stimuli, actions and other sensory features that occur in association with outcomes in a multidimensional array. Has been theorized to guide value-based decision making.

Value updating

The process of registering a change in the neural representation of the desirability or availability of foods

Aspiration lesion

A technique for removing gray matter (i.e., neurons) based on subpial aspiration of tissue. Lesions are typically carried out with the aid of an operating microscope.

Reversal learning

After subjects learn to choose a rewarded item over an unrewarded item, the stimulus-outcome contingencies switch without warning and the subject must now learn to choose the object that was initially unrewarded. The only feedback to guide choices is the occurrence of reward or nonreward.

Excitotoxic lesions

A technique for selectively removing gray matter (i.e., neurons) and sparing white matter (i.e., axons) based on the injection of neurotoxins. Injections are often carried out via a stereotaxic approach based on coordinates obtained from magnetic resonance images of the brain.

Cortical coupling

Patterns of correlated activity between different brain areas discerned from resting state functional magnetic resonance imaging. Has been used to identify brain areas in macaques and humans that have similar connectivity profiles and perhaps comparable functions.

Attentional selection

Concentration of visual or other (e.g., somatosensory, auditory) sensory processing resources towards behaviorally significant spatial locations or visual features. This process enhances sensory perception so that responses can be faster and more accurate.