The orbitofrontal oracle: cortical mechanisms for the prediction and evaluation of specific behavioral outcomes

Abstract

The orbitofrontal cortex (OFC) has long been associated with the flexible control of behavior and concepts such as behavioral inhibition, self-control and emotional regulation. These ideas emphasize the suppression of behaviors and emotions, but OFC’s affirmative functions have remained enigmatic. Here we review recent work that has advanced our understanding of this prefrontal area and how its functions are shaped through interaction with subcortical structures such as the amygdala. Recent findings have overturned theories emphasizing behavioral inhibition as OFC’s fundamental function. Instead, new findings indicate that OFC provides predictions about specific outcomes associated with stimuli, choices and actions, especially their moment-to-moment value based on current internal states. OFC function thereby encompasses a broad representation or model of an individual’s sensory milieu and potential actions, along with their relationship to likely behavioral outcomes.

Introduction

Over a century ago, a team of divers discovered an ancient mechanical device in the wreck of a Roman galleon. Called the Antikythera mechanism, its function remained a mystery for decades. The use of new imaging techniques, together with a painstaking analysis of the device’s inner workings, revealed that this collection of bronze gears can calculate the future locations of the earth, moon and sun. As a result, the Antikythera mechanism accurately predicted eclipses and other celestial events (Freeth et al., 2008). In ancient times, these prognostications probably contributed to planning important cultural activities, and so it is of some interest that the ancients trusted an analog computer for these prophesies, rather than their oracles, auguries or authorities.

Like the chance discovery of the Antikythera mechanism, the first knowledge about orbitofrontal cortex (OFC) function stemmed from a fortuitous event. As the most ventral part of the prefrontal cortex (PFC) (Fig. 1), OFC was directly in harms way when an accidental explosion led to “the passage of an iron rod through the head” of a now-famous railroad worker named Phineas Gage (Harlow, 1848). His subsequent lack of self-control, coupled with otherwise fairly normal cognitive capacities, had an enduring influence on ideas about OFC, along with PFC more generally. Based on patients like Phineas Gage and animals with experimental lesions of OFC, many theories hold that its functions primarily involve behavioral inhibition, inhibitory self-control and emotional regulation (Dias et al., 1996; Mishkin, 1964; Roberts and Wallis, 2000; Rolls, 2000).

Figure 1.

Carmichael and Price’s (1994) parcellation of the ventral surface of the frontal lobe of a macaque brain based on variation in chemo- and cytoarchitecture. Red and purple shaded regions correspond to the lateral and medial subdivisions of OFC, respectively. These two subdivisions are often used in neuropsychological and neurophysiological investigations in primates to grossly subdivide OFC. The medial and lateral orbital sulci are marked by thick black lines. Thick dark arrows indicate connections, based on Carmichael and Price (1995a,b).

Recent research has overturned this view of OFC function. These results show that OFC has a more affirmative function. Rather than primarily suppressing unwanted behaviors and emotions, this new evidence shows that OFC contributes to both by predicting the specific outcomes that should follow either sensory events or behavioral choices and representing an updated valuation of these outcomes. These signals play a particularly important role at the time of a choice, and they underlie both good choices and bad ones.

Evidence from monkeys indicates that OFC encodes the predicted value of objects that might serve as goals, perhaps in an abstract “common currency” (Padoa-Schioppa, 2011). In contrast, results from rodents emphasize specific outcomes more than a general outcome like a common currency. These findings suggest that OFC represents the sensory qualities of available outcomes that have become associated with conditioned stimuli (Schoenbaum et al., 2011) and potentially maps the entire sensory milieu in terms of potentially reinforcing outcomes (Wilson et al., 2014). Although delineating OFC’s differences among mammalian species remains an active area of research, an emerging consensus recognizes that OFC in all mammals plays a critical role in signaling predictions about the future outcomes of behavior, including both appetitive and aversive ones.

Researchers deciphered the Antikythera device through interplay between a detailed analysis of its inner workings and hypotheses about its function, and we would like to do something similar for OFC. Accordingly, this review characterizes the necessary functions of OFC and then delves into its mechanisms. We discuss the information encoded by OFC neurons, the source of these signals, and how distinct parts of OFC contribute to its functions. In doing so, we draw primarily on insights gained from studies of monkeys and rodents, in part because they provide a unique opportunity to examine the causal contribution of OFC to behavior, and partly because they provide insight into its information processing at the single-neuron level. We pose three questions: What does OFC do?; What information does OFC process?; and How does OFC influence behavior?

What does OFC do?

Prediction of specific outcomes

Our strongest evidence regarding OFC’s function comes from studying animals and humans without an OFC. Observations of patients (like the aforementioned Phineas Gage) and monkeys with damage to OFC have led to the idea that OFC is important for inhibitory control, specifically for suppressing prepotent responses in the face of changing circumstances (Mishkin, 1964). This “inhibitory control” hypothesis was based in part on the effects of aspiration lesions of OFC on a task called object reversal learning. When humans, monkeys, cats and rodents with OFC damage face this task, they cannot rapidly reverse their choices after changes in stimulus-reward contingencies (Butter, 1969; Chudasama and Robbins, 2003; Dias et al., 1996; Mishkin, 1964; Schoenbaum et al., 2003a; Teitelbaum, 1964). Over time, and with repeated findings of this kind, the deficits in object reversal learning became accepted as a pathognomonic sign of OFC damage. The inhibitory control hypothesis of OFC function has had a wide-ranging impact on neuropsychology, neurology and psychiatry (Blair, 2010; Clark et al., 2004; Fineberg et al., 2009; Roberts and Wallis, 2000) and has been used to account for the extreme changes in emotional behavior that often follow OFC damage or dysfunction (Rolls, 2000).

The link between OFC function, reversal learning and inhibitory control was based largely on the effects of aspiration lesions of OFC (Iversen and Mishkin, 1970; Jones and Mishkin, 1972; cf. Dias et al., 1996), lesions that not only remove the cortex but also may damage fibers of passage—axons coursing nearby or through OFC. In a study of macaque monkeys, we recently found that complete, excitotoxic, fiber-sparing lesions of OFC (Fig. 1, Fig. 2a) had no effect on either object reversal learning (Fig 2b) or emotional responses to artificial snakes (Rudebeck et al., 2013b) (Fig 2c), both of which are severely disrupted by aspiration lesions of OFC (Butter, 1969; Butter et al., 1970). This surprising result indicates that, contrary to prevailing expert opinion, OFC is not necessary for choosing objects flexibly in the face of changing stimulus-reward contingencies, at least in this classic task. Thus the pathognomonic “OFC task” does not depend on OFC at all, at least not as this anatomical term refers to gray matter as opposed to white matter and fiber tracts.

Figure 2.

The effect of excitotoxic lesions of OFC on reward-guided behavior. A) Excitotoxic OFC lesions; location and extent of the intended lesion shown on drawing of a coronal section through the frontal lobe (Intended lesion, top), representative case with an excitotoxic lesion of OFC (T2-weighted MRI, taken 5 days after the injection of excitotoxins, middle), and representative case with an excitotoxic lesion of OFC (T1-weighted MRI, taken ~3 years after surgery, bottom). B) Serial object reversal learning. Mean (±SEM) number of errors for unoperated controls (CON_EXC and CON_ASP, unfilled circles and squares respectively), macaques with excitotoxic OFC lesions (OFC_EXC, shaded triangles) and macaques with aspiration lesions of OFC (OFC_ASP, shaded diamonds). Unlike monkeys with aspiration lesions of OFC, monkeys with excitotoxic lesions of OFC do not differ from controls in their performance on this task. C) Emotional responses to neutral and fear inducing objects. Mean (±SEM) latency of unoperated controls (unfilled bars) and monkeys with excitotoxic OFC lesions (gray bar) to retrieve a desired food reward in the presence of different objects. Monkeys with excitotoxic lesions of OFC do not differ from controls. D) Devaluation task. Mean (±SEM) difference scores for unoperated controls (unfilled bars) and macaques with excitotoxic lesions of OFC (gray bars) in the object-(left) and action-based (right) devaluation tasks. Same labels as in (C). Monkeys with excitotoxic lesions of OFC are still undergoing behavioral testing; the estimated extents of the OFC lesions as determined from postoperative T2-weighted MR scans ranged from 64 to 96% complete, and later T1-weighted (structural) MR scans are consistent with this picture. There was no correlation of the lesion extent and scores on any behavioral assessment. Adapted from Rudebeck et al. (2013b) and Rhodes and Murray (2013).

These new results also raise doubts about the links between OFC, emotional regulation and behavioral flexibility—for both monkeys and humans (Butter et al., 1970; Damasio et al., 1990; Izquierdo et al., 2005). The locations of white-matter bundles traveling into the frontal lobes (Croxson et al., 2005; Jbabdi et al., 2013) suggest that humans with damage to OFC—whether through strokes, blunt force trauma, tumor excisions or aneurisms—have damage not only to the gray matter of OFC, but also to white matter traveling to frontal regions outside OFC. The close correspondence between macaque and human OFC (Petrides and Pandya, 1994) extends this conclusion to experimental lesions of OFC in monkeys.

Two obvious questions follow from our new results: i) which white matter pathways are damaged by aspiration of OFC; and ii) why do excitotoxic lesions in rodents and marmosets cause deficits in reversal learning (Chudasama and Robbins, 2003; Dias et al., 1996; Izquierdo et al., 2013; Schoenbaum et al., 2003a)?

In answer to the first question, aspiration lesions of medial OFC, alone, do not affect emotional responses to snakes (Noonan et al., 2010a) or cause deficits in object reversal learning (Noonan et al., 2010b). Recall that, in our recent experiment, the selective excitotoxic lesion eliminated both medial and lateral OFC (Fig. 1, Fig. 2a). Taken together, these findings suggest that white matter tracts adjacent to the lateral, but not medial, OFC might be the critical pathways. Anatomical tract tracing studies also show that connections from the medial temporal lobe to the lateral convexity of PFC course near to the fundus of the lateral orbital sulcus, close to where cortex is typically removed by aspiration lesions of OFC (Lehman et al., 2011). Alternatively, the severe impairments in reversal learning seen in the older studies might have resulted from a more widespread disconnection, one involving axons located near both medial and lateral OFC.

There are a number of potential answers to the second question. One is that different species, by virtue of their different foraging niches, solve reversal learning tasks using different strategies. This idea could account for the observation that deficits after excitotoxic lesions of OFC in rats and marmosets differ qualitatively from those caused by aspiration lesions in macaques. Specifically, deficits in rodents and marmosets are usually confined to the first few reversals (Clarke et al., 2008) whereas those in macaques emerge only after a number of reversals (Izquierdo et al., 2004). Alternatively, the enlargement of PFC during primate evolution (Passingham and Wise, 2012; Preuss, 1995) could mean that other parts of PFC, beyond the OFC areas involved in our lesions, can guide choices in reversal learning. Once we identify the cortical areas underlying reversal learning in macaques, we should be in a position to decide among these possibilities.

The results discussed so far show something about what OFC does not do, but not much about what it does. Compared with the negative results obtained for the object reversal learning (Fig. 2b, triangles) and emotional regulation (Fig. 2c) tasks, results from the devaluation task show that OFC plays a critical role in choosing objects based on the expected value of specific outcomes (Fig. 2d, left). This impairment, alone among those observed in the tasks we used, did not depend on disrupting fiber pathways near or passing through OFC.

In devaluation tasks, animals need to update the value of specific outcomes and to use that information to guide choices. Here, when we say “specific outcome”, we mean a representation of the constellation of sensory properties that together comprise the food or fluid that is produced by a particular choice. For a food outcome, for example, this would translate to its taste, smell, visual attributes such as color, shape and texture, and its feel in the mouth. Importantly, whereas object reversal learning examines the ability of animals to respond flexibly to changes in stimulus–reward contingencies when food value is fixed and a single type of reward is available, devaluation tasks examine the ability of animals to respond flexibly to changes in the value of different foods, based on current internal states such as selective satiation. Put another way, reversal learning relates to the likelihood of resource availability given some choice, whereas devaluation tasks probe what a particular resource is worth at the moment.

Given the finding that fiber-sparing lesions of OFC disrupted the choice between two objects based on current valuations (Fig. 2d, left), we conclude that OFC is necessary for updating the value of specific object-outcome associations, not simply tracking the presence, absence or likelihood of a reward. Of course, OFC operates in concert with other brain regions to perform this function, including the amygdala and mediodorsal nucleus of the thalamus (Murray and Rudebeck, 2013), a point taken up later.

In a separate devaluation experiment, monkeys with selective excitotoxic lesions of OFC also had deficits in updating the valuation of expected outcomes in an action-outcome task (Rhodes and Murray, 2013)(Fig. 2d, right). In fact, these results came from some of the same monkeys tested on the object-outcome task (Fig. 2d, left). This result is surprising given that previous results in both rodents (Ostlund and Balleine, 2007) and macaque monkeys (Rudebeck et al., 2008) suggested that OFC was only required for Pavlovian (stimulus-outcome) or object-reward processes, not action-outcome or instrumental responses. These data indicate that, in primates, OFC may be necessary for all computations where sensory-specific properties of the outcome have to be considered. On this view, OFC functions in evaluating such objects both as choice items (in the object devaluation task) and as object outcomes (e.g., food rewards in both object and action devaluation tasks).

Taken together, the results from devaluation studies suggest that OFC encodes a unified representation of outcomes, including their specific sensory properties and current biological value, which could enter into a variety of associations. For example, outcomes could be associated not only with objects and actions, but also with abstract concepts, behavioral rules and strategies. Indeed, one study in macaques that employed an analog of the Wisconsin Card Sort Task implicated OFC in the selection of behavior-guiding rules after rule changes (Buckley et al., 2009).

Our results appear to fit with data from rodents suggesting that OFC is critical for representing the sensory qualities of specific outcomes associated with actions as well as objects (Burke et al., 2008; Gremel and Costa, 2013; Keiflin et al., 2013) and model-based, as opposed to model-free, reinforcement learning (McDannald et al., 2011, 2014). Model-based learning draws on an animal’s detailed knowledge of the world, in contrast to model-free reinforcement learning, which accounts for behaviors based solely on experience with stimulus–response–outcome associations. Primate and rodents diverged long ago, but we assume that as OFC evolved in both lineages the parts inherited from their common ancestor retained a conserved function and new parts (Passingham and Wise, 2012; Preuss, 1995) performed related ones. As a result, we are now in a position to answer the question posed by the title of this section.

Summary: What does OFC do?

Impairments in object reversal learning (Fig. 2b) and emotional responding (Fig. 2c) in macaques previously ascribed to OFC appear to be due to interrupting fiber tracts running nearby (or though) OFC. Accordingly, the available evidence now argues against the idea that OFC per se plays an important role in behavioral inhibition, inhibitory self-control or emotional regulation. Instead, OFC, acting together with the amygdala and mediodorsal thalamus, represents the specific identity of predicted outcomes and their up-to-the-moment value, taking into account an animal’s or human’s current state (Gottfried and Zelano, 2011; Holland and Gallagher, 2004; Passingham and Wise, 2012; Schoenbaum et al., 2009). In this way, OFC plays a crucial role in making beneficial choices among either objects or outcomes (Fig. 2d), and perhaps concepts, rules and strategies, as well. In addition, the outcome representations housed in OFC presumably mediate associations that promote certain Pavlovian responses (i.e., licking).

What information does OFC process?

Signals related to attributes of future outcomes

Now that we can say what the OFC does, we can explore its mechanisms more fully. Physiological recording studies have demonstrated that many neurons in OFC signal information related to reinforcers that have become associated with stimuli, choices and actions, specifically foods, fluids and mildly aversive outcomes. The first neurophysiological studies of OFC showed that its cells encoded stimuli that predicted impending fluid rewards (Niki et al., 1972; Rosenkilde et al., 1981; Thorpe et al., 1983), and a myriad of additional outcome-related signals have been reported since then (for a review see, Abe et al., 2011; Morrison and Salzman, 2011; Padoa-Schioppa and Cai, 2011; Pearson et al., 2014; Schultz et al., 2011; Simmons et al., 2007; Wallis and Kennerley, 2010).

One way to synthesize the diverse array of signals in OFC depends on the idea that its neurons signal predicted outcomes as well as their receipt. Specifically, a substantial proportion of OFC neurons encode the potential for, sensory attributes of and subjective value of outcomes associated with external stimuli. Other OFC neurons signal similar information when the predicted outcomes actually occur (for example, Padoa-Schioppa and Assad, 2006). It is reasonable to think that the stimulus-evoked signals convey the subjective value of biologically significant outcomes because, unlike neurons in other parts of the brain, neurons in OFC distinguish between appetitive and aversive outcomes. In contrast, signals elsewhere appear to code motivational salience, which does not depend as much on whether the outcome is detrimental or beneficial (Morrison and Salzman, 2009; Roesch and Olson, 2004). These neurophysiological attributes, combined with connections from limbic structures and a variety of sensory modalities (Carmichael and Price, 1995a, b; Ghashghaei et al., 2007)(Fig. 1), place OFC in a unique position. Indeed, OFC is one of the few places in the brain where visual, gustatory, olfactory and visceral sensory inputs converge (Price, 2005). Thus, neurons in this part of PFC have access to the information essential for evaluating and choosing advantageously among options.

Functional brain imaging (fMRI) investigations of humans similarly suggest that OFC signals outcome-related attributes (Gottfried and Zelano, 2011; O'Doherty, 2007). One difference between human and animal studies is that the former rarely use primary reinforcement, such as fluid reward or chocolate (cf. Small et al., 2001), whereas studies involving animals almost always do. Instead, human participants usually play games or other tasks to earn money or other secondary reinforcers. This difference complicates the comparison of results from different species and raises the possibility that separable parts of OFC encode primary and secondary reinforcement.

Two neuroimaging studies indicate that primary and secondary reinforcers may be encoded by separate parts of OFC in humans. A posterior to anterior dissociation in signaling primary and secondary reinforcement, respectively, was first reported in an experiment comparing OFC activation for sexually explicit images and money (Sescousse et al., 2010). The images used in that study were more complicated than suggested by terms like primary and secondary reinforcers, however, and many of them probably operated as both.

A recent study provided clearer evidence on this issue (Klein-Flugge et al., 2013). Using an experimental design that relied on repetition-suppression effects to map reward-related representations in OFC, the authors showed that representations in anterior OFC are specific to secondary reinforcers whereas those in the most posterior part of OFC are specific to primary reinforcers. Subjects were taught that two pairs of stimuli predicted specific but different outcomes. For instance, two stimuli predicted strawberries and two other stimuli predicted potato crisps (Fig 3a). To unmask parts of OFC that encoded primary rewards, such as strawberries, the authors looked for areas showing a decrease in activation when a different stimulus predicted the same primary reinforcement as on the previous trial (i.e., different stimulus, same outcome, Fig 3b). In this condition, parts of posterior OFC as well as agranular insular cortex showed reduced activation. For secondary reinforcement, the authors searched for decreased activations after repeated presentations of the same stimulus-outcome combinations (i.e., same stimulus, same outcome, Fig 3c). In this condition, parts of anterior OFC showed reduced activation. Thus, primary and secondary reinforcers appear to be signaled in different parts of the human OFC, especially for distinct stimuli associated with specific reinforcers. Future research on nonhuman primates could determine whether these signals contribute causally to goal selection and outcome evaluation. In addition, fMRI studies in humans could provide additional detail by investigating the generality of the anterior-posterior dissociation, and whether different types of primary and secondary reinforcers map onto distinct portions of OFC.

Figure 3.

Encoding of primary and secondary reinforcers in OFC. A) Two stimuli were paired arbitrarily with each of two different foods, equally valued by participants. Stimuli appeared sequentially for 700 ms separated by a 400-ms intertrial interval. B) Brain regions showing a decrement in BOLD response (green-blue shading) following presentation of two different stimuli that predicted the same food reward (different-stimulus, same-outcome). This measure is thought to represent primary reinforcers. C) Brain regions showing a decrement in BOLD response (red-yellow shading) following the sequential presentation of the same stimulus-reward pairing (same stimulus, same outcome). This measure is thought to represent secondary reinforcers. Z coordinates denote dorsal-ventral level. Adapted from Klein-Flugge et al. (2013).

Outcome signals in OFC: beyond sensory attributes and subjective value

Although some neurons in OFC signal the identity and subjective value of reward (Morrison and Salzman, 2009; Tremblay and Schultz, 1999), OFC neurons might represent other aspects of reward, as well. For example, OFC is thought to play a role in the hedonic experience of reward through its role in valuation (as reviewed by Kringelbach, 2005). If so, then OFC neurons might signal reward irrespective of its context, attributes, or meaning, somewhat similar to theories suggesting that the OFC is important for computing general economic value (Padoa-Schioppa, 2007).

A recent series of experiments by Tsujimoto and colleagues set out to tackle this question. Their results indicate that a proportion of neural responses in OFC convey the sensory or informational aspects of rewards, rather than being directly related to either their identity or hedonic aspects (Tsujimoto et al., 2011, 2012). Macaques were trained to implement different behavioral response strategies based on instructional cues. In one condition, the orientation of a bar presented on a monitor screen instructed macaques whether to make the same response as on the previous trial or to make a different one (i.e., stay or switch). In another, the delivery of one drop or two half-drops of fluid served as the instruction cues, again indicating whether to stay or switch (Fig 4a). Correct performance in either condition was rewarded with the same fluid reward, delivered in the same way. On trials in which fluid served both as an instructional cue and a reward, it was possible to determine whether neurons in OFC encoded reward for either its instructional meaning, as feedback for a correct response, or irrespective of its meaning. If OFC neurons always had the latter property, it would indicate that this cortical area signals reward for its pleasurable or hedonic aspects.

Figure 4.

OFC activity related to rewards delivered as instructional cues or feedback. A) Trial sequence. Trials started with the presentation of an instructional cue, either a single drop of fluid or two smaller drops. These cues signaled to the monkey whether to stay with the response from the previous trial (left or right) or switch to the alternative option. Once the monkey made a saccade to one of the two potential targets, both targets became solid white, whether correctly or incorrectly performed. Successful performance led to the delivery of additional fluid reward as feedback (reinforcement). Red type indicates sequence on example trial. B) Brain regions studied: OFC (orange on ventral view of macaque frontal lobe); frontal polar cortex (FPC, red on dorsal view of macaque frontal lobe); dorsolateral prefrontal cortex (DLPFC, green on lateral view of macaque frontal lobe). C) Percentage of neurons in OFC, FPC and DLPFC encoding the reward when delivered as an instructional cue (cue only), as feedback (feedback only) or both. Color scheme as in (B). Dashed line indicates noise/chance level for cue and feedback responses. Adapted from Tsujimoto et al. (2012).

A proportion of the neurons in OFC only encoded the reward when it was presented as an instruction cue. These same neurons did not respond to the reward delivered for correctly performing the instructed strategy (cue only, Fig 4b, c). A separate population of OFC neurons signaled reward when it was delivered for a correctly performed trial, and yet another population signaled rewards irrespective of when they were delivered (both and feedback only, Fig. 4c). Within OFC, roughly equally sized populations of neurons—about 15 percent of the task-related sample—responded to rewards in these three different categories (Fig 4c). Interestingly, a similar pattern of results was obtained in dorsolateral PFC, but the cortex of the frontal pole differed in that it only signaled reward as feedback (Fig 4c).

The presence of separable populations of OFC neurons signaling the occurrence of fluid reward delivery in these different ways is intriguing. Some neurons in OFC clearly signal the delivery of reward irrespective of its context, potentially suggesting a role for this population in conveying the subjective value of reward or potentially the specific identity of the reward. Neurons encoding reward only as feedback could serve to reinforce or extinguish previously learned associations, a process critical for updating associational knowledge and models of the environment. There is also evidence suggesting that reward-as-feedback encoding in OFC is task specific (Luk and Wallis, 2013).

Finally, neurons in OFC that signal instructional—as opposed to reward-related—information likely encode representations in addition to the “reward as instruction” signals described above. Encoding of task-related signals in OFC unrelated to reinforcement, such as the specific identity of sensory stimuli or the current abstract behavior-guiding rule, have also been reported in a number of studies in humans, macaques and rodents, but their importance has often received little emphasis (for example, Schoenbaum and Eichenbaum, 1995; Wallis et al., 2001; Wallis and Miller, 2003; Zelano et al., 2011). Instead, studies and reviews (including our own) have concentrated on the reward-related attributes of OFC activity (Morrison and Salzman, 2011; Rudebeck and Murray, 2011a). This was in part done to link the data to neuropsychological results and theories concerning the role of OFC in emotion (Rolls, 2000). The existence of non-reinforcement related signals in OFC is potentially important as it suggests a richer representation of the environment beyond reward. It adds weight to the hypothesis that OFC does not simply signal the attributes of a particular reinforcer or stimulus-reinforcer association but instead provides a map of all relevant attributes, related to reinforcement or not, needed to parse the environment (Wilson et al., 2014). It is worth noting that both task relevant and irrelevant signals have been reported in other parts of PFC in macaques, suggesting that encoding of the whole environment maybe a property of PFC areas in general, and not specific to OFC (Genovesio et al., 2014; Mann et al., 1988).

Despite its intuitive appeal, one issue with the idea that OFC represents all relevant attributes of the environment is that it does not readily provide an explanation of how certain sensory features become behaviorally relevant and therefore encoded by neurons in OFC. One possibility is that, in novel settings, all sensory aspects of the task are represented in OFC. Through learning, these representations are pruned until only those that have led to the desired outcomes will be encoded, an idea that has already gained some attention (Huys et al., 2012).

The origin of outcome-related signals

Neurophysiological and neuroimaging studies indicate the diverse types of neural signals present in OFC, but they do not indicate how these signals arise. Understanding the source of these signals could prove to be important for future attempts to influence OFC activity therapeutically. OFC receives connections from parts of the limbic system as well as from many of the different sensory modalities (Fig. 1, Price, 2005). Of all of the areas that interact with OFC, perhaps the amygdala has received the most attention. Interaction between these two structures is thought to be central to emotion and reward-guided behaviors (Gottfried et al., 2003; Holland and Gallagher, 2004; Murray and Wise, 2010; Salzman and Fusi, 2010; Schoenbaum and Roesch, 2005). Exactly how the amygdala contributes to neural signals in OFC remains unclear. The two available studies that have assessed the impact of amygdala damage on OFC outcome-related activity—one in humans and one in rats—offer differing perspectives, potentially due to species differences (Hampton et al., 2007; Schoenbaum et al., 2003b). In addition, studies that involve recording from both the OFC and amygdala in primates have emphasized opposing roles for these two structures in encoding stimulus–outcome associations for rewards and punishments (Morrison et al., 2011), with OFC encoding appetitive outcomes earlier than the amygdala and the opposite results for punishment.

To examine the amygdala’s contribution to outcome encoding, we recorded neural activity in lateral OFC (areas 11 and 13, Fig. 1 red shaded area) and a restricted part of medial prefrontal cortex (MFC, dorsal bank of the cingulate sulcus, corresponding to areas 9v and 24c), of macaques while they performed a choice task for fluid rewards (Fig. 5a) (Rudebeck et al., 2013a). Here, different amounts of fluid reward, which we take to be encoded as an attribute of the outcome, were assigned to the different stimuli available for choice. Recordings were made before and after amygdala lesions. When the amygdala was intact, similar to several previous reports (for example, Wallis and Miller, 2003), we found that many neurons in both OFC and MFC signaled reward or outcome-related aspects of the task, including encoding of the value of anticipated outcomes associated with external stimuli (S1 and S2, Fig. 5b), as well as the value of received outcomes (not illustrated). Neural activity associated with reward was, however, more prevalent in OFC than in the MFC. For instance, relative to MFC, more neurons in OFC signaled the amount of reward associated with the visual cues that the macaques chose, and they did so earlier.

Figure 5.

The effect of amygdala lesions on reward-value encoding in OFC. A) Trial sequence. On each trial monkeys were sequentially presented with two stimuli (S1 and S2) associated with different amounts of reward and were instructed to choose between them. Each stimulus was associated with a different amount of fluid reward and monkeys nearly always chose the stimulus associated with the highest amount. B) Spike density and raster plots illustrating the activity of one neuron in OFC that exhibited its highest firing rate to stimuli associated with smallest amount of reward. Each dot in the raster plot indicates the time that this neuron discharged. The color of the curves and dots in the raster plot correspond to the amount of reward associated with each stimulus, as noted in the key. Inset figures show the relationship between each neuron’s firing rate and reward value, within the periods after S1 and S2 stimulus presentation, respectively. C) The proportion of OFC neurons encoding the absolute value of S2, relative value of S1 (either higher or lower than S2), or interaction between these factors after the presentation of the second stimulus. D) Effect of amygdala lesions. Left: proportion of OFC neurons preoperatively (blue/turquoise) and postoperatively (red/orange) encoding the reward value associated with S1 and S2. Right: time course of stimulus-reward value encoding. Green dots indicate significant reductions in reward encoding after amygdala lesions. Red bar: duration of S1 presentation. Purple bar: duration of S2 presentation. Adapted from Rudebeck et al. (2013a).

Input from the amygdala to frontal cortex was removed by making complete excitotoxic lesions of the amygdala in both hemispheres. This led to a decrease in the proportion of neurons in OFC encoding outcome-related aspects of the task (Fig. 5d). It also altered the latency at which OFC neurons signaled the amount of reward associated with different options. By contrast, outcome-related encoding in MFC was largely unchanged. Thus, despite projecting to both MFC and OFC, the amygdala has a greater influence on outcome encoding in OFC.

Importantly, removing input from the amygdala did not abolish outcome-related activity in OFC, but merely led to an approximately one-third reduction. Based on the work of Tsujimoto and colleagues, discussed previously, it could be that amygdala input is critical for the cells that signal reward irrespective of its behavioral context, a possibility that could be tested empirically. These data also indicate that the amygdala is not the sole origin for the outcome or reward information being signaled by neurons in OFC. Although the specific task parameters and experimental design could have attenuated the decrement that we observed—for example our monkeys were highly familiar with the choice options—it seems likely that other parts of the brain play an integral role in OFC outcome signals. Hypothalamus, inferotemporal cortex, perirhinal cortex, dopaminergic neurons in the midbrain and parts of the thalamus all send projections to OFC (Goldman-Rakic and Porrino, 1985; Ongur et al., 1998; Saleem et al., 2008; Williams and Goldman-Rakic, 1998) and each could provide information related to potential reinforcement or outcomes.

Projections from the basal forebrain also target OFC (Kitt et al., 1987). Like input from the amygdala, inputs from the basal forebrain may influence outcome-related signals and, ultimately, the encoding of specific goals for action in OFC. In rats, bursting activity in basal forebrain neurons in response to behaviorally relevant stimuli is time-locked to task related changes in local field potentials (LFPs) in OFC (Nguyen and Lin, 2014). In addition, electrical stimulation of the basal forebrain elicits alterations in these LFPs, suggesting that neurons in the basal forebrain directly contribute to OFC encoding of behaviorally relevant stimuli. Although input from basal forebrain neurons is unlikely to directly influence the encoding of specific goals or outcomes in OFC (Croxson et al., 2011), temporally discrete signals from this region might serve to heighten the salience of specific options during evaluation and goal selection in OFC. Alternatively, it may support plasticity for behaviorally relevant associations, including stimulus-reward associations, during learning.

These findings highlight the fact that many brain structures contribute to encoding specific outcomes in OFC, including the amygdala. Determining the contribution of each of OFC’s inputs will undoubtedly be important for gaining a mechanistic and systems level understanding of this area’s function. We think this knowledge is important and has the potential to guide therapeutic interventions aiming to affect OFC processing.

Comparison and choice signals

When making a choice, the costs and benefits of a number of alternatives must be weighed before a final selection can be made. In some situations, the presence of alternative options can influence the valuation of the option currently under consideration (Tversky and Itamar, 1993). Given this, we would like to know whether neural signals in OFC are influenced by the relative value of alternative options presented for choice as well as the current option. If only the latter, then OFC’s role in choice would be strictly evaluative. If both, then its role would include relative and evaluative functions.

As already indicated, neurons in lateral OFC (Fig. 1, red shaded area) encode the predicted subjective value of external stimuli (Kennerley et al., 2009; Padoa-Schioppa and Assad, 2006; Roesch and Olson, 2005; Tremblay and Schultz, 1999). In many studies of macaque OFC in choice behavior, two or more choice options are revealed to the subject at the same time. This aspect of the experimental design makes it difficult to disentangle neuronal responses related to the valuation of each individual option. To get around this problem, a few studies have either held the value of one option stable while varying the other (Padoa-Schioppa and Assad, 2008) or they have sequentially presented the options for choice, allowing option values to be dissociated temporally (Luk and Wallis, 2013; Rudebeck et al., 2013a; Wallis and Miller, 2003).

In one of these studies, we sequentially presented macaques with two rewarded options, each associated with a different amount of fluid reward (Fig. 5a)(Rudebeck et al., 2013a). While monkeys were performing the task, neural activity was recorded in lateral OFC. To determine whether reward-related signals in lateral OFC were influenced by the context in which they were presented, we looked at whether encoding of the second option’s (S2) reward value was influenced by that of the first (S1). Although a small population of neurons in lateral OFC (about 8 percent) signaled the reward associated with S2 in a manner that was dependent on S1, a much larger proportion of neurons (about 25 percent) signaled the reward value of the S2 independently of S1 (Fig. 5c). Thus, neurons in lateral OFC predominantly provide an assessment of each individual option independently of the others available.

If OFC signals are critical for making choices, then variations in the activity of OFC neurons should be correlated with choice behavior. Padoa-Schioppa (2013) has examined this issue for alternatives of similar value. Previously, he had characterized three different types of neurons in lateral OFC: “offer value” neurons that encode the value of the two offers presented, “chosen value” neurons that signal the value of the best or chosen option, and “chosen juice” neurons that signal the juice type (or juice identity) associated with the chosen option. Of these, chosen-juice neurons seem to be most specifically related to the outcome of the trial, as opposed to evaluating alternatives. Surprisingly, Padoa-Schioppa reported that slight fluctuations in either the pre-trial or within-trial activity of chosen juice neurons, but not offer-value or chosen-value neurons, correlated with small changes in choice behavior. Later, we take up the finding that it is chosen-juice neurons that show this correlation. For now, we simply note that this study provides evidence that neurons in lateral OFC participate in the process of choosing among options.

Evidence from human neuroimaging, macaque lesion and neurophysiology studies suggest a different view of OFC functional organization, one that questions the role of lateral OFC in choices per se, at least for the part of choice that involves comparisons and final selections. Several studies suggest that whereas lateral parts of OFC are important for evaluating options (Rudebeck and Murray, 2011b; Walton et al., 2010), its more medial parts are engaged when options have to be compared and selected (Boorman et al., 2009; FitzGerald et al., 2009; Hunt et al., 2012; Noonan et al., 2010b; Rudebeck and Murray, 2011b). Note that by medial OFC, we refer to the areas shaded in purple in Fig. 1, and not to the areas called MFC above.

The results of a recent neurophysiology study by Strait et al. (2014) adds weight to the idea that value comparison occurs in the medial OFC. In their study, monkeys chose between two sequentially presented choice options, each associated with different amounts of reward and different probabilities that the reward would be delivered. Neurons in medial OFC signaled the value of the two offers as they were presented using a “common currency” value scale to integrate reward size and probability. Notably, after the presentation of the second option, neurons encoded the value of the options using an inversely correlated encoding scheme. Opposing encoding schemes for the two options indicates that the representations of the two offers may be mutually inhibiting each other. Furthermore, after the presentation of the second option, neurons rapidly transitioned to signal the value of the option that would subsequently be chosen. This type of inversely correlated encoding contrasts with the type of signals seen in lateral OFC, where options are encoded largely independent of each other (Fig. 5c) and use correlated encoding schemes (unpublished observations). Although these findings are intriguing, we note that only a small proportion of neurons in medial OFC (about 10 percent) were found to have these properties, and that neurons in other brain areas were not examined using the same methods (Strait et al., 2014).

In addition to providing evidence for a role for medial OFC in guiding choice, the findings of Strait et al. are consistent with the idea that there are two functionally distinct modules in OFC, one for identifying and signaling the specific characteristics of different outcomes associated with objects or actions and one for comparing these options to select a goal for action (Rudebeck and Murray, 2011a; Rushworth et al., 2011). How the comparison process in medial OFC influences action selection, the next step in the process of attaining ones goals, is an issue we take up later.

Even if we accept the idea that medial OFC plays the largest role in choices and lateral OFC does so for evaluations, this still leaves the question of why the activity of chosen-juice neurons in lateral OFC varied with choice behavior and the activity of chosen-value neurons did not. One possibility is that the fluctuations in activity in these neurons reflect feedback coming from medial OFC about the result of the comparison process. Indeed, this explanation was proposed by Padoa-Schioppa (2013) and, as he noted, would help to explain why pre-trial activity was predictive of the subsequent choice. One way to test this idea could be to record the activity of chosen-juice neurons without input from medial OFC, using temporary interference methods, such as pharmacogenetic inactivation, to specifically suppress the activity of neurons in medial OFC that project to lateral OFC.

Summary: What information does OFC process?

It is commonly accepted that OFC processes information related to reward value in some way. Although this idea has contributed importantly to understanding OFC, recent results point to the need for more specific concepts. OFC neurons signal reward in multiple ways, which depend on the level of association (Fig. 3), the specific sensory properties of the reward, including its instantaneous subjective value (Fig. 2d, 5b, 5d), and the instructional information it conveys (Fig. 4). These signals are not homogeneously distributed within OFC (Fig. 3), which could account for some of the discrepancies in the literature. Converging lines of evidence point to a role for lateral OFC in evaluating options, fairly independently of each other, and medial OFC in comparing and contrasting them in order to make a choice.

How does OFC influence behavior?

Outcome signals: dopaminergic prediction errors

For any organism to learn, a prediction of the outcome associated with an action has to be compared to the actual outcome obtained from making that action (Sutton and Barto, 1998). The difference or “prediction error” can then be used to update expectations and drive learning. Dopaminergic neurons in the ventral tegmental area (VTA) and substantia nigra pars compacta are thought to provide one such prediction-error signal for reward-guided learning (Schultz et al., 1997), but where do the dopamine neurons get information about predicted outcomes?

To address this issue, Takahashi and colleagues investigated whether OFC serves as a source of the prediction signal for VTA neurons (Takahashi et al., 2011). Recordings were made from putative dopaminergic projection neurons in the VTA of rats with ipsilateral OFC lesions (i.e., unilateral OFC lesions in the same hemisphere as the VTA recordings) while the rats performed a behavioral task designed to generate rewardrelated error signals (Fig. 6a). This task had previously been shown to be dependent on both OFC and VTA (Takahashi et al., 2009). Without input from OFC, dopaminergic neurons in the VTA did not exhibit the usual pattern of error-related firing when rats’ expectations about potential rewards were violated. Specifically, the initial increase or decrease in activity in dopamine neurons following unexpected reward delivery or omission, respectively, was attenuated in rats with OFC lesions relative to controls (compare Fig. 6a and b). Furthermore, unlike in controls, the increases and decreases in activity to unexpected changes in reward did not change through repeated experience in rats without an OFC. This can be appreciated by comparing activity on trials immediately after a block switch relative to later in the block (Fig. 6a First 10 vs. Last 10). These data indicate that without input from OFC, prediction error signals conveyed by the firing of VTA neurons are not being updated through experience—a key requirement for any associative learning error signal and model updating in model-based reinforcement learning. Moreover, the contribution of OFC to VTA signals appears to be distinct from that of other frontal areas. Inactivation of MFC in rats produces a different pattern of effects on VTA dopaminergic activity (Jo et al., 2013).

Figure 6.

The effect of OFC lesions on reward-prediction errors in dopaminergic neurons of the ventral tegmental area (VTA). Mean (±SEM) firing rate (spikes / s) of reward responsive VTA neurons in sham-operated (A) and OFC lesion (B) rats following unexpected reward delivery or omission immediately (First 10 trials) and later (Last 10 trials) after a change in the reward contingencies in the task (Block switch). Without input from OFC, VTA neurons do not display the normal patterns of “prediction error” signaling. Dark solid lines represent responses to unexpected delivery of reward. Gray solid lines represent responses to unexpected omission of reward. Gray dashed lines represent baseline firing. Arrows highlight differences in the VTA neuronal responses in the first and last 10 trials after block switches in the sham-operated and lesion rats. Adapted from Takahashi et al. (2011).

In the same study by Takahashi and colleagues, computational modeling of neural activity of VTA neurons in rats that lacked an OFC was consistent with the idea that OFC provides VTA neurons with a specific prediction of the expected outcomes of potential choices, which the authors called a state. Specifically, models where the OFC signaled the current identity of the outcomes or states, not the value of an outcome, most closely matched the firing rate of dopamine neurons after OFC lesions. Thus, the data suggest that signals regarding the current set of specific, predicted outcomes from OFC are used by the dopamine neurons to compute reward prediction errors, which, in turn, are used to promote learning in other brain areas. Indeed, through feedback connections from the VTA to the frontal cortex, dopaminergic error signals likely affect the updating of outcome-related and other model-based signals in OFC, as well as in other parts of the frontal cortex that, like OFC, provide predictions about the future state of the internal or external environment.

Choice signals: action selection, lateral frontal cortex and striatum

While potential alternatives for choice are being evaluated and compared in OFC, how might these signals influence the selection of actions in order to achieve the desired goal? Recent neurophysiological evidence suggests that one way OFC might affect processing in brain structures that control action planning, for example premotor cortex, is through a mechanism similar to top–down attention and biased competition (Pastor-Bernier and Cisek, 2011). In primates, an influence of OFC on premotor cortex could be mediated by connections between OFC and ventral, medial and dorsolateral PFC (Barbas and Pandya, 1989; Carmichael and Price, 1996; Saleem et al., 2013), which contribute multisynaptic pathways to dorsal premotor areas (Takahara et al., 2012).

A systematic analysis of the neurophysiological signals along the pathway(s) from OFC to premotor cortex has yet to be conducted. However, recordings of neural activity in OFC and lateral PFC appear to support the idea that predicted-outcome signals from OFC influence attentional and action processing in ventral and dorsolateral PFC (Cai and Padoa-Schioppa, 2014; Kennerley and Wallis, 2009a, b). For example, in the study by Tsujimoto et al. (2011), activity in OFC specified the strategy to be used for any given trial significantly earlier than did neurons in dorsolateral PFC. Yet neurons in dorsolateral PFC, but not OFC, encoded the location of the upcoming response (left or right). Thus, the timing of these signals indicates that outcome predictions generated in OFC may guide action selection in premotor cortex via dorsolateral PFC.

Direct projections to the striatum are another route by which OFC might bias the selection of actions. OFC densely innervates the ventral and medial striatum (Haber et al., 1995) and interaction between the OFC and distinct parts of the striatum regulates habitual vs. goal-directed behavior in mice (Gremel and Costa, 2013). Taking a mechanistic approach, two optogenetic studies in mice have recently shown that stimulation of OFC projections to the striatum can bias action selection in different ways. In the first study, Ahmari et al. (2013) used virally transfected channel-rhodopsin to stimulate glutamatergic neurons in medial OFC. Selectively enhancing the firing rate of OFC projection neurons to striatum in one hemisphere induced repetitive behaviors. Interestingly, the increase in repetitive behaviors was only observed after repeated stimulation over days, suggesting that the medial OFC’s influence on action control through the striatum is indirect or requires temporally slow adaptations in striatal microcircuits to occur.

In the second study, Burguiere et al. (2013) revealed that selectively enhancing the activity of lateral OFC projections to the striatum in a genetic mouse model of obsessive-compulsive disorder (OCD) alleviated compulsive grooming. Careful inspection of the relevant neural circuits revealed that OFC projections influenced action selection by altering the balance of inhibition and excitation in striatal microcircuits.

Taken together, these studies reveal that OFC can influence action selection by altering processing in the striatum. In addition to taking a mechanistic approach to understanding the influence of OFC, these studies indicate that anatomically distinct parts of OFC play different roles in biasing actions. Whether OFC-striatal interactions in primates influence action in a similar manner is an exciting avenue for future investigation.

Summary: How does OFC influence behavior?

In primates, at least, OFC is not connected to structures that directly affect movements, such as either motor or premotor areas, so it must influence behavior through interaction with other areas. A circuit-level understanding of OFC function helps elucidate its contribution to behavioral choice. Recent work has highlighted some of the routes via which the OFC influences behavior, including through its interaction with the basal forebrain, dopaminergic neurons in the VTA and the striatum.

Summary and conclusions

Despite the complicated design of the Antikythera mechanism—a product of the thousands of years of astronomy and engineering—discerning its inner workings and function was cracked within a century once the tools became available to do so. Likewise, OFC has resulted from millions of years of evolution that have made it an astonishingly complicated gismo. We are only beginning to understand its functions, but by using some new tools, along with older ones, we believe that discerning its inner workings will help unravel its functions. Here we adopted this mechanistic approach, one emphasizing the activity of individual neurons in OFC and an understanding of their causal contributions to behavior.

As we have seen, OFC, especially lateral OFC, signals the outcomes that are likely to occur after the choice of an object or an action. Importantly, these outcomes are not encoded in a featureless manner in OFC. Instead, the constellation of sensory features associated with potential outcomes is encoded as a representation of outcome identity, along with its biological value at any given moment. Thus, the function of OFC resembles that of the Antikythera mechanism in that both devices predict the future, which is why our title refers to OFC as an oracle of sorts.

Such outcome-predictive signals in lateral OFC result from a wide array of inputs, only some of which depend on input from the amygdala. Available evidence from primates indicates that once lateral OFC evaluates potential alternatives, medial OFC then compares and contrasts them using a common currency of universal value. While it seems likely that comparison signals in medial OFC contribute to the selection of actions, lateral OFC may also contribute to this process, especially when a choice is aimed at outcomes with particular features, specific spatial locations or particular contexts, either alone or in combination.

A mechanistic approach to elucidating OFC function aims to characterize how its specific patterns of neural activity result from distinct inputs, especially those related to the updated valuation of specific reinforcers and outcomes. We think that this type of knowledge is important for at least two reasons, both of which involve understanding the function of specific brain circuits, including both areas and pathways. First, it addresses not only how specific pathways produce predictive signals in OFC, but also how, in turn, OFC influences downstream structures to bias the choice of action and compare predicted signals with those that actually occur. Second, this type of knowledge will be critical for understanding disorders that involve dysfunction of OFC, such as major depressive disorder, OCD and substance abuse (Murray et al., 2011).