Abstract
We propose that the entirety of the prefrontal cortex (PFC) can be seen as fundamentally premotor in nature. By this, we mean that the PFC consists of an action abstraction hierarchy whose core function is the potentiation and depotentiation of possible action plans at different levels of granularity. We argue that the apex of the hierarchy should revolve around the process of goal-selection, which we posit is inherently a form of optimization over action abstraction. Anatomical and functional evidence supports the idea that this hierarchy originates on the orbital surface of the brain and extends dorsally to motor cortex. Accordingly, our viewpoint positions the orbitofrontal cortex in a key role in the optimization of goal-selection policies, and suggests that its other proposed roles are aspects of this more general function. Our proposed perspective will reframe outstanding questions, open up new areas of inquiry and align theories of prefrontal function with evolutionary principles.
This article is part of the theme issue ‘Systems neuroscience through the lens of evolutionary theory’.
Keywords: hierarchy, orbitofrontal cortex, premotor cortex, motor cortex, dynamical systems
1. Introduction
As we move around the world, our bodies engage in small movements that are often unrelated to the task at hand. An important recent study shows that, in mice, these small movements account for a large amount of the explainable variance in firing rate of neurons ([1]; see also related findings in [2,3]). These effects were found not just in motor cortex, but, surprisingly, across the entire brain. These findings came about as a result of the careful measure and registration of the full suite of animal behaviour; previous studies that did not measure these movements would have treated them as a source of noise to be ignored. Overall, these results highlight the importance of motor control for the brain as a whole.
It is fascinating how much of the neural response is determined by seemingly unimportant motor activity. Likewise, it is surprising—and humbling—to see the extent to which the cognitive variables that are central to so many models of cognition wind up being relatively small factors in determining the firing rates of neurons [1]. Despite decades of debate about how these regions differ functionally, when we consider factors that drive firing rates the most, these regions turn out to largely have the same function when measured this way. This is not to say that these results support mass action theories. However, they invite us to ask whether studies that focus on differences in brain areas are ignoring the much larger common factors that drive all the regions.
Indeed, from another perspective, these results should not be too surprising. After all, the brain exists, first and foremost, to control behaviour [4–6]. This perspective is found as far back as the work of Sherrington, who argued that ‘Life's aim is an act, not a thought’ [7, p. 201]. From that perspective, the brain's other functions, including the ones that correspond to the chapters of any cognitive neuroscience textbook (attention, reward, memory, executive function, etc.), are there to influence action. If they do not influence action, they are otiose. And if they do influence action, they are minor modulatory factors for the expression of action. What is most surprising is how large the motor effects are even in regions with no obvious motor role. Which suggests that, when factoring in full motor behaviour, so much of the brain has a motor or premotor role. In other words, these results advocate for the primacy of motor expression for understanding neural activity, not just in motor regions, but brainwide. They suggest, at least to us, that motor activity not only accounts for much of the variance in neural firing but serves as the organizing structure for the rest of our mental activity. They also cohere with an evolution-centric view of functional neuroanatomy—anything that does not advance the cause of driving behaviour should not last long as a major part of the brain's repertoire [8].
Here, we make the argument that the premotor perspective is a useful vantage point for thinking about the functional organization of the prefrontal cortex (PFC). While our arguments may apply beyond the PFC, we limit ourselves to that part of the brain because (i) its functions have for long resisted overall theoretical integration, and (ii) aside from the cited papers, the evidence for our claims is much stronger for the PFC than it is for earlier regions, where it is, in our view, mostly theoretical. Beyond these two points, the PFC is our area of scholarly interest, and of interest to people interested in higher cognitive functions and their dysregulation in psychiatric diseases. We will make a special focus on the orbitofrontal cortex (OFC), largely to emphasize the contrast between our position and more conventional theories of PFC function. The OFC, being the most hierarchically distant from the frank motor structures, shows the greatest explanatory difference in our accounting. In making our arguments, we build on the idea that while hierarchical theories of PFC functions have espoused the impact of rule optimization, they have marginalized out the process of selecting goals that situates rule optimization in the first place.
2. The whole prefrontal cortex is a premotor structure
We believe that taking seriously the primacy of motor expression in driving brain activity has important implications for systems neuroscience. In particular, we think that this view can help organize understanding of the ever-mysterious PFC [9–12]. This large portion of the brain is typically associated with non-motor cognitive processes, such as executive function and control, as well as working memory, inhibition, learning and maintaining and switching task set [9,11,13–15]. Note that we are not arguing that other theories of PFC are incorrect. Our ideas, outlined below, will be speculative and need data to support them. Moreover, we believe that full understanding of a structure as complex as PFC benefits from multiple perspectives—all of which, including our own, have limitations. However, we believe that these views can be felicitously augmented by considering the fundamental role of the PFC in driving or setting the stage for action, and seeing the proposed roles of its constituent regions through that lens.
This viewpoint is part of a larger view that advocates for thinking of cognition as an extension of action selection, not as something wholly separate from it [16–22]. We humans (or other animals) move through the world and happen to come upon things that interest us. Those things that we encounter are associated with specific actions. From an economic perspective, the relevant action would be selection; in foraging theory, it would be pursuit or handling [23]. In Gibsonian psychology, the roughly analogous concept is that we encounter options that activate an affordance associated with selection [19,24–26]. Each potential action can either be performed or not performed. If a potential action rises to the level of consideration, the brain gathers available evidence, filters it for relevance, and uses that to militate for or against performing the action. The brain uses the same accept–reject principles for both trivial decisions and serious ones like choosing whether to buy a house or marry a partner [27–30]. The processes that increase or decrease the likelihood of performing an action wind up guiding the selection of actions, and are therefore—in a non-trivial sense—premotor. Because guiding these decisions is the chief function of the PFC, the whole PFC can validly be called premotor cortex (PMC).
3. Wait, what is premotor cortex again?
PMC, as the term is traditionally used, is defined by its relationship with motor cortex [31,32]. Motor cortex is, of course, cortex whose chief function is the regulation either by planning, modifying or executing movements, or some combination of those [33–35]. PMC is thought to have a more abstract and high-level motor function, serving a more regulatory or supervisory role [32,36,37]. The term PMC was originally coined by Hines [31], owing to its adjacent position to and connectivity with motor cortex. This connectivity implies that the PMC is hierarchically earlier than the motor cortex. The major function of PMC is presumed to be to set the stage for motor cortex by potentiating certain motor plans and depotentiating other ones [38,39].
The idea that PMC regulates higher-level motor plans rather than enacts them (e.g. by signalling spinal motor neurons) was first shown by Woolsey [40], who found stimulation of PMC did not produce movements (see also [41]). Later studies showed though that movements can be evoked by premotor stimulation, but are often more complex (e.g. whole hand grasping) those elicited by stimulation of primary motor cortex (e.g. specific muscle innervation; [42]). The notion of PMC as biasing downstream action execution is bolstered by its larger preoccupation during movement planning compared to online execution [43]. A high-level action regulatory role for PMC is supported by recordings showing preparatory activity [44] for reach direction specificity [43], encoding of multiple possible motor plans rather than a singular action [45], switching between action plans and during online control [37,46,47].
These findings indicate that the PMC has an ancillary motor function. That is, while it does not directly drive muscle specific responses, it plays an invaluable function: it sets the stage for action by making some actions more or less likely. In other words, it influences motor function by potentiating or depotentiating actions. In summary, then, the traditional view of the PMC is that, anatomically and functionally, it resides at the first level of what we and others (reviewed next) argue is an action abstraction hierarchy. The essence of our argument is that other prefrontal regions extend this hierarchical control of action and can also be described similarly to premotoric terms as (de)potentiating abstractions of action. In other words, other prefrontal regions do not differ from PMC in kind, just in hierarchical level.
As we will argue below, the functions of these other regions, including their economic ones, can be explained, at least in part, by their premotor role. The major difference between PMC and other prefrontal regions is that the latter are anatomically and hierarchically earlier, and the actions they deal with are probably more abstract, more tangled and more aligned with sensory input features than classically defined PMC [48].
Our view is related to, but distinct from, the philosophical position that the entire brain is a premotor structure. That view is predicated on the fact that all brain activity is aimed at driving behaviour, either directly or indirectly. From that view, even the retinas are premotor since high level form vision is there to identify items in the scene and drive relevant action, so action is just an untangled form of retinal processes. To give another example, in the case of forming long-term memories, the relevance to behaviour is extremely indirect, but is nonetheless eventually action oriented. While we have some sympathy with this viewpoint, the point we are making is narrower. Even if non-PFC regions do guide action, their guidance may be so specialized and indirect that it is more useful to think of their role in terms of that specialized function. For example, a face-detection neuron may ultimately serve the purpose of helping to decide what to do when that face comes into view, but it is more useful to declare it a face detector. We believe the same logic does not apply in PFC, however. Thus, as we will argue, one could think of OFC neurons as value encoders, but this is a less useful and convenient framework for thinking about them than thinking about their premotor roles. In other words, we propose that it is useful to think of the PFC as a premotor structure even if we are not willing to think of the entire brain as a premotor structure.
The PFC has certain features, especially in its integration of information from multiple sources, that make it convenient to start there. We can simultaneously accept two points: (i) form vision is very useful for action selection, and (ii) evolution has apparently selected for a specialized dedicated visual system that serves the purpose of encoding visual form. This second point is critical. Apparently, it is a better design principle for visual inputs to converge and come to some consensus on form identity before integrating with other modalities, such as the visceral and olfactory systems. Regardless of the evolutionary reason, the visual system is conveniently thought of as a visual system—that is, it has a somewhat modular visual function. This is not to discount evidence for non-visual signals in the visual system, just to say that it has a strong bias towards visual function that other sensory cortical regions do not have. Other systems, for example, the brain's olfactory and auditory systems, may also have somewhat modular functions. The argument here is that PFC is not a modular system in the same way these ones are identifiable as modular. Instead, it reflects the convergence of multiple, more modular systems and serves as an important step in a hierarchy that produces action.
4. Hierarchies of action: abstraction, control and goals
In theorizing about the functions of the PFC as premotoric, our view is anticipated by Fuster ([14,49,50]; figure 1). Fuster viewed the PFC as part of a hierarchy oriented towards the control of action. He said, for example, that ‘the entire cortex of the primate's frontal lobe seems dedicated to organismic action. It can, thus, be considered, as a whole, ‘motor’ or ‘executive’ cortex in the broadest sense’ [49, p. 66]. Note that Fuster here uses the word executive in the sense of executing action as distinguished from sensation, not in the sense of a discrete and separate executive or supervisory system.
Figure 1.
Functional organization of the prefrontal cortex, as proposed by Fuster (e.g. in [14,49,50]). In this cartoon, the brain takes in information from the environment, processes it, and generates actions. The processing is hierarchical, and involves a gradual transformation from input to output. Critically, Fuster's proposal ignores a separate central executive.
Of course, Fuster does not mean that the entire PFC is an undifferentiated mass of one extended PMC. There are well described functional differences with the PFC, and there is a larger organization. To quote Fuster again, ‘much of the prevalent confusion in the PFC literature derives from two common errors. The first is to argue for one particular prefrontal function while opposing or neglecting others that complement it; the second is to localize any of them within a discrete portion of PFC’ [50, p. 319]. In other words, Fuster proposes that the core function of the PFC is motor control, that its organization is hierarchical, and that its regions differ in their position, not in their nameable function.
Interestingly, though, Fuster's focus was primarily on the lateral PFC. It may be that, when considering the orbital and medial prefrontal cortices, there is greater evidence for functional specialization. Unfortunately, aside from the dorsal anterior cingulate cortex, the medial wall of the PFC is less well-studied and less well understood than the lateral surface. Our laboratory's research has generally demonstrated functional continuity between orbital and medial structures (e.g. [51,52]). Indeed, our laboratory recently tested these ideas by comparing four medial wall structures in a single task, and found evidence for broad continuity of function, although we found evidence for a gradual gradient of function [53]. These findings suggest that the principle of hierarchy may apply just as well to medial as to lateral structures, and therefore may be a general principle of prefrontal organization.
Nonetheless, the notion of action and abstraction hierarchies already figures heavily into several theories of goal-directed cognitive control and decision-making. Generally, such hierarchical theories propose that areas within the PFC can be described as some type of abstraction hierarchy over action control, culminating in the motor cortex and interacts with basal ganglia circuits [6,54–58]. What differentiates these theories is how the brain deploys hierarchical abstraction to control behaviour after a goal (e.g. feed your friends) is already specified.
This idea of hierarchical control can be illustrated by considering the case of a person who is interested in cooking dinner for a visitor. That goal could be satisfied by any number of possible actions, meaning that successful cooking of the food can be accomplished through any number of specific body movements. And indeed, the higher-level choice (what to cook) can be implemented in multiple steps (which things to cook in which order), and each of those can be executed in multiple specific actions (turn left to grab the saucepan, etc.). So the decision about cooking is at a higher hierarchical level than the execution of the motor actions, although both are parts of control, broadly speaking. Indeed, planning and decision-making takes place at more levels than this—they involve a whole series of levels, including even more abstract ones, like whether cooking is best, or ordering takeout might be smarter.
Indeed, the ideas of abstraction and hierarchy for action already figure heavily in many theories of cognitive control and decision-making in PFC. Perhaps the most influential framework posits that abstraction hierarchies in PFC can be decomposed into different types of cognitive operations [9,54,59]. These operations include abstractions over temporal information, schemas or states and policy abstraction. Policy abstraction is most directly related to action control and aligns with the motor hierarchy in Fuster's [49] conception [60]. Policies in this framework are rule-based mappings that are contextualized by more general rule-based policies. For example, having the goal state of entertaining a new person will map to the higher-order action of ‘cooking for a date’ is an example of policy or state-action mapping. A high-level goal like ‘cooking’ is an example of an abstract policy that generalizes over lower-order policies constrained by contextual information regarding what to cook; context here could be a person's dietary preferences. In this example, policy abstraction links states and goals to potential action, with higher-order contexts like dietary preferences determining lower-order conditions for actions (e.g. make vegetarian not meat dish). Essentially, the least abstract policies during cooking a meal might directly map from states like ‘in front of a heated stove’ to an action of ‘grasp pan with right hand’. The states driving a that map policy actions are essentially a set of features containing information that affect the choice made, such as the person being cooked for combined with knowledge that they enjoy being cooked for is a state [61].
While these ideas of action hierarchy have enjoyed success in describing behaviour, there are several debates surrounding how to define a hierarchy. One organizing principle is driven by determining which cognitive function, such as temporal, schema or policy (rule) abstraction is more higher-order [9]. Another approach is defining it based on anatomical or functional connectivity [62]. This has led to disagreement about whether mid-dorsolateral PFC, rostrolateral PFC or even ventromedial PFC (vmPFC) resides atop the purported PFC control hierarchy. However, to preface our current proposal, we suspect an issue with previous theorizing is they have primarily focused on how aspects necessary for behavioural control are implemented after goals are already specified. Goal-selection processes itself are not factored into these extant ideas. Notably, we will argue that both goal-selection optimization is indeed a higher-order function in the hierarchy that contextualizes all of these functions, and this process is the province of OFC, which has connectivity with all of the aforementioned areas.
Therefore, a formative question raised by these previous theories and others relates to how the goals that are directing behaviour are themselves controlled or selected [63,64]. Building on this need to address the origin of goal-selection, recent ideas on hierarchical action gradients have argued, for example, that either the OFC (including vmPFC; [65,66]) or frontal pole (Broadmann area 10) may be involved in either selecting, maintaining or distributing abstracted goal information (e.g. get food) to other cortical regions [13]. However, these frameworks have largely considered how goals are maintained (e.g. working memory, [9,66]), while the issue of the decision processes underlying goal-selection still awaits further elaboration.
In line with the above ideas on hierarchy and control, we propose that consideration of the homeostatic and motivational drives of behaviour naturally leads to the idea that goal-selection is also an action decision-process that resides atop the PFC abstraction hierarchy. Notably, the motivations for our proposal closely align with Fuster's conception of motor hierarchy and PFC [14,49,50]. Key to our idea is treating goal-selection as a premotor action policy.
5. Goal-selection as driver for policy abstraction
We now turn to delineating our theory in more concrete detail. We are also proposing that PFC is, fundamentally, an abstraction hierarchy. That is, each area moving up the hierarchy has a map between states of different kinds, and progressively more abstract actions. Choice is not just ‘choosing left versus right’ or rules of how to attain goals, but also choosing what goals to follow, goal-selection. And, the difference between these is one of level, not of kind. The PFC as a whole serves the orchestration of goal-directed action (the actions one should take once the goal is specified) as well as the goal-selection (which goal is to be selected). While we take direct inspiration from the hierarchical theories reviewed in the previous section, the novel element of our proposal is that hierarchical abstraction of action policies be extended to include goal-selection as the highest level of the process. We will argue, below, that this can be linked to the OFC.
As an entry point to our idea, consider a descriptive example of a policy. For example, a person walking to work might come to a changing cross-walk light while simultaneously realizing they are already late for work. The individual could arbitrate between running across anyway or biding their time until the cross-light comes on again. All of these events and stimuli (features) constitute a state. Formally, states serve as a summary of all features that affect the choice of actions. Features could include the overall goal of going to work, relative time since the light changed and your distance, traffic conditions, or the emotional or physical cost of being late for work. The cost of being late could outweigh concerns for safety, for example, pushing an individual to cross. In this scenario, the policy is the mapping from how the states (the risk of getting hit, the cost of getting hit, the cost of being late) guide the person in deciding which action (cross or wait) to take.
Having an example of a policy at work, we can define the policy of our example as P(Action|State). The notation for the policy is read as the probability of an action conditioned on the state. The above example of street-crossing captures the idea that crossing or waiting is conditioned on the state and the goal of getting to work. This example accords well with what most theories mean by policy: action is almost always defined in terms of specific physical variables or action rules, e.g. reach or walk if the light is green but not red. Thus, while these policies operate in service of the goal of getting to work, the goal is already defined, and is merely an input that serves lower-level actions.
In our view, though, our imagined person entertains policies that are richer and more abstract than rule-based action mappings that are subservient to a pre-specified goal of ‘go to work’. For example, our person might have flexibility in their choice of daily schedules, such as the job allows them to decide whether to work or forgo it in favour of a pleasurable activity such as kayaking. In totality, we argue this higher-level choice exemplifies how the action of selecting among (potentially competing) goals is an abstract action policy, in line with the premotor notion of (de)potentiating different action.
A first step in defining what this goal-selection policy would look like is defining the relevant state inputs (or, equivalently, motivators). (A very detailed overview of how motivators serve goal-selection is found in [64].) Goal-selection is moulded by things a decision-maker wants or desires (e.g. money or pleasure), the external environment (e.g. opportunities during nice weather), pre-existing goals (e.g. work deadlines or dieting) or their homeostatic needs, such as hunger or thirst. These motivating states are often subject to depletion, meaning that a decision-maker must often prioritize goals that fulfil needs and balance resources based on their expected future depletion levels [63,67,68], and demarcate between wants and needs in goal-selection [66,69,70]. Consequently, the evaluative processes underlying goal-selection dynamics are highly context-dependent, and driven by the needs for resource uptake (e.g. food or money), as well as desires [63,68,69,71].
A major implication of the idea that motivation is often resource dependent is that optimizing goal-selection policies requires an interaction with planning policies. In other words, goal-selection must be future oriented. Anyone who has failed to anticipate their impending hunger, and waited until they were hungry to go to a restaurant with a long-wait has experienced the consequences of failing to plan around these dynamics. One thing to keep in mind is that we probably do not pick a goal and then plan; instead, these things occur simultaneously, and interactively [63,72]. The value of pursuing a particular goal (or multiple goals simultaneously) depends on the availability and feasibility of plans given different constraints. Interactions between planning processes and goal-selection dynamics are thus imperative for ensuring an agent ends up in future states where goals are met in a reasonable time, have trackable progress, and provide sufficient replenishment of resources [66]. Notably, the interaction we are referring to is different from the lower-level isolated process of just planning an action, for example, to reach an already selected target like a coffee cup. The aim of these planning and goal-selection interactions is achieving future states (e.g. satiate hunger) rather than future movements (e.g. hand on cup). The importance of this is that such a system allows anticipating desired states along the way to a goal (goal posts), and how to monitor, correct and replan for deviations away from those states.
Planning interactions, then, is another way in which goal-selection is essentially premotor. To see this, we can compare the general algorithmic nature of planning for goal-selection and sensorimotor control [73]. The two are identical algorithmically. Computationally, optimal planning of actions for achieving goals typically involves learning of a world model or connections between states and using knowledge of the states that satisfy goals to plan actions accordingly. For an agent to optimally plan while minimizing the distance and resources used, they must start planning from a goal satisfying state wherein they know needs or goals will be met, and move backwards to the person's current world state [63,74], e.g. imagining the path from work to home as taken by walking versus driving. This same algorithmic approach has been used to successfully describe sensorimotor control, such as planning motor dynamics to reach to a target [73,75]. These ideas indicate a conceptual and algorithmic overlap between planning for motor control policies and goal-selection policies, blurring the distinction between abstract goal-selection policies and sensorimotor control policies. As a proof of principle, these types of goal-selection dynamics have recently shown to be viable in a biologically realistic neural network [63].
These types of computations for policy optimization are often quite computationally complex—potentially beyond the limits of our brains to implement. Information-processing in the brain is inherently capacity-limited and noisy [76,77]. Owing to these informational constraints, goal-selection cannot be optimized in an error-free manner [78], due in part to a combinatorially high-dimensional state (and temporal) space that goals can be achieved in, uncertainty in whether pursuing a goal will render the desired outcomes, or the error-proneness of complex plans for goal achievement [79]. We suspect that understanding how individuals deal with these constraints will be necessary to elucidate the dynamics of goal-selection. This issue can be usefully reframed as asking how individuals' trade-off between the complexity of a goal-selection policy and plans with the potential benefits or needs of satisfying certain goals.
Several testable predictions emerge when applying these frameworks to goal-selection policies, two of which we consider here. A notable prediction is that goal-selection should exhibit a trade-off between (i) the urgency, amount, and quality of resources gained to fulfil a need, and (ii) the temporal (or distance) and the state-space complexity of a plan for achieving it. This distinction has been experienced by anyone who knows cooking something would be healthier than ordering delivery, but cooking is more state-complex than picking up the phone to order and wait for delivery. Another prediction is that the imperative for compressing the policy is that individuals can learn to abstract over goal-fulfilling states. For example, our person walking to work may represent the world at different levels of abstraction depending on need and desire—if thirsty, they may classify shops into ones that can provide a drink or not; if thirsty and hungry, they may instead classify shops into ones offering drinks and food or not. The extent of goal-fulfilling state abstraction should play a direct role in whether goals are separated or merged. The point we want to convey in discussing this framework and examples is that goal-selection policies are not merely an abstract thought exercise. They represent a plausible component of action abstraction hierarchies, fit the notion of premotor, and have empirically testable predictions that are grounded in extant theories of optimizing decision-making [73,79,80]. Furthermore, while we propose that goal-selection and its interactions with planning processes are key, there already exist modelling frameworks that have attempted to quantify what form these costs might look like [81–83] or explain how such controlled planning could benefit an agent [84]. An open question is how these purported costs and optimization objectives predict a wide variety of behaviours across different goal-selection behaviours.
Finally, given the above, we want to specifically delineate the differences between policies for goal-selection and those for rules. As standard in cognitive control frameworks, rule policy abstraction is about linking states to rules governing action. Rule policy abstraction optimizes different information than policies for goal-selection, where the latter we are concerned with optimizes the goals to pursue. In this view, rule or policy abstraction is indeed subservient to goal-selection. Patently, the policy over rules an agent will entertain will depend on the agent's world state. Imperatively, both (i) the state the agent will end up in in the first place will depend on their goal, and (ii) the policy over rules they entertain is also conditional on their goals (for modern deep reinforcement learning implementations of this idea, see [85]). Therefore, rule-based policy abstractions common in hierarchical theories of cognitive control can and cannot be conditioned on goals. To put this more formally, we can write a rule policy abstraction as goal independent, p(rule|state), or goal conditioned, p(rule|state, goal). Our key point is that the agent must also optimize the selection of goals themselves, potentially based on the current and desired future state: p(goal|current state and future state). The selection of goals through their own policy optimization (as we posit for OFC) is hierarchically higher than rule-based policy abstraction. In a mathematical sense, the notion of generic policy optimization is similar for both rules and goals, but goals sit hierarchically above rules. Thus, our proposal does not dismiss the potential role of the dorsal frontal cortex in rule policy abstraction, it contextualizes the process of how agents might optimize different rule policies based on which goals have been selected.
6. Reconsidering the role of orbitofrontal cortex as residing atop the premotor prefrontal cortex hierarchy
We are proposing that the canonical function of OFC is optimization of goal-selection policies. This proposal is consistent with several recent models that portray OFC as performing goal-selection and representing the value of an agent's current state with respect to its distance to achieving a goal-fulfilling state [63,71,86,87]. Further support for a goal-selection characterization of OFC is already found in extant OFC lesion studies and their behavioural consequences. Specifically, a classic way to examine goal-directed and motivated behaviour has been devaluation studies wherein the value of a reward (e.g. sugar pellet) to the animal is diminished. Under normal conditions, the expectation is that if a reward offer becomes devalued through satiation or aversion pairing, for example, the animal will stop or reduce responding to the devalued reward predictive cue [88]. Imperatively, when either rats or primates have a disrupted OFC function through lesions or optogenetic disruption, the animal's capacity for reducing responding to devalued cues decreases; they still exhibit anticipatory or actual approach towards the stimuli as though it was not devalued [89,90]. Additional evidence for the argument that OFC drives motivated goal-selection, rather than specifically encoding of economic choice variables, is found in a study demonstrating OFC optogenetic disruption had no impact on standard economic choice; in contrast, disruption led the same animals' to still approach a reward cue even after devaluation [91]. Together, we take these types of findings to indicate that OFC will indeed convey an animals ‘wants’ in service of their more abstract goal-selection processes.
Another component serving the idea that OFC optimizes goal-selection is that it is well positioned to be at the top and most abstract portion of the prefrontal hierarchy. One factor in favour of this idea is that OFC has a somewhat unique anatomy [92,93]. It receives inputs from a diverse array of regions with heavily specialized functions, positioning OFC as a hub for integrating disparate information sources and forming inferences. These connections to OFC include four of the five major senses (all except the auditory system), from visceral areas, from hippocampus and amygdala and from the ventral striatum [94–99]. It does not have direct access to motor or premotor regions. However, its ability to influence them indirectly is clear. For example, it has direct projections to the ventral, medial and dorsolateral PFC [100–102], which allow it indirect descending control over dorsal premotor areas. That is, it is possible to place it at the apex of a series of regions that, in a chain, influence the next in the series, to ultimately drive the motor cortex and other regions with direct spinal motor neuron access. What distinguishes OFC from other PFC areas is that it is the first gathering point for distinct and relatively discrete sensory and association streams.
Our theory, which portrays OFC as a premotor structure that optimizes goal-selection, contrasts with the well-known theory that OFC is predominantly an economic structure [103–108]. The economics view emphasizes the contributions of OFC to evaluating options and for comparing values to select a preferred one. While this view has undoubted validity, it has three limitations. First, it is not clear to what extent the OFC is more economic than other brain regions. Indeed, a good deal of evidence supports the idea that economic representations are highly distributed, and that comparisons reflect the outcome of processes occurring in multiple brain regions [21,58,109,110]. Second, it is not clear the extent to which OFC shows specialization for economic functions. That is, OFC appears to participate in many cognitive processes, including those that are only indirectly related to economic decision-making. For example, research implicates OFC in representation of sensory details of predictions (e.g. [111–113]), of abstract rules [114–116] and task- and state-switching [112,117–119]. Third, and most important, OFC's apparent value coding appears, on closer inspection, to reflect expectancy signalling rather than value coding ([120,121] and [122–125]).
The non-economic view has reached its greatest level of sophistication in the cognitive map of task space theory [61,126–128]. That is, its responses serve to encode the set of relevant mappings associated with potential actions and options in the current environment. As such it serves as a potential source of information that can guide decision-making and action selection. This set of mappings may serve as a superset of encodings that also includes reward information, meaning OFC may be more than just an economic predictor [61,127–129]. It may also explain, for example, rule encoding in OFC [114,115]. That should also include information about space. Indeed, the encoding of spatial information has taken on an important position in debates about the mechanisms of choice and valuation in the OFC [51,103,130,131]. Put differently, evidence for a lack of spatial selectivity would support the notion of a modular, non-premotor, purely economic OFC. Despite the debate, a large set of evidence demonstrates spatial selectivity within OFC. Spatial selectivity is observed in neurons in primates [51,131–138] and in rodents [139–145]. Considered together, this work clearly indicates that single neurons in OFC respond with information about the spatial details of the task at hand. We suspect that OFC's capacity for encoding of spatial information is particularly important, as this type of information is necessary for any goal-selecting organism to decide upon the usefulness of a future goal-fulfilling state.
Linking all of these views and extant findings of OFC encoding everything from space, rules or value, we propose that they suggest OFC's previously measured types of encoded information are indicative of a much broader function. They are all necessary for integrating information useful for guiding decision-making over goals. Put differently, measures of OFC activity encodings will vary based on the information needed to determine the current world state and use that to select among the task goals at hand; if one must deliberate between two routes to get to work, spatial information matters, versus if one must select between two meals with limited cash, then economic value factors will dominate. In that view, then, the apparent economic, spatial or cognitive control (rule) based functions of OFC are a consequence of their factorized influence on goal-selection.
7. Conclusion
We propose that the PFC is, in essence, a hierarchically organized premotor structure. Casting its organization this way can help to organize our understanding of its activities. This view, then, sees the PFC as the mirror reversed complement of the sensory systems, especially the ventral visual system, which is organized along the axis of ever more complex form representation [48]. We propose that viewing the PFC in this manner will help resolve important debates and will push researchers away from the quest of identifying the ‘essential function’ of each region within it, and instead to understanding how it coordinates its computations to produce action. We have proposed elsewhere that use of continuous decision-making tasks, such as prey-pursuit tasks, will help to uncover more naturalistic modes of behaviour and brain activity than are captured by standard laboratory tasks [146–148]. Based on the three studies cited at the start of this piece, we suspect that such tasks will also make more apparent the fundamentally premotor nature of the PFC.
Acknowledgements
We thank Sarah Heilbronner, Becket Ebitz, Maya Wang and Michael Yoo for helpful conversations.
Data accessibility
This article has no additional data.
Authors' contributions
Both authors wrote the paper together.
Competing interests
The authors have no competing interests to declare.
Funding
This research was supported by a National Institute on Drug Abuse grant nos. R01 DA038615 and MH124687 (to B.Y.H.).
References
- 1.Musall S, Kaufman MT, Juavinett AL, Gluf S, Churchland AK. 2019. Single-trial neural dynamics are dominated by richly varied movements. Nat. Neurosci. 22, 1677-1686. ( 10.1038/s41593-019-0502-4) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Stringer C, Pachitariu M, Steinmetz N, Reddy CB, Carandini M, Harris KD. 2019. Spontaneous behaviors drive multidimensional, brainwide activity. Science 364, 6437. ( 10.1126/science.aav7893) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Steinmetz NA, Zatka-Haas P, Carandini M, Harris KD.. 2019. Distributed coding of choice, action and engagement across the mouse brain. Nature 576, 266-273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Krakauer JW, Ghazanfar AA, Gomez-Marin A, MacIver MA, Poeppel D. 2017. Neuroscience needs behavior: correcting a reductionist bias. Neuron 93, 480-490. ( 10.1016/j.neuron.2016.12.041) [DOI] [PubMed] [Google Scholar]
- 5.Merel J, Botvinick M, Wayne G. 2019. Hierarchical motor control in mammals and machines. Nat. Commun. 10, 1-12. ( 10.1038/s41467-019-13239-6) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Botvinick MM. 2008. Hierarchical models of behavior and prefrontal function. Trends Cogn. Sci. 12, 201-208. ( 10.1016/j.tics.2008.02.009) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Sherrington CS. 2009. Man on his Nature. Cambridge, UK: Cambridge University Press. [Google Scholar]
- 8.Barrett L. 2011. Beyond the brain: how body and environment shape animal and human minds. Princeton, NJ: Princeton University Press. [Google Scholar]
- 9.Badre D, Nee DE. 2018. Frontal cortex and the hierarchical control of behavior. Trends Cogn. Sci. 22, 170-188. ( 10.1016/j.tics.2017.11.005) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Christoff K, Keramatian K, Gordon AM, Smith R, Mädler B. 2009. Prefrontal organization of cognitive control according to levels of abstraction. Brain Res. 1286, 94-105. ( 10.1016/j.brainres.2009.05.096) [DOI] [PubMed] [Google Scholar]
- 11.Miller EK, Cohen JD. 2001. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167-202. ( 10.1146/annurev.neuro.24.1.167) [DOI] [PubMed] [Google Scholar]
- 12.O'Reilly RC, Frank MJ. 2006. Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 18, 283-328. ( 10.1162/089976606775093909) [DOI] [PubMed] [Google Scholar]
- 13.Mansouri FA, Koechlin E, Rosa MG, Buckley MJ. 2017. Managing competing goals—a key role for the frontopolar cortex. Nat. Rev. Neurosci. 18, 645. ( 10.1038/nrn.2017.111) [DOI] [PubMed] [Google Scholar]
- 14.Fuster J. 2015. The prefrontal cortex. London, UK: Academic Press. [Google Scholar]
- 15.Stuss DT, Knight R (eds). 2013. Principles of frontal lobe function. Oxford, UK: Oxford University Press. [Google Scholar]
- 16.Clark A. 1998. Being there: putting brain, body, and world together again. Cambridge, MA: MIT Press. [Google Scholar]
- 17.Klatzky RL, MacWhinney B, Behrmann M (eds). 2008. Embodiment, ego-space, and action. New York, NY: Psychology Press. [Google Scholar]
- 18.Pezzulo G, Cisek P. 2016. Navigating the affordance landscape: feedback control as a process model of behavior and cognition. Trends Cogn. Sci. 20, 414-424. ( 10.1016/j.tics.2016.03.013) [DOI] [PubMed] [Google Scholar]
- 19.Cisek P, Kalaska JF. 2010. Neural mechanisms for interacting with a world full of action choices. Annu. Rev. Neurosci. 33, 269-298. ( 10.1146/annurev.neuro.051508.135409) [DOI] [PubMed] [Google Scholar]
- 20.Cisek P. 2006. Integrated neural processes for defining potential actions and deciding between them: a computational model. J. Neurosci. 26, 9761-9770. ( 10.1523/JNEUROSCI.5605-05.2006) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Cisek P. 2012. Making decisions through a distributed consensus. Curr. Opin. Neurobiol. 22, 927-936. ( 10.1016/j.conb.2012.05.007) [DOI] [PubMed] [Google Scholar]
- 22.Thelen E, Schöner G, Scheier C, Smith LB. 2001. The dynamics of embodiment: a field theory of infant perseverative reaching. Behav. Brain Sci. 24, 1-34. ( 10.1017/S0140525X01003910) [DOI] [PubMed] [Google Scholar]
- 23.Stephens DW, Krebs JR. 1986. Foraging theory (vol. 1). Princeton, NJ: Princeton University Press. [Google Scholar]
- 24.Cisek P. 2007. Cortical mechanisms of action selection: the affordance competition hypothesis. Phil. Trans. R. Soc. B 362, 1585-1599. ( 10.1098/rstb.2007.2054) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Gibson JJ. 1979. The ecological approach to visual perception: classic edition. New York, NY: Psychology Press. [Google Scholar]
- 26.Turvey MT. 1992. Affordances and prospective control: an outline of the ontology. Eco. Psychol. 4, 173-187. ( 10.1207/s15326969eco0403_3) [DOI] [Google Scholar]
- 27.Kacelnik A, Vasconcelos M, Monteiro T, Aw J. 2011. Darwin's ‘tug-of-war' vs. starlings ‘horse-racing’: how adaptations for sequential encounters drive simultaneous choice. Behav. Ecol. Sociobiol. 65, 547-558. ( 10.1007/s00265-010-1101-2) [DOI] [Google Scholar]
- 28.Hayden BY. 2018. Economic choice: the foraging perspective. Curr. Opin. Behav. Sci. 24, 1-6. ( 10.1016/j.cobeha.2017.12.002) [DOI] [Google Scholar]
- 29.Hayden BY, Moreno-Bote R. 2018. A neuronal theory of sequential economic choice. Brain Neurosci. Adv. 2, 2398212818766675. ( 10.1177/2398212818766675) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Kolling N, Wittmann MK, Behrens TE, Boorman ED, Mars RB, Rushworth MF. 2016. Value, search, persistence and model updating in anterior cingulate cortex. Nat. Neurosci. 19, 1280-1285. ( 10.1038/nn.4382) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Wise SP. 1985. The primate premotor cortex: past, present, and preparatory. Annu. Rev. Neurosci. 8, 1-19. ( 10.1146/annurev.ne.08.030185.000245) [DOI] [PubMed] [Google Scholar]
- 32.Wise SP, Boussaoud D, Johnson PB, Caminiti R. 1997. Premotor and parietal cortex: corticocortical connectivity and combinatorial computations. Annu. Rev. Neurosci. 20, 25-42. ( 10.1146/annurev.neuro.20.1.25) [DOI] [PubMed] [Google Scholar]
- 33.Georgopoulos AP, Schwartz AB, Kettner RE. 1986. Neuronal population coding of movement direction. Science 233, 1416-1419. ( 10.1126/science.3749885) [DOI] [PubMed] [Google Scholar]
- 34.Graziano MS, Taylor CS, Moore T. 2002. Complex movements evoked by microstimulation of precentral cortex. Neuron 34, 841-851. ( 10.1016/s0896-6273(02)00698-0) [DOI] [PubMed] [Google Scholar]
- 35.Omrani M, Murnaghan CD, Pruszynski JA, Scott SH. 2016. Distributed task-specific processing of somatosensory feedback for voluntary motor control. Elife, 5, e13141. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Coallier É, Michelet T, Kalaska JF. 2015. Dorsal premotor cortex: neural correlates of reach target decisions based on a color-location matching rule and conflicting sensory evidence. J. Neurophysiol. 113, 3543-3573. ( 10.1152/jn.00166.2014) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Dekleva BM, Kording KP, Miller LE. 2018. Single reach plans in dorsal premotor cortex during a two-target task. Nat. Commun. 9, 1-12. ( 10.1038/s41467-018-05959-y) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Mirabella G, Pani P, Ferraina S. 2011. Neural correlates of cognitive control of reaching movements in the dorsal premotor cortex of rhesus monkeys. J. Neurophysiol. 106, 1454-1466. ( 10.1152/jn.00995.2010) [DOI] [PubMed] [Google Scholar]
- 39.Pastor-Bernier A, Cisek P. 2011. Neural correlates of biased competition in premotor cortex. J. Neurosci. 31, 7083-7088. ( 10.1523/JNEUROSCI.5681-10.2011) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Woolsey CN. 1952. Patterns of localization in precentral and supplementary motor areas and their relation to the concept of a motor area. Publ. Assoc. Res. Nerv. Ment. Dis. 30, 238-264. [PubMed] [Google Scholar]
- 41.Travis AM. 1955. Neurological deficiencies following supplementary motor area lesions in Macaca mulatta. Brain 78, 174-175. ( 10.1093/brain/78.2.174) [DOI] [PubMed] [Google Scholar]
- 42.Graziano MS, Taylor CS, Moore T, Cooke DF. 2002. The cortical control of movement revisited. Neuron 36, 349-362. ( 10.1016/S0896-6273(02)01003-6) [DOI] [PubMed] [Google Scholar]
- 43.Weinrich M, Wise SP. 1982. The premotor cortex of the monkey. J. Neurosci. 2, 1329-1345. ( 10.1523/JNEUROSCI.02-09-01329.1982) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Churchland MM, Byron MY, Ryu SI, Santhanam G, Shenoy KV. 2006. Neural variability in premotor cortex provides a signature of motor preparation. J. Neurosci. 26, 3697-3712. ( 10.1523/JNEUROSCI.3762-05.2006) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Cisek P, Kalaska JF.. 2005. Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45, 801-814. ( 10.1016/j.neuron.2005.01.027) [DOI] [PubMed] [Google Scholar]
- 46.Ames KC, Ryu SI, Shenoy KV. 2019. Simultaneous motor preparation and execution in a last-moment reach correction task. Nat. Commun. 10, 1-13. ( 10.1038/s41467-018-07882-8) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Pastor-Bernier A, Tremblay E, Cisek P. 2012. Dorsal premotor cortex is involved in switching motor plans. Front. Neuroeng. 5, 5. ( 10.3389/fneng.2012.00005) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Yoo SBM, Hayden BY. 2018. Economic choice as an untangling of options into actions. Neuron 99, 434-447. ( 10.1016/j.neuron.2018.06.038) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Fuster JM. 2000. Executive frontal functions. Exp. Brain Res. 133, 66-70. ( 10.1007/s002210000401) [DOI] [PubMed] [Google Scholar]
- 50.Fuster JM. 2001. The prefrontal cortex—an update: time is of the essence. Neuron 30, 319-333. ( 10.1016/S0896-6273(01)00285-9) [DOI] [PubMed] [Google Scholar]
- 51.Strait CE, Sleezer BJ, Blanchard TC, Azab H, Castagno MD, Hayden BY. 2016. Neuronal selectivity for spatial positions of offers and choices in five reward regions. J. Neurophysiol. 115, 1098-1111. ( 10.1152/jn.00325.2015) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Azab H, Hayden BY. 2018. Correlates of economic decisions in the dorsal and subgenual anterior cingulate cortices. Eur. J. Neurosci. 47, 979-993. ( 10.1111/ejn.13865) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Maisson DJN, Cash-Padgett TV, Wang MZ, Hayden BY, Heilbronner SR, Zimmermann J. 2021. Choice-relevant information transformation along a ventrodorsal axis in the medial prefrontal cortex. Nat. Commun. 12, 1-4. ( 10.1038/s41467-021-25219-w) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Badre D. 2008. Cognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes. Trends Cogn. Sci. 12, 193-200. ( 10.1016/j.tics.2008.02.004) [DOI] [PubMed] [Google Scholar]
- 55.Badre D, D'esposito M. 2009. Is the rostro-caudal axis of the frontal lobe hierarchical? Nat. Rev. Neurosci. 10, 659. ( 10.1038/nrn2667) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Koechlin E, Ody C, Kouneiher F. 2003. The architecture of cognitive control in the human prefrontal cortex. Science 302, 1181-1185. ( 10.1126/science.1088545) [DOI] [PubMed] [Google Scholar]
- 57.Hunt LT, Malalasekera WN, de Berker AO, Miranda B, Farmer SF, Behrens TE, Kennerley SW. 2018. Triple dissociation of attention and decision computations across prefrontal cortex. Nat. Neurosci. 21, 1471-1481. ( 10.1038/s41593-018-0239-5) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Hunt LT, Hayden BY. 2017. A distributed, hierarchical and recurrent framework for reward-based choice. Nat. Rev. Neurosci. 18, 172-182. ( 10.1038/nrn.2017.7) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Nee DE, D'Esposito M. 2016. The hierarchical organization of the lateral prefrontal cortex. Elife 5, e12112. ( 10.7554/eLife.12112) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Botvinick MM. 2007. Multilevel structure in behaviour and in the brain: a model of Fuster's hierarchy. Phil. Trans. R. Soc. B 362, 1615-1626. ( 10.1098/rstb.2007.2056) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Niv Y. 2019. Learning task-state representations. Nat. Neurosci. 22, 1544-1553. ( 10.1038/s41593-019-0470-8) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Yeterian EH, Pandya DN, Tomaiuolo F, Petrides M. 2012. The cortical connectivity of the prefrontal cortex in the monkey brain. Cortex, 48, 58-81. ( 10.1016/j.cortex.2011.03.004) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Fine JM, Zarr N, Brown JW. 2020. Computational neural mechanisms of goal-directed planning and problem solving. Comput. Brain Behav. 3, 472-493. ( 10.1007/s42113-020-00095-7) [DOI] [Google Scholar]
- 64.O'Reilly RC, Hazy TE, Mollick J, Mackie P, Herd S. 2014. Goal-driven cognition in the brain: a computational framework. arXiv, 1404.7591.
- 65.Holroyd CB, Verguts T. 2021. The best laid plans: computational principles of anterior cingulate cortex. Trends Cogn. Sci. 25, 316-329. ( 10.1016/j.tics.2021.01.008) [DOI] [PubMed] [Google Scholar]
- 66.O'Reilly RC, Hazy TE, Mollick J, Mackie P, Herd S.. 2014. Goal-driven cognition in the brain: a computational framework. arXiv, 1404.7591. [Google Scholar]
- 67.Cannon WB. 1929. Organization for physiological homeostasis. Physiol. Rev. 9, 399-431. ( 10.1152/physrev.1929.9.3.399) [DOI] [Google Scholar]
- 68.Keramati M, Gutkin B. 2014. Homeostatic reinforcement learning for integrating reward collection and physiological stability. Elife 3, e04811. ( 10.7554/eLife.04811) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Juechems K, Summerfield C. 2019. Where does value come from? Trends Cogn. Sci. 23, 836-850. ( 10.1016/j.tics.2019.07.012) [DOI] [PubMed] [Google Scholar]
- 70.Hull CL. 1943. Principles of behavior (vol. 422). New York, NY: Appleton-Century-Crofts. [Google Scholar]
- 71.Juechems K, Balaguer J, Castañón SH, Ruz M, O'Reilly JX, Summerfield C.. 2018. A network for computing value homeostasis in the human medial prefrontal cortex. bioRxiv, 278531. [DOI] [PubMed] [Google Scholar]
- 72.Pezzulo G, Donnarumma F, Maisto D, Stoianov I. 2019. Planning at decision time and in the background during spatial navigation. Curr. Opin. Behav. Sci. 29, 69-76. ( 10.1016/j.cobeha.2019.04.009) [DOI] [Google Scholar]
- 73.Todorov E, Jordan MI. 2002. Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5, 1226-1235. ( 10.1038/nn963) [DOI] [PubMed] [Google Scholar]
- 74.Daw ND. 2012. Model-based reinforcement learning as cognitive search: neurocomputational theories. In Cognitive search: evolution, algorithms and the brain (eds P Todd, T Robbins), pp. 195-208. Cambridge, MA: MIT Press. [Google Scholar]
- 75.Shadmehr R, Krakauer JW. 2008. A computational neuroanatomy for motor control. Exp. Brain Res. 185, 359-381. ( 10.1007/s00221-008-1280-5) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Hick WE. 1952. On the rate of gain of information. Q. J. Exp. Psychol. (Colchester) 4, 11-26. ( 10.1080/17470215208416600) [DOI] [Google Scholar]
- 77.Sims CR. 2016. Rate–distortion theory and human perception. Cognition 152, 181-198. ( 10.1016/j.cognition.2016.03.020) [DOI] [PubMed] [Google Scholar]
- 78.Bhui R, Lai L, Gershman SJ. 2021. Resource-rational decision making. Curr. Opin. Behav. Sci. 41, 15-21. ( 10.1016/j.cobeha.2021.02.015) [DOI] [Google Scholar]
- 79.Lai L, Gershman SJ.. 2021. Policy compression: an information bottleneck in action selection. In The psychology of learning and motivation, vol. 74 (ed. KD Federmeier), pp. 195-232. Amsterdam, The Netherlands: Elsevier. [Google Scholar]
- 80.Genewein T, Leibfried F, Grau-Moya J, Braun DA. 2015. Bounded rationality, abstraction, and hierarchical decision-making: an information-theoretic optimality principle. Front. Robotics AI, 2, 27. ( 10.3389/frobt.2015.00027) [DOI] [Google Scholar]
- 81.Ho MK, Abel D, Cohen JD, Littman ML, Griffiths TL. 2020. The efficiency of human cognition reflects planned information processing. arXiv, 2002.05769.
- 82.Moreno-Bote R, Ramírez-Ruiz J, Drugowitsch J, Hayden BY. 2020. Heuristics and optimal solutions to the breadth–depth dilemma. Proc. Natl Acad. Sci. USA 117, 19 799-19 808. ( 10.1073/pnas.2004929117) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 83.Piray P, Daw ND. 2021. Linear reinforcement learning in planning, grid fields, and cognitive control. Nat. Commun. 12, 1-20. ( 10.1038/s41467-021-25123-3) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Shenhav A, Botvinick MM, Cohen JD. 2013. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron, 79, 217-240. ( 10.1016/j.neuron.2013.07.007) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 85.Nasiriany S, Pong VH, Lin S, Levine S.. 2019. Planning with goal-conditioned policies. arXiv, 1911.08453. [Google Scholar]
- 86.Castegnetti G, Zurita M, De Martino B. 2021. How usefulness shapes neural representations during goal-directed behavior. Sci. Adv. 7, eabd5363. ( 10.1126/sciadv.abd5363) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 87.Zarr N, Brown JW.. 2019. Foundations of human problem solving. bioRxiv, 779322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 88.Pickens C, Saddoris M, Setlow B, Gallagher M, Holland P, Schoenbaum G. 2003. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J. Neurosci. 23, 11 078-11 084. ( 10.1523/JNEUROSCI.23-35-11078.2003) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 89.Rudebeck PH, Saunders R, Prescott A, Chau L, Murray E. 2013. Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating. Nat. Neurosci. 16, 1140-1145. ( 10.1038/nn.3440) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 90.West EA, DesJardin JT, Gale K, Malkova L. 2011. Transient inactivation of orbitofrontal cortex blocks reinforcer devaluation in macaques. J. Neurosci. 31, 15 128-15 135. ( 10.1523/JNEUROSCI.3295-11.2011) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 91.Gardner MP, Conroy JS, Shaham MH, Styer CV, Schoenbaum G. 2017. Lateral orbitofrontal inactivation dissociates devaluation-sensitive behavior and economic choice. Neuron 96, 1192-1203. ( 10.1016/j.neuron.2017.10.026) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 92.Rudebeck PH, Murray EA. 2014. The orbitofrontal oracle: cortical mechanisms for the prediction and evaluation of specific behavioral outcomes. Neuron 84, 1143-1156. ( 10.1016/j.neuron.2014.10.049) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 93.Rushworth MF, Noonan MP, Boorman ED, Walton ME, Behrens TE. 2011. Frontal cortex and reward-guided learning and decision-making. Neuron 70, 1054-1069. ( 10.1016/j.neuron.2011.05.014) [DOI] [PubMed] [Google Scholar]
- 94.Öngür D, Price JL. 2000. The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cerebral Cortex 10, 206-219. ( 10.1093/cercor/10.3.206) [DOI] [PubMed] [Google Scholar]
- 95.Haber SN, Knutson B. 2010. The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35, 4-26. ( 10.1038/npp.2009.129) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 96.Price JL. 2005. Free will versus survival: brain systems that underlie intrinsic constraints on behavior. J. Comp. Neurol. 493, 132-139. ( 10.1002/cne.20750) [DOI] [PubMed] [Google Scholar]
- 97.Carmichael ST, Price JL. 1995. Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. J. Comp. Neurol. 363, 615-641. ( 10.1002/cne.903630408) [DOI] [PubMed] [Google Scholar]
- 98.Carmichael ST, Price JL. 1995. Sensory and premotor connections of the orbital and medial prefrontal cortex of macaque monkeys. J. Comp. Neurol. 363, 642-664. ( 10.1002/cne.903630409) [DOI] [PubMed] [Google Scholar]
- 99.Ghashghaei HT, Hilgetag CC, Barbas H. 2007. Sequence of information processing for emotions based on the anatomic dialogue between prefrontal cortex and amygdala. Neuroimage 34, 905-923. ( 10.1016/j.neuroimage.2006.09.046) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 100.Barbas H, Pandya DN. 1989. Architecture and intrinsic connections of the prefrontal cortex in the rhesus monkey. J. Comp. Neurol. 286, 353-375. ( 10.1002/cne.902860306) [DOI] [PubMed] [Google Scholar]
- 101.Carmichael ST, Price JL. 1996. Connectional networks within the orbital and medial prefrontal cortex of macaque monkeys. J. Comp. Neurol. 371, 179-207. () [DOI] [PubMed] [Google Scholar]
- 102.Saleem KS, Miller B, Price JL. 2014. Subdivisions and connectional networks of the lateral prefrontal cortex in the macaque monkey. J. Comp. Neurol. 522, 1641-1690. ( 10.1002/cne.23498) [DOI] [PubMed] [Google Scholar]
- 103.Padoa-Schioppa C. 2011. Neurobiology of economic choice: a good-based model. Annu. Rev. Neurosci. 34, 333-359. ( 10.1146/annurev-neuro-061010-113648) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 104.Wallis JD. 2007. Orbitofrontal cortex and its contribution to decision-making. Annu. Rev. Neurosci. 30, 31-56. ( 10.1146/annurev.neuro.30.051606.094334) [DOI] [PubMed] [Google Scholar]
- 105.Schoenbaum G, Takahashi Y, Liu TL, McDannald MA. 2011. Does the orbitofrontal cortex signal value? Ann. NY Acad. Sci. 1239, 87. ( 10.1111/j.1749-6632.2011.06210.x) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 106.Wallis JD. 2012. Cross-species studies of orbitofrontal cortex and value-based decision-making. Nat. Neurosci. 15, 13-19. ( 10.1038/nn.2956) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 107.O'Doherty JP. 2004. Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr. Opin. Neurobiol. 14, 769-776. ( 10.1016/j.conb.2004.10.016) [DOI] [PubMed] [Google Scholar]
- 108.Rushworth MF, Noonan MP, Boorman ED, Walton ME, Behrens TE. 2011. Frontal cortex and reward-guided learning and decision-making. Neuron 70, 1054-1069. ( 10.1016/j.neuron.2011.05.014) [DOI] [PubMed] [Google Scholar]
- 109.Vickery TJ, Chun MM, Lee D. 2011. Ubiquity and specificity of reinforcement signals throughout the human brain. Neuron 72, 166-177. ( 10.1016/j.neuron.2011.08.011) [DOI] [PubMed] [Google Scholar]
- 110.Hunt LT, Dolan RJ, Behrens TE. 2014. Hierarchical competitions subserving multi-attribute choice. Nat. Neurosci. 17, 1613. ( 10.1038/nn.3836) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 111.Burke KA, Franz TM, Miller DN, Schoenbaum G. 2008. The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards. Nature 454, 340-344. ( 10.1038/nature06993) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 112.Tsujimoto S, Genovesio A, Wise SP. 2011. Comparison of strategy signals in the dorsolateral and orbital prefrontal cortex. J. Neurosci. 31, 4583-4592. ( 10.1523/JNEUROSCI.5816-10.2011) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 113.Tsujimoto S, Genovesio A, Wise SP. 2012. Neuronal activity during a cued strategy task: comparison of dorsolateral, orbital, and polar prefrontal cortex. J. Neurosci. 32, 11 017-11 031. ( 10.1523/JNEUROSCI.1230-12.2012) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 114.Sleezer BJ, Castagno MD, Hayden BY. 2016. Rule encoding in orbitofrontal cortex and striatum guides selection. J. Neurosci. 36, 11 223-11 237. ( 10.1523/JNEUROSCI.1766-16.2016) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 115.Wallis JD, Anderson KC, Miller EK. 2001. Single neurons in prefrontal cortex encode abstract rules. Nature 411, 953-956. ( 10.1038/35082081) [DOI] [PubMed] [Google Scholar]
- 116.Buckley MJ, Mansouri FA, Hoda H, Mahboubi M, Browning PG, Kwok SC, Phillips A, Tanaka K. 2009. Dissociable components of rule-guided behavior depend on distinct medial and prefrontal regions. Science 325, 52-58. ( 10.1126/science.1172377) [DOI] [PubMed] [Google Scholar]
- 117.Sleezer BJ, LoConte GA, Castagno MD, Hayden BY. 2017. Neuronal responses support a role for orbitofrontal cortex in cognitive set reconfiguration. Eur. J. Neurosci. 45, 940-951. ( 10.1111/ejn.13532) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 118.Young JJ, Shapiro ML. 2009. Double dissociation and hierarchical organization of strategy switches and reversals in the rat PFC. Behav. Neurosci. 123, 1028. ( 10.1037/a0016822) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 119.Gremel CM, Costa RM. 2013. Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat. Commun. 4, 1-12. ( 10.1038/ncomms3264) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 120.Jones JL, Esber GR, McDannald MA, Gruber AJ, Hernandez A, Mirenzi A, Schoenbaum G. 2012. Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 338, 953-956. ( 10.1126/science.1227489) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 121.Zhou J, Jia C, Montesinos-Cartagena M, Gardner MP, Zong W, Schoenbaum G. 2021. Evolving schema representations in orbitofrontal ensembles during learning. Nature 590, 606-611. ( 10.1038/s41586-020-03061-2) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 122.Zhou J, Gardner MP, Schoenbaum G. 2021. Is the core function of orbitofrontal cortex to signal values or make predictions? Curr. Opin. Behav. Sci. 41, 1-9. ( 10.1016/j.cobeha.2021.02.011) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 123.Gardner MP, Schoenbaum G. 2021. The orbitofrontal cartographer. Behav. Neurosci. 135, 267. ( 10.1037/bne0000463) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 124.Wang MZ, Hayden BY. 2017. Reactivation of associative structure specific outcome responses during prospective evaluation in reward-based choices. Nat. Commun. 8, 1-13. ( 10.1038/s41467-016-0009-6) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 125.Farovik A, Place RJ, McKenzie S, Porter B, Munro CE, Eichenbaum H. 2015. Orbitofrontal cortex encodes memories within value-based schemas and represents contexts that guide memory retrieval. J. Neurosci. 35, 8333-8344. ( 10.1523/JNEUROSCI.0134-15.2015) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 126.Schuck NW, Cai MB, Wilson RC, Niv Y. 2016. Human orbitofrontal cortex represents a cognitive map of state space. Neuron 91, 1402-1412. ( 10.1016/j.neuron.2016.08.019) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 127.Wilson RC, Takahashi YK, Schoenbaum G, Niv Y. 2014. Orbitofrontal cortex as a cognitive map of task space. Neuron 81, 267-279. ( 10.1016/j.neuron.2013.11.005) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 128.Wikenheiser AM, Schoenbaum G. 2016. Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex. Nat. Rev. Neurosci. 17, 513-523. ( 10.1038/nrn.2016.56) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 129.Murray EA, Rudebeck PH. 2018. Specializations for reward-guided decision-making in the primate ventral prefrontal cortex. Nat. Rev. Neurosci. 19, 404-417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 130.Grattan LE, Glimcher PW. 2014. Absence of spatial tuning in the orbitofrontal cortex. PLoS ONE 9, 11. ( 10.1371/journal.pone.0112750) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 131.Yoo SBM, Sleezer BJ, Hayden BY. 2018. Robust encoding of spatial information in orbitofrontal cortex and striatum. J. Cogn. Neurosci. 30, 898-913. ( 10.1162/jocn_a_01259) [DOI] [PubMed] [Google Scholar]
- 132.Tsujimoto S, Genovesio A, Wise SP. 2009. Monkey orbitofrontal cortex encodes response choices near feedback time. J. Neurosci. 29, 2569-2574. ( 10.1523/JNEUROSCI.5777-08.2009) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 133.Abe H, Lee D. 2011. Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 70, 731-741. ( 10.1016/j.neuron.2011.03.026) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 134.Luk CH, Wallis JD. 2013. Choice coding in frontal cortex during stimulus-guided or action-guided decision-making. J. Neurosci. 33, 1864-1871. ( 10.1523/JNEUROSCI.4920-12.2013) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 135.McGinty VB, Rangel A, Newsome WT. 2016. Orbitofrontal cortex value signals depend on fixation location during free viewing. Neuron 90, 1299-1311. ( 10.1016/j.neuron.2016.04.045) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 136.Roesch MR, Olson CR. 2004. Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304, 307-310. ( 10.1126/science.1093223) [DOI] [PubMed] [Google Scholar]
- 137.Baeg E, Jedema HP, Bradberry CW. 2020. Orbitofrontal cortex is selectively activated in a primate model of attentional bias to cocaine cues. Neuropsychopharmacology 45, 675-682. ( 10.1038/s41386-019-0499-0) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 138.Costa VD, Averbeck BB. 2020. Primate orbitofrontal cortex codes information relevant for managing explore–exploit tradeoffs. J. Neurosci. 40, 2553-2561. ( 10.1523/JNEUROSCI.2355-19.2020) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 139.Roesch MR, Taylor AR, Schoenbaum G. 2006. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51, 509-520. ( 10.1016/j.neuron.2006.06.027) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 140.Feierstein CE, Quirk MC, Uchida N, Sosulski DL, Mainen ZF. 2006. Representation of spatial goals in rat orbitofrontal cortex. Neuron 51, 495-507. ( 10.1016/j.neuron.2006.06.032) [DOI] [PubMed] [Google Scholar]
- 141.Furuyashiki T, Holland PC, Gallagher M. 2008. Rat orbitofrontal cortex separately encodes response and outcome information during performance of goal-directed behavior. J. Neurosci. 28, 5127-5138. ( 10.1523/JNEUROSCI.0319-08.2008) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 142.Sul JH, Kim H, Huh N, Lee D, Jung MW. 2010. Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66, 449-460. ( 10.1016/j.neuron.2010.03.033) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 143.van Wingerden M, Vinck M, Lankelma J, Pennartz CM. 2010. Theta-band phase locking of orbitofrontal neurons during reward expectancy. J. Neurosci. 30, 7078-7087. ( 10.1523/JNEUROSCI.3860-09.2010) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 144.Bryden DW, Roesch MR. 2015. Executive control signals in orbitofrontal cortex during response inhibition. J. Neurosci. 35, 3903-3914. ( 10.1523/JNEUROSCI.3587-14.2015) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 145.Young JJ, Shapiro ML. 2011. Dynamic coding of goal-directed paths by orbital prefrontal cortex. J. Neurosci. 31, 5989-6000. ( 10.1523/JNEUROSCI.5436-10.2011) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 146.Yoo SBM, Tu JC, Piantadosi ST, Hayden BY. 2020. The neural basis of predictive pursuit. Nat. Neurosci. 23, 252-259. ( 10.1038/s41593-019-0561-6) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 147.Yoo SBM, Tu JC, Hayden BY. 2021. Multicentric tracking of multiple agents by anterior cingulate cortex during pursuit and evasion. Nat. Commun. 12, 1-4. ( 10.1038/s41467-021-22195-z) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 148.Yoo SBM, Hayden BY, Pearson JM. 2021. Continuous decisions. Phil. Trans. R. Soc B 376, 20190664. ( 10.1098/rstb.2019.0664) [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
This article has no additional data.

