Foraging for foundations in decision neuroscience: insights from ethology

Abstract

Modern decision neuroscience offers a powerful and broad account of human behaviour using computational techniques that link psychological and neuroscientific approaches to the ways that individuals can generate near-optimal choices in complex controlled environments. However, until recently, relatively little attention has been paid to the extent to which the structure of experimental environments relates to natural scenarios, and the survival problems that individuals have evolved to solve. This situation not only risks leaving decision-theoretic accounts ungrounded but also makes various aspects of the solutions, such as hard-wired or Pavlovian policies, difficult to interpret in the natural world. Here, we suggest importing concepts, paradigms and approaches from the fields of ethology and behavioural ecology, which concentrate on the contextual and functional correlates of decisions made about foraging and escape and address these lacunae.

It begins to be difficult, and even in some cases impossible, to say where ethology stops and neurophysiology begins. Tinbergen, 1963

Ethology, the scientific study of animal behaviour under natural conditions, and its offspring, behavioural ecology, which studies how behaviour may vary adaptively in response to environmental variation, have provided valuable insights into animal decision-making in the natural world. The paradigms used by ethologists and behavioural ecologists examine decisions relating to the exploitation of resources and optimal foraging, the ways that predators can be avoided and deterred, the reduction of competition and the maximization of reproductive success¹. Such decisions are often complex because individuals need to balance energy costs with other survival needs; a central lesson from behavioural ecology is that trade-offs are ubiquitous. In addition, the computations that characterize these decisions are described in formal mathematical accounts — such as optimal escape theory², foraging theories³ including the marginal value theorem (MVT)⁴, ideal free distribution (IFD) theory⁵ and state-dependent valuation⁶ — using methods such as stochastic dynamic programming⁷. These theories fall into the global frameworks of Huxley, Mayr and Tinbergen (BOX 1), which seek to answer wider questions of the proximate (for example, physiological) and ultimate (for example, adaptive or Darwinian fitness-increasing) repercussions of an animal’s decisions.

Box 1 |. The overlapping frameworks of Huxley, Tinbergen, Mayr and Marr.

It has long been recognized that there are two fundamentally different classes of questions that biologists can ask: why a behaviour or neural structure takes the form it does and how this comes to happen, with subtle and complex nuances within each class. Many different authors in cognitive science^18,108 and ethology^107,110,111 have proposed versions of these questions. One of the first to offer guidelines for biobehavioural questioning was Julian Huxley, who proposed that one should attempt to address three issues. First, what are the behavioural and neurobiological mechanisms that facilitate ecological decision-making (the mechanistic-physiological question)? second, what is the functional or survival value of such decisions (the adaptive-function question)? Last, what is the evolutionary course of the physiology or behaviour that, for example, supports decision-making processes (the historic (palaeontological) question)? these three questions were extended and further refined by tinbergen¹⁰⁷, who proposed that biologists should also answer questions of the ontogeny of the behaviour and biology (how does decision-making change over an individual’s lifespan?). Mayr combined these four questions to form two core questions. First, what are the proximate causes of the behaviour (short-term factors such as physiological mechanisms or development during a lifetime)? second, what are the ultimate causes of the behavior (long-term functional trade-offs in terms of natural selection or how phylogeny has developed as a series of small changes over ‘evolutionary time’)¹¹⁰?

Perhaps the most influential framework in decision neuroscience is that of Marr, which focuses particularly on information processing and the different levels at which such processing can be viewed. His computational level of investigation has a rough parallel with the adaptive-function question — except with a focus on the information processing aspects of the task and the logic of the (computational) strategy used to solve a problem. Marr then split the mechanistic-physiological question into two separate algorithmic and implementational levels of investigation that aim to answer questions concerning how problems are solved. there are, in principle, many algorithms that can realize a particular computation and many implementations in computational (biophysical) hardware that can run any particular algorithm. Biologists are typically aiming to understand the particular implementation at hand and therefore lack such freedom. there are various Marrian approaches to aspects of ontogeny¹¹², although this framework does not appear to have been used to characterize evolutionary change (see REF.¹⁸ for a detailed discussion of the relationship between the theories of Huxley, tinbergen, Mayr and Marr). the figure shows how the questions of Huxley, tinbergen, Mayr and Marr map onto each other. ^aHypothesized levels of Marr’s theory as they apply to tinbergen’s theory.

By contrast, decision neuroscience has built accounts of the proximate causes of behaviour that tie optimizing theories such as dynamic programming and Bayesian decision theory to their psychological and neurobiological substrates. Such optimizing theories are shared with ethology and behavioural ecology, along with other fields such as economics, operations research and control theory. However, the bulk of the work in decision neuroscience focuses on building and understanding process models, such as models of inference (for example, diffusion-to-bound decision-making⁸, realized in neural substrates such as the parietal and prefrontal cortex⁹) and learning models (for example, rules based on prediction error, such as the Rescorla–Wagner rule¹⁰, or temporal-difference learning¹¹, which is realized in substrates such as the phasic activity of dopamine neurons¹²). Data from a wealth of anatomical, pharmacological, neurophysiological, optogenetic and neuroimaging paradigms are regularly integrated in decision neuroscience.

Central features of most work in decision neuroscience are carefully controlled and structured environments that rigorously specify the costs and benefits of behaviours — that is, what needs to be optimized and under what constraints. This has led to many functional MRI (fMRI) studies making use of only a relatively modest collection of tasks that reflect aspects of the structural and functional implementation of choice. However, whether these tasks are ethologically credible and comprehensive is not always clear. These two questions are of particular importance given the influential observation in decision theory that completely optimal behaviour is uncommon because of its computational and mnestic complexity, implying that approximations must be ubiquitous. Such approximations are presumably tailored to work well in common and relevant environments at the expense of rare ones; thus, it is vital to consider problems that are ethologically well founded and thus functionally relevant.

In this Opinion article, we lay out a blueprint for a closer integration of ethological and behavioural ecological approaches with work on the neural basis of decision-making. The objective is to build biological realism into decision neuroscience with a focus on translating the extensive work that has been conducted on non-human animals to the study of humans. We concentrate on foraging, which is an early harbinger of this approach^13–17. We consider three constraints on foraging that are of interest for human decision neuroscience: first, energy-based decisions in which metabolic needs, including energy consumption and fuelling, need to be balanced; second, competitive foraging; and, third, foraging under the risk of predation, when the requirement is to be strategic about averting threats and facilitating the possibility of escape. Along with these examples, we discuss formal tools developed by behavioural ecologists to examine various decisions that are made by animals. Taken together, these constraints and tools provide new insights and help integrate human decision-making into the guiding frameworks (BOX 1). Indeed, we note that this form of integration is already becoming popular in comparative neuroscience^13,18,19.

Decision neuroscience

Human decision neuroscience has drawn upon a set of experimental decision paradigms from the fields of economics and psychology. These approaches have been highly successful because they carefully control the variables of interest and constrain computational models that fit the data. One stalwart of human decision neuroscience is the bandit modelling approach, which involves binary (or multiple) choice static or restless ‘bandits’ that deliver rewards and punishments. This approach is used to measure value-based and explore-or-exploit decision processes²⁰. Other paradigms have been used to measure other aspects of the decision processes, including working memory (for example, the AX-continuous performance task²¹), sequential information integration and diffusion-to-bound decision-making (for example, motion discrimination using random dot kinematograms⁹), retrospective and prospective planning (for example, the two-step task²²), social decision-making (for example, trust games²³) and intertemporal choice^24,25. This relatively limited in number set of paradigms, together with their offshoots, has provided us with our current set of potentially oversimplified tests of how the brain computes decisions. Crucially, most realizations of the aforementioned paradigms leave substantial gaps in our understanding of the ways that decisions are made in the real world, or indeed of solutions to “The problems that the brain evolved to solve” (REF.²⁶).

More broadly, ecologically important, fitness-determining decisions relate to core problems that all animals face. These decisions include rules underlying fighting, foraging, fleeing and reproduction. Such rules are under strong selection pressures and probably underpin all decision processes²⁶. Although it is widely understood that the brain contains a set of heuristic mechanisms that evolved to make fast and accurate decisions with as little effort as possible, there has been less work in decision neuroscience paying due heed to the selection pressures or the bounds on their use (when one task paradigm transitions to another). Ethologically inspired theorists have begun to consider the evolutionary roots of human decision-making to explain how we systematically deviate from rational-choice models²⁷ and the extent to which these biases are uniquely human. However, such ethological thinking remains rather rare in human decision neuroscience, which, in most cases, has yet to embrace fully the synthesis that ethological and behavioural ecological approaches offer between theory, mathematical modelling and natural observation.

Some contemporary researchers have started to exploit ethological insights when designing and interpreting more conventional decision-making problems. For instance, binary decision tasks are being adapted to address foraging concerns such as the average value of the environment or the cost of changing choices^14–17. Equally, impulsivity, in the form of a preference for smaller rewards sooner rather than larger ones later, can be studied in the light of changing natural environments or threats²⁷. In particular, because it may be computationally difficult to make the right decisions, adopting an ethological perspective will tell us the types of environment to which the necessary heuristics will have been tailored²⁸. Here, we argue that a more comprehensive synthesis of the two traditions, marrying the approaches used by decision neuroscientists (but see REF.29) with the naturalistic importance of ethology and behavioural ecology, is an important but, as yet, incompletely realized goal. We also offer some prospects for the future.

Foraging

Behavioural ecologists and ethologists have used value-based decision-making to study foraging decisions and the management of food resources (and of other contributors to homeostatic well-being), raising at least two sets of unanswered questions in human decision neuroscience. One of these sets concerns the external environment: the spatiotemporal arrangement of different sorts of resources; the explorative, energetic and other costs required to find and extract them; and the risk of predation while doing so. There have been numerous studies focusing on how variation in the foraging environment and temporal variation in predation risk influence decisions^30,31. The second set of questions concerns the subjects themselves — notably state dependence, which arises from the frequent propinquity of catastrophic boundaries whereby animals run out of reserves and starve to death³². This scenario creates an inevitable trade-off between dying from lack of resources and death from predation^6,30,31,33.

Behavioural neuroscientists such as LeDoux³⁴ have also proposed that energy management exerts a critical influence over survival-based decision-making. Indeed, adaptive behaviour system theory outlines how foraging may be triggered by neural systems that monitor energy reserves (for example, fat)³⁵. This theory is proposed to result in animals attempting to maximize net energy intake rate (a proposal we qualify later) by measuring various key quantities: the average success in acquiring prey, the cost of time travel between foraging patches and the time since the last capture of the prey. A related model characterizing the optimal diet also includes consideration of the energy or caloric value of the prey, the time that is required to prepare and consume it and its ease of discovery³.

Net rate maximization.

If we temporarily ignore predation risk and state dependence, then the obvious approach to understanding foraging is provided by the principle of rate maximization, which optimizes the difference between the benefits and the costs per unit time. In this context, the elements influencing decisions include energy, various forms of opportunity costs and time. Rate maximization has ramifications for what food type to choose (and how to handle it) and the choice of a foraging location. Given a choice between different food types, an individual should choose the one with the most energy (if everything else is equal; for example, see REF.³⁶) and, when the energy content of the food options is equal, they should choose the option that requires the least energy to acquire it, taking time into appropriate account. For example, classic work with starlings has shown that they attempt to maximize the rate of net energy gain for themselves as well as their (energetically expensive) begging offspring when hunting for leatherjackets³⁷; it is also known that gulls and crows modify the heights at which they drop crabs and walnuts, respectively, to minimize energetic costs and avoid kleptoparasitism³⁸.

In decision neuroscience, there are a wealth of experiments manipulating the amount of work that subjects have to do to get outcomes, showing, for instance, a complex and crucial role for dopamine in overcoming subjective effort costs³⁹. It has been suggested that the dorsal anterior cingulate cortex (dACC) is involved in the cost–benefit analysis of control-demanding situations and that it is engaged in energizing behaviour⁴⁰. This proposed function would account for the dACC’s entanglement in decisions associated with energy consumption. Indeed, human imaging work has begun to examine the role of physical⁴¹ and cognitive⁴² effort in decision-making processes. Several groups have shown that anticipation of high-effort investment is associated with activity in the dACC^43,44. Supporting these claims, comparative research has shown that when the dACC region is damaged, it impairs effort-based decisions^45,46. In the healthy human brain, such impairments in decision-making are often mitigated by a set of systems that increase vigour when the payoff is high⁴¹, yet persistent effort can result in decision impairments⁴⁷.

Together, these ethologically inspired studies are consistent with human lesion studies showing that damage to the dACC can result in impairments in motivation, whereas electrical stimulation of this region elicits perseverance and vigour⁴⁸. Foraging tasks show that anticipation and exertion of effort are key components of representation, valuation and action selection. These components will allow us to probe the dACC and its functional role further; this can be done by testing for the effects of how depleting resources, time and effort influence engagement of this region. A different sort of potential cost comes from the possibility of predation. We discuss aspects of this later in the article.

The marginal value theorem.

Another important issue for rate maximization concerns a property of many environments — that is, resources deplete over time as they are exploited. The MVT identifies an optimal foraging strategy in this context^4,49. Consider a notional apple-picking task (FIG. 1). When an agent reaches a new apple tree, the number of apples that can be acquired every minute is high; however, over time, this depletes as the easy-to-pick, low-hanging fruits become scarce. The agent must repeatedly choose between two options: remaining at the same tree and trying to obtain more apples or spending time (and/or energy) travelling to another tree with its own low-hanging fruits. The MVT calculates the policy that maximizes the rate of energy intake using these two options and includes an opportunity cost for the passage of time to help make the comparison. The MVT is an important theory because it extends classic cost–benefit behavioural economic tools and makes testable predictions about the effect on this choice of the time needed to transit between patches and the dynamics of resource depletion.

Fig. 1 |. Example of classic apple-picking task.

Subjects forage in an unlimited orchard and have to decide whether to continue picking from their current tree or to switch to another one to maximize their harvest¹⁶. Switching is associated with a time penalty; however, if the subject decides to stay and pick apples from the tree, the rate of return will decrease. Thus, at some point, the subject should switch to a new tree. To maximize the long-term harvest rate of apples, the subject repeatedly decides between choosing to stay at a tree and switching to a new one.

Of course, it is possible to mimic MVT-relevant conditions in controlled studies on humans (for examples, see refs^50,51). For instance, in one study, the authors designed a version of the apple-picking task mentioned above and administered it to humans¹⁶. They analysed leaving decisions by comparing the classic MVT with temporal-difference learning models, which focus on the learning of values for different options. The authors performed two ‘apple-tree-picking’ experiments in which they varied opportunity cost across blocks, thereby making the subject adjust their behaviour to the changing profitabilities in their environment (FIG. 1). By comparing the predictions made by MVT and temporal-difference learning, they found that the MVT was a better model when subjects adjusted their decisions on the basis of environmental profitability. This study¹⁶ and another investigation¹⁵ were among the first to compare models used in decision neuroscience directly with those used in ethology and behavioural ecology, and they show how these models may be fruitfully adopted for use in human decision-making studies.

Given the popularity of MVT in ethology, it is perhaps surprising that it has very rarely been used in neurophysiological experiments. One important exception is a study of the neural basis of MVT in non-human primates (Macaca mulatta)¹³. In this investigation, monkeys were presented with two options, with one being a stay option in which individuals were given juice at a specific patch. With time, the amount of juice would reduce, thereby leading to less and less reward. The other option was to leave, which provided no reward and a delay. However, once an individual switched, the juice would be replenished. Therefore, monkeys had to balance the trade-off of the immediate reward against the long-term benefits of switching to a new patch.

The results showed that monkeys made near-optimal decisions in all experimental conditions, leading the authors to conclude that the context of foraging reduces biases in time preferences, a finding that is inconsistent with classic intertemporal choice findings¹³. While individuals performed the task, the authors also recorded neuronal activity in the dACC. The dACC was chosen because, as discussed above, it is a region that is thought to integrate reward and cost signals with action selection. The results showed that as the patch gradually depleted in juice, neurons in the dACC would increase their firing rates to the point where, at a specific threshold, monkeys would switch patches. The dACC neuronal signals rose more slowly when the travel time increased, suggesting that multiple control signals work in harmony to regulate patch leaving. This finding supports a large body of work from the field of behavioural ecology. For example, one study showed both experimentally and theoretically that myopic ‘short-term’ rules can be favoured over feeding because they have adaptive ‘long-term’ consequences⁵².

Recent work suggests that the MVT foraging framework can be generalized to intangible resources beyond money and other primary-value domains that are central to traditional decision-making research. Indeed, one study used foraging models to examine how humans allocate limited attention⁵³, and in a different study, the MVT framework was applied to rhesus macaques’ quests for social information⁵⁴. The latter involved an environment in which social information (pictures of other monkeys) was available for up to a fixed time following a choice (in an abstraction of a ‘patch’). However, having looked at a picture, the subjects had to wait until a waiting time had elapsed before making another choice (abstracting the travel cost between patches). Monkeys duly spent more time ‘foraging’ for social information when the travel cost was greater⁵⁴. Equally, in a study of the recall of items from semantic memory by humans, individuals generated a whole collection of answers from one set of overlapping subcategories (the informational equivalent of a patch) and then moved to another set as recall of items in the first one ‘dried up’. The choice of when to switch was consistent with MVT-like processes⁵⁵.

Exploration versus exploitation in foraging.

The analysis in the MVT is based on a simplifying assumption about the regularity of finding a new patch (or tree). In a study on humans, the authors¹⁵ designed a task that made the trade-off between exploration (search) and exploitation (encounter) more explicit, as in an optimal diet model⁵⁶. They used simple binary decision-making tasks in which participants were presented with two stimuli that represented the encounter values; above these two stimuli, there were six additional shapes that represented the ‘search value’, drawn from a set of 12 objects that the subjects had learned about on a previous session. Each individual’s goal was to decide whether to engage with (and ultimately to choose between) the encounter options or search for an alternative. Searching resulted in a cost (or loss of money); note further that, unlike our previous discussion of exploration, searching did not result in the acquisition of information that could be useful over the long run and instead merely led to a potentially better alternative. The results showed that the dACC encoded both the average value of the foraging environment and the costs of foraging. Furthermore, the authors integrated their results with those from REF.¹³ and proposed that the dACC is also involved in searching for alternatives, whereas the ventromedial prefrontal cortex (vmPFC) is involved in well-defined ‘economic’ values of options (but see the subsequent debate elewhere^57,58).

State dependence.

Despite the power of rate maximization in explaining some foraging choices, it does not address certain important factors. In particular, when reserves are tight and homeostatic challenges become critical, the variance of outcomes is important³. Subjects can become more risk-seeking in terms of patch choice and predation risk³⁰. With a few important exceptions (for examples, see refs^59,60), most behavioural neuroscience decision-making studies have taken the view, typically implicitly, that individuals are not constrained by energy or homeostatic challenges. Indeed, it is rare for subjects even to be able to run out of experimentally provided money. Given the ubiquity of energetic constraints in nature, this seems an unfortunate assumption when the brain may often process the acquisition of money (an exogenous resource) in a similar manner to the acquisition of food resources (that is, primary reinforcers that influence one’s endogenous abilities^61,62). Furthermore, research shows that social rewards also modulate activity in overlapping rewards circuits⁶³. Perhaps some of the debates about neural structures such as the dACC, the involvement of which in conventional decision-theoretic tasks is complex and confusing⁵⁷, might be resolved by adopting a more naturalistic stance in which crucial resources are limiting.

Competitive foraging

When the best choice by (and payoff to) one individual is influenced by the actions of another, the appropriate modelling framework is game theory^64,65. In nature, the context of such decisions may occur when making territorial decisions to avoid despotic (for example, dominant) individuals, or when studying parasitic behaviours as captured in the producer–scrounger game model⁶⁶. One goal of game theory, particularly when it is applied to non-human animals, is to predict behaviours that, in conjunction with one another type of behaviour, are evolutionarily stable strategies (a particular type of Nash equilibrium⁶⁷).

Ideal free distribution.

One useful concept is captured by the theory of IFD, which proposes that foragers geographically distribute themselves in relation to the proportion of food available and to the density of competition⁵. The IFD assumes that the forager has perfect information about patch quality and the density of competition (the ideal aspect of IFD) and can move freely among patches without time delay (the free aspect of IFD). In a very simple form, this distribution is expressed by the input-matching rule in which, if habitat A contains more food and less competition than habitat B, habitat A will be the preferred location. Although despotic distributions have also been modelled (for example, dominance and territoriality), theorists have suggested that the input-matching rule is an evolutionarily stable strategy⁶⁷, and pioneering work from behavioural ecology has shown it to be pervasive across various species from invertebrates to humans^5,68.

One notable study relating to the IFD examined foraging behaviours in mallard ducks (Anas platyrhynchos)⁶⁹ — specifically, how mallards distributed themselves when researchers dropped bread into a lake at fixed locations approximately 20 m apart. This approach mimicked a patch foraging task, and the researcher could keep track of the movement of the mallards while altering the amount of bread being dropped on either side of the lake. The study showed that mallards generally followed the assumptions of the IFD, but it also found that some individuals exhibited despotic behaviour (dominance), resulting in some mallards receiving unequal amounts of food. These findings suggest that the behaviour of these birds depends on two types of information: the supply rate in each patch and the competitive value of the other birds⁶⁹.

In one application of the IFD in decision neuroscience, fMRI was used to examine the human brain during the sort of competitive foraging that underlies the IFD¹⁴. The authors created a foraging task in which the available reward and competition varied across two patches. The competition was manipulated by telling the participants that they were performing the task live online with real competitors (represented as red dots), allowing the experimenters to manipulate the number of competitors in each patch. The subject’s goal was to acquire as many tokens as possible by pressing a response key as soon as a token appeared, before any of their competitors. They could switch between two habitats to minimize competition and compete for tokens in the patch with the highest supply rate. At a small cost of time, subjects could switch patches as the rewards and competition increased or decreased in each patch. The increased drive to switch patches resulted in increased activity in several regions, including the dACC, the supplementary motor area and the insula; the authors speculated that all three regions might be involved in the urge to switch habitats. By contrast, staying put in an advantageous habitat was associated with activity in the reward-related areas — namely, the striatum and medial prefrontal cortex. Moreover, the level of amygdala activity steered individual preferences in competition avoidance (for example, high amygdala activity was associated with high levels of avoidance). This study suggests that input-matching decisions are made through the medium of a distributed set of neural circuits.

Foraging under predation risk

Behavioural ecologists have shown the importance of predation risk in understanding foraging decisions^3,70. The near-ubiquitous importance of predation risk suggests a deficiency in decision-making paradigms that ignore it. An early example examined how risk of predation affected rate maximization in grey squirrels (Sciurus carolinensis)⁷¹. Like other animals, squirrels leave their safety refuge to reach food patches, a phenomenon referred to as central-place foraging⁷². Once a squirrel reaches the food patch, it must make a decision to eat in the patch or to use the energy to take the food back to the safety of its refuge. Going back to the refuge makes sense only if there is a cost to staying in the food patch (for example, predation risk). Empirical observations confirm the predictions of mathematical models as to what happens: the squirrels acquire more food if it is close and therefore can be taken back to the refuge cheaply. Similar trade-offs affect the portion sizes they pick. Together, these behaviours reduce the chance of being eaten by a predator.

Common currency.

To understand, at a functional level, how decisions should trade off different benefits (or risks), such as the worth of foraging, versus avoiding predation risk or attempting to court a mate, a common currency is required⁷³. By comparing the expected future reproductive success of an individual (often referred to as ‘reproductive value’ (REF.⁷⁴)) under different strategies, the choice that maximizes reproductive value can be identified as the normative expectation under natural selection. How the value of decisions alters with environmental conditions and an individual’s internal state has been studied extensively from a theoretical perspective^29,75. We suggest that these ethologically inspired insights about optimal behaviour can help to ground expectations about conditions under which particular trade-off paradigms should come into play under the guiding principle that mental processes are likely to produce outcomes that approximate the behaviours that maximize the reproductive value. Consequently, this approach may help to define and understand the boundaries of particular experimental paradigms and to understand the apparent ‘boundaries’ (or framing effects) of different mental processes. As a very simple example, if an individual is close to starvation, then their reproductive value will approach zero if they do not soon find food. Therefore, the normative perspective suggests that their mental processes should take relatively less account of predation risk when their reserves are low.

Escape theory.

Ydenberg and Dill⁷⁶ theorized that when a predator approaches, prey make economic decisions in terms of the cost–benefit ratio of remaining compared with fleeing. Studies of flight initiation distance in hundreds of species of birds, mammals, lizards and other taxa have shown that prey are remarkably adept at escape, making decisions on the basis of prior experience with the predator and the predator’s relative position and bearing, lethality and velocity^2,77. This well-understood natural paradigm offers substantial opportunities for decision neuroscience (FIG. 2).

Fig. 2 |. Example of foraging and escape choice, and the cost-of-fleeing and cost-of-staying curves.

a | The solitary prey (for example, a zebra) must make a decision about whether to flee to safety when a predator is approaching (for example, a lion). The flight initiation distance (FID) is the measurable distance at which the prey makes the decision to flee from an approaching threat². The FID is influenced by several factors, including the degree of threat posed by the predator (for example, whether it is a fast-moving or slow-moving predator) and the value of its current location (for example, there is an abundance of food or mates). b | The schematic represents the cost of fleeing versus remaining and is based on the model proposed by Ydenberg and Dill⁷⁶. As the distance between the prey and the predator decreases, the cost of fleeing decreases (red line), whereas the cost of not fleeing increases (blue line). d* represents the optimal distance between the prey and the predator at which the prey should flee. c | In situations in which there are multiple benefits of remaining at a particular patch but a single cost of fleeing, d* is longer for the higher of the two cost-of-not-fleeing curves. d | Conversely, d* is longer for the lower of the two cost-of-fleeing curves when there is a single cost-of-not-fleeing curve². d*_CH, high cost; d*_CL, low cost; d*_NH, not fleeing, high cost; d*_NL, not fleeing, low cost. Adapted with permission from REF.²,Cambridge University Press.

Flight initiation distance and the neural basis of escape.

In humans, fMRI has been employed during ‘active escape’ paradigms in which the goal was to escape from a virtual predator that possessed the capacity to chase, capture and shock the participant. In this experiment, the vmPFC was preferentially active when the virtual predator was distant, but this activity switched to the midbrain periaqueductal grey (PAG) as the threat moved closer⁷⁸. This finding was replicated⁷⁹ and extended by showing that panic-related motor errors correlated with activity in the midbrain encompassing the PAG and the dorsal raphe nucleus⁸⁰. Other researchers have shown that evading virtual predators is also associated with greater connectivity between the amygdala, the anterior cingulate cortex and the vmPFC⁸¹, and that the hippocampus is activated when individuals engage in threat engagement, a finding supported by investigations of patients with hippocampal⁸² or amygdala lesions⁸³ as well as animal and theoretical models^79,84–87. Together, these findings support a network of survival circuits that are involved in ecologically defined threats.

More recently, a flight initiation distance task was developed to investigate how survival circuits facilitate escape decisions when subjects encounter fast-attacking or slow-attacking threats⁸⁸. Fast-attacking predators were characterized by the virtual predator quickly switching from a slow-approach to a fast-attack velocity, thereby requiring the subject to make quick escape decisions. By contrast, slow-attacking predators slowly approached for longer time periods, resulting in larger buffer zones and more time to strategize escape. When subjects were faced with quick decisions to flee virtual predators with far attack distances, regions associated with reactive fear including the midbrain PAG and midcingulate cortex (MCC) were activated. Conversely, the close-attacking virtual predators with larger buffer zones elicited the activation of brain regions that are presumed to be involved in cognitive fear and strategic planning, including the vmPFC, hippocampus, posterior cingulate cortex and retrosplenial cortex. The authors then investigated how close each subject was to adopting the optimal escape strategy by applying a Bayesian decision-theory model. The model revealed the MCC as a candidate region for optimal escape from fast-attacking predators, consistent with the known role for this area in adaptive motor learning⁸⁹. Hippocampal activity was associated with optimal escape from slower-attacking threats, supporting a role for this region in escape only when the subject is not under time pressure. This study provides one example of how models from the field of ethology and behavioural ecology can be applied to decision neuroscience and demonstrates a previously unknown link between defensive survival circuits and their role in adaptive escape decisions.

Future directions

The development of a new decision neuroscience framework that is based on ethology and behavioural ecology provides a new set of paradigms that can be adapted to understand (human) decision-making. Such a framework links the guiding frameworks of evolutionary biology (BOX 1) with the formal tools used by ethologists and behavioural ecologists (FIG. 2). Moreover, it specifies experimental protocols and mathematical models that characterize the mechanistic and dynamic signals that support moment-to-moment decisions and highlights the role of state dependence in decision-making. Furthermore, it provides information about the environments in which priors and heuristics, which are necessary to cope with the complexity of natural decision-making tasks, evolved and developed.

Neuroscience has made considerable progress in identifying the function of particular brain regions in relation to particular tasks and, in some cases, generalization across tasks (for example, reward functions of the basal ganglia⁹⁰). Arguably, the next challenge is to understand the effect of trade-offs across tasks — how a particular mechanism is adapted for use on multiple tasks (thus being suboptimal on any one) and the more immediately tractable question of how coordination and control take place among different functions and the brain regions that support them (for example, in the case of self-control^91–93). Behavioural ecology may be particularly helpful in identifying relevant trade-off situations.

This more biologically relevant framework, along with evolutionary approaches to medicine⁹⁴, also sits well with the Research Domain Criteria (RDoC) framework advocated by the US National Institute of Mental Health, which aims to integrate disparate levels of information and provide a more foundational understanding of the neural and psychological dimensions that underpin healthy and abnormal brain function⁹⁵. Homologous studies in humans can create dynamic virtual ecologies that provide new dimensions to investigate adaptive and affective decision-making in healthy humans and across a broad range of psychiatric disorders. For example, a recent study showed that subjects who gambled more showed increased exploration on an explore–exploit foraging task and earned less money on a patchy foraging task^96,97. Moreover, other researchers have linked foraging behaviour to depression⁹⁸. Shifts in neural activity that are associated with changing proximity to threat allow researchers to examine the dynamic relationship between anxiety and fear circuits in individuals with affective disorders. Studies are also possible into how decisions associated with effort are impaired in mood disorders such as depression, in which lethargy and fatigue are rampant⁹⁹.

Finally, the approach we advocate is not a one-way street: it can be reciprocally illuminating. Ethological thinking by decision neuroscientists will in turn have an impact on ethologists and behavioural ecologists. The key issues of ethology are often based around ‘why’ questions; as Stephens and Krebs point out, “Asking what a machine is for helps the engineer understand how it works” (REF.³¹). Conversely, understanding relevant brain mechanisms might help ethologists understand what specific condition the mechanisms evolved for and will certainly help identify important constraints on information processing. There is evidence that decision science is having an impact on ethology, including the use of classic paradigms from behavioural economics such as risk aversion, delayed discounting and framing effect paradigms^100–102. The question of the calculational complexity of decision-making, which informs much thinking about process models and heuristics^28,103 in decision neuroscience, could offer relevant constraints for ethological paradigms. Following on from Tinbergen’s work, which identified different classes of biological questions (BOX 1), McNamara and Houston²⁹ have pushed for us to understand how function and mechanism have co-evolved; such ‘evo-mecho’ approaches are necessary for a comprehensive understanding of decision-making because it is not tenable to consider the evolution of just function or mechanism and to pretend that the other remained fixed. Gaining a deeper understanding of cognitive processes can help enable this integration and, in turn, benefit the fields of ‘cognitive ecology’ (REF.¹⁰⁴) and the study of social behaviour^105,106.

Concluding remarks

Like Mayr and Huxley, Tinbergen¹⁰⁷ suggested that we must take a multilevel approach to scientific questions in biology, writing, “If we do not continue to give thought to the problem of our overall aims, our field [ethology] will be in danger of becoming an isolated ‘ism’”. Likewise, Marr and Poggio opined that “Complex systems, like a nervous system … must be analysed and understood at several different levels” (REF.¹⁰⁸). Although decision neuroscience largely abides by Marr and Poggio’s framework, we suggest that it is in some danger of isolating itself by, as Tinbergen put it, “Losing touch with the natural phenomena” (REF.¹⁰⁷). We have attempted to address this problem by discussing three examples from the foraging literature that are central to studying decision-making in the natural environment and that bring together disparate fields to generate a multilevel approach to human decision-making.

A small, but growing, band of researchers are adopting new approaches to task design and analysis that are inspired by ethology and behavioural ecology. These approaches have given rise to new information about how the brain computes decisions under various conditions. A more critical outcome of these approaches is that they move the laboratory closer to nature¹⁰⁹, thereby allowing better understanding and prediction of how we make decisions in the real world. Crucially, the functional approach of ethology (and behavioural ecology) identifies fitness trade-offs that help us to understand how one set of approximately optimal rules (such as rate maximization when foraging) can, under different circumstances, conflict with rules that are approximately optimal in another task (such as predator avoidance). By understanding such trade-offs, we may better understand how to design neuroscientific studies to identify the (bounds on) conditional use of particular neural structures. Adopting this framework will thus enhance our understanding of natural diversity and likely improve applications for human health and well-being.