Neuroethological studies of fear, anxiety, and risky decision-making in rodents and humans

Abstract

Prey are relentlessly faced with a series of survival problems to solve. One enduring problem is predation, where the prey’s answers rely on the complex interaction between actions cultivated during its life course and defense reactions passed down by descendants. To understand the proximate neural responses to analogous threats, affective neuroscientists have favored well-controlled associative learning paradigms, yet researchers are now creating semi-realistic environments that examine the dynamic flow of decision-making and escape calculations that mimic the prey’s real world choices. In the context of research from the field of ethology and behavioral ecology, we review some of the recent literature in rodent and human neuroscience and discuss how these studies have the potential to provide new insights into the behavioral expression, computations, and the neural circuits that underlie healthy and pathological fear and anxiety.

Predation presents the organism with an omnipresent problem to solve including how to predict, avoid, escape and combat threats. This problem is compounded by other goal-oriented needs including sustenance, sexual replication via the best genetic mates and protection of kin. Nature is unforgiving, and solving the puzzles it presents is critical to survival and this has resulted in a set of actions that are learned during the organism’s lifetime or passed down from its ancestors. These innate and learned behaviors are instantiated in Darwinian theory, are both conscious and nonconscious and are supported by a physiology that is shaped by the organism’s ecological niche. This proposes that animals make dynamic decisions when under threat that include the tension between goal-oriented needs, energy consumption, and tactic choice, are each supported by independent, dependent and interacting biological systems. Progress in understanding these biological systems and their proximate behaviors, therefore, involves the convergence of disciplines across the life sciences that include ethology, behavioral ecology, computational, behavioral, and cognitive neuroscience, and evolutionary biology.

Ethology, the scientific study of the organism’s behavior in its natural environment, has a long history of investigating how organisms innately respond to threat. For example, Douglas Spalding initially observed that young chicks instinctively fear a hawk hovering over them[1]. Subsequently, Lorenz and Tinbergen used a wooden silhouette that mimicked a Hawk if moved right (short neck, long tail) or a goose if the silhouette was moved to left (long neck, short tail), and found that naïve chicks would exhibit escape response only when the silhouette’s ‘flying’ mimicked a Hawk [2]. These findings, albeit controversial, have been supported by studies showing that looming stimuli results in innate fear responses in non-human primates [3] and rodents ([4]). Contemporary questions in the field of ethology include how animals forage in patches with varying densities of predators, the distance at which the animal flees from threat, why animals live in groups and in what contexts different anti-predatory responses are evoked [5]. In the adjacent field of behavioral neuroscience, contemporary neurobiological models of fear have largely been based on fear conditioning studies (e.g., [6–8]) which are derived from ‘snapshot’ information. In this paper, we argue that for the neuroscientist to fully understand these as biological questions, it is critical that one should examine the organism’s reactions to stimuli in paradigms that mirror the environment in which the behavior evolved providing a ‘panoramic’ view that fill gaps in current understanding of fear.

Fear, anxiety, risk and survival: an ethological perspective

To the ethologist, the terms fear and anxiety are survival responses and reflect broad categories each having their own behavioral and contextual profile. Behavioral neuroscientists, ecologists and ethologist have carefully laid out both these profiles. For example, Kavaliers and Choleris [9] have proposed an ‘apprehension gradient’ which extends from the prey exhibiting no interest to complete preoccupation with the predator. Similarly, Blanchard and colleagues studies using rodents suggests three levels of danger; potential threat, distal threat, and proximal threat [10]. From the other side of the fence, behavioral ecologists such as Lima and Dill [11] have proposed that a set of scenarios can play out when the predator and prey come into contact (Figure 1). These include situations where the prey attempt to detect the predator first (p) and makes the decision to avoid (a). If detected by the predator, the prey will alter its decision and make the most appropriate response for that situation. Alternatively, the predator aims to optimize the situation to detect the prey (q) first and can decide to attack the unaware prey (1-i₂) or ignore (i₂). Importantly, the Lima and Dill model suggests that prey will optimize the avoidance of predators through risk allocation and escape strategies (e) and these depend upon the context and behavior of the predator.

Fig. 1.

Lima and Dill’s Predator-Prey Model. Flow chart displaying the permutations of a predator-prey encounters. The symbols signify the conditional probabilities of each step of the pathway. a= avoid; e = escape; i = ignore; p=probability that the prey detects the predator first; q = probability that the predator detects the prey first [11].

Several pioneers have made the link between ethological models and empirical behavioral neuroscience. One of the best known models is Bolles and Fanselow’s perceptual-defensive-recuperative (PDR) model [12] which states that when the animal perceives threats, its fear motivation system inhibits other motivational systems, such as pain, that impede defensive behavior. Presumably, attending to pain while facing a predator is not an adaptive behavioral trait. The PDR concept is supported by findings that during fear conditioning rats do not display injury-related behavior because fear elicits analgesia, via endogenous opioids, that suppresses pain [13]. The anti-predatory defensive behavior seems to be determined by the ecological niche of animals. For example, the woodland living P. m. austerus deermouse tends to freeze, which is effective against its natural predators such as a weasel, whereas the arid region residing P. m. gambeli deermouse displays vertical leap which is effective against its natural predators, such as a gopher snake [14].

Building on the PDR model, Fanselow and Lester [15] put forward the “Threat Imminence Continuum” model, which posits that distinct threat-states change depending on whether the threat context is absent, detected, or attacking. These different contexts of imminence evoke stereotyped defensive behaviors in rodents, where the animal will choose strategies to prevent or defer its progression down the imminence continuum. This continuum encompasses four core stages: the preferred phase is the time period when the animal is in a safe place, such as a burrow or nest; the pre-encounter phase is where the risk of threat is present, although there is no detectable presence of danger and characterized by increased vigilance and arousal; the post-encounter threat is when a threat is detected, but there is no direct interaction between the prey and predator (e.g. the predator has not yet detected the prey) resulting in freezing behaviors and adaptive autonomic responses; and the circa-strike threat is where the predator starts to pursue the aware prey with the intention of capture and consumption resulting in the pray either fleeing or fighting if the threat is inescapable. These contexts are further determined by the actual or perceived proximity to a threat [16]. Together, these models operationalized a set of boundaries from which to create semi-realistic experiments that attempt to exam defensive behaviors and their related neural circuits.

The emergence of neuroethological approaches to survival circuits

The emerging need for semi-realistic paradigms comes with the recent acceptance that threat responses and decisions-making are represented along a set of overlapping neural circuits. For example, several theorists have proposed that survival circuits are mapped along a distal-proximal danger hierarchy extending from the ventral prefrontal cortex - anterior cingulate - amygdala - hypothalamus – periaqueductal gray (PAG) pathway [17]. This neuroanatomical pathways is supported by research on non-human primates [18,19], rodents [20–22] and are also closely aligned with what Panksepp calls the FEAR circuitry [23]. These circuits have been further clarified by the mapping of parallel circuits via the hippocampus, septum, hypothalamus and PAG [24*] and the recent recognition of other important structures including the habenula, medial dorsal and paraventricular thalamus [25,26]. The most studied part of these circuits are the amygdala and PFC, both believed to be the hub and modulator of threat, yet new approaches including optogenetics and high resolution human neuroimaging are disseminating these circuits showing both complex local (basolateral amygdala-intercalated neurons-central nucleus of the amygdala) and global connectivity (e.g. PAG-PFC; [27,28]). The evolutionary purpose of these circuits is speculative, yet given the evidence from ethology and behavioral neuroscience these primitive to higher cortical survival pathways have likely evolved to enrich behavioral flexibility.

There is also increasing recognition that more than one circuit can produce the identical behavior and this may differ within and between species [29]. This becomes even more complex when we consider the hormonal, autonomic and neurotransmitter variables that are evoked during danger. To account for the diverse processes that occur during threat LeDoux [30**] has proposed that “defensive organismic state” is evoked and supported by “defensive motivational circuits” that are triggered in the presence of threat. Presumably, these “defensive motivational circuits” are determined by the context, which in turn result in a set of survival strategies that are optimized to escape predators [31**]. While LeDoux’s elegant theory puts forward the differences between humans and animals (e.g. the role of introspection and subjective states), it does not integrate theories from ethology and behavioral ecology instead focusing on the insightful findings from laboratory studies. Likewise, ethologists and behavioral ecologists have been equally guilty in their skirting of theories from behavioral and affective neuroscience. Elaborating on, and combining with, LeDoux’s theories, Fanselow and Lester’s TIC model and the work by behavioral ecologists such as Dill, Lima, Nonac, and Blumstein among others, Mobbs and colleagues have recently attempted to synthesize these fields by proposing that humans and possibly other organisms posses five core survival strategies when encountering contextually distinct levels of danger. These include the ability to (i) predict the sensory landscape by simulating possible encounters with threat and selecting the appropriate pre-encounter action and ability to evoke (ii) prevention strategies in which the organism manufactures safe environments. When a threat is encountered the (iii) threat orienting system is engaged to determine whether the organism ignores the stimulus or switches into a process of (iv) threat assessment, where the organism monitors the stimulus, weighs the threat value, predicts the actions of the threat, searches for safety, and guides behavioral actions crucial to directed escape. When under imminent attack, (v) defensive systems evoke fast reflexive indirect escape behaviors (i.e., fight or flight). These strategies map on to the aforementioned survival circuits and the threat imminence continuum, where prediction can result in both preferred and pre-encounter context resulting in vigilance; threat orienting and threat assessment results in post-encounter freezing and flight (depending on proximity and refuge); and circa-strike elicits flight or fight [15]. These strategies can be flexibly altered by a conscious modulatory system (e.g. reappraisal of threat) and updated via a number of learning processes characterized by computational theorists (e.g. [32].

Active Escape in Rodents and Humans

The Mouse Defense Test Battery (MDTB) has been developed to measure defensive behaviors when rodents are presented with an unconditioned predator stimulus [33,34]. The MDTB measures the rodent’s threat responses including flight, freezing, threat assessment and defensive attack (i.e. fight [35]. In the classic version of the experiment, the human experimenter is used as the threat stimulus. A rat (Rattus norvegicus) is placed in a 6m long runway, and presented with the threat stimulus. Results show that rapid flight is observed 97% of the times when the human approaches the rat, but abruptly switches to 100% freezing when the threat becomes imminent or if the escape route is blocked (i.e. closer of a door). When the threat is extremely close to the rat (about 1m) defensive attack in the form of jumping and biting is observed. These reactions are observed in both wild-type and laboratory-bred rats, yet some differences in timing and magnitude are observed in other strains (Long–Evans strain; [36–38]. These seminal studies demonstrated that distance and escapability can be used as a powerful tool to evoke distinct defensive reactions (e.g. fight, flight and freezing; Fig 2A).

Fig 2.

(A) The Blanchard model proposing that physical distance to threat and escape (flight) availability evokes distinct defensive response. (B) The AET showing the neural switches between the vmPFC and PAG associated with distal and proximal threat and midbrain activity correlated with panic-related motor errors. (C) Experimental set up for oscillating tarantula task and (D) an example of monitoring the threats movement showing that as the Tarantula move closure based on it previous position compared to moving further away from a closure position there was increased activity in the dorsal amygdala and bilateral BNST.

In humans, the existence of survival circuits is supported by brain imaging research using functional magnetic resonance imaging (fMRI) and employing Active Escape Paradigms (AEP) where the goal is to actively evade an artificial predator with the capacity to chase, capture and shock the subject. Mobbs and colleagues [39] used the AEP to show that when the artificial predator is distant, increased activity is observed in the ventromedial prefrontal cortex (vmPFC; Fig. 2B). However, as the artificial predator looms closer, a switch to increased activity in the midbrain PAG is observed. In another experiment, Mobbs and colleagues [39] attempted to directly examine the neural basis of Fanselow and Lester’s “Threat Imminence Continuum” by creating three contexts that mirrored pre and post-encounter threat and circa-strike attack. Consistent with Fanselow and Lester’s model [15], post-encounter threat elicited activity in forebrain areas, including the vmPFC, hippocampus, and amygdala. Conversely, active escape during circa-strike threat increased activity in midbrain areas. Furthermore, subjects showed increased coupling between the midbrain and mid-dorsal ACC and decreased coupling with the vmPFC, amygdala, and hippocampus, supporting the proposal of mutual inhibition between these defensive circuits (i.e. both defensive circuits cannot be active at the same time). Finally, the authors found that panic-related motor errors (i.e. wrong button presses resulting in collisions with the virtual walls of the maze) correlated with increased activity in the midbrain PAG and dorsal raphe nucleus [40] (Fig. 2B). These finding have been supported by studies using more realistic stimuli [41] (e.g. placing a Tarantula progressively closer to the subject’s foot) while also showing that keeping track of the threat movements is associated with increased activity on the bed nucleus of the stria terminalis (BNST), a region implicated in threat vigilance and sustained fear [42]. Together, these ethological inspired studies suggest that higher forebrain areas are involved in slower, deliberate actions to distant or potential threat, whereas imminent danger results in fast, “hard-wired,” defensive reactions mediated by the midbrain.

Risky Foraging in Rodents and Humans

According to the risk allocation hypothesis, animals allocate most of their defensive resources to situations of high predator threat [43]. These defensive responses are further determined by the frequency, or pulses, of high-risk predation and lost foraging opportunities are allocated to times of low predatory risk. Natural observations support the risk allocation hypothesis (e.g. [43] and suggest that predation plays a major role in foraging decisions. One of the earliest experiments—that simulated naturalistic situations of fear, avoidance, and appetitive behaviors being a meaningful, integrated part of animal’s lives—utilized a ‘Closed Economy’ paradigm where rats lived for extended periods in individual chambers consisting of a safe nest and a foraging arena that had to be entered to press levers to procure food and that could be rendered dangerous by the administration of footshocks [15,44]. Helmstetter and Fanselow [44] found that introduction of random shocks caused rats to decrease meal frequency but increase meal size such that they reduce exposure to footshocks in the foraging area while maintaining caloric intake. A recent study showed that random footshocks also caused rats to decrease pressing the lever distal to the nest, where it will take the animal longer to escape from shock, and increase pressing the lever proximal to the nest, where escape from shock will be quicker, and that the amygdala is necessary for the reorganization of foraging patterns [45**]. These shock-induced changes in the foraging pattern are consistent with the risk allocation hypothesis.

Another ethobehavioral paradigm exposed rats foraging for food to a programmed predator-like robot [46]. In this study, as the hunger-motivated rat approached the food, the artificial predator surged towards it, eliciting the rat to flee into the safety of its nest. The robot effectively mimicked a naturalistic threat because its size is relatively larger than the rat, and its shape (with eyes, moving jaw, and tail) and surging action simulate a predatory strike. The robot’s disruptive effects on the animal’s foraging varied as a function of the nest-food and food-robot distances, which indicates that rats can innately discern safe vs. dangerous foraging distances. This adaptive behavior depends on the amygdala as amygdalalesioned and -inactivated rats did not flee to the looming robot. The main advantage of using a robotic predator is that it allows reliable and quantitative interaction with the rat, which is not possible with real predatory animals, such as a cat. The fleeing behavior can be elicited reliably in naïve rats simply by stimulating their amygdala or dorsal PAG each time they approached the food[45**]. In contrast to rats that faced the predatory robot, however, with amygdala/PAG stimulation the animals were unable to procure food placed near the nest, presumably because the brain stimulation evoked the same magnitude of fear regardless of the nest-food distance. Interestingly, the amygdala stimulation effect was intact in PAG lesioned rats, but the dorsal PAG stimulation was blocked in amygdala lesioned/inactivated rats, indicating that the amygdala is downstream of the dPAG. These animal studies demonstrate that rats adjust their foraging behavior consistent with the risk assessment-based antipredator defensive models, such as predatory imminence [15], that postulate fear behaviors as coordinated reaction and action to the specific threat situation and its perceived proximity.

Behavioral ecologist and ethologists have noted the importance of predation during foraging [47]. Although there is no human analog to these studies, several researchers are beginning to examine the neural basis of risk taking during foraging. One of the first human brain studies of foraging was put forward by Mobbs, Hassabis and colleagues [48*]). In this study, the authors used a continuous-input foraging task where subjects were presented with two patches (left and right of the screen) and had to decide to stay or switch to the other patch based on the increasing or decreasing competition and reward frequency. The goal of the task was to maximize points, which were exchanged for money, by avoiding patches with high density of competition and low-reward rate. As it became increasingly disadvantageous to be in a patch (i.e. increasing competition and decreasing reward frequency), the authors observed increased activity in the dorsal anterior cingulate cortex (dACC) and the insula, which the authors speculate may be involved in the conscious urge to switch patches. Furthermore, individual differences in competition avoidance and reward drive were found. Results suggested that the amygdala steers preferences to avoid competition, while the dorsal putamen activity was associated with a drive to pursue reward. Other foraging studies have proposed that the vmPFC encodes the value of clear options and the dACC encodes the cost of foraging and average value of the foraging environment ([49] c.f. [50]).

Future directions

Ethologically inspired paradigms attempt to mirror the ecological conditions under which survival behaviors evolved. The preliminary studies reviewed here supplement current evidence using more traditional methods and have provided new insights into the brain’s survival circuits. The preliminary studies reviewed here supplement current evidence using more traditional methods and have provided new insights into the brain’s survival circuits. For example, the aforementioned studies on humans provide new information on how increasing danger results in the transfer of threat from corticolimbic ‘anxiety’ systems to midbrain ‘fear’ centers, how maladaptive motor responses are linked to the midbrain and the brain tracks the spatial patterns of the threat (Figure 2). Likewise, recent ethobehavioral studies in rodents indicate that the contemporary fear models derived largely from fear conditioning studies may be inadequate to address risky foraging behavior in a naturalistic, dynamic fear environment (Figure 3). Moving forward, semi-realistic studies will allow researchers to further elucidate contextual switching between defensive strategies, help formulate new approaches to test the changing dynamics of competition, reward and predation risk and how the brain integrates this information to produce the optimal foraging decisions and open up the use of formal computational approaches used by behavioral ecologists. These studies will require a paradigmatic shift in experimental design, moving beyond the oversimplified methods used in classical and instrumental conditioning, yet overcoming the obstacles of balancing tight control over conditions with the fluid dynamic parameters that are often noisy. However, the benefits are clear in that these new approaches will allow theorists to create new computational models that map closer to how humans and animals react to threat in the real world and unify a diverse set of fields from behavioral ecology, to cognitive neuroscience, and evolutionary biology. In this exciting age of the human connectome project, innovations in molecular-genetics techniques (e.g., optogenetics), advances in human brain imaging and computational methods, the creation of ethologically inspired paradigms will provide a greater match to real-world threat and provide researchers with a new window into the neural circuits that underlie fear and anxiety.

Fig. 3.

(A) A foraging rat facing a ‘predatory’ robot. Each time the rat approached the food pellet, the looming motion of the robot caused the rat to flee into the safety of the nest. Animals were unable to procure pellet located beyond certain distance but were able to retrieve pellet placed closed to the nest. (B) Same experimental design except either the amygdala or the dPAG is stimulated in naïve rats as they came near the pellet. Both amygdala and dPAG stimulation always elicited fleeing response in animals regardless of the pellet location. (C) Histology photographs show the tip locations for stimulation electrode and guide cannulae, and the extent of lesions. (D) Representative track plots from a rat with basolateral amygdala (BLA) stimulation, a PAG-lesioned rat with BLA stimulation, a BLA-lesioned rat with dPAG stimulation, and BLA-inactivated rat with dPAG stimulation. (E) Group mean (±SEM) latency to procure pellet (180 s = unsuccessful), and group mean (±SEM) number of times animals approached the pellet during the 180 s allotted time.