Protocol to Study Spatial Subgoal Learning Using Escape Behavior in Mice

Abstract

Rodent spatial navigation is a key model system for studying mammalian cognition and its neural mechanisms. Of particular interest is how animals memorize the structure of their environments and compute multi-step routes to a goal. Previous work on multi-step spatial reasoning has generally involved placing rodents at the start of a maze until they learn to navigate to a reward without making wrong turns. It thus remains poorly understood how animals rapidly learn about the structure of naturalistic open environments with goals and obstacles. Here we present an assay in which mice spontaneously memorize two-step routes in an environment with a shelter and an obstacle. We allow the mice to explore this environment for 20 min, and then we remove the obstacle. We then present auditory threat stimuli, causing the mouse to escape to the shelter. Finally, we record each escape route and measure whether it targets the shelter directly (a ‘homing-vector’ escape) or instead targets the location where the obstacle edge was formerly located (an ‘edge-vector’ escape). Since the obstacle is no longer there, these obstacle-edge-directed escape routes provide evidence that the mouse has memorized a subgoal location, i.e., a waypoint targeted in order to efficiently get to the shelter in the presence of an obstacle. By taking advantage of instinctive escape responses, this assay probes a multi-step spatial memory that is learned in a single session without pretraining. The subgoal learning phenomenon it generates can be useful not only for researchers working on navigation and instinctive behavior, but also for neuroscientists studying the neural basis of multi-step spatial reasoning.

Background

In previous work, rodent escape behavior has been used to study goal-directed spatial navigation in the context of obstacle-free environments. In the standard Barnes Maze assay (Barnes, 1979; Harrison et al., 2006), rodents are placed in an open-field arena with an underground shelter and presented with an ongoing aversive stimulus such as a bright light. Rodents locate the enclosed space in the environment (the shelter), instinctively adopt it as their home base, and run to it when faced with threatening stimuli such as loud sounds. Over multiple sessions, animals learn to navigate efficiently to the shelter using spatial memory. Work from our laboratory has provided three updates to this protocol (Vale et al., 2017, 2018): the learning period and escape testing all occur within a single session; the mouse’s self-motivated exploratory behavior proceeds without having to remove the mouse from the arena; and sudden-onset threat stimuli are used to evoke robust, shelter-directed escape paths. Our new protocol builds on that assay by adding (and dynamically modifying) structure in the environment. In this environment with an obstacle, mice learn a hierarchy of goals: the ultimate goal (the shelter) and two subgoal locations (the sides of the obstacle) that the animal uses to plan efficient routes to the ultimate goal. Thus, our protocol can allow researchers to investigate how animals rapidly learn multi-step routes to a shelter during a single session of self-motivated exploration.

Unlike this approach, prior work on multi-step spatial reasoning has focused on repeatedly placing an animal at the start of a constrained maze environment and testing how it learns to reach a food reward while minimizing erroneous turns (Tolman and Honzik, 1930; Sharma et al., 2010). These assays have several advantages: they assess long-term memory over multiple sessions/days; they induce stereotyped paths across animals; and they rely on a particularly stable and controllable source of motivation, i.e., hunger. However, they also leave out several key aspects of spatial reasoning. For one, they disregard how animals explore and rapidly compute routes in natural environments, which include both open space (allowing a much wider range of possible actions) and obstacles (necessitating multi-step reasoning). In addition, they lack a stimulus that can trigger immediate, goal-directed behavior. Thus, it is unclear if the animal’s ‘errors’ reflect a lack of understanding or merely a decision not to exploit a known route to the goal. Our assay—escape to shelter in the presence of obstacles—complements previous work on maze learning by incorporating these elements into the study of multi-step route learning.

Finally, escape to shelter in the presence of obstacles has been studied before, with gerbils (Ellard and Eller, 2009). Our protocol differs from this work primarily in its methods for evoking immediate and robust escape responses, ruling out visual-guided strategies, and quantitatively measuring subgoal learning. Our protocol was initially developed in Shamash et al. (2021) and then improved upon in Shamash and Branco (2021), which includes the automated threat presentation and removable obstacle panel features described below.

Materials and Reagents

1. Paper towels

2. C57BL/6J adult male or female mice, purchased from Charles River, 8–12 weeks of age. Animals were tested during the light phase of a 12-h light/dark cycle and had been singly housed for ≥5 days by the time of testing (see Note 1). No pretraining or habituation is necessary.

3. 70% ethanol

Equipment

1. Behavioral arenas

   A 92 cm diameter circular arena, made of 5 mm thick white acrylic, with a 10 cm × 50 cm rectangular hole cut from the middle; and a similar 92 cm diameter circular arena without a hole in it, to be placed beneath the platform with the hole (Figure 1).

2. Panel with obstacle

   50 cm long × 10 cm wide panel with a 50 cm-long × 12.5 cm tall obstacle attached along the central axis, made of 5 mm thick white acrylic (Figure 1).

3. Flat panel

   An identical 50 cm long × 10 cm wide panel (Figure 1B).

4. Stand to support the elevated behavioral arena, ≥20 cm tall

   The stand should stably support the 92 cm diameter arena without sticking out beyond the perimeter of the arena. For example, a 62.5 cm cubic base would work (Figure 1A).

5. Shelter

   10 cm long × 10 cm wide × 15 cm tall cube of transparent red acrylic (opaque to the mouse). It includes a mouse-hole-shaped entrance at the front (Figure 1).

6. Video camera

   acA1300-60gmNIR, Basler, with a near-infrared selective filter, positioned 1 m above the behavioral arena (Figure 1A).

7. Amplifier (Topaz AM10, Cambridge Audio) and speaker (L60, Pettersson) hooked up to the computer used to control the experiment.

8. Infrared LED illuminators (TV6700, Abus; 850 nm light) distributed above the platform for infrared video recording (we use six illuminators).

9. Sound level meter (Castle GA213)

10. Auditory stimulus sound files ( Supplementary Audio 1-2 ). Two sounds were downloaded from soundbible.com (‘smashing’ and ‘crackling fireplace’). They were then edited manually in Audacity 2.3.0, such that they were 1.5  s long and continuously loud.

11. Recommended: Infrared LED (850 nm OSLON PowerStar IR LED), hooked up to the auditory threat signal coming from the amplifier and therefore flashing infrared light toward the video camera whenever an auditory stimulus occurs.

12. Optional: A large, sound-proof box (160 cm wide × 165 cm deep × 190 cm tall). Alternatively, surround the arena with any dark material such as thin black plastic, at least 30 cm above the height of the arena, to make the environment less exposed and so less stressful for the mice.

13. Optional: Projector (BenQ) and projector screen (Xerox). We used a projector to illuminate the arena, because we also perform experiments with visual stimuli in the same arena. However, illuminating the arena with a lamp should be sufficient. During experiments, the screen was illuminated with uniform, gray light at 5 cd m^-2 (this measures the incoming light from the projector coming onto the projector screen). A rectangular hole was cut in the projector screen, and the video camera and speaker were placed just above this hole, 1 m above the center of the behavioral arena.

Figure 1. Shelter + obstacle environment.

A. Schematic of the relative positions of the equipment used to build and record from the shelter + obstacle environment. B. Top view of the behavioral arena. During the obstacle removal experiment, the obstacle panel is replaced with the flat panel, by the experimenter. The suggested threat zone (not visibly marked on the actual platform) is 50 cm long and extends 15 cm from rightmost point of the platform. C. Picture of the platform with the obstacle panel in place and the mouse peeking out of the shelter.

Software

1. Video recording and automated stimulus presentation program, written using Bonsai 2.4.1. Online mouse tracking was based on the mouse being darker than the white acrylic platform (sample code available at github.com/philshams/bonsai-behavior).

2. Post-hoc animal tracking, DeepLabCut (Mathis et al., 2018)

3. https://github.com/philshams/behavior-opto-analysis)

Procedure

Perform two types of experiments, in two groups of mice: a baseline-escape experiment in which escapes are triggered in an arena with no obstacle present, and an obstacle removal experiment in which mice explore the arena with an obstacle present for 20 min, and then the obstacle is removed and escapes are triggered. We recommend performing these experiments in two separate groups of mice. If it is necessary to reuse mice, allow 3–7 days in-between experimental sessions.

1. Set up and test the behavioral rig

   1. Place the acrylic arena with no hole on top of the stand and the arena with the rectangular hole on top of that. Obstacle removal experiment: Place the panel with the obstacle in the hole. Baseline-escape experiment: Place the flat panel in the hole.

   2. Test the auditory stimuli through the overhead speaker by placing the sound level meter on the perimeter of the arena, pointing towards the speaker. Ensure that it is playing at a volume of 84 dB.

   3. Test the video and automated stimulus presentation program, by moving a black, mouse-sized object into the threat zone and ensuring that an auditory stimulus is triggered.

   4. Ensure that the arena is lit just well enough for the mouse to easily see and target the obstacle (~5 cd m^-2; see Equipment 12 and Notes).

   5. Clean the arena, shelter, and obstacle by wiping them with 70% ethanol, and then wait at least 5 min before beginning an experiment.

2. Begin the behavioral session with a spontaneous exploration period

   1. Place the mouse in the arena in a consistent location across sessions, e.g., in between the obstacle and shelter, and initiate the behavioral program. Mice were picked up by the base of the tail, with a hand positioned to support their body as they were placed onto the arena. Previous handling of mice was limited to routine animal husbandry.

   2. The mouse should spontaneously enter the shelter within the first 7 min, and henceforth treat it as its preferred location. If this does not consistently occur, see the Notes below for help.

   3. Obstacle removal experiment: Allow the mouse to spontaneously explore the arena for 20 min total. Baseline-escape experiment: Allow the mouse to spontaneously explore the arena for 7 min total.

   4. Throughout the session: if the mouse does not leave the shelter for 5 min or does not enter the threat zone for 10 min, scatter one pinch (1 g) of bedding from its home cage in the threat area, for a maximum of once per session.

3. Obstacle removal experiment: Remove the obstacle from the arena

   1. After 20 min, as soon as the mouse enters the shelter, quietly lift and remove the obstacle panel and replace it with the flat panel. If this is not done stealthily, it could frighten the mouse, causing it to stay in the shelter for most of the experiment. Details will depend on the layout of the experimental room, but typically, this should be done with 15 s. If the mouse starts to leave the shelter after the panel switch was already initiated, we recommend calmly completing the switch.

4. Trigger auditory threat stimuli

   1. Use an automated software program (we use custom software in Bonsai) to track the mouse location online and trigger a threat stimulus in the following conditions: the mouse is currently in the threat zone, the mouse was in the threat zone 1.5 s prior, and the mouse has positive velocity in the direction opposite of the shelter.

   2. The threat stimulus consists of 1.5-s sounds, automatically played on repeat until the mouse gets within 20 cm of the shelter, or for a maximum of 9 s. A ‘crash’ sound and a ‘smash’ sound (Supplementary Audio 1-2) are alternated every other trial, to prevent stimulus habituation.

   3. If the mouse does not successfully escape during the stimulus, increase the stimulus volume by 2 dB, up to a maximum of 88 dB.

   4. End the experiment after one hour or six successful escape trials, whichever comes first.

Data analysis

1. Post-hoc tracking

   1. Use DeepLabCut to track the position of the mouse at each frame in the video. We find that averaging across multiple tracked body parts leads to more robust tracking. Specifically, we average 13 points: nose, left eye, right eye, left ear, right ear, neck, left upper limb, right upper limb, upper back, left lower limb, right lower limb, lower back, and tail base. However, a single point in the area of the mouse’s neck or upper back will work as well.

   2. Compute the mouse’s speed at each frame in the video and its speed relative to the shelter location, convert these to cm s^-1, and smooth these data with a Gaussian filter (σ = 100 ms, length = 800 ms).

2. Escape target score and trajectory classification

   1. To quantify escape trajectories, calculate the ‘initial escape target score’ (Figure 2). This is a metric where a vector aimed directly at the shelter receives a value of 0; one aimed at either obstacle edge receives a value of 1.0; a vector halfway between these scores 0.5; and a vector that points beyond the edge receives a value greater than 1.0. The initial escape target was computed by taking the mouse’s position when it is 10 cm in front of the obstacle and comparing the offset between this latter position to where it would have been if it escaped directly toward the shelter or toward the obstacle edge. This is calculated as

      $$\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}=\frac{|{\mathrm{o}\mathrm{f}\mathrm{f}\mathrm{s}\mathrm{e}\mathrm{t}}_{\mathrm{H}\mathrm{V}}-{\mathrm{o}\mathrm{f}\mathrm{f}\mathrm{s}\mathrm{e}\mathrm{t}}_{\mathrm{E}\mathrm{V}}+{\mathrm{o}\mathrm{f}\mathrm{f}\mathrm{s}\mathrm{e}\mathrm{t}}_{\mathrm{H}\mathrm{V}-\mathrm{E}\mathrm{V}}|}{2\mathrm{*}{\mathrm{o}\mathrm{f}\mathrm{f}\mathrm{s}\mathrm{e}\mathrm{t}}_{\mathrm{H}\mathrm{V}-\mathrm{E}\mathrm{V}}}$$

      where offset_HV is the distance from the mouse to the line between the mouse’s starting position and the shelter (the ‘shelter direction’ line in Figure 2); offset_EV is the distance from the mouse to the line between the mouse’s starting position and the obstacle edge vector (the ‘edge direction’ line in Figure 2); and offset_HV–EV is the distance from the shelter-direction line to the edge-direction line (Figure 2). If the escape path deviates to the left of the homing-vector path, then the left obstacle edge is used to calculate offset_EV and offset_HV–EV (and vice versa). Each data point receiving a score corresponds to one escape route.

   2. To classify an escape as either an edge vector or homing vector, threshold the initial escape target score using the 95^th percentile of scores from a group of mice escaping in an environment with no obstacle. In our hands, this value was 0.65. Expect 40–60% edge-vector responses in the subgoal-learning group.

   3. Computing where the mouse would have been if it ran directly toward the shelter or obstacle edge requires first identifying the escape initiation point. For a simple measurement of escape initiation, use the point at which the mouse exceeds a speed of 20 cm s⁻¹ relative to (towards) the shelter location. This is computed by taking the frame-by-frame difference in the mouse’s distance from the shelter. The key is to use a metric that is approximately where the escape begins but does not include the non-escape movements that sometimes occur after threat onset but before escape initiation.

3. Statistical permutation test

   1. Use a non-parametric permutation test to give more weight to mice that have performed more escape trials, while still scaling the degrees of freedom with the number of mice. Alternative tests do not work as well here: using each trial’s data in a t-test or Mann-Whitney test would improperly scale the degrees of freedom with the number of trials, and using a repeated-measures ANOVA would impose a Gaussian-noise assumption that the data are unlikely to follow. To test if group A and group B have different propensities to perform edge-vector escapes, use the signed difference in the groups’ mean pooled edge-vector probability as the test statistic (i.e., group B% edge vectors – group A% edge vectors). Generate a null distribution of this value by randomly shuffling the group labels for each mouse (i.e., in each shuffle, all of a given mouse’s escape trials are assigned to either group A or B). Then, find the percentile that the actual difference in means occupies within this distribution—this is the p-value for the hypothesis that group B does more edge vectors than group A.

4. Additional parameters

   1. Data exclusion: exclude mice with zero successful escape trials.

   2. Number of replicates: use 8–14 mice per condition.

Figure 2. Classifying escape trajectories.

The initial escape target uses the mouse’s position 10 cm in front of the obstacle (gray and green dots), normalized between 0 (direct path from the escape initiation point to shelter) and 1 (direct path from the escape initiation point to the obstacle edge location). Escape initiation (mouse silhouette on top) is where the mouse’s speed relative to the shelter exceeds 20 cm/s. The dotted line represents the location where the obstacle had been during the 20-min exploration period. A. A homing-vector escape response, with an initial escape target of 0.53. This is less than the threshold value of 0.65 at which point escapes are classified as edge vectors. B. An edge-vector escape response. C. Closeup of the offset distances used to compute the escape target score. Offset_EV (green) is the distance from the mouse to the edge-vector line. Offset_HV (gray) is the distance from the mouse to the homing-vector line. Offset_HV-EV (blue) is the length of the line connecting the edge-vector and homing-vector lines, with the constraint that this line must pass through the average of the two vectors at the point 10 cm in front of the obstacle (i.e., it must pass through the point corresponding to an escape target score of 0.5). Figure adapted from Shamash and Branco (2021).

Notes

There is one key ‘dimension’ along which mouse behavior tends to vary across individuals: shelter-resting vs. vigorously exploring. Both extremes can be problematic in terms of producing enough escape trials. Mice that stay in the shelter may never enter the threat zone. On the other hand, mice that prefer exploration to being in the shelter may fail to escape to shelter in response to the threat stimulus. For example, group-housed mice explore more but may fail to respond to multiple trials of threat stimuli, while singly housed mice explore less but respond more consistently to threat stimuli. Additional factors that promote shelter resting include: adding a pinch of bedding from the mouse’s cage into the shelter, increasing the background illumination, and using a mouse in multiple behavioral sessions. Factors that promote exploration include: adding a pinch of bedding to the threat zone, decreasing background illumination, performing experiments during the dark phase of the light-dark cycle, using food-restricted mice and using young (7–8 week old) mice.

Competing interests

The authors declare that there are no any conflicting and/or competing interests.

Ethics

All experiments were performed under the UK Animals (Scientific Procedures) Act of 1986 (PPL 70/7652) after local ethical approval by the Sainsbury Wellcome Centre Animal Welfare Ethical Review Body.