Hippocampal Processing of Ambiguity Enhances Fear Memory

Abstract

Despite the ubiquitous use of Pavlovian fear conditioning as a model for fear learning, the highly predictable conditions used in the laboratory do not resemble real-world conditions, in which dangerous situations can lead to unpleasant outcomes in unpredictable ways. In the current experiments, we varied the timing of aversive events after predictive cues in rodents and discovered that temporal ambiguity of aversive events greatly enhances fear. During fear conditioning with unpredictably timed aversive events, pharmacological inactivation of the dorsal hippocampus or optogenetic silencing of cornu ammonis 1 cells during aversive negative prediction errors prevented this enhancement of fear without affecting fear learning for predictable events. Dorsal hippocampal inactivation also prevented ambiguity-related enhancement of fear during auditory fear conditioning under a partial-reinforcement schedule. These results reveal that information about the timing and occurrence of aversive events is rapidly acquired and that unexpectedly timed or omitted aversive events generate hippocampal signals to enhance fear learning.

When one is driving a car, an unexpected swerve from another motorist may signal danger, yet it is virtually impossible to predict precisely when, or if, a collision will occur. Indeed, in a natural setting, most cues that signal an unpleasant experience provide little information about the precise timing or regular occurrence of the aversive stimulus. It is surprising, then, that cued Pavlovian fear conditioning, the dominant laboratory paradigm of aversive learning across most animal taxonomies, including humans, rarely incorporates these forms of variability that are present in virtually all fearful experiences. Although Pavlovian fear conditioning is an excellent tool for dissecting the substrates and mechanisms of associative learning, its most common implementations do not accurately model real-world fear learning.

Many neural structures support fear learning. These include the amygdala, a brain region often considered a store for long-term fear memories (Campese et al., 2016), and the hippocampus, a brain region that relays multisensory information to the amygdala (McDonald & Mott, 2016). Understanding the neural computations that occur in the brain during real-world fear learning directly informs hypotheses about the basis of dysfunction in pathological fear. Thus the absence of variability in the timing and occurrence of the aversive stimulus in standard cued-fear-conditioning paradigms is even more surprising when considered from a clinical perspective. Human patients with fear and anxiety disorders are known to be disproportionately affected by ambiguity surrounding the occurrence of aversive events. When aversive stimuli are administered unpredictably, patients with panic disorder display enhanced startle reactivity relative to healthy comparison subjects (Grillon et al., 2008). In addition, patients with generalized anxiety disorder are more likely than healthy control subjects to interpret ambiguous stimuli as threatening (Dugas, Gagnon, Ladouceur, & Freeston, 1998). Intolerance of uncertainty (Buhr & Dugas, 2002), even when uncertainty is not specifically related to aversive outcomes, is greater in patients with anxiety disorders than in human subjects without such disorders (Holaway, Heimberg, & Coles, 2006).

Numerous reports indicate that aversive learning systems acquire and use information about the expected time of reinforcement. For example, conditioned fear responses peak at the time of the expected aversive reinforcement (Drew, Zupan, Cooke, Couvillon, & Balsam, 2005; Shionoya et al., 2013). In addition, changes in the timing of aversive foot shock after an auditory cue renders an established fear memory labile (Diaz-Mataix, Ruiz Martinez, Schafe, LeDoux, & Doyere, 2013). Fewer studies have examined the impact on aversive learning of variability in the timing of reinforcement. Some researchers have reported that neurons in brain regions that process aversion, such as the amygdala, show greater firing (Belova, Paton, Morrison, & Salzman, 2007) and activation (Herry et al., 2007) in response to unpredictable stimuli than to predictable stimuli. Although many forms of ambiguity can influence aversive learning, we focused in the current work on how ambiguity in the timing of aversive stimuli relative to the onset of predictive cues influenced aversive learning.

We conducted a series of behavioral experiments in which we systematically varied ambiguity in the timing and occurrence of aversive reinforcement during auditory Pavlovian fear conditioning in rats. We speculated that these simple yet largely unexplored manipulations would change the neural substrate and strength of long-term fear memory. We hypothesized that ambiguity surrounding the occurrence of the aversive reinforcer during cued fear conditioning would enhance the strength of fear memory, despite a decrease in the information provided by the cue. We also hypothesized that the dorsal hippocampus, a brain region linked to temporal processing (MacDonald, Lepage, Eden, & Eichenbaum, 2011) and ambiguity (Vanni-Mercier, Mauguiere, Isnard, & Dreher, 2009), would be necessary for such an enhancement, despite a substantial literature arguing against a role for the hippocampus in cued fear memory (Kim & Fanselow, 1992; Maren & Holt, 2004).

Experiment 1

We aimed to determine whether variability in the timing of aversive reinforcement altered the strength of auditory fear memory. Rats were fear conditioned using either standard conditions, in which a tone of fixed duration coterminated with foot-shock administration (predictable-shock groups), or unpredictable conditions, in which a foot shock was delivered at a pseudorandom interval following each tone onset but during the tone presentation (unpredictable-shock groups). Contingency (defined as the probability of foot shock following the tone, held at 100%) and contiguity, factors that regulate associative memory strength (Bauer, LeDoux, & Nader, 2001; Rescorla, 1968), were consistent across the predictable- and unpredictable-shock groups. Additional factors held constant across the two groups included the intertrial interval (ITI), cumulative exposure to the auditory stimulus, and the number of foot-shock presentations. The primary difference between the two conditions was whether the tone onset provided information about the specific timing of subsequent foot shock. Although some prior studies presented foot shock at variable times within a predictive cue during fear conditioning (Rescorla, 1966, 1968), the strength of the fear memory that resulted was not compared with the strength of fear memory after fear conditioning in which the foot shock follows cue onset at a fixed interval. In addition, studies using probabilistic timing of reinforcement within predictive cues are unusual; the vast majority of contemporary studies using Pavlovian fear conditioning in rodents use fixed times relative to cue onset for reinforcement.

Method

An initial pilot experiment was conducted with at least 5 rats per group. After this, the experiment was replicated with additional rats. Because there were no significant differences between the results of the original and replication experiments (see Table S1 in the Supplemental Material available online), the two data sets were combined and analyzed as a single data set; this yielded group sizes that are standard for the field. All replications included representation across all of the groups. We adhered to the principles described in the Animal Welfare Act (2013), which include minimizing the number of animals used. Our targeted minimal group size, after any potential exclusions, was set a posteriori at 6. We stopped collecting data after we replicated each experiment twice; this rule was set a priori.

Subjects

The subjects were adult male Long-Evans rats (200–225 g) obtained from a commercial supplier (Taconic, Germantown, NY). Rats were housed individually in plastic cages on a 12-hr light/12-hr dark cycle (lights on at 7:00 a.m.). Chow and water were provided ad libitum. All procedures were approved by the Committee on Animal Care at the Massachusetts Institute of Technology and the Animal Care and Use Review Office of the Army Research Office.

Fear conditioning

Training and testing were conducted in conditioning chambers (30 × 24 × 21 cm; MED-Associates, St. Albans, VT) with aluminum sides and a clear polycarbonate door. The removable grid floors consisted of 19 steel rods (0.5 cm in diameter, 1.6 cm apart) that delivered the foot-shock unconditioned stimuli (USs). Chambers were located in sound-attenuating cubicles containing speakers through which auditory conditioned stimuli (CSs) were delivered. All training and testing was conducted between the hours of 10:00 a.m. and 5:00 p.m.

The interior of the chambers was manipulated to produce two distinct contexts. For Context A, the chambers were cleaned with 0.3% Pine-Sol (The Clorox Company, Oakland, CA), and chamber and room lights were on. Ventilation fans provided background noise. The animals were transported to and from the conditioning room in clear boxes. In Context B, a white plastic insert covered the chamber’s grid floor, and a rounded white plastic insert was placed against the back wall. In this context, the chambers were cleaned with 1% acetic acid. Chamber and room lights were turned off, and a red light provided room illumination. Animals were transported to and from the conditioning room in black boxes.

Rats were handled for at least 3 days before fear conditioning. Rats received fear conditioning in Context A with five pairings of tones (80 db, 2 kHz) and foot shocks (1 s, 0.7 mA); there was a 3-min period after the tone. For some rats, fear conditioning involved a 30-s tone (Fig. 1a); for other rats, fear conditioning involved a 42-s tone (Fig. 1b). Two groups of rats received predictable training, in which the foot shock was delivered at a consistent time after tone onset for every trial: 30 s after the onset of a 30-s tone (the predictable-shock/short-tone group) or 17 s after the onset of a 42-s tone (the predictable-shock/long-tone group). The other two groups of rats received one of two types of unpredictable training in which the timing of each foot shock varied after the onset of each tone. Rats in one unpredictable-training group (unpredictable-shock/short-tone) received fear conditioning with 30-s tones, but each foot shock was delivered at a pseudorandom time within the tone (6, 12, 18, 24, or 30 s after tone onset). Rats in the other unpredictable-training group (unpredictable-shock/long-tone) received fear conditioning with 42-s tones; foot shocks were also delivered at pseudorandom times within the tone (17, 23, 29, 35, or 41 s after tone onset).

Fig. 1.

Method and results for Experiment 1. Two of the groups of rats (n = 10 per group) were fear conditioned with five pairings of a 30-s tone with a 1-s foot shock, as illustrated in (a); there was a 210-s intertrial interval (ITI). For the predictable-shock/short-tone group (which received predictable training and a 30-s tone), each tone coterminated with a foot shock. For the unpredictable-shock/short-tone group (which received unpredictable training and a 30-s tone), each tone was paired with a foot shock that occurred pseudorandomly during the tone. Thus, interstimulus intervals (ISIs) were shorter for this group than for the predictable-shock/short-tone group. The other two groups of rats, the predictable-shock/long-tone and unpredictable-shock/long-tone groups, were fear conditioned the same way, except that the tones were 42 s instead of 30 s, as illustrated in (b). For the unpredictable-shock/long-tone group, each tone was paired with a foot shock that occurred pseudorandomly during the tone, so the average ISI was longer for this group than for the predictable-shock/long-tone group. The bar graphs show the mean percentage of time the rats displayed freezing behavior in each (c) short-tone group and (d) long-tone group, separately for contextual fear-memory recall and auditory fear-memory recall. The small open circles represent the percentage of time that individual rats displayed freezing behavior. Error bars represent +1 SEM. Asterisks indicate significant differences between groups (*p < .05, **p < .01).

The day after fear conditioning, all rats were returned to Context A for context extinction (20 min). The following day, an initial tone-extinction session was conducted; rats were placed in Context B and received 20 tone presentations. The day after that, rats in the predictable-shock/short-tone and unpredictable-shock/short-tone groups were returned to Context B for a second tone-extinction session.

Statistics

Freezing behavior (i.e., lack of motion) in rats was used as a measure of the strength of fear memory. Behavior was recorded throughout all sessions (digitized at 30 Hz), and freezing was detected offline using commercial software (Video Freeze; Med Associates, Fairfax, VT). Using a proprietary formula, the software computes a motion index throughout the recorded session; this value increases in proportion to the amount of movement in the test box. The threshold of freezing (i.e., the value of the motion index below which no movement is detectable) was determined, and the percentage of observations below this threshold was calculated for the times of interest (interval before the tone, tone presentation, and interval after the tone). Thus, freezing is reported as the percentage of time that rats displayed the freezing behavior within each period of interest. Motion had to be below the threshold for at least 1 s to be scored as freezing. Rats were excluded from all data analysis if, during the test of auditory fear-memory recall, they displayed high levels of freezing (> 80%) before the first tone presentation. Such behavior reflects inappropriate contextual generalization and interferes with the ability to attribute freezing specifically to the tone. Two rats were excluded from all analyses on the basis of this criterion. Conditioned freezing was compared using analyses of variance (ANOVAs), and planned comparisons were performed when the results of the analyses showed a significant omnibus F ratio. The data were analyzed using an ANOVA with factors of group (predictable-shock/short-tone vs. unpredictable-shock/short-tone, or predictable-shock/long-tone vs. unpredictable-shock/long-tone) or time (Bins 1–18 in the interval after the second tone).

Results

Despite the decrease in the information content of the tone onset under unpredictably timed foot-shock delivery, rats trained under these conditions displayed significantly stronger associative fear memories during auditory recall conducted in a novel context. There was a significant main effect of group, F(1, 18) = 6.31, p = .022 (Fig. 1c, right). In contrast, rats in the predictable-shock/short-tone and unpredictable-shock/short-tone groups displayed similar levels of freezing for contextual fear recall, F(1, 18) = 0.001, p = .98, which suggests that rats trained with unpredictably timed foot-shock delivery did not simply have nonspecific elevations in fear. The difference in contextual versus auditory fear memory might arise because the context provided no reliable information about timing of the aversive foot shock and was thus a comparably ambiguous cue for rats in the predictable-shock/short-tone and unpredictable-shock/short-tone groups, whereas the auditory stimulus was significantly less informative for the unpredictable-shock/short-tone group than for the predictable-shock/short-tone group.

Enhanced fear-memory strength after fear conditioning with unpredictably timed foot shocks cannot be attributed to second-order conditioning to the context in which the recall test was conducted or to altered fear extinction (see Experiment 1 in Supplemental Results in the Supplemental Material). It also cannot be attributed to changes in inhibition of delay (see Experiment 1 in Supplemental Results), a phenomenon in which conditioned behaviors are most highly expressed during the portion of the predictive cue that is closest to the timing of the reinforcer. It is also not likely to be due to the use of shorter intervals between the onset of the CS and the onset of the US in the unpredictable groups compared with the predictable groups (see Supplemental Discussion in the Supplemental Material).

A straightforward interpretation of these results suggests that ambiguity in the timing of aversive outcomes can enhance fear. Alternatively, multiple theories suggest that timing plays an important role in regulating the formation of associative memories (for a review, see Kirkpatrick & Balsam, 2016). In particular, these theories suggest that differences in the time between the onset of the predictive cue and the onset of reinforcement (the ISI; sometimes called the trial time) are critical for determining the rate and asymptote of learning. When the timing of the foot shock was altered from trial to trial (Fig. 1a), the ISI value and other related measures also varied across the predictable-shock/short-tone and unpredictable-shock/short-tone groups (see Table S2, top two rows, in the Supplemental Material). Thus, differences in memory strength across the two groups could have arisen from group differences in any of these conditioning parameters.

To test whether fear-memory strength was enhanced by ambiguity per se or was determined simply by the temporal duration of conditioning parameters, we ran a second set of rats for which the temporal differences between the predictable and unpredictable groups were systematically reversed from those used for the previous groups (see Table S2 in the Supplemental Material). The rats in the unpredictable-shock/long-tone group displayed significantly greater auditory fear-memory recall than rats in the predictable-shock/long-tone group; there was a significant main effect of group, F(1, 24) = 7.84, p = .0099 (Fig. 1d). As before, contextual fear recall was unaffected, F(1, 24) = 0.004, p = .95 (Fig. 1d), and no differences in extinction were observed (see Experiment 1 in Supplemental Results).

Thus, conditioned freezing in the unpredictable groups relative to that in the predictable groups was determined by the ambiguity surrounding the timing of the foot shock and not by temporal parameters that varied across the groups (see Table S2 in the Supplemental Material). These findings should not be used to discount the important role of timing in the acquisition of associative memory. However, it does suggest that the restricted differences in ISIs that we used in this experiment (e.g., 12-s maximum difference in mean ISI) were not sufficient to drive differences in learning, at least for amygdala-dependent fear behaviors.

These results reveal that fear learning, as measured by the magnitude of conditioned freezing, is exquisitely sensitive to temporal variability in the occurrence of negative events following predictive stimuli. This finding is surprising because rapidly acquired Pavlovian fear (as used in this experiment) evokes many behavioral and endocrine responses that are not regulated in a temporally precise manner. For example, the expression of most fear responses, including conditioned freezing, release of stress hormones, and changes in blood pressure and heart rate, is not limited to the CS. Instrumental avoidance responses, by contrast, must occur within a specific period relative to the CS to produce successful avoidance of aversive stimuli.

To determine when information about the timing of an aversive event is acquired during learning, we examined freezing on the conditioning day (Fig. 2). The percentage of time that the rats in the predictable-shock and unpredictable-shock groups displayed freezing behavior was statistically indistinguishable until the period after the second pairing of tone and foot shock; a 2 (group: predictable shock/short tone vs. unpredictable shock/short tone) × 18 (Time: Bins 1–18 in the interval after the second tone) ANOVA revealed a significant interaction, F(17, 306) = 1.6, p = .050; at this point, rats in the unpredictable-shock group exhibited higher levels of freezing than rats in the predictable-shock group. Thus, fear was heightened after the first trial in which the timing of the foot shock became ambiguous. This result strongly suggests that information about the timing between the onset of a cue and the occurrence of a subsequent aversive event is acquired during the first pairing of the cue and aversive event, which is surprising because novel cues do not necessarily have predictive value. Thus, one might expect that the need to encode the passage of time after the onset of a novel cue is minimal until repeated presentations of a cue indicate that the cue is associated with the occurrence of a significant event. However, the current finding is consistent with findings from other studies showing that, within aversive learning, temporal control of conditioned responding can sometimes emerge after very few conditioning trials (Davis, Schlesinger, & Sorenson, 1989; Drew et al., 2005).

Fig. 2.

Results from Experiment 1. Freezing behavior during the first two conditioning trials is graphed as a function of time, separately for rats in the predictable-shock/short-tone group (n = 10) and rats in the unpredictable-shock/short-tone group (n = 10). Each data point during the tone presentations represents the mean percentage of time that groups of rats displayed freezing behavior during a 2-s period; each data point during the intervals after the tones represents the mean percentage of time that groups of rats displayed freezing behavior during a 10-s period. Error bars indicate ±1 SEM. The asterisk indicates a significant difference between the behavior of the two groups (*p < .05).

Experiment 2

To further investigate the relationship between temporal ambiguity and fear, we examined whether systematic increases in temporal ambiguity predicted long-term strength of fear memory.

Method

Subjects

The subjects were obtained from the same source as in Experiment 1, and the veterinary care and experiment-approval process were identical to that of Experiment 1.

Fear conditioning

The fear conditioning and extinction testing were very similar to that of Experiment 1, with the following exceptions. Rats received fear conditioning with six pairings of 30-s tones (80 db, 2 kHz) and 1-s foot shocks (0.5 mA) with a 3-min posttone period. Relative to Experiment 1, a lower foot-shock intensity (0.5 mA vs. 0.7 mA) was used to insure ample room for any enhancement in fear by unpredictability of foot shock. Rats were assigned to one of three groups (Fig. 3a). Rats in the one-unpredictable-trial group received one foot shock 2 s after tone onset, and the five remaining foot shocks were given 20 s after tone onset. Rats in the three-unpredictable-trials group received three foot shocks 20 s after tone onset, and the timing of the three remaining foot shocks was variable (2, 12, or 28 s after tone onset). Rats in the six-unpredictable-trials group received six foot shocks, the timing of which varied (2, 9, 16, 20, 25, or 30 s after tone onset). Thus, the average interval from CS onset to US onset was held constant across all groups (17 s). The day after fear conditioning, all rats were returned to Context A for context extinction (20 min). The day after that, rats were placed in Context B and received 15 tone presentations.

Fig. 3.

Method and results from Experiment 2. Three groups of rats (n = 6–8 per group) were fear conditioned with pairings of a 30-s tone with a 1-s foot shock, as illustrated in (a); there was a 3-min interval after the tone. The three conditions had one, three, or six pairings of a tone and a foot shock. The average interval between tone onset and foot shock onset was held constant across the three groups. The day after fear conditioning, all the rats were returned to the conditioning context for 20 min. The day after that, the rats were placed in a novel context, and auditory fear recall was measured as average freezing across the first 2 of 15 tone presentations. In (b), results are shown separately for the two groups. The bars show the mean percentage of time that groups of rats displayed freezing behavior, and the small open circles represent the percentage of time that individual rats displayed freezing behavior. Error bars represent +1 SEM. Asterisks indicate significant differences between the group with one predictable trial and each of the other groups (**p < .01).

Statistics

The initial experiment was conducted with 2 to 3 rats per group. Two replications were run with an additional 2 to 3 rats per group, for a total of three runs. Because there were no significant differences between the results of the original and replication experiments (see Table S1 in the Supplemental Material), the three data sets were combined and analyzed as a single data set. Sample size and the point at which data collection stopped were determined as described for Experiment 1; the second replication was added at the request of the reviewers. Statistical analysis was conducted as described in Experiment 1. The data were analyzed using an ANOVA with a factor of group (predictable-shock/short-tone, unpredictable-shock/short-tone). Two rats were excluded from all data analysis because of high (> 80%) pretone freezing during the tone-extinction test; such behavior indicates inappropriate generalization of fear.

Results

We found that increasing levels of ambiguity also increased fear-memory strength; there was a significant main effect of group, F(2, 16) = 6.76, p = .0070 (Fig. 3b), which provides further support for a direct relationship between temporal ambiguity and the strength of long-term fear memory. In addition, because the average interval between CS onset and US onset was held constant across the three conditions, this experiment provided further support for the idea that ambiguity in the timing of the aversive foot shock is the key factor in fear-memory enhancement observed after conditioning with an ambiguously timed aversive reinforcer.

Experiment 3

The previous results show that temporal ambiguity of aversive reinforcement enhances fear. However, little is known about the neural substrates of ambiguity signals. Some studies using neuroimaging in humans have shown that the hippocampus is one brain region that consistently exhibits an elevated blood-oxygenation-level-dependent response when an unexpected aversive stimulus is presented (Ploghaus et al., 2000). However, numerous fear-conditioning studies have argued that the hippocampus is not involved in fear conditioning in response to discrete cues (Kim & Fanselow, 1992; Maren & Holt, 2004). Here, we explore the role of the hippocampus in ambiguity.

Method

Subjects

The subjects were obtained from the same source as in Experiment 1, and the veterinary care and experiment-approval process were identical to that of Experiment 1.

Cannula implantation

One week after the rats’ arrival, cannulae were implanted. Rats were anaesthetized using a cocktail of ketamine, xylazine, and acepromazine (75, 8, and 1.5 mg/kg, respectively, ip) and then mounted in a stereotaxic apparatus (David Kopf Instruments, Tujunga, CA). Small burr holes were drilled in the skull for placement of the guide cannulae and three jeweler’s screws. Bilateral stainless steel guide cannulae (23 gauge, 10 mm) were implanted, aimed at the dorsal hippocampus (3.8 mm posterior and 2.5 mm lateral to bregma and 1.8 mm ventral to dura; for the rationale behind targeting dorsal rather than ventral hippocampus, see Supplemental Discussion in the Supplemental Material). The cannulae and screws were affixed to the skull using dental acrylic, and a dummy cannula (11 mm) was inserted into each guide cannula to prevent obstruction. Each animal received a postoperative injection of buprenorphine (0.03 mg/kg sc). Animals recovered for at least 5 days before behavioral training commenced.

Intracranial infusions

Rats received intrahippocampal infusions 20 min before fear conditioning. Injection cannulae (30 gauge, 11 mm) were attached to 10-µl Hamilton syringes (Hamilton, Reno, NV) via polyethylene tubing (PE-20; Intramedic, Sparks, MD), and the syringes were mounted in an infusion pump (Harvard Apparatus, South Natick, MA). Animals were transported to the infusion room, and the dummy cannulae were replaced with injectors. The rats were placed in plastic buckets containing shredded wood chips. Rats were bilaterally infused with either 0.9% saline (vehicle) or 1 µg/µl muscimol (labeled with fluorescent boron-dipyrromethene; Thermo Fisher Scientific, Waltham, MA) at a rate of 0.15 µl/min for a total infusion volume of 0.5 µl per side (Figs. 4a and 4b). Muscimol is an agonist of the γ-aminobutyric acid type A receptor that causes transient inactivation of targeted regions. After infusion, the injection cannulae were left in place for an additional minute to allow the drug to diffuse before injector removal.

Fig. 4.

Results from Experiment 3. As illustrated in (a), the rats (n = 8–12 per group) received intrahippocampal infusions of muscimol or vehicle before auditory fear conditioning (30-s tone, 2-s foot shock, and 210-s intertrial interval) with foot shocks delivered at either predictable or unpredictable intervals after tone onset. The next day, all the rats were returned to the conditioning context for an 8-min contextual fear-memory test. The day after that, the rats were placed in a novel context, and eight tones were presented (auditory fear-memory test). A representative photomicrograph (b) shows one brain hemisphere after infusion with fluorescent muscimol. The pink fluorescent signal indicates the spread of the muscimol. The bar graphs (c) depict the mean percentage of time the rats displayed freezing behavior over the test sessions for contextual fear-memory recall (left) and auditory fear-memory recall (right). The bars show the mean for each group, and the small open circles represent the percentage of time that individual rats displayed freezing behavior. Asterisks represent significant differences between groups (*p < .05, **p < .01, ****p < .0001). Freezing behavior during the first two conditioning trials (d) is graphed as a function of time, separately for rats from each group. Each data point represents freezing during a 3-s period averaged over the first two trials of tone presentation during extinction. Error bars in (c) represent +1 SEM; error bars in (d) represent ±1 SEM.

Fear conditioning

The fear-conditioning procedure was similar to that in Experiment 1, except that rats received auditory fear conditioning with a lower foot-shock intensity (0.5 mA). The day after fear conditioning, all the rats were returned to Context A for context extinction (8 min). The day after that, the rats were placed in Context B and received eight tone presentations.

Histology

The animals were anaesthetized with isoflurane and perfused transcardially with 0.9% saline followed by 10% formalin. The brains were removed and postfixed in 10% formalin for 48 hr, then transferred to a solution of 30% sucrose and 10% formalin in saline. Using a cryostat, the brains were sectioned into 50-µm slices, and alternating slices were mounted. Sections were stained with 1% cresyl violet and examined by light microscopy to visualize cannula placement.

Statistics

The initial experiment was conducted with 3 to 4 rats per group. Two replications were run with an additional 3 to 4 rats per group, for a total of three runs. Because there were no significant differences between the results of the original and replication experiments (see Table S1 in the Supplemental Material), the three data sets were combined and analyzed as a single data set. Sample size and the point at which data collection stopped were determined as described for Experiment 1; the second replication was added at the request of the reviewers. Statistical analysis was conducted as described in Experiment 1. The data were analyzed using ANOVAs with factors of training type (predictable shock vs. unpredictable shock), infusion condition (vehicle vs. muscimol), or time (ten 3-s bins within tone presentation). Eight rats were excluded from all data analyses because of incorrect cannula placement (n = 6) or extensive gliosis at the cannula tip (n = 2).

Results

Rats received infusions of either muscimol or vehicle in the dorsal hippocampus before fear conditioning with either temporally predictable or unpredictable foot shocks (Figs. 4a and 4b). One indication of the effectiveness of our hippocampal-inactivation procedure is that contextual fear memory was abolished in all the rats that received intrahippocampal muscimol before fear conditioning (Fig. 4c); there was a significant main effect of infusion condition, F(1, 35) = 18.72, p = .0001, a finding consistent with results from other studies (Zhang, Bast, Xu, & Feldon, 2014).

In contrast, although we observed a significant effect of hippocampal inactivation on auditory fear memory, the effects of hippocampal inactivation were different in the PRED and UNPRED groups (Fig. 4c); the Training Type × Infusion Condition interaction was significant, F(1, 35) = 6.70, p = .014 (see also Experiment 3 in Supplemental Results in the Supplemental Material). Hippocampal inactivation before standard auditory fear conditioning had no effect on fear-memory strength (Fig. 4c, right, black vs. white bars; planned comparison, p = .57), which is a finding consistent with those from prior studies showing that hippocampal activity does not play a necessary role in the acquisition of learned fear (Kim & Fanselow, 1992). However, inactivation of the hippocampus during temporally unpredictable fear conditioning completely abolished the memory-enhancing effects of temporal unpredictability of the aversive event (Fig. 4c, red vs. gray bars; planned comparison, p < .0001). Because muscimol reduced fear only in the unpredictable-shock group, it is clear that the auditory stimuli used in our temporally unpredictable conditioning paradigms (30 s in length) did not function as diffuse, contextual cues to be associated with foot shock. If our auditory stimuli functioned as contextual cues, hippocampal inactivation would have reduced fear memory in both the predictable- and unpredictable-shock groups.

To determine whether the temporally unpredictable foot shock or muscimol infusions simply altered the time course over which conditioned freezing was expressed, we performed an analysis of tone-evoked conditioned freezing in 3-s bins during auditory fear extinction (Fig. 4d). Replicating our previous findings, we observed that there were significant changes in conditioned freezing over the course of the tone presentation, F(9, 153) = 7.00, p < .0001, and higher levels of conditioned freezing during the last 3 s relative to the first 3 s, F(1, 20) = 12.3, p = .0020. These findings suggest mild inhibition of delay across groups. However, the predictable- and unpredictable-shock groups did not differ in the rate at which conditioned freezing changed across the presentation of the tone. Neither the Training Type × Time interaction, F(9, 153) = 1.37, p = .21, nor the Training Type × Infusion × Time interaction, F(9, 153) = 0.61, p = .79, was significant. Thus, neither muscimol infusions nor temporally unpredictable fear conditioning altered conditioned freezing by reshaping the temporal window during which behavior was expressed.

Experiment 4

Prediction errors, which are generated when outcomes and expectations do not match, are thought to drive associative learning; larger prediction errors lead to greater changes in learning (Pearce & Hall, 1980; Rescorla & Wagner, 1972). Many factors may enhance the magnitude of a prediction error, including the size and the timing of reinforcement. The hippocampus has been implicated in aversive prediction errors (Goosens, 2011; Ploghaus et al., 2000). When aversive reinforcement follows predictive cues at variable intervals, this theoretically leads to two forms of prediction error: (a) when the reinforcer occurs at an unexpected time (positive prediction error) and (b) when the reinforcer is omitted at an expected time (negative prediction error). Because dorsal hippocampal inactivation does not affect fear conditioning when foot shock is given at a consistent temporal interval after onset of the auditory cue, it is highly unlikely to be involved in generating positive prediction errors to drive learning. However, the selective recruitment of the dorsal hippocampus when foot shock is administered at ambiguous times suggests that the hippocampus may be involved in generating or processing aversive negative prediction errors. To further investigate this possibility, we performed an experiment to determine whether the dorsal hippocampus enhances fear during partial reinforcement, another type of fear conditioning in which aversive negative prediction errors may be generated.

Method

Subjects

The subjects were obtained from the same source as in Experiment 1, and the veterinary care and experiment-approval process were identical to that of Experiment 1.

Cannula implantation and intracranial infusions

The surgical and infusion procedures were similar to those used in Experiment 3, but nonfluorescent muscimol (Sigma-Aldrich, St. Louis, MO) was used.

Fear conditioning

The fear conditioning and extinction testing were similar to those used in Experiment 1, with the following differences. Rats received auditory fear conditioning with shorter tones (10 s) and a shorter ITI (70 s). One cohort of rats received auditory fear conditioning in Context A with either a fully reinforced schedule (full-reinforcement group) of three pairings of the 10-s tone (80 db, 2 kHz) with a 2-s foot shock (0.7 mA), whereas a second group received fear conditioning under a partially reinforced schedule (partial-reinforcement group) of three pairings of tone and foot shock, with three additional unpaired tones pseudorandomly interspersed between the pairings (Fig. 5a). On this schedule, the relationship between the predictive cue and the foot-shock reinforcer is degraded. The total time in the fear-conditioning chamber (9 min), as well as the timing of the pairings of tones and foot shocks, relative to placement of the rat in the fear-conditioning chamber, was held constant across both experimental conditions. The day after fear conditioning, all rats were returned to Context A for context extinction (10 min). The day after that, rats were placed in Context B and received 14 tone presentations (Fig. 5a).

Fig. 5.

Method and results from Experiment 4. Four groups of rats (n = 5–7 per group) received intrahippocampal infusions of muscimol or vehicle before auditory fear conditioning (10-s tone, 2-s foot shock; 70-s intertrial interval), as illustrated in (a). Conditioning was either partially reinforced (50% chance of a foot shock following a tone) or fully reinforced (100% chance of a foot shock following a tone). The next day, all the rats were returned to the conditioning context for 10 min. The day after that, all rats received 14 tone presentations, and auditory fear recall was measured as average freezing across the first two tone presentations for each group. The bar graphs in (b) show the mean percentage of time the rats displayed freezing behavior in each reinforcement-schedule group, separately for contextual fear-memory recall and auditory memory-fear recall. The small open circles represent the percentage of time that individual rats displayed freezing behavior. Error bars represent +1 SEM. Asterisks represent significant differences between groups (*p < .05, **p < .01).

Histology

Placement of cannulae was assessed as described for Experiment 3.

Statistics

The initial experiment was conducted with 3 to 4 rats per group. A replication was run with an additional 3 to 4 rats per group, for a total of two runs. Because there were no significant differences between the results of the original and replication experiments (see Table S1 in the Supplemental Material), the two data sets were combined and analyzed as a single data set. Sample size and the point at which data collection stopped were determined as described for Experiment 1. Statistical analysis was conducted as described in Experiment 1. The data were analyzed using ANOVAs with factors of infusion condition (vehicle vs. muscimol) and training type (partial reinforcement vs. full reinforcement). Some rats were excluded from all data analysis because of poor cannula placement (n = 3) or high pretone freezing (> 80%) during the tone-extinction test (n = 4). Because of a software error, video data from the context extinction test for one group of rats (n = 4) were not saved; data from these rats were not included in that analysis (Fig. 5b, left).

Results

Rats received intrahippocampal infusions of either muscimol or vehicle before auditory fear conditioning either under a partial reinforcement schedule or a full reinforcement schedule (Fig. 5a). Partial reinforcement can be achieved either by adding additional presentations of the cue in the absence of foot shock or by removing presentations of the foot shock after some of the cues during conditioning (Bouton & Sunsay, 2003). The latter strategy would be expected to decrease the overall level of fear learning achieved by partial reinforcement (relative to the levels achieved after full reinforcement) because the asymptote of associative learning is determined, in part, by the number of pairings of the CS and the US (Rescorla & Wagner, 1972). Thus, we pursued the former strategy, holding the number of pairings of the CS and the US constant between groups; this enabled us to insure that any fear-conditioning differences that arose between rats trained under the partial- and full-reinforcement schedules were not attributable to different numbers of pairings of the CS and the US. We deliberately used conditions that would produce moderate levels of conditioned freezing in rats in the full-reinforcement conditions (three CS-US pairings) so that any effect of ambiguity on freezing in the PAR group could be more readily detected.

We found that hippocampal inactivation significantly affected auditory fear-memory recall (Fig. 5b, right); there was a significant main effect of infusion condition, F(1, 19) = 5.59, p = .029. The partial-reinforcement schedule led to a decreased contingency relationship (50% chance of a foot shock following a tone) between the cue and foot shock compared with the full-reinforcement schedule (100% chance of a foot shock following a tone). However, we found that the fear memory that resulted from the partial-reinforcement schedule was significantly stronger than the fear memory that resulted from the full-reinforcement schedule (Fig. 5b); there was a significant main effect of training type, F(1, 19) = 4.47, p = .048. Although higher levels of contingency can support greater asymptotes of learning, there are numerous cases in which conditioned responding is greater after partial reinforcement than full reinforcement (Goodrich, 1959), an effect that may be exacerbated when reinforcement is strong (Wagner, 1961). In addition, partial reinforcement has long been known to elevate conditioned responding (Humphreys, 1939), a phenomenon termed resistance to extinction.

Note that hippocampal inactivation fully reversed the fear-enhancing effect of ambiguity (Fig. 5b, right, red bar vs. gray bar; planned comparison, p = .0020) but had no effect on fear memory in rats trained under the full-reinforcement schedule (Fig. 5b, right, black bar vs. white bar; planned comparison, p = .87). These manipulations did not affect within-session fear extinction (see Experiment 4 in Supplemental Results in the Supplemental Material). The stronger conditioned responding after partial reinforcement is often explained by postulating that animals trained under partial reinforcement take longer to appreciate that the extinction phase is distinct from training, whereas animals trained under full reinforcement can instantaneously appreciate the difference between training and extinction on the first nonreinforced presentation of the CS during extinction. Our results raise the intriguing possibility that the enhanced conditioned responding seen after partial reinforcement arises, in part, from a signal generated by dorsal hippocampus during conditioning itself.

The two muscimol-infused groups showed minimal conditioned freezing during the contextual fear-memory test (Fig. 5b, left); there was a significant main effect of infusion condition, F(1, 16) = 9.42, p = .007, which demonstrated the efficacy of our hippocampal inactivation and revealed that the acquisition of contextual fear was blocked by the inactivation procedure.

The addition of CS presentations not only degraded the contingency relationship between the CS and US in rats trained under partial reinforcement, it also increased the CS density in the conditioning session, relative to conditioning with full reinforcement (i.e., the percentage of the session occupied by the CS was 11.1 and 5.5, respectively). Higher CS densities have been shown to support higher levels of conditioning (Kitaguchi, 2000); thus, one might predict that the higher levels of conditioned fear observed in the partial-reinforcement group relative to the full-reinforcement group arise from the higher CS density during conditioning rather than from any difference in the ambiguity of the CS. By this logic, one might conclude that the dorsal hippocampus is selectively recruited by conditioning paradigms with higher CS densities, a measure that requires assessment of the passage of time. However, other studies have shown that dorsal hippocampal inactivation with muscimol does not affect the auditory fear memory that results from conditioning using a CS density comparable to (9.4%; Maren & Holt, 2004) or even higher than (15.3%; Raybuck & Lattal, 2011) that used in our study. Thus, our observation that inactivation of the dorsal hippocampus eliminates the fear-memory-enhancing effect of partial reinforcement is most consistent with a role for the hippocampus in processing ambiguity. Collectively, these data reveal that ambiguity of predictive cues leads to enhanced fear memory for those cues, and the dorsal hippocampus plays an essential role in the computation of this ambiguity during fear learning.

Experiment 5

The results of Experiments 3 and 4 are consistent with the idea that the dorsal hippocampus generates temporally based aversive negative prediction errors to enhance fear-memory strength. However, they are also consistent with the idea that the dorsal hippocampus may function as a stopwatch, computing the passage of time between the start of a predictive cue and the occurrence of the reinforcer (MacDonald et al., 2011).

To distinguish between these two roles for the dorsal hippocampus, we performed an experiment in which we used optogenetics to selectively and briefly silence dorsal hippocampal cells in cornu ammonis 1 (CA1). This silencing took place in mice that received fear conditioning in which foot shock occurred pseudorandomly after predictive auditory cues. The cues occurred either during putative aversive negative prediction errors or during times in which foot shock was never presented. Mice were used because viral infusion and laser-light delivery could be targeted to a greater portion of the dorsal hippocampus than would be the case if rats were used. If silencing simply disrupts a stopwatch function, then silencing at any time within the CS presentations during fear conditioning should produce an equivalent decrease in subsequent fear-memory strength. In contrast, if silencing blocks prediction errors, then the specific timing of the inactivation should determine whether there is an effect on fear memory.

Method

Subjects

The subjects were adult male mice (7–8 weeks of age) obtained from a commercial supplier (Charles River Laboratories, Wilmington, MA). The mice were group housed in plastic cages on a 12-hr light/12-hr dark cycle. Chow and water were provided ad libitum. There were two groups of mice: mice that received surgical implants and mice that did not. All the mice were moved to individual cages 1 month before behavior training. In the case of the mice with implants, this was immediately after surgery. The mice were left undisturbed for 3 weeks to recover from surgery; they were then handled for at least 7 days before the optogenetic and behavioral experiments. Approvals were as described for Experiment 1.

Viruses

An expression cassette containing the gene coding for the light-driven outward proton pump archaerhodopsin from Halorubrum sodomense strain TP009 (ArchT; X. Han et al., 2011) fused to green fluorescent protein (GFP) was placed in a plasmid under the control of the CAG promoter. A second plasmid without ArchT expressed GFP under the control of the CAG promoter. The two types of plasmids were each packaged in with adenoassociated virus (AAV) serotype 1 capsids. Viruses were obtained from the Vector Core at the University of North Carolina, Chapel Hill. Viral stocks were diluted to 1.25 × 10¹¹ infectious particles per milliliter, and a total of 2 µl was injected into each hemisphere of a mouse’s dorsal hippocampus. Because ArchT is a membrane-trafficked protein, it is preferentially expressed in the dendrites and axons, rather than the cell bodies, of transduced cells. The fused GFP can be readily observed in the dendritic branches and axon terminals of the CA1 field.

Virus injection and fiber-optic cannula implantation

Mice that had been assigned to receive implants were mounted in a stereotaxic apparatus under isoflurane anesthesia. Small burr holes were drilled on the skull for bilateral infusion (2 µl at 0.1 µl/min) of AAV expressing either GFP only (GFP groups) or GFP and the silencing opsin ArchT (ArchT groups) targeting the CA1 of dorsal hippocampus at 2.3 mm posterior and ±1.75 mm medial to bregma, and 1.5 mm ventral to dura. Anatomical specificity of the targeting was conferred by the stereotaxic coordinates, not the properties of the virus; any mice with viral infections extending beyond CA1 were excluded from all analyses. Injections were made using an UltraMicroPump 3 (World Precision Instruments, Sarasota, FL) containing a 10-µl Hamilton syringe with a 33-gauge needle. After the completion of the infusion, the needle remained in position for an additional 10 min. Two small jeweler’s screws were placed in the skull. Zirconia ceramic ferrules (Kientec Systems, Palm City, FL) containing a multimode optical fiber (with a diameter of 200 µm) were then lowered into the site of injection and secured with dental acrylic. Mice recovered for at least 4 weeks before behavioral experiments.

Optical silencing of hippocampal CA1 neurons

A 532-nm green laser diode (100 mW; Shanghai Laser & Optics Century Co., Shanghai, China) was coupled to a 200-µm multimode silica-core optical fiber through an FC/PC adapter. A fiber-optic rotary joint (Doric Lenses, Quebec, Canada) was used to release torsion in the fiber caused by the animal’s rotation. Laser output was controlled via a transistor-transistor logic pulse generator (National Instruments, Austin, TX), with timing controlled by Python 2.6 software (Python Software Foundation, Beaverton, OR). Before implantation, an optical power meter (815-C; Newport, Irvine, CA) was used to ensure that the fiber optics delivered 10 mW of constant laser light. Laser light was applied for 4-s periods to induce photoinhibition.

Fear conditioning

Mice without implants were fear conditioned in Context A with three pairings of a 30-s tone (85 db, 2.2 kHz) and a 2-s foot shock (0.6 mA). One cohort of mice received predictable conditioning, in which the offset of each tone triggered foot-shock delivery. Another cohort of mice received unpredictable conditioning, in which each foot shock was delivered at a pseudorandom time within the tone (6, 18, or 29 s after tone onset). The following day, mice were placed in Context B and received four tone presentations (85 dB, 30 s) with a 150-s ITI.

After several weeks of surgical recovery and expression of ArchT or GFP, mice with implants were subjected to three trials of auditory fear conditioning, as described for mice without implants. Fiber-optic cables equipped for the delivery of green laser light were attached to the cranial implants before fear conditioning. Light (12 s total) was delivered to mice with implants during three periods (4 s each) within the second and third tones of the fear-conditioning session.

Mice in the unpredictable-conditioning/light-during-negative-prediction-error groups and predictable-conditioning/random-light groups received light starting 5 s into the second tone presentation and 5 s and 17 s into the third tone presentation. Thus, on Trial 2, light was applied during the time at which the foot shock had occurred on Trial 1. On Trial 3, light was applied during the times at which foot shock had occurred on Trials 1 and 2. Mice in the unpredictable-conditioning/random-light group received light starting 24 s into the second tone presentation and at 10 and 24 s into the third tone presentation; these were not times at which foot shock was received on any trial. For all mice with implants, long-term auditory fear memory was assessed 2 days later in a novel context without a cable attached to the implant.

For mice in the ArchT groups, it was expected that application of laser light to CA1 cells infected with ArchT would produce robust photoinhibition of neuronal firing, as was shown in a previous study (Sakaguchi et al., 2015). For mice in the GFP groups, the application of the laser served as a control for light-induced thermal effects in brain tissue.

Histology

The animals were deeply anaesthetized with isoflurane and perfused transcardially with 1× phosphate-buffered saline (PBS; 8,000 mg/L NaCl, 2,160 mg/L Na₂HPO₄, 200 mg/L KCl, 200 mg/L KH₂PO₄, 100 mg/L MgCl_{2 •} 6H₂O, and 100 mg/L CaCl₂) followed by 4% paraformaldehyde. The brains were harvested and postfixed in 4% paraformaldehyde for 24 hr, then cryoprotected by 30% sucrose in 1× PBS. Using a cryostat, the brains were sectioned (20 µm), and slices were mounted with coverslips and Vectashield-4,6-diamidino-2-phenylindole mounting medium (Vector Laboratories, Burlingame, CA). Sections were examined using an LSM 710 confocal scanning microscope equipped with a motorized xy-stage (Zeiss, Thornwood, NY). Image tiles (acquired in a 4 × 6 grid) were scanned using a 20× objective lens under optimal acquisition parameters (frame size = 1,024 × 1,024, pixel size = 0.42 µm, pinhole = 51.1 airy units, gain = 800, digital offset = 0, and digital gain = 1.0). Tiles were reassembled into a single image using Zeiss Zen Black software. Any mice with transduction of the dentate gyrus or CA3 were excluded from all analyses.

Statistics

An initial experiment and one replication were performed with 4 mice without implants per group. Because there were no significant differences between the results of the original and replication experiments (see Table S1 in the Supplemental Material), the two data sets were combined and analyzed as a single data set. In mice that received the implants, a pilot experiment was conducted with 3 to 4 mice per group. Two additional replications were run (3 to 4 mice per group per replication), for a total of three runs. Because there were no significant differences between the results of the original and replication experiments (see Table S1 in the Supplemental Material), the three data sets were combined and analyzed as a single data set. Sample size was determined as described for Experiment 1; we added a replication at the request of the reviewers. Statistical analysis was conducted as described in Experiment 1. The data were analyzed using an ANOVA with a factor of group. Three mice were excluded from all data analyses because an equipment problem omitted the foot shock on the fear-conditioning day. Mice were also excluded from all analyses because of inadequate or inappropriate viral spread (n = 4) or high pretone freezing (> 60%) when placed in the novel context for auditory fear extinction (n = 2).

Results

The mice without implants received auditory fear conditioning with either temporally predictable or unpredictable foot shocks after auditory cue onset (Fig. 6a). As was found for rats in Experiments 1 through 3, the mice in the unpredictable-conditioning group exhibited higher conditioned freezing during auditory fear recall than the mice in the predictable-conditioning group (Fig. 6b), F(1, 11) = 5.28, p = .04.

Fig. 6.

Method and results for Experiment 5. Mice without implants (n = 6–7 per group) were fear conditioned with three pairings of a 30-s tone with a 2-s foot shock delivered with predictable or unpredictable timing, as illustrated in (a). Auditory fear recall was then tested in a novel context. The graph in (b) shows mean percentage of time the mice without implants displayed freezing behavior, separately for predictable- and unpredictable-conditioning groups. The small open circles represent the percentage of time that individual mice displayed freezing behavior. The mice in the ArchT groups received a bilateral infusion of an adenoassociated virus (AAV) expressing the silencing opsin ArchT fused to green fluorescent protein (GFP); the mice in the GFP groups received an AAV expressing GFP. Both viruses targeted cornu ammonis (CA) 1 of dorsal hippocampus. The mice that received brain implants were fear conditioned with three pairings of a 30-s tone with a 2-s foot shock, delivered along with 4-s applications of green laser light, under one of three conditions, as illustrated in (c). Mice in the unpredictable-conditioning/light-during-negative-prediction-error group (n = 10–12) received intrahippocampal delivery of green light (λ = 575 nm) during 4-s periods (the 2 s in which foot shock was actually delivered, and an additional second of light delivery before and after the time of the previous foot shock) surrounding the times at which foot shock had been administered on previous trials. Thus, on Trial 2, light was applied during the time at which the foot shock had occurred on Trial 1. On Trial 3, light was applied during the times at which foot shock had occurred on Trials 1 and 2. Mice in the predictable-conditioning/random-light group received intrahippocampal green light at the same times (relative to conditioned stimulus, or CS, onset) as mice in the unpredictable-conditioning/light-during-negative-prediction-error group, but foot shock had never occurred at these times for the predictable-conditioning/random-light groups. Mice in the predictable-conditioning/random-light group (n = 5–8) received intrahippocampal green light during 4-s periods in which foot shock had never been delivered. Expression of GFP after infection with ArchT is shown for a representative brain section in (d). The white overlay indicates the location of the fiber-optic tip and the estimated light spread. DG = dentate gyrus. Auditory fear-memory recall was measured across two tone presentations during a subsequent laser-free extinction session. The bar graphs show mean percentage of time that the mice displayed freezing behavior as a function of infusion type for (e) the unpredictable-conditioning/light-during-negative-prediction-error group, (f) the predictable-conditioning/random-light group, and (g) the unpredictable-conditioning/random-light group. The small open circles represent the percentage of time that individual mice displayed freezing behavior. Error bars indicate +1 SEM. Asterisks represent significant differences between infusion conditions (*p < .05, **p < .01).

The mice with implants received auditory fear conditioning with either temporally predictable or unpredictable foot shocks after auditory cue onset combined with the transient application of light (Fig. 6c). For these mice, viral infusions were aimed at CA1, and produced robust transduction of cells in this subregion of the hippocampus (Fig. 6d) consistent with results from other studies (Sakaguchi et al., 2015).

As we observed for rats in Experiments 1 through 5 and for mice without implants in this experiment, GFP-infused mice in either unpredictable-conditioning group exhibited higher conditioned freezing during auditory fear recall compared with GFP-infused mice in the predictable-conditioning groups; there was a significant main effect of training type, F(1, 22) = 6.00, p = .02. Photoinhibition of CA1 during putative aversive negative-prediction errors produced a significant decrease in long-term auditory fear-memory recall (Fig. 6e); there was a significant main effect of group, F(1, 20) = 11.21, p = .003, an effect that was not observed when photoinhibition was applied at the same times (relative to auditory cue onset) during temporally predictable fear conditioning (Fig. 6f); there was a significant main effect of group, F(1, 14) = 0.44, p = .52. Thus, these results were virtually identical to what we observed when the dorsal hippocampus was inactivated with muscimol during fear conditioning in Experiment 3 (Fig. 4c). This finding suggests that the effects of the muscimol infusion were probably due to effects specifically within the dorsal hippocampus, rather than from any possible diffusion to nearby structures such as the subiculum. Brief photoactivation of CA1 cells at times when foot shock has never occurred was not sufficient to elevate fear-memory strength in mice that received temporally predictable auditory fear conditioning (see Experiment 5 in Supplemental Results in the Supplemental Material), which suggests that changes in structures outside of the dorsal hippocampus are also required to enhance fear-memory strength during temporally unpredictable fear conditioning.

We ran an additional set of mice with implants that received unpredictable auditory fear conditioning and received photoinhibition at random times within the auditory CS presentation. Thus, ArchT-infused mice in the unpredictable-conditioning/random-light group and ArchT-infused mice in the unpredictable-conditioning/light-during-negative-prediction-error group received an equivalent amount of dorsal hippocampal silencing. There was no difference in the GFP and ArchT groups under these conditions (Fig. 6g); there was no significant main effect of group, F(1, 10) = 1.04, p = .33. The observation that mice in the ArchT group exhibited conditioned freezing levels as high as those of mice in the GFP group suggests that negative prediction errors were present in both groups. This further suggests that, insofar as information about the passage of time since the CS onset is important for generating negative prediction errors, such information must be computed outside the dorsal hippocampus. Because silencing of CA1 reduced fear only when it was applied during aversive negative prediction errors, these data suggest that the dorsal hippocampus enhances fear during temporally unpredictable fear conditioning by generating prediction errors.

General Discussion

These studies advance the knowledge in our field in several ways. We have shown that associative fear-memory strength is influenced by ambiguity surrounding the timing and occurrence of aversive reinforcement (see Supplemental Discussion in the Supplemental Material), and this information is rapidly acquired during fear learning. We have also demonstrated that the dorsal hippocampus plays a critical role in generating error signals (specifically, when a reinforcer is expected but does not occur) and show a causal role for these signals in promoting fear memory. Our results identify an important novel role of the dorsal hippocampus in Pavlovian fear conditioning (see Supplemental Discussion). They also place new emphasis on the relevance of timing to this form of learning and the ability of altered timing to drive prediction errors (see Supplemental Discussion). In addition, we have argued that models of fear learning should embody the features of fear learning found in natural settings because these may lead to qualitatively different fear memories.

At least two distinct processes may engage the dorsal hippocampus during computations of ambiguity in aversive outcomes. The dorsal hippocampus may calculate the passage of time; time cells within the hippocampus are thought to facilitate associations between cues and reinforcers that are discontiguous in time (MacDonald et al., 2011; Modi, Dhawale, & Bhalla, 2014) and maintain memories across delay periods (Gill, Mizumori, & Smith, 2011). Alternatively, the hippocampus may encode the ambiguity of predictive cues on either an ongoing basis (Harrison, Duggins, & Friston, 2006) or a trial-by-trial basis (Vanni-Mercier et al., 2009). Our results are most consistent with the latter idea and further suggest that the hippocampus uses the encoding of ambiguous outcomes to enhance fear memory (see Supplemental Discussion in the Supplemental Material).

Although an emerging body of literature suggests that the hippocampus plays a role in the prediction of future events (Goosens, 2011), it is not known how the brain uses these predictions. We show that the hippocampus is essential for the enhancement of associative fear by ambiguity in the timing or occurrence of aversive outcomes. The role of the hippocampus in prediction per se is not important for determining associative fear-memory strength, given that hippocampal inactivation (Maren & Holt, 2004) and overt hippocampal damage (Kim & Fanselow, 1992) do not affect associative fear memory to discrete cues when aversive outcomes occur at predictable times. Such hippocampal manipulations do not affect predictable auditory fear memory strength, which suggests that the dorsal hippocampus does not play an essential role in positive prediction errors (when reinforcement occurs unexpectedly) or that any such hippocampal prediction errors do not enhance fear-memory strength. In other words, the hippocampus plays an important role in associative fear-memory strength only when (a) cues have an ambiguous relationship with outcomes and (b) outcome-related predictions during learning are sometimes incorrect. Our study is one of the first to identify a specific neural substrate and role for negative aversive prediction errors in fear memory.

There are two primary classes of theoretical models that account for how prediction errors are used to change learning. In one class, positive prediction errors directly enhance associative learning, whereas negative prediction errors weaken associative learning (Rescorla & Wagner, 1972; Sutton & Barto, 1998). In a second class, prediction errors influence the rate of learning by modulating attention to the predictive stimuli: Larger negative or positive prediction errors enhance learning (Pearce & Hall, 1980). We found that hippocampal signaling during times of putative negative prediction errors strengthens learning (Fig. 6), which suggests that the hippocampus contributes to attentional (e.g., Pearce-Hall) enhancement of fear. Our finding that the hippocampus plays a role in the enhancement of fear, both by temporal (Figs. 4 and 6) and by nontemporal (Fig. 5) ambiguity, provides further support for this claim. Indeed, other groups have argued that the hippocampus may play a role in Pearce-Hall-like changes in attention (J. S. Han, Gallagher, & Holland, 1995).

Because the amygdala is an important storage site of long-term fear memories (J. H. Han et al., 2009) and single amygdalar neurons show Pearce-Hall-like changes in associative plasticity across fear-conditioning trials (Roesch, Esber, Li, Daw, & Schoenbaum, 2012), it seems likely that the dorsal hippocampus enhances fear learning by sending an error signal to the amygdala, which then implements a Pearce-Hall algorithm to modify fear learning. Hippocampal error signals may enhance fear-memory strength by increasing excitability or synchrony among neurons in the amygdala during fear learning (Whalen, 2007). In support of this, temporal unpredictability of a sensory stimulus, even in the absence of a reinforcer, is sufficient to increase spontaneous activity in the basolateral amygdala (Herry et al., 2007). The mechanism by which hippocampal error signals affect fear learning in the amygdala is an important topic for future studies.

It may seem surprising that the difference between the predictable- and unpredictable- conditioning groups was determined by the ambiguity of the time of foot-shock delivery (Fig. 1) and not the duration of conditioning parameters known to play an important role in regulating the speed and asymptote of associative learning (see Table S2 in the Supplemental Material). It is important to note that ambiguity surrounding the timing (Figs. 1, 3, 4, and 6) and occurrence (Fig. 5) of aversive reinforcement provides a modulatory influence that enhances fear beyond a baseline level. This baseline level of freezing is shared across the predictable and unpredictable conditions and is probably determined by a combination of fear-conditioning parameters, including the number of pairings of CS and US, intensity of US, and temporal values of specific parameters. Our findings, therefore, do not discount the importance of timing theories of associative learning (Kirkpatrick & Balsam, 2016), and our findings are not inconsistent with these theories.

Together, our data highlight the sensitivity of fear to ambiguity. These results show that ambiguity can produce an unexpected enhancement of fear-memory strength and at the same time either have no effect on (Figs. 1, 3, 4, and 6) or degrade (Fig. 5) the contingency between predictive cues and aversive reinforcement. Although our studies reveal that ambiguity regulates fear in normal rodent populations, ambiguity is likely to exert an even more important influence on fear levels in pathological conditions. Mouse models of anxiety (Tsetsenis, Ma, Lo Iacono, Beck, & Gross, 2007), patients with fear and anxiety disorders (Grillon et al., 2008), and humans who experience unexpected aversive events (Grillon, Baas, Lissek, Smith, & Milstein, 2004) are disproportionately affected by ambiguity during trauma. Thus, strategic interventions aimed at reversing changes in hippocampal circuits in patients with fear and anxiety disorders may help normalize fear learning.