Aversive Prediction Error Signals in the Amygdala

Abstract

Prediction error signals are fundamental to learning. Here, in mice, we show that aversive prediction signals are found in the hemodynamic responses and theta oscillations recorded from the basolateral amygdala. During fear conditioning, amygdala responses evoked by footshock progressively decreased, whereas responses evoked by the auditory cue that predicted footshock concomitantly increased. Unexpected footshock evoked larger amygdala responses than expected footshock. The magnitude of the amygdala response to the footshock predicted behavioral responses the following day. The omission of expected footshock led to a decrease below baseline in the amygdala response suggesting a negative aversive prediction error signal. Thus, in mice, amygdala activity conforms to temporal difference models of aversive learning.

Introduction

Learning that certain stimuli predict negative outcomes is essential to survival. This can be studied experimentally by pairing an initially neutral cue (e.g., a tone) with an aversive unconditioned stimulus (US; e.g., electric shock). With sufficient pairings, the tone becomes a conditioned stimulus (CS+) that predicts the occurrence of the US and elicits emotional responses in the absence of the US. This type of Pavlovian fear conditioning requires the amygdala (Maren et al., 1996), but the mechanisms by which the CS+ acquires the capacity to predict impending aversive events are not known.

In contrast, there is strong evidence that predictions about impending reward are encoded by midbrain dopaminergic (DA) signals. Initially, DA neurons are strongly activated by US delivery, but as the animal learns that the CS+ is a reliable predictor of the US, US evoked firing decreases concomitantly with an increase in CS+ evoked firing. When an expected US is withheld, DA neurons inhibit their firing (Schultz et al., 1997). Thus, DA neurons encode the disparity between expected and actual reward, i.e., the prediction error. Notably, however, DA neurons do not respond in this way when the US is unpleasant and therefore do not encode aversive prediction errors (Fiorillo, 2013).

So where are aversive prediction signals found in the brain? In rats, amygdala neurons reduce their firing-rate to a repeatedly encountered footshock, consistent with a role in predicting aversive events. However, unlike DA cells, the same amygdala neurons do not exhibit a concomitant increase in CS+ (auditory cue) evoked firing nor do they reliably inhibit their firing below baseline when the expected aversive outcome is withheld (Johansen et al., 2010). Thus, the extent to which aversive prediction signals are represented in amygdala single-unit activity is unclear.

Human neuroimaging studies have also failed to establish whether aversive prediction signals are present in the amygdala. Some studies report higher amygdala blood–oxygen level-dependent (BOLD) responses to an unexpected versus an expected or blocked US, consistent with a role in predicting aversive events (Dunsmoor et al., 2008; Eippert et al., 2012; Wood et al., 2012; Metereau and Dreher, 2013). However, several studies explicitly investigating aversive prediction signals do not report significant amygdala involvement at all (Seymour et al., 2004; Menon et al., 2007; Schiller et al., 2008). These discrepant findings may reflect methodological differences. Alternatively, regions outside of the amygdala may generate and process these signals (McNally et al., 2011; Kobayashi, 2012).

To investigate aversive prediction error signals, here we recorded hemodynamic responses and local field potentials (LFPs) from the basolateral amygdala in freely moving mice during discriminative Pavlovian fear conditioning. Hemodynamic responses were recorded in the form of tissue oxygen (T_O2) signals, which are driven by the same physiological mechanisms as the BOLD signal, and therefore provide a close proxy measure (Offenhauser et al., 2005; Logothetis, 2007; Lowry et al., 2010; McHugh et al., 2011, 2013). Positive aversive prediction errors were investigated during fear learning and negative aversive prediction errors were investigated during fear extinction, when the expected footshock was omitted.

Materials and Methods

Subjects

This study used 24 male mice on a CBA × C57bl/6 background that were 5- to 6-months-old at the time of surgery. Mice were maintained on a 12 h light/dark cycle (lights on 7:00 A.M.; off 7 pm) with ad libitum food and water. Mice were housed in groups of two to four before surgery and individually after surgery. The experiments were conducted in accordance with the United Kingdom Animals Scientific Procedures Act (1986) under project licenses 30/2561 and 30/3068 and were approved by local ethical review.

Surgery

Under isoflurane anesthesia, mice were implanted with a carbon paste electrode (CPE; gifted by Lilly UK, Surrey, UK) into the basolateral amygdala (BLA) of one hemisphere to measure tissue oxygen and a silver wire electrode into the BLA of the contralateral hemisphere to measure LFPs (approximately equal numbers received left CPE/right LFP and right CPE/left LFP). CPEs were constructed from Teflon-coated 200- μm-diameter silver wire (∼270 μm coated diameter, Advent Research Materials) with the insulation pulled down the wire to produce a 2 mm cavity which was subsequently packed with carbon paste and smoothed (O'Neill et al., 1982). LFP electrodes were made from 125-μm-diameter silver wire (∼177 μm coated diameter, Advent). Coordinates for BLA implantations were −1.35 mm anterior/posterior, ±3.10 mm medial/lateral and −5.00 mm dorsal/ventral, relative to bregma. Auxiliary and reference electrodes (200-μm-diameter silver wire) were implanted into parietal cortex. A ground electrode was wrapped around a skull screw. Each electrode was soldered to a gold pin (E363/0, Plastics One), which was inserted into a pedestal plug (MS363, Plastics One) and secured with skull screws and dental cement (“Simplex Rapid,” Associated Dental Products). Mice were allowed to recover for at least 7 d after surgery.

Apparatus

T_O2 was measured using constant potential amperometry, as described previously in detail (Bolger et al., 2011; McHugh et al., 2011; Barkus et al., 2014). A constant potential (−650 mV relative to a reference electrode) was applied to the electrode using a low-noise potentiostat (Biostat, ACM Instruments). The applied potential produces the electrochemical reduction of dissolved O₂ on the surface of the electrode, inducing an electrical current, which is measured by the potentiostat. The availability of O₂ for this two-step reaction (O₂ + 2H⁺ + 2e⁻ → H₂O₂; H₂O₂ + 2H⁺ + 2e⁻ → 2H₂O) is determined by the local T_O2 concentration. Thus, changes in O₂ concentration around the tip of the electrode produce directly proportional changes in the measured Faradaic current (Hitchman, 1978). The spatial resolution is estimated to be a sphere with diameter approximately twice the diameter or the electrode, i.e., 400 μm (Thompson et al., 2003; Li et al., 2011b).

Mice were connected to the potentiostat via a six-channel rotating commutator (SL6C, Plastics One) held on a counter-weighted arm (PHM-110P1, Med Associates) using screened cables (363-363 6TCM, Plastics One). A Powerlab 8/30 interface (AD Instruments) was used for analog/digital conversion and data were collected on a Windows PC running Chart v5 software (AD Instruments). LFPs were recorded using a differential amplifier (DP-301, Warner Instruments) or using the potentiostat with no potential applied, in which case the potentiostat acts as a differential amplifier. T_O2 and LFPs were sampled continuously at 4 kHz.

Fear conditioning was conducted in one of three operant chambers (ENV-307A, Med Associates), each with distinct visual and olfactory cues (Fig. 1A). Stimulus delivery was controlled by a custom-written script in the MED-PC language. Timed transistor–transistor logic pulses to the AD converter ensured that stimulus delivery was synchronized with the electrophysiological recordings at 1 ms resolution.

Figure 1.

Experimental design and histology. A, Diagram of the recording setup. B, Reconstructions of electrode placements in the basolateral amygdala. Filled circles/squares = T_O2 electrodes, open circles/squares = LFP electrodes. Circles denote mice that underwent extinction; squares denote mice that underwent the probe session containing expected and unexpected shocks. C, Schematic of the experimental sessions. Twenty-four mice underwent fear conditioning, eight of these also underwent the probe session. The remaining 16 mice underwent one extinction session, with 10 of these doing two sessions. D, Fear conditioning sessions consisted of a 10 min pre-session period, followed by a 23 min session with 10 auditory cues (5 × tone, 5 × white noise), presented in pseudorandom order. On training days, one of the cues (CS+) was always paired with coterminating footshock; the other cue (CS−) was never paired with footshock. No shocks were given during the pre-exposure or extinction sessions. During the probe session three CS+ (with shock), three CS− (no shock), and three unsignaled shocks were given. E, Left, Example of raw T_O2 data during a CS+ presentation (CS onset at 0 s, offset at 30 s). E, Right, T_O2 data transformed into ΔT_O2 signal by subtracting the baseline (mean T_O2 signal in the 5 s before CS onset) and binning the raw T_O2 signal into fifteen 2 s epochs. Arrow shows the peak T_O2 signal. F, Left, Example power spectral density (PSD) for an LFP signal in the pre-CS+ period (blue trace), and the CS+ period (red trace). F, Right, Proportional PSD calculated by expressing the power in each frequency bin as a proportion of the total power between 1 and 45 Hz. Arrow shows peak theta power.

Procedure

Twenty-four mice underwent fear conditioning. The procedure on each day was virtually identical. First, mice were connected to the recording equipment and placed in the “neutral context,” i.e., a chamber in which the mouse had been previously habituated for at least 2 h and in which they never received shocks. The potential was then applied to the T_O2 electrode for 10 min before the experiment began to ensure that T_O2 signals were stable. Pre-exposure (day 1) was performed entirely in the neutral context: five tone (2900 Hz) and five white noise stimuli (both 30 s duration, 80 dB) were played in pseudorandom order with a mean ITI of 80 s (range 60–100 s), with no shocks administered. There were no differences in behavioral, T_O2, or LFP responses to the auditory cues before training and these results are not reported further. On training days 1–3 (experimental days 2–4) mice were placed into the neutral context for 10 min and then transferred to one of the conditioning chambers (e.g., context A or B, counterbalanced). Mice then received five tone and five white-noise stimuli presentations, the same as pre-exposure, but one of the stimuli (counterbalanced across mice) was always paired with coterminating footshock (0.3 mA, 0.5 s).

After fear conditioning, in one subset of mice (n = 8) we investigated amygdala responses to expected versus unexpected footshock (day 5). Mice were placed into a novel conditioning chamber (e.g., context B if trained in context A) and underwent a probe session containing expected shocks (i.e., CS+ followed by footshock), unsignaled shocks (i.e., footshock occurring in the absence of any auditory cue), and presentations of the CS− without footshock. Three of each trial type were used and presentation order was pseudorandomized and counterbalanced across mice. Behavioral data and LFP recordings were obtained from eight mice and T_O2 data were obtained from seven of these mice.

A separate subset of mice underwent either one (n = 16) or two (n = 10) fear extinction sessions (days 5 and 6). On each session, mice were placed first into the neutral context for 10 min and then placed into a novel conditioning chamber (e.g., context B if trained in context A) and the five tone and five white noise stimuli were presented with no shocks administered (Figs. 1C,D).

Histology

Mice were injected with sodium pentobarbitone; 200 mg/kg) and perfused transcardially with physiological saline (0.9% NaCl) followed by 10% formol saline (10% formalin in 0.9% NaCl). Their brains were removed and placed in 10% formol saline for 3 d, and then transferred to a 30% sucrose–formalin solution for 24 h and frozen. Coronal sections (40 μm) were then cut on a freezing microtome and stained with cresyl violet to enable visualization of the electrodes. Only mice with electrodes in the basolateral amygdala were included in the analyses (Fig. 1B).

Data analysis

T_O2 responses.

Auditory cue-evoked T_O2 responses were calculated by subtracting the mean T_O2 signal during the 5 s before cue onset (i.e., baseline) from the T_O2 signal during the 30 s cue presentation. The magnitude of US evoked responses was also calculated by subtracting the mean T_O2 signal during the 5 s before shock onset, which in all cases except unsignaled shock would be the last 5 s of CS presentation. Then, the 30 s signal was divided into fifteen 2 s time bins (i.e., 0–2 s, 2–4 s, 4–6 s…28–30 s), with each data point equal to the mean value during each 2 s time bin (Fig. 1E). For analysis, we averaged T_O2 responses over the five (or three in the case of the probe day) CS+, CS−, US, or US omission periods of each session and determined the time bin with the highest value (max T_O2; Fig. 1E).

On the extinction days, we were interested in whether the T_O2 signal decreased when the expected footshock was omitted and so we did not use the maximum value for analysis. Instead we analyzed how the signal changed in the 30 s following CS offset by dividing the 30 s T_O2 signal into fifteen 2 s time bins and directly comparing responses following CS+ versus CS− offset (ANOVA model: day_{2(extinction day 1, extinction day 2)} × stimulus type_{2(CS+, CS−)} × timebin₁₅, n = 10 mice). A decrease below baseline could occur simply as a consequence of the offset of the auditory cue but if this was the case, then this decrease should be equivalent following CS+ and CS− presentations. Note that the baselines for shock omission (and the equivalent period following CS− presentations) were based on the mean signal during the last 5 s of CS+ or CS− presentation and so values below zero during the shock omission periods indicate signals lower than those at the end of CS presentation.

LFPs.

Stimulus-evoked LFPs were bandpass filtered between 1 and 45 Hz. We calculated power spectra during the 10 s after CS onset, US onset, or US omission (i.e., the 10 s following CS offset). Note that the 0.5 s period during shock administration was not contained in these epochs. Spectra were averaged over the five CS+, five CS−, five US, or five US omission periods for each mouse for each day of training and extinction. Spectra were computed in MATLAB (Mathworks) using a fast Fourier Transform size of 2000 samples (with a Hamming window, 50% overlap) at a sampling rate of 1 kHz, giving a frequency resolution of ∼0.5 Hz. For statistical analysis, the power spectral density in each frequency bin (Φ_i,) was transformed into a proportion of the total power between 1 and 45 Hz:

where P_i = proportional power, Φ_i = raw power density (mV² /Hz).

From these proportional spectra, we determined the peak power in the theta frequency range (5–10 Hz) as shown in Figure 1F. Spectrograms (time × frequency × power) were generated in MATLAB, using a sliding time window of 1 s, with 500 ms overlap, a frequency resolution of ∼0.25 Hz, and the “jet” color map (blue, lower-power; red, higher-power). To analyze US omission, we used proportional spectra (as described above) and normalized spectrograms, which were obtained by z-transforming the frequency dimension of each spectrogram as follows:

where Z_ij is the power at the i^th frequency and j^th time bin expressed in SD units, X_ij is the raw power at the i^th frequency and j^th time bin, X̄_i is the mean power at the i^th frequency across all time bins, and SD(X_i) is the SD at the i^th frequency across all time bins. For analysis, we calculated the mean z-transformed power in the 10 s before CS+ onset and compared this to the mean z-transformed power in the 10 s period after CS+ offset (i.e., in the 10 s after an expected footshock was not delivered). Because of noise in some LFP recordings on extinction day 2, we restricted LFP analysis of US omission to the first extinction day. In addition, on the probe day in Experiment 2, we excluded the first expected shock and the first unexpected shock trials due to noise issues in the LFP recordings.

Behavior: locomotor and freezing data.

Locomotor activity was monitored via a video camera in the roof of the operant chamber and measured using a script in NIH Image (Schneider et al., 2012), which compared consecutive video frames for pixel changes (1 Hz sampling rate, therefore 1 s resolution). Freezing was determined on the frame-to-frame percentage pixel change being below a set threshold, calibrated for an absence of movement except for breathing (Richmond et al., 1998). If the percentage pixel change was above threshold then a score of 0 was given, if it was below threshold then a score of 1 was given. These scores were converted into percentages by summing the values during CS presentation, dividing by the CS duration, and multiplying by 100. To reduce the impact of freezing evoked by the training context, a “difference score” was calculated such that percentage freezing during the 30 s before cue presentation was subtracted from the percentage freezing during the 30 s cue presentation (i.e., positive freezing scores indicate increased freezing to the cue and negative freezing scores indicate decreased freezing to the cue relative to the pre-cue period). For the regression analyses, we used a freezing discrimination score based on the absolute difference between CS+ evoked freezing minus CS− evoked freezing. For example, if a mouse had a percentage pixel change below threshold in 18 of the 1 s time bins during a 30 s CS+ presentation, this would yield a freezing score of 60% [i.e., (18/30) × 100 = 60]. If the equivalent freezing during CS− presentation was 40%, then the freezing discrimination score would be: 60–40% = 20%.

Statistical procedures.

Data were analyzed using t tests, ANOVA, or multiple linear regression in SPSS. ANOVAs are described using a modified version of Keppel's notation (Keppel, 1982) in which the dependent variable is defined in the form: A₂ × B₃, where A is a factor with two levels and B is a factor with three levels. All graphs show the mean ± 1 SEM. The familywise error was set at α = 0.05.

Results

Behavioral responses during discriminative fear learning

Over the 3 d of training, the mice learned to discriminate between the auditory cue that predicted footshock (CS+) and the other auditory cue that did not predict footshock (CS−), such that freezing levels were markedly higher during CS+ than CS− presentations (Fig. 2). During training, mice froze less during CS− presentations compared with the 30 s before CS− onset (hence negative scores in Fig. 2A) and froze more during CS+ presentations compared with the 30 s before CS+ onset (hence positive scores). Analysis of CS− versus CS+ evoked freezing responses during training (ANOVA: CS type_{2(CS−, CS+)} × day₃, n = 24 mice) confirmed that mice froze more during CS+ presentations (CS type × day interaction: F_(2,46) = 4.8, p = 0.01, CS+ vs CS− on day 3, p = 0.03). In short, the mice learned that the CS+, and not the CS−, predicted the aversive outcome.

Figure 2.

Aversive prediction error signals in amygdala T_O2 signals during fear learning. A, Freezing responses during aversive conditioned (CS+) or nonaversive conditioned (CS−) cues for training days 1–3 (T1–T3). Freezing is plotted as a difference score (freezing during the 30 s of cue presentation minus freezing in the 30 s before cue presentation), with negative scores indicating less freezing during the cue compared with the 30 s before the cue. B, Amygdala T_O2 responses during the 30 s CS+ period and the 30 s post-US and the CS− and the equivalent 30 s no US period for training days 1–3. C, Peak T_O2 responses during CS+ and US periods, and CS− and no US periods, during training days 1–3. A–C, Mean ± SEM (n = 24 mice); *p < 0.05.

Aversive prediction errors in amygdala hemodynamic responses

Amygdala T_O2 responses changed markedly over the 3 d of training such that responses to the CS+ increased, responses to the footshock (US) decreased, whereas responses to the CS− changed little (Fig. 2B,C). We analyzed these data using ANOVA with within-subjects factors of CS type (CS+, CS−), CS phase (cue phase, shock phase), and day (days 1, 2, 3). Crucially, there was a three-way interaction between these factors (F_(2,46) = 4.7, p = 0.01), allowing us to perform further analyses to investigate the changes in T_O2 responses over training.

The first thing to note is that T_O2 responses evoked by the CS+ were significantly higher than those evoked by the CS− and T_O2 responses following footshock were higher than those in the “no US” period following the CS− (CS type × CS phase interaction: F_(1,23) = 4.6; p = 0.04, CS+ > CS−, p = 0.04; US > no US, p < 0.001), consistent with the behavioral data showing higher freezing levels during CS+ trials.

Second, and most importantly, amygdala T_O2 responses evoked by the CS+ and the US changed systematically over training. On training day 1, responses evoked by the US were higher than those evoked by the CS+ but by training day 3, the US evoked response had significantly decreased, whereas the CS+ evoked response had significantly increased (Fig. 2B,C). Analysis (ANOVA: stimulus type_{2(CS+, US)} × day₃, n = 24 mice) revealed an interaction between stimulus type and day (F_(2,46) = 6.4, p = 0.003). US evoked responses were significantly higher during day 1 than day 3 (p = 0.008), whereas CS+ evoked T_O2 responses were significantly higher during day 3 than day 1 (p = 0.005). These patterns suggest that amygdala hemodynamic responses contain aversive prediction error signals.

Moreover, the systematic changes in CS+ and US evoked T_O2 responses were not seen for CS− evoked responses and the equivalent no US period during the 30 s after CS− offset (no stimulus type × day interaction: F_(2,46) = 0.4, p = 0.7; Fig. 2B,C). Although CS− evoked signals were higher than those during the no US period, this did not reach significance (F_(1,23) = 4.0; p = 0.06). Importantly, T_O2 signals in the CS− and no US period did not change across days (all pairwise comparisons p > 0.5), arguing that the prediction error-like changes were specific for CS+ and US evoked amygdala T_O2 responses.

Aversive prediction error signals in theta oscillations

The onset of the CS+ and the US evoked theta frequency oscillations (5–10 Hz), and theta power changed in a similar way to T_O2 responses over training (Fig. 3). As with T_O2 signals, analyses of theta oscillations revealed a three-way interaction between CS type, CS phase, and day (F_(2,46) = 13.1, p < 0.001). Three key findings emerged.

Figure 3.

Aversive prediction error signals in amygdala-recorded theta oscillations during fear learning. A, Spectrogram from a representative mouse on training day 1 for the pre-CS, CS+, and US periods. US (footshock) onset at 29.5 s (arrow), offset at 30 s. B, Same as in A but showing training day 3. Note the shift to theta-dominant signals during the CS+ and delta-dominant signals following the US on T3 compared with T1. C, Peak theta (θ) power during CS+ and US periods for T1–T3. Note the increase in CS+ evoked and the decrease in US evoked theta power. D, Spectrogram from a representative mouse on training day 1 for the pre-CS, CS−, and no US periods. E, Same as in D but showing training day 3. There was slightly higher delta (1–4 Hz) power on T3 but no shift to theta-dominant signals on either T1 or T3. F, Peak theta (θ) power during CS− and no US periods for T1–T3. There was a decrease in theta evoked by the CS− but no change during the no US period. C, F, Mean ± SEM (n = 24 mice); *p < 0.05.

First, theta oscillations evoked by the CS+ were higher than those evoked by the CS− (CS type × CS phase interaction: F_(1,23) = 11.5, p = 0.003; CS+ vs CS−, p = 0.02), consistent with both the behavioral and T_O2 data.

Second, like the amygdala T_O2 responses, US onset evoked an increase in theta power on training day 1 which diminished over the 3 d of training. Conversely, theta oscillations evoked by the CS+ increased in power over this period (Fig. 3A–C). Analysis (ANOVA: stimulus type_{2(CS+, US)} × day₃, n = 24 mice) confirmed these observations (stimulus type × day interaction, F_(2,46) = 10.9, p < 0.001). CS+ evoked theta power was higher on day 3 compared with day 1 (p = 0.03), whereas US evoked theta power was higher on day 1 compared with day 3 (p = 0.045). Thus, positive aversive prediction errors are represented via a decrease in US evoked theta oscillations and a concomitant increase in CS+ evoked theta oscillations as the mouse learns that the CS+ predicts the US.

Third, analysis of CS− evoked responses and the no US epoch after CS− offset (i.e., the equivalent period to that following US delivery on CS+ trials) also yielded a stimulus type × day interaction (F_(2,46) = 3.3, p = 0.047). Theta power was higher during CS− presentations on day 1 compared with day 2 (p = 0.04) and day 3 (p = 0.02), and therefore showed the opposite pattern to CS+ evoked theta oscillations (Fig. 3D–F). Moreover, theta oscillations during the no US period did not show any change over days (all p > 0.5). In short, unlike the CS− and no US evoked signals, theta oscillations changed systematically to the CS+ and the US during training in a manner congruent with an aversive prediction error signal.

The changes we observed in amygdala T_O2 responses and theta oscillations could not be explained by changes in locomotor activity (Fig. 4A,B). Whereas CS+ evoked T_O2 responses and theta oscillations increased over training, locomotor activity during CS+ presentations decreased over training (day 1 vs day 3: p = 0.03). Moreover, whereas US evoked T_O2 responses and theta oscillations significantly decreased over training, the burst in locomotor activity evoked by the US (i.e., the unconditioned response to the shock) remained constant over training (day 1 vs day 3: p = 0.6; Fig. 4B).

Figure 4.

Locomotor activity during aversive CS+ and US presentations could not explain the patterns of amygdala T_O2 responses or theta oscillations. A, Locomotor activity during the 30 s CS+ presentation and the 30 s post-US period (black trace) for T1–T3. Movement during CS− and the equivalent no US period is shown for comparison (gray trace). The burst in activity at ∼30 s in the black trace is in response to the footshock US. B, The magnitude of the CS+ evoked locomotor response decreased over training (*p < 0.05), whereas T_O2 responses and theta oscillations significantly increased over training. Also, US evoked locomotor responses stayed constant over training, whereas T_O2 responses and theta oscillations significantly decreased over training (Fig. 2). C, Locomotor activity during CS+ and CS− trials and the post-CS period during extinction trials. There was a slight increase in movement following US omission on CS+ trials (black trace from 30 to 40 s), whereas T_O2 signals and theta oscillations decreased during this period. A–C, Mean ± SEM. A, B, n = 24 mice; C, n = 10 mice).

Responses to expected versus unexpected shock

Fear learning resulted in decreased T_O2 responses and theta oscillations to the footshock over training, consistent with an aversive prediction error signal. However, this decreased response could reflect habituation to the shock, rather than a prediction error per se. To test this, a subset of the fear-conditioned mice experienced a probe session in which they were given both expected shocks (i.e., shocks signaled by CS+ presentations) and unexpected shocks (i.e., shocks unsignaled by any auditory cue). If these signals contained aversive prediction errors, then responses to unexpected shock should be greater than those to expected shock.

Unexpected shock evoked significantly higher amygdala T_O2 responses than expected shock (t₍₆₎ = 3.8, p = 0.01; Fig. 5A,B). Moreover, unexpected shock was associated with increased theta oscillations (8–12 Hz) compared with expected shock (t₍₇₎ = 2.4, p = 0.05; Fig. 5C). This difference in amygdala responses was not due to locomotor activity differences because expected shock and unexpected shock evoked almost identical locomotor responses (mean movement during 30 s postshock: t₍₇₎ = 0.2, p = 0.8; maximum movement during 30 s postshock: t₍₇₎ = 0.3, p = 0.8; Figure 5D). In short, the decrease in US evoked amygdala responses with training reflect expectation of the shock and do not simply reflect habituation to the shock.

Figure 5.

A, T_O2 signals were higher to an unexpected compared with an expected footshock (i.e., one that was preceded by the CS+). Arrow indicates onset of footshock. B, Peak T_O2 data responses to expected and unexpected shock. Although the error bars overlap, the difference (unexpected minus expected) was significantly greater than zero (gray bar). C, Theta power (8–12 Hz) was higher for unexpected than expected shock, again with the difference shown in the gray bar. D, Locomotor responses did not differ between the expected and unexpected shocks. A–D, Mean ± SEM. A, B, n = 7 mice; C, D, n = 8 mice; *p < 0.05.

Responses during fear extinction learning

In the remaining fear conditioned mice, we investigated what happened when the expected footshock was omitted, to see whether the amygdala signals represent negative aversive predictions errors. Over the two extinction sessions, behavioral discrimination between the CS+ and CS− remained relatively constant (Fig. 6A). Analysis (ANOVA: stimulus type_{2(CS−, CS+)} × trial₁₀, n = 10 mice) confirmed there was significantly higher freezing during CS+ than CS− presentations (main effect of stimulus type: F_(1,9) = 7.0, p = 0.03), but there was no evidence of extinction learning over the 10 unreinforced CS+ trials (no effect of trial, or stimulus type × trial interaction: F < 1.6, p > 0.1).

Figure 6.

Negative aversive prediction signals in the amygdala during fear extinction. A, Freezing responses during CS+ or CS− cues for extinction days 1 and 2 (Ext1, Ext2). B, C, T_O2 signals evoked by the absence of an expected footshock (US omission) were significantly lower than pre-US omission baseline signals and the pre-CS+ baseline. This was not the case for the equivalent CS− period. D, Power spectra illustrating lower theta power in the 10 s following US omission (USo, black line) versus the 10 s before US omission (pre-USo, gray line). E, Z-transformed, group-averaged spectrogram illustrating increased theta power in a narrow frequency range (∼6 Hz) during CS+ presentation and the marked decrease of activity in this range following US omission. F, As in E, but for the equivalent CS− period. G, Theta activity at 6 Hz (expressed in SD units) was significantly lower following US omission compared with the pre-CS+ period. A–D, G, Mean ± SEM. A–C, n = 10 mice, D–G, n = 16 mice; *p < 0.05.

Negative aversive prediction errors in amygdala hemodynamic responses

During extinction, the omission of expected footshock led to a negative T_O2 response, i.e., a decrease below baseline (Fig. 6B,C). When the expected US was not delivered at the end of CS+ presentations, T_O2 signals decreased to a level that was significantly lower than the equivalent period following CS− offset (CS type × time bin interaction, F_(14,126) = 2.0; p = 0.03; CS+ < CS−, 16–20 s after expected US was omitted; note that the baselines for this analysis were based on the last 5 s of CS+ or CS− presentation and so US omission resulted in a T_O2 signal decrease below levels at the end of CS+ presentation). Importantly, the decrease in T_O2 signals following US omission was also below the pre-CS+ baseline (t₍₉₎ = 2.7, p = 0.03; Fig. 6C). Thus, negative aversive prediction errors are also present in amygdala hemodynamic responses.

Omission of expected footshock was also associated with reduced theta power compared with the CS+ period (and the pre-CS+ period), and this reduction was focused within the same narrow frequency band that was elevated during CS+ presentation (Fig. 6D–G). Analysis of power spectra (ANOVA: phase_{(before omission, after omission)2} × freq₁₉, n = 16 mice) revealed a significant reduction in theta power in the 10 s after US omission compared with the 10 s before US omission (phase × frequency interaction, F_(18,270) = 2.3, p = 0.003, 6–6.5 Hz, p < 0.05; Fig. 6D). Moreover, when analyzing the z-transformed power spectra (see Materials and Methods), theta power after US omission was lower than theta power in the period before CS+ onset (based on theta power at 6 Hz: t₍₁₄₎ = 2.5, p = 0.03; Fig. 6E–G; note this was also the case for the averaged power between 6 and 7 Hz: t₍₁₄₎ = 2.3, p = 0.04). The decreased amygdala signals during US omission could not be explained by locomotor activity changes because there was increased movement following US omission (Fig. 4C). Thus, negative aversive prediction errors are associated with significantly reduced theta power below baseline.

Amygdala responses to footshock predict subsequent fear-related behavior

Finally, we used multiple linear regression to see whether T_O2 signals on day n could predict behavioral discrimination between the aversive (CS+) and nonaversive (CS−) conditioned cues on day n + 1. We calculated a simple behavioral discrimination score for each mouse based on the absolute difference in percentage freezing during CS+ and CS− presentations (i.e., CS+ freezing minus CS− freezing, see Materials and Methods). Three independent variables were used in the regression model: the maximum T_O2 signal during CS+ presentations (T_O2_CS+), the maximum T_O2 signal during CS− presentations (T_O2_CS−), and the maximum T_O2 signal in the 30 s following footshock delivery (T_O2_US). In the regression model, T_O2_CS+, T_O2_CS− and T_O2_US on training days 1, 2, and 3 were used to predict behavioral discrimination on training days 2 and 3 and the extinction session, respectively. Stepwise linear regression was used to determine the maximum variance explained with the fewest independent variables.

The optimum regression model contained only T_O2_US and accounted for 18% of the variance in behavioral discrimination, which was significantly greater than zero (F_(1,46) = 11.5, p = 0.001, adjusted R² = 0.18). The correlation between T_O2_US and behavioral discrimination was positive, with the standardized β coefficient = 0.45 (t₍₄₆₎ = 3.4, p = 0.001). In other words, mice who exhibited a larger T_O2 response to the footshock showed better behavioral discrimination the following day.

We repeated this regression analysis using the maximum theta power during CS+, CS−, and US presentations (θ_CS+, θ_CS−, θ_US, respectively). As above, the optimum model contained only θ_US, accounting for 7% of the variance which was significant (F_(1,46) = 4.4, p = 0.04, adjusted R² = 0.07). The correlation between θ_US and behavioral discrimination was positive, with the standardized β coefficient = 0.3 (t₍₄₆₎ = 2.1, p = 0.04). In short, both theta oscillations and T_O2 responses to the footshock predicted subsequent behavioral discrimination. Unfortunately, because we observed little in the way of extinction, we could not use this regression approach to test whether the negative T_O2 and theta responses following US omission predicted the amount of extinction learning.

Discussion

Summary of results

Here we show that hemodynamic responses and theta oscillations recorded from the amygdala show activity patterns consistent with aversive prediction error signals. The significance of these findings is threefold. First, prediction errors are necessary for learning and the amygdala is necessary for fear learning; and yet the evidence for amygdala aversive prediction error signals has been equivocal. Our data demonstrate for the first time that aversive prediction error signals, analogous to appetitive prediction error signals found in DA neurons, can been seen in the amygdala, even though these may not be detectable at the single-unit level (Johansen et al., 2010). Second, this is the first study to track theta oscillations evoked by both the CS+ and the US over the course of fear learning. Our data show that the changes in CS+ and US evoked theta oscillations are systematic and proceed in opposite directions as the mouse learns that the CS+ predicts the US. Furthermore, omission of an expected US led to reduced theta oscillations. These patterns of theta oscillations provide a novel neural correlate for fear learning. Third, we found that mice who exhibit strong T_O2 and theta responses to the footshock (US) show better behavioral discrimination between the aversive (CS+) and the nonaversive (CS−) conditioned cues the following day. Thus, the magnitude of US evoked responses may reflect the processing or attention directed to the US, strengthening the association between the predicting cue and the aversive outcome.

Linking rodent T_O2 signals with human neuroimaging

This is the first study to show that amygdala T_O2 responses in mice represent aversive prediction errors. T_O2 and BOLD signals are closely related, for example, both are hemodynamic responses dependent upon increased cerebral blood flow (Ogawa et al., 1990; Lowry et al., 1997), allowing for cross-species comparisons that are not usually possible. Consistent with our data, several human neuroimaging studies have reported reduced US evoked BOLD responses over training and greater BOLD responses to an unexpected or partially predicted US versus an expected, blocked, or fully predicted US (Dunsmoor et al., 2008; Eippert et al., 2012; Wood et al., 2012; Metereau and Dreher, 2013).

However, other neuroimaging studies investigating aversive prediction errors do not report significant amygdala involvement (Seymour et al., 2004; Menon et al., 2007; Schiller et al., 2008). These failures could reflect methodological issues. First, it is unclear whether the scanning parameters were optimized to measure amygdala responses. The amygdala exhibits small BOLD responses (Norbury et al., 2010) and is vulnerable to susceptibility artifacts (Morawetz et al., 2008). Moreover, intersubject variability in human amygdala volume makes normalization to a standard atlas problematic (Robinson et al., 2004). Consequently, amygdala activations may be missed in whole-brain analyses. Second, the typical CS+ duration (i.e., the interval between CS+ onset and US onset) is <10 s, whereas the hemodynamic response evolves over >10 s. Therefore, it is difficult to disambiguate responses due to CS+ versus US onset on reinforced trials. Consequently, many studies have used partial reinforcement between CS and US and only analyzed responses from nonreinforced CS+ trials. However, partial reinforcement limits the attenuation of the US response and reduces CS+ evoked responses over training (Dunsmoor et al., 2007, 2008). To avoid these issues, in the present study we recorded hemodynamic responses directly from the amygdala, used a CS duration of 30 s, and used 100% reinforcement between the CS+ and US during training.

One recent study has argued that amygdala BOLD signals encode the associability between the CS and US rather than the prediction error (Li et al., 2011a). Our T_O2 data are not consistent with this interpretation. Specifically, their model incorporates part of Pearce–Hall theory, which states that associability will decrease over trials as the organism learns that the CS+ is a good predictor of the US (Pearce and Hall, 1980), and therefore predicts a decrease in CS+ evoked amygdala signals over training, contrary to what we observed. This could reflect a species difference but there are also important methodological differences. First, Li et al. (2011a) used angry faces as the conditioned stimuli, which are known to evoke unconditioned amygdala responses even before being paired with an aversive outcome (i.e., pairing angry faces with shock may represent a US–US association, rather than CS–US association). Second, Li et al. (2011a) used partial reinforcement (33%) and a short CS duration of 4 s, which complicates the interpretation of US and US omission evoked responses, as discussed above. Third, Li et al. (2011a) did not analyze amygdala responses to the US. These methodological differences limit comparison with our study.

Seymour et al. (2005) report that amygdala BOLD signals are consistent with appetitive but not aversive prediction errors. In their paradigm, participants experienced persistent pain and visual cues predicted either the exacerbation (aversive signal) or relief (appetitive signal) of pain. Their failure to see aversive prediction errors could be due to the participants experiencing persistent pain rather than distinct aversive versus nonaversive epochs.

Theta oscillations and the amygdala

Several studies have investigated cue (CS+) evoked theta oscillations during fear learning (Seidenbecher et al., 2003; Narayanan et al., 2007) but we are the first to show systematic (and opposite) changes in CS+ and US evoked theta during fear learning. However, although our study recorded oscillations from electrodes in the basolateral amygdala, theta may be generated elsewhere. Most models of neuronal oscillation assume that there must be spatial separation of current sinks and sources (to produce a dipole), and that these must be in coherently aligned neurons for oscillations to be observed in the LFP. As a nonlaminar structure, it is unclear whether the cellular architecture of the basolateral amygdala allows for the generation of theta oscillations. Nevertheless, intracellular theta oscillations can be induced in amygdala neurons by near-threshold membrane depolarization (Pare et al., 1995; Pape et al., 1998) and lateral amygdala single-units are modulated by theta oscillations during discriminative fear conditioning (Pare and Collins, 2000). Thus, amygdala neurons are influenced by theta, even if it is not generated locally.

What is the function of theta oscillations?

There are several theories about the function of theta, and these are not mutually exclusive. First, theta may synchronize activity across different structures to facilitate reciprocal information transfer (Buzsáki, 2002; Seidenbecher et al., 2003; Sirota et al., 2008). Second, theta oscillations influence spike timing and so can influence synaptic plasticity (Song et al., 2000). Third, theta may set the “gain” (attention/arousal) for stimuli, boosting the signal without influencing its content, which could also facilitate learning (Sejnowski and Paulsen, 2006).

Relationship between amygdala signals and freezing behavior

How do CS and US evoked signals relate to the behavioral correlates of learning? We found that mice with larger US evoked T_O2 and theta responses exhibited better behavioral discrimination between the aversive (CS+) and nonaversive (CS−) conditioned cues the following day. This result was surprising because, at a group level, US evoked responses diminished over the course of learning whereas behavioral discrimination improved, suggesting that lower US evoked signals might indicate greater US expectancy and hence greater likelihood of CS+/CS− discrimination. However, our results suggests that, at the level of individual mice, higher US evoked responses may reflect greater attention to the US, which in turn may facilitate the association between the CS+ and the US. One important caveat in our findings is that we could not relate the amygdala negative prediction error signals to behavior because we did not see any significant loss of fear across the two extinction sessions. Future studies using prolonged extinction may be required to investigate this further.

What is the origin of the aversive prediction signal?

Why do we see aversive prediction signals if they are not present in amygdala single-unit activity (Belova et al., 2007; Johansen et al., 2010)? First, hemodynamic responses and LFPs sample from a larger spatial area and over a longer temporal window, pooling activity from many neurons. Second, hemodynamic responses and LFPs predominantly reflect the input to a region whereas unit activity reflects output (Logothetis et al., 2001). Thus, amygdala T_O2 responses and LFPs may reflect aversive prediction signals generated elsewhere. There is good evidence implicating the periaqueductal gray in the generation of these signals, although it does not project directly to the amygdala (McNally and Cole, 2006; Johansen et al., 2010). Another candidate structure for generating aversive prediction error signals is the habenula. Cells in the primate lateral habenula increase their firing to a CS that predicts aversive events and fire more to unpredictable than predictable aversive events (Matsumoto and Hikosaka, 2009). However, lesions of the lateral habenula do not prevent fear learning (Heldt and Ressler, 2006) and the habenula does not project directly to the amygdala (Kim, 2009). An alternative theory is that serotonergic neurons might provide aversive prediction signals, although evidence for this theory is currently limited (Daw et al., 2002).

Conclusion

The amygdala is necessary for both the acquisition and extinction of conditioned fear. By demonstrating that positive and negative aversive prediction errors are represented in the amygdala, we provide new insights into the mechanisms of fear learning.