Effects of amygdala lesions on overexpectation phenomena in food cup approach and autoshaping procedures

Abstract

Prediction error (PE) plays a critical role in most modern theories of associative learning, by determining the effectiveness of conditioned or unconditioned stimuli (CS or USs). Here, we examined the effects of lesions of central (CeA) or basolateral (BLA) amygdala on performance in overexpectation tasks. In two experiments, after two CSs were separately paired with the US, they were combined and followed by the same US. In a subsequent test, we observed losses in strength of both CSs, as expected if the negative PE generated on reinforced compound trials encouraged inhibitory learning. CeA lesions, known to interfere with PE-induced enhancements in CS effectiveness, reduced those losses, suggesting that normally the negative PE also enhances cue associability in this task. BLA lesions had no effect. When a novel cue accompanied the reinforced compound, it acquired net conditioned inhibition, despite its consistent pairings with the US, consonant with US effectiveness models. That acquisition was unaffected by either CeA or BLA lesions, suggesting different rules for assignment of credit of changes in cue strength and cue associability. Finally, we examined a puzzling autoshaping phenomenon previously attributed to overexpectation effects. When a previously-food-paired auditory cue was combined with the insertion of a lever and paired with the same food US, the auditory cue not only failed to block conditioning to the lever, but also lost strength, as in an overexpectation experiment. This effect was abolished by BLA lesions but unaffected by CeA lesions, suggesting it was unrelated to other overexpectation effects.

Modern theories of associative learning emphasize a critical role for prediction error (PE, the difference between received and expected events). Most widely recognized (e.g., Rescorla & Wagner, 1972) is the idea that PE determines the effectiveness of a reinforcer or unconditioned stimulus (US): surprising reinforcers are more effective than expected ones. Decades of behavioral research have provided wide-ranging support for this view (e.g., Miller, Barnet, & Grahame, 1995). However, according to many models of associative learning (e.g., Esber & Haselgrove, 2013; LePelley, 2004; Pearce & Hall, 1980; Pearce & Mackintosh, 2010), the induction of PE in a learning trial by the surprising presentation or omission of a US may also enhance attention to the cues or conditioned stimuli (CSs) present on that trial, as reflected in the rate at which those CSs subsequently enter into associations (their associabilities). Considerable behavioral evidence supports this assertion as well (e.g., Pearce & Mackintosh, 2010).

We have examined the function of a brain network comprising amygdala, midbrain, basal forebrain, striatal and cortical structures in generating and implementing these associability changes (see Holland & Maddux, 2010, and Holland & Schiffino, 2016, for reviews). Damage to portions of this system prevents these enhancements in cue associability under many but not all circumstances. Our investigations have primarily focused on the occurrence of such changes after the induction of negative PEs. One reason for this emphasis is the availability of conditioning procedures that distinguish between the role of negative PE in modulating the effectiveness of USs and the associability of CSs. Although reinforcement models such as that of Rescorla and Wagner (1972) unequivocally predict losses in associative strength with negative PEs, under some circumstances attention models such as that of Pearce and Hall (1980) predict that enhanced cue associability may facilitate excitatory learning.

Within the Rescorla-Wagner (1972) model, the effectiveness of a US in modifying CS-US associations is determined by the difference between the value (associative strength, V) supportable by the US (λ) and the aggregate strength of all CSs present on a learning trial (V_agg). Thus, if the US is unexpected (V_agg is low, and λ-V_agg is high) the increment in conditioning (ΔV) for each CS is large, whereas if the US is already anticipated on the basis of past learning (and λ-V_agg is low), new learning is minimal. Formally, ΔV_A = α_Aβ(λ-V_agg), where A represents a stimulus, and α and β refer to (constant) rate parameters for the CS and US, respectively. An important feature of RW is that its equation provides a simple basis for distinguishing between excitatory CSs (for which V_CS > 0) and inhibitory CSs (for which V_CS < 0), as well as specifying the conditions under which excitatory learning [(λ-V_agg) > 0, a positive PE] and inhibitory learning [(λ-V_agg) < 0, a negative PE] occur. Thus, in this model, the difference between the expected and received values of a US determines both the magnitude and direction of learning. By contrast, in the Pearce-Hall (1980) model, the effectiveness of the US remains constant, but PE modulates processing of the cues present when the PE is induced. Specifically, the associability (α) of a cue is proportional to the absolute value of the aggregate PE on previous trials. More formally, the change in the strength of a cue A on a learning trial n is ΔV_A = α_Anλ, where α_An ≈ |(λ-V_agg)_n-1|. Thus, somewhat counterintuitively, within this model if a US is consistently predicted (such that PE is small), the associabilities of accompanying CSs decline, whereas if a US is poorly predicted (such that PE remains large), a cue’s associability may be maintained at a higher level. Notably, because in this model cue associability is related to the absolute value of PE, either positive or negative PEs may enhance subsequent learning about a CS.

Increased excitatory learning about a CS, despite that cue’s accompaniment by negative PE, has been observed in a number of experimental paradigms, including an “unblocking” procedure (Dickinson, Hall, & Mackintosh, 1976) in which a serial reinforcer (e.g., US1→US2) is downshifted to a single (US1) reinforcer, a serial prediction task (Wilson, Boumphrey, & Pearce, 1992) in which a serial CS (CS1→CS2) is downshifted to a single CS (CS1), a temporal uncertainty task (Swan & Pearce, 1988), and a serial negative patterning procedure (Rescorla, 1991). In each of these tasks, evidence suggests that normal learning reflects the enhancement of cue associability by negative PEs, and that those enhancements are absent in rats with lesions of the amygdala central nucleus (CeA) or other parts of this system (Holland, Thornton, & Ciali, 2000; Holland & Gallagher, 1993a,b; Wheeler & Holland, 2011).

Common to these procedures is the induction of negative PE by the omission of an expected event. Importantly, induction of a negative PE by reduction in the value of a US, rather than its explicit omission, does not appear to enhance cue associability sufficiently to enhance excitatory learning, nor to generate CeA-dependent learning of any kind. For example, Esber and Holland (2014) found that whereas omission of US2 in an anticipated US1→US2 serial reinforcer produced excitatory learning in a downshift unblocking experiment, reductions in the magnitude or concentration of a US in an otherwise identical unblocking procedure instead produced only inhibitory learning (as predicted by the Rescorla-Wagner, 1972, model), which was unaffected by amygdala lesions. Citing electrophysiological evidence that CeA neurons that code US omission do not appear to code details of US value in a reward shift procedure (Calu et al., 2010) and immediate early-gene expression data indicating that CeA neurons code US omissions (downshifts in number) but not downshifts in concentration of sucrose USs (Esber & Holland, 2014), we suggested that CeA’s role might be limited to using information about probability (omission) of expected events to recalculate cue associability (Esber & Holland, 2014).

However, Haney et al. (2010) reported that transient reductions in AMPA neurotransmission in CeA interfered with learning in an “overexpectation” procedure (Rescorla, 1999), which produces negative PEs without changing either the probability or value of the US. After two CSs (A and B) were each separately paired with the US until learning was near asymptote, they were compounded (A + B) and followed by the same US. Because the aggregate predicted value (that is, the sum of the strengths of A and B) was greater than the value of the delivered US, a negative PE should have been produced. According to the Rescorla-Wagner (1972) model, that PE should encourage inhibitory learning to A and B (reductions in their strength) until their aggregate prediction matches the delivered US. Indeed, Haney et al. (2010) found that in control rats, responding to A in a final test was lower than responding to a control CS, C, which had been separately paired with another, but equivalent, US. Nevertheless, Haney et al. (2010) noted that within the Pearce-Hall (1980) model, the PE induced in this compound overexpectation training might also enhance the associabilities of A and B, allowing more rapid losses in their strengths. Consistent with that conjecture, Haney et al. (2010) found that rats that received A+B compound training after receiving infusions of the AMPA receptor antagonist NBQX into CeA responded similarly to A and C, thus failing to show the overexpectation effect found in control rats. Furthermore, Haney et al. (2010) also found that comparable infusions into the basolateral amygdala (BLA) did not interfere with the normal overexpectation effect.

From our perspective, two aspects of Haney et al.’s (2010) results are surprising. First, disruptions in CeA function apparently interfered with associability enhancements after a PE-inducing manipulation that not only did not involve US omission, but also required a fairly subtle comparison of the summed strengths of two separate CSs with the current US value. Recall that Calu et al. (2010) found no evidence that CeA neurons that coded the omission of an expected US also coded the magnitude of that US. Unfortunately, in Haney et al.’s (2010) study, NBQX infusions into CeA did not actually affect responding to the compounded cue A: Responding to A was no greater in rats with CeA infusions than in rats with BLA or vehicle infusions. Instead, responding to the other, control CS, C, was lower in CeA-infused rats than the other rats. Although there is no obvious reason why responding to that control CS (which was paired with a different US) should be affected by perturbed CeA activity, the possibility remains that alteration of CeA activity did not in fact alter overexpectation itself in Haney et al.’s (2010) study. Second, given that manipulations of BLA function have been found to disrupt other examples of PE-induced changes in cue associability (Chang, McDannald, Wheeler, & Holland, 2012; Esber & Holland, 2014) and the frequent assertion that a major function of BLA in appetitive condiftioning is to represent US value and to adjust those values when new information is experienced (e.g., Corbit & Balleine, 2005; Pickens, et al., 2003; Roesch, Calu, Esber, & Schoenbaum, 2010), it is puzzling that manipulation of BLA activity had no discernible effect on the overexpectation phenomenon. Furthermore, using a procedure similar to Haney et al.’s (2010), Takahashi et al. (2009) found that contralateral infusions of GABA agonists into the orbitofrontal cortex (OFC) and the ventral tegmental area (VTA), both of which have much richer connectivity with BLA than with CeA, also eliminated the overexpectation effect.

In the present study, we reconsidered the roles of CeA and BLA in overexpectation. In Experiment 1, we extended Haney et al.’s (2010) research in several ways. First, by using pretraining excitotoxic lesions rather than NBQX infusions during the compound phase only, we could consider whether Haney et al.’s (2010) failure to find an effect of BLA perturbation may have been due to neural activity spared by (or enhanced by) NBQX infusions (e.g., Jakawich, et al., 2010) or a BLA role in the expression of overexpectation effects at the time of test rather than in the computation or use of negative PE during compound training. Second, by using a different control procedure, and a single-reinforcer, between-subjects rather than a two-reinforcer, within-subjects design, we attempted to minimize extraneous factors that may have encouraged differential responding to the control CS, but not the overexpectation CS, in Haney et al.’s (2010) experiment. Despite these substantial procedural differences, we replicated Haney et al.’s (2010) major findings, except that the effects of CeA lesions were observed in our overexpectation condition and not in our control condition.

In Experiment 2, we again examined the effects of CeA and BLA lesions on negative PE, while simultaneously examining their effects on another predicted consequence of overexpectation, the conditioning of conditioned inhibition to a novel cue presented on compound overexpectation trials. The Rescorla-Wagner (1972) model makes the counterintuitive prediction that such a cue should acquire conditioned inhibition despite its consistent pairings with the US. This procedure allowed us to compare the effects of CeA and BLA lesions on the use of negative PE in generating conditioned inhibition itself with their putative effects on the use of negative PE in recalculating cue associability, and hence modulating the rate of acquisition of that inhibition.

Finally, in Exp. 3 we examined the roles of CeA and BLA in another puzzling phenomenon that might be attributable to overexpectation effects. Holland et al. (2014) found that when insertion of a lever was added to a previously-food-paired auditory cue and paired with the same food US, rats rapidly acquired conditioned responding to that lever, but the auditory cue lost some of its ability to elicit conditioned responding. That is, not only did the auditory cue fail to block (Kamin, 1968) conditioning to the added lever, but also the lever appeared to suck previously-established strength from the pretrained auditory cue. Holland et al. (2014) suggested that this “vampire effect” might be mediated by an overexpectation-like effect: as the lever acquired additional associations with the food US, the consequent negative PE could result in losses in the strength of the auditory CS. In that case, CeA (but not BLA) lesions might be expected to abolish this effect.

For convenience in exposition, this study was described as three experiments. However, all rats participated in all three experiments, which were conducted sequentially, in three replications.

General Methods and Materials

Subjects

The subjects were 72 male Long-Evans rats (Charles River Laboratories, Raleigh, NC, USA), which weighed 300–325 g on arrival to the laboratory vivarium. Rats were individually housed in a colony room with a 12:12 hr light-dark cycle. They received about a week of free access to food and water prior to surgery. Surgery was followed by 10–14 days of recovery before behavioral training. During the recovery period, the rats were handled daily. Seven days before the beginning of behavioral training, their access to food was restricted, such that their weights reached and were then maintained at 85% of their free feeding weights. Behavioral training sessions were conducted during the light portion of the light-dark cycle. The care and experimental treatment of rats was conducted according to the National Institutes of Health’s Guide for the Care and Use of Laboratory Animals, and protocols were approved by the Johns Hopkins University Animal Care and Use Committee.

Apparatus

The behavioral training apparatus consisted of eight individual chambers (20.5 cm × 22.0 cm × 22.5 cm) with stainless steel front and back walls, clear acrylic sides, and a floor made of 0.48-cm stainless steel rods spaced 1.9 cm apart. A food cup was recessed in a 5.0 × 5.0 cm opening in the front wall, and photocells at the front of the food cup recorded entries and time spent in the cup. Grain pellets (45 mg, Formula 5TUM, Test Diets, Richmond, IN, USA) were delivered to the food cups by pellet feeders (Coulbourn H14–22, Allentown, PA, USA). A 1-w lamp was mounted behind a perforated steel hemisphere on the front wall, 10 cm above the food cup; illumination of this lamp served as the visual stimulus. In Experiment 3, locally-fabricated retractable levers, which were operated quietly by pneumatic controls, were located on either side of the food cup. An infrared activity monitor (Coulbourn H24–61) and a bank of infrared LEDs to provide bright illumination for video observation were mounted on the top of each chamber. Neither video nor activity data are reported here. Each chamber was enclosed inside a sound attenuating shell. A piezoelectric device for presenting a 75-db, 4500-hz tone, a speaker for presenting a 78-db white noise, and a video camera was mounted on the side wall of the shell. A relay clicker (6 hz) was mounted on the floor of the shell at the front of the side wall. A 1-W lamp mounted behind a red lens near the tone source provided constant dim illumination visible to the rats, and ventilation fans provided masking noise (66 dB).

Surgery

Prior to all experiments, the rats were assigned to one of three lesion conditions: NMDA lesions of the BLA alone (Group BLA), ibotenic acid lesions of the CeA alone (Group CeA), and sham lesions that involved infusion of the PBS vehicle. Because the CeA has a lower density of NMDA receptors than BLA, the use of NMDA for BLA lesions permitted greater regional specificity (Holland & Gallaher, 2003). Rats from each lesion condition were represented in each replication. Stereotaxic (Kopf Model 902, Tujunga, CA, USA) surgery was conducted under aseptic conditions. Rats were maintained under anesthesia with 2–3% isoflurane mixed with oxygen. Bilateral BLA lesions were made using 10 mg/ml NMDA (Sigma, St. Louis, MO, USA) in Dulbecco’s phosphate-buffered saline (PBS) solution infused into each of two sites in each hemisphere with a 2.0 µl Hamilton syringe over a 4-min period. The injectors remained in place for 3 min after infusions before they were removed, to allow diffusion away from the tip. The coordinates used were 2.8 mm posterior of bregma and 5.1 mm from the midline, with infusions at depths of 8.7 mm (0.16 µl) and 8.4 mm (0.08 µl) from the skull surface. Bilateral CeA lesions were made by infusing .08 µg 10 mg/ml ibotenic acid (Sigma) at 2.2 mm posterior of bregma, 4.3 mm from the midline, and 8.1 mm below the skull surface. Sham lesions were made by infusing PBS alone in the same manners.

After surgery, the incision was closed with surgical staples and topical antibiotic ointment was applied to the wound edges. After removal from the stereotaxic apparatus, each rat received a single 0.3-ml subcutaneous injection of 0.02 mg/ml buprenorphine hydrochloride (Sigma) for amelioration of pain, and was allowed to recover from surgery for 10–14 days before beginning behavioral training.

Histological procedures

After completion of Experiment 3, the rats were deeply anesthetized with isoflurane and perfused intracardially with 0.9% saline followed by 3.7% formalin. The brains were removed and stored at 4°C in 3.7% formalin + 12% sucrose solution. A freezing microtome was used to take 40-µm sections from each brain. Of every 3 consecutive sections, the first was mounted on glass slides, dehydrated in ascending concentrations of alcohol, defatted in xylene, and Nissl-stained with thionin for evaluation of lesions.

Lesion evaluation

Lesions were evaluated from photographs of the Nissl-stained sections at 5 coronal planes of the amygdala (−1.80, −2.30, −2.80, −3.30, and −3.80 mm relative to bregma). Outlines of the lesion extents were drawn on digital images from Paxinos and Watson (1998) using Adobe Photoshop CC 2014.2.2. Calculation of percentage damage was performed within Photoshop by comparing the area of the intersection of lesion and region extent with the area within the region’s borders. The lesion outlines for each rat at each plane were then filled in Photoshop with an opacity of 100/n, where n was the number of lesions represented, and stacked onto a single atlas section image, such that the darkness of an area reflected the number of lesions that included that area.

Experiment 1

Experiment 1 was designed to examine the effects of pretraining excitotoxic lesions of CeA or BLA on the overexpectation phenomenon. All rats first received pairings of each of two auditory cues (A and C) with a food US, and nonreinforced presentations of a third auditory CS (B). Then, rats in the overexpectation condition (Group Over), received pairings of an A+C compound with the food US, whereas rats in the control condition (Group CTL) received pairings of a B+C compound with the food US. Because both A and C, but not B, predicted food delivery, within the Rescorla-Wagner (1972) model, the A+C compound in Group Over would be followed by the induction of a negative PE, whereas delivery of the B+C compound in Group CTL would not. Thus, the Rescorla-Wagner (1972) model predicts losses in response strength to the A and C cues in Group Over, relative to Group CTL. In preparation for Experiment 2, a novel visual cue accompanied the auditory compound CSs in half of the rats in each training and lesion condition of Experiment 1; this manipulation might reduce the magnitude of the anticipated overexpectation effect, but would be unlikely to eliminate it.

Procedures

Behavioral training procedures

Once their weights reached 85% of their ad lib weights, rats were given 10–20 grain pellets in their home cages, to familiarize them with the reinforcer. The next day, the rats were trained to eat pellets from the recessed food cups, in one or two (as needed) sessions, each including 16 unsignaled reinforcer deliveries, with random intertrial intervals (ITIs, mean = 4 min, range = 2 to 6 min).

Table 1 provides an outline of the procedures of Experiment 1. All rats first received 8 16-min sessions designed to establish conditioning to 2 10-s auditory cues (A, either the tone or noise, and C, the clicker) but not to a third 10-s cue (B, either the noise or the tone). On reinforced trials, the termination of the auditory cue coincided with the delivery of 2 45-mg pellets over a 1-s period. Each session included 6 training trials with each cue, randomly intermixed, with random ITIs.

Table 1.

Outline of behavioral training procedures

Experiment 1
Group	Element training	Compound	Test
Over (no Vis)	A→food, B→nothing, C→food	AC→food	A? B? C? V?
Over (Vis)	A→food, B→nothing, C→food	VAC→food	A? B? C? V?
CTL (no Vis)	A→food, B→nothing, C→food	BC→food	A? B? C? V?
CTL (Vis)	A→food, B→nothing, C→food	VBC→food	A? B? C? V?

Experiment 2
Group	Compound training	Summation test	Retardation test
Over	VAC→food, A→food, B→no food, C→food	A, VA, C, VC, B, VB, V, VAC, VBC	V→food
CTL	VBC→food, A→food, B→no food, C→food	A, VA, C, VC, B, VB, V, VAC, VBC	V→food

Over = Overexpectation treatment; CTL = control treatment; Vis and no Vis refer to the presence or absence of a visual cue (V) in the compound training phase. A and B were a tone or a noise (counterbalanced) and C was a clicker. Rats with excitotoxic or sham lesions of central or basolateral amygdala were equally represented in each training condition (group).

Over = Overexpectation treatment; CTL = control treatment; A and B were a tone or a noise (counterbalanced), C was a clicker, and V was a visual conditioned stimulus. All CS element and compound presentations in the summation test were nonreinforced. Rats with excitotoxic or sham lesions of central or basolateral amygdala were equally represented in each group.

Next, the rats received 2 sessions of either control (Group CTL) or overexpectation (Group Over) training. In each of these sessions, the rats in Group CTL received 12 presentations of a compound of cues B and C, reinforced. Because only C had been previously paired with pellets, no negative prediction error should have occurred, and responding to cues C and B (and to the unpresented A) should be maintained. By contrast, the rats in Group Over received 12 presentations of a compound of cues A and C, reinforced with delivery of two food pellets. Because each of these cues had previously been separately paired with the delivery of 2 pellets, the 2-pellet delivery should be accompanied by a negative prediction error. Thus, losses in responding to both A and C, relative to that in Group CTL, would be expected. For half of the rats in each of these groups, the compound cues were accompanied by the visual (V) cue (that is, they received either VAC or VBC compounds), whereas the other half received no visual cues (that is, they received either AC or BC compounds). The purpose of the visual cue accompaniments was to evaluate overexpectation-induced losses in responding to A and C under stimulus conditions to be used in Experiment 2, after only a brief period of overexpectation training.

Finally, to evaluate overexpectation effects, we examined responding to the auditory cues A, B, and C in all rats in a single 64-min test session, which included 4 nonreinforced presentations of each of those cues, randomly intermixed. Overexpectation would be indicated by less responding to cues A and C in Group Over than in Group CTL. In this test, the rats also received 4 10-s presentations of V alone intermixed with the auditory cues, to provide a baseline for responding to V alone in Experiment 2.

Behavioral measures and analyses

The behavioral response measures were the percentages of time spent in the food cup during a 5-s stimulus-free pre-CS period (immediately prior to the CS), and during the final 5-s of each 10-s CS period. Conditioned food cup responding was assessed during the latter half of CS presentations because in that interval, food cup CRs are more frequent and less contaminated by conditioned ORs (e.g., Holland, 1977).

CRs during the pre-CS and CS periods were each analyzed with separate mixed analyses of variance (ANOVAs) with replication, group treatment (over or CTL), compound treatment (with or without visual cue), lesion (BLA, CeA, or sham), and identity of the A cue (tone or noise) as between-subject variables, and the various test stimuli as within-subject repeated-measures variables. Initial ANOVAs showed no significant main effects of replication or A cue identity, and only a small number of (mostly higher-order) interactions with the other variables, so those 2 variables were dropped from the analyses presented here. When within-subject variables were included in the analyses, the Greenhouse-Geisser procedure was used to compensate for potential violations of sphericity assumptions. Unless otherwise noted, all individual comparisons were post hoc and used the Tukey honestly-significant difference (HSD) procedure for unequal ns. Effect sizes are given as partial η².

Results and Discussion

Lesion results

Twenty of the 24 CeA lesions and all of the 18 BLA lesions were judged as acceptable, with at least 50% damage to the target (CeA or BLA) structure and less than 10% damage to the other structure. Figure 1 shows the full extent of each lesion at each of 5 coronal sections. At each section, the darkness of an area reflects the number of lesions that included that area. Table 2 shows the mean percentage damage to CeA, including its medial (CeAm) and lateral/capsular (CeAl/c) subregions, and to BLA, including the lateral (LA), basolateral (BL), and basomedial (BM) nuclei. CeA lesions frequently included minor damage to the medial portions of BL, but otherwise the lesions were highly specific to their target region.

Figure 1.

Amygdala lesions in experiment 1. (A) Extent of lesions at 5 anterio-posterior planes in the rats that received ibotenate lesions of amygdala central nucleus (CeA). (B) Extent of lesions at 5 anterio-posterior planes in the rats that received NMDA lesions of the basolateral amygdala (BLA). In both A and B, each lesion is represented; the darker an area, the more lesions included that area (see text for details). The scale between panels A and B relates the shades of gray in a location to the percentage of lesioned rats with damage to that location. The lesions are drawn on sections from The Rat Brain in Stereotaxic Coordinates (4th ed.), pp. 26, 29, 31, 33 and 35, by G. Paxinos & C. Watson, 1998, New York, NY: Academic Press. Copyright 1998 by Elsevier Academic Press. Adapted with permission. The numbers to the right of each section indicate mm posterior (−) to bregma.

Table 2.

Mean (± s.e.m.) percentage damage to amygdala subregions

	CeA	CeAl/c	CeAm	BLA	LA	BL	BM
CeA Over	77.4 ± 3.5	84.8 ± 2.6	67.5 ± 5.1	6.7 ± 1.0	2.8 ± 0.7	12.1 ± 1.8	2.9 ± 1.9
CeA CTL	71.9 ± 3.7	80.1 ± 3.6	60.9 ± 4.3	5.1 ± 1.3	2.6 ± 1.0	9.2 ± 2.3	0.7 ± 0.3
BLA Over	3.5 ± 2.0	5.1 ± 2.7	1.3 ± 1.1	71.8 ± 4.0	64.2 ± 4.6	83.8 ± 3.7	59.5 ± 4.6
BLA CTL	1.1 ± 0.4	1.7 ± 0.7	0.2 ± 0.2	77.5 ± 3.4	77.9 ± 3.5	86.9 ± 2.9	60.0 ± 6.1

Note that percentage damages for subregions do not average to percentage total damage because the areas of the subregions differ. CeA = amygdala central nucleus (all); CeAl/c = lateral/capsular divisiions of CeA; CeAm = medial division of CeA; BLA = basolateral complex of the amygdala (all); LA = lateral amygdala; BL = basolateral nucleus; BM = basomedial nucleus.

Behavioral Results

Element training

Figure 2 shows responding to the A, B, and C cues during the initial conditioning phase. All rats acquired CRs to the A and C cues, but not the B cue. acquisition of responding was more rapid to the C cue (clicker) than to A (tone/noise) cue. No aspect of acquisition was significantly affected by lesion. A lesion X treatment X compound X cue X session ANOVA showed significant effects of cue, F_(2,92) = 168.07, p < .001, η² = .785, and session, F_(7,322) = 163.8, p < .001, η² = .781, as well as their interaction, F_(14,644) = 68.78, p < .001, η² =.599. No other effects or interactions were significant, ps > 0.104, η²s < .056. Separate ANOVAs for each cue also showed only significant effects of session ps < .001, η² = .197 except for the nonreinforced B stimulus, for which the Lesion X Session interaction was significant F_(14,322) = 1.99, p = .018, η² = .080. Individual comparisons showed that responding to B was lower (ps < .050) in sham-lesioned rats than in either of the lesion groups on the final session, but did not differ significantly on any other session. Finally, pre-CS responding showed only a main effect of session, F_(7,322) = 14.16, p < .001, η² = .235, and a Compound X Session interaction, F_(7,322) = 3.95, p < .001, η²= .079. The differences between pre-CS CRs for rats that were to later receive the visual cue accompanying the auditory compounds versus those to be trained without that cue appeared randomly distributed among the 8 sessions, and neither the main effect of compound nor any of its other interactions was significant ps > .148, η²s < .033.

Figure 2.

Mean ± s.em. responding during the reinforced A (tone or noise, counterbalanced) and C (clicker) and the nonreinforced B (noise or tone) conditioned stimuli (CSs), and the pre-CS periods in the initial acquisition phase (sessions c1→c8) of Experiment 1 and the reminder sessions of Experiment 2 (r1) and Experiment 3 (r3). CeA = rats with lesions of amygdala central nucleus; BLA = rats with lesions of basolateral amygdala.

Compound training

Figure 3 shows responding to the compound cues during the overexpectation training phase, in which rats in Group Over received compound presentations that included two previously-reinforced cues, A and C, whereas rats in Group CTL received compounds of the previously-reinforced C and the previously nonreinforced B cue. Addition of the visual cue suppressed responding to the auditory compounds on the first block of trials. There was little evidence of summation, that is, greater responding to the A+C compound than to the B+C compound, either overall or in any lesion/compound condition.

Figure 3.

Mean ± s.em. responding during the reinforced compound conditioned stimuli (CSs) and pre-CS periods in the compound training phase of Experiment 1. Panel A shows responding in rats in which the auditory compound CSs were presented alone and Panel B shows responding in rats in which the auditory compounds were accompanied by a visual stimulus. Rats in the Over condition received compounds of A and C, the two CSs that were reinforced in the first phase, and rats in the CTL condition received compounds of C with B, the previously-nonreinforced CS. CeA = rats with lesions of amygdala central nucleus; BLA = rats with lesions of basolateral amygdala.

A lesion X treatment X compound X blocks ANOVA showed significant main effects of compound (visual cue present or not), F_(1,46) = 4.68, p = .036, η² = .092 and block, F_(3,138) = 17.96, p < .001, η² = .281, and a significant Compound X Block interaction, F_(3,138) = 4.63, p = .005, η² = .091. Simple effects analysis showed that the visual compound evoked significantly less responding than the compound without visual cues only in the first half-session block (p = .046). Neither the main effect of treatment (Over vs. CTL) nor any of its interactions was significant, ps > .470, nor were any contrasts of Over vs. CTL in any lesion or lesion/compound condition, ps > .331. Pre-CS responding declined over the session blocks, F_(3,138) = 2.76, p = .045, η² = .057

Overexpectation test

Figure 4 shows the primary results of Experiment 1, responding to the two excitatory CSs, A and C. Among both sham-lesioned rats and rats with BLA lesions, responding to these cues was significantly lower in Group Over than in Group CTL, signifying the overexpectation effect, whereas among CeA-lesioned rats responding was equally high in both treatment groups. These outcomes were observed regardless of whether the compounds had included the visual cue or not. A lesion X treatment X compound X cue (A or C) ANOVA showed significant main effects of treatment, F_(1,46) = 34.88, p < .001, η² = .431, and cue, F_(1,46) = 39.87, p < .001, η² = .464, and a significant Lesion X Treatment interaction, F_(2,46) = 12.11, p < .001, η² = .345. Ps for all other effects and interactions were > .116, η²s < .089. Individual comparisons showed that for each of cues A and C, responding was lower after the overexpectation treatment than after the CTL treatment in both sham-lesioned (ps < .003) and BLA-lesioned (ps < .001) rats, but not in CeA-lesioned rats (ps > .896). Furthermore, CeA-lesioned rats in Group Over responded significantly more to each of the cues than either sham-lesioned (ps < .012) or BLA-lesioned (ps < .030) rats, whereas among rats in Group CTL there were no significant differences in responding to those cues across lesion condition (ps > .338).

Figure 4.

Mean ± s.em. responding to the cues A (panel A) and C (panel B) in the test phase of Experiment 1. Cues A and C had each been paired with food in the initial conditioning phase. In the compound training phase, rats in the Over condition received reinforced AC compound presentations intended to produce an overexpectation of food (negative prediction error), whereas rats in the CTL condition received reinforced BC compound presentations, intended to produce little prediction error. The overexpectation phenomenon is indicated by lower responding to A and C after the Over treatment than after the CTL treatment. CeA = rats with lesions of amygdala central nucleus; BLA = rats with lesions of basolateral amygdala.

There was no evidence for conditioning of the visual cue among rats that had received reinforced presentations of it in compound with the auditory cues in the compound phase (range across all 12 lesion/treatment/compound conditions = 0.9 ± 0.9% to 10.0 ± 6.6 %). A lesion X treatment X compound ANOVA of responding to that cue showed neither a main effect of compound nor a Treatment X Compound interaction, Fs < 1, ps > .862. Finally, prior training of C apparently blocked acquisition of conditioning to the originally-nonreinforced auditory cue B during the overexpectation phase in the CTL rats: Those rats showed no more responding to B in test than the rats in Group Over, which received no B presentations in the overexpectation phase. A lesion X treatment X compound ANOVA of responding to B showed no significant main effects or interactions (ps > .129), although the lesion X treatment interaction approached significance, F_(2,46) = 2.96, p = .062, η² = .114. In the CTL condition, responding to B was marginally (ps < .15) lower in sham-lesioned rats (5.9 ± 2.2%) than in the CeA-lesioned rats (14.6 ± 6.8%) or BLA-lesioned rats (16.5 ± 4.2%). However, that same difference in responding to B was observed at the end of Phase 1 training (Figure 2).

Consistent with Haney et al.’s (2010) findings, we found that CeA, but not BLA, dysfunction interfered with the normal overexpectation effect of losses in conditioned responding after two CSs that were previously separately-paired with the US were jointly reinforced with that US. Unlike in Haney et al.’s (2010) experiment, that lesion deficit was expressed as more responding (less loss) in the overexpectation condition (Group Over) in CeA-lesioned rats than in sham- or BLA-lesioned rats, and performance in the control condition (Group CTL) did not differ among the lesion conditions. Thus, Experiment 1 extended Haney et al.’s (2010) results to permanent lesion procedures, a between-groups design, and a different control procedure, and removed doubts about the origins of the CeA lesion deficit. Both sets of results are consistent with the view that in rats subjected to overexpectation procedures, losses in conditioned responding to the cues produced by negative PE are normally amplified by PE-induced enhancement of the associability of those cues, which are absent in rats with CeA lesions.

Experiment 2

The Rescorla-Wagner (1972) model describes the overexpectation losses observed in Experiment 1 as resulting from inhibitory learning to the A and C cues after negative PEs were induced by reinforcement of the AC compound. Kremer (1978) noted that adding a new cue, V, when the A and C cues are compounded (VAC) should result in the accrual of net inhibition to V, despite its having been presented only when paired with a reinforcer. Because the joint presentation of A and C followed by a single reinforcer generates a negative PE, inhibitory learning should occur to all three of the A, C, and V cues, reducing the associative strength of A and C from their prior levels, and reducing the strength of V from 0 to a negative value. In an initial experiment with that simple design, Kremer (1978) failed to find that outcome, but he noted that inhibitory learning is often slow, and that as the strengths of A and C decline, the negative PE decreases. Thus, Kremer (1978) suggested that net inhibition to V might be better revealed if separate reinforced presentations of the A and C cues were intermixed with reinforced VAC pairings. By maintaining the strengths of A and C alone, these additional presentations would prolong the induction of a negative PE on VAC trials. Indeed, Kremer (1978) reported evidence that this procedure yielded net inhibition to the added V cue. Unfortunately, Kremer (1978) used only a retardation test for conditioned inhibition, and in that test compared the rate of learning about the V cue to learning about that cue in rats that had never experienced it before. Thus, slower learning to V in the retardation test after overexpectation training could have simply reflected latent inhibition (Lubow & Moore, 1959; Rescorla, 1971), slower learning about a previously-presented cue than about a novel cue.

The first goal of Experiment 2 was to replicate Kremer’s observation under proper control conditions, using both summation and retardation tests of conditioned inhibition (Rescorla, 1969). After reestablishment of conditioning to cues A and C in a reminder session, rats received reinforced presentations of VAC (Group Over) or VBC (Group CTL) compounds, intermixed with reinforced A and C trials, and nonreinforced B trials. Thus, in Group Over, but not in Group CTL, the visual cue was followed by a negative PE. After evaluating overexpectation effects to A and C (as in Experiment 1), we then used both summation and retardation test procedures to evaluate the conditioned inhibitory powers of cue V. In the summation test, responding to A and C cues presented alone was compared to responding to VA and VC compounds. If responding to A and C was suppressed more by V in Group Over than in Group CTL, we could claim that the overexpectation treatment established conditioned inhibition to V. In the retardation test, cue V was repeatedly paired with food. If acquisition of responding to V was slower in Group Over than in Group CTL, we could claim that the overexpectation treatment established conditioned inhibition to V.

We expected that the assessment of overexpectation (losses in responding to A and C) would reveal the same lesion effects as in Experiment 1. Of more interest is the effects of those lesions on the acquisition of conditioned inhibition to V. If CeA lesions interfered with overexpectation effects by interfering with the calculation or use of negative PEs to produce inhibitory learning, as described by the Rescorla-Wagner (1972) model, then CeA lesions should also interfere with the acquisition of inhibition to V in Group Over. Similarly, within the Pearce-Hall (1980) model, to the extent that inhibitory learning about V is normally enhanced by surprise-induced increases in V’s associability, the prevention of such increases by CeA lesions should slow inhibitory learning about V, just as it slows loss of excitation to the A and C cues.

This last prediction depends on Pearce and Hall’s (1980) assumption that PE forces adjustments in the associability of all cues present at the time (α_A ≈ |λ- V_agg|). However, Holland (2012) suggested an alternative assignment of credit rule for the Pearce-Hall (1980) model, such that only cues that participate in generating a negative PE benefit from that PE in altering associability. Because V’s strength in Group Over would start at zero and decline, V would not contribute to the negative PE induced when the VAC compound was reinforced, and hence V’s associability would not benefit. Thus, although conditioned inhibition would be expected to accrue to all cues present on VAC trials, resulting in the loss of responding to A and C and the acquisition of net conditioned inhibition to V, only changes in the strengths of A and C would benefit from associability enhancements, and hence only those changes would be impaired by CeA lesions. Thus, from this modified Pearce-Hall perspective, although CeA lesions would interfere with the acquisition of inhibitory associations to the A and C cues, and hence with losses in responding to them (overexpectation), they would not interfere with the acquisition of conditioned inhibition to V.

Procedures

Behavioral training procedures

Table 1 provides an outline of the procedures of Experiment 2. All rats from Experiment 1 first received a “reminder” conditioning session, identical to those of the first phase of Experiment 1, to reestablish responding to A and C, but not B. Next, the rats received 4 64-min sessions in which they received 2 reinforced presentations of A, 2 reinforced presentations of C, 2 nonreinforced presentations of B, and 12 reinforced compounds, either VAC (Group Over) or VBC (Group CTL), randomly intermixed. The rats maintained their group memberships (Over or CTL) established in Experiment 1. Because in Group Over, the delivery of a single 2-pellet reinforcer would induce a negative PE on VAC trials, conditioned inhibition should accrue to each of cues V, A, and C. Thus, losses in responding to A and C would be expected, as in Experiment 1, but in addition, because V started as a neutral cue, it should accrue net negative (inhibitory) value. By contrast, in Group CTL, because only cue C generated an expectancy of reinforcement, no negative PE would occur on VBC trials, and hence neither losses in the strength of C nor the establishment of net inhibition to V would be anticipated.

Finally, the rats received two tests to assess overexpectation effects to A and C, and inhibition to V. The first, a 64-min summation test, included two 10-s presentations each of A, B, C, V, VA, VB, VC, VAC, and VBC. The order of these trials was randomized across subjects, with the constraint that all nine trial types were presented before any was repeated. Overexpectation effects (as in Experiment 1) would be revealed if responding to A and C were lower in Group Over than in Group CTL. Inhibition to V would be revealed as greater suppression of responding to A and C by V in Group Over than in Group CTL. That is, if the overexpectation procedure established net inhibition to V, the difference between responding to C and to VC, and between A and VA, should be greater in Group Over than in Group CTL. Finally, all rats received a retardation test of the inhibitory power of V. In each of two 64-min sessions, they received 16 reinforced presentations of V. If the overexpectation procedure (Group Over) established conditioned inhibition to V, then acquisition of excitatory V-food associations should be slowed in that group relative to acquisition in Group CTL.

Behavioral measures and analyses

The behavioral measures and analyses were the same as in Experiment 1. Although in Experiment 2 all rats received visual + auditory compounds in the initial training phase, the compound variable (visual cue versus no visual cue) was retained in the ANOVAs, because it distinguished between rats that had received 2 sessions with visual + auditory compounds in Experiment 1 from those that received them for the first time in Experiment 2.

Results and Discussion

Reminder session

In the reminder session (Figure 2, session r1), responding to A, B, and C rapidly returned to levels comparable to those of Experiment 1. A lesion X treatment X compound X cue ANOVA showed only an effect of cue, F_(2,92) = 218.61, p < .001, η² = .826; other ps > .250, η²s < .030. As in Phase 1, a separate ANOVA for responding to the nonreinforced B cue showed a marginally significant effect of lesion F_(2,46) = 2.96, p = .062, η² = .114, although individual contrasts showed no significant differences (ps> .100) between any pair of groups.

Compound training

Figure 5 shows acquisition of responding to the visual + auditory cue compounds (VAC in Group Over and VBC in Group CTL), the separately-reinforced A and C cues (combined), and the separately-nonreinforced B cue. Neither lesion nor treatment condition affected responding to any of the cues, but responding to the compounds was suppressed on the first day of training for rats that were receiving them for the first time (that is, the rats in the no-visual cue condition of Experiment 1; not shown separately in Figure 5). A lesion X treatment X compound X cue X session ANOVA showed significant effects of cue, F_(3,138) = 315.51, p < .001, η² = .873, and session, F_(3,13) = 2.87, p = .039, η² = .059, and significant Cue X Session, F_(9,414) = 7.32, p < .001, η² = .137, and Compound X Cue X Session, F_(9,414) = 2.45, p = .010, η² = .051, interactions. No other effect or interaction was significant, ps > .105, η²s < .043. A comparable ANOVA of responding to the visual + auditory compounds alone showed a main effect of session, F_(3,138) = 12.38, p < .001, η² = .212, and a significant Compound X Session interaction, F_(3,138) = 3.92, p = .010, η² = .079. ANOVAs of responding to the separately-presented cues (A, B, and C) showed only effects of sessions, Fs_(3,138) > 3.08, ps < .030, η²s > .062.

Figure 5.

Mean ± s.em. responding during the reinforced compound (Vcomp) conditioned stimuli (CSs), reinforced element CSs (A and C, averaged), and nonreinforced B stimulus, and during pre-CS periods in the compound training phase of Experiment 2. Panels A, B, and C show responding in sham-lesioned rats, rats with lesions of the amygdala central nucleus (CeA) and rats with lesions of the basolateral amygdala (BLA), respectively. Rats in the Over condition received compounds that comprised a visual stimulus (V), A, and C, and rats in the CTL condition received compounds that comprised V, B, and C.

Overexpectation test

The filled bars in Figure 6A show responding to the excitatory A and C element cues in the summation test (averaged). As in Experiment 1, responding was lower to those cues after overexpectation (Over) training than after CTL training, in both sham-lesioned and BLA-lesioned rats, but not in CeA-lesioned rats. Thus, we replicated our observation of Experiment 1, extending the findings of Haney et al. (2010), and supporting the view that in intact rats exposed to the overexpectation treatment, losses in cue strength induced by negative PEs are amplified by PE-induced enhancement of the associability of those cues, which is absent in rats with CeA lesions. A lesion X treatment X compound ANOVA showed a significant main effect of treatment, F_(1,46) = 13.26, p < .001, η² = .224, and a significant Lesion X Treatment interaction, F_(2,46) = 6.91, p = .002, η² = .231. Subsequent contrasts showed that responding was lower in the Over condition than in the CTL condition in both sham-lesioned (p = .005) and BLA-lesioned (p = .006) rats, but not in CeA-lesioned rats (p = .946). Furthermore, responding in the Over condition was greater in the CeA-lesioned rats than in the sham-lesioned rats (p = .006) rats, and marginally greater than in the BLA-lesioned rats (p = .089). Comparable ANOVAs and post-hoc tests for cues A and C separately showed equivalently significant results, except that in the Over condition, responding to A was significantly (p = .010) greater in CeA-lesioned rats than in BLA-lesioned rats, and responding to C was not (p = .156).

Figure 6.

Mean ± s.em. responding during the summation (Panel A) and retardation (Panel B) tests of Experiment 2. Panel A shows responding to the A and C (averaged) element conditioned stimuli (CSs, solid bars) and their compounds (open bars) with the suspected inhibitor, V. Lower responding to A/C in the Over condition than in the CTL condition indicates the overexpectation effect, and greater differences between A/C and VA/VC responding in the Over condition than in the CTL condition implies that V was a conditioned inhibitor in the Over condition. Although significant overexpectation was observed only in sham-lesioned rats and rats with lesions of the basolateral amygdala (BLA), significant (and equivalent) conditioned inhibition was observed in all three lesion conditions, including rats with lesions of the amygdala central nucleus (CeA). Responding to the other stimuli presented during the summation test is given in Table 3. Panel B shows acquisition of conditioned responding to the visual CS alone in the retardation test. In all three lesion conditions, acquisition of excitatory learning was significantly slowed in the rats previously trained in the Over condition compared to the acquisition of responding in the rats previously trained in the CTL condition.

One might wonder why the reinforced VAC presentations in the Over procedure of Experiment 2 would produce losses of conditioned responding to the A and C cues, given that the A and C cues were individually paired with reinforcement as well. According to the Rescorla-Wagner (1972) model, in the Over procedure, asymptotically, the strengths of A and C should each approach λ, and the strength of V should approach –λ. Likewise, in the CTL procedure, the strength of A should approach λ, the strength of C should remain at λ, and the strength of V should remain near zero (as should the strength of B). The simplest rejoinder to this concern is that the 4 compound training sessions used in Experiment 2 were insufficient to produce asymptotic learning. Thus, despite equivalent reinforced A and C presentations in both groups Over and CTL, the Over rats (but not the CTL rats) still overexpected the food US on compound trials, and hence pre-asymptotically the Over rats would show lower responding to A and C than the CTL rats. But it is unclear why such differences were revealed only in test (Figure 6A), and not during the compound phase itself (Figure 5), although one might reasonably invoke a ceiling effect to explain the lack of differences in the compound phase, or some reinstatement of the nonereinforced test conditions of Experiment 1 in the summation test of Experiment 2.

Summation test of conditioned inhibition

A summation test of conditioned inhibition to the visual cue V compares responding evoked by the A and C excitors when presented alone (filled bars in Figure 6A) to responding evoked by their compounds with V (VA and VC, open bars in Figure 6A). If the overexpectation (Over) treatment established conditioned inhibition to V, then responding to the VA and VC compounds should be lower than responding to the A and C elements alone. By contrast, because a negative PE was not generated in the CTL condition, V should not have acquired conditioned inhibition, and hence would not be expected to suppress responding to A or C. Thus, suppression of responding to A and C by V in the Over condition should be greater than any such suppression observed in the CTL condition. We found exactly that effect. Furthermore, unlike the overexpectation losses in responding to A and C described in the previous paragraph, V’s inhibitory powers were unaffected by the CeA lesion. A lesion X treatment X compound X cue (A/C combined vs. VA/VC combined) ANOVA showed significant main effects of treatment, F_(1,46) = 31.70, p < .001, η² = .408, and cue, F_(1,46) = 55.91, p < .001, η² = .549. A significant Lesion X Treatment, F_(1,46) = 9.39, p < .001, η² = .290, interaction reflected the higher responding in CeA-lesioned rats in Group Over, as noted previously, and a significant Treatment X Cue interaction, F_(1,46) = 12.08, p = .001, η² = .208, showed that the difference between responding to A/C and VA/VC was greater in the Over rats than in the CTL rats. Importantly, the Lesion X Treatment X Cue interaction, F < 1, p = .798, η² = .010, did not approach significance, indicating that V’s inhibitory strength in Group Over was unaffected by the lesions.

Retardation test of conditioned inhibition

Figure 6B shows the results of the retardation test of V’s inhibitory powers in Experiment 2. If the overexpectation treatment resulted in the acquisition of net conditioned inhibitory associations to V, then subsequent establishment of excitatory V-food associations should be retarded in rats in Group Over relative to rats in Group CTL. We obtained that outcome, regardless of lesion condition. A lesion X treatment X compound X session ANOVA showed significant main effects of treatment, F_(1,46) = 27.99, p < .001, η² = .378, and session, F_(3,138) = 90.41, p < .001, η² = .663, and a significant Treatment X Session interaction, F_(3,138) = 4.66, p = .004, η² = .092. Importantly, neither the effect of lesion nor any of its interactions was significant, ps > .171, η²s < .063. ANOVA of pre-CS responding (not shown) showed only an effect of session, F_(3,138) = 3.69, p = .014, η² = .074; other ps > .142, η²s < .039.

Thus, we extended Kremer’s (1978) claims that a negative PE produced by the overexpectation procedure could establish net conditioned inhibition to a new cue, using more appropriate control conditions and both summation and retardation tests, which ruled out a number of other interpretations to which Kremer’s (1978) data were susceptible. The observation of net conditioned inhibition to V despite its presentation only when accompanied by the US is a remarkable confirmation of the Rescorla-Wagner (1972) model’s notion that the establishment of conditioned inhibition to a cue is determined by negative PE, and not mere absence or reduced probability of the US.

Furthermore, although the overexectation losses in strength of the A and C cues depended on the integrity of the CeA (as in Experiment 1), the acquisition of conditioned inhibition to V was unaffected by CeA lesions, or by lesions of the BLA. These contrasting observations have important implications. Because both the overexpectation losses in cue strength and the acquisition of conditioned inhibition depend on negative PE, the CeA lesion effect observed cannot reflect a deficit in registering or computing reward predictions or PEs, or in generating inhibitory learning, but rather some deficit unique to the overexpectation phenomenon. One possibility is that negative PEs only enhance the associability of cues whose expectations contribute to generating that negative PE (Holland, 2012), in this case, A and C, but not V, for which any PEs would have been positive. Notably, evidence suggests that associability enhancements produced by positive PEs are unaffected by CeA lesions (e.g. Holland, 2006; Holland & Gallagher, 1993b).

Excitatory summation effects

Although the summation test revealed the expected summation of cues’ excitatory and inhibitory strength, the evidence for summation of responding when excitatory (reinforced) CSs were combined was mixed. Table 3 shows responding to the B, V, VB, VAC, and VBC stimuli presented in the summation test, as well as pre-CS responding (responding to A, C, VA, and VC in that test are portrayed in Figure 6A). Consistent with summation of excitatory strengths, responding to the VAC compound, which included 2 reinforced cues, was substantially greater than responding to the VBC compound, which included only one reinforced cue, F_{(1, 46)} = 25.22, p < .001, n² = .354. Furthermore, this difference was observed regardless of treatment or lesion condition, ps > .135, n²s < .048. However, this difference was not observed in the compound phase (Figure 5). Of course, one might again invoke a ceiling effect to account for this discrepancy. Similarly, in the summation test, although in Group Over overall responding to the VAC compound was greater than responding to VA or VC compounds in all 3 lesion conditions, in Group CTL that observation was only true (marginally) in CeA-lesioned rats. A lesion X treatment X compound X cue ANOVA showed significant main effects of cue, F_(1,46) = 29.95, p < .001, n² = .394, and treatment, F_(1,46) = 10.36, p = .002, n² = .184, and significant Treatment X Cue, F_(1,46) = 20.53, p < .001, n² = .309, and Lesion X Treatment X Cue, F_{(2, 46)} = 6.92, p = .002, n² = .231, interactions. Individual comparisons showed significantly greater responding to VAC than to the VA and VC compounds (averaged) in the Over group in all 3 lesion conditions (ps < .032), and marginally more responding to VAC in the CeA-lesioned rats in Group CTL (p = .066); other ps > .167.

Table 3.

Responding in the Summation Test of Experiment 2

	V	B	VB	VAC	VBC	pre-CS
Sham-Over	0.8 ± 0.4	5.3 ± 5.0	3.6 ± 2.9	51.8 ± 8.2	32.0 ± 7.4	2.1 ± 1.6
Sham-CTL	0.7 ± 0.4	2.1 ± 0.9	1.5 ± 0.7	54.2 ± 4.4	43.7 ± 2.7	1.4 ± 0.7
CeA-Over	3.0 ± 2.7	3.9 ± 1.8	4.6 ± 1.9	51.0 ± 7.6	27.2 ± 7.7	0.6 ± 0.4
CeA-CTL	0.6 ± 0.6	0.3 ± 0.3	4.4 ± 2.1	57.0 ± 7.2	46.7 ± 3.4	1.4 ± 1.1
BLA-Over	1.1 ± 1.0	6.7 ± 4.7	5.5 ± 4.7	53.4 ± 7.5	33.8 ± 6.7	2.2 ± 1.5
BLA-CTL	1.8 ± 1.1	7.3 ± 3.3	4.8 ± 1.5	50.6 ± 8.8	44.8 ± 7.3	1.6 ± 0.8

Entries are mean ± s.e.m. percentage behavior. Over = overexpectation treatment; CTL = control treatment; CeA = Lesions of amygdala central nucleus; BLA = lesions of basolateral amygdala. V = visual stimulus; B = previously nonreinforced CS (conditioned stimulus). During the previous training phase, rats in the Over groups received reinforced presentations of the VAC compound CS, whereas rats in the CTL groups received reinforced presentations of the VBC compound CS. See text for details of identities and training histories of each cue. Responding to cues A, C, VA, and VC is presented in Figure 6A.

Finally, individual Lesion X treatment X compound ANOVAs showed no significant effects or interactions for B, V, VB, or VAC, ps > .098, η²s < .096. However, ANOVA of responding to VBC showed a significant effect of treatment, F(_1,46) = 5.06, p = .029, η² = .010, with greater responding in the CTL rats than in the Over rats. This observation is consistent with the training of these two groups: only Group CTL had received reinforced VBC compound presentations. No other effect or interaction was significant, ps > .216, η²s < .033. ANOVA of pre-CS responding showed a significant main effect of Experiment 1 compound treatment, F_(1,46) = 8.00, p = .007, η² = .148, with greater pre-CS responding in rats that had received visual + auditory compounds in Experiment 1 (3.0 ± 1.0% vs. 0.6 ± 0.6%; not shown in Table 3). We have no explanation for this small effect in pre-CS rates‥

Experiment 3

The nearly-universally-observed phenomenon of blocking (Kamin, 1968) was a major stimulant for the development of modern learning theories. Consider two groups of rats that receive pairings of a tone + light compound stimulus with a US. For a blocking group, the tone was pretrained to asymptote with the same US, whereas for a control group it was not. Although rats in the control group acquire substantial conditioned responding to the light, the rats in the blocking group do not. Within the Rescorla-Wagner (1972) model, substantial conditioning accrues to both tone and light in the control group because the US was initially unexpected on compound trials. By contrast, in the blocking group, because the US was already well-predicted by the tone, the US is no longer effective in supporting new learning in the compound phase. Likewise, within the Pearce-Hall (1980) model, because the US is well-predicted by the tone when the light is introduced, the light’s associability is rapidly reduced, curtailing the formation of light-US associations.

However, Holland et al. (2014) found no evidence for blocking in a rat autoshaping experiment. Prior conditioning of an auditory CS failed to block acquisition of lever-press “sign-tracking” responses when insertion of a lever was accompanied by the auditory CS and reinforced with the same food reinforcer. Notably, not only was there no evidence of blocking of conditioning to the lever, but also the auditory cue lost some of its ability to evoke conditioned responding, despite its continued pairings with the reinforcer, as if conditioning to the lever sucked strength from the auditory cue. Holland et al. (2014) suggested that this “vampire” effect might reflect a type of overexpectation: if the lever could somehow acquire conditioning despite the presence of the previously-conditioned auditory cue, then the resulting overexpectation of food might eventually generate losses in responding to the auditory cue. If that account is correct, then one might expect the vampire effect, like the overexpectation effects observed in Experiments 1 and 2, to normally be potentiated by negative PE-induced enhancements in cue associability. In that case, one might expect that this effect would be reduced in rats with CeA lesions, but not in rats with BLA lesions, as in Experiments 1 and 2. However, if the vampire effect is unique to some privileged aspect of lever incentive learning (e.g., Flagel et al., 2011; Robinson & Flagel, 2009), then the opposite pattern of lesion effects might be expected, because the performance of autoshaped lever pressing is impaired by BLA (Chang, Wheeler, & Holland, 2012b), but not CeA (Chang, Wheeler, & Holland, 2012a) lesions.

In Experiment 3, the rats from Experiments 1 and 2 first received food-paired presentations of a compound of one of the previously-reinforced auditory CSs and the insertion of a lever.Then, in a final test, responding to that auditory CS was compared to responding to the other previously-reinforced CS, which was not presented during the autoshaping training sessions.

Procedures

Behavioral training procedures

All rats from Experiments 1 and 2 first received a reminder conditioning session, identical to those of the first phase of Experiment 1, to reestablish responding to A and C, but not B. Then, all rats received 8 64-min sessions of autoshaping training, each of which included 16 reinforced presentations of a 10-s compound of either A (noise or tone) or C (counterbalanced) and the extension of the right lever into the experimental chamber. If the vampire effect were observed, then the previously-reinforced auditory cue that accompanied the lever (termed the blocking or BLK cue) would lose responding relative to the other previously-reinforced auditory cue (termed the control or CON cue). To the extent that a lesion interfered with that effect, responding to the lever-accompanied BLK cue would not be reduced by that experience. The vampire effect was assessed in two final 64-min sessions. In the first of these sessions, there were 8 nonreinforced presentations each of BLK and CON, pseudorandomly intermixed, with the stipulation that there were 4 of each trial type in each half of the session. The second test session began with 4 nonreinforced presentations each of BLK and of CON, randomly intermixed, followed by 8 nonreinforced presentations of the right lever alone.

Results and Discussion

Reminder session

The reminder session reestablished appropriate response levels to A, B, and C (Figure 2, session r2). A lesion X Experiment and Experiment 2 treatment (Over vs. CTL) X Experiment 1 compound (visual vs. no visual cue) X cue ANOVA showed only an effect of cue F_(2,92) = 197.90, p < .001, η² = .811; other ps > .224, η²s < .063. Unlike in Experiments 1 and 2, the effect of lesion on responding to the nonreinforced B cue did not approach significance, F_(2,46) = 1.37, p = .265, η² = .008.

Autoshaping acquisition phase

Lever-press responding was acquired readily in the autoshaping sessions, despite the presence of a previously-conditioned auditory cue on lever-insertion trials (Figure 7), consistent with Holland et al.’s (2014) finding that a pretrained auditory CS did not block acquisition of lever-pressing in intact rats. Furthermore, in accordance with Chang at al.’s (2012a, b) findings, although the course of autoshaping was unaffected by CeA lesions, BLA-lesioned rats showed lower levels of lever-press responding and higher levels of food-cup behavior than either sham-lesioned or CeA-lesioned rats.

Figure 7.

Mean ± s.em. responding to the compound of lever presentation + previously-reinforced auditory cue in the autoshaping phase of Experiment 3. Panels A, B, and C show lever press rates, percentage of trials on which a lever-press occurred, and percentage of time spent in the food cup during the lever + auditory cue compounds, respectively. Panel C also shops the percentage of time spent in the food cup in the 10-s periods prior to lever presentations (pre). CeA = rats with lesions of amygdala central nucleus; BLA = rats with lesions of basolateral amygdala.

A lesion X treatment X compound X session ANOVA of lever-press rates (Figure 7A) showed significant main effects of lesion, F_(2,46) = 4.14, p = .022, n² = .153, and session, F_(7,322) = 25.29, p < .001, n² = .355, and a significant Lesion X Session interaction, F_(14,322) = 1.69, p = .050, n² = .069. No other effect or interaction was significant, ps > .182, n²s < .031. BLA-lesioned rats showed significantly lower lever-press rates than either sham-lesioned (p = .020) or CeA-lesioned rats (p = .029), which did not differ (p = .986). A similar analysis of lever-press probability (Figure 7B) yielded a marginally-significant effect of lesion, F_(2,46) = 3.14, p = .053, n² = .120. Individual comparisons showed that BLA-lesioned rats were significantly less likely to respond than CeA-lesioned rats (p = .050), and marginally less likely to respond than sham-lesioned rats (p = .062), which did not differ (p = .994). The ANOVA also showed a significant effect of session, F_(7,322) = 81.42, p < .001, n² = .639, and significant Lesion X Treatment X Session, F_(14,322) = 1.88, p = .027, n² = .076, and Treatment X Compound X Session, F_{(7, 322)} = 5.70, p < .001, n² = .110, interactions. These interactions did not appear systematic across sessions. No other effect or interaction was significant, ps > .167, n²s < .056.

Figure 7C shows food cup responding during the autoshaping phase. As pairings of the auditory + lever cue with food continued, initially high levels of food cup responding rapidly decreased, except among BLA-lesioned rats, which maintained somewhat higher levels of food-cup responding throughout. ANOVA showed significant main effects of lesion, F_(2,46) = 4.67, p = .014, n² = .169, and session, F_(7,322) = 99.36, p < .001, n² = .684, as well as Lesion X Session, F_(14,322) = 1.86, p = .030, n² = .074, and Treatment X Session, F_(7,322) = 2.89, p = .006, n² = .059, interactions. The former of these interactions reflected rapidly diverging responding among the groups, whereas the latter seemed unsystematic across sessions. ANOVA of pre-CS food cup behavior showed only a main effect of sessions, F_(7,322) = 10.19, p < .001, n² = .182; other ps > .114, n²s < .035.

Autoshaping acquisition phase: sign-trackers and goal-trackers

One of the 20 sham-lesioned rats, 1 of the 20 CeA-lesioned rats, and 5 of the 18 BLA-lesioned rats were initially classed as goal-trackers, showing maintained food-cup responding to the auditory + lever CS, and lever-pressing on fewer than 25% of the trials. The remaining rats were classified as sign-trackers. A chi-square test showed that the proportions of sign trackers differed significantly across the groups, χ²(2) = 6.069, p = .048. Interestingly, Chang et al. (2012b) did not find more goal-trackers among BLA-lesioned rats in simple lever-presentation autoshaping; perhaps the presence of a previously-trained auditory cue, which already controlled a goal-tracking (food cup) response, increased the probability of observing a goal-tracking response to the auditory cue + lever CS in the present experiment. Indeed, it is possible that the BLA-lesioned “goal-trackers” were simply maintaining their response to the auditory CS, rather than acquiring a food-cup response to the lever. This possibility is supported by the results of the final tests (described later) of responding to auditory and lever cues when they were presented separately.

When the analysis of autoshaping acquisition data was confined to sign-trackers, the lesion effects were numerically smaller, because the lowest lever-press and highest food-cup responders among BLA-lesioned rats were not included. ANOVA of lever press rates among sign-trackers alone showed only the main effect of session, F_(7,273) = 22.44, p < .001, n² = .365, and the Lesion X Session interaction, F_(7,273) = 1.73, p = .049, n² = .08, to be significant. Individual contrasts of overall responding showed no significant differences across lesion groups, but similar comparisons of responding over the last half of training showed significantly lower lever-press rates in BLA-lesioned rats than in either CeA- or sham-lesioned rats (ps < .050), which did not differ (p =.867). For lever-press probability, the effect of sessions was significant, F_(7,273) = 95.54, p < .001, n² = .710, but the Lesion X Session interaction was only marginally significant, F_{(14, 273)} = 1.69, p = .057, n² = .080. Individual comparisons showed only marginally-lower responding (ps < .12) in BLA-lesioned rats than in the other rats. Finally, ANOVA of food cup responding among sign-trackers showed only a main effect of sessions, F_(7,273) = 164.29, p < .001, n² = .808, suggesting that the higher food-cup responding of BLA-lesioned rats shown in Figure 7C largely reflected responding among goal-trackers. Notably, these more limited effects of BLA lesions on autoshaping correspond more closely to those reported by Chang et al. (2012b).

Auditory cue test

Figure 8 shows the primary data of Experiment 3, food-cup responding to the two previously-reinforced auditory cues. As noted by Holland et al. (2014), continued reinforcement of an auditory cue in the presence of the lever resulted in a loss of responding to that (BLK) cue, relative to responding to the CON cue, which was not presented during the autoshaping phase. If, as Holland et al. (2014) suggested, that loss reflected an overexpectation effect, then from the results of Experiments 1 and 2, we might expect CeA lesions, but not BLA lesions, to eliminate that effect. However, instead we found this loss to be substantial and equivalent in sham-lesioned and CeA-lesioned rats, but absent in BLA-lesioned rats.

Figure 8.

Mean ± s.em. test responding to the previously-reinforced auditory conditioned stimuli (CSs) in Experiment 3. BLK = CS that was presented in compound with the lever in the autoshaping phase; CON = CS that was not presented during the autoshaping phase. Responding to the BLK cue was significantly lower than responding to the CON cue in both sham-lesioned rats and rats with lesions of the amygdala central nucleus (CeA) but not in the rats with lesions of the basolateral amygdala (BLA).

A preliminary ANOVA showed that Experiment 1 and Experiment 2 history variables (group treatment and compound) did not significantly affect responding in this final test, so they were not included in the analysis of test responding. A lesion X cue treatment (BLK vs. CON) X identity of the BLK cue (clicker, tone, or noise) ANOVA showed a significant main effect of cue treatment, F_(1,49) = 32.44, p < .001, n² = .398, and a significant Lesion X Cue Treatment interaction, F_(2,49) = 4.55, p = .015, n² = .157. The cue treatment effect was significant for sham-lesioned (p = .001) and CeA-lesioned (p < .001) rats, but not BLA-lesioned rats (p = .969). Importantly, responding to these CSs did not differ prior to the autoshaping training: During the reminder session (r2 in Figure 2), responding to the BLK vs. CON CSs averaged 81.7 ± 4.5% vs. 82.1 ± 4.5%, respectively, in sham-lesioned rats, 84.4 ± 6.4% vs. 86.3 ± 4.2% in CeA- lesioned rats, and 80.9 ± 10.0% vs. 83.3 ± 8.%% in the BLA-lesioned rats.

In the final test ANOVA, the Cue Treatment X Cue Identity interaction was significant, F_(2,49) = 6.71, p = .003, n² = .215. Overall, the cue treatment effect was smaller with the clicker (21.7 ± 3.3% vs. 30.7 ± 4.5%) than with the noise (18.3 ± 4.6% vs. 36.3 ± 4.6%) or tone (17.1 ± 3.5% vs. 29.0 ± 5.1%) cues. Importantly, cue identity did not affect the lesion effect: the Lesion X Cue Treatment X Cue Identity interaction was not significant, F_(4,49) = 1.84, p = .136, n² = .131.

The observation that BLA lesions prevented the “vampire” effect is consistent with the view that it is derived from some special incentive property established to levers in sign-tracking. If so, the failure to observe that effect might be especially obvious in goal-trackers (which were over-represented in the BLA-lesioned rats). Among the 5 BLA-lesioned goal-trackers, that appeared to be the case: test responding to the BLK cue was 54.1 ± 13.3% and responding to the CON cue was 32.6 ± 4.3%, and responding to the BLK cue was significantly higher in those rats than in the BLA-lesioned sign-trackers (21.6 ± 3.1%), t(16) = 3.502, p = .003. However, this pattern was found only among the BLA-lesioned rats: the two goal-trackers in the CeA- and sham-lesioned groups both showed less responding to the BLK cue (25.3% and 23.2%, respectively) than to the CON cue (32.4% and 37.4%). Furthermore, removing the goal-trackers from the ANOVA of responding to the auditory cues had only a small effect: The Lesion X Cue interaction was again significant, F_(2,39) = 3.70, p = .034, n² = .160, and individual comparisons again showed significantly less responding to the BLK cue than to the CON cue in both sham-lesioned (17.2 ± 1.7% vs. 31.1 ± 2.7%, p = .001) and CeA-lesioned (21.6 ± 2.1% vs. 36.6 ± 4.5%, p < .001) sign-trackers, but not in BLA-lesioned sign-trackers (26.6 ± 4.5% vs. 31.0 ± 4.2%, p = .766). Thus, the vampire effect was not seen in BLA-lesioned rats, even if they were sign-trackers.

Final lever-alone test

The results of the final test of responding to the lever alone (Table 4) shed additional light on the effects of the BLA lesions. Consider first food cup responding on lever presentation trials (right two columns of Table 4). There was little evidence that the rats initially identified as goal-trackers in the acquisition phase had acquired a food cup response to the lever. Instead, it is likely that the high level of food cup responding observed in those rats to lever + auditory cue presentations in the acquisition phase reflected maintenance of responding to the auditory (BLK) cue, as was observed in response to that cue alone in the final tests. By contrast, the 2 sham-lesioned and CeA-lesioned rats identified as goal-trackers both displayed substantial amounts of food-cup responding in response to lever presentation in test. Nevertheless, a lesion X auditory cue identity ANOVA of food cup responding during lever presentations among all rats showed no effect of lesion, F_(2,49) = 1.44, p = .247, n² = .055. Furthermore, among the BLA-lesioned rats, sign-trackers and putative goal-trackers did not differ in their food-cup responding to presentations of the lever alone, t(16) = 1.00, p = .335. Thus, BLA-lesioned “goal-trackers” are probably better described as rats that failed to acquire much conditioning to lever insertion.

Table 4.

Mean (± s.e.m.) responding during the lever-alone test in Experiment 3.

rats	n	lever-press rate	lever-press probability	food-cup % time pre	food-cup % time dur
all sham	20	33.5 ± 2.9	88.1 ± 5.0	0.8 ± 0.3	4.2 ± 1.7
all CeA	20	33.2 ± 2.7	91.9 ± 4.6	1.5 ± 0.5	5.7 ± 1.5
all BLA	18	22.1 ± 2.9	73.6 ± 6.3	2.3 ± 1.4	6.0 ± 1.8
GT sham	1	0.0	0.0	2.6	30.4
GT CeA	1	0.4	12.5	9.6	28.7
GT BLA	5	8.3 ± 3.1	35.0 ± 8.3	0.0 ± 0.0	8.8 ± 4.0
ST BLA	13	27.5 ± 2.7	88.4 ± 3.0	3.2 ± 1.9	4.9 ± 1.9

The left two data columns show lever-presses/min and the percentage of trials with a lever-press, respectively. The right two data columns show the percentage of time spent in the food cup in the 10-s periods prior to (pre) and during (dur) lever presentations. CeA = rats with lesions of the amygdala central nucleus; BLA = rats with lesions of the basolateral amygdala; GT = goal-trackers; ST = sign-trackers.

Next, consider lever-press responding in this final test segment. The rate of lever-pressing to the lever insertion cue alone (first data column of Table 4) was reduced in rats with BLA lesions, as was observed to the lever + auditory cue in the acquisition phase. A lesion X auditory cue identity ANOVA showed a significant effect of lesion, F_(2,49) = 4.42, p = .017, n² = .153, and individual contrasts showed that responding of the BLA-lesioned rats was significantly lower than responding in either the sham-lesioned (p = .026) or CeA-lesioned (p = .031) rats, which did not differ (p = .996). Similarly, ANOVA of lever press probability (second data column of Table 4) showed a significant main effect of lesion, F_(2,49) = 3.15, p = .050, n² = .124, with significantly lower responding in BLA-lesioned rats than in CeA-lesioned rats (p = .050), but not sham-lesioned rats (p = .157). Furthermore, rats that failed to show lever-press responding in the initial acquisition phase also failed to do so in this test of the lever alone, indicating that the former failure was not due simply to competition between lever pressing and food cup responding controlled by the auditory cue during acquisition of autoshaping. Among BLA-lesioned rats, the 13 rats identified as sign-trackers showed significantly higher rates and probabilities of lever pressing ts(16) = 4.05, 7.72, respectively, ps < .001, than the other 5 rats.

General Discussion

The results of Experiments 1 and 2 support the major conclusions of Haney et al. (2010), using very different training and control procedures, and permanent lesions rather than transient pharmacological manipulations of amygdala function. Rats with sham lesions and rats with BLA lesions both showed losses of conditioned responding to each of two individual food-paired CSs after those two CSs were compounded and paired with food (“overexpectation”; Rescorla, 1999). No such losses occurred in rats with CeA lesions.

One account for these results is that rats with lesions of CeA (but not those with lesions of BLA) are impaired in their ability to compute negative PEs under these conditions, and are thus unable to use those PEs to acquire inhibitory learning about the cues. Another account is that CeA-lesioned (but not BLA-lesioned) rats are not impaired in computing negative PEs or using those PEs to establish inhibitory learning, but instead are impaired in their ability to use negative PEs to enhance the associability of the two CSs. Thus, compared to sham-lesioned or BLA-lesioned rats, which could show rapid inhibitory learning to those CSs (and hence losses in conditioned responding) in the compound overexpectation phase, CeA-lesioned rats would be more slow to acquire that learning.

The results of Experiment 2 supported this second account. In Experiment 2, a novel cue was added when the two CSs were combined and paired with reinforcement. According to the Rescorla-Wagner (1972) model, the induction of a negative PE should generate inhibitory learning not only about the two original CSs, but also this novel cue. Thus, despite its consistent pairing with food, it should become a net conditioned inhibitor. Both summation and retardation tests identified this cue as a conditioned inhibitor in Experiment 2. Furthermore, the acquisition of that inhibition was unaffected by either CeA or BLA lesions. Thus, the lesions did not impair the rats’ ability to compute negative PEs or to use those PEs to generate inhibitory learning, as specified by the Rescorla-Wagner (1972) model. At the same time, CeA-lesioned rats were impaired in their acquisition of inhibitory learning about the original CSs (as expressed in reduced responding, that is, the overexpectation effect). This pattern of data is consistent with the possibility that overexpectation normally involves CeA-dependent cue associability enhancements, combined with Holland’s (2012) claim that only cues that contribute to a negative PE show CeA-dependent associability enhancements in response to that PE: although both of the pretained cues contributed to the overexpectation of food on compound trials, the novel cue did not.

This account demands different rules for assignment of credit for changes in associative strength and for changes in cue asscociability: changes in strength are distributed among all cues present whereas changes in associability are distributed only among cues that contributed to the PE. Holland (2012) presented data in support of the latter assertion. Although many procedures that induce negative PEs produce CeA-dependent associability enhancements, the conditioned inhibition (A→US, XA→nothing) procedure appeared not to. According to the Pearce-Hall (1980) model, under most circumstances, the large negative PE on early XA trials should enhance or maintain the associability of both A and X, contributing to rapid learning of inhibitory X→US (or X→“no US”) associations, which underlie solution of the discrimination. However, across a range of cue modalities, saliences, temporal arrangements, and amounts of training (Holland et al., 2000; Holland, 2012; Holland & Kenmuir, 2005) we never observed an effect of CeA lesions on the acquisition of these discriminations. Importantly, in the A→US, XA→nothing procedure, X is never reinforced and so does not itself generate a prediction of the US. If associability increments are assigned only to cues that contributed to the PE, rather than distributed to all cues present, as the Pearce-Hall (1980) model assumed, we would expect A→US, AX→nothing procedures to enhance A’s associability but leave X’s unchanged. Holland (2012) found that to be the case, and A’s enhanced associability was eliminated by CeA lesions.

Holland’s (2012) data are also consistent with an earlier modification to the Pearce-Hall (1980) model (Pearce & Mackintosh, 2010), in which cue associability is determined by individual PE (α_A ≈ |λ-V_A|), rather than the aggregate error. Early in conditioned inhibition training, the individual PE for A is large and negative, whereas that for X is zero. This modification is mathematically more elegant than our imprecise assertion that changes in associability are distributed only among cues that contributed to the PE. Unfortunately, although assuming that cue associability is linked to individual PE accounts for Holland’s (2012) conditioned inhibition results, it is inconsistent with the associability enhancements observed in the A→food, C→food | A + C→ food overexpectation procedure. In that procedure, although both A and C contribute to the negative aggregate PE, the individual PE for each of A and C is zero, and thus their associabilities should remain low. In that case, the effects of CeA dysfunction on overexpectation observed by Haney et al. (2010) and in the present Experiments 1 and 2 would not be anticipated. Furthermore, because in Experiment 2, the added cue X had a substantial individual PE, the Pearce and Mackintosh (2010) view would predict that X’s associability would be enhanced or maintained initially, and hence might be especially sensitive to CeA lesion effects, the opposite of what we observed. Allowing both individual and aggregate errors to contribute to associability changes in a multiplicative manner, that is, α_{A ≈} |(λ – V_A)(λ – V_agg)|, would fare no better, for similar reasons. However, some of the results of the present experiments could be generated from a model in which individual PE and aggregate PE interacted additively in determining associability, α_{A ≈} |(λ – V_A)| + |(λ – V_agg)|, if it were also assumed that only associability enhancements generated by negative PEs are CeA-dependent (Holland, 2005). Within such a model, the basic overexpectation phenomenon (losses in responding to A and C in Experiments 1 and 2) would be impaired by CeA lesions, because lesioned rats would lack the normal contribution of the (negative) aggregate PE to associability (as in the original Pearce-Hall, 1980, model). Furthermore, in Experiment 2, impairments in inhibitory learning would be smaller for stimulus X than for A and C, because the contribution of the (positive) individual PE to X’s associability would be unaffected by the CeA lesion. Nevertheless, one should still expect to see some deleterious effect of the lesion on learning about X in that experiment, reflecting the CeA-dependent aggregate PE’s contribution to associability, which we did not. Thus, we favor our less elegant assignment of credit rule.

Of course, these conclusions may rest more on assumptions about the nature of CeA’s roles in associability changes than on assumptions about rules for associability changes themselves. For example, the effects of CeA lesions on responding in the overexpectation procedure may be based on CeA involvement in some process other than associability enhancement. Although the failure of CeA lesions to disrupt inhibitory learning about the added V cue in Experiment 2 limits the range of what those processes might be, we have no independent evidence, beyond the lesion effects, that overexpectation procedures in fact enhance the associability of the pretrained A and C cues more than they enhance the associability of the added V cue. Furthermore, it must be recognized that alternate assignment of credit rules for associative strength have also been proposed (e.g., Rescorla, 2000, 2001); it is possible that CeA lesions might alter assignment of credit in the distribution of strength changes. However, such models would be unable to account for examples of CeA-dependent enhancement of excitatory learning after the induction of negative PEs, such as those described in the Introduction, and later in this Discussion. Thus, it seems more parsimonious to adopt our assignment of credit rule and attribute the observed CeA lesion effects on overexpectation to impairments in associability enhancements that normally contribute to learning in overexpectation procedures.

Experiment 3 examined the potential contribution of CeA-dependent overexpectation effects to the “vampire” effect noted in autoshaped lever-pressing by Holland et al. (2014). In Experiment 3, as in Holland et al.’s (2014) study, repeated reinforced presentations of a lever-insertion CS in compound with a previously-reinforced auditory cue not only failed to reveal blocking of conditioning to the lever, but also resulted in the loss of conditioned responding to that auditory cue relative to responding to another previously-reinforced auditory cue. Holland et al. (2014) suggested that this loss might reflect an overexpectation effect: if lever-insertion independently acquired conditioning, the ensuing negative PE might result in both decreases in conditioned responding to, and increases in associability of, the auditory cue within the auditory + lever compound. If the associability increases were CeA-dependent (as in Experiments 1 and 2), then CeA (but not BLA) lesions might be expected to reduce the losses in conditioned responding to the auditory cue. However, in Experiment 3, CeA lesions had no effect, on either this vampire effect or, consistent with the findings of Chang et al. (2012a), the acquisition of conditioned responding to the lever. Notably, the same auditory cue successfully blocked conditioning to another auditory cue in Experiments 1 and 2 in the CTL (BC→food or VBC→food) condition, and there was no evidence in those experiments that the presence of the second (B) cue reduced responding to the C cue, relative to responding to the A cue. By contrast, BLA lesions both impaired the acquisition of lever-pressing (confirming Chang et al.’s, 2012b, findings) and prevented the vampire effect: training of a lever + auditory cue compound did not reduce conditioned responding to that auditory cue relative to responding evoked by another previously-trained auditory cue.

Taken together, the results of Experiment 3 indicate that the vampire effect does not require CeA-dependent associability enhancements like those critical to overexpectation effects in Experiments 1 and 2, but instead demands a contribution of BLA to some special feature of autoshaped lever-pressing, as advocated by Flagel et al. (2011) and other investigators. For example, Chang et al. (2012b) suggested that although BLA was not critical to the conditioning of incentive salience to food-paired levers, it appeared to amplify that property. Perhaps that amplification is responsible for levers’ resistance to blocking and ability to draw value away from other contemporaneously-presented stimuli.

Given the well-established roles for BLA in coding reinforcer value information (e.g., Esber & Holland, 2014; Pickens et al., 2003; Roesch et al., 2010) and an unsigned PE like that specified by the Pearce-Hall (1980) model and thought responsible for associability enhancements (Esber et al., 2012; Roesch et al., 2010), it is surprising that rats with BLA lesions were unimpaired in the display of overexpectation effects in Experiments 1 and 2. One possibility, mentioned earlier in another context, is that these effects do not involve associability enhancements, but some other CeA-dependent conditioning function. However, it is notable that other experiments also suggest a more limited role for BLA than for CeA in associability enhancements. Although CeA is critical to associability enhancements that occur with negative PEs in overexpectation, the serial prediction task (Holland & Gallagher, 1993b; Holland, Hatfield, & Gallagher, 2001), serial negative patterning (Holland et al., 2000), and unblocking with downshifts in reinforcer number (Esber & Holland, 2014; Holland & Gallagher, 1993b), normal BLA function is needed only for the last of these. Esber and Holland (2014) suggested that BLA may only be critical for associability enhancements when different reinforcer values are compared. As in the serial prediction and negative patterning tasks, in the overexpectation procedure used here (and that used by Haney et al., 2010), each of the cues that contributed to the overexpectation was paired with the same reinforcer. It is possible that PE is processed in many brain regions in parallel, and that other regions, such as CeA, are sufficient to enhance cue associabilities when comparison of more detailed representations of reinforcer values is unnecessary. It would be interesting to reexamine the effects of BLA and CeA lesions in overexpectation procedures in which the compounded CSs are initially paired with different USs (Rescorla, 1999). Finally, although Esber and Holland (2014) suggested that CeA’s role might be limited to circumstances in which expected events are completely omitted, the results of Experiments 1 and 2 (and Haney et al.’s, 2010, observations) extend that domain to situations in which negative PE is induced when a single US is overexpected on the basis of multiple predictions.

Gallagher and Holland (1992) proposed a simple circuit in which CeA-processed surprise information was propagated to the cholinergic basal forebrain, which in turn broadly modulated cortical activity. More recent evidence suggests a broader network. Holland and Schiffino (2016) described surprise, storage, and expression modules involved in PE-induced associability enhancements. The surprise module, in which PEs are coded for use in altering cue associability, includes CeA, BLA, posterior parietal cortex (PPC), substantia nigra pars compacta and reticularis, and perhaps portions of medial pre-frontal cortex, although the role of some of these structures may be task-specific. Currently, only the PPC has been identified as a potential storage site for altered associability values (Schiffino, Zhou, & Holland, 2014; Schiffino, Zhou, Trageser, & Holland, 2014). Finally, we have identified several regions that are critical to the expression of enhanced associability in faster learning but not to the original encoding of altered associability, including substantia innominata/nucleus basalis magnocellularis, the secondary visual cortex, the dorsolateral striatum, and the lateral hypothalamus. Circuitry by which recalculated associability values are conveyed to PPC for storage and by which these altered associability memories are transmitted to components of the expression module remains a subject of speculation (Schiffino, 2015; Schiffino, Zhou, & Holland, 2014).