Variations in Unconditioned Stimulus Processing in Unblocking

Abstract

Three experiments examined the mechanisms by which downward shifts in reinforcer value influence learning in appetitive unblocking procedures. The downward shift was accomplished by omitting the second of a two-reinforcer sequence (food-food or food-sucrose). Performance of normal rats was compared with that of rats with lesions of the central nucleus of the amygdala, which are thought to interfere with surprise-induced enhancements of event processing. The results suggested that in normal rats, omission of the second reinforcer enhanced processing of the first reinforcer, rather than processing of the conditioned stimuli, and that lesions of the central nucleus eliminated this enhancement. The roles of reinforcement error signals in conditioning were discussed.

Animals are faced with a broad array of stimuli, but often learn selectively about a small subset of that array. Specification of mechanisms by which this stimulus selection occurs has been a perennial concern for learning theorists. Although within the study of human information processing the general problem of stimulus selection is typically studied in the context of relatively extensive arrays of stimuli, associative learning theorists have tended to concentrate on simple experimental models, often examining stimulus selection within nominally two-element compounds.

The most widely used procedure for studying stimulus selection within this tradition has been the blocking procedure (Kamin, 1969). In a basic blocking study, animals receive pairings of a compound conditioned stimulus (CS) with an unconditioned stimulus (US), for example, light + noise → food. Prior to this compound training, animals in a Blocking treatment receive pairings of one of the stimulus elements, e.g., the light, with the same US, whereas animals in a Control treatment do not. Prior conditioning of the light blocks conditioning to the noise: a test of responding to the noise alone reveals considerably more conditioned responding (CRs) after the Control treatment than after the Blocking treatment, despite identical conditioning experience with the noise in both treatments. Thus, the likelihood of selecting the noise for new learning in the compound phase depends on prior experience with the light.

Two broad classes of learning theories have reconciled blocking with simple contiguity learning views by reformulating the idea of contiguity to apply to mental events instigated by CSs and USs, rather than to the events themselves. One set of theories emphasizes the role of past learning in modulating the effectiveness of USs, whereas another set focuses on changes in the ability of CSs to participate in associative learning. For example, within the Rescorla-Wagner model (Rescorla & Wagner, 1972), the effective reinforcement value of a US on any learning episode is derived from a reinforcement error signal: the discrepancy between the maximum value of that US and the current associative strength of all stimuli present on that episode. Thus, when the compound stimulus is introduced in a blocking experiment, the added (noise) element is paired with a relatively ineffective reinforcer, because the US is already anticipated on the basis of the prior training of the other (light) element.

By contrast, other theorists have suggested that prior training of one cue can modify the effectiveness of CSs in blocking studies (see LePelley, 2004, for a recent review). For example, within the Pearce-Hall model (Pearce & Hall, 1980), a reinforcement error signal affects the ability of a CS to enter into new associations (its learning rate parameter, ∀, often termed “associability”). When the absolute magnitude of the error signal is large, the CS is highly associable, whereas if that signal is small, as when the reinforcer is already well-predicted, the associability of the CS is low. Thus, when the compound stimulus is introduced in a blocking experiment after extensive training of one of its elements, the associability of the added element will decrease rapidly, minimizing the extent to which that element can be associated with the US.

Although there is substantial independent evidence that experience can alter processing of both CSs and USs (see Dickinson & Mackintosh, 1974; Rescorla & Holland, 1982; Wasserman & Miller, 1997, for reviews), variations on the blocking procedure have provided a major test arena for these learning theories. Especially informative are those procedures for which each approach has apparently unambiguous, but contradictory predictions. One such example is the case of “unblocking” when the value of the US is shifted downward when the compound stimulus is introduced (Dickinson, Hall, & Mackintosh, 1976). For example, in one experiment from our laboratory (Holland & Gallagher, 1993b), rats first received pairings of a visual cue with a two-US sequence, e.g., a food pellet followed 5 sec later by two more pellets. Then, an auditory CS was added to the visual CS, and the compound paired with the single pellet US alone, omitting the subsequent pellets. Within the framework of the Rescorla-Wagner (1972) model, because of the prior training of the visual CS with a higher-valued US, this procedure results in an overexpectation of US value when the auditory CS is added. This overexpectation, a negative error signal, should generate inhibitory learning about the added auditory CS. By contrast, within the Pearce-Hall (1980) model, any discrepancy between expected and obtained US values maintains or enhances CS processing, thus enabling the formation of excitatory associations between the auditory CS and the single food pellet US.

Holland and Gallagher (1993b) found that rats acquired substantial excitatory learning about the added auditory CS, supporting models like Pearce and Hall’s (1980). Although some investigations using downshift procedures have revealed inhibitory learning (e.g., Cotton, Goodall, & Mackintosh, 1982; Holland, 1988, Wagner, Mazur, Donegan, & Pfautz, 1980), the frequent observation of substantial excitatory learning (e.g., Bucci, Holland, & Gallagher, 1998; Dickinson, et al., 1976; Dickinson & Mackintosh, 1979; Holland, 1984, 1985, 1988; Holland & Gallagher, 1993b) has been a major source of support for the claim that the disconfirmation of reinforcement expectancies can enhance processing of CSs. Indeed, as we discuss later in this article, we have used excitatory learning in unblocking as a tool for investigating brain mechanisms involved in attentional processes such as those specified in the Pearce-Hall (1980) model (Bucci, et al., 1998; Holland & Gallagher, 1993b, 1999).

However, a simple alternative account for the occurrence of excitatory learning in unblocking downshift procedures is that the surprising omission of the second US (the 2 pellets in Holland and Gallagher’s study) enhances the reinforcing power of the first US (Holland, 1988; Kamin, 1969). Within this view, the occurrence of excitatory learning reflects enhanced processing of the US, rather than of the CS. Casually speaking, if surprise can enhance the effectiveness of a CS, why not the effectiveness of a US? The experiments reported here considered the roles of alterations in CS and US processing in unblocking. In Experiments 1 and 2 we used standard unblocking procedures, extending previous findings of Holland and Gallagher (1993b) and Holland (1988). In Experiment 3 we used procedures derived from those of unblocking, but which permitted the examination of alterations in US processing unconfounded with changes in CS processing.

In all 3 experiments, we also examined the effects of lesions of the central nucleus of the amygdala (CN) on the behavioral consequences of unblocking procedures. Several studies conducted in this laboratory (see Holland & Gallagher, 1999, for a review) suggest that the surprise-induced enhancements of CS associability described by Pearce and Hall (1980) depend on the integrity of CN function. Although experimental lesions of the CN have little or no impact on many aspects of appetitive conditioning, they eliminate a number of phenomena often attributed to these associability enhancements. For example, excitotoxic lesions of CN (Holland & Gallagher, 1993a) eliminated the enhanced CS associability typically observed when a consistent predictive relation between two CSs is shifted to a less consistent relation (Wilson, Boumphrey, & Pearce, 1992). Normal rats showed enhanced rates of both excitatory (Holland & Gallagher, 1993a) and inhibitory (Holland, Chik, & Zhang, 2001) learning about a visual stimulus that had been made an unreliable predictor of a tone CS, relative to learning about that same cue when it had been a reliable predictor of the tone. By contrast, lesioned rats showed no such enhancements, but otherwise performed similarly to the normal rats. Most relevant to the research reported here, in the unblocking experiment described earlier, Holland and Gallagher (1993b) found that, unlike sham-lesioned control rats, rats with CN lesions failed to show excitatory learning about the added cue. Holland and Gallagher (1993b) claimed that the CN lesion interfered with the enhancement of CS associability that is normally induced by reductions in the anticipated US value, thus preventing the observation of unblocking.

Experiment 1

In Experiment 1, we examined the performance of rats with neurotoxic lesions of the amygdala CN and rats with sham lesions in unblocking procedures like those used by Holland and Gallagher (1993b). However, we examined both the acquisition of excitation to the added auditory CS during the compound training phase and the acquisition of conditioned inhibition to that auditory CS in a subsequent inhibitory conditioning phase.

Rats in the unblocking condition (Group Down) first received training in which a visual CS was paired with a sequence of a single pellet (food1) followed 5 s later by two similar pellets (food2). In a second phase they received pairings of a compound of that visual CS and a noise, paired with delivery of food1 only. Control rats were trained with a blocking procedure (Group Low) in which food1 was the reinforcer in both the initial element training and the compound conditioning phases. Excitatory learning about the noise alone was examined in test sessions administered during and just after the compound training phase. Subsequent inhibitory learning about the noise was assessed by examining the rate of acquisition of a feature negative discrimination at the conclusion of the experiment. In that discrimination, presentations of a second visual CS were paired with the immediate delivery of food2, and presentations of a compound of that new visual cue and the noise were nonreinforced.

If the omission of food2 enhanced the associability of the noise CS, then any learning about that CS should be enhanced. Thus, in sham-lesioned rats, both excitatory learning during compound conditioning and subsequent inhibitory learning about the noise should be enhanced in Group Down, relative to rats in Group Low. Furthermore, if CN lesions interfere with this enhancement of CS associability, then lesioned rats in Group Down should fail to show facilitation of either excitatory or inhibitory learning. It is notable that this technique of showing similar changes in the rate of both excitatory and inhibitory learning after some experimental treatment has been frequently used to permit inferences about changes in the associability of a CS (e.g., Reiss & Wagner, 1972; Rescorla, 1971; Swan & Pearce, 1986; Wilson, Boumphrey & Pearce, 1992), including the effects of CN lesions (Holland, et al, 2001; Holland, Thornton, & Ciali, 2000).

By contrast, if omission of food2 enhanced processing of food1, but left the associability of the noise unchanged, the effects on excitatory and inhibitory learning might differ. The establishment of excitatory noise-food1 associations would still be facilitated (that is, unblocking would be observed) in sham-lesioned rats. Although the effects of CN lesions on enhanced US processing of this sort have not been investigated independently, to account for our previous observations of lesion deficits in unblocking (Holland & Gallagher’s, 1993b), from this perspective, CN lesions must be assumed to interfere with such US processing enhancements. By contrast, enhanced processing of food1 in compound training would be unlikely to increase the rate of subsequent inhibitory noise-food2 learning in either sham- or CN-lesioned rats, and so no lesion effects would be anticipated in the final, inhibitory learning test.

Methods

Subjects

The subjects were 40 male Long-Evans rats, obtained from Charles River Laboratories, Inc. (Raleigh, NC) and maintained in a Duke University Department of Psychological and Brain Sciences facility. They were maintained at 85% of their ad-lib body weights by measured feedings at the end of each session. Water was available at all times in their individual home cages. All subjects were experimentally naive.

Surgical procedures

All surgery was performed under Nembutal (50 mg/kg) anesthesia with aseptic conditions. Twenty-four rats received bilateral lesions of the CN, using stereotaxic coordinates 2.3 mm posterior to bregma and 4.2 mm from the midline, with infusions at a depth of 7.9 mm from the skull surface. The CN lesions were made using 0.25 μl of 10 μg/μl ibotenic acid (Sigma, St. Louis, MO) in a phosphate-buffered saline (PBS) solution, infused with a Hamilton 2.0 μl syringe over a 2-min period. Sixteen sham lesion rats received injections of the PBS vehicle alone in a comparable manner.

Histological procedures

After completion of behavioral testing, the rats were deeply anaesthetized with Nembutal (150 mg/kg), and perfused with 0.1-M phosphate-buffered saline (PBS), followed by 10% (v/v) formalin. The brains were removed and stored in 0.1-M PBS with 20% (w/v) sucrose and 1% (w/v) DMSO at 4 C° for 24–48 hr. Sections (60-μm) were taken from each brain, and alternate sections were mounted on slides and Nissl-stained to verify lesions.

Apparatus

The behavioral training apparatus consisted of eight individual chambers (22.9 × 20.3 × 20.3 cm) with aluminum front and back walls, clear acrylic sides and top, and floor made of 0.48-cm stainless steel rods spaced 1.9 cm apart. A dimly illuminated food cup was recessed in the center of one end wall and an identical cup for sucrose was recessed in the center of the opposite end wall; this cup was occluded by a stainless steel plate throughout Experiment 1. Infrared photocells placed just inside the cups were polled (1 kHz) by computer circuitry. Each chamber was enclosed in a sound-resistant shell. A 6-w lamp, which served as the source of one visual CS (house light), was mounted on the inside wall of the shell, 10 cm above the experimental chamber and even with the end wall opposite the food cup. A second lamp was mounted behind a jeweled lens on the front panel (panel light), 10 cm above the food cup. Ventilation fans provided masking noise (70 dB). Constant dim illumination was provided by a 6-w lamp behind a dense red lens mounted on the ceiling of the shell. A TV camera was mounted within each shell to provide a view of the chamber; the output from each camera was digitized, merged into a single image of all four chambers, displayed on a monitor, and recorded on videotape. Data from videotapes are not presented in this article.

Procedure

The rats were first trained to consume the food pellets to be used as unconditioned stimuli. In a single 64-min session, there were 16 deliveries of a 45-mg food pellet (P.J. Noyes, Lancaster, NH; now Research Diets, New Brunswick, NJ) to the food cup.

Table 1 shows an outline of the procedures of Experiment 1. After food cup training, the rats were divided into two groups, and Phase 1 training of the panel light CS (V1) was started. In each of 12 64-min sessions, the rats in Group Down (12 lesioned and 8 sham-lesioned) received 8 10-s presentations of V1 paired with a single pellet (food1), followed 5 s later by two additional pellets (food2), whereas the rats in Group Low (12 lesioned and 8 sham-lesioned) received pairings of V1 with the delivery of a single pellet only. In each of the eight Phase 2 compound training sessions, the rats in each group received eight 10-s presentations of a compound of V1 and a 78-db white noise, reinforced with the delivery of food1. Thus, rats in Group Down received the added noise CS on trials on which the reinforcer was shifted from the food1-food2 sequence down to food1 alone, whereas the rats in Group Low were exposed to the noise on trials on which the value of the reinforcer was maintained the same as in the previous training phase.

Table 1.

Outline of Procedures of Experiments 1–2

Experiment 1
Group	Phase 1	Phase 2	Test	Inhibitory Learning Test
Down	V1→food1→food2	V1N→food1	N	V2→food2, V2N→nothing
Low	V1→food1	V1N→food1	N	V2→food2, V2N→nothing
Experiment 2
Group	Phase 1	Phase 2	Test	Inhibitory Learning Test
Down	V1→food→suc	V1N→food	N
	V2→suc	V2→suc	N	V2→suc, V2N→nothing
Low	V1→food	V1N→food	N
	V2→suc	V2→suc	N	V2→suc, V2N→nothing
High	V1→food→suc	V1N→food→suc	N
	V2→suc	V2→suc	N	V2→suc, V2N→nothing

Note. N = noise, V1 and V2 = panel light and house light visual stimuli, respectively, in Experiment 1; counterbalanced in Experiment 2, suc = sucrose, → = followed by.

Excitatory conditioning to the added noise CS was assessed in two 32-min probe tests, one administered between the fourth and fifth compound training session and one after the eighth compound session. Each of these tests included four nonreinforced presentations of the noise CS alone. Finally, the rate of acquisition of conditioned inhibition to the noise was assessed in 16 test sessions. First, in each of the first four 64-min sessions, all rats received eight 10-s presentations of the house light (V2), paired with the immediate delivery of food2. These sessions were designed to establish conditioning to V2 cue, which was used as the reinforced cue in the inhibitory learning test. In the remaining 12 64-min sessions, there were 2 10-s presentations of V2, reinforced with food2, and six nonreinforced 10-s presentations of a noise + V2 compound.

Response measures

The measure of conditioning was the time the photobeams for the food cups were broken (presumably indicating the presence of the rat’s head in the food cup), expressed as a percentage of the 5-s sampling interval. Because these responses occur predominantly during the few seconds immediately before delivery of the reinforcer (Holland, 1977), we examined these behaviors during the last 5s of the 10-s CS-US intervals, as in previous studies (e.g. Holland & Gallagher, 1993b). To reduce within-group variance, we reported the elevation of these behaviors over the baseline response levels during a comparable 5-s interval immediately before each trial (% time during CS minus % time pre-CS). Baseline response levels did not differ significantly between groups in any of these studies, justifying the use of elevation scores. Finally, in the inhibitory learning test, as a measure of conditioned inhibition learning, we presented a simple index of discrimination performance, the difference (in % time) between the elevation scores for the reinforced and nonreinforced CSs.

Data analysis

The data analyses were comparable in all studies presented here. First, groups (Down or Low) X lesion (CN or sham) ANOVAs of the pre-CS data from each phase were conducted to verify comparability of the baselines across training and lesion conditions. Next, groups X lesion X session ANOVAs were conducted on the elevation scores. Planned comparisons used simple F-tests (LSD procedure) and post-hoc comparisons used the Tukey HSD procedure. The level of statistical significance adopted throughout these studies was p < .05.

Results

Histological results

Seventeen brains were judged as having acceptable lesions, 9 in Group Low and 8 in Group Down. Lesions were rejected (n = 7) if there was less than 30% damage to the CN on either side, or if there was more than minimal damage to adjoining regions. The damage was substantial in the medial portions of the CN in all rats, and extended to the lateral portions of CN in ten rats. Six brains showed significant damage to the basolateral amygdala, but in both cases that damage was unilateral. Figure 1 shows the extent of the largest and smallest acceptable lesions at various rostral-caudal planes. Except around the injector tracks, no cellular damage was evident in any of the vehicle control brains.

Figure 1.

Extent of smallest (dark shading) and largest (hatching) acceptable lesions. Coronal sections are shown at 4 planes, 1.8, 2.3, 2.8, and 3.3 mm posterior to bregma. Sections are based on Swanson (1994).

Behavioral results

Baseline levels of food cup behavior ranged between 5.7% ± 3.6% and 12.1% ± 5.6% in the four group/lesion subgroups of rats over the course of each phase of this experiment. Separate group (Down or Low) X lesion (CN or sham) ANOVAs for each phase showed no reliable effects or interactions, Fs < 1.

The left side of Figure 2 shows the elevation in food cup responding during V1 in Phase 1. More food cup responding was acquired to V1 when it was paired with the food1-food2 sequence (Group Down) than when it was paired with food1 (Group Low). A group X lesion X session blocks ANOVA showed only a significant effect of blocks, F(5, 140) = 28.78, and a group X sessions interaction, F(5, 140) = 2.72. Over the last two blocks, responding was greater in Group Down than in Group Low, F(1, 28) = 4.71.

Figure 2.

Mean elevation (during CS minus pre-CS) in food cup responding during the element (left side) and compound (right side) training phases of Experiment 1. The error bars indicate two times the overall between-groups MSE for each phase. CN = amygdala central nucleus, CS = conditioned stimulus.

The right side of Figure 2 shows elevation in responding during the compound trials in Phase 2. Although responding was maintained in both sham and CN-lesioned rats in Group Low, which continued to receive the same reinforcer as in the previous phase, the effects of omitting food2 in Group Down differed as a function of the lesion. After an initial loss of responding in both sets of rats, responding recovered in sham-lesioned rats but declined further in CN-lesioned rats. A group X lesion X session blocks ANOVA showed reliable lesion X block, F(3, 84) = 5.90, and group X lesion X block, F(3, 84) = 6.18, interactions. An analysis of the simple effects of lesion in Group Down was significant, F(1, 28) = 34.4, as was a contrast of the linear trend of session blocks between lesioned and sham rats of that group, F(1, 28) = 5.10.

Figure 3 shows food cup responding during the two probe tests (combined) of responding to the noise alone (there were no differences in responding between the two tests). Consistent with the findings of Holland & Gallagher (1993b), sham-lesioned rats exhibited unblocking with a downshift in reinforcer value, but rats with CN lesions did not. Responding of sham-lesioned rats was greater in Group Down than in Group Low, but responding of CN-lesioned rats did not differ between groups. A group X lesion ANOVA of the elevation scores showed a significant interaction, F(1, 28) = 6.28. Multiple comparisons using the Tukey HSD test showed significantly higher response levels in the sham-lesioned rats in Group Down than in the other three sets of rats.

Figure 3.

Mean (± s.e.m.) elevation (during CS minus pre-CS) in food cup responding during the noise-alone test sessions of Experiment 1. CN = amygdala central nucleus, CS = conditioned stimulus.

All rats showed rapid acquisition of responding to V2, the visual CS paired with food2, prior to the inhibitory training test. On the first training session, responding ranged from 30.0 ± 4.8% to 34.1 ± 5.9% across the four groups, and from 64.1 ± 4.7% to 68.9 ± 4.5% on the final session. A group X lesion X sessions ANOVA showed a significant effect of sessions, F(3, 84) = 374.78, but no between-groups effects, Fs(3, 84) < 1.35.

Figure 4 shows the results of the final test phase, in which the noise was trained as an inhibitor in a feature negative discrimination, with food2 as the reinforcer. Figure 4A shows responding on reinforced V2 and nonreinforced V2N compound trials on the first inhibitory training session, which might be viewed as a test of summation of the associative strengths of the noise CS and the food2-paired V2. There was little effect of combining the noise with V2, except in Group Down-CN, which showed an inhibitory effect of the noise. A group X lesion X stimulus ANOVA showed significant group X stimulus, F(1, 28) = 10.12, and three-way interactions, F(1, 28) = 5.90. A contrast of the difference between V2 alone and V2N compound trials in Group Down-CN with that of the other three groups was significant, F(1, 28) = 20.86.

Figure 4.

Panel A: Mean (± s.e.m.) elevation scores for reinforced V2-alone and nonreinforced V2N trials on the first session of the final inhibitory conditioning phase. Panel B: Mean discrimination difference scores (elevation scores for reinforced noise trials minus elevation scores for nonreinforced V2N trials) during the inhibitory savings test sessions of Experiment 1. Panel C: Mean elevation (during CS minus pre-CS) in food cup responding on reinforced (CS+) and nonreinforced (CS-) trials in those sessions. The error bars show two times the overall between-groups MSE. CN = amygdala central nucleus, CS = conditioned stimulus, N = NOise stimulus, V2 = house light stimulus.

Figures 4B and 4C show the acquisition of the final V2→food2, V2N→nothing feature negative test discrimination. Discrimination learning was more rapid in Group Down for both CN- and sham-lesioned rats, which did not differ after session 1. Separate ANOVAs of the discrimination difference scores (Figure 4B) and responding on nonreinforced compound trials alone (Figure 4C) each showed significant effects of blocks, Fs(5, 140) = 121.52 and 46.43, respectively, and groups X blocks interactions, Fs(5, 140) = 4.96 and 2.27, with no effects or interactions with lesion, Fs < 1. Contrasts of the quadratic trends over blocks between Groups Down and Low were significant, Fs(1, 28) = 10.51 and 6.71. ANOVA of responding on reinforced trials (Figure 4C) showed no significant main effects or interactions, Fs < 1, except for a main effect of session blocks, F(5, 140) = 3.04.

Discussion

The performance of the sham-lesioned rats was consistent with the claim that the downshift procedure enhanced the associability of the noise CS: Both excitatory learning during the compound training phase (unblocking) and inhibitory learning in the final test phase were enhanced in Group Down, relative to learning of sham-lesioned rats in Group Low. The performance of CN-lesioned rats, however, failed to support this claim. Although, consistent with the findings of Holland and Gallagher (1993b), lesioned rats in Group Down failed to show unblocking, they showed as great an enhancement in their rate of inhibitory learning in the final test phase (relative to rats in Group Low) as sham-lesioned rats. If CN lesions interfere with surprise-induced enhancements of the associability of the noise, then lesioned rats should be impaired at acquiring both excitatory and inhibitory associations to that noise.

The selective effects of the CN lesions on excitatory, but not inhibitory, learning observed here could be accommodated in several ways. For example, it might be argued that these selective lesion effects were artifacts. The inhibitory learning advantage in Group Down observed in this experiment was small and hence may have been insufficiently sensitive to reveal lesion effects. Alternately, different brain circuitry may be involved in the expression of associability differences in excitatory and inhibitory learning, such that damage to CN affects only the former. However, in other studies of surprise-induced enhancement of CS associability, rats with CN lesions failed to show enhancement of either excitatory or inhibitory learning (Holland et al., 2000, 2001). Notably, inhibitory training procedures similar to those used in the final test of Experiment 1 were sufficiently sensitive in Holland et al.’s (2001) study to reveal effects of CN lesions.

We suggest instead that the key to understanding the results of Experiment 1 is to recognize that the omission of food2 in the compound (blocking/unblocking) training phase may establish inhibitory noise-food2 learning as well as enhance excitatory noise-food1 learning. From this perspective, the greater inhibitory noise-food2 learning in Group Down relative to Group Low in the final test phase of Experiment 1 reflected savings produced by previous learning during the compound phase, rather than a benefit due to enhanced CS associability in the test phase. Given previous evidence that CN lesions do not affect the acquisition of conditioned inhibition in the absence of manipulations thought to enhance CS associability (e.g. Holland et al., 2000, 2001), this account is consistent with both the enhanced inhibitory learning in the test phase in Group Down relative to Group Low, and the lack of a CN lesion effect on this enhancement. If it is further presumed that omission of food2 enhances noise-food1 learning by enhancing processing of food1, the results of Experiment 1 may be simply understood without reference to changes in CS associability.

The plausibility of these claims depends on the assumptions that the added cue in downshift procedures may simultaneously acquire excitatory associations with the remaining US and inhibitory associations with the omitted US, and that omission of the second US may indeed enhance processing of the first US. Experiments 2 and 3 were designed to provide evidence for these two assumptions, respectively. However, previous data provide some support for the first of these claims.

Holland (1988; Exps. 3 and 4) found that in a number of circumstances, downshifts from a two-reinforcer sequence to a single reinforcer simultaneously produced both excitatory and inhibitory learning about the added cue. In particular, in one experiment (Holland, 1988, Exp. 4), relative to control treatments, a downshift from a food1-food2 sequence (identical to that used in Experiment 1) to food1 alone resulted in both greater responding to the added noise when it was tested alone, and slower learning about that cue when it was subsequently paired with food2 in a retardation test of inhibition. In this context, the results of the inhibitory feature-negative discrimination training phase of Experiment 1 could be viewed as a companion summation/savings test of inhibition, which supplements the results of the retardation test provided by Holland (1988, Exp. 4). Thus, although responding evoked by the added noise itself provides evidence for excitatory noise-food1 learning after downshifts, the results of both retardation and summation tests of inhibition (Rescorla, 1969) show that those downshifts also produced inhibitory noise-food2 learning.

It could be argued that the inhibitory learning savings test might be relatively insensitive to the inhibition acquired during compound training. Denniston, Blaisdell, and Miller (2004) found that, just as excitatory CSs code the time of US delivery (e.g. Holland, 2000), conditioned inhibitors code the time of nonreinforcement. If the inhibitory power of a CS is maximal at the time at which the expected US is omitted, then the omission of food2 would establish maximal inhibition 5 sec after CS termination, and maximal excitation just before CS termination. Assessments of responding during the CS itself then would be more likely to yield evidence for excitatory than inhibitory associations. From this perspective, we might have found greater evidence for inhibitory learning about the added noise CS had we extended the CS-US interval in the final inhibitory test phase so that the US was delivered at the same time after CS onset as it was omitted in the compound, unblocking phase. However, previous investigations in our laboratory found substantially broader temporal generalization of inhibition than Denniston et al. (2004) reported. Although, comparable to Denniston et al’s (2004) data, Holland (1988, Experiment 4) found the highest levels of conditioned inhibition when the CS-US interval in a retardation test matched the CS-nonreinforcement interval in the compound training phase, he found significant evidence for inhibition with a 10-sec CS-US test interval (as was used in Experiment 1) after training intervals as long as 30 see.

The notion that the rats in Group Down in both Holland’s (1988) experiment and the present Experiment 1 simultaneously acquired net excitatory noise-food1 associations and net inhibitory noise-food2 associations requires that they distinguished between food1 and food2. Because both reinforcers supported the same CR, it is difficult to assess their associations independently. For example, at least early in the inhibitory training phase, in Group Down-Sham the noise might be expected to enhance food cup entry because of its excitatory associations with food1 at the same time that it inhibits food cup entry because of its inhibitory associations with food2. In this regard it is notable that Group Down-CN, which by our view had equal opportunity for inhibitory noise-food2 learning but did not exhibit excitatory noise-food1 responding, showed significantly greater evidence for inhibition in the first inhibitory training session than the rats in Group Down-Sham. Likewise, the rats in Group Down-CN showed a decline in responding to the noise+V1 compound over the course of Phase 2, consistent with the acquisition of inhibition in that phase. It is likely that comparable levels of inhibition in Group Down-Sham were masked by food cup responding based on noise-food1 learning. In Experiment 2 we attempted to provide a clearer separation of the roles of the first and second USs by using a heterogeneous US sequence, in which the first and second USs supported distinguishable CRs.

Finally, it is notable that the use of extended Phase 2 compound training may have encouraged separation of the effects of the enhancement of processing of food1 in that phase, from processing of food2 in the final inhibitory testing phase. Despite the similarity of food1 and food2, when V2 was initially paired with food2 in the final test phase, there was no evidence of generalization of enhanced processing of food1 to food2 in Group Down-Sham. These rats showed no more rapid acquisition of responding to V2 than the rats in the other groups. However, this equivalent responding might be anticipated even if food1 and food2 generalized substantially. By the end of the extended compound training phase, the new reinforcer (food1 alone) in Groups Down-Shift and Down-CN should have been relatively well-predicted, and hence its processing substantially reduced once again. Thus, by the time V2-food2 training was begun, processing of food1 would have been substantially reduced, and so the effects of generalization between food1 and food2 would be minimal in the test phase. Note that from this perspective, the effects of omitting food2 on both enhanced noise-food1 excitatory learning and inhibitory noise-food2 learning were likely to have occurred mostly in the early sessions of compound conditioning. This possibility is supported by the data; both the increases in responding in Group Down-Sham and the decreases in Group Down-CN had stabilized over the final 4 compound training sessions (right portion of Figure 2), and there were no differences between the levels of responding in the noise-alone test administered after the fourth compound session and those in the test after the eighth session.

Experiment 2

In Experiment 2, we extended our analysis of the basis of unblocking with downshifts in reinforcer value by using a variation of the unblocking procedure in which the reinforcer sequence included two qualitatively different foods, food pellets and liquid sucrose. The use of the heterogeneous sequence provided two advantages over the procedures of Experiment 1. First, it was designed to enhance the amount of inhibitory learning expected to occur in the compound training phase. Holland (1988) found that such heterogeneous sequences favored the development of inhibitory learning about the added US. Second, because the pellets and sucrose supported distinguishable CRs, we could be more precise about the origin of CRs observed to the added noise CS, and the rats’ ability to distinguish between the two USs.

The procedure of Experiment 2 was similar to that of Experiment 1, with three variations. First, the 2-pellet second reinforcer was replaced by the delivery of a sucrose solution delivered to a liquid cup on the opposite side of the chamber from the food cup. Thus, the rats’ choice of food or sucrose cup entry could provide a measure of whether responding was based ultimately on the first (food) or second (sucrose) US. Second, an additional behavioral control group was added. Group High received pairings of the visual (blocking) cue with the food-sucrose reinforcer sequence in both training phases. Finally, the second visual cue, used in the feature-negative assessment of inhibitory learning, was included in all training phases.

Methods

Subjects and apparatus

The subjects were 56 male Long-Evans rats, obtained from Charles River Laboratories Inc. (Raleigh, NC) and maintained as in Experiment 1. The apparatus was the same as that used in Experiment 1, except that the sucrose cup was available.

Surgical and histological procedures

The surgical and histological procedures were identical to those of Experiment 1. Thirty-two rats received bilateral lesions of the CN, and 24 received sham lesions.

Procedure

The rats were first trained to consume the food pellet and liquid sucrose reinforcers used in the experiment, in four 64-min sessions. In each of the first two sessions, there were 16 deliveries of a single 45-mg food pellet to the food cup and in each of the second two sessions there were 16 0.4-ml deliveries of 0.2 M sucrose solution to the sucrose cup.

Table 1 shows an outline of the procedures of Experiment 2. The rats were divided into three groups, and Phase 1 training of the two visual CSs was started. In each of 28 64-min sessions, the rats in each group received four 10-s presentations of one of the visual cues (V1) paired with food delivery, and four 10-s presentations of the other visual CS (V2) paired with sucrose, randomly intermixed. In Group Low, V1 was reinforced with the delivery of one food pellet only, whereas in Groups High and Down, the delivery of that pellet was followed 5 s later by the delivery of 0.4 ml of 0.2 M sucrose. The identities (house light or panel light) of V1 and V2 were counterbalanced. In each of the eight Phase 2 blocking sessions, the rats in each group received four V2-sucrose pairings, as in Phase 1, and four V1N compound trials, randomly intermixed. The V1N compound comprised a 10s 78-db white noise presented simultaneously with the V1 visual cue trained previously. It was reinforced with the delivery of a single food pellet in Groups Down and Low, and by a single pellet followed 5 s later by a 0.4 ml delivery of 0.2 M sucrose in Group High. Thus, rats in Group Down received the added noise CS on trials on which the reinforcer was shifted from a pellet-sucrose sequence down to a pellet-only delivery, whereas the rats in the other two groups were exposed to the noise on trials on which the value of the reinforcer was maintained the same as in the previous training phase.

As in Experiment 1, excitatory conditioning to the added noise CS was assessed in two 32-min probe tests, one administered between the fourth and fifth blocking session and one after the eighth blocking session. Each of these tests included four nonreinforced presentations of the noise CS alone. Finally, the rate of acquisition of conditioned inhibition to the noise was assessed in 12 test sessions. In each of these 64-min sessions there were 2 10-s presentations of V2, reinforced with sucrose, and six nonreinforced 10-s presentations of a V2N compound. As described earlier, we intended this test to provide a measure of how much inhibitory learning was established in the compound conditioning phase.

The response measures were the same as those used for Experiment 1, with the addition of sucrose cup responding, measured in the same manner as food cup responding.

Results

Histological results

Eighteen brains were judged as having acceptable lesions. Lesions were rejected (n = 14) if there was less than 30% damage to the CN on either side, or if there was more than minimal damage to adjoining regions. The damage was substantial in the medial portions of the CN in all rats, and extended to the lateral portions of CN in ten rats. Four brains showed significant damage to the basolateral amygdala, but in both cases that damage was unilateral. Except around the injector tracks, no cellular damage was evident in any of the vehicle control brains.

Behavioral results: Pre-CS responding

Over the course of each phase of Experiment 2, pre-CS levels of food cup behavior ranged between 2.4 ± 0.8% and 7.1 ± 3.1% across the six group/lesion subgroups, and sucrose cup behavior ranged between 2.2 ± 0.9% and 15.3 ± 10.1%. Separate group X lesion ANOVAs for each phase showed no reliable effects of group, Fs(2, 36) < 2.59, lesion, Fs(1, 36) < 1.71, or their interaction, Fs(2, 36) < 1.35, for either response measure.

Phase 1 visual CS training

The left side of each panel of Figure 5 shows food or sucrose cup responding in the visual CS training phase. As in Experiment 1, acquisition of CRs in this phase was unaffected by the lesions. All rats acquired sucrose-cup (Figure 5D), but not food-cup (Figure 5B) responding to V2, which was paired with sucrose alone in all groups. A group X lesion X session blocks ANOVA of sucrose-cup responding showed a reliable effect of session blocks only, F(13, 468) = 84.19, and an ANOVA of food-cup responding showed no significant effects or interactions.

Figure 5.

Mean elevation (during CS minus pre-CS) in food cup (panels A and B) and sucrose cup (panels C and D) responding during the element (blocks 1–4) and compound (blocks b1–b4) training phases of Experiment 2. Responding during V1 (element training) and V1+noise (compound training) trials is shown in panels A and C, and responding during V2 alone (both phases) is shown in panels B and D. 1the The error bars show two times the overall between groups MSE for each phase. CN = amygdala central nucleus, CS = conditioned stimulus, V1 and V2 = counterbalanced visual stimuli (panel light and house light).

Likewise, all rats acquired food cup responding to V1 (Figure 5A), which was paired with food in all groups. However, the rats in Group Low, which received only food as the US, showed more food cup responding than the rats in the other two groups, which also received sucrose on V1 trials. ANOVA showed a significant effect of session blocks, F(13, 468) = 98.46, and a significant group X session blocks interaction, F(26, 468) = 1.57. A contrast of food cup responding in Group Low with that in the other two groups was significant, F(1, 36) = 4.71.

The rats in Groups High and Down also acquired sucrose-cup responding to V1 (Figure 5C); these two groups of rats received sucrose delivery 5 s after food delivery on V1 trials. ANOVA of sucrose-cup responding during V1 showed a reliable effect of groups, F(2, 36) = 4.09, and session blocks, F(13, 468) = 4.68, but the interaction of those two variables was not reliable, F(26, 468) = 1.43. A contrast of sucrose cup responding in Groups High and Down with that in Group Low (which did not receive sucrose presentations on V1 trials), was significant, F(1, 36) = 7.53.

Phase 2 compound training

The right side of each panel of Figure 5 shows responding during the blocking phase of the experiment, in which V2 was paired with sucrose as before, but the noise was added to V1. As in Phase 1, there were no differences among the groups in either food cup or sucrose responding during V2; group X lesion X blocks ANOVAs showed no significant effects other than of session blocks.

Similarly, for the most part, the patterns of responding to V1 were maintained. However, omission of the sucrose US in Group Down appeared to alter two aspects of conditioned responding. First, sucrose CRs were reduced in that group. ANOVA showed a significant effect of group, F(2, 36) = 11.08, and post-hoc contrasts among the three groups (Tukey HSD) showed significantly more sucrose cup behaviors in Group High (which continued to receive sucrose on V1N trials) than in the other two groups. Second, there was some suggestion that food CRs were enhanced in sham-lesioned rats in Group Down relative to CN-lesioned rats in that group, but supporting statistical analyses (contrasts of the performance of CN- and sham-lesioned rats in the final session alone) F(1,36) = 5.80, and of the lesion-dependent differences in linear trend over session blocks, F(1, 36) = 5.26, fell short of acceptable levels of significance for post-hoc comparisons.

Test phase

Figure 6A shows food cup responding during the two tests (combined) of responding to the noise alone. Consistent with the results of Experiment 1 and of Holland & Gallagher (1993b), sham-lesioned rats exhibited unblocking of food cup responding with a downshift in reinforcer value, but rats with CN lesions did not. Food cup responding of sham-lesioned rats was greater in Group Down than the two control groups, but responding of lesioned rats did not differ across the groups. A group X lesion ANOVA showed a significant interaction, F(2, 36) = 3.70. A planned contrast of food cup responding in Group Down with responding in the pooled controls was significant in sham-lesioned rats, F(1, 36) = 6.20, but not in the CN-lesioned rats, F(1, 36) = 1.74. In addition, within Group Down, responding was greater in sham-lesioned than in CN-lesioned rats, F(1, 36) = 10.23.

Figure 6.

Mean (± s.e.m.) elevation (during CS minus pre-CS) in food cup (panel A) and sucrose cup (panel B) responding during the noise-alone test sessions of Experiment 2. CN = amygdala central nucleus, CS = conditioned stimulus.

Figure 6B shows sucrose cup responding during those same tests of unblocking. Not surprisingly, sucrose cup responding to the noise was greatest in the rats in Group High, the only group in which sucrose was presented on compound trials. However, a more notable feature of these data was the observation of greater sucrose cup responding in Group Down in the CN-lesioned rats than in the sham-lesioned rats, or in CN-lesioned rats in Group Low. Implications of this finding will be discussed later. ANOVA showed a significant main effect of group, F(2, 36) = 3.49. Planned comparisons showed that sucrose-cup responding was greater in Group High than in the other two groups, F(1, 36) = 4.16. Post-hoc contrasts (Tukey HSD) showed greater sucrose cup responding in CN-lesioned rats in Group Down than in either the sham-lesioned rats in that group or in CN-lesioned rats in Group Low.

Figure 7 shows the acquisition of the V2→sucrose, V2N→nothing feature negative discrimination, intended as an assessment of the consequences of the compound conditioning phase for either the associability or associative strength of N. As in Experiment 1, the feature-negative discrimination was learned more rapidly in Group Down than in either of the control groups. Also as in Experiment 1, there was no effect of the CN lesion on the acquisition of that discrimination. Separate ANOVAs of the discrimination difference scores (Figure 7A) and responding to the nonreinforced V2N compound (Figure 7C) showed significant main effects of group, Fs(2, 36) = 6.01 and 4.34 (respectively), and session blocks, Fs(5, 180) = 43.17 and 47.08, but no group X lesion interactions, Fs < 1. Finally, contrasts of the discrimination difference scores in Group Down with those of the two control groups were significant, Fs(1, 36) = 11.54 and 8.63. Separate ANOVA of responding during the reinforced V2 trials alone (Figure 7B), showed only a main effect of session blocks, F(5, 180) = 6.11; other Fs < 1.12. Unlike in Experiment 1, a separate group X lesion X stimulus (V2 or V2N) ANOVA of the first session’s data showed no significant main effects or interactions except for that of stimulus, F(1, 36) = 10.00; other Fs < 1.39.

Figure 7.

Panel A: Mean discrimination difference scores (elevation scores for reinforced noise trials minus elevation scores for nonreinforced noise + V2 trials) during the inhibitory savings test sessions of Experiment 2. Panels B and C: Mean elevation (during CS minus pre-CS) in sucrose cup responding on reinforced (CS+) and nonreinforced (CS-) trials in those sessions. The error bars show two times the overall between-groups MSE. CN = amygdala central nucleus, V2 = visual stimulus not used in previous training (panel light or house light).

Discussion

As in Experiment 1, sham-lesioned rats trained with the unblocking downshift procedure (Group Down) acquired excitatory associations between the added noise CS and the initial, food pellet reinforcer, whereas CN-lesioned rats did not. Also as in Experiment 1, both sham and CN-lesioned rats in Group Down showed more rapid acquisition of inhibitory learning in the final inhibitory learning test than the rats in the two blocking control groups. We argue that this latter outcome indicates that rats in the downshift condition acquired inhibitory associations between the noise and the omitted sucrose reinforcer in the compound conditioning phase. Because only excitatory noise-food learning was impaired by the CN lesions, we suggest the enhanced noise-food learning in the sham-lesioned rats of Group Down reflected enhanced processing of the food US rather than of the noise CS. Furthermore, from this viewpoint, the lesion deficits reflected impairment in surprise-induced enhancements of US processing rather than in processing of the noise CS itself.

Indeed, in Experiment 2, the CN-lesioned rats in Group Down showed some evidence of greater learning about the noise CS than the sham-lesioned rats. In response to the noise, lesioned rats displayed more sucrose-cup behavior, appropriate to the omitted sucrose reinforcer, than normal rats in that group. It is difficult to reconcile this enhanced learning of CN-lesioned rats relative to normal rats with lower associability of the noise relative to those same rats.

Because the added noise CS was never paired with the sucrose reinforcer in Group Down, the origins of the sucrose-cup behavior observed to that cue are of interest. Sucrose behavior was not acquired to the noise in either lesioned or normal rats in Group Low, which had received equivalent pairings of V2 (but not V1) with sucrose. Thus, the acquisition of sucrose cup behavior in Group Down was apparently dependent on the prior pairing of V1 with the food-sucrose sequence. As Rescorla and Colwill (1983) and Rescorla and Holland (1982) noted, compound stimulus presentations in blocking/unblocking procedures provide opportunity for the formation of within-compound associations between the noise and V1. Because in Group Down (but not in Group Low) V1 had been previously paired with the food-sucrose sequence, the establishment of noise-V1 stimulus-stimulus associations could endow the noise with the ability to activate a representation of V1, which in turn would evoke sucrose-cup behavior.

To evaluate this account for the origin of sucrose cup behavior in Group Down, in a companion experiment we examined the effects of extinguishing the visual blocking CS (V1) after compound training, but before assessing responding to the added noise CS. Sixteen sham-lesioned rats and 14 rats with histologically-verified CN lesions received initial visual CS and compound training that was identical to that of Group Down in Experiment 2. However, the test of excitatory conditioning to the noise alone was not administered until after the conclusion of the compound conditioning phase and an extinction phase. In the extinction phase, half of the rats received 8 sessions in which they received a total of 48 nonreinforced presentations of the visual blocking CS (V1) and 16 reinforced presentations of the other visual CS (V2), while the other half of the rats received only the reinforced V2 presentations.

As in Experiment 2, CN-lesioned rats that did not receive extinction of the blocking CS showed more sucrose cup behavior (19.1 ± 3.1%) than sham-lesioned rats (5.4 ± 3.3%). However, that advantage was eliminated by extinction of the blocking CS; lesioned rats that received extinction of V1 averaged 4.5 ± 3.4% sucrose cup behavior, comparable to that of sham-lesioned rats (4.7 ± 3.6%). A contrast among the four conditions, performed after a lesion X extinction ANOVA yielded a significant interaction, F(1, 22) = 4.79, showed sucrose responding to be reliably greater in nonextinguished, CN-lesioned rats than in any of the other three conditions. Also consistent with the findings of Experiment 2, sham-lesioned rats showed higher levels of food cup responding than CN-lesioned rats. Furthermore, that superiority was unaffected by extinction of V1 prior to testing. Food cup responding was 19.8 ± 5.4% and 17.6 ± 2.7% in sham rats that received no extinction or extinction, respectively, and 6.4 ±5.6% and 3.4 ± 5.1% in the corresponding lesioned rats. A lesion X extinction ANOVA showed a significant effect of lesion, F(1, 22) = 10.55, but not of extinction, F < 1, or the interaction of those variables, F < 1. Thus, the food cup response was probably not due to within-compound associations but rather to noise-food associations (Holland, 1984).

At first glance, this account for the acquisition of sucrose cup behavior to the noise would seem to apply equally to intact and lesioned rats. One possible account for the greater responding of lesioned rats appeals to greater generalization between V1 and V2 in lesioned rats. In this case, noise-V1 pairings would establish more sucrose cup behavior to the noise in CN-lesioned rats because those rats displayed more of that behavior to V1 than sham-lesioned rats. Indeed, inspection of Figure 5C provides some support for that possibility, with the rats in Group Down-CN apparently showing more sucrose cup behavior to V1 in Phase 1 than the rats in Group Down-Sham. However, that difference was not statistically significant (see above), and furthermore, there was no evidence for such greater generalization in CN-lesioned rats in Group Low.

A second account is suggested by previous observations from our laboratory. Using a conditioning procedure similar to that used here, Holland (1980 Holland (1985a) found that presentation of a food US after serial compound cues interfered with within-compound learning. Furthermore, Holland (1980) found that such interference was enhanced when processing of the interfering US was enhanced by procedures that rendered that US more surprising. In the context of Experiment 2, in sham-lesioned rats, enhancement of processing of the food US (by sucrose omission) might especially interfere with the formation of the noise-V1 associations that are responsible for the sucrose cup behavior controlled by the noise. If CN lesions interfered with the surprise-induced enhancement of processing of the food US, then that relatively lower processing of the food US might yield both reduced noise-food learning and enhanced noise-V1 learning, because the food would be both a less effective reinforcer and a less effective interfering event.

All in all, the observation of CN lesion deficits in excitatory noise-food learning but not in either excitatory or inhibitory noise-sucrose learning is consistent with the view that omission of the sucrose enhanced processing of the food reinforcer (Holland, 1988; Kamin, 1969), and that CN lesions interfered with that enhancement. By contrast, these observations are difficult to reconcile with the standard view that US omission in the downshift procedures generally enhances CS associability.

Experiment 3

In Experiment 3 we attempted to directly examine changes in the associability or reinforcement power of the first US when the second is omitted, under conditions in which changes in CS associability were unlikely to be critical determinants of the learned behavior. An extended initial conditioning phase designed to reduce the effectiveness of a food US was followed by a brief downshift phase designed to restore that effectiveness. Finally, the effectiveness of the food as a reinforcer was evaluated by monitoring the acquisition of new learning supported by that food, using a novel CS. As in the previous studies, performance of sham- and CN-lesioned rats was compared throughout.

Table 2 shows an outline of the procedures of Experiment 3. First, the rats received extended training of a visual CS with either a single food reinforcer or a food-sucrose sequence. According to models like that of Rescorla and Wagner (1972), either treatment should reduce the effectiveness of the food when it is preceded by that visual CS. However, if the Pearce and Hall (1980) model were extended to cases of US-US learning, the repeated presentation of the food-sucrose sequence might also render the food less effective overall. Establishment of food as a consistent predictor of sucrose may reduce the associability of the food, just as establishing a light as a consistent predictor of a tone reduced the associability of that light in the task used by Holland and Gallagher (1993a) and Wilson et al (1992). Likewise, in the single-food condition, the food’s effectiveness as a reinforcer might be reduced because it consistently predicted the absence of anything but the experimental context. Next, surprise was induced after the food in half of the rats in each treatment condition, either by omitting the anticipated sucrose (downshift) in the food-sucrose sequence condition or adding an unanticipated sucrose after the food alone (upshift). Finally, the reinforcing power of the food was assessed by using the food-sucrose sequence or food alone as the reinforcer for a new, auditory CS, in the absence of the visual CS.

Table 2:

Outline of Procedures of Experiment 3

Group	Phase 1	Phase 2	Test
Down	V1→food→suc	V1→food	N→food→suc
	V2→suc
High	V1→food→suc	V1→food→suc	N→food→suc
	V2→suc
Low	V1→food	V1→food	N→food
	V2→suc
Up	V1→food	V1→food→suc	N→food
	V2→suc

Note. N = noise, V1 and V2 = counterbalanced visual stimuli, suc = sucrose, → = followed by.

The procedures thus differed from unblocking procedures in that reinforcer shifts occurred in the absence of the auditory test stimulus, and the visual blocking CS was omitted at the time of training of the auditory test CS. In this manner, any changes in the effectiveness of the food attributable to changes in CS associability would be eliminated. Furthermore, in contrast to the procedures of Experiment 2, the surprise treatment phase was brief, to reduce the opportunity for acquisition of new expectancies appropriate to the altered reinforcer.

Methods

Subjects and apparatus

The subjects were 72 male Long-Evans rats, obtained from Charles River Laboratories (Raleigh, NC) and maintained as in the previous experiments. The apparatus was the same as that used in Experiment 2.

Surgical and histological procedures

The surgical and histological procedures were identical to those of the previous experiments. Forty rats received bilateral lesions of the CN, and 32 received sham lesions.

Behavioral training procedures

The initial food and sucrose cup training was identical to that of Experiment 2. Phase 1 training of Groups Down and High was identical to Phase 1 training of Group Down in Experiment 2, and Phase 1 training of Groups Low and Up was identical to Phase 1 training of Group Low in Experiment 2.

Next, the rats were given two 32-min sessions, each of which included four pairings of the visual CS V1 with either the food alone (Groups Down and Low) or the food-sucrose sequence (Groups Up and High). In Groups Low and High, this experience was consistent with their previous training, but in Groups Down and Up, these experiences represented either downshifts or upshifts, respectively, in the value of the reinforcer.

Finally, the reinforcing value of the food was assessed by examining the acquisition of food cup behavior to a noise paired with either the food alone (Groups Low and Up) or the food-sucrose sequence (Groups Down and High). In each of six 64-min sessions, there were 16 pairings of a 78 db noise with the appropriate reinforcer.

Results

Histological results

Twenty-nine brains were judged as having acceptable lesions, by the same criteria as used in the previous experiments. The final numbers of lesioned rats in Groups Down, High, Low, and Up were 8, 7, 6, and 8, respectively. Except around the injector tracks, no cellular damage was evident in any of the vehicle control brains (all subgroup ns = 8).

Behavioral results

Group X lesion X sessions ANOVAs of pre-CS food cup and sucrose cup behavior showed significant effects of sessions in phase 1 and testing, Fs > 2.65, with all other Fs < 1.

Table 3 shows elevation in food cup and sucrose cup responding to the two visual CSs in both groups during the final block of four Phase 1 training sessions. This responding was comparable to that observed in the corresponding phases of the procedurally-identical Experiment 2. All rats acquired sucrose-cup, but not food-cup responding, to V2, which was paired with sucrose alone in all groups. Group X lesion ANOVAs of sucrose-cup and food-cup responding showed no reliable effects or interactions, Fs < 1. Likewise, all groups acquired food cup responding to V1, which was paired with food in all groups. A group X lesion ANOVA showed a significant effect of group, F(3, 53) = 10.75, but the remaining main effects and interactions were not reliable, Fs < 1. Post-hoc individual comparisons showed that Groups Low and Up showed more food cup responding than Groups High and Down; note that V1 was paired with only food in Groups Low and Up, but with a food-sucrose sequence in the other groups. By contrast, only the rats in Groups High and Down acquired sucrose cup behavior; ANOVA showed only a significant effect of group, F(3, 53) = 22.76; individual post-hoc comparisons showed greater sucrose cup responding in Groups High and Down than in the other groups.

Table 3:

Terminal responding in each training phase of Experiment 3.

			Response			Response
Group	Lesion	Phase 1	food	sucrose	Phase 2	food	sucrose
Down	Sham	V1→fd→suc	13.6±2.9	21.9±4.7	V1→fd	23.4±1.7	20.5±3.6
		V2→suc	−0.1±1.5	62.4±4.6
	CN	V1→fd→suc	14.2±4.9	23.8±3.9	V1→fd	13.5±2.3	20.5±3.5
		V2→suc	0.0±1.7	59.0±5.2
High	Sham	V1→fd→suc	14.5±1.4	19.9±3.6	V1→fd→suc	20.7±1.4	11.7±2.2
		V2→suc	−0.8±1.9	56.1±4.1
	CN	V1→fd→suc	15.4±1.7	20.9±3.9	V1→fd→suc	19.2±2.5	15.3±2.7
		V2→suc	−1.7±1.1	55.4±6.5
Low	Sham	V1→fd	24.5±4.0	2.2±1.4	V1→fd	28.4±4.2	5.7±1.9
		V2→suc	−0.9±1.3	54.3±6.1
	CN	V1→fd	28.4±3.8	0.6±1.3	V1→fd	24.8±3.8	−0.2±1.7
		V2→suc	0.8±2.1	46.5±0.9
Up	Sham	V1→fd	28.5±4.3	0.5±2.2	V1→fd→suc	33.6±4.2	4.4±1.4
		V2→suc	1.2±1.8	54.2±7.1
	CN	V1→fd	29.4±5.0	3.5±1.3	V1→fd→suc	29.7±4.9	5.5±1.6
		V2→suc	−0.4±1.6	54.5±6.7

Note. Entries are mean (± s.e.m) elevation scores for food cup and sucrose cup responding over the last two sessions of each phase. V1 and V2 were two visual conditioned stimuli, counterbalanced. CN = central nucleus of amygdala, fd = food, suc = sucrose, → followed by.

Responding on V1 trials in Phase 2 (Table 3) maintained the patterns observed in Phase 1, with comparable levels of statistical significance in all cases.

Figure 8 shows the primary data from this experiment, the acquisition of food cup CRs to the noise CS in the final test phase, which served as an index of the reinforcing power of the food pellet US. Exposure to downshifts in Phase 2 (Figure 8A) enhanced the reinforcing power of the food in the test phase in sham, but not CN-lesioned rats. Food cup responding elevation scores for the noise CS were higher in Group Down-Sham than in Group High-Sham, whereas there was no such elevation among the CN-lesioned rats. By contrast, exposure to upshifts in Phase 2 (Figure 8B) appeared to enhance the reinforcing power of the food US in both sham and CN-lesioned rats; food cup elevation scores for the noise CS were greater in the Up training condition than in the Low condition.

Figure 8.

Mean elevation (during CS minus pre-CS) in food cup responding during the test phase of Experiment 3. The error bars indicate two times the overall between-groups MSE for each phase. CN = amygdala central nucleus, CS = conditioned stimulus.

The test data of rats that received the food-sucrose reinforcer in Phase 1 training and testing (Groups Down and High) and those that received the food-alone reinforcer in those phases (Groups Up and Low) were analyzed separately. A group X lesion X sessions ANOVA of the elevation scores for Groups Down and High showed a significant effect of sessions, F(5, 135) = 201.20, and significant group X lesion, F(1, 27) = 7.25, group X sessions, F(5, 135) = 2.83, and group X lesion X sessions, F(5, 135) = 2.46, interactions. Planned comparisons showed significantly greater responding in Group Down-Sham than in either Group High-Sham, F(1, 27) = 9.26, or Group Down-CN, F(1, 27) = 7.24, but no difference in responding between Groups Down-CN and High-CN, F < 1. A comparable analysis of elevation scores for Groups Up and Low showed significant main effects of group, F(1, 26) = 3.97, and sessions, F(5, 130) = 187.46, and a significant group X sessions interaction, F(5, 130) = 2.82. Notably, there were no reliable effects or interactions involving lesion, Fs < 1.

Finally, rats that received sucrose in the test phase (Groups Down and High) acquired low levels of sucrose behavior (not shown in Figure 8; elevation scores ranging from 19.6 ± 8.5% to 25.7 ±7.9%), but there was no evidence that omitting the sucrose reinforcer briefly in Phase 2 in Group Down affected learning about that reinforcer in the test phase, in either CN- or sham-lesioned rats. A treatment X lesion X sessions ANOVA of the sucrose cup elevation scores for those groups showed only a significant effect of sessions, F(5, 135) = 4.13.

Discussion

In a typical unblocking experiment (e.g., Experiments 1–2), the reinforcer is changed when a new CS is added to a previously-trained CS. Variations in the amount of conditioning established to the new CS as a result of that change then may be attributable to variations in the effectiveness of either that CS or the reinforcer itself. By contrast, in Experiment 3, the reinforcer was changed prior to the introduction of the novel test CS, and returned to its original value when the test CS was presented. Only the original training CS and the food reinforcer were present when surprise was induced by the omission of the sucrose reinforcer. Thus, the observation of more rapid conditioning of food cup behavior to the new noise CS in Groups Down-Sham and Up-Sham than in Groups High-Sham and Low-Sham, respectively, likely reflects increased value of the food reinforcer. The lack of such an increase in Group Down-CN is consistent with past data showing that CN is critical to enhancements in learning rate when expected events are omitted (Holland & Gallagher, 1993b).

It might be argued, however, that the critical violation of event expectancies occured in the test phase, rather than in Phase 2. Within this view, the shift from the (altered) Phase 2 reinforcer back to the original reinforcer in the test phase enhanced processing of the test CS, rather than of the food US. Thus, the enhanced learning rate in Group Down-Sham would reflect the upshift in the reinforcer in the test, and the enhanced learning rate in Group Up-Sham would reflect the downshift in the reinforcer in the test. This possibility seems unlikely for two reasons. First, the brief Phase 2 treatment, in which the shifted reinforcers were delivered in Groups Down and Up, was probably inadequate to establish new food→no sucrose or food→sucrose expectancies, whereas even small numbers of disconfirmation trials have been observed to be sufficient to enhance processing of events (e.g., Hall & Pearce, 1979; Holland, Bashaw, & Quinn, 2002). Second, previous data indicated that CN lesions eliminated enhanced learning effects due to downshifts in reinforcers, but not those due to upshifts (Holland & Gallagher, 1993b). The observation of CN lesion effects in Group Down and not in Group Up thus supports the claim that the shift critical to enhanced learning in the test was the shift that occurred in Phase 2, rather than the return to the original reinforcers in the test sessions.

It is notable that the results from Group Up provide additional support for Holland and Gallagher’s (1993a) suggestion that the role of amygdala CN may be specific to the case of omission of expected events, and may not extend to the delivery of unexpected events. In two experiments, Holland and Gallagher (1993b) found no evidence for CN lesion effects on unblocking when the value of the US was shifted upward, from a single food pellet to a 1 pellet→2 pellet sequence. However, as they pointed out, the additional learning observed when the 2-pellet US was added might simply have reflected additional conditioning supported by that event, rather than enhanced processing of the added CS (or the original, 1-pellet US). In Group Up of the present experiment, the added sucrose US supported a different CR than the food initial US. Thus, the enhanced effectiveness of the food US observed in the final test phase in both sham- and CN-lesioned rats could not be easily attributed to direct conditioning effects of the sucrose US.

In summary, the results of Experiment 3 suggest that omission of an anticipated second reinforcer or presentation of an unanticipated second reinforcer may enhance the effectiveness of the first reinforcer, just as these manipulations are often described as enhancing the effectiveness of the CS. Furthermore, amygdala CN is critical to such enhancements when they are produced by omission of expected events, but not when they are produced by delivery of unexpected events.

General Discussion

Evidence for changes in US processing in unblocking

Several aspects of the results of these experiments support the conclusion that downshifts in reinforcement value, accomplished by omitting the second of a two-US sequence, may enhance the effectiveness of the remaining US. Perhaps the most straightforward support comes from Experiment 3. In Experiment 3, these downshifts enhanced the rate of learning associations between a novel cue and the first US (food) when that cue was later paired with the original two-US (food-sucrose) sequence. Because only the US, and not the CS, occurred in the same session in which the expectancies of the sucrose US were disconfirmed, it seems more likely that these disconfirmations altered processing of the food US than processing of a CS.

Other support is provided by differences between the performances of rats with lesions of the amygdala CN and those with sham lesions in all of these experiments. Each of these differences is consistent with the view that omission of the second of two reinforcers enhanced processing of the first reinforcer, and had little or no effect on processing of the added auditory CS. First, in Experiments 1 and 2, CN-lesioned rats showed less acquisition of excitatory noise-US1 learning than sham-lesioned controls after a downshift procedure, but no deficit in the acquisition of inhibitory noise-US2 learning. If these downshifts generally enhanced the associability of the noise CS, then both excitatory and inhibitory conditioning of the noise should reflect that enhanced associability in sham-lesioned rats, and thus CN-lesioned rats should show deficits in both inhibitory and excitatory learning. By contrast, if these downshifts enhanced only the processing of the first US, then CN lesions would only interfere with excitatory noise-US1 learning and not inhibitory noise-US2 learning. Second, in Experiment 2, CN-lesioned rats showed more acquisition of within-compound noise-light learning after downshifts than sham-lesioned controls. As outlined earlier, previous data (Holland, 1980, 1985a) suggest that greater within-compound learning occurs when processing of subsequent USs is minimized. If sucrose omission enhanced processing of food in sham-lesioned but not CN-lesioned rats, then greater noise-light learning would be anticipated in CN-lesioned rats. At the same time, the observation of less acquisition of within-compound learning in sham-lesioned rats relative to CN-lesioned rats makes it even more unlikely that sucrose omission generally enhanced the associability of the added noise in sham-lesioned rats.

Changes in CS and US processing in compound conditioning procedures

Our claim that sucrose omission enhanced processing of the food US in our unblocking procedures is also supported by previous investigations of the contributions of changes in CS and US processing to blocking in the conditioning preparation used here. The results of those studies (Baxter et al, 1999; Holland & Fox, 2003) indicate that in this conditioning preparation, blocking is produced by reductions in the processing of the US, rather than in the processing of the added CS. Thus, it is reasonable to infer that unblocking might reflect the amelioration of those reductions in US processing.

Although those studies showed blocking to be the consequence of reductions in US processing, at the same time they clearly demonstrated that the blocking procedures also induced reductions in CS associability. As in the present series, Baxter et al (1999) and Holland and Fox (2003) used lesion procedures to make inferences about the nature of changes in stimulus processing in this conditioning preparation. Rats with impaired hippocampal function fail to show losses in CS associability in a number of tasks that produce such losses in normal rats (e.g., Baxter, Holland, & Gallagher, 1997; Baxter et al, 1999; Han, Gallagher, & Holland, 1995). Holland and Fox (2003) found that normal rats exposed to a blocking procedure showed reductions in the associability of the added cue relative to rats exposed to overshadowing or single cue conditioning procedures. These reductions were assessed by subsequent excitatory and inhibitory savings tests with the added cue, administered after the assessment of blocking itself. By contrast, rats with lesions of the hippocampus (Holland & Fox, 2003) or the cholinergic input to the hippocampus (Baxter et al., 1999) failed to show those reductions in CS associability with exposure to blocking procedures, showing more rapid learning about the added cue than sham-lesioned rats in savings tests administered after assessment of blocking. Nevertheless, both lesioned and control rats in those studies showed identical amounts of blocking, as measured in tests of responding acquired to the added CS over the course of the blocking procedure itself. The implication of this pattern of results is that although the blocking procedure indeed induced reductions in CS associability, blocking itself occurred because the reinforcer was rendered ineffective by prior conditioning of the first CS. If the reinforcer was ineffective, conditioning of the added CS would not be supported, regardless of its associability.

The pattern of results observed by Baxter et al (1999) and Holland and Fox (2003) has direct implications for the nature of learning in unblocking procedures as well. A typical CS-processing account for unblocking with reinforcer downshifts argues that omitting the second sucrose reinforcer enhances processing of the added CS, allowing it to be associated with the remaining food reinforcer. However, if the effectiveness of the food reinforcer has been reduced to near-zero levels, no amount of enhancement of CS processing will permit CS-food learning. Thus, if blocking is the consequence of reductions in US- rather than CS-processing in a particular conditioning protocol, then only enhancements of US processing would permit unblocking.

Analogous considerations reconcile the apparent discrepancy between the implications of the present studies and those of Mackintosh and Turner (1971) and Holland (1985b; Holland & Gallagher, 1993a, Experiment 2). In those studies, unblocking with either upshifts or downshifts in reinforcer number was reduced if the second CS was added several sessions before the reinforcer shift. This outcome is consistent with CS-processing accounts of unblocking: Presentations of the added CS with the original reinforcer would result in reduced associability of the added CS, lowering the baseline of associability to be enhanced when the US was later shifted. But if downshifts acted primarily by enhancing the effectiveness of the initial US, then on the surface these shifts should be equally effective regardless of whether they occurred at the initial introduction of the added cue or after training it with the original reinforcer. However, if, as suggested by many CS-processing theories (e.g. Mackintosh, 1975; Pearce & Hall, 1980), exposure to the added cue with no change in the reinforcer produces decrements in that CS’s associability, then its associability would be low when the reinforcer shift occurred. Thus, the impact of changes in US processing on conditioning to the added cue would initially be small. Because most modern learning theories assume a multiplicative relation between CS and US processing parameters, reductions in either to low levels should reduce the impact of subsequent variations in the other.

The recognition that processing of both CSs and USs can be enhanced by disconfirmation of outcome expectancies may help us understand some of apparently contradictory conclusions drawn from the results of studies of stimulus selection. How might we predict the relative contributions of changes in CS and US processing to the results of any particular compound conditioning experiment? We think a reasonable first approach is to assume that disconfirmation of outcome expectancies can enhance processing of any event present when that surprise is induced, and is not limited to arbitrary classes of “CS” vs. “US” events. More salient events might be expected to gain more benefit from this surprise, and so it is perhaps not surprising that the primary consequence of sucrose omission in the present studies was the enhancement of food’s reinforcement power rather than the noise’s associability. By contrast, other procedures might be more easily interpreted as altering the associability of CSs. For example, in a procedure we have used extensively to study brain mechanisms of alterations in CS processing, trials on which expectancies are violated include only a CS, and no US. As noted in the introduction, Wilson et al (1992) induced overexpectation of outcomes by first presenting rats with a consistent light-tone sequence, reinforced with food on half these presentations. Later, for some rats, the tone was omitted on nonreinforced trials. Because only the light was present on trials that violated light-tone expectancies, enhancements in stimulus processing that occur as a consequence of surprise would be confined to the light CS. Indeed, both excitatory (Holland & Gallagher, 1993a; Wilson et al, 1992) and inhibitory (Holland, et al, 2001; Wilson et al, 1992) new learning about the light was enhanced in subsequent tests with these rats, compared to rats for which the tone was never omitted. Importantly, both of these enhancements were absent in rats with CN lesions (Holland & Gallagher, 1993a; Holland et al, 2001), as would be expected if tone omission produced CN-dependent enhancements in CS associability across the board.

Implications for theories of learning

Given that the observation of excitatory conditioning after downshift procedures is typically construed as exclusive evidence for enhanced CS processing, an important aspect of the results reported here is that they show that downward shifts in reinforcer value may produce unblocking by altering processing of the US rather than of the CSs. It is possible that other examples of presumed enhancements of CS processing by disconfirmation of event expectancies may reflect instead alterations in US processing. For example, Young and Fanselow (1992) showed that under some circumstances a negative transfer effect demonstrated by Hall and Pearce (1979) and attributed to alterations in CS processing may be instead due to changes in US processing.

Clearly, theories of learning need to be more complete in specifying how discrepancies between the actual and expected trial outcomes affect processing of various events present on conditioning trials. The Rescorla-Wagner (1972) model provided a simple rule, that this reinforcement error signal determines the effective reinforcer value for each conditioning episode. If the error signal is positive (the actual reinforcer is underpredicted by the cues present on that trial), the effective reinforcer value is positive, and cues present on that trial gain excitatory strength. If the error signal is negative (the actual reinforcer is overpredicted), the effective reinforcer value is negative and inhibition is acquired. If there is no prediction error, then no new learning occurs. By this simple assumption, the Rescorla-Wagner model accounted for a wide range of conditioning phenomena, including contingency learning effects, conditioned inhibition (including the acquisition of inhibition with downshifts in reinforcer value), blocking, and unblocking with upshifts in reinforcer value. However, because the reinforcement error signal determines the effective reinforcer value, the Rescorla-Wagner model could not predict the occurrence of excitatory learning in the presence of a negative error signal, as in unblocking with downshifts in reinforcer value. Nor could it deal with any examples of alterations in CS learning rate parameters, which were considered constants in that model.

An alternate approach was taken by Pearce and Hall (1980), who suggested that the reinforcement error signal did not directly determine the reinforcer value, but instead the unsigned magnitude of that signal determined the learning rate (associability) parameters for CSs present on conditioning trials. Thus, this model easily predicted the occurrence of unblocking with either upshifts or downshifts in reinforcer value. And, because it incorporated a Rescorla-Wagner-like mechanism by which the effectiveness of overpredicted reinforcers could be modified, the Pearce-Hall model could also predict the occurrence of most of the overexpectation and inhibition phenomena handled by the Rescorla-Wagner model, including in principle the observation of inhibitory learning with downshifts in reinforcer value in unblocking procedures. However, the Pearce-Hall model does not include sufficient specification to determine when the net effects of reinforcer will be excitatory or inhibitory. Nor does it specifically provide for enhanced US processing as a result of reinforcement error signals, as is claimed to underlie the results of the present studies.

All in all, it seems likely that reinforcement error signals influence processing of both CSs and USs. First, the signed value of that error signal may determine the reinforcement value of the US on a given conditioning trial, as specified by the Rescorla-Wagner model. Second, the absolute magnitude of that error (“surprise value”) may influence processing of CSs present on that trial, for example by determining their learning rate parameters (∀s), as in the Pearce-Hall model. Third, the error magnitude may influence processing of the USs that are physically present on that trial. Although it would seem most parsimonious to suggest that surprise value influences processing of a US in the same way that it influences CS processing, that is, by altering its learning rate parameter (∃ in the Rescorla-Wagner model), this mechanism is not consistent with unblocking data like those presented here. If the reinforcing value of a US is determined by the reinforcement error signal, then no amount of change in its learning rate parameter will permit an expected or overexpected US to serve as an excitatory reinforcer. Instead, the reinforcement error signal itself must be altered after surprise, for example by the temporary suppression of excitatory input from the blocking CS or the temporary enhancement of its supported learning asymptote (λ).

The view that reinforcement error signals influence processing of both CSs and USs is reminiscent of Kamin’s (1969) suggestion that surprise produced by violations of reinforcer expectancies provoked a “backward scan” of all events on that trial, enhancing their processing on that trial. Interestingly, this suggestion is consistent with the popular view that the amygdala is particularly critical for post-trial processing of information, which enhances subsequent memory for that information (e.g., McGaugh, et al., 2000). It is notable however that many studies of blocking and unblocking (e.g., Mackintosh, Bygrave, & Picton, 1977) suggest that surprise on a given learning trial does not affect learning on that trial, as implied by Kamin’s (1969) notion, but rather, by altering the effectiveness of stimuli for learning on subsequent trials. This idea is captured by many theories of variations in CS processing (e.g., Mackintosh, 1975; Pearce & Hall, 1980), but it remains to be seen whether it applies equally to CS and US processing.

Finally, these results add to our understanding of amygdala function in appetitive conditioning. Although Holland and Gallagher (1999) proposed that amygdala CN was critically involved in the enhancement of attentional processing of CSs when expected events are omitted, the results reported here suggest a broader role of CN in the modulation of effectiveness of reinforcers as well. Notably, CN has both direct and indirect connections with mesolimbic dopamine systems, which are often implicated in reinforcement processes (Gonzales & Chesselet, 1990; McDannald, Groshek, & Holland, 2004), and which are often thought to provide behavioral action systems with the reinforcement prediction error signals needed for learning (Schultz & Dickinson, 2000).