Effects of ventral striatal lesions on first- and second-order appetitive conditioning

Abstract

Rats with bilateral lesions of the ventral striatal nucleus accumbens failed to acquire Pavlovian second-order conditioning to auditory stimuli paired with visual stimuli that had previously received first-order pairings with food. This deficit in second-order conditioning was specific to learning driven by incentive properties of the first-order cues, and was observed whether the first-order training had occurred prior to or after lesion surgery. Lesions also produced deficits in the display of conditioned responses to the first-order conditioned stimulus, but only when they were made after first-order training. These results suggest a specific role for the ventral striatum in acquiring and expressing incentive properties of conditioned stimuli through second-order conditioning, as well as a more general role in expressing previously acquired Pavlovian conditioned responses.

Introduction

Pavlovian pairings of an initially neutral conditioned stimulus (CS) with a food reinforcer endow a CS with many properties, including the ability to elicit overt conditioned responses (CRs) and various attentional, representational, and motivational or “incentive” functions (Rescorla, 1988). Especially in the context of addiction, much research has explored how relatively arbitrary stimuli may, through associative learning, come to serve powerful motivational functions, including the ability to modulate ongoing behavior (Holmes et al., 2010; Holland & Petrovich, 2005), reinforce new learning (Everitt & Robbins, 2005), and control powerful stimulus-directed approach behaviors (Flagel et al., 2010).

Considerable evidence suggests that the ventral striatal nucleus accumbens (ACB) is involved in the acquisition or expression of learned incentive to appetitive cues, but not in the acquisition of many overt conditioned responses to those cues. In experiments with rats, in which response-independent food delivery is signaled by the insertion of a lever into the experimental chamber, ACB lesions selectively impair sign-tracking (lever-directed responses) while leaving goal-tracking (food-directed responses) intact (Chang et al., 2012). Similarly, ACB core lesions impair the performance of cue-directed responses to a visual stimulus paired with food, but not the acquisition of food-directed responses to that same cue (Cardinal et al., 2002). Furthermore, although ACB lesions do not impair acquisition of instrumental lever-pressing or Pavlovian food-directed responses to an appetitive cue, lesions do impair Pavlovian-instrumental transfer (PIT), the ability of the Pavlovian cue to enhance instrumental responding (Hall et al., 2001). These studies suggest the ACB is necessary for attributing incentive to cues for food but not to the formation of cue-food associations in general.

Some incentive functions of the ACB may depend on communication with the basolateral amygdala (BLA). Like bilateral ACB lesions, lesions that disrupt BLA-ACB communication also impair sign-tracking (Chang et al. 2012). Further supporting this account, BLA-ACB disconnection lesions impair a cue’s ability to serve as a reinforcer in second-order conditioning, without impairing the acquisition of food-directed CRs or cue-specific conditioned orienting responses (Setlow et al. 2002b). Noting that bilateral lesions of BLA made after acquisition of first-order conditioning did not affect a cue’s ability to reinforce second-order conditioning (Setlow, Gallagher, & Holland, 2002a), Setlow et al. (2002b) suggested dissociable roles for the BLA in acquisition and ACB in expression of acquired incentive. If correct then ACB lesions made after first-order conditioning would also impair second-order conditioning, because they would prevent rats from expressing the first-order cue’s learned incentive value. Notably, Cardinal et al. (2002) found that post-training lesions of ACB core severely impaired the performance of previously-trained cue-directed responses.

Here, we first examined the effects of ACB lesions on the expression of previously-established CRs to a CS, and on a measure of incentive learning, the ability of that CS to reinforce second-order conditioning. Next, in those same lesioned rats, we examined the acquisition of CRs to a previously nonreinforced CS, and that cue’s ability to reinforce new second-order conditioning. Thus, we could potentially contrast ACB’s role in acquiring incentive with its role in expressing incentive.

Materials and methods

Animals

The subjects were 16 male Long-Evans rats (Charles River Laboratories, Raleigh, NC), which weighed 300-325 g on arrival. Rats were individually housed in a climate controlled colony room that was illuminated from 7:00 A.M. to 7:00 P.M. Rats were given ad libitum access to food and water before and continuing two weeks after surgery. They were then placed on a food restriction schedule and maintained at 85% of their ad libitum weights throughout the experiment. The care and experimental treatment of the rats were carried out in accordance with the Guidelines of the US National Institutes of Health, and were approved by the Animal Care and Use Committee of Johns Hopkins University, where this research was conducted.

Surgical Procedures

Our lesions targeted the entire ACB. This targeting permitted the most reasonable comparison with the results of a related previous study (Setlow et al. 2002b), which examined the effects of accumbens-BLA disconnection produced by making unilateral whole-accumbens lesions contralateral to a unilateral BLA lesions. Furthermore, not knowing a priori which accumbens subregion might contribute more to the function under investigation, and given recent data suggesting that several related incentive learning phenomena engage both shell and core subregions (Blaiss & Janak, 2009; Chang & Holland, 2013; Change et al., 2012; Flagel et al., 2011), we thought it more reasonable to start with examining effects of damage to the entire structure.

Rats were anesthetized with isoflurane and placed in a stereotaxic frame (Kopf Instruments, Tujunga, CA) fitted with an isoflurane gas anesthesia system. A midline incision was made, the skin and periosteum retracted, and holes drilled through the skull over the lesion sites. Lesions were made with N-methyl-D-aspartic acid (NMDA; Sigma, St. Louis, MO) in phosphate buffer vehicle (12.5 μg/μl). Sham lesions were made by infusing phosphate buffer vehicle alone. A 30-gauge needle attached by a length of plastic tubing to a 10-μl microsyringe (Hamilton, Reno, NV) mounted on a syringe pump (Sage Instruments, Boston, MA) was used for the infusions. The needle was lowered to the flat skull coordinates AP +2.1 mm, ML ± 1.6 mm from bregma, DV −7.2 mm from the skull surface, and 0.4 μl NMDA or vehicle was infused. Following the infusion, the needle was left in place for 4 min to allow for diffusion. After surgery, the incision was closed with wound clips and antibiotic ointment was applied to the wound site. Rats were monitored until recovery from anesthesia and on subsequent days for behavioral disturbances and signs of infection.

Apparatus

The behavioral training apparatus consisted of eight individual chambers (22.9 cm x 20.3 cm x 20.3 cm) with aluminum front and back walls, clear acrylic sides and top, and a floor made of 0.48-cm stainless steel rods spaced 1.9 cm apart. A dimly illuminated food cup was recessed in the center of one end wall. A 6-W jeweled panel light, which served as the source of one visual CS, was located 5 cm above the opening to the food cup recess. An infrared photocell placed inside the food cup was used to register food cup entries. Each chamber was enclosed in a sound-attenuating shell, where ventilation fans provided masking noise (70 dB), and a 6 W lamp behind a red lens provided continuous dim background illumination. Another 6-W lamp, which served as the house light visual CS, was mounted on the inside wall of the shell, 5 cm above and 20 cm to the left of the panel light. A speaker, which was used to present auditory cues was mounted next to the house light. A television camera was mounted within each shell for video recording of behavioral training sessions.

Behavioral training procedures

The experimental procedures are summarized in Table 1. All rats first received 2 64-min sessions of training to approach the food cup; each session included 16 deliveries of 2 45-mg grain pellets (Test Diets, Richmond, IN), the reinforcer used throughout this experiment. Next, the rats received four 64-min training sessions, each of which included 16 presentations of a 10-s visual stimulus (V1+), either the intermittent (3 hz) illumination of the houselight or the steady illumination of the panel light (counterbalanced), reinforced with food pellets. The next four 64-min discrimination training sessions included 8 reinforced V1+ presentations, and 8 nonreinforced 10-s presentations of the other visual stimulus (V2−), randomly intermixed.

Table 1.

Outline of procedures

	FOC Disc	SOC 1	reversal	SOC 2	extinction	test
Group SOC:	V1+, V2− | surgery | A1→V1, V2− | V2+, V1− | A2→V2, V1− | V2− or V1− | V2−, A2−
Group CTL:	V1+, V2− | surgery | A1→V2, V1− | V2+, V1− | A2→V2, V1− | V2− or V1− | V2−, A2−

Notes: There were 3 ACB-lesioned and 4 sham-lesioned rats in each of Groups SOC and CTL. A1, A2, counterbalanced auditory stimuli; CTL, control; FOC Disc, first-order conditioning discrimination; SOC, second-order conditioning; +, reinforced by food; − nonreinforced; V1, V2, counterbalanced visual stimuli; →, followed by.

After surgery and 14-17 days of recovery, the rats received testing designed to assess the effects of the lesions on the ability of the CS+ to both elicit previously-established conditioned responding and to serve as a conditioned reinforcer for the establishment of second-order conditioning to an auditory stimulus. The first session of this phase was 48 min in duration. All rats initially received 4 presentations of a 10-s 80-db auditory cue (A1, white noise or 1,500-hz tone, counterbalanced), spaced over the first 16 min, as a pretest of their unconditioned responding to that cue. In the remaining 32 min of this session, the rats in the second-order conditioning (SOC) treatment group received four second-order conditioning trials, in which A1 was immediately followed by V1, but without food delivery, and 4 V2− presentations, randomly intermixed. In that same period, the rats in the control (CTL) treatment group received 4 pairings of A1 with V2−, and 4 nonreinforced presentations of V1, randomly intermixed. Sessions 2 and 3 of this phase were each 64 min in duration, and included 8 A1→V1 and 8 V2− trials (SOC) or 8 A1→V2 and 8 V1− trials (CTL), all nonreinforced, in each session. Trials were in pseudorandom order, with the constraint that each half of each session included equal numbers of the two trial types.

Next, all rats received reversal training with the two visual cues. In each of 12 64-min sessions, there were 8 reinforced V2+ trials and 8 nonreinforced V1− trials, randomly intermixed. Thus, whereas pairings of V1 with food had occurred before the lesions were made, V2− food pairings were given after the lesions were made.

At the conclusion of reversal training, the ability of V2 to serve as a conditioned reinforcer to establish second-order conditioning to a new auditory stimulus (A2) was assessed. Unlike in the previous second-order conditioning test, all rats were tested in the SOC condition. In the first 32 min of the first session, rats received 4 presentations of A1, and 4 presentations of A2 (either tone or noise, whichever had not been used as A1) as a test of residual responding to A1 and of unconditioned or generalized responding to A2. In the remaining 32 min of this session, and in each half of the next 2 64-min sessions, the rats received 4 A2→V2 and 4 V1− trials, all nonreinforced, randomly intermixed.

Finally, to assess the nature of second-order conditioned responding (see Categorization and origins of second-order CRs section, above), the effects of extinction of V2 on responding to the second-order A2 stimulus were examined. Half of the ACB-lesioned and half of the sham-lesioned rats received nonreinforced presentations of V2 and the other half received nonreinforced presentations of V1. If second-order CRs were mediated by stimulus-stimulus associations between A2 and V2, extinction of V2 should reduce those responses more than additional nonreinforced presentations of V1. If second-order CRs were mediated by stimulus-response associations based on the prior “cached” (Lucantonio et al., 2012) value of V2, extinction of V2 should not affect responding to A2 any more than nonreinforced presentations of V1. In each of 2 64-min sessions, rats in the EXT condition received 16 nonreinforced V2 presentations and rats in the CTL condition received 16 nonreinforced presentations of V1. In a third session, the rats first received 4 V2 or V1 presentations, as before, and then 4 test presentations of A2 alone. This session was 32 min in duration, preserving the 4-min ITIs used throughout the study.

Response measures

Automated measure

In this conditioning preparation, rats come to approach and enter the food cup as a CR to both auditory and visual CSs paired with food. We measured that response as the percentage of time during the CS that the food cup photocells were activated. To reduce within-treatment variance, we subtracted the percentage of time the photocells were activated during the 10-s periods before CSs from that during comparable CS periods to form elevation scores. Pre-CS scores did not differ across treatments. We reported both the elevation scores and the pre-CS scores.

Behavioral observation procedure

In addition to food cup responses, rats acquire stimulus-specific orienting responses (ORs), rearing to visual cues and startle to auditory cues (Holland, 1977), which engage neural systems independent of those responsible for food cup behavior (e.g., Gallagher et al., 1990) and which are not readily assessed automatically. Furthermore, second-order conditioned responses to auditory cues have mixed topography (e.g. Holland, 1977, 1981), including the normal OR to auditory cues (startle), increased overall activity (such as increased walking around the chamber), agitated head movements (headjerk), food cup behaviors, and behaviors characteristic of responding elicited by the first-order stimulus, in this case, rearing. Thus, assessments of conditioned responses during the two second-order conditioning phases were made by a behavioral observation procedure. These assessments included behaviors evoked by the first-order visual CSs as well as those occurring to the second-order auditory CSs. Finally, we also used the behavioral observation period to assess rearing to the visual CSs on the final session of initial V1+, V2− discrimination training and on sessions 4, 8, and 12 of V2+, V1− reversal training.

All observations were made from videotapes of the behavioral training sessions and paced by auditory signals recorded onto the tapes. Behavior was scored at 1.25-s intervals during each 10-s CS and pre-CS period, such that there were eight observations per period. At each observation, only one behavior was recorded. The categories of behavior recorded from the tapes were: food cup, nose within the food cup (essentially identical to the photocell-assessed food cup response); head jerk, short, rapid, horizontal or vertical head movements, usually occurring in the vicinity of the food cup; walk, walking, running, or circling; rearing, standing on the hind legs with the front legs off the cage floor but not grooming; and startle, a jump in response to stimulus onset.

The primary observers were aware of the purpose of the experiment, but were blind to the lesion condition and group membership of the rats observed, and the previous-phase training history of the stimuli presented. However, some variables, such as whether a stimulus was presented within a compound or as an element, could not be blinded. Approximately 2,000 judgments of the same performances in each of the two second-order conditioning phases were made by two observers, one of whom was also unaware of the purpose of the experiment. Inter-observer agreement ranged from 90-99%, depending on the behavior scored.

Previous work (Holland, 1977, 1980) showed that rearing to visual cues paired with food is more frequent during the first half of 10-s visual CSs paired with food, whereas food-cup behavior to those cues is more frequent during the second half. Accordingly, for the visual CSs (V1 and V2) we reported rearing data for only the first 5 s, and food-cup data for only the second 5 s period. However, because most behavior during second-order CSs (in this case, food cup, head jerk, walk, and rear) tends to be distributed more evenly across the CS periods (e.g., Holland, 1977), we reported those behaviors throughout the 10-s auditory CS period. An exception was startle responding, which occurred only at auditory cue onset; we reported the percentage of trials in a session on which that response occurred. The frequency of all other behaviors was expressed as “percentage behavior”, which was obtained by dividing the number of observations of a particular behavior in a given interval by the total number of observations in that interval. Thus, a rat that performed food-cup behavior on five of the eight observations during a tone CS trial would have a 62.5% percentage food-cup behavior for that trial. Because observations occurred at a constant rate, that measure is an absolute (not relative) measure of response frequency. Finally, to reduce within-treatment variance, we subtracted baseline behavior from that during the CSs to form elevation scores. Pre-CS behavior scores did not differ across treatments. We reported both the elevation scores and the pre-CS scores.

Categorization and origins of second-order CRs

Considerable evidence supports distinguishing three classes of responses to second-order CSs. First, the occurrence of rearing to an auditory CS paired with a visual first-order CS reflects the formation of stimulus-stimulus, auditory-visual associations. In that case, the display of rear ORs to the second-order auditory CS would depend not on the acquired reinforcement value of the visual first-order CS, but only on its ability to elicit rear ORs. Indeed, previous studies showed that extinction of a first-order visual cue’s ability to elicit rearing ORs, after the completion of second-order conditioning, also eliminates the display of that response to an auditory second-order CS that was paired with that cue (Setlow et al., 2002a). By contrast, the acquisition of second-order startle ORs and food CRs reflects the formation of stimulus-response associations that depend on the acquired reinforcement value of the first-order CS. These second-order responses are unaffected by the extinction of the first-order CS that reinforced the second-order learning (Holland & Rescorla, 1975; Setlow et al., 2002a). Finally, considerable evidence shows that brain systems involved in acquiring both first- and second-order conditioned ORs differ from those involved in the acquisition of other CRs (e.g., Holland & Gallagher, 1999). Thus, we reported 3 second-order conditioned behaviors that occurred to the auditory CSs: rearing and startle ORs, and a composite “food-related” CR that included food cup, head jerk and walk responses.

Data analysis

Both elevation and pre-CS scores for each behavior were analyzed within mixed design analyses of variance (ANOVAs), with lesion (ACB or sham) and various treatments (SOC or CTL in the initial second-order conditioning phase, EXT or CTL in the extinction phase) as between-subject variables, and trial type (reinforced or nonreinforced in the first-order conditioning and reversal phases, and second-order compound or first-order element-alone in the second-order conditioning phases), and session as within-subject variables. The p< .05 level of statistical significance was adopted. Levene tests for violations of homogeneity of variance assumptions were not significant for any measure in any analysis set. The Greenhouse-Geisser correction for violations of sphericity assumptions was used in all repeated-measures ANOVAs.

Histological procedures

At the completion of the experiments, the rats were given an overdose of pentobarbital and perfused intracardially with 0.9% saline followed by 4% formaldehyde. Brains were removed and stored in 4% formaldehyde for 24 hr, followed by 30% sucrose in 4% formaldehyde until slicing. The brains were sliced on a freezing microtome and coronal sections (40μm) collected through the areas of the accumbens. Sections were mounted on glass slides, stained with thionin, and coverslipped with Permount. Lesions were evaluated by first drawing lesions on four sections (see Figure 1) shown on plates from Paxinos and Watson (1998) and then calculating the percentage damage. Percentage damage was calculated by dividing the total lesion area (as drawn) across all sections by the total ACB area across those sections. Calculated percentages for lesions on the left and right sides were averaged to yield a single percentage damage value for each brain. Lesion drawings and damage assessments were made blind with respect to group membership and behavioral performance.

Figure 1. Histological results.

A: Smallest (dark) and largest (outline) lesion extents at each of four coronal planes, 2.70mm, 1.70mm, 1.20 mm, and 0.70 mm anterior to bregma (top to bottom). Section outlines are from Paxinos and Watson (1998) and are used by permission of Elsevier.

Results

Histological results

Figure 1 presents a schematic representation of neuronal damage in accepted ACB-lesioned rats (n = 6). On average, 89 5% (mean ± SEM) of ACB was eliminated, with comparable damage to both its shell and core. Data of rats with less than 50% damage to ACB (n = 2) were discarded. Sham-lesioned rats (n = 8) had no observable damage other than near the needle track.

Behavioral results

First-order conditioning (pre-lesion)

Both simple conditioning and visual discrimination learning (not shown) was rapid, and did not differ as a function of subsequent group assignment. ANOVA of food cup elevation scores in the four training sessions showed only an effect of cue (47.4 ± 8.2% CS+ and 7.6±5.4% CS-; F_1,12 = 40.44, p < .001) and a significant cue X session block interaction (F_3,36 = 6.08, p < .001), all other ps > .20. ANOVA of the elevation in rear behavior on the final session showed only an an effect of cue (37.5 ± 5.2% CS+ and 12.0 ± 5.6% CS-; F_1,12 = 53.09, p < .001. Pre-CS rearing was 10.3 ± 3.2%.

Post-lesion test: First-order conditioned responding

ACB-lesioned rats showed substantial deficits in their performance of both food cup CRs and rearing ORs during the previously-trained first-order visual cue (V1). Figures 2A and 2B, respectively, show food cup elevation scores when V1 was presented as the reinforcer on second-order conditioning (compound) trials, and when it was presented alone. A lesion X trial type (compound or element-alone) X cue (identity of reinforcer cue on compound trials (V1+ or V2−) X session ANOVA showed significant main effects of cue (F_1,10 = 12.37, p = .006) and session (F_2,20 = 10.58, p < .001). Although the main effect of lesion was only marginally significant (F_1,10 = 4.27, p = .066), the lesion X cue (F_1,10 = 5.55, p = .040) and lesion x session (F_2,20 = 5.14, p = .016) interactions reached conventional significance levels, as did the cue x session interaction (F_2,20 = 3.89, p = .037). Individual comparisons showed that sham-lesioned rats responded more to the previously-reinforced V1 than to the previously-nonreinforced V2 (F_1,10 = 22.78, p < .001) but ACB-lesioned rats did not (F_1,10 = 1.29, p = .282). Pre-CS food cup behavior did not differ significantly between ACB-lesioned (8.5±2.9%) and sham-lesioned (3.6±1.0%) rats (F_1,12 = 3.14, p > .101).

Figure 2. Responding to visual first-order conditioned stimuli in the initial second-order conditioning phase.

Panels A and C show mean ± s.e.m. responding to V1 and V2 when they were presented within second-order conditioning (compound) trials, and Panels B and D show responding to V1 and V2 when they were presented alone (element). Panels A and B show food cup behavior and Panels C and D show rear behavior. Open symbols refer to behavior of rats that received nucleus accumbens (ACB) lesions prior to this phase and filled symbols refer to behavior of rats that received sham lesions of this region. V1+ and V2− refer to the (counterbalanced) visual stimuli that had been reinforced or nonreinforced, respectively, prior to lesion surgery. In this phase, the 4 sham-lesioned and 3 ACB-lesioned rats in Group SOC received V1 on compound trials and V2 on element trials, whereas the 4 sham-lesioned and 3 ACB-lesioned rats in Group CTL received V2 on compound trials and V1 on element trials.

ORs to the first-order visual cues were also reduced by the ACB lesion, although the pattern was more complex because ORs to the nonreinforced V2, likely unconditioned ORs, recovered somewhat on compound trials (in which they were presented under novel circumstances, that is, after the novel A1). Figures 2C and 2D show rear elevation scores to the visual cues on compound and visual element-alone trials, respectively. ANOVA showed significant main effects of cue (F_1,10 = 5.14, p = .047), session (F_2,20 = 4.14, p = .031), and lesion (F_1,10 = 6.95, p = .025), as well as a trial type (compound or element) X cue X session interaction (F_2,20 = 5.79, p = .010). As with food cup CRs, the sham-lesioned rats showed more rearing to V1+ than to V2− (F_1,10 = 6.87, p = .025), but the ACB-lesioned rats did not (F_1,10 = 0.53, p =.483). However, unlike food cup CRs, ORs to the previously nonreinforced V2− showed substantial dishabituation when V2− was presented after A1 on compound trials: rearing to V2− was significantly more frequent on compound trials than on element-alone trials (F_1,10 = 5.88, p = .036). Individual comparisons for each session showed that this difference was only significant on session 1 (F_1,10 = 6.11, p = .033; other ps > .138). Pre-CS rearing did not differ between SCB-lesioned (5.2 ± 1.2%) and sham-lesioned (7.4 ± 2.3%) rats (F < 1, p > .458.

Post-lesion test: Second-order conditioning

Sham-lesioned rats acquired multiple classes of second-order CRs (see Categorization and origins of second-order CRs section, above), including food-related CRs (Figure 3A) and rear ORs characteristic of the first-order visual cues (Figure 3C). In addition, startle ORs appropriate to the second-order auditory cues (Figure 3B) were also maintained at higher levels in SOC than CTL rats. By contrast, lesioned rats showed no evidence for the acquisition of food-related CRs or rear ORs, although startle responding of ACB-lesioned rats was maintained in a manner similar to that observed in sham-lesioned rats.

Figure 3. Responding to the auditory second-order conditioned stimulus in the initial second-order conditioning phase.

Panels A, B, and C show mean ± s.e.m. food-related, startle, and rear behaviors, respectively, in response to the auditory stimulus A1. Rats in the second-order conditioning (SOC) groups received pairings of A1 with the previously-reinforced visual stimulus V1, and rats in the control (CTL) groups received pairings of A1 with the previously-nonreinforced visual stimulus V2. Open symbols refer to behavior of rats that received nucleus accumbens (ACB) lesions prior to this phase and filled symbols refer to behavior of rats that received sham lesions of this region. Session PT refers to responding to A1 alone on a block of trials presented prior to A1-V1 or A1-V2 pairings. There were 4 sham-lesioned and 3 ACB-lesioned rats in each of the two treatment (SOC and CTL) conditions.

A lesion X treatment X sessions ANOVA of food-related CR elevation scores showed only a marginally significant effect of lesion (F_1,10 = 3.81, p = .080). However, a lesion X treatment (SOC or CTL) ANOVA of elevation scores on the final second-order conditioning session showed significant main effects of both lesion (F_1,10 = 11.46, p = .007) and treatment (F_1,10 = 11.95, p = .006), and a significant lesion X treatment interaction (F_1,10 = 8.37, p = .016). Individual comparisons on that session showed greater food-related CRs in the SOC treatment condition than in the CTL condition among sham-lesioned rats (F_1,10 = 23.52, p < .001) but not among ACB-lesioned rats (F_1,10 = 0.14, p = .717). Comparable ANOVAs of pre-CS food-related behavior showed no significant effects (ps > .201); overall percentages of pre-CS food-related behaviors were 8.1 ± 1.7% in sham-lesioned rats and 8.0 ± 1.9% in ACB-lesioned rats.

For rearing, a lesion X treatment X sessions ANOVA showed significant effects of lesion (F_1,10 = 13.57, p = .004) and treatment (F_1,10 = 14.72, p = .003), as well as a significant lesion X treatment interaction (F_1,10 = 14.06, p = .004). Individual comparisons showed greater rearing in the SOC condition than in the CTL condition among sham-lesioned rats (F_1,10 = 33.57, p < .001), but not among ACB-lesioned rats (F_1,10 < 0.01, p = .955.) Although rearing to A1 was also significantly lower in ACB-lesioned rats on the pretest trials (F_1,10 = 5.70, p = .038) prior to the beginning of second-order training, neither the effect of treatment nor the lesion X treatment interaction was significant (ps > .228) on that block of trials. Comparable ANOVAs of pre-CS rearing showed no significant effects (ps > .177); overall pre-CS rearing was 7.2 ± 4.2% in sham-lesioned rats and 6.1 ± 4.3% in lesioned rats.

Finally, a lesion X treatment X session ANOVA of startle responding showed only a main effect of treatment (F_1,10 = 10.54, p = .009; other ps > .265). Individual comparisons showed more startle responding in the SOC than CTL conditions among both sham-lesioned rats (F_1,10 = 5.63, p = .039 and ACB-lesioned rats (F_1,10 = 5.02, p = .049.) Because startle responding began at a high level in all conditions and there was no evidence of an acquisition function, we suspect this observation reflects protection from habituation of unconditioned startle responding to A1 in the SOC treatment condition, rather than A1-V1 learning. Habituation to a target cue is frequently slowed or prevented when the habituating cue is presented together with another cue (e.g., Joynes & Grau, 1996).

In summary, ACB lesions made after the completion of first-order conditioning of a visual cue left that cue unable to reinforce new food-related CR and rearing responses, as would be expected if ACB was critical to the expression of previously-learned incentive value to that cue. However, given that the ACB lesion also interfered with the visual cue’s ability to evoke overt first-order food cup and rear CRs, these results do not permit identifying a special role for ACB in the expression of learned incentive alone. Instead, we must conclude that ACB function was critical to the expression of prior learning in general. The remainder of the experiment was designed to evaluate the effects of the lesion on the establishment of new learning subsequent to the lesion damage.

Reversal learning

Despite the substantial deficits ACB-lesioned rats showed in responding to stimuli that were conditioned prior to the lesions, those rats showed no significant deficits in acquiring new learning to the previously nonreinforced cue or in extinguishing reposing to the previously reinforced cue (food cup responding, Figure 4A). This lack of an effect of ACB lesion is consistent with the results of many previous investigations of the effects of ACB manipulations on reversal learning (e.g., Calaminus & Hauber, 2007; Castane et al., 2000; Schoenbaum & Setlow, 2003; but see Haluk & Floresco, 2009). A lesion X cue X session ANOVA showed main effects of cue (F_1,12 = 47.10, p < .001) and session (F_11,132 = 7.37, p < .001) as well as a cue X session interaction (F_11,132 = 36.64, p < .001), but no significant effect or interactions involving lesion (lowest p = .111 for the 3-way interaction, other ps > .638). A comparable ANOVA of pre-CS food cup responding showed a significant three-way interaction (F11,132 = 2.15, p = .021), but the differences did not appear systematic; lesioned rats showed significantly higher pre-CS responding than sham-lesioned rats on sessions 4,5,6, and 10 (ps < .05, no correction for multiple comparisons).

Figure 4. Performance in the reversal learning phase.

A: Mean ± s.e.m. food cup responding. B: Mean ± s.e.m. rear responding, which was scored only for sessions 4, 8, and 12. V2+, reinforced visual stimulus; V1-, nonreinforced visual stimulus. In the previous phases, V1 had been reinforced and V2 nonreinforced. The data points portray performance of all 8 sham-lesioned or 6 ACB-lesioned rats.

A lesion X cue X session ANOVA of rearing (Figure 4B) also showed significant effects of cue (F1,12 = 36.69, p < .001), session (F_2,24 = 5.19, p = .013) and cue X session interaction (F_2,24 = 9.71, p < .001), but no significant effect or interactions with lesion (lowest p = .441). ANOVA of pre-CS rearing showed no significant effects or interactions, ps > .340.

Post-reversal second-order conditioning phase

Figure 5 shows food cup (A) and rearing (B) responses to the first-order cue, V2, on second-order conditioning trials. As in the reversal phase, but unlike in the post-lesion test (Figure 2), ACB-lesioned rats showed no deficits in either response compared to sham-lesioned rats. ANOVAs of elevation scores showed significant effects of sessions for both behaviors (F_2,24 > 18.11, ps < .001), but no other effects or interactions (ps > .215). ANOVAs of pre-Cs responding showed no significant effects or interactions for either measure (ps > .160). Overall pre-CS food cup behavior was 2.9 ± 1.8% and 2.7 ± 0.9% in sham- and ACB-lesioned rats, respectively, and pre-CS rear behavior was 8.7 ± 1.1% and 9.9 ± 1.8%, respectively.

Figure 5. Responding to the visual first-order conditioned stimulus V2 in the second, second-order conditioning phase.

A: Mean ± s.e.m. food cup responding. B: Mean ± s.e.m. rear responding. Open symbols refer to behavior of all 6 rats that received nucleus accumbens (ACB) earlier and filled symbols refer to behavior of all 8 rats that received sham lesions of this region.

Despite their normal first-order responding, the ACB-lesioned rats showed substantial deficits in second-order conditioning. Figure 6 shows responding to A2, the second-order auditory cue, A2. As in the previous second-order test (with A1), sham-lesioned rats acquired food-related CRs (Figure 6A), startle ORs appropriate to the auditory second-order stimulus (Figure 6B), and rearing ORs (Figure 6C) normally elicited by the visual first-order stimulus. However, ACB-lesioned rats acquired only the rearing response. This pattern of data suggests that the ACB lesions interfered with the development of second-order associations that depended on the emotional value of the first order V2 stimulus, but not with the establishment of stimulus-stimulus associations between the auditory second-order stimulus and the visual first-order CS, which itself elicited rear responses (see Categorization and origins of second-order CRs section, above). Thus, the lesions did not produce a general deficit in second-order learning.

Figure 6. Responding to the auditory second-order conditioned stimulus in the final second-order conditioning phase.

Panels A, B, and C show mean ± s.e.m. food-related, startle, and rear behaviors, respectively, in response to the auditory stimulus A2. All rats received pairings of A2 with the most-recently reinforced visual stimulus V2. Open symbols refer to behavior of all 6 rats that received nucleus accumbens (ACB) lesions prior to this phase and filled symbols refer to behavior of all 8 rats that received sham lesions of this region. Session PT refers to responding to A2 alone on a block of trials presented prior to A2-V2 pairings.

ANOVA of elevation in food-related behavior scores showed a marginal effect of lesion (F_1,12 = 3.89, p = .072), a significant effect of sessions (F_2,24 = 6.03, p = .008) and a significant lesion X session interaction (F_2,24 = 4.36, p = .024). ANOVA of startle scores showed a significant effect of lesion (F_1,12 = 11.94, p = .005) and a lesion X session interaction (F_2,24 = 3.90, p = .034). However, ANOVA of elevation in rearing showed only an effect of sessions (F_2,24 = 6.03, p = .008; other ps > .664). Neither food-related CRs (9.9 ± 2.1% and 10.1 ± 1.9% in sham- and ACB-lesioned rats, respectively) nor rear ORs (above) in the pre-CS periods was affected by lesion or session (ps < .273); startle responding did not occur in those periods.

First-order extinction and final second-order test

The extinction procedure was successful in reducing first-order responding to V2. On the V2 test trials (Table 2), both food cup and rearing ORs were less frequent in rats that had received nonreinforced V2 presentations than in rats that had received V1 presentations. ANOVAs showed the effects of extinction to be significant for rearing (F_1,10 = 20.67, p = .001) and marginally significant for food cup (F_1,10 = 4.01, p = .073), which had already declined to low levels during second-order conditioning training. Neither the effect of lesion nor its interaction with extinction was significant for either measure (ps > .310), nor did pre-CS behavior differ among conditions (ps > .265).

Table 2.

Elevation scores (mean ± s.e.m.) to V2 in final test

group (n)	food cup	rear
sham ext (4)	−6.9 ± 6.1	4.7 ± 3.0
sham ctl (4)	7.3 ± 4.4	25.0 ± 4.4
lesion ext (3)	−1.0 ± 3.9	−3.1 ± 9.4
lesion ctl (3)	6.3 ± 5.0	29.7 ± 6.9

Notes. Food cup entries are percentages of time in the food cup during V2 presentations minus the percentages of time in the food cup prior to V2 presentations. Rear entries are the percentages of observations on which rear behavior was scored during V2 presentations minus the percentages of observations on which rear behavior was scored prior to V2 presentations. Ctl, control groups; ext, extinction groups; n = number of rats, V2, visual stimulus that was reinforced in the reversal training phase and then nonreinforced (ext) or not presented (ctl) in the extinction phase.

If second-order CRs were mediated by stimulus-response associations based on incentive value of V2 (as is hypothesized for food-related and startle responses), extinction of V2 should not affect responding to A2 any more than nonreinforced presentations of V1. Indeed, in sham-lesioned rats, neither of these responses (Figures 7A and 7B) was affected by extinction of V2. As in the second-order conditioning phase, ACB-lesioned rats showed only low levels of those responses. ANOVA showed a significant effect of lesion for food-related CRs (F_1,10 = 13.93, p = .004) and a marginally significant effect of lesion for startle (F_1,10 = 4.63, p = .057), but neither the effect of extinction nor the lesion X extinction interaction was significant for either of those responses, Fs_1,10 < 1, ps > .815. By contrast, if second-order CRs were mediated by stimulus-stimulus associations between A2 and V2 (as is hypothesized for the rear response), extinction of V2 should reduce those responses more than additional nonreinforced presentations of V1, and should do so in both sham- and ACB-lesioned rats. Figure 7C shows that extinction of V2 reduced rear ORs elicited by A2 (F_1,10 = 10.81, p = .008). There was no effect of lesion nor a lesion X extinction interaction (ps > .637) for this measure. There were no recorded instances of any of these behaviors in the pre-CS periods in this test session.

Figure 7. Responding in the final test.

Panels A, B, and C mean ± s.e.m. food-related, startle, and rear behaviors, respectively, in response to the second-order conditioned auditory stimulus A2. The dark bars refer to responding in the rats that had received nonreinforced extinction trials with V2, the first-order reinforce for A2, prior to this test (4 sham-lesioned and 3 ACB-lesioned rats). The light bars refer to responding in the rats that had received nonreinforced trials with V1 (4 sham-lesioned and 3 ACB-lesioned rats).

Discussion

The results of this experiment join others (e.g., Everitt & Robbins, 2005) in demonstrating an important role for ACB in acquired incentive. Lesions of ACB made after Pavlovian conditioning impaired the expression of all measured aspects of prior learning about a stimulus paired with food, including the ability to serve as a conditioned reinforcer in second-order conditioning of another cue. Although those rats were subsequently unimpaired in their ability to learn and express food-approach and CS-orienting response to a new stimulus (V2), they failed to display evidence for incentive learning about that stimulus: lesioned rats failed to acquire second-order CRs known to depend on such learning when a new cue was paired with V2. Thus, although the nature of the impairment in the display of previously-established first-order CRs is unclear, the deficit in the ability of first-order CSs to reinforce second-order S-R learning was consistent across both second-order conditioning phases.

The initial purpose of this experiment was to separate the effects of ACB lesions on acquisition and expression of learned incentive (conditioned reinforcement) value. Setlow et al. (2002a) suggested that whereas BLA function was critical for the acquisition, but not expression, of such incentive, ACB function might be necessary for its expression. In that case, ACB lesions made after first-order V1-food training would prevent the expression of V1’s already-acquired incentive value in reinforcing new second-order conditioning of A1. Consistent with that prediction, ACB-lesioned rats showed little evidence of second-order conditioning with A1→V1 pairings, whereas sham-lesioned rats showed substantial second-order learning. Furthermore, the conclusion that ACB lesions interfered with the expression of learned incentive is also consistent with the subsequent failure to find second-order conditioning after A2→V2 pairings in ACB lesioned-rats, because those rats would be unable to express such value in the conditioned reinforcement of second-order learning, even if they did acquire incentive to V2.

A salient feature of these results is the contrasting effect of ACB lesions on first-order CRs acquired prior to and after the lesions. In the initial second-order conditioning phase, the lesioned rats showed substantial decrements in the display of all aspects of previously-established first-order conditioning to V1, including the display of food-cup CRs, rear ORs, and the ability to reinforce second-order conditioning. Thus, contrary to Setlow et al.’s (2002b) suggestion, ACB function was not responsible for the expression of V1’s reinforcement power uniquely, but rather for all measured aspects of previously-acquired learning about V1. However, if ACB function was necessary for the expression of all of these aspects of Pavlovian learning, then they should also not have been expressed in the subsequent reversal or second second-order conditioning phases. But in those phases of the experiment, only the second-order conditioning of startle and food cup responses was affected by the lesions.

Two possibilities for the differences in the effects of ACB lesions on previously-established and subsequently-learned first-order CRs come to mind. First, over the additional two weeks needed for the reversal training phase and second second-order conditioning test, surviving ACB neurons may have compensated for the neurons lost to the lesions. We have no way of evaluating this possibility, but note that the damage to ACB was substantial (nearly 90% complete). Second, when ACB function is intact, rats may depend on ACB circuitry for the storage or expression of information needed for the display of overt Pavlovian responses, but if ACB function is compromised, they may compensate by using other circuitry for processing that information. From this perspective, the disruption of second-order conditioning in both tests in this experiment indicates that any such extra-ACB compensation is not possible for the storage or expression of conditioned reinforcement power; instead, ACB function is critical to those functions.

This latter interpretation is supported by the results from Pavlovian first- and second-order conditioning conducted after the bilateral neurotoxic or sham lesions of ACB. Although those lesions had no effect on the ability of the visual CS (V2) to acquire or express new first-order conditioned ORs (rearing) or food-cup CRs, they prevented V2 from acquiring (or expressing) the ability to reinforce second-order conditioning of startle ORs or food-related CRs to a novel auditory cue (A2) later. Notably, these second-order CRs are attributable to S-R learning reinforced by acquired incentive properties of the first-order CS, and are not mediated by the concurrent ability of the first-order CS to control CRs. Furthermore, in that later second-order conditioning phase, ACB-lesioned rats showed no deficits in the acquisition of second-order rear CRs, which are attributable to learning of S-S A2-V2 associations, and which are mediated by the concurrent ability of the first-order CS to control those CRs, rather than its incentive value. One interpretation of that set of observations is that the ACB lesions interfered with the acquisition or display of incentive properties, specifically, conditioned reinforcement power, to V2, but not with the acquisition of other aspects of V2−food or A2-V2 associations.

It is instructive to compare these results with those of Setlow et al. (2002a,b). Consider first the acquisition of the rear response to the second-order auditory CS, which putatively reflects S-S associations between that cue and the visual first-order CS, which elicits a conditioned rearing response. Thus, the second-order auditory CS controls rearing because it activates a representation of the visual first-order CS, in turn generating the rear response, a claim supported both by previous research (Setlow et al., 2002a) and the results of the final post-V2 extinction test in the present study. The acquisition of this second-order rear response was unaffected by bilateral lesions of either ACB (in the second, second-order conditioning phase in this study) or BLA (Setlow et al., 2002a). Furthermore, in the initial second-order conditioning phase of the present study, in which ACB-lesioned rats showed deficits in the ability of the first-order V1 to elicit rearing, those rats also failed to show acquisition of the second-order rear response, as would be anticipated if its performance was mediated by the first-order CS’s ability to control that response.

Next, consider the acquisition of food-related CRs (walking, headjerk and food-cup behaviors) to the second-order CS, which are thought to depend on S-R learning reinforced by the incentive value of the first-order CS. Bilateral lesions of either BLA (Setlow et al., 2002a) or ACB (this study), or unilateral lesions that disconnected ACB and BLA (Setlow et al., 2002b) each prevented the acquisition of second-order food-related CRs, as would be expected if convergence of processing by BLA and ACB was critical to the acquisition or expression of value by the first-order CS. However, the effects of this same set of lesions on second-order conditioning of startle ORs, also thought to depend on S-R learning reinforced by the incentive value of the first-order CS, remain somewhat puzzling. Although bilateral lesions of either BLA (Setlow et al., 2002a) or ACB (present study) impaired the acquisition of these responses, ACB-BLA disconnection lesions had no measurable effects (Setlow et al., 2002b). Thus, ACB and BLA must interact in different ways in the learning or expression of second-order CRs and ORs. Apparently, convergence of BLA and ACB processing (absent in rats with disconnection lesions) is necessary for incentive-value driven second-order CRs, whereas separate processing by BLA and ACB is sufficient for second-order conditioned ORs. Notably, first-order conditioned ORs, but not CRs, are mediated by a circuit that includes amygdala central nucleus, the midbrain substantia nigra pars compacta, and the dorsolateral striatum (Lee et al., 2005: Han et al., 1997), a circuit left intact bilaterally in all of the lesion conditions above. It is possible that this system may require communication with both BLA and ACB but not necessarily within the same hemisphere, but we have no grounds for speculating further.

In the present study we considered whether lesions of the entire ACB, including both the shell and core subregions, impaired the acquisition or expression of indices of first- and second-order conditioning. Although this treatment of ACB as a whole contrasts with studies that identified dissociable contributions of the shell and core to motivated behavior (Saddoris et al., 2013), it is consistent with others, which found no functional differences between the two subregions (e.g., Blaiss & janak, 2009; Flagel et al., 2011). Furthermore, the whole ACB lesion approach was most suitable for the current study because of its relation to previous work from our laboratory. Initially, Hatfield et al. (1996) showed that BLA was critical for the acquisition of second-order conditioned responding that reflected incentive salience. Because of its role in motivated behavior and its dense input from the BLA, the ACB was a likely site for expression of second-order conditioned responding. Notably, BLA projections to ACB do not distinguish between the shell and core, but strongly target both subregions (Kita and Kitai, 1990). To begin to characterize the involvement of ACB in second-order conditioning, the previous BLA-ACB disconnection study (Setlow et al., 2002b) and the present study assessed the effects of lesions of the entire ACB. The present finding that ACB as a whole is important for second-order conditioned responding reflecting incentive salience provides a platform for future studies which could identify any distinct contributions of the shell and core subregions and their BLA afferents.

Conditioned reinforcement is important to the acquisition and maintenance of behavior, both normal and pathological. For example, cues paired with primary reinforcers can serve as subgoals in organized sequences of instrumental behavior, allowing substantial time intervals to elapse between performance of a response and ultimate primary reinforcement, with both natural (e.g., food) and drug reinforcers (e.g., Everitt & Robbins, 2000). Similarly, conditioned reinforcers can support the learning of potent new instrumental food- or drug-seeking behaviors (e.g., DiCiano & Everitt, 2004). ACB has been shown to be important in all of these functions (Everitt & Robbins, 2005).

Behaviors based on S-R second-order learning, which were specifically affected by ACB lesions in the present experiment, may be especially important in this regard, because they are often immune to the alterations in the value or availability of either the conditioned reinforcer or the primary reinforcer from which its value was derived (Rescorla, 1980). Thus, for example, drug-seeking behavior established in this fashion may be maintained despite efforts to devalue or minimize the availability of drugs and drug-related stimuli.

Abbreviations

ACB

nucleus accumbens

ANOVA

Analysis of variance

BLA

basolateral amygdala

CR

conditioned response

CS

conditioned stimulus

CTL

control

OR

orienting response

PIT

Pavlovian-instrumental transfer S-R stimulus-response

S-S

stimulus-stimulus

SOC

second-order conditioning