Differential Involvement of the Basolateral Amygdala, Orbitofrontal Cortex, and Nucleus Accumbens Core in the Acquisition and Use of Reward Expectancies

Abstract

In this study, the authors tested the hypothesis that the basolateral amygdala (BLA), orbitofrontal cortex (OFC), nucleus accumbens core (NA-core), and the extended hippocampus mediate different aspects of the development–maintenance of unique reward expectancies produced by the differential outcomes procedure (DOP). Rats were trained with either DOP or a nondifferential outcomes procedure (NOP) on a simple discrimination task. Fornix lesions did not affect either version of the task, demonstrating that the extended hippocampal system has no role in stimulus–outcome (S-O) associations. In contrast, in the DOP condition, BLA lesions impaired performance throughout training, OFC lesions impaired choice accuracy only in the later maintenance phase, and NA-core lesions resulted in enhanced learning. These results suggest that BLA and OFC are important for establishment (BLA) and behavioral maintenance (OFC) of S-O associations, whereas the NA-core is not needed and can in fact impede using multiple S-O associations. No impairments were observed in the NOP condition, demonstrating that these structures are not critical to stimulus–response learning.

Learned expectations are the product of various associations (stimulus–outcome [S-O], stimulus–response [S-R], response–outcome [R-O]) that provide accurate predictive information for behavioral adaptation. The differential outcomes procedure (DOP) is a manipulation of reward contingencies that modifies the cognitive strategy–brain regions used on conditional discrimination tasks (Savage, 2001; Trapold, 1970; Trapold & Overmier, 1972; Urcuioli, 2005). During the generic conditional discrimination task, a common or randomized reward procedure is used (nondifferential outcomes procedure [NOP]). Under the NOP condition, an S-R association driven by memory of the sample is used to guide behavior. In contrast, with the DOP condition, each sample stimulus and the subsequent correct response are consistently paired with a specific reinforcer that results in S-O and/or R-O associations. As a result, the subject develops an expectation of the specific reward before the actual reinforcer (Demarse & Urcuioli, 1993; Overmier & Linwick, 2001; Savage, 2001; Savage & Langlais, 1995; Trapold, 1970; Trapold & Overmier, 1972; Urcuioli, 2005).

The DOP produces a more rapid learning curve and higher terminal accuracy than the NOP, presumably through the use of unique reward expectations called the differential outcomes effect (DOE; Overmier & Linwick, 2001; Savage, 2001; Savage & Parsons, 1997; Trapold, 1970; Urcuioli, 1990, 2005). We propose that the functional difference produced by the DOP changes the neural systems used and their associated learning and memory strategies, as compared with the NOP (see Ramirez, Buzzetti, & Savage, 2005; Savage, 2001; Savage, Buzzetti, & Ramirez, 2004; Savage & Parsons, 1997; Savage, Pitkin, & Careri, 1999).

Previous findings suggest that several structures, including the basolateral amygdala (BLA), the orbitofrontal cortex (OFC), and the nucleus accumbens (NA) are part of the neural circuitry that is integral for the representation of reward values and goal-directed behavior (Blundell, Hall, & Killcross, 2001; Cardinal, Parkinson, Hall, & Everitt, 2002; Corbit, Muir, & Balleine, 2001; Holland & Gallagher, 2004). Substantial evidence suggests that the BLA is integral for encoding of incentive value by the pairing of a previously neutral stimulus (i.e., discriminative stimulus [S^D]) with a specific reward (Aggleton & Saunders, 2000; Baxter & Murray, 2002; Cardinal et al., 2002; Holland & Gallagher, 1999, 2004; McDonald & White, 1993; White & McDonald, 2002). BLA lesions impair performance on appetitive Pavlovian second-order conditioning, reinforcer devaluation (Hatfield, Jan, Conley, Gallagher, & Holland, 1996), conditional discrimination involving the DOE, and reward-specific Pavlovian-to-instrumental transfer (Blundell et al., 2001).

The OFC is a cortical region with extensive connections to the BLA as well as other cortical and subcortical regions (hippocampus, basal ganglia, and thalamic nuclei) that may act as the goal- directed correlate of reward expectancy in the presence of a discriminative stimulus (Barbas, 2000; Cardinal et al., 2002; Holland & Gallagher, 2004; McDannald, Saddoris, Gallagher, & Holland, 2005; Pickens, Saddoris, Gallagher, & Holland, 2004; Schoenbaum, Setlow, Saddoris, & Gallagher, 2003). More specifically, it has been inferred that the OFC specializes in processing features essential for the guidance of behavior based on outcome expectancies (Schoenbaum, Chiba, & Gallagher, 1998). The BLA and OFC appear to process different aspects of reward information. Whereas the BLA may be responsible for the neutral stimulus gaining associative strength with the incentive value of the reward but not for the actual use of these expectancies in shaping future behaviors, the OFC seems to guide responses on the basis of the BLA-mediated expectancies (Holland & Gallagher, 2004; Pickens, Saddoris, Gallagher, & Holland, 2005; Pickens, Saddoris, Setlow, Gallagher, & Holland, 2003; Setlow, Gallagher, & Holland, 2002; Schoenbaum, Chiba, & Gallagher, 1999).

The NA receives inputs from structures central to processing goal-directed information, such as the medial prefrontal cortex, hippocampus, and amygdala, placing it in a key position to mediate behavioral responses to reward information. The two subregions of the NA (core and shell) appear to have structural and functional distinctions (see Corbit et al., 2001; Heimer, Zahm, Churchill, Kalivas, & Wohltmann, 1991; Kelley, 1999; Zahm & Brog, 1992). Interestingly, NA lesions do not prevent rats from detecting changes in cue–reward relationships (Balleine & Killcross, 1994; Cardinal & Cheung, 2005). Rather, the dysfunction seems to lie in choosing the appropriate course of action on the basis of reward value (Cardinal et al., 2002; Corbit et al., 2001; Everitt et al., 1999; Everitt & Robbins, 1992; Lavoie & Mizumori, 1994). The core and shell may control different associative aspects of this type of complex behavior. Evidence suggests that the core is more involved in R-O associations, whereas the shell may be more involved in S-O associations (Corbit et al., 2001; Kelley, Smith-Row, & Holahan, 1997).

Although the hippocampal system connects extensively with other learning and memory processing regions, it does not appear to be involved in the expression of the DOE. In rats with damage to the extended hippocampal system, the DOP reduces or eliminates the negative behavioral impact of these lesions when compared with the behavior of rats trained with the NOP (Savage, 2001; Savage et al., 2004). This suggests that the hippocampal system is not essential for either type of reward expectancies (S-O or R-O).

In contrast, emerging findings suggest that the BLA and OFC reward circuits are necessary for the production of the reward expectancies produced by the DOP (Blundell et al., 2001; McDannald et al., 2005; Saddoris, McDannald, Gallagher, & Holland, 2004). Collectively, these findings support Savage's claim that altering reward contingency by embedding the DOP (e.g., pairing specific cues with a specific reward condition) into a task alters both the neural system and strategy engaged, as compared with the NOP (Ramirez et al., 2005; Savage, 2001; Savage et al., 1999, 2004). However, this hypothesis has not been directly tested, and previous studies have not investigated the role of reward expectancies on multiple reward-related structures using the DOP.

The present study examined the individual effect of lesions of the extended hippocampus, BLA, OFC, and NA-core on learning a conditional discrimination task with (DOP) and without (NOP) the use of reward expectancies. Further, to assess whether the contribution of these structures changes with extended training, an additional maintenance phase was added. We predicted that pre-training lesions to the BLA, OFC, and NA-core would differentially alter the ability of rats to use reward expectancies to solve a conditional discrimination task. However, in line with past research, we did not expect fornix lesions (representing the extended hippocampus) to impact the use of reward expectancies or S-R learning.

Method

Subjects

Male Sprague–Dawley rats (2−3 months old; 225−250 g) were randomly assigned to five conditions: fornix lesioned, BLA lesioned, NA-core lesioned, OFC lesioned, and surgical sham. Following the surgery, subjects were individually housed to recover and were thereafter pair housed within plastic breeder cages. All subjects had ad libitum access to Purina rodent chow and water in our vivarium, which was temperature regulated (20 °C) and had a 16:8-hr light–dark cycle (onset at 0600; offset at 2200). Two weeks prior to the onset of behavioral testing, a food deprivation regimen was instituted to reduce rats to 85% of their ad libitum feeding weight. However, the rats' normal growth curve (5−10 g per week—growth curve information acquired from Harlan Sprague–Dawley) was still maintained throughout behavioral testing.

Lesion Procedures

All subjects were anesthetized with sodium pentobarbital (Nembutal, 50 mg/ml) according to weight in grams (50 mg/kg ip) and treated with atropine sulfate (0.1 mg/kg ip) to maintain respiration. Nembutal was administered rather than ketamine–xylazine, a more common method of intraperitoneal anesthesia, to avoid potential interactions between N-methyl-D-aspartate (NMDA) and ketamine. After the subjects were fully anesthetized, bilateral brain lesions were made with a standard stereotaxic frame (Kopf Instruments, Tujunga, CA). All stereotaxic coordinates were calculated from Paxinos and Watson (1998), with bregma as the point of reference. Following surgery, all rats were monitored until fully awake. The food deprivation regimen began 1 week after surgery was completed.

Fornix lesioning

Electrolytic lesions of the fimbria–fornix (n = 20) were made bilaterally at the following coordinates adapted from Aggleton, Keith, Rawlins, Hunt, and Sahgal (1992): anterior–posterior (AP) = −1.3 mm, medial–lateral (ML) = ±1.2 and ±0.5 mm, and dorsal–ventral (DV) = 4.4 and 4.8 mm. Fornix lesions were made by administering a 3-mA anodal current for 10 s through an electrode that was insulated with the exception of 0.8 mm at the end. Electrolytic lesioning was used because of the location and structure of the fornix: The fornix is situated immediately anterior to the dorsal third ventricle as well as directly medial to the lateral ventricles. Neurotoxic lesioning would involve high doses of toxin to destroy this fiber tract and would run the risk of not impacting all of the fibers.

BLA lesioning

Bilateral neurotoxic lesions of the BLA (n = 20) were made via infusions of 0.3 μl of 0.25 M NMDA dissolved in 0.01 M phosphate buffer saline, with a pH of 7.4 (Sigma, St. Louis, MO), with a 2-μl microsyringe (Hamilton, Reno, NV), driven by a stereotaxic syringe pump (Stoelting, Wood Dale, IL). Each infusion was made over 32 s and allowed to diffuse for an additional 2 min before removal of the injector. The coordinates for the BLA lesions were AP = −2.8 mm, ML = ±5.0 mm, and DV = −8.4 and −8.7 mm (see Pickens et al., 2003).

OFC lesioning

OFC lesion rats (n = 20) were fully anesthetized prior to receiving bilateral lesions made with 0.25 M NMDA through a 2-μl microsyringe using a syringe pump (see Pickens et al., 2003). The needle was placed into the OFC, and NMDA was infused over 32 s and left in place for an additional 2 min to allow for diffusion. A total of 0.05 μl of NMDA was injected at the following coordinates: AP = +3.5, ML = ±3.2, ±4.2, DV = −5.3 and AP = +2.5, ML = ±4.7, DV = 6.2. An additional 0.025 μl of NMDA was administered at AP = +2.5, ML = ±3.5, and DV = 6.2.

NA-core lesioning

Neurotoxic lesions to the NA-core (n = 20) were produced by bilateral microinjects of NMDA (.25 M, .3 μl) for 32 s with a 2-min diffusion period at the following coordinates: AP = +1.9, ML = ±1.9, and DV = −7.7. (Procedures were adapted from Roozendaal, de Quervain, Ferry, Setlow, & McGaugh, 2001.)

Surgical sham lesioning

Surgical sham subjects representing each lesion group (fornix-lesioned sham, BLA-lesioned sham, OFC-lesioned sham, and NA-core-lesioned sham) underwent surgical procedures identical to those outlined for their lesioned counterparts, although phosphate buffer saline infusions were made rather than NMDA infusions. However, no differences were observed as a function of sham group, so sham data were collapsed, F(1, 24) < 1.

Behavioral Testing

Following 1 week of recovery from surgery, all lesion groups (fornix, BLA, OFC, NA-core, and sham) were randomly assigned to behavioral testing squads that included a representation of DOP and NOP from the lesion and control groups.

Apparatus

All behavioral testing occurred in eight small animal operant chambers (29 × 24 × 30 cm; Med Associates, St. Albans, VT). The front and rear walls of the chambers and ceiling were made of aluminum; the sidewalls were composed of clear Plexiglas. The floors of the chambers contained evenly spaced aluminum bars (1 cm apart, measuring from center to center of the bars). A water–food aperture (5 × 5 cm) was located centrally on the front wall, 1.6 cm above the grid floor. This reward dispenser contained a photobeam sensor at the entrance to detect head entries. Two identical 5-cm retractable levers were situated 3 cm to each side of the reward aperture and 6.3 cm from the floor. Located 6 cm above each lever was a cue lamp (2.5 cm; 28 V DC, 100 mA). Each operant chamber was illuminated by a houselight (2.5 cm; 28 V DC, 100 mA) that was positioned in the center of the rear wall, 24 cm above the floor. A speaker located in the left corner of the rear wall of the chamber 3 cm from the ceiling delivered a 2900-Hz tone (Med PC ENV 223A tone module, Med Associates).

Each operant chamber was enclosed in an environmental isolation chamber. A ventilation fan on the rear wall of the isolation chamber served the dual purpose of ventilation and providing background noise (60 dB). Control of the operant chambers as well as data collection was provided by a Med PC interface and the accompanying software (Med Associates) using a PC computer (Compaq Presario).

Behavioral Procedure

Upon reaching food deprivation criterion, rats were semirandomly divided into either DOP training (fornix–DOP, n = 10; BLA–DOP, n = 10; OFC–DOP, n = 10; NA-core–DOP, n = 10; sham–DOP, n = 10) or NOP training groups (fornix–NOP, n = 10; BLA–NOP, n = 10; OFC–NOP, n = 10; NA-core–NOP, n = 10; sham–NOP, n = 10) that were used throughout experimentation.

Pretraining

Following reward condition designation (DOP or NOP), pretraining began with two sessions of autoshaping. Autoshaping sessions contained 50 trials, with a mean intertrial interval of 65 s with a range of 35−90 s. Rats were presented with either the left or the right lever for 8 s (semirandomly determined; neither lever was presented for more than 3 consecutive trials), after which the lever was retracted and a randomized noncontin-gent reward was immediately presented. In the event that the rat pressed the lever, the lever subsequently was retracted, and a randomized reward (either a 45-mg pellet [Bioserve, Frenchtown, NJ] or 0.1 cc of a 20% wt/vol sucrose solution) was immediately presented. Both reinforcers have been previously shown to be efficacious in appetitive paradigms.

Following autoshaping, all subjects were trained to respond on the levers during two sessions with a continuous reinforcement schedule (fixed ratio 1 [FR1]). Regardless of reward condition (DOP, NOP), a single lever (left or right, counterbalanced between lesion and reward conditions) remained extended throughout the session, allowing continuous opportunity to earn a semirandomized reward—neither reward (sucrose pellet or sucrose water) was administered for more than three consecutive trials. This FR1 session continued until the rat had earned 50 reinforcers. In the event that the rat did not begin pressing the lever after an hour in the chamber, the subject was run on the following day with another FR1 session, and the same lever was extended. During this session, the experimenter hand-shaped behavior until the rat reliably pressed the lever. When the FR1 schedule had been successfully completed for one lever, the next daily session operated under the same schedule parameters with the exception that the opposite lever was available to the rat. The day after completion of 50 trials of FR1 on this lever, a session of variable interval 30 (VI30) schedule with only the lever used for the first FR1 session available was imposed. This schedule was also composed of 50 reinforced VI30 trials. Following successful completion of the VI30 left lever program, the next day a VI30 on only the right lever was administered. Separation of left and right lever training was given to promote equal responding on both levers (see Blundell et al., 2001).

Conditional discrimination

Discrimination sessions were composed of eight alternating segments in which either a tone stimulus (78 dB, 2 kHz) or a white noise stimulus (10 Hz) was presented (see Blundell et al., 2001). Each stimulus presentation lasted for 5 min, making the total session duration 40 min. Throughout discrimination sessions, the houselight remained on and both the left and right levers were extended. In the presence of the tone, the rat had to press the left lever to receive a reward, which was given on a VI30 schedule. Pressing the left lever during the tone resulted in the presentation of a 45-mg pellet (BioServ) in the DOP condition. In the NOP condition, the reward was not paired with the lever; therefore the rat had an equal chance of receiving the 45-mg pellet or the 20% sucrose solution. During the presentation of the white noise stimulus, pressing the right lever was necessary to attain the reward (VI30 schedule). For rats in the DOP condition, the right lever was paired with4sof access to 0.1 cc of 20% sucrose solution. In contrast, subjects assigned to the NOP group received a semirandomly determined reward for pressing the correct (right) lever. An incorrect response resulted in no reward delivery but no further repercussions. There was a total of 20 daily sessions of conditional discrimination.

Histology: Lesion Verification

Following the completion of behavioral testing, the subjects were deeply anesthetized with an intraperitoneal injection of Sleepaway (26% sodium pentobarbital in 7.8% isopropyl alcohol and 20.7% propylene glycol solution, Fort Dodge Laboratories, Fort Dodge, IA) followed by the removal of the brain. The brains were postfixed in a 10% wt/vol formalin solution for 6 days then submerged in a 20% sucrose solution for cryoprotection for at least 48 hr before sectioning. A cryostat (Leica, Bannockburn, IL) was used to cut all brains into coronal sections (50 μm), and every third section was slide mounted throughout the brain region containing the lesion. The slides were stained with cresyl violet, and a determination of lesion positioning and extent was made with a light microscope (Nikon Eclipse E400 with an attached digital camera [Scion Corporation, Frederick, MD]) at 4×–20× magnification.

Statistical Analyses

The mean percentage correct (correct responses/correct + incorrect responses) and the log transformation of the number of responses on both the correct and incorrect lever for all subjects were calculated for each of the 20 sessions, and sessions were divided into early (Session 1−10) and late (Session 11−20) phases. Group differences as a function of lesion status (BLA, fornix, OFC, or NA-core vs. sham) and reward contingency (DOP, NOP) were analyzed across early and late phases of learning as a function of session (two between-subjects and two within-subject repeated measures analysis of variance [AVOVA], α = .05). Main effects of reward contingency, lesion status, session, phase, and the pertinent interactions are noted.

Results

Histology

The electrolytic lesion of the fornix removed all tissue surrounding the probe; there was a total loss of fibers. The neurotoxic lesions of the remaining structures also resulted in significant tissue loss. Although neurotoxic lesions can spare fibers, in this study there was a complete loss of tissue (including neurons, fiber tracts, and neuropil) in a significant number of subjects (BLA = 67%; OFC = 73%; NA-core = 33%). Figure 1 shows photomicrographs of these extensive lesions with the corresponding sham-lesioned brains as well as the extent of the greatest and least damage across the anterior–posterior boundaries of the regions of interest (fornix, BLA, OFC, and NA-core) as representations from the atlas of Paxinos and Watson (1998).

Figure 1.

Photomicrographs showing cresyl violet stained sections of representative lesions and a control for comparison (left) and schematic representations (Paxinos & Watson, 1998) of lesion placement (right) of each brain region of interest. Shaded areas represent the maximum (light gray) and minimum (dark gray) extent of the lesions for the animals included in the behavioral analyses. From The Rat Brain in Stereotaxic Coordinates (Figures 5, 7, 9, 10, 12, 14, 15, 22−24, 26, 27, 29, 31, 33), by G. Paxinos and C. Watson, 1998, San Diego, CA: Academic Press. Copyright 1998 by Elsevier. Adapted with permission.

Fornix Lesioning

Subjects receiving fornix lesions were included in the data analysis if they had sustained damage to the fimbria–fornix posterior to the septum and anterior to the dorsal hippocampus. In addition, subjects with extensive damage to the neighboring septal area were excluded from the data analysis. Fifteen rats had acceptable lesions of the fornix (DOP = 8, NOP = 7). The rat containing the largest lesion considered acceptable for analysis had a lesion that extended into the anterior border of the retrosplenial cortex. However, 5 rats were excluded from the analyses because of extensive damage to the neighboring septal area or because of minimal damage to the fimbria–fornix. Figure 1 displays a representation of an average lesion as well as the variability across subjects.

BLA Lesioning

Inclusion criteria for rats with BLA damage was destruction of lateral, basal, accessory basal, and posterior nuclei without significant damage to the adjacent nuclei. Lesions must have encompassed 70% or more of these regions to be considered acceptable. The majority of subjects (64%) had complete tissue loss that included neurons, fibers, and neuropil. This resulted in absent regions being surrounded by glia (see Figure 1). The remaining 36% of subjects had neuron loss with an increase in glial cells and some tissue cavitation within the amygdala region. Despite moderate variability in lesion size across the anterior–posterior boundaries of the structure, acceptable lesions had significant damage to the main portion of the lateral and basal nuclei. Histological analyses revealed that 4 rats had damage to either the central nucleus of the amygdala or the internal capsule, and they were subsequently dropped from all analyses. All other rats were retained in the analyses (DOP = 7, NOP = 9).

OFC Lesioning

OFC lesions that destroyed neurons in a target area including medial, lateral, and ventrolateral orbital regions as well as the dorsal and ventral agranular insular cortices were included in data analyses (Pickens et al., 2003). The area of interest encompassed the dorsal edge of the rhinal sulcus, which connects to the BLA (Kita & Kitai, 1990). The majority of lesions in the cortex (73%) included complete tissue loss (neurons, fibers, and neuropil) and resulted in glial cells sealing off the damaged region. One rat that was considered acceptable for data analysis had the majority of damage centralized at the anterior segment of the lateral OFC but also displayed unilateral damage to the neighboring motor cortices (as displayed in Figure 1).

OFC-lesioned rats were excluded if damage was less than 60% of the structure in either hemisphere or if there was extensive bilateral damage in extra-OFC cortical regions. However, some rats were still considered acceptable if they contained unilateral damage to the lateral agranular motor cortex. Fifteen rats (DOP = 7, NOP = 8) reached this criterion and remained in the study (see Figure 1). Five OFC-lesioned rats were excluded because of extensive bilateral damage in extra-OFC cortical regions such as medial orbital and gustatory areas posterior to the genu of the corpus callosum in the insular cortex or because of a lesion that was ≤ 60% in either hemisphere.

NA-Core Lesioning

Rats sustaining damage to 70% or more of the NA-core without significant damage to the shell (less than 10% damage to the shell region) or encroaching upon the septal nuclei or preoptic area were included in the data analysis (see Zahm & Brog, 1992; DOP = 7, NOP = 8). There was some degree of gross morphological changes, which included ventricular enlargement and alteration in the shape of the anterior commissure, as well as gross tissue loss in 33% of the subjects. Figure 1 displays a rat with complete tissue loss within the core region. In the remaining rats (67%), gliosis, scarring, and cell loss were present within the core region, with some slight ventricular enlargement as a result of tissue cavitation. In all brains, the lateral and ventral parts of the NA, including the majority of the shell, were spared. Five rats had lesions that were either too dorsal or did not contain lesions that reached the 70% criterion.

Conditional Discrimination Learning

Choice accuracy

The mean percentage accuracy in lever pressing (correct responses/correct + incorrect responses) for each daily session across all 20 sessions was calculated for each lesion condition (sham-lesioned control, fornix, BLA, OFC, and NA-core) and is portrayed in Figure 2. The data presented in these figures were analyzed using a Lesion Status (sham vs. lesion) × Reward Contingency (DOP vs. NOP) × Phase (Sessions 1−10 vs. Sessions 11−20) × Session repeated measures ANOVA. In addition, differences in accuracy performance between DOP and NOP groups as a function of lesion status are shown in Figure 3.

Figure 2.

The mean percentage correct and standard error during acquisition and maintenance of a conditional discrimination for (a) sham-lesioned, (b) fornix-lesioned, (c) basolateral amygdala (BLA)-lesioned, (d) orbitofrontal cortex (OFC)-lesioned, and (e) nucleus accumbens (NAc)-lesioned rats trained with the differential outcomes procedure (DOP) or the nondifferential outcomes procedure (NOP). These data are presented across 20 sessions, separated into early learning phase (Session 1−10) and late learning phase (Sessions 11−20).

Figure 3.

The mean difference (± SEM) between differential outcomes procedure (DOP) and nondifferential outcomes procedure (NOP) groups accuracy scores (percentage correct DOP – percentage correct NOP) for three time points (early [Session 1], middle [Sessions 8−10], and late [Sessions 18−20]) of the acquisition–maintenance curve for the conditional discrimination task. BLA = basolateral amygdala; OFC = orbitofrontal cortex; NAc = nucleus accumbens core.

Number of bar press responses

The log number of correct and incorrect bar presses was calculated for each lesion condition (sham-lesioned control, fornix, BLA, OFC, and NA-core) for each session; see Figure 4. A log transformation was conducted as a result of the variability across subjects in response rates. These data were analyzed using separate Lesion × Reward Contingency × Phase × Session mixed design repeated measures ANOVAs for both correct and incorrect of levers. Each lesion condition was contrasted individually to the sham groups.

Figure 4.

The mean number (± SEM) of responses on both the correct and incorrect levers across all sessions (Session 1−20) for subjects from each lesion condition (sham, fornix, basolateral amygdala [BLA], orbito-frontal cortex [OFC], nucleus accumbens core [NAc-core]) trained with either the differential outcomes procedure (DOP) or the nondifferential outcomes procedure (NOP), shown as a log function. These data are divided into early and late learning phases.

Sham-Lesioned Controls

Analysis of the sham-lesioned controls alone revealed that the DOP condition, relative to the NOP condition, resulted in enhanced learning throughout behavioral testing. There were main effects for reinforcement contingency (Sessions 1−20), F(1, 18) = 13.99, p < .01; phase, F(1, 18) = 13.99, p < .01; and session, F(9, 162) = 11.25, p < .001. There were significant interactions between reward contingency and phase, F(1, 18) = 6.91, p < .05, and between reward contingency and session, F(9, 162) = 2.44, p < .05. However the four-way interaction between reward contingency, phase, day, and session was not significant, F(9, 162) = 1.57, p = .13. These findings indicate that the DOE becomes stronger between the early and late learning phases and with successive sessions (see Figures 2 and 3).

Fornix Lesion Versus Sham

When fornix-lesioned rats were compared with sham-lesioned controls, they did not differ in choice accuracy on the conditional discrimination task (main effect of lesion status), F(1, 31) < 1. There were strong main effects of reward contingency, F(1, 31) = 20.30, p < .001; phase, F(1, 31) = 64.19, p < .001; and session, F(9, 279) = 11.76, p < .001. However, lesion status did not interact with reward contingency, F(1, 31) < 1; phase, F(1, 31) < 1; or session, F(1, 31) < 1, and the Reward Contingency × Session interaction was not significant, F(1, 31) < 1. In fact, the DOE continued to increase between phases, F(1, 31) = 7.50, p < .01, and across sessions, F(9, 279) = 4.19, p < .001 (see Figure 2). Thus, both sham-lesioned and fornix-lesioned rats improved across the sessions, but regardless of lesion status, rats trained with the DOP had the highest accuracy scores (see Figure 3). These data suggest a high degree of similarity in prevalence of the DOP between fornix-lesioned and sham-lesioned rats. Planned comparisons support the lack of effect of extended hippocampal damage on performance as a function of session or phase for rats trained with either the DOP (both Fs < 1) or the NOP (both Fs < 1.5).

All rats increased responding on the correct lever across training: phase, F(1, 31) = 29.91, p < .01, and session, F(9, 279) = 9.67, p < .01. Fornix-lesioned rats displayed an elevated mean total amount of bar pressing behavior, across both correct and incorrect levers, when compared with sham-lesioned control rats, both Fs(1, 31) = 13.8, both ps < .01. However, for correct bar presses, there was a significant Lesion Status × Reward Contingency × Phase × Session interaction, F(9, 279) = 2.59, p < .01. The fornix-lesioned DOP trained rats had the highest response rate on the correct lever, particularly late in training. For the number of responses on the incorrect lever, fornix-lesioned rats had a higher response rate on the incorrect lever as shown by a significant Lesion Status × Session interaction, F(9, 279) = 3.02, p < .01. Thus, regardless of the responding on a correct or incorrect lever, fornix-lesioned rats displayed an elevated number of bar presses (see Figure 4).

BLA Lesion Versus Sham

A comparison of BLA-lesioned and sham-lesioned rats did not result in a significant main effect of lesion status, F(1, 32) = 1.32, p = .26, on choice accuracy, although there were significant main effects for reward contingency, F(1, 32) = 11.85, p < .01; phase, F(1, 32) = 99.58, p < .001; and session, F(9, 288) = 12.79, p < .001. There were also significant interactions between phase and session, F(9, 288) = 5.66, p < .001. Note that the Reward Contingency × Phase × Session interaction was significant, F(9, 288) = 1.94, p < .05, and the Lesion Status × Reward Contingency × Phase interaction approached significance, F(1, 32) = 4.07, p = .052.

Further assessment of the interactions revealed that the sham-lesioned rats displayed the DOE—demonstrated by a main effect of reward contingency, F(1, 18) = 19.45, p < .001—as a function of both phase—demonstrated by a significant Reward Contingency × Phase interaction, F(1, 18) = 7.37, p < .02—and session—demonstrated by a significant Reward Contingency × Session interaction, F(9, 162) = 2.20, p < .05. In contrast, the DOE was significantly reduced in the BLA-lesioned rats— demonstrated by a main effect of reward contingency, F(1, 14) = 1.04, p = .32. Although both the DOP and NOP trained BLA-lesioned rats improved across the two phases, F(1, 14) = 33.12, p < .001, and sessions, F(9, 126) = 5.03, p < .001, reward contingency did not interact with either of these variables (all Fs < 1). Thus, relative to sham-lesioned controls, the BLA-lesioned rats displayed a significant reduction in the DOE across all training sessions (see Figures 2 and 3). This was further supported by planned comparisons of BLA lesions to sham controls on performance as a function of sessions for rats trained with the DOP, Lesion Status × Session interaction, F(9, 144) = 2.01, p < .05. However, no such interaction was observed in the NOP condition, F(9, 153) = 1.64, p > .10.

Lever pressing on the correct lever increased across training in all rats, Phase × Session interaction, F(9, 288) = 5.37, p < .01. BLA-lesioned rats, relative to sham-lesioned controls, had higher amounts of bar pressing on the correct lever, a lesion status main effect, F(1, 32) = 6.28, p = .05, and there was a trend for this elevated responding on the incorrect lever, F(1, 32) = 3.93, p = .07. In addition, regardless of lesion status, rats in the DOP condition had higher response rates on the correct lever, F(1, 32) = 6.35, p < .05. For responding on the incorrect lever, there was a significant Reward Contingency × Phase × Session interaction, F(9, 288) = 1.98, p < .05. Rats in the NOP condition made more responses on the incorrect lever early in training, but late in training, the numbers of responses on the incorrect lever were equal across the DOP and NOP conditions.

OFC Lesion Versus Sham

When comparing OFC-lesioned rats with sham-lesioned rats, there was no main effect of lesion status, F(1, 31) = 1.16, p = .29, on choice accuracy. There was a significant main effect of reward contingency, F(1, 31) = 23.09, p < .001; phase, F(1, 31) = 82.77, p < .001; and session, F(9, 279) = 13.93, p < .01. The Phase × Session interaction, F(9, 279) = 5.84, p < .001; Reward Contingency × Phase × Session interaction, F(9, 279) = 2.65, p < .01; and Lesion Status × Reward Contingency × Phase interaction, F(1, 31) = 4.64, p < .05, were significant. These interactions were due to the fact that in the NOP condition the OFC-lesioned rats did slightly better at the end of the phases than the sham-lesioned rats, Lesion Status × Session interaction, F(1, 16) = 6.26, p < .05; whereas in the DOP condition, there was a trend for the OFC-lesioned rats to display worse performance than the sham-lesioned rats at the end of testing, Lesion Status × Phase × Session interaction, F(9, 135) = 1.92, p = .06.

Given the previous findings that pretraining lesions to the OFC removed the DOE in a test of conditional discrimination after training with each S-R condition separately (McDannald et al., 2005), we conducted planned comparisons to compare choice accuracy solely on the OFC-lesioned rats trained with the DOP with those trained with the NOP. There was a significant main effect of phase, F(1, 13) = 26.98, p < .001, and session, F(9, 117) = 5.78, p < .001, and there was also a significant Phase × Session interaction, F(9, 117) = 4.65, p < .001, on choice accuracy. Analyzing these data as a function of phase revealed that during the early learning phase, rats with OFC lesions did display the DOE, main effect of reward contingency, F(1, 13) = 12.53, p < .01. However, during the late discrimination sessions, the DOE was not prevalent, F(1, 13) = 1.47, p = .25, as shown in Figures 2 and 3. Performing the same analyses on the sham-lesioned controls showed evidence of the DOP in the early learning phase, F(1, 18) = 5.65, p < .05, as well as a stronger effect in the later phase, F(1, 18) = 21.76, p < .001.

With successive sessions, the amount of bar pressing on the correct lever increased across sessions in all subjects, F(9, 279) = 4.52, p < .01 (see Figure 4). However, there was a significant Lesion Status × Reward Contingency × Phase × Session interaction, F(9, 279) = 2.37, p < .05. In sham-lesioned rats, those trained with the DOP had higher response rates on the correct lever than those trained with the NOP. However, DOP and NOP rats with OFC lesions responded at similar rates on the correct lever, until the last three sessions. In addition, all rats showed a decline in the number of lever presses on the incorrect lever, F(1, 31) = 13.22, p < .01. There was also a significant Lesion Status × Reward Contingency × Phase × Session interaction for the response rate on the incorrect lever, F(9, 279) = 3.47, p < .01. At the start of training, rats trained with the DOP made the fewest responses on the incorrect lever, but at the end of training, rats with OFC lesions and trained with the NOP made the fewest number of responses on the incorrect lever.

NA-Core Lesion Versus Sham

We found significant differences between NA-core-lesioned rats and sham-lesioned controls in choice accuracy, main effect of lesion status, F(1, 31) = 8.58, p < .01, and between subjects trained with the DOP or NOP, F(1, 31) = 44.12, p < .001. Given that the NA-core-lesioned rats performed better than the sham rats across all levels of training, neither the Lesion Status × Phase interaction nor the Lesion Status × Session interaction was significant (both Fs < 1.7, both ps > .20). The ANOVA also revealed a significant difference across phases, F(1, 31) = 43.30, p < .001, and sessions, F(9, 279) = 15.50, p < .001. The Reward Contingency × Phase interaction, F(9, 279) = 3.67, p < .001; Phase × Session interaction, F(9, 279) = 5.07, p < .001; and Reward Contingency × Phase × Session interaction, F(9, 279) = 5.93, p < .001, were significant. Note that the NA-core-lesioned rats showed a larger DOE than sham-lesioned controls (see Figures 2 and 3). A comparison of DOP trained NA-core-lesioned rats with sham-lesioned controls trained with the DOP exposed a main effect of lesion status, F(1, 15) = 11.914, p < .01. Conversely, the same comparison between NA-core-lesioned and sham-lesioned rats trained with the NOP did not indicate a performance enhancement in the presence of NA-core lesions, F(1, 16) = 2.08, p = .10.

This augmentation in conditional discrimination performance seen in DOP trained NA-core-lesioned rats is not due to a change in motivation level: A comparison of the log number of bar presses in NA-core-lesioned and sham-lesioned rats on the correct lever revealed no difference as a function of lesion status, F(1, 31) = 2.15, p > .10. All subjects increased the number of lever presses on the correct lever as a function of Phase × Session, F(9, 279) = 3.04, p < .01. There was a significant reward contingency effect, F(1, 31) = 4.29, p < .05: DOP-trained rats had the highest number of lever presses on the correct lever (see Figure 4). However, the mean number of lever presses on the incorrect lever revealed a significant Lesion Status × Reward Contingency × Phase × Session interaction F(9, 279) = 2.95, p < .05. Rats with NA-core lesions and trained with the DOP made the fewest number of responses to the incorrect lever later in training.

Discussion

Overall, we found that damaging three brain regions associated with reward-related learning (BLA, OFC, and NA-core) had differential effects on the expression of the DOE on a simple discrimination task modified from Blundell et al. (2001). First, we replicated their findings, suggesting that BLA lesions impair the acquisition of reward expectancies produced by the DOP. Second, we also extended training from 10 to 20 sessions, allowing us to pull apart functional differences between the OFC and the BLA. The role of the OFC in the maintenance of reward expectancies is evident much later in training than is the role of the BLA. Third, and perhaps most interesting, are the data suggesting that lesions to the NA-core actually facilitate and enhance the use of reward expectancies produced by the DOP. In addition, we further dem- onstrated that the extended hippocampal system is not needed for either the acquisition or maintenance of reward expectancies.

Below, we discuss the implications of these data for understanding how the brain processes reward-related associations. However, the lesions in the present study, across all brain regions, were extensive, with a large portion including the loss of neurons, neuropil, and fibers. Thus, the lesions may reflect more systems-level damage than discrete regional neuronal loss.

Role of the BLA in Reward Expectancies

This study demonstrates that lesioning the BLA results in an impaired DOE across both the acquisition and maintenance learning phases, supporting the theory that one role of the BLA is in forming unique S-O associations for goal-directed behavioral adaptation (for a review, see Holland & Gallagher, 2004). This should be contrasted with the fact that BLA-lesioned rats are capable of learning simple Pavlovian conditioned stimulus–unconditioned stimulus associations (see Hatfield et al., 1996) and discriminative rules (S-R associations). We found that all rats, regardless of lesion status (BLA lesioned or sham lesioned) or reward condition (DOP or NOP), improved performance across sessions. BLA-lesioned rats trained with the DOP or NOP reached their maximal choice accuracy (60%−65%) at Day 7 and remained at about this level for the next 13 sessions. These findings are in contrast to DOP trained sham or fornix-lesioned rats, which continued to improve discrimination performance across both early and late learning sessions, reaching a significantly higher asymptotic performance (70%−80%) by the end of behavioral testing.

These findings, coupled with those obtained by Blundell et al. (2001), are consistent with previous studies indicating that pre-training BLA lesions adversely affect the ability of animals to update reward representations (reinforcer devaluation; see Hatfield et al., 1996) or form differential associations between neutral and motivationally significant stimuli (appetitive Pavlovian second-order conditioning; see Setlow et al., 2002; Pavlovian-to-instrumental transfer; see Blundell et al., 2001). These learning paradigms require animals to use multiple distinct S-O representations to adapt behavior. The BLA appears to be a critical component for such a task.

Role of the OFC in Reward Expectancies

In the current experiment, OFC-lesioned rats, which performed similar to sham-lesioned controls, displayed the DOE in the early phase, with DOP trained rats reaching 65%−70% accuracy by Session 10 and NOP rats reaching 55%− 60%. In contrast to the performance of sham-lesioned rats, choice accuracy in OFC-lesioned rats trained with the DOP reached their peak performance levels by Session 11 but then showed a conspicuous decline in performance for the next eight sessions that was not observed under NOP conditions. These results suggest that after the distinct S-O associations are formed, lesions to the OFC then disrupt the ability to use this information in an adaptive behavioral manner.

The present findings provide further support for a role of the OFC in processing reward-related information on a conditional discrimination task with the DOP embedded (see McDannald et al., 2005). However, our pattern of results differs from McDannald et al.'s (2005) results, which can be attributed to differences in training procedures. Despite these differences, OFC-lesioned rats in McDannald et al.'s study displayed a blunted DOE throughout the period of time after the rats learned the discrimination rule, just as the decreased ability to use unique reward information in the OFC-lesioned rats of the present study appears immediately following peak performance around Session 10. Our data suggest that the OFC is not essential for establishing the predictive relationship between the specific reward and the antecedent stimulus conditions.

Role of the NA in Reward Expectancies

There are two general theories of NA function: (a) It mediates how Pavlovian conditioned reward expectancies stimulate behavior (incentive motivation or salience), and (b) it is part of the neural circuitry enabling animals to attach incentive value to the representations of instrumental-contingent rewards (de Borchgrave, Rawlins, Dickinson, & Balleine, 2002). These two functions may map onto data demonstrating that the core and shell subdivisions of the NA are functionally and structurally distinct (see Corbit et al., 2001; Zahm & Brog, 1992). The core appears to mediate the associative processes relating the incentive value of outcomes with goal-directed actions (bidirectional R-O associations; see Corbit et al., 2001; Kelley et al., 1997). In contrast, the shell may be more involved in mediating S-O associations into goal-directed performance—as tested by the Pavlovian-to-instrumental transfer paradigm with multiple reinforcers (Corbit et al., 2001).

Our NA lesions encompassed the core and spared the great majority of the shell. In the present study, rats with NA-core lesions displayed an enhanced rate of acquisition on a simple conditional discrimination only when trained with the DOP compared with sham-lesioned controls. NA-core-lesioned rats trained with the DOP reached their peak accuracy at 75%−80% by Session 6, which is comparable to sham-lesioned rats at Session 20, and remained at this level for the duration of behavioral testing. In contrast, under the NOP condition, lesions to the NA-core had no effect on performance. These findings demonstrate that the NA-core is not critical for this type of S-R learning. It has been suggested that NA-core dysfunction does not impede the ability to learn a contingency or to determine changes in either contingency or motivational value; rather, the deficit becomes apparent in modulating subsequent behavior to reflect these changes (Balleine & Killcross, 1994; Cardinal & Cheung, 2005; Corbit et al., 2001). Previous findings demonstrated that NA-core lesions resulted in a moderate learning facilitation for the initial discrimination— but not subsequent discriminations—in a serial discrimination task in which distinct odor cues predicted either appetitive or aversive outcomes (Schoenbaum & Setlow, 2003).

There is also evidence that general NA lesions do not affect the development of cue–reward relationships (S-O) and detection of changes to this association (Balleine & Killcross, 1994). Along similar lines, Cardinal and Cheung (2005) further demonstrated that rats with NA-core lesions are more sensitive to differences in reward magnitude than are their control counterparts. Comparisons of concurrent responding on schedules with either a high or low concentration of sucrose pellets indicated that NA-core-lesioned rats displayed enhanced sensitivity to the difference between the distinct rewards.

Our results suggest that the NA-core is not necessary for the use of distinct S-O associations to guide instrumental behavior. In fact, it appears that an intact NA-core can interfere with this process. The occurrence of an enhanced DOE in NA-core-lesioned rats may stem from the presence of an intact BLA and OFC integration; the NA-core circuit may process other reward information in parallel. Removal of this process may produce a more rapid learning of S-O associations. However, if the experimental parameters were to change, for example, reversing the cue-specific reward relationship, the absence of NA-core processing may become evident.

Conclusions

Although reward expectancies can influence instrumental behavior, different associations could be modulating the enhanced behavior under the DOP condition. For example, reward expectancies can be formed via S^D–outcome associations or from bidirectional R-O associations. A limitation of our study was that we were unable to determine which type of association was controlling behavior. However, previous research does shed some light on this issue. Blundell et al. (2001) used an extinction procedure after the discrimination training to examine this issue. They found that during the extinction phase—when no reward was delivered—the overall patterns of group differences (sham-lesioned rats displayed the DOE, whereas BLA-lesioned rats did not) shown during acquisition were maintained (Blundell et al., 2001). Similarly, Mc-Dannald et al. (2005) examined their data prior to the delivery of the first outcome, and these preoutcome data were similar to those from the entire stimulus duration (there was a DOE effect in shams, but not in rats with OFC lesions). Both of these measures demonstrated that without the direct experience of the outcome, sham animals display the DOE to the S^D alone, whereas animals with lesions to either the BLA or OFC do not (see Figure 3 of Blundell et al., 2001, and Figure 4 of McDannald et al., 2005).

Both of these studies support other behavioral data suggesting that under certain training conditions S-O associations are sufficient to guide instrumental behavior (for a review, see Urcuioli, 2005). Our behavioral procedure was a replication of the discrimination task used by Blundell et al. (2001), thus one can extrapolate that similar cognitive–behavioral processes occur across the studies. However, regardless of the exact associations (S-O or R-O) used to enhance behavior with the DOP, the data from the present study, along with the data from Blundell et al. (2001) and McDannald et al. (2005), suggest that the BLA and the OFC, respectively, are important for the development and maintenance of reward expectancies produced by differential reward of stimuli and actions.

Our data also suggest that the NA-core is not needed for, and may impede, the development of reward expectancies that are potentially created by S-O associations, under the present experimental conditions. The current experiment was the first to assess the role of the NA-core using the DOP. As carefully documented by Urcuioli (2005), what may appear as minor differences in training protocols (such as whether differentially reinforced responses are trained in different sessions or discrete S^D are used to signal behavioral choice) can change the type of outcome expectancy (S-O or R-O) that gains stimulus control. Most of the studies that have examined R-O associations in rats trained with a DOP-like procedure have not used discrete S^D, and/or each differentially reinforced response was trained separately in different sessions (see Urcuioli, 2005). Such parameters have been used to study the role of thalamus and the medial prefrontal cortex in R-O learning (see Corbit, Muir, & Balleine, 2003; Ostlund & Balleine, 2005).

It is possible that under different training conditions—those that are more likely to evoke R-O expectancies—the NA-core may be involved. To date, it has not been determined whether the functional distinctions between the NA-core and NA-shell could be dissociated when trained with the DOP under various parameter conditions that selectively evoke the use of S-O or R-O expectancies. Thus, the manipulation of behavioral parameters may be powerful tools for understanding brain function as it relates to the acquisition and maintenance of reward expectancies.