A visual, position-independent instrumental reinforcer devaluation task for rats

Introduction

Animals depend on the ability to guide behavior in order to achieve specific goals e.g., outcomes or consequences. Because a behavior that at one time produced a positive outcome, can at a later time produce a negative outcome, it is important to adjust behavior to adapt to changes in consequences. This flexibility allows rapid alterations in behavior in the face of changing outcomes, conferring a survival advantage. This type of flexible goal-directed behavior is enabled by the integration of frontal and limbic neural circuitry.

An example of goal-directed behavior is food seeking. The natural tendency to avoid heavily consumed foods in order to seek relatively novel foods ensures that animals will consume a diversity of foods, thereby increasing the probability of balanced nutritional intake. By extension, a non-edible cue associated with the heavily consumed food changes in value. This process is studied in a laboratory setting using a “Conditioned Reinforcer Devaluation” task.

Such tasks have been used extensively to study flexible goal-directed behavior in multiple species of laboratory animals (Balleine et al., 2003; Corbit and Balleine, 2005; Hatfield et al., 1996; Izquierdo et al., 2004; Johnson et al., 2009; Malkova et al., 1997; Nelson and Killcross, 2006; Ostlund and Balleine, 2008, 2007; Pickens, 2008; Pickens et al., 2005; Pickens et al., 2003). Initial training on these tasks aims to instantiate associations between cues (e.g. a specific object or sound) and specific primary reinforcer(s) (e.g. a specific food or type of juice). A specific primary reinforcer is subsequently “devalued” either by selective satiation (feeding the specific food to satiety) or inducing a taste aversion (causing internal illness following consumption of the specific food). Following devaluation, subjects adjust their responding to the cue in a way that reflects the new “value” of the reinforcer, i.e., they reduce their selection of the cue that predicts devalued reinforcer. The recalibration of the cue does not require additional pairings of the cue with the devalued reinforcer. Instead, the animals must recall which cue predicts the food and integrate that information with the updated value of the food.

The conditioned devaluation sequence described above depends upon the integrity of the basolateral subdivision of the amygdala (BLA) (Balleine et al., 2003; Hatfield et al., 1996; Johnson et al., 2009; Malkova et al., 1997; Wellman et al., 2005), the orbitofrontal cortex (OFC) (Izquierdo and Murray, 2004; Izquierdo et al., 2004; Pickens et al., 2005; Pickens et al., 2003), the mediodorsal thalamus (MDT) (Izquierdo and Murray; Mitchell et al., 2007; Ostlund and Balleine, 2008; Pickens, 2008), and the connections between OFC and BLA (Baxter et al., 2000). While each of these neural substrates appears critical for the process, there are inconsistencies across studies with respect to the differential impairment associated with compromised function of amygdala, OFC and MDT (Balleine et al., 2003; Hatfield et al., 1996; Johnson et al., 2009; Malkova et al., 1997; Ostlund and Balleine, 2008, 2007; Pickens, 2008; Pickens et al., 2005; Pickens et al., 2003; Wellman et al., 2005).

The neural substrates that are required for conditioned reinforcer devaluation may depend upon the modality of the conditioned reinforcers (e.g., spatial versus non-spatial) and/or the nature of the responses required (i.e., instrumental versus pavlovian). Tasks using distinct types of cues or responses have revealed relationships between task-specific demands and neural substrates (Johnson et al., 2009; Ostlund and Balleine, 2008; Pickens et al., 2003). Currently, we lack an instrumental reinforcer devaluation task for rats that is independent of spatial cues. The ability to resolve the task using spatial cues may be a critical factor that determines the sensitivity to OFC lesions. It is possible that the absence of spatial cues in the reinforcer devaluation task used in monkeys may account for the OFC-dependent nature of conditioned reinforcer devaluation in the monkey, which contrasts the OFC-independent nature of instrumental conditioned reinforcer devaluation in the rat (Izquierdo et al., 2004; Ostlund and Balleine, 2007). In contrast to instrumental tasks, the performance in pavlovian conditioned reinforcer devaluation tasks in rats is dependent on OFC (Gallagher et al., 1999; Pickens et al., 2003, 2005).

Instrumental tasks that have been used to test, conditioned reinforcer devaluation in the rat can be performed using positional cues, e.g., left and right levers (Balleine et al., 2003; Johnson et al., 2009; Ostlund and Balleine, 2008, 2007) whereas the standard task for monkeys requires the use of visual cues only, with position being irrelevant. Because different cortical regions have been implicated in task performance depending on whether the secondary reinforcer is positional or visual (Rushworth et al., 2007a; Rushworth et al., 2007b), the nature of the cues used in a task are likely to determine behavioral vulnerability to damage of particular structures. For example, OFC, which receives information about stimuli, may be important for reinforcer devaluation tasks when an animal must learn to associate a specific reinforcer with a visual or auditory stimulus, but not with a spatial location (Rushworth et al., 2007a; Rushworth et al., 2007b).

During the training phase of the standard conditioned reinforcer devaluation task used in monkeys, a series of object discrimination trials are presented; in each trial, one object is baited with a primary reinforcer (food) and the other object is unbaited. Different pairs of objects are presented in each trial. Over the course of multiple repetitions of the same series of object pairs, the subjects learn to remove the appropriate object to uncover the reinforcer (instrumental action). Half of the reinforced objects in a series are baited with one type of specific reinforcer (e.g., fruitsnack) while the other half of the reinforced objects are baited with an alternative reinforcer (e.g., peanut). The spatial position (left or right) of the reinforced objects is determined according to a pseudorandom sequence. During training, an association is formed between the reinforced objects and the primary reinforcer(s) those objects predict. During a probe session, subjects are presented with a series of trials, offering a choice between two objects. One object predicts one type of primary reinforcer (which has been devalued) and the other object predicts the alternative primary reinforcer. Under baseline conditions, the monkeys switch their preference away from the objects baited with the devalued reinforcer.

The goal of the present study was to design and implement a conditioned reinforcer devaluation task in rats for which spatial cues would be irrelevant. To accomplish this, we incorporated the following characteristics of the standard monkey task: 1) a visual-cue dependent discrimination 2) instrumental responses to obtain food and 3) cues independent of spatial position.

Here, we developed a rat task using two visual cues as secondary reinforcers for each of two different foods (primary reinforcers). As in the monkey task, the cues during the probe trial were presented in pairs, changing their left and right positions pseudo-randomly. During the training sessions, rats were trained to press a lever in response to the presentation of a visual cue. In between training and probe sessions, the rats were satiated on one of the two foods. We hypothesized that if the task was successful in detecting devaluation of the cues, the rats would make fewer responses to the cue associated with the devalued reinforcer than to the cue associated with the non-devalued reinforcer.

Materials and Methods

Subjects

Behavioral testing was conducted with 32 female Sprague-Dawley rats (Harlan, Indianapolis, Indiana) weighing approximately 250 g at the start of the study. The rats were pair housed in the animal vivarium at Georgetown University Medical Center. The animal rooms were climate controlled and illuminated on a standard light-dark cycle (light on from 6:00 A.M. to 6:00 P.M. For this study, the rats were food restricted to 85% of their pre-study body weight (10–15g chow/day) with water available ad libitum. Food restriction began 5 days prior to the start of behavioral training and continued throughout training and testing. The study was conducted under a protocol approved by the Georgetown University Animal Care and Use Committee and in accordance with the Guide for Care and Use of Laboratory Animals adopted by the National Institutes of Health.

Apparatus

Standard rat operant chambers (Habitest, Coulbourn Instruments, Whitehall, PA) were used for food-tray training, instrumental training, cue training, instrumental (cue) preference test and instrumental probe of reinforcer devaluation. Graphic State software (Coulbourn Instruments, Whitehall, PA) was used to program the training and testing sequences and to collect and record the rats’ responses. Each chamber contained two levers (Coulbourn H21-03R), with one light panel (Coulbourn H11-02R) located above each lever as shown in Figure 1. Each light panel consisted of three Light Emitting Diodes (LEDs) in a row. Each LED emitted a specific color light, but only two lights were used as cues in this experiment:

Figure 1.

Phase III: Cue Training. A) Presentation of Cue 1 (green light) signaled the active lever. A correct response (i.e., pressing the active lever) resulted in the delivery of a food reinforcer (sugar pellet). B) Presentation of Cue 2 (red light) signaled the active lever. Incorrect response (i.e., pressing on the inactive lever) resulted in no food delivery.

• Cue 1a green light on the left side of panel (λ=565nm) with a flash rate of 1/sec.
• Cue 2a red light on the right side of panel (λ=635nm) with a flash rate of 5/sec.

Both lights fall within the visible spectrum for rats (Jacobs et al., 2001). Thus color, flash frequency, and cue position on the light panel distinguished the two lights. A food tray was located in between the levers. Both levers were present throughout training and experimental conditions. A speaker delivered constant 70 dB white noise to the operant chamber. All training and testing was done in the absence of a houselight (except where otherwise indicated). Each operant chamber was located inside a standard isolation cubicle (Coulbourn H10-24).

Standard cages (10 X 7 X 5 inches) were used for selective satiation, consummatory preference test, and consummatory probe of reinforcer devaluation (see below). For these experimental procedures, food pellets were available ad libitum in a small Petri dish.

Food reinforcers

Rats were divided into two groups (16 rats/group). One group (Group GB) received two types of sucrose dustless precision pellets, grape flavored and banana flavored (45mg BioServ, Frenchtown, NJ). These two had identical nutritional content but different flavors. The other group (Group SC) received two different types of pellets, sugar dustless precision pellets and chocolate purified formula pellets (45 mg, BioServ, Frenchtown, NJ). These two were different from each other in both nutritional content and flavor. For each group, the choice of pellets remained constant across all experimental procedures.

Behavioral Training

Table 1 shows the order of behavior training and testing procedures. Training consisted of four phases. Within two of the phases (Phases II and III) there were different “levels” with progressively increasing schedules of reinforcement. The number of daily sessions in each level of training depended on each rat’s performance. Before starting instrumental training, the rats were given Petri dishes filled with both foods in their home cages. This procedure was used to expose the rats to each of the two types of pellets daily for one week. After this food pre-exposure period, the rats started the food-tray training sessions.

Table 1.

Training Schedule for Reinforcer Devaluation Task

Phase		Level	Training	Criterion	Average # of days to reach criterion	# of rats (%total) that completed the level
I	Food Tray Training		One pellet delivered every 15 s in the food tray	50 entries into food try and all pellets consumed	1.5 ± 0.18	32 (100%)
II	Instrumental Training	1	FR1 schedule with reinforcement on both levers	50 reinforced trials /session	3.5 ± 0.31	32 (100%)
		2	FR3 schedule with reinforcement on both levers	50 reinforced trials /session	1.8 ± 0.23	32 (100%)
		3	FR5 schedule with reinforcement on both levers	50 reinforced trials /session	1.2 ± 0.09	31 (97%)
III	Cue Training	1	Cue above each of the levers, one at a time- FR5* (one minute)	100 responses on each lever	1.7 ± 0.28	31 (97%)
		2	Cue above each of the levers, one at a time - FR5 (30 s)	3 days with >200 responses on each lever on the third day	3.1 ± 0.08	30 (94%)
		3	Cue above each of the levers, one at a time - VR9 (30 s)	200 responses on each lever	1.2 ± 0.08	30 (94%)
		4	Cue above each of the levers, one at a time - VR9 (30 s) with a timeout for incorrect responses	85% correct over 3 days	12.3 ± 0.7	20 (63%)
IV	Cue - choice training		Both cues presented simultaneously (VR9) reinforced with specific foods (15s)			20 (63%)

All averages ± Standard Error of the Mean (SEM)

Phase I- Food-tray training

In Phase I, the rats were trained to approach the food tray in order to receive a food pellet without an instrumental response required. In the operant chamber, one reinforcer was delivered every 15 seconds, alternating with the second reinforcer. The session included 25 deliveries of each specific reinforcer and lasted 15 minutes.

Phase II- Instrumental training

In Phase II, the rats were trained to press a lever in order to receive a food pellet. Each daily session consisted of 25 deliveries of each specific reinforcer (banana and grape pellets for group GB, and sugar and chocolate pellets for group SC). This phase of training consisted of three levels. The duration of each session was limited to 30 minutes. In level 1, the rats were trained on a fixed ratio schedule where one response on either lever resulted in the delivery of one food pellet (FR1). Criterion was set at the delivery of 50 food pellets (25 pellets of each reinforcer) for each rat per session to move to the next level. In Levels 2 and 3, the rats were trained on fixed ratio schedule where every third (FR3) and every fifth (FR5) response, respectively, on either lever was reinforced with a food pellet. The criterion was the same as in level 1. Once the rats reached criterion on an FR5 schedule, the rats began cue training.

Phase III- Cue training

In Phase III, the rats were trained to press a lever with either Cue 1 or Cue 2 flashing above it, in order to receive a food pellet as depicted in Figure 1. This phase of training consisted of four levels. The duration of each session was limited to 20 minutes. In level 1, the rat had to respond by pressing a lever with the cue flashing above it (active lever) on a FR5 schedule in order to receive a food reinforcer (Figure 1a). There was no cue presented above the inactive lever. If the rat pressed the inactive lever, no food was delivered (Figure 1b). The trial was terminated after one minute or by a pellet delivery. There was a 5s inter-trial interval between trials. Lever presses in response to Cue 1 consistently resulted in the delivery of a grape pellet for group GB (or sugar pellet for group SC), whereas responses to Cue 2 consistently resulted in the delivery of a banana pellet (or chocolate pellet). This phase of training aimed to enable the formation of an association between the specific cue and a particular food reinforcer (e.g., Cue 1 with a grape pellet). The left-right location of the active lever followed a pseudo-random order between trials and was balanced across trials. After the rat responded at least 100 times on each lever within a session, it moved to the next level. In level 2, the rat was allowed 30 sec to respond 5 times (FR 5) on the active lever; at least 200 responses per session for at least 3 sessions were required to proceed to the next level. In level 3, the schedule switched to a variable ratio (VR9), with pellets delivered in response to between 5 and 13 lever presses. After the rats responded more than 200 times on each lever within a session, the final training level was initiated. In level 4, the rat was allowed 30 sec to respond on the active lever with reinforcement delivered on a VR9 schedule. Responses on the inactive lever resulted in a 10s timeout period signaled by illumination of the house light. Thus, there were three possible outcomes for each trial: reinforcer delivery, omission (the rat did not respond enough times to receive a food pellet in 30 s), or a timeout (the inactive lever was pressed). Criterion for advancement to the next training phase (cue choice training) was set at 85% correct responses over three days. The percent correct responses were determined by the number of food deliveries divided by the total number of trials, expressed as a percentage. Upon reaching the criterion of 85% correct, the SP rats moved to Phase IV.

Phase IV- Cue choice training

In Phase IV, the rats were simultaneously presented with Cue 1 over one lever and Cue 2 over the other lever. In this stage, rats could press either one of the two levers while both cues were presented and reinforced with the particular reinforcers. The purpose of this phase was to expose the rats to a condition in which both cues were presented simultaneously as would take place subsequently in the instrumental probe during extinction. During each session, the cues over both levers were flashing, with the left-right positions of the specific pellets and their cues following a pseudo-random order between trials. The rat was allowed 15 seconds to respond on either lever to receive a food pellet on a VR9 schedule. All sessions were limited to 15 min.

Instrumental (cue) preference test

During the last Phase IV session, the number of lever presses for each cue was recorded and analyzed to determine whether the rats showed a preference for one of the two cues associated with the specific foods.

Consummatory preference test

The consummatory preference test was used to determine if there was a difference in food preferences between the food pairs in the absence of secondary reinforcers. Each rat was given access to two Petri dishes, one filled with 10g of Food 1 (e.g. grape pellets or sugar pellets) and other filled with 10g of Food 2 (e.g., banana or chocolate pellets). Rats were given 30 min to consume the food. The amount of each type of food pellets consumed was measured by subtracting the food remaining from the original amount.

Selective satiation

For selective satiation, each rat was placed in a standard cage, was presented with a Petri dish filled with 15g of one of the two foods, and was allowed to eat for 15 minutes. The amount of food each rat consumed was measured by weighing the food remaining in the Petri dish after 15 min. Rats received four selective satiations; 2 satiations (1 per food) were followed by a consummatory probe and the other 2 satiations (1 per food) were followed by an instrumental probe. The satiations were separated by at least 3 days.

Consummatory probe of reinforcer devaluation

The consummatory probe was used to determine the extent to which a specific food was devalued following selective satiation. To test the devaluation, 15 minutes after the selective satiation, each rat was given access to two Petri dishes, one filled with 10g of the satiated food (e.g. chocolate pellets) and the other filled with 10g of the non satiated food (sugar pellets). Rats were given 15 min to consume the food. The amount of each type of food pellets consumed was measured by subtracting the food remaining from the original amount. At least 3 days later, the rats were satiated on the other food. The total amount of devalued (satiated) food pellets (in grams) consumed after each devaluation was then summed for each rat across the two sessions and compared with the total amount of non-devalued food pellets consumed across the two sessions. Rats that consumed more non-devalued (non-satiated) food pellets than the devalued (satiated) food pellets by at least 1g, were considered to show devaluation.

Phase V- Instrumental probe of reinforcer devaluation

This was the final phase of testing, during which we evaluated whether the devaluation of the primary reinforcer was transferred to the particular cue. Successful transfer would manifest as fewer lever presses to the cue associated with the devalued reinforcer (Cue_D) than the cue associated with the non-devalued reinforcer (Cue_ND). In order to prevent additional conditioning from occurring during testing no pellets were delivered during the probe trials. Following selective satiation with one of the food reinforcers (e.g. chocolate pellets), the rats were given a 5 min instrumental probe during which the cues above both levers were illuminated simultaneously with one lever below Cue 1 and the other below Cue 2 counterbalanced across the lever positions. Rats were allowed 15 sec to respond in each trial. The total number of times the rat pressed the lever associated with each cue (Cue_D or Cue_ND) was recorded. Subsequently (at least 3 days later), the alternative reinforcer (e.g. sugar pellets) was devalued and the rat was tested again on the instrumental probe. The total number of responses in both instrumental probes following selective satiation with each reinforcer were summed across the two instrumental probe tests. In addition, the cumulative number of responses on the lever associated with each cue was measured across the session (trials 1–10) and summed across the two instrumental probe tests. Between the two instrumental probes, all rats were re-exposed to Phase III level 4 training as well as one session of Phase IV with food pellet delivery.

Data Analysis

All data analyses were done using SPSS software. Consummatory preference test was analyzed by two-way ANOVA with food choice (GB-grape versus banana; SC-sugar versus chocolate) as a within-subject variable and group (GB versus SC) as a between-subject variable. Instrumental (cue) preference was analyzed by two-way ANOVA with cue choice (Cue 1 versus Cue 2) as a within-subject variable and group (GB versus SC) as a between-subject variable. Consummatory probe was analyzed by two-way ANOVA with devaluation status (devalued versus non-devalued) as a within-subject variable and group (GB versus SC) as a between-subject variable. For the instrumental probe, the number of responses to Cue_D vs. Cue_ND was analyzed by two-way ANOVA with repeated measures with two within-subject factors: devaluation status and trial number (1–10). The cumulative number of responses (sum of trials 1–10) to the Cue_D and Cue_ND were analyzed using a paired t-test. Finally, a relationship between the magnitude of the devaluation effect in the consummatory probe (i.e., percent of total food eaten that was the devalued food) and the magnitude of the devaluation effects in the instrumental probe (i.e., percent of total instrumental responses that were to the Cue_D) was assessed by Spearman’s correlation coefficient.

Results

Table 1 shows the average number of days the rats took to reach criterion for each level and how many rats completed each level. The maximum number of sessions (across all levels) to reach criterion was set at 35. Two rats were eliminated before reaching level 4 of Phase III and 10 were eliminated because they did not reach the 85% criterion at level 4 within 35 days of total training. Thus, 20 out of 32 (63%) rats completed all 4 levels of Phase III.

Phase III- Cue Training (level 4)

Rats required an average of 12 sessions (range 7–17) to reach criterion on this level. To ensure that the rats had the same amount of experience in the operant boxes, training was continued until all rats, including those that already had reached criterion, completed a total of 35 sessions. With this requirement and depending on the performance on the previous levels, each rat completed at least 18 sessions at this level. These data are represented in Figure 2, which shows the mean percent correct over the first 18 trials.

Figure 2.

Cue training learning curve. The graph depicts the mean percent correct (+/− SEM) on Phase III level 4 in each of the first 18 training session, including the sessions where the rats reached criterion.

Consummatory preference test

Group by food-choice two-way ANOVA with repeated measures did not show significant main effect of the food choice (F_1,30=2.5; p>0.05) or group (F_1,30=0.0; p>0.05), but there was a significant effect of food-choice by group interaction (F_1,30=10.4; p<0.05). Figure 3a shows that the rats in group GB consumed significantly more of the grape pellets compared to the banana pellets (paired t-test: t=3.4 p<0.05). However, when the choice was between chocolate and sugar pellets there was no significant difference between these foods in the amount that the rats consumed of each in group SC (paired t-test: t=−1.1 p>0.05).

Figure 3.

A) Consummatory preference test. The bars represent the mean (+SEM) amount of each type of food consumed; grape and banana pellets (group GB) or sugar and chocolate pellets (group SC) *denotes a significant difference (p<0.05) between food choices (between grape and banana). B) Instrumental (cue) preference test. The bars represent the mean (+/− SEM) number of lever presses to the cue associated with each food; Cue 1 (predicts grape pellets) and Cue 2 (predicts banana pellets) for group GB and Cue 1 (predicts sugar pellets) and Cue 2 (predicts chocolate pellets) for group SC *denotes a significant difference (p<0.05) between Cue 1 (when it predicts grape) and Cue 2 (when it predicts banana).

Instrumental (cue) preference test

A total of 20 rats were tested on the instrumental preference test (11 rats in group GB and 9 in group SC). Group by cue-choice two-way ANOVA with repeated measures showed a significant main effect of the cue choice (F_1,18=16.7; p<0.05) and a significant cue-choice by group interaction (F_1,18=5.6; p<0.05). There was no significant effect of group (F_1,18=1.8; p>0.05). Figure 3b shows that the rats in group GB responded significantly more to Cue 1, which predicted grape pellets, compared to Cue 2, which predicted banana pellets (paired t-test: t=4.4 p<0.05). This result is consistent with the preference observed in the consummatory test. In group SC, there was no significant difference in the number of responses to Cue 1 as compared with Cue 2 (which predicted sugar and chocolate pellets, respectively; paired t-test: t=1.5 p>0.05); This result is also consistent with the data obtained in the consummatory test. (Figure 3 here please)

Consummatory probe of reinforcer devaluation

Twenty-six (81%) out of 32 (12 in GB and 14 in SC) rats showed a devaluation effect by eating less of the devalued food than of the non-devalued. Group by devaluation-status two-way ANOVA with repeated measures resulted in a significant main effect of the devaluation status, i.e., non-devalued food versus devalued-food (F_1,30=30.5; p<0.05). Thus, following devaluation, rats consumed significantly less of the food on which they had just been sated (devalued food) as compared with the food on which they were not sated (non-devalued food). There was neither a significant main effect of group (F_1,30=2.616; p>0.10) nor a devaluation status by group interaction (F_1,30=1.083; p>0.05). Figure 4 shows that within each group the rats consumed significantly less of the devalued pellets than of the non-devalued pellets (Group GB; t=−2.833, p<0.05; Group SC; t=−5.35, p<0.001). These results confirmed that both types of pellets were suitable for reinforcer devaluation and that both groups of rats could be used for further behavioral testing in the instrumental probe. (Figure 4 here please)

Figure 4.

Consummatory probe of reinforcer devaluation. The bars represent the mean (+/− SEM) amount of each type of food consumed after devaluation by selective satiation; black bars represent the devalued food and gray bars the non-devalued food. Grape vs. Banana: the group (GB) that received grape and banana food pellets, Sugar vs Chocolate: the group (SC) that received sugar and chocolate food pellets. *denotes a significant difference (p<0.05) between the devalued and non-devalued food within each group. No significant difference between groups was found.

Instrumental probe of reinforcer devaluation

When no food pellets were delivered, the rats continued responding throughout the first 10 trials, after which most rats stopped responding. Therefore, we used only the first 10 trials for the analysis. A total of 20 rats (i.e., those that reached criterion) were tested on the instrumental probe of reinforcer devaluation (11 rats in group GB and 9 in group SC). Of the 20 rats, two did not show devaluation in the consummatory probe, two did not respond on the levers during the instrumental test, and one rat only hit one lever throughout testing. Therefore only data from the remaining 15 rats were analyzed (8 rats in group GB and 7 in group SC). Figure 5a shows the average number of lever presses to Cue_D (devalued) versus Cue_ND (non-devalued), in each of the first 10 trials within a session. Figure 5b shows the cumulative number of lever presses in each of the first 10 trials. A group by devaluation- status analysis showed no significant effect of group. Therefore the data were collapsed across groups. ANOVA revealed a significant main effect of devaluation status (F_1,14=7.1; p<0.05). These data show that following devaluation, rats responded significantly less on the lever corresponding to the Cue_D compared to the lever corresponding to Cue_ND (paired t-test: t=−2.66 p<.05). There was also a significant main effect of trial number (F_9,126=7.1; p<0.05). Rats responded more on the last 5 of the 10 trials. There was no trial by devaluation-status interaction (F_9,126=3.5; p>0.05).

Figure 5.

Instrumental probe of reinforcer devaluation. A: Each curve represents the mean (+/− SEM) number of lever-presses per trial across the first 10 trials. Gray squares represent responses to the cue associated with the non-devalued food, black circles represents responses to the cue associated with the devalued food. B: Each curve represents the mean (+/− SEM) number of cumulative lever-presses per trial across the first 10 trials. White squares and black circles are the same as in A.

Figure 6 shows the total number of responses across the first 10 trials to Cue_D (26.2 +/− 3.1) and Cue_ND (44.4 +/− 6.7). These results suggest that the rats associated each of the two reinforcer with the particular cue and responded to it regardless of the lever position (left-right). In addition, this final phase of the task demonstrated that after satiation on the primary reinforcer, the rats successfully updated the value of the cue to reflect the new value of the primary reinforcer and accordingly reduced responding in the presence of that cue. (Figure 6 here please)

Figure 6.

Instrumental probe of reinforcer devaluation. The bars represent the mean (+/−SEM) cumulative number of lever presses across the first 10 trials. Gray bar represents responses to the cue associated with the non-devalued food (Cue_ND), black bar responses to the cue associated with the devalued food (Cue_D). *denotes a significant difference (p<0.05) between the responses to Cue_ND and Cue_D.

Correlation between Consummatory Probe and Instrumental Probe Performance

A significant correlation (R=0.6, p<0.05) was found between the magnitude of the devaluation effect detected by the instrumental probe (i.e., percent of instrumental responses that were in response to Cue_D) and the magnitude of the devaluation effect detected by the consummatory probe (i.e., percent of food eaten that was the devalued food) as shown in Figure 7. This result suggests that the degree to which individual rats shift food choices in the consummatory probe following devaluation transfers to the cues used in the instrumental probe.

Figure 7.

Instrumental probe and consummatory probe correlation. The percent of responses to the cue associated with the devalued cue (Cue_D) in the instrumental probe compared with the percent of the devalued food consumed in the consummatory probe test for each rat in the two different tests show a significant linear relationship (R²=.6, p<0.05).

Discussion

The results presented here support our hypothesis that the animals will make fewer responses to the cue associated with the devalued reinforcer than to the cue associated with the non-devalued reinforcer. These results indicate that the new value of the reinforcer was successfully transferred to the visual cue and was used to guide the instrumental choices. We also showed that selective satiation of two distinct solid foods can be successfully employed as a devaluation procedure in rats comparable to that previously seen with two distinct liquid foods (Johnson et al., 2009). This task, analogous in specific features to the monkey task, can now be used to further reconcile results across laboratories with respect to the neural substrates involved in the individual phases of goal-directed behavior.

We have further demonstrated that rats can discriminate two concurrent visual cues that predict different foods. Importantly, in our new task, this discrimination is not aided by spatial cues, thereby eliminating a major difference between the standard instrumental tasks currently used in rats and the task used in monkeys. In the instrumental reinforcer devaluation tasks previously used in rats (Balleine et al., 2003; Johnson et al., 2009; Ostlund and Balleine, 2008, 2005, 2007), responses are directed towards one of two spatially fixed levers. In the task we have developed, the positional cues are replaced with position-independent visual cues.

The Sprague Dawley rats that were used in our devaluation study completed training within an average of 26 daily sessions (range 20–34), well within the range of the published experiments by Johnson et al. (2009) and Ostlund and Balleine (2007) that reported an average of 22 and 34 sessions, respectively, using spatial cues. Thus, the absence of positional cues does not increase the amount of training required. However, the absence of positional cues probably makes our task more difficult for albino rats, which have poor visual acuity (Jacobs et al., 2001; Szel and Rohlich, 1992). This may account for the relatively high rate of attrition during the stage of training when the visual discrimination is introduced (10 out of 30 did not successfully reach criterion on this stage). Supporting this possibility, we have more recently obtained training statistics on Long-Evans pigmented rats, which we have been using for a lesion study in progress. These rats (see Supplemental Table 1) completed training with an average of 25 daily sessions (range 15–33), and all but 5 rats (out of 36) reached criterion in the final stage of cue training (West et al., 2010).

Our task is the first reinforcer devaluation task to utilize two solid foods that are identical in size, texture, and nutritional content. Most previous studies in rats have used either two liquid foods or a solid and liquid combination (Balleine et al., 2003; Corbit and Balleine, 2005; Corbit et al., 2001; Johnson et al., 2009; Ostlund and Balleine, 2008, 2005, 2007). Our data show that distinct flavors are sufficient to support a robust devaluation effect. Moreover, rats also exhibited a consummatory and instrumental preference for one flavor (grape) over the other (banana), verifying the distinction of the stimuli. These observations in rats are consistent with those in monkeys, as several studies reported that monkeys often strongly prefer one type of solid food reinforcer over the other (Baxter et al., 2000; Machado and Bachevalier, 2007; Malkova et al., 1997; Thornton et al., 1998; Wellman et al., 2005), yet still show a clear devaluation effect for either.

The results during the instrumental probe revealed a similar devaluation effect as in the consummatory probe. Moreover, across rats the extent of the shift of responses to the non-devalued food in the absence of secondary reinforcers is positively correlated with the extent of the shift of responses to the cues.

This task now serves as a basis to compare the neural substrates of goal-directed behavior within species across tasks, as well as, across species within analogous tasks. In particular, we can use this task in attempt to reconcile certain inconsistencies across studies (Balleine et al., 2003; Hatfield et al., 1996; Johnson et al., 2009; Malkova et al., 1997; Ostlund and Balleine, 2008, 2007; Pickens, 2008; Pickens et al., 2005; Pickens et al., 2003; Wellman et al., 2005) with respect to the necessary involvement of amygdala, OFC and MDT for conditioned reinforcer devaluation in rats. Moreover, although there are several differences between the task for rats and that for monkeys including the use of extinction during testing and the use of only two cues in rats, the task we have described for rats eliminates a major difference by introducing the use of position-independent visual cues.

Research Highlights.

• Rats can discriminate two concurrent visual cues that predict different foods

• Discrimination is not aided by spatial cues

• This eliminates a major difference between rat and monkey tasks

• Rats devalue two foods by satiation and transfer the new value to the visual cues

• Rats adjust responses by choosing the devalued cue less than the non-devalued