Extinction of Goal-Tracking Also Eliminates the Conditioned Reinforcing Effects of an Appetitive Conditioned Stimulus

Abstract

Previous studies have suggested that the effects of extinction are response-specific. The present study investigated whether an extinction treatment that eliminated goal-tracking elicited by an appetitive conditioned stimulus (CS) would also eliminate the conditioned reinforcing effects of that CS. Rats were first trained on a goal-tracking procedure where an auditory CS was paired with a food unconditioned stimulus. Animals learned to approach the location where the food was delivered. In a subsequent phase, rats in one group received extinction training that eliminated goal-tracking elicited by the CS. Rats in the other group did not experience extinction of the food-paired CS. Then, both groups received a test for conditioned reinforcement where lever presses resulted in the brief presentation of the stimulus previously paired with food. This stimulus did not act as a conditioned reinforcer in the group where it was subjected to extinction, but did serve as a conditioned reinforcer in the group that did not experience extinction. These results indicate that the effects of extinction generalize from the approach-eliciting to the conditioned reinforcing effects of an appetitive CS.

In extinction, a conditioned stimulus (CS; e.g., a tone) is presented without the unconditioned stimulus (US; e.g., food) with which the CS was previously paired. This typically results in a reduction in the frequency or magnitude of the conditioned response (CR; e.g., salivation) elicited by the CS. There are several potential explanations for how extinction produces this decrease in conditioned responding. Perhaps the most intuitive of these is that extinction erases or weakens the CS-US association (Rescorla & Wagner, 1972). Findings such as spontaneous recovery (e.g., Brooks & Bouton, 1994), reinstatement (e.g., Rescorla & Heth, 1975), and context renewal (e.g., Bouton & Bolles, 1979) are problematic for this view, however, since these are all instances where the CR reappears after extinction seemed to have eliminated the CS-US association. An alternative view is that extinction involves the learning of a new inhibitory CS-US association (or CS-no US association) while the original CS-US association is preserved (Bouton, 1994, 2004; Pavlov, 1927). After extinction, presentation of the CS activates the inhibitory CS-US association (or CS-no US association) and therefore the CS does not elicit the CR. However, certain events (e.g., passage of time, change in context) can disrupt or prevent activation of this inhibitory association and the intact original CS-US association is revealed through a reappearance of the CR.

A third view of how extinction reduces conditioned responding is that an inhibitory CS-CR association is learned during extinction. Several studies have supported this view by showing that the effects of extinction are response-specific (e.g., Bonardi, 1989; Delamater, 1996; Rescorla, 1993). For example, Delamater (Exp. 1, 1996) first trained rats on a procedure where two CSs, a noise and a light, were each separately paired with a food or sucrose US. After repeated pairings, both CSs came to elicit a magazine approach (goal-tracking) CR. Then, the CR elicited by one of the CSs was eliminated through extinction (i.e., the CS was presented without its US), while the other CS was not subjected to extinction. Surprisingly, there was no difference between the two stimuli on a transfer-of-control test where each CS was superimposed on an operant baseline of lever pressing or chain pulling (for food or sucrose). That is, each CS was equally effective in facilitating operant responding, regardless of whether it had been previously extinguished or not. This and similar results led to the proposal that CS-US associations are preserved through extinction and the observed decrease in CR frequency is the result of an inhibitory stimulus-response association that is response-specific (Colwill, 1991; Delamater, 1996; Rescorla, 1993, 1996). The present experiment sought to determine whether similar response-specificity of extinction would be observed when an extinguished CS that formerly elicited an appetitive CR was tested for its conditioned reinforcing properties.

Rats were first trained on a goal-tracking procedure where an auditory CS was paired with a food US. After the goal-tracking response was established, rats were assigned to one of two groups. In a second phase, the Extinction (Ext) Group received extinction training (the CS was no longer paired with food) that eliminated goal-tracking elicited by the CS. The No Extinction (No Ext) Group did not experience extinction of this CS. Then, a test of the conditioned reinforcing effects of the CS was administered to both groups. Evidence that extinction is response-specific would be provided if the previously food-paired CS served as an effective conditioned reinforcer in both groups.

Method

Subjects

Twenty naïve adult male Long-Evans rats served as subjects. Rats were deprived to approximately 80% of their free-feeding weights (~350–450 g) throughout the experiment and were individually housed in stainless-steel hanging cages. They had free access to water and were fed approximately 15 g of laboratory rat chow following training sessions. The colony room where subjects were housed was on a 12:12 h light:dark cycle, with lights on at 08:00 h. Training sessions were conducted during the light phase. Throughout the experiment, rats were treated in accordance with the Guide for the Care and Use of Laboratory Animals (National Academy of Sciences, 1996) as well as the guidelines of American University's Institutional Animal Care and Use Committee (IACUC).

Apparatus

Experimental sessions were conducted in five Coulbourn Modular Test Cages (30.0 cm × 25.5 cm × 29.0 cm), whose front and rear walls and ceiling were aluminum. The two side walls were clear Plexiglas. Each chamber was equipped with two Coulbourn retractable levers located on the chamber's front wall. The left and right levers were positioned approximately 2 cm from the left and right side walls, respectively. Each lever measured 3.3 cm wide and, when inserted, extended approximately 1.5 cm into the chamber approximately 2.25 cm above the grid floor. Depressions of a lever in the extended state activated a microswitch that sent a −28 VDC input to the computer interface. Food was delivered via a Coulbourn food dispenser to a food receptacle (magazine) located inside an aperture in the front wall of the chamber between the two levers. A photobeam extended across this aperture approximately 0.5 cm inside the front chamber wall and was used to record magazine-approach responses. Three Coulbourn stimulus lights that signaled the start and end of a session were located in the center of the front wall approximately 10 cm above the floor. A speaker connected to a Med-Associates ANL 926 audio generator provided the tone (4000 Hz, approximately 80 dB) and white noise (approximately 73 dB) stimuli. Experimental procedures were controlled by Med-Associates software (Med-PC) running on a PC located in an adjacent room.

Procedure

Phase 1

Rats were first trained in a single session to eat from the food magazine on a procedure where the food US was presented according to a variable-time (VT) 180-s schedule (range: 150–210 s). Each US consisted of three food pellets delivered in rapid succession (0.2 s apart). There were 12 US presentations over the course of an approximately 36-min session. In addition, five food pellets were placed in the food magazine prior to the start of the session. The levers remained in the retracted position.

Rats were then trained on a procedure where one of the auditory CSs (the CS⁺) was paired with food. The other auditory CS (the CS⁻) served as a control stimulus and was not paired with food. The tone and the noise were counterbalanced over CS⁺/CS⁻ designations. CS⁺ trials lasted 15 s and terminated with the delivery of three food pellets. CS⁻ trials also lasted 15 s but did not end in food delivery. Inter-trial intervals lasted 90 s on average (range: 60–120 s) and were timed from the offset of one CS to the onset of the next CS. CS⁺ and CS⁻ trials alternated randomly with the restriction that there be no more than 2 consecutive trials of the same type. There were 18 presentations of each CS per session. Food-magazine photobeam breaks were used to measure the goal-tracking CR. Photobeam breaks were recorded but had no programmed consequences at any time. There were 6 sessions on this procedure. The levers remained in the retracted position.

Phase 2

Rats were then assigned to either the Ext Group or the No Ext Group. Group assignment was made with the goal of matching groups on CS⁺ and CS⁻ goal-tracking response rates. The numbers of subjects in each group that had the tone as CS⁺ and noise as CS⁻, and vice versa, were approximately counterbalanced. (Due to an equipment malfunction, two rats had to be excluded from the study. This resulted in the numbers of rats trained with tone as CS⁺ vs. noise as CS⁺ being 5 and 4, respectively, for the No Ext Group and 4 and 5, respectively, for the Ext Group.) The Ext Group received 7 sessions on an extinction procedure where the CS⁺ and the CS⁻ stimuli were presented without food. As previously, trials lasted 15 s and inter-trial intervals lasted 90 s on average (range: 60–120 s) and there were 18 trials of each type per session. As before, goal-tracking CRs were recorded, but had no consequences. The levers remained in the retracted position. The No Ext Group was trained on the same procedure except that CS⁺ trials were omitted. That is, no stimulus was presented during those periods where CS⁺ was scheduled to occur. Thus, during Phase 2, rats in the No Ext Group experienced the same amount of exposure to the chamber and to CS⁻ as the Ext Group did.

Conditioned Reinforcement Testing

Rats in both groups then received 3 sessions to test for the conditioned reinforcing effects of the CSs. Both levers were inserted into the chambers for the duration these sessions. A single lever-press response on the left lever turned on the noise for 1 s. A single press on the right lever turned on the tone for 1 s. Food was not presented at any time during conditioned reinforcement sessions. Sessions lasted for 60 min.

Results

Phase 1

The left-hand portion of Figure 1 presents mean (± SEM) goal-tracking responses per session for CS⁺ and CS⁻ during the 6 sessions of goal-tracking acquisition for both groups. As the figure illustrates, rats' mean CS⁺ responses rose to approximately 120 by the end of the phase, while CS⁻ responding was approximately one third that amount. A 2 × 6 × 2 (Stimulus X Session X Group) repeated measures ANOVA indicated that there was a significant Stimulus X Session interaction (F[5,80] = 15.9, p < 0.001) as well as a significant effect of Stimulus (F[1,16] = 35.7, p < 0.05), but there was no significant effect of Group or significant interactions involving the Group variable (all F's < 1).

Figure 1.

The left portion of the figure presents mean (± SEM) goal-tracking responses per session for CS⁺ and CS⁻ for the No Ext and Ext Groups during the conditioning phase. The right-hand portion presents mean (± SEM) CS⁺ and CS⁻ goal-tracking responses during Phase 2.

Phase 2

The right-hand portion of Figure 1 presents mean (± SEM) goal-tracking responses during the 7 sessions of Phase 2. Mean CS⁺ responding significantly declined over sessions for the Ext Group (the No Ext Group did not experience CS⁺) and on the final session of this phase, rats made fewer than 10 total CS⁺ responses. A repeated measures ANOVA performed on CS⁺ responding confirmed that the effect of Session was significant (F[1,8] = 6.6, p < 0.05). A paired-samples t-test indicated that there was no difference between CS⁺ and CS⁻ responding for the Ext Group on the final session of the phase (t[8] = 0.1, p > 0.9).

Conditioned Reinforcement Testing

Figure 2 presents the mean (± SEM) total number of lever presses made on the levers associated with CS⁺ and CS⁻ over the 3 conditioned reinforcement test sessions. Rats in the No Ext Group made twice as many responses on the CS⁺ lever as they did on the CS⁻ lever. In contrast, rats in the Ext Group made a similar number of responses on both levers. A 2 X 2 (Lever X Group) repeated measures ANOVA confirmed that there was a significant Lever X Group interaction (F[1, 16) = 5.1, p < 0.05). Subsequent paired-sample t-tests (Bonferroni corrected) indicated that rats in the No Ext Group responded significantly more on the CS⁺ lever as compared to the CS⁻ lever (t[8] = 3.9, p < 0.005), but there was no difference in responding over levers for the Ext Group (t[8] = 0.2, p > 0.8).

Figure 2.

Mean (± SEM) total number of responses made during the 3 conditioned reinforcement test sessions for the No Ext and Ext Groups. White and black bars represent responding on the CS⁺ and CS⁻ levers, respectively. Asterisk indicates p < 0.005.

Discussion

In the present experiment, rats in the Ext Group received during Phase 2 an extinction treatment that eliminated goal-tracking elicited by a stimulus (CS⁺) previously paired with food. On a subsequent test for conditioned reinforcement, this group did not press a lever to turn on this stimulus any more than they pressed a lever to turn on a control stimulus (CS⁻) that was never paired with food. In contrast, the No Ext Group, that did not receive extinction of the food-paired stimulus during Phase 2, pressed a lever to turn on this stimulus at double the rate that they pressed a lever to turn on the control stimulus. These results indicate that the effects of extinction generalize from a goal-tracking CR elicited by a formerly food-paired CS to the conditioned reinforcing properties of that CS.

The present outcome contrasts with those of previous studies (e.g., Delamater, 1996) suggesting that extinction produces an inhibitory stimulus-response association that is specific to a particular response. Instead, the present results are consistent with the more common view that extinction decreases the net excitatory strength of the CS. This reduction in net excitation may occur through a direct weakening of the excitation conditioned to the CS (e.g., Rescorla & Wagner, 1972) or through the learning of an inhibitory CS-US association in addition to the originally learned excitatory CS-US association (e.g., Bouton, 1994, 2004; Pavlov, 1927).

The idea that a response controlled by a CS will only be observed if the CS's net excitation surpasses a threshold (Weidemann & Kehoe, 2003) can help to reconcile the results of the present experiment with those of the previous report that extinction of goal-tracking elicited by a CS did not affect its transfer properties (Delamater, 1996). If the various behavioral effects produced by a CS require different thresholds of excitation, then at a given level of net excitation, some behavioral effects may be observed while others are not. For example, extinction may reduce an appetitive CS's net excitation to a level that is below the threshold required to observe goal-tracking and conditioned reinforcement, but not below the threshold required to observe transfer of control. (Evidence that instrumental transfer requires very little net excitation comes from previous experiments showing that the operant-rate-influencing effects of stimuli are retained even after very extensive extinction [Hendry, 1982; Kearns & Weiss, 2005; Reberg, 1972]). That different behavioral effects of a CS have different net excitation thresholds can explain why some studies (e.g., Delamater, 1996) have observed response-specific extinction and others (e.g., present study) have found that extinction suppresses multiple behavioral effects of a CS.