Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jun 1.
Published in final edited form as: Learn Behav. 2016 Jun;44(2):151–161. doi: 10.3758/s13420-015-0195-9

Discriminative properties of the reinforcer can be used to attenuate the renewal of extinguished operant behavior

Sydney Trask 1, Mark E Bouton 1
PMCID: PMC4805475  NIHMSID: NIHMS725674  PMID: 26400498

Abstract

Previous research on the resurgence effect suggests that reinforcers that are presented during the extinction of an operant behavior can control inhibition of the response. To further test this hypothesis, three experiments with rat subjects examined the effectiveness of using reinforcers that were presented during extinction as a means of attenuating or inhibiting the operant renewal effect. In Experiment 1, lever pressing was reinforced in Context A, extinguished in Context B, and then tested in Context A. Renewal of responding that occurred during the final test was attenuated when a distinct reinforcer that had been presented independent of responding during extinction was also presented during the renewal test. Experiment 2 established that this effect depended on the reinforcer being featured as a part of extinction (and thus associated with response inhibition). Experiment 3 then found that the reinforcers presented during extinction suppressed performance in both the extinction and renewal contexts; the effects of the physical and reinforcer contexts were additive. Together, the results further suggest that reinforcers associated with response inhibition can serve a discriminative role in suppressing behavior and may be an effective stimulus that can attenuate operant relapse.

Behavior that has been acquired through either Pavlovian or operant conditioning can be reduced when the outcome or reinforcer is withheld following the CS or response. Although such extinction can eliminate behavior, it is known that it does not erase the original learning. For example, extinguished responding can return when the animal is removed from the physical context of extinction in both Pavlovian (Bouton & Bolles, 1979; Bouton & Peck, 1989) and operant (Bouton, Todd, Vurbic, & Winterbauer, 2011; Nakajima, Tanaka, Urushihara, & Imada, 2000) procedures. This return of responding is known as renewal. Renewal suggests that extinction results in the creation of new learning that is especially context-dependent, rather than erasing the original learning. In Pavlovian procedures, contextual cues appear to disambiguate a CS that now has two distinct meanings (from both acquisition and extinction; e.g., Bouton, 2002). However, recent evidence suggests that during operant extinction, animals learn to inhibit a specific response in a specific context (e.g., Bouton & Todd, 2014; Todd, 2013; Todd, Vurbic, & Bouton, 2014). Removal of contextual cues associated with inhibition of the response is enough to cause a return of responding. This is the case even when the contexts are matched for associative history (Todd, 2013). Renewal has also been shown when the response has been eliminated through punishment, rather than extinction (Bouton & Schepers, 2015; Marchant, Khuc, Pickens, Bonci, & Shaham, 2013).

Several other “relapse” phenomena that have been demonstrated after extinction make a similar point (e.g., Bouton & Woods, 2008; Vurbic & Bouton, 2014). For example, in experiments on resurgence, animals learn to perform one response (R1) to receive a food outcome. Once this behavior is established, R1 is extinguished and a newly introduced second response, R2, is now reinforced. When reinforcement for R2 behavior is subsequently removed, R1 responding increases or “resurges” (Leitenberg, Rawson, & Bath, 1970; Leitenberg, Rawson, & Mulick, 1975). While there are several accounts of resurgence (see Leitenberg et al., 1970; Shahan & Sweeney 2011), it has been suggested that resurgence is a special case of the renewal effect, in which removal of reinforcers creates the contextual change necessary to produce relapse (e.g., Winterbauer & Bouton, 2010). On this view, receiving reinforcers for R2 creates a context for the extinction of R1, and the removal of those reinforcers creates a context change that causes behavior to return. If resurgence occurs due to a lack of generalization between the context with reinforcers and the testing context without, then encouraging generalization between these two phases should theoretically decrease resurgence. Consistent with this idea, it has been shown across a wide array of experimental procedures that leaner schedules of R2 reinforcement (that should encourage greater generalization to the testing phase, where no reinforcers are delivered) do attenuate (and sometimes abolish) the resurgence effect (Bouton & Schepers, 2014; Bouton & Trask, 2015; Leitenberg et al., 1975; Schepers & Bouton, 2015; Sweeney & Shahan, 2013; Winterbauer & Bouton, 2012).

The contextual explanation of resurgence follows a research tradition suggesting that reinforcers have discriminative, as well as a reinforcing, roles. For example, Reid (1958) found that an extinguished operant behavior could return when a reinforcer that had previously been used to establish that behavior was presented noncontingently across rat, pigeon, and human subjects. Presenting the reinforcer returned a stimulus (or context) that had originally set the occasion for the response. In an experiment reported by Ostlund and Balleine (2007), rats learned to perform one response (R1) for one outcome (O1). However, R1 only produced O1 following a noncontingent presentation of a second outcome (O2), which served as the discriminative stimulus to signal the R1-O1 relationship (i.e., O2: R1-O1). Concurrently, animals were also taught to perform a second response (R2) to earn the O2 reinforcer, but only after free presentation of the O1 reinforcer (O1: R2-O2). When tested for reinstatement following O1 and O2 presentations, it was found that when O1 was presented, R2 was elevated relative to R1, and when O2 was presented, R1 was elevated relative to R2. In other words, each reinforcer selectively elevated the response that it preceded rather than followed, further suggesting that the stimulus properties of the reinforcer (and not its reinforcing properties) accounted for the reinstatement effect. Stimulus properties of the reinforcer have been invoked to explain a number of interesting phenomena in animal learning (e.g., Neely & Wagner, 1974; Sheffield, 1949).

The idea that reinforcers can have discriminative properties has been expanded to include the idea that reinforcers can also control extinction performance. For example, in an experiment reported by Bouton, Rosengard, Achenbach, Peck, and Brooks (1993), rats received repeated cycles of conditioning, in which a tone CS was paired with a food US, followed by extinction, in which the CS was no longer paired with the US. One group of animals, Group HiLo, received unsignaled US presentations during the intertrial interval (ITI) throughout conditioning, but not extinction. A second group, Group LoHi, received unsignaled US presentations during the ITI as a feature of extinction, but not of conditioning. When finally tested with and without US presentations, animals in Group HiLo showed enhanced conditioned responding when the US was presented in the ITI, while Group LoHi showed a suppression of conditioned responding when the US was presented in the ITI. In other words, when the extra US presentations were a feature of acquisition, they enhanced responding, and when they were a feature of extinction they inhibited responding. Similarly, Lindblom and Jenkins (1981) reduced conditioned responding through either negatively correlated or noncorrelated presentations of the CS and US; both procedures involve unsignaled presentations of the US during response elimination (as in Bouton et al., 1993). They found that removal of the food US entirely resulted in a renewal-like return of responding, wherein the context change was created by removal of the US presentations.

In a direct test of the idea that removal of reinforcers in the resurgence paradigm causes renewal through context change, Bouton and Trask (2015, Experiment 2) found that distinct reinforcers associated with extinction can serve as an effective cue to inhibit operant responding. In that experiment, rats were taught to perform a response (R1) for a distinct outcome (O1). In a second phase, R1 was placed on extinction, while a newly introduced response (R2) produced a new reinforcer (O2). In a final phase, where both levers were available but neither was reinforced, rats were tested in one of three conditions: with O1 reinforcers given freely (these had been associated with acquisition of R1), with O2 reinforcers given freely (these had been associated with acquisition of R2 and inhibition of R1), or with no reinforcers. While animals tested with either O1 reinforcers or no reinforcers showed a robust increase in R1 responding (i.e., resurgence), animals tested with the O2 reinforcers showed no increase in R1 responding. The presentation of O2 reinforcers during the test made testing more similar to response elimination conditions. In terms of the context hypothesis, the animals had learned to suppress their R1 behavior in a distinct “context” signaled by the addition of O2.

If reinforcers delivered in extinction can come to suppress or inhibit performance, as the context account of resurgence suggests, then they should also be able to inhibit other examples relapse after extinction. The current experiments therefore asked whether a distinct O2 that had been presented during extinction might actively attenuate the renewal of a free-operant response. In all experiments, rats received operant conditioning in Context A, extinction in Context B, and then renewal testing in Context A. In Experiment 1, presentations of an O2 reinforcer delivered noncontingently during extinction attenuated operant ABA renewal. A second experiment replicated this effect and demonstrated that in order for O2 to attenuate ABA renewal, it had to be associated with response inhibition. A reinforcer that was not presented during extinction did not suppress behavior to the same degree. Experiment 3 then showed that when combined, changing the reinforcer context and the physical context could have additive effects. While changing either was enough to cause an increase in responding, response recovery increased in magnitude when both were changed.

Experiment 1

Experiment 1 examined whether an O2 reinforcer presented during extinction could attenuate ABA renewal. In a within-subject design, rats learned to press a lever for one outcome, O1 (either grain-based food pellets or sucrose-based food pellets, counterbalanced) in Context A. Once animals had acquired lever pressing, they were switched to a new context, Context B, where lever pressing no longer produced O1. At this time, however, a new reinforcer, O2 (either sucrose-based pellets or grain-based pellets, counterbalanced), was presented independently of responding throughout the session. Here, rats could potentially learn to inhibit lever pressing in both the physical Context B and the reinforcer Context O2. During the test, animals were switched back to Context A and tested for the renewal of lever pressing (where lever pressing produced no outcomes). They were tested both with free O2 reinforcers presented as they were during extinction and without reinforcers. If O2 reinforcers can cue or control the inhibition of responding that develops in extinction, then the renewal effect should be attenuated by the presentation of O2.

Method

Subjects

The subjects were 16 naïve female Wistar rats purchased from Charles River Laboratories (St. Constance, Quebec). They were between 75 and 90 days old at the start of the experiment and were individually housed in suspended wire mesh cages in a room maintained on a 16:8-h light:dark cycle. Experimentation took place during the light period of the cycle. The rats were food-deprived to 80% of their initial body weights throughout the experiment.

Apparatus

Two sets of four conditioning chambers housed in separate rooms of the laboratory served as the two contexts (counterbalanced). Each chamber was housed in its own sound attenuation chamber. All boxes were of the same design (Med Associates model ENV-008-VP, St. Albans, VT). They measured 30.5 cm × 24.1 × 21.0 cm (l × w × h). A recessed 5.1 cm × 5.1 cm food cup was centered in the front wall approximately 2.5 above the level of the floor. A retractable lever (Med Associates model ENV-112CM) positioned to the left of the food cup protruded 1.9 cm into the chamber. The chambers were illuminated by one 7.5-W incandescent bulb mounted to the ceiling of the sound attenuation chamber, approximately 34.9 cm from the grid floor at the front wall of the chamber. Ventilation fans provided background noise of 65 dBA.

In one set of boxes, the side walls and ceiling were made of clear acrylic plastic, while the front and rear walls were made of brushed aluminum. The floor was made of stainless steel grids (0.48 cm diameter) staggered such that odd- and even-numbered grids were mounted in two separate planes, one 0.5 cm above the other. This set of boxes had no distinctive visual cues on the walls or ceilings of the chambers. A dish containing 5 ml of Rite Aid lemon cleaner (Rite Aid Corporation, Harrisburg, PA) was placed outside of each chamber near the front wall.

The second set of boxes was similar to the lemon-scented boxes except for the following features. In each box, one side wall had black diagonal stripes, 3.8 cm wide and 3.8 cm apart. The ceiling had similarly spaced stripes oriented in the same direction. The grids of the floor were mounted on the same plane and were spaced 1.6 cm apart (center-to-center). A distinct odor was continuously presented by placing 5 ml of Pine-Sol (Clorox Co., Oakland, CA) in a dish outside the chamber.

The reinforcers were a 45-mg grain-based rodent food pellet (5-TUM: 181156) and a 45-mg sucrose-based food pellet (5-TUT: 1811251, TestDiet, Richmond, IN, USA). Both types of pellet were delivered to the same food cup. The apparatus was controlled by computer equipment located in an adjacent room.

Procedure

Magazine training

On the first day of the experiment, all rats were assigned to a box within each set of chambers. They then received one 30-min session of magazine training in Context A with their O1 reinforcer (grain-based or sucrose-based food pellet, counterbalanced). On the same day, the animals also received a second 30-min session of magazine training in Context B with their O2 reinforcer (sucrose-based or grain-based food pellet, counterbalanced). Half the animals were trained first in Context A, and half were trained first in Context B. The sessions were separated by approximately 1 hr. Once all animals were placed in their respective chambers, a two-minute delay was imposed before the start of the session. In each magazine training session, approximately 60 reinforcers were delivered freely on a random time 30-s (RT 30-s) schedule. The levers were not present during this training.

Acquisition

On each of the next six days, all rats received two 30-min sessions of instrumental training in Context A. The sessions were separated by approximately 1 hr. Following a two-minute delay, sessions were initiated by the insertion of the lever into the chamber. Throughout the sessions, presses on the lever delivered O1 reinforcers on a variable interval 30-s (VI 30-s) schedule of reinforcement. No hand shaping was necessary.

Extinction

On each of the next four days, all animals then received two sessions of response extinction in Context B. As before, following a two-minute delay, the lever was inserted and available for 30 min. During this phase, responding on the lever had no programmed consequences. During both the delay period and throughout the session, however, O2 reinforcers were delivered freely (i.e. not contingent on responding) according to an RT 30-s schedule of reinforcement.

Test

On the final day of the experiment, all rats were given two 10-min renewal tests in Context A. Following a two-minute delay, each test session began with the insertion of the lever and ended with the retraction of the lever. One test session occurred with O2 reinforcers delivered on the RT 30-s schedule during the delay and throughout the session. The other test session was conducted without any reinforcer presentation. Testing order was counterbalanced such that half of the animals were tested first with reinforcers present, and half of the animals were tested first without.

Data Analysis

The data were subjected to either t-tests or analysis of variance (ANOVA) as appropriate. For all statistical tests, the rejection criterion was set at p < .05.

Results

Results from acquisition (left panel), extinction (center panel), and the test (right panel) are shown in Figure 1. Animals increased responding throughout the acquisition phase and decreased responding throughout the extinction phase. During the test, animals showed an increase in responding (the standard ABA renewal effect) that was significantly attenuated by presentation of the O2 reinforcer.

Figure 1.

Figure 1

Open in a new tab

Results of Experiment 1. Acquisition in Context A with O1 (left panel), extinction in Context B with noncontingent O2 (center panel), and renewal testing in Context A (right panel) with both free O2 reinforcers and no reinforcers delivered. All available comparisons are within-subject. Note changes in y axes.

Acquisition

As expected, the rats increased their responding over the twelve sessions of acquisition. This was confirmed by a repeated measures ANOVA conducted to assess responding throughout the twelve sessions of the acquisition phase, which found a significant main effect of session, F(11, 165) = 21.67, MSE = 80.42, p < .001, ηp2 = .59.

Extinction

Animals decreased their responding throughout the eight sessions of the extinction phase, as confirmed by a repeated measures ANOVA conducted to assess responding over this phase, F(7, 105) = 14.01, MSE = 29.64, p < .001, ηp2 = .48. On the final day of extinction, rats that were to be tested first with reinforcers (M = 2.21) did not differ from those to be tested first without reinforcers (M = 4.89), t(14) = .73, p = .48.

Test

During the test, there was substantial responding in Context A. However, there was less responding in test sessions where free O2 reinforcers were presented on an RT 30-s schedule than test sessions where no reinforcers were presented. This was confirmed by a paired samples t-test that assessed responding throughout the first three minutes of each condition, t(15) = 2.42, p < .05, η2 = .28. Additionally, t-tests were conducted to assess renewal of responding from the last day of extinction to both the free O2 reinforcer and no reinforcer condition. These revealed significant renewal in both the free O2 reinforcer condition, t(15) = 4.79, p < .001, η2 = .60, and the no reinforcer condition, t(15) = 7.33, p < .001, η2 = .78. Together, the results indicate a robust contextual renewal effect that was attenuated in the free O2 test relative to the no reinforcer test.

Discussion

As predicted, animals responded significantly less in Context A when O2 reinforcers (previously associated with extinction) were presented than when tested without reinforcers. O2 presentations did not abolish the renewal effect entirely, as responding was still increased when animals were tested with reinforcers relative to the final day of extinction, at which time responding was relatively low. However, the suppressive effects of the reinforcers are consistent with the idea that their addition to the test in Context A made the testing context more similar to that of extinction and provided a retrieval cue to signal response inhibition. In the present experiment, while the physical context changed between extinction and both tests, the reinforcer context only remained consistent between extinction and the O2 test. Thus, during the test without reinforcers present, the contextual change was more complete as both the physical context and the reinforcer context had changed, resulting in a greater renewal of responding.

Experiment 2

One alternative explanation of the results of Experiment 1 is that reinforcer presentation might have a generally suppressive effect on behavior. For example, presentation of O2 during testing might merely cause the animal to engage in competing behaviors, such as entering the food magazine or eating. Shahan and Sweeney (2011) have also emphasized the potentially disruptive effects of presenting reinforcers during Phase 2 in the resurgence design. However, Bouton and Trask (2015, Experiment 3) found that presentation of an O2 reinforcer noncontingently in extinction after operant conditioning with O1 actually augmented, rather than suppressed, responding relative to a group that received no extra reinforcers (see also Baker, 1990; Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011). Such results suggest that the presence of noncontingent reinforcers on their own does not necessarily suppress operant responding. The approach under investigation here implies that they need to be featured in extinction first.

Experiment 2 therefore examined whether noncontingent O2 presentations needed to be featured in extinction in order to suppress the renewal effect. As in Experiment 1, all rats acquired lever pressing for an O1 reinforcer in Context A; lever pressing was then extinguished in Context B in the presence of noncontingent O2 reinforcers. Animals were then returned to and tested in Context A. In one group, testing occurred as in Experiment 1 with both noncontingent O2 reinforcers and without any reinforcers. For a second group, animals were tested with both noncontingent O1 reinforcers and without any reinforcers. O1 was presented at the same rate that O2 reinforcers had been delivered during extinction. If O2 reinforcers create a unique context in which extinction learning took place, then O2 reinforcers, but not O1 reinforcers, should suppress the ABA renewal effect. In other words, we hypothesized that in order for a reinforcer to be an effective retrieval cue to signal inhibitory learning, it had to be uniquely associated with response inhibition.

Method

Subjects and Apparatus

The subjects were 24 naïve female Wistar rats of the same stock and maintained in the same conditions as Experiment 1. The same apparatus was used. Each animal was given two daily sessions.

Procedure

Magazine Training, Acquisition, and Extinction

Magazine training, acquisition, and extinction proceeded in exactly the same way as Experiment 1, with the sole exception being that for each animal, sessions were separated by 1.5 hrs instead of 1 hr.

Test

On the final day of the experiment, rats were each given two 10-min renewal tests in Context A. During the first session, eight animals were tested with O1 reinforcers delivered on an RT 30-s schedule, eight were tested with O2 reinforcers delivered on an RT 30-s schedule, and eight were tested with no reinforcer delivery. During the second test, the animals that had been tested with either O1 or O2 reinforcers were now tested without any reinforcers. Half the animals that had been tested first with no reinforcer presentation were now tested with O1 presentations delivered on an RT 30-s schedule and half were tested with O2 presentations delivered on an RT 30-s schedule. In addition to a between-subject comparison of the effects of O1, O2, and no reinforcer presentations on the first test, the experiment thus resulted in a group of rats that was tested both with and without O2, and another group that was tested with and without O1. Although the counterbalancing of test order in the latter grouping was not complete (8 rats were tested first with free reinforcers and 4 were tested first without), an ANOVA on test data that included test order as a factor found that test order did not interact with the other factors, Fs ≤ 1.55, p ≥ .23.

Data Analysis

All data were subjected to t-tests or analysis of variance (ANOVA) where appropriate with a rejection criterion of p < .05. One animal failed to learn extinction by the final day of extinction (when it still made 56.6 responses per minute, Z = 3.21) and was therefore excluded from all analyses (Field, 2005).

Results

Results from all three phases of Experiment 2 are shown in Figure 2. Responding increased throughout acquisition (left panel) and decreased throughout extinction (center panel). During the test (right panel), responding was significantly attenuated by O2, but not O1, presentations compared to the test with no reinforcers present.

Figure 2.

Figure 2

Open in a new tab

Results of Experiment 2. Acquisition in Context A with O1 (left panel), extinction in Context B with noncontingent O2 (center panel), and renewal testing in Context A (right panel) with both free reinforcers (either O1 or O2) and no reinforcers delivered. Error bars are only appropriate for between-group comparisons. Note changes in y axes.

Acquisition

All animals increased responding throughout the twelve sessions of acquisition, as confirmed by a 2 (Group) × 12 (Session) ANOVA in which a significant main effect of session was found, F(11, 231) = 54.59, MSE = 46.27, p < .001, ηp2 = .72. Neither the main effect of group nor the group by session interaction were significant, Fs < 1.

Extinction

The rats decreased responding over the eight sessions of extinction. A 2 (Group) × 8 (Session) ANOVA found a significant main effect of session (7, 147) = 8.99, MSE = 20.98, p < .001, ηp2 = .30, but no main effect of group nor a group by session interaction, Fs < 1. A one-way ANOVA conducted on response rates on the final day of extinction confirmed that animals did not differ based on whether they were to be tested first with O1 (M = 9.1), O2 (M = 7.4), or no reinforcers (M = 2.0), F < 1.

Test

As in Experiment 1, the rats responded less in the renewal test when O2 reinforcers were presented freely than when tested without the reinforcer presentations. However, animals tested with O1 reinforcers showed no difference in responding relative to when tested without reinforcers. Data for the rats tested with and without O1 and with and without O2 over the two test sessions are presented at right in Figure 2. A 2 (Group) × 2 (Test Condition: Free Reinforcers vs. No Reinforcers) ANOVA conducted over the first two minutes of the test found a significant main effect of test condition, F(1, 21) = 5.40, MSE = 55.61, p < .05, ηp2 = .20, with rats responding less in the reinforcer condition than the no reinforcer condition. There was no main effect of group, or a significant interaction, Fs < 1. However, planned comparisons to examine within-subject differences revealed that while Group O1 showed no difference between the free O1 reinforcer and no reinforcer conditions, F < 1, Group O2 showed a significant reduction in responding when presented with free O2 reinforcers (as had been presented in extinction) compared to when tested with no reinforcers, F(1, 21) = 5.17, p < .05, ηp2 = .20. Interestingly, groups did not differ at either the free reinforcer condition or the no reinforcer condition, Fs < 1.

Analyses that focused on the first test session supported the same conclusions. Recall that during the first test, eight rats each were tested with free O1, free O2, and no reinforcers (None). Animals with O1 reinforcers and no reinforcers showed a significant renewal effect when assessed during the test; but this was not true of animals in Group O2. The pattern was confirmed by the 3 (Group) × 2 (Session) ANOVA conducted to assess responding from the last day of extinction to the first three minutes of the test found a significant main effect of session, F(1, 20) = 16.22, MSE = 68.47, p < .001, ηp2 = .45, with animals increasing responding in the test relative to extinction. There was neither a main effect of group nor a significant interaction, Fs < 1. Planned comparisons that assessed within-subject changes from extinction to the test (i.e., renewal) revealed that while animals in both Group O1, F(1, 20) = 5.83, p < .05, ηp2 = .55, and Group None, F(1, 20) = 9.81, p < .01, ηp2 = .80, showed an increase in responding from the last day of extinction to the test (i.e., a renewal effect), Group O2 did not, F(1, 20) = 1.90, p = .18. The mean response rates on the last extinction session were 9.1, 7.4, and 2.0, and the mean rates on the first test session were 19.1, 13.1, and 15.8, for Groups O1, O2, and None, respectively. The results continue to suggest that the suppression of renewal depends on receiving a reinforcer that had been featured in extinction.

Discussion

As predicted, subjects tested with response-independent O2, but not O1, showed a significant attenuation of responding when tested in the free reinforcer condition as compared to the no reinforcer condition. In a complementary way, rats first tested with O1 presentations showed a significant renewal effect, whereas rats first tested with O2 presentations did not. This pattern, coupled with previous findings suggesting that reinforcers delivered independently of responding during or after extinction do not usually suppress responding (e.g., Baker, 1990; Bouton & Trask, 2015; Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011), suggests that in order for a reinforcer to attenuate ABA renewal, it needs to be a feature of extinction learning. The results are not consistent with the idea that the results of Experiment 1 were due to O2 unconditionally eliciting competing behaviors, or otherwise disrupting operant responding (e.g., Shahan & Sweeney, 2011); in that case, O1 should have been equally effective here. The results are instead consistent with the idea that noncontingent O2 presentations reduced renewal because they were associated with extinction or response inhibition.

Interestingly, and perhaps surprisingly, O1 presentation during testing did not augment or reinstate renewed responding above that seen when the rats were tested without reinforcers. This suggests that reinstatement by O1 reinforcers does not add to the renewal effect that occurs when testing occurs in Context A after extinction has occurred in Context B. This aspect of the findings will be discussed in more detail in the General Discussion.

Experiment 3

In resurgence, the context hypothesis proposes that removal of alternative reinforcers has a renewing effect that parallels the simple renewal effect that occurs when the animal is removed from the physical context of extinction. On this view, reinforcer removal and physical context change are held to have the same effect. The parallel may be similar to a parallel that has been noted previously regarding physical context change and a retention interval (i.e., a temporal context change; e.g., Bouton, 1993). Indeed, previous research suggests that a physical context change and a temporal context change can have additive effects in producing response recovery of an extinguished response (Rosas & Bouton, 1998) and in attenuating latent inhibition (Rosas & Bouton, 1997). Experiment 3 thus examined how changing both the physical context and the “reinforcer context” would impact behavior after extinction. As in Experiments 1 and 2, all rats lever pressed for a distinct O1 reinforcer in Context A and the response was then extinguished in Context B with noncontingent O2 reinforcers. For testing, animals were split into two groups. In Group ABA, rats were tested for responding back in Context A with O2 reinforcers presented as in extinction and with no reinforcers presented (in a counterbalanced order). For rats in Group ABB, however, testing occurred in Context B, again with O2 reinforcers presented as in extinction and with no reinforcers delivered. We hypothesized that Group ABA would replicate the effect of Experiment 1 and 2, with rats responding less in the free O2 reinforcer condition than in the no reinforcer condition, thus attenuating the ABA renewal effect (due to greater generalization between extinction and the O2 test). We predicted the same pattern in Group ABB, but that overall responding would be lower in this group than in Group ABA, as there would be no change in the physical context between extinction and testing. If the reinforcer/physical context relationship is similar to the temporal/physical context relationship (Rosas & Bouton, 1997, 1998), then the effects of changing the reinforcer and physical contexts should be additive: The ABA group tested without O2 (which received both physical and reinforcer context change) should show more response recovery than the ABA group tested with O2 reinforcers (which received physical context change only) and the ABB group tested without O2 (which received reinforcer context change only).

Method

Subjects and Apparatus

The subjects were 32 naïve female Wistar rats of the same stock as Experiments 1 and 2. Animals were housed and maintained exactly as in Experiments 1 and 2. The same apparatus was used. Each animal was given two daily sessions.

Procedure

Magazine Training, Acquisition, and Extinction

Magazine training, acquisition, and extinction proceeded in exactly the same way as Experiments 1 and 2, with the sole exception being that for each animal, sessions were separated by 2 hrs instead of 1 hr (Experiment 1) or 1.5 hrs (Experiment 2).

Test

On the final day of the experiment, rats were separated into two groups (Groups ABA and ABB, ns = 16) and given two 10-min renewal tests during which responding had no programmed consequences. One test for each animal occurred with O2 reinforcers delivered freely according to an RT 30-s schedule during both the delay and the other test session occurred without any reinforcer presentation. For animals in Group ABA, these tests occurred in Context A. For animals in Group ABB, testing occurred in Context B. Testing order was counterbalanced so that half the animals in each group were tested first with free O2 reinforcers, and half were tested first without reinforcer presentations.

Data Analysis

All data were subjected to t-tests or analysis of variance (ANOVA) where appropriate with a rejection criterion of p < .05.

Results

Results from Experiment 3 are shown in Figure 3. As before, all rats increased responding throughout acquisition (left panel), and responding decreased during extinction (middle panel). In the test (right panel), there were clear and additive effects of changing both the physical context (ABA vs. ABB groups) and the reinforcer context (O2 vs. no reinforcer groups).

Figure 3.

Figure 3

Open in a new tab

Results of Experiment 3. Acquisition in Context A with O1 (left panel), extinction in Context B with noncontingent O2 (middle panel), and testing in both the free O2 reinforcer and no reinforcer conditions. Error bars are only appropriate for between-group comparisons. Note changes in y axes.

Acquisition

As in Experiment 1, all animals increased their responding over the twelve sessions of acquisition. This was confirmed by a 2 (Group) × 12 (Session) ANOVA in which a significant main effect of session was found, F(11, 330) = 78.74, MSE = 29.83, p < .001, ηp2 = .72. Neither the main effect of group, F(1, 30) = 1.32, MSE = 928.32, p > .05, nor the interaction, F < 1, was significant.

Extinction

All animals decreased their responding throughout the extinction phase. A 2 (Group) × 8 (Session) ANOVA conducted over this phase found a significant main effect of session, F(7, 210) = 23.50, MSE = 15.80, p < .001, , ηp2 = .44. Neither the main effect of group, F < 1, nor the interaction, F (7, 210) = 1.73, MSE = 15.80, p > .05, was significant. Importantly, on the final day of extinction, animals in Group ABA who were to be tested first with free O2 reinforcers (M = 3.75) did not differ from those who were to be tested first without reinforcers (M = 2.88), t(14) = .27, p > .05. Similarly, animals in Group ABB who were to be tested first with reinforcers (M = 2.96) did not differ from those who were tested first without reinforcers (M = 1.31), t(14) = 1.37, p > .05.

Test

As in Experiments 1 and 2, rats in Group ABA responded more in the test session in which no reinforcers were presented than in the session in which O2 was presented freely. The same was true of animals in Group ABB, although overall responding was substantially lower in this group. A 2 (Group) × 2 (Testing Condition: Reinforcers vs. No Reinforcers) ANOVA run to assess responding during the test found significant main effects of both group (test context), F(1, 30) = 14.74, MSE = 26.70, p = .001, ηp2 = .33, and of reinforcer testing condition, F(1, 30) = 32.14, MSE = 9.93, p < .001, ηp2 = .52, but no interaction between the two, F < 1. Pairwise comparisons revealed that animals in Group ABA responded less when tested in the free O2 condition than in the no reinforcer condition, F(1, 30) = 19.45, p < .001, ηp2 = .39. Similarly, animals in Group ABB showed more suppression of responding during the free O2 test than during the no reinforcer test, F(1, 30) = 13.01, p = .001, ηp2 = .30. In both the free O2 reinforcer condition, F(1, 30) = 12.54, MSE = 13.00, p = .001, ηp2 = .29, and the no reinforcer condition F(1, 30) = 9.89, MSE = 23.63, p < .01, ηp2 = .25, animals in Group ABB responded less than animals in Group ABA.

Discussion

For both Group ABA and Group ABB, responding was significantly reduced during testing by noncontingent presentations of O2. However, in both testing conditions, Group ABB was suppressed relative to Group ABA. Thus, both the physical context of extinction (B) and the reinforcer context of extinction (noncontingent O2) had suppressive effects on behavior. In concordance with results found on changing both the physical context and temporal context (Rosas & Bouton, 1997, 1998), this pattern of results suggests that the reinforcer context and the physical context have separate but additive effects. The results replicate the findings of Experiment 1 and extend the notion of the hypothesized “reinforcer context” (see Bouton et al., 1993; Bouton & Schepers, 2014). The results are also consistent with the finding in pigeons that the combination of a context change (created by changing key light color) and the discontinuation of Phase-2 reinforcers causes more resurgence in the resurgence paradigm than reinforcer discontinuation alone (Kincaid, Lattal, & Spence, 2015).

General Discussion

The results of the present experiments indicate that reinforcers that have been associated with extinction can attenuate the renewal effect when they are presented during the renewal test. While Experiment 1 established this effect, Experiment 2 replicated it and found that it depends on whether the reinforcer has been specifically associated with extinction. Experiment 3 then demonstrated that removal of reinforcers (O2) from the context of extinction (B) produced a resurgence-like relapse effect. The results also suggested that the reinforcer context in which extinction is learned can have a separate but additive effect with that produced by the physical context. Together, the results are consistent with the view that distinct reinforcers presented during extinction can serve to signal response inhibition during extinction. Through this mechanism, reinforcer presentations might also serve as a retrieval cue to attenuate renewal when responding is tested outside of the context of extinction. To our knowledge, this is to date the only evidence of a retrieval cue attenuating operant renewal.

The current results fit well with our interpretation of resurgence, which emphasizes the discriminative role of the alternative reinforcer. According to the context hypothesis, resurgence occurs when reinforcement is removed because animals have learned to inhibit their responding in the context of alternative reinforcement (e.g., Bouton & Schepers, 2014; Schepers & Bouton, 2015; Winterbauer & Bouton, 2010). Thus, removing reinforcers changes the context sufficiently to produce a relapse similar to ABC renewal (e.g. Bouton et al., 2011). This idea is clearly demonstrated in Experiment 3: Group ABB received training with O1 in Context A, extinction with free O2 in Context B, and testing with or without O2 in Context B. When tested with O2, there was a continuation of the extinction conditions and thus suppressed responding was maintained. But when tested without O2 (i.e., alternate reinforcement was removed), this constituted a context change and animals demonstrated a significant increase in responding. Although the present experiments did not examine resurgence per se, the results suggest a clear role for the reinforcer context in controlling relapse in that paradigm. They also extend the findings, reviewed in the Introduction, that indicate that Pavlovian extinction performance can also be cued by reinforcers presented in the background (e.g., Bouton et al., 1993).

It should be noted that in Experiment 2, the O1 reinforcer failed to reinstate responding beyond the level seen in the no-reinforcer condition. This finding suggests that presentation of a reinforcer from acquisition did not add to the basic renewal effect. Because the response had originally produced the O1 reinforcer in Context A, it would not have been surprising to see augmented or reinstated responding with O1 presentations. One reason why such an effect was not observed may be that experience with O2 presentations during extinction reduced the ability of the O1 reinforcer to reinstate extinguished behavior. Related studies have found that free presentations of a reinforcer during extinction can reduce or eliminate the ability of that reinforcer to augment responding during a reinstatement test (e.g., Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011); such a result is consistent with the idea that reinforcer presentations in extinction extinguish the reinforcer's ability to set the occasion for the operant response. It is worth noting that the earlier studies did not use different reinforcers in conditioning and extinction as we did here. However, in a related design, Bouton and Trask (2015; Experiment 3) found that animals that received free presentations of an O2 reinforcer during extinction after initial acquisition with O1 likewise showed no augmenting effect of O1 presentations during a final test. Given these results, it seems that free reinforcer presentations in extinction may have an effect on the ability of similar reinforcers to augment or reinstate extinguished responding. Although the present O1 and O2 reinforcers differed in some sensory properties, they presumably shared some sensory as well as motivational properties. Any generalization between O2 and O1 could have allowed O2 presentations in extinction to reduce the possible reinstating effects of O1.

Previous writers have noted that renewal has interesting implications for relapse after treatment for drug abuse disorders (e.g., Bouton et al., 2011; Crombag & Shaham, 2002). The idea is that if a patient undergoes treatment in a therapeutic setting, this behavior could be susceptible to relapse following the cessation of that treatment (or simple removal from the therapeutic setting, or context). The current results suggest that a salient cue (in the present case, a reinforcer) from the treatment situation could potentially attenuate relapse or renewal if presented in the settings where relapse is likely to occur. They thus extend previous research on renewal and spontaneous recovery in Pavlovian conditioning suggesting that relapse effects can be attenuated by presenting a retrieval cue, just prior to the test, that had been featured in extinction (Brooks & Bouton, 1993, 1994).

Acknowledgments

This research was supported by NIH Grant RO1 DA 033123. We thank Scott Schepers and Eric Thrailkill for their comments.

References

  1. Baker AG. Contextual conditioning during free-operant extinction: Unsignaled, signaled, and backward-signaled noncontingent food. Animal Learning & Behavior. 1990;18:59–70. [Google Scholar]
  2. Bouton ME. Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin. 1993;114:80–99. doi: 10.1037/0033-2909.114.1.80. [DOI] [PubMed] [Google Scholar]
  3. Bouton ME. Context, ambiguity, and unlearning: Sources of relapse after behavioral extinction. Biological Psychiatry. 2002;52:976–986. doi: 10.1016/s0006-3223(02)01546-9. [DOI] [PubMed] [Google Scholar]
  4. Bouton ME, Bolles RC. Contextual control of the extinction of conditioned fear. Learning and Motivation. 1979;10:445–466. [Google Scholar]
  5. Bouton ME, Peck CA. Context effects on conditioning, extinction, and reinstatement in an appetitive conditioning preparation. Animal Learning & Behavior. 1989;17:188–198. [Google Scholar]
  6. Bouton ME, Rosengard C, Achenbach GG, Peck CA, Brooks DC. Effects of contextual conditioning and unconditional stimulus presentation on performance in appetitive conditioning. The Quarterly Journal of Experimental Psychology. 1993;41B:63–95. [PubMed] [Google Scholar]
  7. Bouton ME, Schepers ST. Resurgence of instrumental behavior after an abstinence contingency. Learning & Behavior. 2014;42:131–143. doi: 10.3758/s13420-013-0130-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bouton ME, Schepers ST. Renewal after the punishment of free operant behavior. Journal of Experimental Psychology: Animal Learning and Cognition. 2015;41:81–90. doi: 10.1037/xan0000051. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bouton ME, Todd TP. A fundamental role for context in instrumental learning and extinction. Behavioural Processes. 2014;104:13–19. doi: 10.1016/j.beproc.2014.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bouton ME, Todd TP, Vurbic D, Winterbauer NE. Renewal after the extinction of free operant behavior. Learning & Behavior. 2011;39:57–67. doi: 10.3758/s13420-011-0018-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Bouton ME, Trask S. Role of the discriminative properties of the reinforcer in resurgence. Learning & Behavior. 2015 doi: 10.3758/s13420-015-0197-7. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Bouton ME, Woods AM. Extinction: Behavioral mechanisms and their implications. In: Byrne JH, Sweatt D, Menzel R, Eichenbaum H, Roediger H, editors. Learning and Memory: A Comprehensive Reference. Vol. 1. Elsevier; Oxford, UK: 2008. pp. 151–171. [Google Scholar]
  13. Brooks DC, Bouton ME. A retrieval cue for extinction attenuates spontaneous recovery. Journal of Experimental Psychology: Animal Behavior Processes. 1993;19:77–89. doi: 10.1037//0097-7403.19.1.77. [DOI] [PubMed] [Google Scholar]
  14. Brooks DC, Bouton ME. A retrieval cue attenuates response recovery (renewal) caused by a return to the conditioning context. Journal of Experimental Psychology: Animal Behavior Processes. 1994;20:366–379. [Google Scholar]
  15. Crombag HS, Shaham Y. Renewal of drug seeking by contextual cues after prolonged extinction in rats. Behavioral Neuroscience. 2002;116:169–173. doi: 10.1037//0735-7044.116.1.169. [DOI] [PubMed] [Google Scholar]
  16. Field A. Discovering statistics using SPSS. Sage Publications; Thousand Oaks, CA: 2005. [Google Scholar]
  17. Kincaid SL, Lattal KA, Spence J. Super-resurgence: ABA renewal increases resurgence. Behavioural Processes. 2015;115:70–73. doi: 10.1016/j.beproc.2015.02.013. [DOI] [PubMed] [Google Scholar]
  18. Leitenberg H, Rawson RA, Bath K. Reinforcement of competing behavior during extinction. Science. 1970;169:301–303. doi: 10.1126/science.169.3942.301. [DOI] [PubMed] [Google Scholar]
  19. Leitenberg H, Rawson RA, Mulick JA. Extinction and reinforcement of alternative behavior. Journal of Comparative and Physiological Psychology. 1975;88:640–652. [Google Scholar]
  20. Lindblom LL, Jenkins HM. Response eliminated by noncontingent or negatively contingent reinforcement recover in extinction. Journal of Experimental Psychology: Animal Behavior Processes. 1981;7:175–190. [PubMed] [Google Scholar]
  21. Marchant NJ, Khuc TN, Pickens CL, Bonci A, Shaham Y. Context-induced relapse to alcohol seeking after punishment in a rat model. Biological Psychiatry. 2013;73:256–262. doi: 10.1016/j.biopsych.2012.07.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Nakajima S, Tanaka S, Urushihara K, Imada H. Renewal of extinguished lever-press responses upon return to the training context. Learning and Motivation. 2000;31:416–431. [Google Scholar]
  23. Neely JH, Wagner AR. Attenuation of blocking with shifts in reward: The involvement of schedule-generated contextual cues. Journal of Experimental Psychology. 1974;102:751–763. [Google Scholar]
  24. Ostlund SB, Balleine BW. Selective reinstatement of instrumental performance depends on the discriminative stimulus properties of the mediating outcome. Learning & Behavior. 2007;35:43–52. doi: 10.3758/bf03196073. [DOI] [PubMed] [Google Scholar]
  25. Reid RL. The role of the reinforcer as a stimulus. British Journal of Psychology. 1958;49:202–209. doi: 10.1111/j.2044-8295.1958.tb00658.x. [DOI] [PubMed] [Google Scholar]
  26. Rescorla RA, Skucy JC. Effect of response-independent reinforcers during extinction. Journal of Comparative and Physiological Psychology. 1969;67:381–389. [Google Scholar]
  27. Rosas JM, Bouton ME. Additivity of the effects of retention interval and context change on latent inhibition: Toward resolution of the context forgetting paradox. Journal of Experimental Psychology: Animal Behavior Processes. 1997;23:283–294. doi: 10.1037//0097-7403.23.3.283. [DOI] [PubMed] [Google Scholar]
  28. Rosas JM, Bouton ME. Context change and retention interval can have additive, rather than interactive, effects after taste aversion extinction. Psychonomic Bulletin & Review. 1998;5:79–83. [Google Scholar]
  29. Schepers ST, Bouton ME. Effects of reinforcer distribution during response elimination on resurgence of an instrumental behavior. Journal of Experimental Psychology: Animal Learning and Cognition. 2015;41:179–192. doi: 10.1037/xan0000061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Shahan TA, Sweeney MM. A model of resurgence based on behavioral momentum theory. Journal of the Experimental Analysis of Behavior. 2011;95:91–108. doi: 10.1901/jeab.2011.95-91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Sheffield VF. Extinction as a function of partial reinforcement and distribution of practice. Journal of Experimental Psychology. 1949;38:511–526. doi: 10.1037/h0057242. [DOI] [PubMed] [Google Scholar]
  32. Sweeney MM, Shahan TA. Effects of high, low, and thinning rates of alternative reinforcement on response elimination and resurgence. Journal of the Experimental Analysis of Behavior. 2013;100:102–116. doi: 10.1002/jeab.26. [DOI] [PubMed] [Google Scholar]
  33. Todd TP. Mechanisms of renewal after the extinction of instrumental behavior. Journal of Experimental Psychology: Animal Behavior Processes. 2013;39:193–207. doi: 10.1037/a0032236. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Todd TP, Vurbic D, Bouton ME. Mechanisms of renewal after the extinction of discriminated operant behavior. Journal of Experimental Psychology: Animal Learning and Cognition. 2014;40:355–368. doi: 10.1037/xan0000021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Vurbic D, Bouton ME. A contemporary behavioral perspective on extinction. The Wiley Blackwell Handbook of Operant and Classical Conditioning. 2014:53–76. [Google Scholar]
  36. Winterbauer NE, Bouton ME. Mechanisms of resurgence of an extinguished instrumental behavior. Journal of Experimental Psychology: Animal Behavior Processes. 2010;36:343–353. doi: 10.1037/a0017365. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Winterbauer NE, Bouton ME. Mechanisms of resurgence II: Response-contingent reinforcers can reinstate a second extinguished behavior. Learning and Motivation. 2011;42:154–164. doi: 10.1016/j.lmot.2011.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Winterbauer NE, Bouton ME. Effects of thinning the rate at which the alternative behavior is reinforced on resurgence of an extinguished instrumental response. Journal of Experimental Psychology: Animal Behavior Processes. 2012;38:279–291. doi: 10.1037/a0028853. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES