Mechanisms of resurgence II: Response-contingent reinforcers can reinstate a second extinguished behavior

Abstract

Three experiments with rat subjects examined resurgence of an extinguished instrumental response using the procedure introduced by Epstein (1983) with pigeons. There were three phases: (1) initial acquisition of pressing on a lever (L1) for pellet reward, (2) extinction of L1, and (3) a test session in which a second lever (L2) was inserted, briefly reinforced, and then extinguished. Experiment 1 confirmed that if pressing L2 delivered 20 pellets followed by extinction, rats would resume L1 responding in the final test. Experiment 2 compared the effects of response-contingent and non-contingent rewards delivered upon insertion of L2. Although insertion of L2 alone did not increase L1 responding, response-contingent and non-contingent rewards led to comparable increases in L1 responding. Experiment 3 found that the delivery of non-contingent pellets during extinction of L1, which would be expected to reduce the ability of pellets to set the occasion for the L1 response, also reduced the effects of both response-contingent and non-contingent rewards during the final test. The results indicate that in this method, the resurgence treatment leads to an increase in L1 pressing due to simple presentation of the pellet; delivering the reinforcer after extinction of L1 reinstates L1 responding by setting the occasion for the L1 response.

Once acquired, learned behavior is strikingly resilient. Attempts to eliminate behavior through extinction do not permanently eliminate performance-- instead, a wide range of manipulations uncover continuing control over behavior by the associations established during acquisition (e.g., Bouton, 2004). Even treatments with the potential to deepen extinction, or to interfere with expression of the to-be-extinguished association, are often ineffective. For example, removal of punishment can reveal that the original learning remains and can recover its control of behavior (Estes, 1944). Common to most of the paradigms that reveal the impermanence of extinction is a role for the context (Bouton, 2004). For example, if the test context is different from the extinction context, behavior often returns (shows renewal, e.g., Bouton & Bolles, 1979; Bouton, Todd, Vurbic, & Winterbauer, 2011; Nakajima, Tanaka, Urushihara, & Imada, 2000; Welker & McAuley, 1978). Extinction is a relatively context-specific form of new learning (e.g., Bouton, 2004).

In instrumental learning, resurgence is another example of response recovery after extinction. Two key forms of resurgence may be distinguished. First, in what we will call Type I resurgence, a second response is reinforced during extinction of a target response (e.g., Leitenberg, Rawson, & Bath, 1970). Compared to a control group that receives only extinction, reinforcing the second response typically increases the rate of extinction. Yet, subsequent removal of the reinforcer causes the target response to recover or resurge (e.g., Leitenberg et al., 1970; Winterbauer & Bouton, 2010). Introduction of the alternative reward may hasten the loss of the target response by introducing behaviors that competed with performance of the target. Type I resurgence was initially attributed to such prevention of the target response during extinction, because it reduced exposure to the response - no outcome contingency (Rawson, Leitenberg, Mulick, & Lefebvre, 1977). However, prevention of the target response is not necessary to observe the Type I resurgence effect (Winterbauer & Bouton, 2010). Given that the alternative reward is introduced simultaneously with the beginning of extinction, it might also provide a context marking extinction. Removal of this contextual marker could then trigger resurgence of the original behavior in an example of "ABC" renewal (e.g., Bouton et al., in press). Recent experimental evidence suggests that this renewal account may be the most parsimonious explanation of Type I resurgence (Winterbauer & Bouton, 2010).

The purpose of the present experiments was to investigate a second form of resurgence introduced by Epstein (1983, 1985). In what we will call Type II resurgence, the original target response is extinguished before the reinforcement and extinction of the new response. Epstein (1983) demonstrated Type II resurgence in pigeons trained to keypeck for grain on a variable interval 1-min schedule of reinforcement. Keypecking was extinguished for several sessions and then a test was conducted. In this test, the birds were given an initial period of continued keypeck extinction, followed by a period in which another response such as head turning (the actual response varied from bird to bird) was reinforced 20 times with grain. All of the pigeons acquired the alternative response and quickly earned the 20 reinforcers, after which this new response was extinguished. Epstein found that keypecking behavior resurged soon after extinction of the alternative response, which itself showed rapid, nearly complete extinction following its final reinforcement. More recently, Lieving and Lattal (2003, Experiment 1) similarly conducted extinction of a keypecking response before introducing a rewarded alternative response (treadle pressing). After reinforcement of the treadle pressing for many sessions, extinction led to resurgence of the original response.

There has been little, if any, research designed to uncover the mechanisms underlying Type II resurgence. Our contextual account of Type I resurgence may not apply as clearly to the Type II design (Epstein, 1983). For one thing, in the Type II situation, extinction of the alternative response returns the animal to the nonreinforced conditions (context) that had prevailed during previous extinction of the target response. A return to the context of extinction would not be expected to renew responding, although it is worth noting that the previous extinction phase occurred without the presence of (or recent reinforcement of) an alternative behavior. But equally important, there is another candidate that can contribute to the recovery of the extinguished target behavior in the Type II design—activation of the extinguished target response by simple reintroduction of the reinforcer. The presentation of reinforcers alone after extinction can “reinstate” extinguished responding in the reinstatement paradigm (Baker, Steinwald, & Bouton, 1991; Franks & Lattal, 1976; Ostlund & Balleine, 2007; Reid, 1958; Rescorla & Skucy, 1969). Type II resurgence might therefore depend to some extent on the mechanisms that underlie reinstatement.

The present article reports three experiments that examined the generality of Type II resurgence and sought to determine the mechanisms behind it. Experiment 1 demonstrated the phenomenon for the first time in rats rather than pigeons. Experiment 2 compared the resurgence treatment, where an alternative behavior was explicitly reinforced and extinguished, with a reinstatement treatment, in which yoked non-contingent reinforcers were delivered. Experiment 3 asked whether Type II resurgence, like the reinstatement produced by non-contingent reinforcers, could be reduced by delivering non-contingent reinforcers during the initial extinction phase. The results suggest that Type II resurgence occurs because presentation of the reinforcer sets the occasion for the original response.

Experiment 1

Experiment 1 was designed to replicate Epstein's (1983) Type II resurgence effect in rats. All animals were given initial acquisition training (Phase 1) in which pressing a lever (L1) delivered pellets on a random interval 30-sec (RI 30) schedule of reinforcement. Following acquisition, reinforcement for responses on L1 was terminated for several sessions, extinguishing that responding (Phase 2). Finally, testing occurred in a single session that began with continued L1 extinction. After 15 min, a novel lever (L2) was introduced and presses of it were briefly reinforced on a continuous schedule of reinforcement (CRF). Each rat was given the opportunity to earn 20 pellets on L2, after which L2 was put on extinction. If Type II resurgence occurs in rats, then animals should show a return to responding on L1 once L2 begins extinction.

Method

Subjects

The subjects were eight female Wistar rats obtained from Charles River, Inc. (St. Constance, Quebec). They were approximately 85–95 days old at the start of the experiment and were individually housed in suspended stainless steel cages in a room maintained on a 16:8-hr light:dark cycle. Sessions occurred during the illuminated portion of the cycle, at the same time each day. Each rat was food deprived to 80% of its free-feeding weight and maintained at that level throughout the experiment by a single feeding following each session.

Apparatus

Conditioning proceeded in two rooms, each with a different set of four standard conditioning boxes (Med-Associates, St. Albans, VT) that were modified as described below for use as separate contexts in other experiments (box context was balanced across groups in all of the present experiments). Boxes from both sets measured 31.5 × 25.4 × 24.1 cm (l × w × h), with side walls and ceilings made of clear acrylic plastic and front and rear walls made of brushed aluminum. Recessed 5.1 × 5.1-cm foodcups were centered in the front wall about 3 cm above the grid. In one set of boxes, the floor was composed of stainless steel rods (0.5 cm in diameter) in a horizontal plane spaced 1.6 cm center to center, while in the other set of boxes, the floor was composed of identical rods spaced 3.2 cm apart in two separate horizontal planes, one 0.6 cm lower than the other and horizontally offset by 1.6 cm. The boxes with the planar floor grid had a side wall with black panels (7.6×7.6 cm) placed in a diagonal arrangement, and there were diagonal stripes on both the ceiling and back panel, all oriented in the same direction, 2.9 cm wide, and about 4 cm apart. The other boxes, with the staggered floor, were not adorned in any way. Retractable levers (1.9 cm when extended) were positioned approximately 3.2 cm to the right and to the left of the food cup and 6.4 cm above the grid. Both sets of boxes were housed in sound-attenuating chambers and were illuminated by two 7.5-W incandescent light bulbs (houselights) mounted above the plexiglass box top on the chamber ceiling. Food reward consisted of 45-mg MLab Rodent Tablets (TestDiet, Richmond, IN).

Procedure

Daily sessions were employed throughout the experiment. Sessions were conducted with 6–8 hr of lighted colony time remaining.

Magazine training

All animals received an initial session of magazine training in which the levers were retracted (i.e., unavailable). Food pellets were delivered freely on a random time 30-sec schedule (RT 30); delivery occurred with a uniformly distributed 1 in 30 probability in each second. On average, 60 pellets were delivered over the course of the rats' first 30 min in the conditioning chamber.

L1 acquisition (Phase 1)

All animals were then given five sessions of instrumental conditioning. Each session was defined as the period between insertion and then retraction of the lever(s) 30 min later. For half the animals the left lever was inserted during these sessions, while for the other half the right lever was inserted. Regardless of which lever was available, presses earned delivery of a single pellet on an RI 30 schedule of reinforcement throughout this phase. Reinforcement availability for responding was initiated with a 1 in 30 probability in each second following the prior rewarded press; subsequent RI schedules used the same general procedure but adjusted the probability as required.

L1 extinction (Phase 2)

Animals were next given three sessions of extinction. During these sessions, presses on the lever no longer delivered pellets; all other features of these sessions were unchanged from Phase 1.

Resurgence test (Phase 3)

A final test session was given to all animals. L1 was inserted and had no programmed consequences throughout the session (as in Phase 2). L2 was inserted 15 min into the session, and each press of L2 delivered a pellet until 20 pellets had been delivered. After the 20^th pellet, no further pellets were delivered. Both L1 and L2 remained in the chamber on extinction schedules for 26 more min after L2 insertion. In addition, the animals were given a second 30-min session with both L1 and L2 inserted without programmed consequences on the next day.

Statistical analysis

For this and following experiments, analysis of variance with a rejection criterion of p < 0.05 was employed in all inferential tests. Test analyses examined pre-, during-, and post-L2 treatment periods. In all cases, the pre- and post-periods were 3-min periods before the beginning of the test treatment and 3-min periods following the final pellet delivery defining that treatment. A wide range of window lengths supported identical conclusions.

Results and Discussion

The results of acquisition and extinction are presented in Figure 1. Acquisition of L1 responding in Phase 1 proceeded smoothly, as indicated by a reliable increase in number of lever presses over sessions, F(4, 28) = 9.99, MSE = 38.12. Extinction likewise proceeded uneventfully, with rats decreasing lever pressing over sessions, F(2, 14) = 83.81, MSE = 1.66. Rats rapidly earned the available L2 reinforcers; a median of 125 s elapsed between the insertion of L2 and delivery of the 20^th pellet.

Figure 1.

Responding on Lever 1 during the sessions of acquisition and extinction in Experiment 1.

Responding on both L1 (upper panel) and L2 (lower panel) during the entire resurgence test session is presented in Figure 2. Although there was some L1 responding at the start of the session owing to spontaneous recovery after the third extinction session 24 hrs earlier, responding declined substantially before the resurgence treatment (at bin 0, upper panel). At that point, there was a clear increase in L1 responding, particularly after L2 had been introduced and reinforced. The resurgence effect with L1 was analyzed by isolating the 3-min bin before L2 was inserted and the 3-min bin after the final (20^th) pellet was delivered for each rat. An ANOVA revealed a reliable increase in L1 pressing following reward and extinction of L2, F(1, 7) = 29.38, MSE = 0.97. The rats, however, responded on L2 at a reliably higher rate than on L1 during the first 3-min post- bin, F(1, 7) = 36.96, MSE = 11.74.

Figure 2.

Responding on Lever 1 (L1, top) and Lever 2 (L2, bottom) during the test session of Experiment 1. L2 was inserted 15 mins into the session. Lever pressing during the resurgence treatment (0) is at center, and the bin length varies by subject; the remaining data points correspond to the 3-minute bins that preceded (left) and followed (right) the resurgence treatment. Note that L1 and L2 ordinates differ.

The results indicate a Type II resurgence effect in rats. Several characteristics of the present phenomenon are worth mentioning. First, extinction learning might not have been complete before introduction of the resurgence treatment, as suggested by the spontaneous recovery evident at the start of the test session. The extinction method used here was linked to other studies of relapse in our laboratory (Bouton et al., 2011; Winterbauer & Bouton, 2010); rats often continue to emit some responding at the start of extinction sessions even after extensive extinction training (e.g., Rescorla & Skucy, 1969). A second characteristic of the present effect is that, as suggested by Figure 2, the animals emitted some L1 responses while L2 was being reinforced—that is, before L2 began extinction. The modest increase from the pre- to the during treatment periods, however, was not reliable, F < 1, MSE = 1.7. Finally, it is worth noting that resurgence did not take the form of an absolute preference for L1 over L2 after L2 was put on extinction. Interestingly, L2 responding was still higher than L1 responding even in the 30-min extinction session that followed the resurgence test (not shown). During that session, the rats pressed L2 at a rate of 1.67 responses / min and L1 at 0.8 responses / min, F(1, 7) = 19.3, MSE = 0.16. The results of Experiment 1 nonetheless demonstrate a clear Type II resurgence effect in rats. The next two experiments were organized to study it in more detail.

Experiment 2

The resurgence effect demonstrated in Experiment 1, as well as that reported by Epstein (1983), can be explained in several ways. In Experiment 1, response recovery was correlated with the addition of reinforcement, insertion of L2, the instrumental L2-pellet contingency, and the extinction of L2. Any of these variables could cause an increase in behavior in designs like that employed in Experiment 1. Although resurgence might be a unique consequence of terminating reward for a recently learned behavior, other explanations are possible. Perhaps the most likely alternative is the possibility that reintroduction of the reinforcer alone might “reinstate” L1 responding (e.g., Bouton, 2004). Reinstatement has often been studied in Pavlovian conditioning preparations (Rescorla & Heth, 1975; Bouton & Bolles, 1979), but it also occurs in instrumental conditioning paradigms when non-contingent reinforcers are presented after extinction (e.g., Baker et al., 1991; Franks & Lattal, 1976; Rescorla & Skucy, 1969).

Experiment 2 tested the possible role of reinstatement in Experiment 1’s resurgence of L1 pressing. One set of animals, Group Resurge, received acquisition, extinction, and testing in a manner like that received by the rats in Experiment 1. For another group, Group Reinstate, L2 was inserted and pellet delivery occurred, but the pellets were independent of the rats' behavior. Instead of earning its own pellets, each rat in Group Reinstate received a pellet whenever a master rat in Group Resurge earned one. The final group, Group Ext, received extinction throughout testing, but L2 was inserted into the chamber at the same time as in the other groups. It is conceivable that L2 insertion alone might increase L1 responding through disinhibition (e.g., Brimer, 1970).

Method

Subjects and Apparatus

The subjects were 24 naive female Wistar rats and again obtained from Charles River, Inc. The apparatus was the same as that in Experiment 1.

Procedure

Consecutive daily 30-min sessions were again employed except where noted.

All animals received an initial 30-min session of RT 30 magazine training with levers withdrawn, and then five sessions of instrumental conditioning (Phase 1). For half the animals the left lever was inserted, and for the other half the right lever was inserted. Presses of either lever delivered a single pellet on an RI 30 schedule. All animals were subsequently given three extinction sessions in which they were able to press L1 without reward (Phase 2). As in Experiment 1, each session began with lever insertion and ended with lever withdrawal 30 min later.

A resurgence test (Phase 3) was then conducted. L1 was available from the start of the session, but presses on it were never reinforced. L2 was inserted 15 min into the session. For Group Resurge, the first 20 L2 presses delivered a pellet, after which extinction occurred. For Group Reinstate rats, a non-contingent pellet was delivered [[using a yoking procedure]] whenever a matched animal in Group Resurge earned a pellet. Both L1 and L2 were inactive. Group Ext received no pellets, and both L1 and L2 were inactive. In all groups, lever presses were recorded for an additional 26 min after insertion of L2.

For analysis of the resurgence test data, the pre-treatment period for all groups was defined as the 3-min period prior to L2 insertion; the during-treatment period was defined as the period from L2 insertion until the 20^th pellet was delivered in Groups Resurge and Reinstate (notice that there was no corresponding treatment in Group Ext); and the post-treatment period was defined as 3 min following cessation of pellet delivery in both Groups Resurge and Reinstate, and as the 3-min period immediately following L2 insertion in Group Ext. (The period used for Group Ext was considered conservative because it was most likely to be influenced by disinhibition caused by L2 insertion; however, alternative analyses of responding during later “post” periods yielded the same results.)

Results and Discussion

Acquisition and extinction data are presented in Figure 3. Acquisition of responding proceeded smoothly and equivalently in all groups. There was a reliable increase in responding over sessions, F(4, 84) = 98.31, MSE = 10.5, but there was no Group effect, F < 1, MSE = 159, and no Group×Session interaction, F < 1, MSE = 10.5. Extinction likewise proceeded similarly in all groups, with an effect of Session, F(2, 42) = 106.91, MSE = 5.3, but neither a Group effect, F < 1, MSE = 14.7, nor a Group×Session interaction, F < 1, MSE = 5.3. Group Resurge rats again rapidly earned all L2 reinforcers, with a 142-sec median delay from introduction of L2 to the 20^th pellet delivery.

Figure 3.

Responding on Lever 1 for each group during the sessions of acquisition and extinction in Experiment 2.

Figure 4 summarizes responding of the groups during the test session. As in Experiment 1, there was some L1 responding (upper panel) at the start of this session that declined before L2 was introduced (at bin 0). Not surprisingly, L2 responding (lower panel) increased more in Group Resurge than the other groups; the rate of L2 pressing was reliably greater in Group Resurge than in Group Reinstate during the treatment period, F(1, 14) = 17.73, MSE = 6.6. Immediately following the treatment (hence immediately following L2 insertion in Group Extinction, but after 20 pellet deliveries in Groups Resurge and Reinstate), Group Resurge continued to press reliably faster than both Group Reinstate, F(1, 14) = 44.39, MSE = 7.3, and Group Extinction, F(1, 14) = 9.01, MSE = 14.7. The latter two groups did not reliably differ, F(1, 14) = 2.87, MSE = 14.7.

Figure 4.

Responding on Lever 1 (L1, top) and Lever 2 (L2, bottom) during the test session of Experiment 2. L2 was inserted 15 mins into the session. During treatment (0), Group Resurge was reinforced for the first 20 presses on L2, Group Reinstate received non-contingent pellets (yoked to members of Group Resurge), and Group Ext received no pellets; bin length varied with yoked pairs in Groups Resurge and Reinstate. The remaining data points correspond to the 3-minute bins that followed (right) and preceded (left) the groups’ differential treatment. L1 and L2 ordinates differ.

The crucial results concerned the fate of L1 responding after differential treatment (bin 0), when systematic group differences emerged. During the treatment period, Groups Resurge and Reinstate both increased their L1 responding relative to the last preceding 3-min bin, F(1, 14) = 5.84, MSE = 3.8; there was neither an effect of Group, F < 1, MSE = 3.0, nor a Group×Period interaction, F(1, 14) = 1.85, MSE = 3.8. Groups Resurge and Reinstate also showed a similar increase in L1 responding between the 3-min periods before and after pellet delivery, while Group Ext showed no change. An ANOVA on these data revealed a reliable Group×Period interaction, F(2, 21) = 5.4 MSE = 2.3, a reliable Period effect, F(1, 21) = 23.70, MSE = 2.3, but no overall effect of Group, F(2, 21) = 2.44, MSE = 2.4. Post-hoc analyses examined the interaction. Groups Resurge and Reinstate both increased their responding between the pre- and post-treatment periods, Fs(1, 7) = 13.58, MSE = 2.1, and 12.96, MSE = 4.0, respectively, but there was no change in responding in Group Ext, F < 1, MSE = 0.89. The increase in L1 responding in Group Resurge and Group Reinstate was further examined in a Group×Period ANOVA that isolated these two groups. Although there was a clear effect of Period, F(1, 14) = 25.79, MSE = 3.0, there was no Group effect, F < 1, MSE = 2.5, and more importantly, no interaction between Group and Period, F < 1, MSE = 3.0. The Resurge and Reinstate groups thus showed similar increases in L1 responding.

The results of this experiment suggest that reinforcers that were either contingent or not contingent on L2 responding led to a statistically indistinguishable increase in L1 responding during testing. That increase began during the treatment period, i.e., during pellet delivery. The results have at least two important implications. First, the fact that noncontingent and L2-contingent reinforcers had essentially equivalent effects on L1 responding indicates that the resurgence effect produced by L2-contingent reinforcers is not merely due to generalization (or confusion) between Levers 2 and 1. A role for such generalization would clearly imply a stronger effect of the L2-contingent reinforcers given the Resurge group. Second, the apparently equivalent increase in both groups' responding could have come about directly if the reinforcer played the role of a discriminative stimulus and set the occasion for L1 responding. The results are thus consistent with the view that the present example of Type II resurgence reflects reinstatement, rather than a unique effect of the acquisition and extinction of a new instrumental behavior.

Experiment 3

The possibility that the present version of the Type II resurgence is a product of reinstatement would be strengthened if it could be shown that a treatment that modified the strength of reinstatement also modified the strength of the resurgence effect. Experiment 3 was therefore designed to examine the susceptibility of resurgence to a reinstatement-mitigating treatment.

Two general mechanisms appear to be responsible for reinstatement of extinguished instrumental behavior (Baker et al., 1991). First, free pellet deliveries might have their impact by conditioning the context. Consistent with this possibility, Baker et al. (1991) found that manipulating the strength of context - reinforcer associations that developed during post-extinction exposure to free reinforcers affected the degree of reinstatement they observed in a lever pressing response (e.g., reinstatement was weakened by post-treatment extinction exposure to the context). Second, as noted above, the pellet may develop discriminative control over L1 responding during acquisition, and free pellet deliveries might simply reactivate a chain of behavior that was acquired in Phase 1. Either of these mechanisms can be degraded through the delivery of non-contingent pellets during extinction. For example, L1 responses that might follow presentation of a pellet in extinction would not be reinforced, and the procedure would thus weaken the pellet’s discriminative control. Consistent with this view, while free pellets during extinction slow the rate of response loss in extinction, they also eliminate reinstatement (Rescorla & Skucy, 1969).

Experiment 3 therefore examined the effects of free extinction pellets on the resurgence and (yoked) reinstatement treatments examined in Experiment 2. The design was a 2×2 factorial that manipulated test treatment (Resurgence vs. Reinstatement) and whether or not the rats had previously received non-contingent pellets during extinction. As in Experiment 2, both Resurge and Reinstate animals were expected to show a similar increase in L1 responding following the test treatment if they were in the normal extinction condition. In contrast, as described above, non-contingent pellets in extinction should reduce the reinstatement effect. The question was whether the non-contingent pellets would have the same effect on rats given the resurgence treatment.

Method

Subjects and Apparatus

The subjects were 48 naive female Wistar rats from the same supplier, tested in the same apparatus, and otherwise treated identically to those employed in the previous experiments.

Procedure

The experiment was run in two replications, one with eight and one with four animals per group. Daily 30-min sessions were once again employed except where noted. All animals received an initial session of RT 30 magazine training, and then five sessions of instrumental conditioning (Phase 1). For half of the animals the left lever was inserted during instrumental conditioning, and for the other half the right lever was inserted. Presses delivered a single pellet on an RI 30 schedule. All animals were then given four sessions of extinction in which reinforcement was discontinued (Phase 2). One half of the animals (None) received extinction of L1 without any other programmed events, while the other half (Free) were given free pellets throughout the session on an RT schedule that increased over days (from RT 60 to 120, 240, and 480 in Sessions 1 – 4). The rats in the Free condition also received 20 pellets starting 15 min into every session on an RT 5 schedule. These were included to mimic and thus further extinguish discriminative control by the actual pellet pattern that would be received during testing.

A resurgence test session (Phase 3) was then conducted. As in the previous experiments, L1 was inserted initially in extinction, and after 15 min L2 was inserted. For animals in the Resurge condition (one group each from the None and Free conditions), every press on L2 delivered a pellet until 20 pellets had occurred, after which L2 responses were on extinction. The remaining animals (Reinstate treatment, with one group each from the None and Free conditions) received non-contingent pellet deliveries when a yoked animal in the Resurge condition earned pellets. In all groups, lever presses on both L1 and L2 were recorded for an additional 26 min after L2 insertion.

Results and Discussion

The data from acquisition and extinction are presented in Figure 5. One animal from the Free Resurge group was excluded from the figure and data analyses, due to a failure to learn to press L1. The figures collapse over replication, because at test no interactions with replication were significant, Fs(1, 39) < 1.82, MSE ≥ 3.9. (Replication was also therefore excluded from the data analyses reported below.) Acquisition of L1 pressing in Phase 1 proceeded smoothly in all groups. An ANOVA indicated a reliable increase in lever pressing over sessions, F(4, 172) = 102.34, MSE = 10.2, but there were no interactions with the treatment variables (which were dummy variables at this point), Fs < 1.14, MSEs = 10.2. Extinction likewise proceeded normally in all groups. There was an overall decrease in responding over sessions, F(3, 129) = 174.52, MSE = 3.4. Test treatment a (dummy variable at this point) had no main effect, F < 1, MSE = 23.1. Importantly, the non-contingent delivery of extinction pellets slowed extinction, increasing the overall level of L1 responding, F(1, 43) = 8.10, MSE = 23.1. No other interaction was significant, greatest F(1, 43) = 1.00, MSE = 23.1. Resurgence condition rats once again rapidly earned all L2 reinforcers, with a 140-s median delay from introduction of L2 to the 20^th pellet delivery.

Figure 5.

Responding on Lever 1 during the sessions of acquisition and extinction in Experiment 3. Groups that received no free pellets or free pellets during extinction are each shown in the left and right panels, respectively.

The results of the resurgence testing session are presented in Figure 6, which shows L1 and L2 responding (upper and lower panels, respectively) for the groups that had been given no free pellets during extinction (left) and those that had (right). In general, the behavior of the groups given no free pellets in extinction replicated the results of Experiment 2: The resurgence and reinstatement treatments produced equivalent increases in L1 responding. In contrast, the groups given free pellets (right) showed no such increase in responding to L1.

Figure 6.

Responding on Lever 1 (L1, top) and Lever 2 (L2, bottom) during the test session of Experiment 3. L2 was inserted 15 mins into the session. During treatment (0), Resurge groups were reinforced for the first 20 presses on L2 and the Reinstate groups received non-contingent pellets (yoked to a member of the corresponding Group Resurge; bin lengths again varied with yoked pairs). The remaining data points correspond to the 3-minute bins that followed (right) and preceded (left) the groups’ differential treatment. L1 and L2 ordinates again differ.

We first report L2 results. As in Experiment 2, response-contingent pellets given to the Resurge groups caused more L2 responding than non-contingent pellets given the Reinstate groups: The rate of L2 pressing was again reliably higher in Group Resurge than in Group Reinstate during treatment (bin 0), F(1, 44) = 53.59, MSE = 5.4. There was no difference in L2 pressing between the None and Free conditions, F < 1, MSE = 5.4, and no reliable interaction between the two factors, F(1, 44) = 3.22, MSE= 5.4. This pattern was maintained in the critical post-period, where Resurge rats once again pressed L2 more than Reinstate rats, F(1, 44) = 23.27, MSE = 24.9, but Extinction treatment (None vs. Free) once again had no main effect, F < 1, MSE = 24.9, and did not interact with the test treatment, F < 1, MSE = 24.9.

We next report L1 results. As in the preceding experiments, spontaneous recovery was evident at the start of the session (left part of each panel), but responding in the None condition (left panel) had declined substantially before L2 was inserted and the resurgence and reinstatement treatments began (bin 0). To compare responding before vs. during treatment, a Period (pre- vs. during)×Extinction Pellet (None vs. Free)×Test Treatment (Reinstate vs. Resurge) ANOVA was conducted. This showed that only the Period×Extinction Pellet interaction was reliable, F(1, 43) = 12.56, MSE = 2.7. Effects of Period, and all other interactions with Period, were not, Fs(1, 43) ≤ 1.92, MSE = 2.7. The main effects of the Extinction Pellet (None vs. Free) and Test Treatment (Reinstate vs. Resurge), as well as their interaction, were likewise not reliable, Fs(1, 43) ≤ 1.75, MSE = 3.2. The response rate of animals in the no extinction pellet condition marginally increased during the test treatment, F(1, 22) = 3.06, MSE = 2.0. In contrast, there was a reliable decrease in response rate in animals given Free pellets during extinction, F(1, 21) = 9.52, MSE = 3.5. In both cases, the Period×Test Treatment interactions remained unreliable, Fs < 1.

The most important analysis of L1 responding isolated the change in responding before and after the resurgence/reinstatement treatments. A Period (pre- vs. post-)×Extinction Pellet (None vs. Free)×Test Treatment (Resurge vs. Reinstate) ANOVA found an overall increase in rate of responding from the pre- to the post-treatment Period, F(1, 43) = 6.31, MSE = 2.7. However, this effect interacted with the Extinction Pellet factor, F(1, 43) = 13.72, MSE = 2.7. There was no main effect of Extinction Pellet, no interaction between Extinction Pellet and Test type, and no overall difference between animals tested under Resurgence vs. Reinstatement, Fs(1, 43) < 1.69, MSE = 4.1. The overall pre- to post-period change did not interact with Test Treatment, F < 1, MSE = 2.7, and there was no Extinction Pellet×Test Treatment×Period interaction, F(1, 39) = 1.88, MSE = 2.7. The critical Period×Extinction Pellet interaction was further analyzed with separate ANOVAs testing the effect of Period within each level of the Extinction Pellet factor. Although animals in the None treatment, which were given simple extinction, showed a robust increase in responding from the pre- to the post- period, F(1, 22) = 19.63, MSE = 2.7 that did not interact with Test Treatment, F < 1, MSE = 2.7, animals in the Free treatment, which were given non-contingent pellets throughout extinction, showed no reliable change over these periods, F < 1, MSE = 2.8, and no differential change as a consequence of Test Treatment, F(1, 21) = 1.43, MSE = 2.8. The results thus indicate that the presentation of non-contingent reinforcers in extinction abolished both the resurgence and the reinstatement effects.

The addition of free pellets in extinction slowed down the loss of L1 responding during extinction (see also, e.g., Rescorla & Skucy, 1969), and more importantly, abolished the effects of the present resurgence and reinstatement treatments. The overall pattern of results supports the hypothesis that in the present method, resurgence and reinstatement both reflect discriminative control of responding by the reinforcer. Both recovery effects were robust in the absence of exposure to pellets during extinction, but essentially disappeared when such exposure occurred. Interestingly, there was again no evidence that the contingency between L2 responding and reinforcement during testing played any role in the recovery of L1 responding. The present example of Type II resurgence thus appears to be linked to the discriminative stimulus properties of reinforcer presentation.

General Discussion

The present experiments documented a Type II resurgence effect in rat subjects, and identified a major mechanism that controls it. The results of Experiment 1 suggested that reinforcement and extinction of an alternative behavior (L2) after a target behavior (L1) had been conditioned and extinguished could cause the target behavior to return. Experiment 2 then further demonstrated that either L2-contingent or L2-independent reinforcers produced comparable increases in the extinguished L1 response, beginning during delivery of those reinforcers. The findings supported the idea that reinstatement and Type II resurgence may be a consequence of the same process and that the resurgence effect that results from L2-contingent reinforcers is not merely a product of generalization between responding on Levers 2 and 1. (Such generalization would have produced a greater level of responding on L1 after L2-contingent pellets). Experiment 3 then studied the effects of a treatment designed to reduce or eliminate reinstatement of L1 responding. It showed that free pellets delivered in extinction eliminated both resurgence and reinstatement at test. The most straightforward interpretation of that result is that the presentation of free pellets extinguished their discriminative control over L1 responding. This result strengthened the conclusion that in the present method, Type II resurgence is an example of reinstatement of extinguished behavior. The results thus establish that response-contingent reinforcer presentations, like response-independent presentations, can reinstate an extinguished behavior.

The reinstatement of instrumental behavior may represent the influence of several processes. One possibility is that presentation of the reinforcer causes excitatory conditioning of the context (e.g., Baker et al., 1991). Such contextual conditioning might energize or arouse any behavior, or it might be particularly effective at energizing an instrumental response, as demonstrated by Pavlovian-instrumental transfer (e.g., Rescorla & Solomon, 1967). Another possibility is that because contextual conditioning is a feature of the original acquisition context, it might renew extinguished responding when it is presented again after extinction (e.g., Bouton, Rosengard, Achenbach, Peck, & Brooks, 1993). Whatever the underlying process, the context - pellet association can be critical to reinstatement: Baker et al. (1991) found reduced reinstatement if the food was signaled during reinstating pellet delivery, if the context was subsequently extinguished, or if pellet delivery occurred in a context other than the test context.

It is also possible that non-contingent pellets may act as discriminative stimuli for contemporary lever pressing (e.g., Baker et al., 1991). Such an effect has clearly been demonstrated in instrumental responding under choice, and is distinct from the rewarding properties of the reinforcer (Dickinson & de Wit, 2003; de Wit, Ostlund, & Balleine, 2009). Reinstatement following extinction appears to depend more on the discriminative stimulus function than on the reward value of the reinforcer (Ostlund & Balleine, 2007). The stimulus aspects of the reinforcer likewise seem wholly responsible for the recovery of responding in the present experiments. Even the instrumentality of the L2 response (the fact that pellets were contingent on L2 responding for some animals) produced no added benefit in the resurgence condition of Experiments 2 and 3. And in Experiment 3, extinction of the discriminative property of the pellet was uniformly sufficient to abolish recovery of L1 responding, whether it was produced by response-contingent (resurgence) or response-independent (reinstatement) reinforcers in the final test.

Given the present results, both Type I and II resurgence now seem consistent with known principles of response recovery and relapse. Winterbauer and Bouton (2010) found that Type I resurgence was supported by renewal, in that the presence of reinforcers and reinforced behavior during extinction established a contextual marker for that extinction; the removal of the marker caused renewal of the extinguished behavior. The present experiments demonstrate that Type II resurgence in rats can occur due to the reinstating effects of reward delivery, without an additional contribution of the novel lever - reward contingency. Other examples of resurgence may reflect a combination of these mechanisms. For example, Lieving and Lattal (2003, Experiment 1) introduced a novel rewarded response following extinction, as in the present Type II experiments, but continued the reinforcement of that response for an extended period, as in the Type I resurgence paradigm. In comparison with the present procedure, the extended procedure could more easily establish reinforcement of the alternative behavior as a new extinction context, the removal of which might cause renewal.

One remaining question is how well the present results capture the main features of Epstein’s (1983) initial experiment demonstrating Type II resurgence in pigeons. At least two differences deserve comment. First, Epstein’s procedure induced deeper extinction prior to introduction of the resurgence treatment (little spontaneous recovery of pecking was observed during the test session). However, there is no a priori reason to expect that discriminative control of responding by the reinforcer would decrease if extinction had been more complete. Indeed, Rescorla and Skucy (1969, Experiment 2) found no difference in reinstatement of lever pressing in rats given one or eight sessions of extinction training prior to free pellet delivery. Second, while nothing excludes a role for the discriminative stimulus properties of the reinforcer in Epstein’s experiment, it is notable that resurgence in that experiment did not appear until a median of 32 s had elapsed after presentations of the last reinforcer. In the present experiments, there was evidence that resurged responding began while the alternative behavior was being reinforced, although response competition could presumably play a stronger role in some situations. The present findings suggest a need for additional research on the Type II resurgence effect in pigeons.

Basic research on operant conditioning has had an important influence on the clinical treatment of behavior disorders, including substance abuse (e.g., Bouton, Vurbic, & Winterbauer, in press; Higgins, Heil, & Lussier, 2004), and it is consequently worth considering the possible broader implications of the present results. Type I resurgence is known to occur in rats reinforced with other rewards, including alcohol (Podlesnik, Jimenez-Gomez, & Shahan, 2006). Although Type II resurgence has been the focus of less research, it might also be general; for example, the reinstatement mechanism implicated here has been shown using a variety of reinforcers (see, e.g., Lu, Shepard, Hall, Shaham, 2003). In fact, the current experiments can be viewed as extending the generality of the basic reinstatement phenomenon because they suggest that response-contingent as well as response-independent reinforcers can reinstate extinguished behavior. Response-contingent reinforcers (rather than response-independent) may be the norm in the natural world outside the laboratory.

Therapies targeting problematic habits (e.g., overeating, drug abuse, and excessive gambling) often leave patients vulnerable to relapse (Bouton et al., in press). Laboratory effects such as resurgence may provide a framework for understanding some relapse effects in the clinic. Contingency Management, for example, is a therapeutic intervention that employs a strategy that parallels the extinction treatment used in Type I resurgence. That is, a novel reward for abstinence is introduced during treatment, and its presence improves initial outcomes during the treatment of drug abuse (reviewed in Higgins et al., 2004; Petry, 2000). This effect might tentatively correspond to the ability of L2 reward to accelerate the extinction of L1 pressing in Type I resurgence. However, after the novel reward is discontinued, abstinence may diminish, and drug-taking may resurge (e.g., Petry, Alessi, Carroll, Hanson, MacKinnon, Rounsaville, & Sierra, 2006).

Effective treatment strategies might be designed to target the relapse mechanisms uncovered in the laboratory. The contexual basis of Type I resurgence suggests that treatments designed to decrease the salience of the transition out of treatment might reduce relapse. The current finding that a reinstatement mechanism underlies the present form of Type II resurgence might have a different implication for treatment. Experiment 3 provides some evidence that non-contingent delivery of the reinforcer during treatment, for example, may provide some protection against Type II resurgence and reinstatement. In a general way, the results suggest that treatments that target the reinstating effects of reinforcers associated with problematic behavior might protect against not only the ability of non-contingent exposure to those reinforcers to induce relapse, but also voluntary exposure to those reinforcers. Similar issues have been discussed by Bouton, Woods, and Pineño (2004) and Woods and Bouton (2007).