Mechanisms of resurgence of an extinguished instrumental behavior

Abstract

Four experiments examined "resurgence" of an instrumental behavior after extinction. All experiments involved three phases in which rats were (1.) trained to press one lever for food reward, (2.) trained to press a second lever while the first leverpress was extinguished, and (3.) tested under conditions in which neither leverpress was rewarded. In each experiment, the first leverpress recovered (resurged) in Phase 3, when the second leverpress was extinguished. The results demonstrated that resurgence occurred when the schedules of reinforcement employed in Phases 1 and 2 yielded either an upshift, downshift, or no change in the rate of reward delivery between those phases. They also demonstrated that initial training on the first lever was required to observe a robust increase in pressing at test (resurgence is thus an associative effect). Resurgence was shown to occur over a wide variety of schedules of reinforcement in Phase 2 (including ratio, interval, and leverpress-independent schedules). Finally, the results do not support the view that resurgence occurs because response competition suppresses leverpressing of the first lever during extinction. Overall, they are consistent with the view that resurgence is a renewal effect in which extinction of an instrumental behavior is specific to the context provided by rewarded leverpressing during the extinction phase.

A characteristic feature of extinction in both instrumental and Pavlovian conditioning paradigms is the relative fragility of the reduction in behavior that the procedure produces (e.g., Bouton & Swartzentruber, 1991). Indeed, although "unlearning" accounts of extinction are sometimes discussed, and embodied in otherwise powerful models of conditioning (e.g., Rescorla & Wagner, 1972), relatively enduring extinction seems to be the exception, not the rule. Continuing effects of original training on post-extinction performance may be uncovered by a wide range of manipulations, from simply allowing spontaneous recovery via the passage of time, to reinstating pre-extinction behavior via the reintroduction of primary reward or an unconditioned stimulus, to renewing behavior by removing the animal from the extinction context. As different as these procedures appear, the common thread is that all seem to depend upon the failure of what is learned in extinction to generalize beyond the extinction context (Bouton, 2002).

Resurgence, a form of relapse that has primarily been examined in instrumental learning tasks, has seen little recent empirical scrutiny. In resurgence, when a previously acquired behavior is no longer rewarded while a second, novel behavior delivers reward, subsequent extinction of that new behavior will result in recovery of the original one. That is, resurgence is a return to performance of the original behavior found when all reward is discontinued (for a discussion of this and other uses of the term "resurgence" see Cleland, Guerin, Foster, & Temple, 2001). This definition is broad enough to capture several potentially heterogeneous phenomena. For example, a resurgence effect was produced when Lindblom and Jenkins (1981) provided unpredicted rewards during extinction of autoshaped keypecking behavior, then at test stopped the novel reward deliveries; when Epstein (1983) rewarded and then extinguished an alternative behavior following extinction of the original one; and when Leitenberg, Rawson, and Bath (1970) first rewarded an alternative behavior during extinction of the original and then stopped rewarding it at test. Although these procedures all induced a recovery of the original behavior when the alternative source of reward was removed or extinguished, they may well have acted via distinct mechanisms.

The present experiments explored the form of resurgence first investigated by Leitenberg and colleagues (Leitenberg et al., 1970; Leitenberg, Rawson, & Mulick, 1975; see also Lieving & Lattal, 2003, Experiments 2 and 3), who developed and tested a specific hypothesis to account for the increasing amount of activity on the original behavior at test (Rawson, Leitenberg, Mulick, & Lefebvre, 1977). Their typical experiment employed three phases: (1.) an initial training phase where rats were, for instance, allowed to press one lever (L1) for food reward, (2.) an extinction training phase where L1 presses were not rewarded while presses on a novel, alternative lever (L2) were, and (3.) a test phase in which both levers were presented in extinction. Upon extinction of L2, animals resumed pressing L1 (i.e. resurgence was observed during Phase 3), and during extinction of L1, animals rewarded on L2 pressed L1 much less frequently than control animals not rewarded for L2 presses (i.e. response competition was observed in Phase 2). Based upon the latter finding, Rawson et al. argued that if the number of extinction presses emitted is a critical determinant of the strength of extinction, then animals with an alternative source of reward might have developed much weaker extinction because of response competition. When the distracting reward source was then removed, itself via extinction, these animals would have been able to manifest the relatively more intact acquisition associations in the form of greater L1 pressing. Resurgence thus occurred because animals were functionally prevented, by response competition, from emitting sufficient L1 presses to exhibit robust extinction. Rawson et al. therefore compared resurgence to an explicit response prevention procedure as a determinant of later extinction performance, and found comparable effects of the two treatments. Although this "response prevention" account is difficult to apply to other examples of resurgence, such as Epstein’s (1983) result (in which the first behavior was fully extinguished before the second behavior was trained), it does appear consistent not only with the body of Leitenberg's studies, but also with the effect detected by Lindblom and Jenkins (1981), where free reward could have produced behavior that competed with extinction of L1.

The difficulties the response prevention account has with Epstein's (1983) findings suggest that different forms of resurgence may be controlled by different mechanisms. A common mechanism, however, may be involved: key to all resurgence procedures is the introduction of a clear change in background stimuli during both the transition to extinction and the shift to testing. Since extinction learning is highly context-specific (e.g., Bouton, 2004), extinction performance of the original behavior may depend on the continued presence of either the second behavior or its associated reward -- both of which are reduced or removed during the resurgence test. “Renewal” of an extinguished instrumental behavior can be observed when training occurs in one context (i.e., an "A" context), extinction occurs in a different context (i.e., a "B" context), and animals are returned to the training "A" context for test (this is "ABA" renewal -- see e.g., Nakajima, Tanaka, Urushihara, & Imada, 2000; Bouton & Swartzentruber, 1991). It is plausible that "context" may be provided by both the actions the animal performs (e.g., Weise-Kelley & Siegel, 2001) and the reward it earns through that performance (e.g., Bouton, Rosengard, Achenbach, Peck, & Brooks, 1993). An inactive or absent L2 during L1 training could act as Context A, while the establishment of L2 as a source of reward could act as Context B. At the final extinction test, if L2 was present during initial acquisition the animal would return to Context A, or if it was absent the animal would experience a "C" context ("ABC" renewal is well established in Pavlovian conditioning [e.g., Bouton & Bolles, 1979], although demonstrations in instrumental conditioning are currently lacking). Renewal of extinguished performance is thus a direct and general explanation of the return to leverpressing evinced at the resurgence test phase.

The present article reports four experiments that examined several questions that remain open about the form of resurgence studied by Leitenberg et al. (1970). All previous demonstrations of resurgence involved an increase in the rate of reward between Phase 1, when L1 was trained, and Phase 2, when L1 was extinguished during concurrent L2 training. Such an increase might produce an L2 that is especially effective at competing with L1, and hence an L2 that is particularly likely to release suppressed L1 pressing when shifted to an extinction contingency. Experiments 1 and 2 therefore asked whether resurgence could occur when Phase 2 reward for L2 pressing was delivered at a rate equal to, or lower than, that experienced during Phase 1 L1 pressing. A second open question was whether resurgence depends on initial reward and extinction of L1. It is conceivable that extinction of L2 might cause an increase in any behavior, regardless of its history of association with reward, because a decline in L2 pressing produces a vacuum in behavior, or because frustration would invigorate any available behavior. Experiment 3 therefore asked whether the resurgence effect is in fact another example of the recovery of a learned-then-extinguished behavior. A third question was whether resurgence depends on the nature of the response contingency employed in Phase 2. Experiment 4 thus examined the effects of the contingency between L2 pressing and reward (which was delivered according to ratio, interval, or noncontingent schedules of reinforcement, with rate matched across treatments). Overall, the results suggest that resurgence can occur over a wide set of conditions, and that response prevention plays little necessary role.

Experiment 1

Experiments 1 and 2 examined the effect of the schedule of reinforcement used in Phase 2, and in particular the relationship between the rate of reward delivery in Phase 1 (for pressing L1) and the rate of reward delivery in Phase 2 (for pressing L2). In Experiment 1, we compared resurgence in groups that received either no shift or an upshift in potential reward rate between Phases 1 and 2. In Experiment 2, we compared resurgence in groups that received either no shift or a downshift in rate of reward. Previous research on this version of resurgence in rats has only shown resurgence when there has been an upshift in reward rate between Phases 1 and 2 (although pigeons show resurgence when the schedule is not changed, see Lieving and Lattal, 2003). Additionally, most past research employed variable interval schedules of reinforcement in Phase 1, while using fixed ratio schedules of reinforcement in Phase 2, complicating analysis of the effect by introducing potentially confounding effects of the changing schedules of reinforcement. These experiments directly addressed the possibility that the transition from Phase 1 to Phase 2 would produce resurgence even without a qualitative change in the schedule of reinforcement.

The design of Experiment 1 is illustrated in Table 1. Three groups received initial L1 training on a random interval 30-sec (RI 30) schedule of reinforcement. During Phase 2, while L1 was being extinguished, one group received training on L2 with an increased rate of reward relative to Phase 1 (Group RI 10) and another group received training on L2 with the same rate of reward employed in Phase 1 (Group RI 30). A control group received only extinction (Group EXT). The question was whether there would be a difference in resurgence in the two RI groups. From the perspective of the response prevention hypothesis, there might be, because an RI 10 schedule of reinforcement on L2 could cause more substantial interference with L1 pressing (e.g. because of either a higher rate of L2 pressing or more time spent in foodcup-related behavior). This interference would prevent L1's effective extinction. Since L2 data was typically not reported in detail in previous experiments, we examined performance on both levers in Phases 2 and 3, during their concurrent availability.

Table 1.

Experimental Designs

Experiment	Group	Phase 1	Phase 2	Phase 3
1	RI 10	L1 - RI 30	L1 - Ø ; L2 - RI 10	L1 - Ø v. L2 – Ø
	RI 30	L1 - RI 30	L1 - Ø ; L2 - RI 30	L1 - Ø v. L2 – Ø
	EXT	L1 - RI 30	L1 - Ø ; L2 - Ø	L1 - Ø v. L2 - Ø
2	RI 10	L1 - RI 10	L1 - Ø ; L2 - RI 10	L1 - Ø v. L2 - Ø
	RI 30	L1 - RI 10	L1 - Ø ; L2 - RI 30	L1 - Ø v. L2 – Ø
	EXT	L1 - RI 10	L1 - Ø ; L2 - Ø	L1 - Ø v. L2 - Ø
3	RSRG	L1 - RI 30 ; L2 - Ø ; Ø	L1 - Ø ; L2 - RI 10	L1 - Ø v. L2 – Ø
	FREE	L1 - Ø ; L2 - Ø ; YPEL	L1 - Ø ; L2 - RI 10	L1 - Ø v. L2 – Ø
	X	L1 - Ø ; L2 - Ø ; Ø	L1 - Ø ; L2 - RI 10	L1 - Ø v. L2 - Ø
4	FR 10	L1 - RI 30	L1 - Ø ; L2 - FR 10 ; Ø	L1 - Ø v. L2 - Ø
	Yoked-VT	L1 - RI 30	L1 - Ø ; L2 - Ø ; YPEL	L1 - Ø v. L2 – Ø
	Yoked-VI	L1 - RI 30	L1 - Ø ; L2 - YPEL ; Ø	L1 - Ø v. L2 – Ø
	EXT	L1 - RI 30	L1 - Ø ; L2 - Ø ; Ø	L1 - Ø v. L2 – Ø

Note. Reward always consisted of pellet delivery. L1 indicates the initially rewarded lever, while L2 indicates the alternative source of reward available during the extinction of L1. RI 10 and RI 30 denote random interval 10- respectively 30-sec schedules of reinforcement. FR 10 indicates a fixed-ratio 10 schedule of reinforcement. YPEL indicates yoked pellet availability or delivery.

Method

Subjects and Apparatus

The subjects were 32 female Wistar rats obtained from Charles River, Inc. (St. Constance, Quebec). They were approximately 85–95 days old at the start of the experiment and were individually housed in suspended stainless steel cages in a room maintained on a 12:12-hr light:dark cycle. They were run during the illuminated portion of the cycle at the same time each day. The rats were food deprived to 80% of their free-feeding weights and maintained at that level throughout the experiment.

The rats were conditioned in two rooms, each with a different set of four standard conditioning boxes (Med-Associates, St. Albans, VT), modified slightly as described below for use as separate contexts in other experiments (box context was balanced across groups in all of the present experiments). Boxes from both sets measured 31.5 × 25.4 × 24.1 cm (l × w × h), with side walls and ceilings made of clear acrylic plastic and front and rear walls made of brushed aluminum. Recessed 5.1 × 5.1-cm foodcups with infrared photobeams positioned approximately 1.2 cm behind the plane of the wall and 1.2 cm above the bottom of the cup were centered in the front wall about 3 cm above the grid. In one set of four boxes, the floor was composed of stainless steel rods (0.5 cm in diameter) in a horizontal plane spaced 1.6 cm center to center, while in the other set of four boxes, the floor was composed of identical rods spaced 3.2 cm apart in two separate horizontal planes, one 0.6 cm lower than the other and horizontally offset by 1.6 cm. The boxes with the planar floor grid had a side wall with black panels (7.6 × 7.6 cm) placed in a diagonal arrangement, and there were diagonal stripes on both the ceiling and back panel, all oriented in the same direction, 2.9 cm wide, and about 4 cm apart. The other boxes, with the staggered floor, were not adorned in any way. Retractable levers (1.9 cm when extended) were placed approximately 3.2 to the right and to the left of the magazine and 6.4 cm above the grid. Both sets of boxes were housed in sound-attenuating chambers, and could be illuminated by two 7.5-W incandescent light bulbs (the houselights) mounted to the ceilings of the chambers. Food reward consisted of 45-mg MLab Rodent Tablets (TestDiet, Richmond, IN).

Procedure

Consecutive daily sessions of 30-min duration were employed throughout the experiment. Sessions were conducted with 4–6 hr of lighted colony time remaining, and all rats were fed after all daily conditioning sessions were complete. Group running order was mixed, with animals from each group in each session in a partially balanced order. The rats were placed in darkened chambers. The start of each session was then indicated by houselight illumination and, for instrumental sessions, lever insertion. The end of each session was indicated by houselight termination and, for instrumental sessions, lever retraction.

Magazine training

All animals received an initial session of magazine training. Food pellets were delivered freely on a random time 30-sec schedule (RT 30), such that all rats received approximately 60 pellet deliveries over the course of their first 30 min in the conditioning chamber. This schedule was programmed by delivering a pellet in a given second with a 1 in 30 probability.

L1 conditioning (Phase 1)

All animals were then given five sessions of instrumental conditioning. For half the animals the left lever was inserted during these sessions, and for the other half the right lever was inserted. Regardless of which lever was available, presses earned a single pellet delivery on an RI 30 schedule of reinforcement throughout training. This schedule was programmed by initiating pellet availability in a given second with a 1 in 30 probability. Pellets remained available until the next lever press, at which point the pellet was delivered and the schedule mechanism was restarted.

L1 extinction and L2 conditioning (Phase 2)

Animals were then given four further sessions of instrumental conditioning. For all animals, both the left and the right lever were inserted during these sessions. From the outset they experienced one of three contingencies upon the novel L2, which was in the opposite position of L1. Group RI 30 was rewarded with pellets on an RI 30 schedule of reinforcement on L2, Group RI 10 on an RI 10 schedule, and Group EXT was an extinction control group where press on both L1 and L2 had no consequences. Twelve animals were assigned to each experimental group (RI 30 and RI 10), and eight were assigned to the extinction control group.

Resurgence test (Phase 3)

All animals were given a final test session, in which both levers were inserted, and neither presses of the left nor of the right lever delivered pellets. The resurgence test was thus a pure extinction test for all animals.

Statistical analysis

For this and the following experiments, analysis of variance with a rejection criterion of p < .05 was employed in all inferential tests.

Results and Discussion

All training and test data are presented in Figure 1. Acquisition of the original leverpress proceeded normally in all groups (Figure 1A), which performed a comparable number of presses in the final training session, F < 1, MSE = 45405. Analysis of acquisition revealed a reliable increase in L1 presses over days, F(4, 116) = 131.63, MSE = 9215, but neither a group effect, F < 1, MSE = 120692, nor a group x day interaction, F < 1, MSE = 9215.

Figure 1. Results of Experiment 1.

A. Daily leverpressing means from Phase 1 acquisition of L1. B. Phase 2 extinction of L1. C. Phase 2 acquisition of L2. D. Phase 3 (resurgence) test of L1 (Note reduced y-axis scale). E. Phase 3 extinction of L2. Group labels and treatments are described in the text.

Availability of the alternative source of reward (L2) facilitated the rate of extinction of L1, but only in Group RI 10 (Figure 1B). Analysis of L1 presses during Phase 2 revealed a reliable group x day interaction, F(3, 87) = 3.97, MSE = 3040. Pairwise factorial analyses indicated that the interaction was carried by the apparent difference between Group RI 10 and both other groups. Analyses of Group RI 10 against Group RI 30, and of Group RI 10 against Group EXT, both revealed decreasing L1 pressing in all groups, with reliable day effects, Fs(3, 54) ≥ 58.59, MSEs ≤ 3012; more importantly, Group RI 10 pressed reliably less than either Group EXT or Group RI 30, with group effects, Fs(1, 18) ≥ 12.64, MSEs ≤ 7362, and Group RI 10 leverpress performance changed differently than in the other groups, with reliable group x day interactions, Fs(3, 54) ≥ 4.10, MSEs ≤ 3012. In contrast, Group RI 30 and Group EXT both decreased L1 performance during extinction, F(3, 54) = 79.65, MSE = 4114, but there was neither an effect of group, F < 1, MSE = 10007, nor a group x day interaction, F < 1, MSE = 4114. All groups appeared similarly extinguished by the final day of extinction training, as analysis of the that day failed to reveal group differences in performance, F(2, 29) = 1.92, MSE = 886.

L2 pressing showed clear sensitivity to the delivery of reward (Figure 1C). In Group EXT, it reliably declined over days, F(3, 21) = 23.45, MSE = 164; factorial analysis of L2 in the RI groups showed a reliable increase over days, F(3, 66) = 34.55, MSE = 20945, a reliable group x day interaction, F(3, 66) = 4.07, MSE = 20945, but no overall effect of group, F < 1, MSE = 161134. Comparison of final extinction day L2 pressing in these groups, however, revealed no reliable difference between Group RI 10 and Group RI 30, F(1, 22) = 2.40, MSE = 81058.

According to the response prevention hypothesis, because Group RI 10 emitted fewer L1 presses in extinction, it should have shown the strongest resurgence effect. In the resurgence test, Group EXT showed continued extinction of L1-directed behavior, but both of the other groups increased L1 pressing (Figure 1D). An analysis comparing the amount of L1 pressing during the final day of extinction with that in the test session found a reliable group x day interaction, F(2, 29) = 4.93, MSE = 1519. Two planned comparisons for each treatment group were used to examine this interaction: a between-subjects analysis of test day L1 performance in each group versus the performance of Group EXT, and a within-subjects analysis of the change in each group's L1 pressing from the final extinction day to the test day. L1 pressing in Group RI 10 was reliably elevated on the test day in comparison both to Group EXT, F(1, 18) = 8.12, MSE = 1985, and to its own final extinction day performance, F(1, 11) = 8.78, MSE = 2253. Similarly, Group RI 30 pressing was elevated both relative to Group EXT, F(1, 18) = 9.08, MSE = 1894, and with respect to its own behavior on the final extinction day, F(1, 11) = 6.11, MSE = 1357. Furthermore, analysis of just Groups RI 10 and RI 30 showed a reliable day effect, F(1, 22) = 14.87, MSE = 1805, but neither a group effect, F < 1, MSE = 2118, nor a group x day interaction, F < 1, MSE = 1805.

This experiment succeeded in replicating the resurgence effect: in both Groups RI 10 and RI 30, L1 pressing returned when L2 underwent extinction. Although it is tempting to imagine resurgence as a return to absolute preference for L1 at the test session, that is, as a pure form of primacy of the original conditioning, these data clearly show that the resurging animals produced a greater number of L2 presses than L1 presses during the test. Resurgence therefore did not take the form of an absolute preference for the initially-rewarded lever.

The results were also not consistent with the response prevention hypothesis. Groups EXT and RI 30 performed comparably on L1 throughout the extinction training phase, and yet Group RI 30 still showed a reliable return to pressing that lever when L2 was extinguished for the resurgence test. Neither a qualitative nor a quantitative change in schedule of reinforcement between Phases 1 and 2 was required, as resurgence was detected here when initial conditioning and alternative lever training both utilized the same RI 30 schedule.

Experiment 2

The design of Experiment 2 is presented in Table 1. Three groups received initial L1 training with an RI 10 schedule of reinforcement. During Phase 2, L1 was once again extinguished in all groups. However, at this time, one group received L2 training with a relatively low rate of reward (an RI 30 schedule was employed) whereas another group received training on L2 with the same rate of reward used in Phase 1 (RI 10). The third (control) group received merely extinction. We were especially interested in whether a decrease in rate of reward would allow a resurgence effect to emerge in Phase 3, when both L1 and L2 were tested on extinction.

Leitenberg et al. (1975, Experiment 4) previously tested for resurgence with a downshift in rate of reward. They trained pigeons to peck a key (K1) on a VI 120 schedule, then shifted the schedule either to a leaner (VI 240) or richer (VI 30) schedule of reinforcement on a new key (K2) while K1 was placed in extinction. As usual, they tested the pigeons with both keys in extinction. Only the shift to a richer schedule of reinforcement on K2 during Phase 2 produced resurgence; it also produced a pronounced reduction in K1 pecking during the extinction training phase (whereas rewarding K2 on the leaner, VI 240 schedule did not). These results were interpreted as consistent with the response prevention hypothesis. However, the VI 240 schedule was so lean that it is likely that the birds underwent considerable extinction of K1 before they ever encountered reward in Phase 2. Thus, it is not clear that the experiment used a leaner schedule that was discriminably different from extinction (which would lead to the non-renewal-producing "ABB" set of contextual transitions), and the question of whether resurgence occurs after shifting to a leaner schedule in Phase 2 therefore still appears to be an open one. The present experiment sought to test the resurgence effect in animals either maintained at a constant density of reward in the two phases (Group RI 10) or shifted from a high density of reward to a lower density of reward that remained discriminably different from extinction (Group RI 30).

Method

Subjects and Apparatus

The subjects were 31 female Wistar rats, again obtained from Charles River, Inc. and otherwise treated identically to those employed in the previous experiment. The apparatus was also unchanged.

Procedure

The current procedure followed that of Experiment 1 except as noted. After an initial magazine training session, all animals received five sessions of instrumental conditioning of L1 on an RI 10 schedule of reinforcement. For 16 of the animals the left lever was inserted during these sessions, and for the other 15 the right lever was inserted. All rats then received four further sessions of instrumental conditioning. All animals were able to press L1 without reward delivery, while experiencing one of three other contingencies upon the novel L2, which was once again for all animals inserted in the opposite position of L1. Group RI 30 was given pellets on an RI 30 schedule for L2 presses, Group RI 10 on an RI 10 schedule, and Group EXT earned no reward for L2 presses. There were 12 animals in each experimental group (RI 10 and RI 30), and seven were assigned to the extinction control group. All animals were given a resurgence test in a final session in which both levers were inserted and leverpresses delivered no pellets.

Results and Discussion

Training and test data are presented in Figure 2. Acquisition of L1 again proceeded normally in all groups (Figure 2A), which performed a comparable number of presses in the final training session, F < 1, MSE = 44689. Analysis of the acquisition data revealed a reliable increase over days, F(4, 112) = 84.53, MSE = 9008, but there were neither overall differences in the amount of L1 pressing across groups, F(2, 28) = 1.04, MSE = 79159, nor was there an interaction between group and training day, F < 1, MSE = 9008.

Figure 2. Results of Experiment 2.

A. Daily leverpressing means from Phase 1 acquisition of L1. B. Phase 2 extinction of L1. C. Phase 2 acquisition of L2. D. Phase 3 test of L1 (Note reduced y-axis scale). E. Phase 3 extinction of L2. Group labels and treatments are described in the text.

Availability of an alternative source of reward in pressing L2 again affected extinction of L1. Interestingly, the shift to a leaner schedule of reinforcement on L2 appears to have, if anything, retarded extinction of L1 relative to simple extinction (Figure 2B). Examination of these effects revealed a reliable effect of day of extinction, F(3, 84) = 119.59, MSE = 3667, a marginal group effect, F(2, 28) = 3.31, p < .10, MSE = 7095, and no interaction, F < 1, MSE = 3667. Pairwise group x day ANOVAs revealed a group difference between Groups RI 10 and RI 30, F(1, 22) = 5.50, MSE = 7817, a marginal group effect between Groups RI 30 and EXT, F(1, 17) = 4.04, p < .10, MSE = 5270, and no group effect between Groups RI 10 and EXT, F < 1, MSE = 7985; no interactions approached reliability, Fs(3, max 66) ≤ 1.52, MSEs ≥ 2782, and all day effects were reliable, Fs(3, min 51) ≥ 74.74, MSEs ≤ 4331. The groups were no longer different by the final day of extinction training, F < 1, MSE = 1029.

L2 pressing again showed effects of the various schedules of reinforcement (Figure 2C). Group EXT pressing reliably declined over days, F(3, 18) = 3.53, MSE = 1095. Analysis of pressing in the RI groups showed a reliable increase over days, F(3, 66) = 39.38, MSE = 24362, reliably greater leverpressing overall in Group RI 10, F(1, 22) = 6.70, MSE = 134263, and no interaction between group and day, F < 1, MSE = 24362.

In the resurgence test, Group EXT again showed continued extinction of L1 pressing, but both of the other groups increased L1 pressing (Figure 2D). An analysis comparing L1 pressing during the final day of extinction with that during the test session in all groups revealed a reliable group x session interaction, F(2, 28) = 5.76, MSE = 913. Two planned comparisons for each treatment group were again used to examine the reliability of the resurgence effect: a between-subjects analysis of test day L1 performance in each group versus the performance of Group EXT, and a within-subjects analysis of the change in the group's L1 pressing from the final extinction day to the test day. Group RI 10 showed a reliable elevation of L1 pressing on the test day with respect both to Group EXT, F(1, 17) = 27.47, MSE = 686, and to its own final extinction day performance, F(1, 11) = 15.22, MSE = 1303. Group RI 30 was also elevated relative to Group EXT, F(1, 17) = 5.24, MSE = 1157, but with respect to its behavior on the final extinction day, it did not reliably increase performance, F(1, 11) = 1.85, MSE = 838.

Further analysis, however, supported the idea that Group RI 30 did show a resurgence effect at test. It is important to note that L2 pressing in this group during Phase 2 was relatively low. The shift to a leaner schedule of reinforcement in Phase 2 (a transition from RI 10 to RI 30) evidently made acquisition of the new leverpress difficult. Interestingly, one animal in Group RI 30 failed to extinguish on L1; her daily Phase 2 L1 press totals were 142, 101, 132, and 104, leaving her with the highest rate of L1 pressing of all of the rats on the final extinction day. She also showed very slow acquisition of L2 pressing, emitting just over 50% of the L2 presses emitted by the next slowest rat, and achieving only 33% of the group mean over the phase. Thus, this animal failed to convincingly learn the contingencies of Phase 2. When she was eliminated from the data analysis, Group RI 30 showed a clear within-subjects, across-days resurgence effect, F(1,10) = 5.37, MSE = 563. A group effect in Phase 2 L1 pressing also emerged, F(2, 27) = 3.46, MSE = 7243, but the pattern of reliability reported above was otherwise unchanged by exclusion of this rat. In the interest of brevity, Figure 2 therefore depicts data omitting this rat from all phases.

The results of this experiment confirm that a resurgence effect is possible when L2 is rewarded on a leaner reinforcement schedule than was L1. Since the leaner schedule did not cause suppression of L1 pressing during Phase 2, the finding further challenges the response prevention hypothesis. This experiment shows that even when a group performed more L1 presses during extinction than an extinction control group, it was still capable of producing a reliable resurgence effect at test. Consistent with the contextual analysis, it appears that a transition to either a leaner or richer reinforcement schedule from Phase 1 to 2 followed by the transition to extinction in Phase 3 may be sufficient to cause resurgence of L1 pressing.

Experiment 3

Although many potential mechanisms could produce the resurgence effect observed in Experiments 1 and 2, one possibility has seen no direct empirical scrutiny in this preparation (cf. Epstein, 1983). Increased L1 pressing in Phase 3 might be a unique consequence of the history of reward of both levers, but alternatively, it is in principle possible that Phase 1 training is in fact not required to produce the apparent "resurgence" in behavior. Instead, extinction of L2 could produce a heightened level of activity that could result in considerable pressing on a previously non-rewarded lever via generalization, frustration, or other non-specific mechanisms. The absence of experiments directly addressing the role of Phase 1 training in producing resurgence is a significant omission in the resurgence literature.

To address this concern, Experiment 3 was designed to contrast the full resurgence training procedure in which rats received training on both L1 and L2 with a version that omitted initial instrumental conditioning of L1. It compared resurgence of leverpressing in animals (Group RSRG) that received a treatment similar to Group RI 10 in Experiment 1 with that in two groups of animals with no initial instrumental conditioning of L1 (see Table 1). Both of these new groups received Phase 1 sessions in which L1 was available, but presses were not rewarded. Group X received no pellets at all during these sessions, while rats in Group FREE received free pellets using a yoking procedure in which individual rats received a pellet whenever a matched rat from Group RSRG earned a pellet. For all groups, Phases 2 and 3 proceeded identically. Phase 2 consisted of the unrewarded availability of L1 while presses on the novel L2 delivered pellets on an RI 10 schedule of reinforcement. Phase 3 consisted of the extinction of all pellet delivery with both levers simultaneously available. If resurgence in Phase 3 occurs due to the previous reward of a behavior, Group RSRG alone should show the resurgence effect by increasing L1 pressing when shifted from Phase 2 to Phase 3.

Method

Subjects and Apparatus

The subjects were 24 female Wistar rats, again obtained from Charles River, Inc., tested in the same apparatus as, and otherwise treated identically to those employed in Experiments 1 and 2.

Procedure

Consecutive daily sessions of 30-min duration were again employed throughout the experiment, with session timing identical to the first experiment. The start of each session was indicated by houselight illumination, and lever insertion for instrumental sessions. The end of each session was indicated by houselight termination, and when applicable, lever retraction.

All animals received an initial 30-min session of RT 30 magazine training, and then five sessions of instrumental conditioning. Rats were randomly assigned to three groups, and were run daily in three squads, with animals from each group represented in each squad. During Phase 1, both levers were concurrently available, but L2 presses were never rewarded. Group RSRG earned reward on L1 on an RI 30 schedule of reinforcement. Group FREE received no programmed consequences for L1 pressing but received a pellet every time a matched member of Group RSRG earned one. Group X received neither programmed consequences for L1 pressing nor unearned pellets.

All groups then received four further sessions, in which all animals were able to press L1 without reward delivery. All animals were also able to press L2 (e.g., the right lever when the left was originally conditioned) for reward. The first 10 L2 presses delivered pellets on CRF, and then all following pellets were delivered on RI 10. Both schedules were chosen to maximize the likelihood that the rats, all of which had extensive unrewarded experience of L2 in Phase 1, would acquire the L2 - pellet contingency.

Finally, all animals were given a test session in which both levers were inserted and neither presses of the left nor the right lever delivered pellets. The resurgence test was thus again a pure extinction test for all animals.

Results and Discussion

All training and test data are presented in Figure 3. Acquisition of the L1 pressing proceeded normally in Group RSRG, which performed an increasingly greater number of presses than both Groups X and FREE over the course of Phase 1 training (see Figure 3A). Group RSRG showed a reliable increase in pressing over days, F(4, 28) = 23.53, MSE = 12826; Groups X and FREE did not differ from one another, F(1, 14) = 1.97, MSE = 274, and did not change differently over days of training, F < 1, MSE = 60, but L1 pressing in both groups reliably declined from its initially low level over the course of training, F(4, 56) = 5.13, MSE = 60, indicating the elimination of a baseline tendency to press even inactive levers.

Figure 3. Results of Experiment 3.

Daily leverpressing means from Phase 1 acquisition of L1. B. Phase 2 extinction of L1. C. Phase 2 acquisition of L2. D. Phase 3 test of L1. E. Phase 3 extinction of L2. Group labels and treatments are described in the text.

During Phase 2, the groups appeared to acquire L2 pressing at roughly comparable rates (see Figure 3C). Analysis of these data confirmed this impression, revealing a reliable increase in L2 pressing during L1 extinction, F(3, 63) = 50.51, MSE = 14310, but neither a group effect, F(2, 21) = 1.50, MSE = 190612, nor a group x day interaction, F < 1, MSE = 14310. L1 pressing was, however, differently affected by Phase 2. Group RSRG pressed L1 less over days, F(3, 21) = 21.42, MSE = 841; Group FREE and X L1 pressing differed over Phase 2, with a reliable group x day interaction, F(3, 42) = 3.93, MSE = 83, where Group FREE reliably decreased over days, F(3, 21) = 3.91, MSE = 77, but Group X showed no change over days, F < 1, MSE = 89. Of more importance, however, is the fact that all groups exited the final day of Phase 2 extinction with comparable L1 performance, as assessed by analysis of L1 presses during the final session, F(2, 21) = 2.61, MSE = 322.

During the resurgence test, all groups increased L1 pressing relative to the final day of Phase 2, but Group RSRG did so to a much greater degree than either Group EXT or Group FREE (see Figure 3D). In the analysis, a reliable interaction between group and session was detected, F(2, 21) = 13.92, MSE = 347. Performance of each group on the final day of Phase 2 and the test day was contrasted in order to analyze this interaction. All three groups reliably increased L1 pressing from the final extinction day to the resurgence test: Group RSRG, F(1, 7) = 27.84, MSE = 869, Group Free, F(1, 7) = 16.18, MSE = 148, and Group X, F(1, 7) = 27.17, MSE = 23. Comparison of the groups on the test day, however, revealed a reliable effect of group, F(2, 21) = 16.21, MSE = 892. At this time, Groups X and FREE failed to differ, F < 1, MSE = 418, while both showed reliably less pressing than Group RSRG, Fs(1, 14) ≥ 18.60, MSEs ≤ 1166.

These results suggest that extinction of L2 during Phase 3 can cause a slight, but detectable, increase in pressing of a never-rewarded lever. This result is consistent with frustration theory (Amsel, 1962), which would emphasize the general energizing effects upon behavior of the frustration generated by nonreward. Alternatively, rats undergoing extinction of L2 might be left with a vacuum in behavior, and that vacuum could be filled by any activity available to the animal. Whatever the mechanism for the slight increase observed in the absence of a history of reward for pressing L1, it is also clear that the increase was much more pronounced in the group with such a history. Thus, resurgence does appear to be another post-extinction phenomenon indicating that extinction -- and reward of an alternative behavior -- neither erases nor destroys the original learning.

Experiment 4

The fourth experiment examined the effects of different types of reinforcement schedule in Phase 2, as well as the leverpress - pellet contingency itself, on the resurgence effect (see Table 1). There were four groups, all of which received L1 variable interval training and then extinction, but the relationship between L2 presses and pellet delivery in Phase 2 was manipulated across groups. One group received pellets on a ratio schedule (Group FR 10), as have rats in past research into this form of the resurgence effect (Leitenberg et al., 1975; Rawson et al., 1977). A second group received a free pellet whenever a paired animal in the ratio group earned reward (Group Yoked-VT). A third group received a functional variable interval schedule in which a pellet became available as reward for the next L2 press when a paired animal in the ratio group earned a pellet (Group Yoked-VI). A fourth group was an extinction only control (Group EXT). Note that the density of pellet delivery across the three experimental groups was potentially equated in Phase 2. If the leverpress - pellet contingency, or L2 pressing rate (which would be predictably higher in Group FR 10 than in either yoked group) contributes to the resurgence effect, we expected to see corresponding differences in resurgence.

Method

Subjects and Apparatus

The subjects were 32 female Wistar rats, again obtained from Charles River, Inc., tested in the same apparatus as and otherwise treated identically to those in the previous experiments.

Procedure

The procedure was the same as that employed in the other experiments except as noted. After an initial magazine training session, all rats received four sessions of instrumental conditioning in which either the left lever or right lever was inserted (each for half the animals) and given pellet deliveries for presses on an RI 30 schedule of reinforcement. Animals then experienced three further sessions of instrumental conditioning, in which pressing the L1 lever no longer led to reward. Four groups were employed, differing only on L2 - reward contingencies (L2 was, for all rats, the opposite lever to that used as L1 in Phase 1). For Group EXT, L2 presses also had no consequences, and thus these sessions were conducted under total extinction. Group FR 10 was able to earn pellets for pressing L2 on a fixed ratio 10 (FR 10) schedule of reinforcement. The two novel groups had pellet deliveries yoked to those experienced by Group FR 10 via two different schedules. Group Yoked-VT animals were given pellets whenever the animal to whom they were yoked in Group FR 10 earned pellets; pellet delivery was thus not contingent upon their leverpressing. Group Yoked-VI animals were given the opportunity to earn a pellet whenever the animal to whom they were yoked in Group FR 10 itself earned reward; pellet delivery was thus contingent upon leverpressing, but with a variable interval schedule of reinforcement in which each successive interval was determined by pellet delivery in a member of Group FR 10.

All groups were then given a final resurgence test session in which both levers were inserted and neither presses of the left nor the right lever were rewarded -- a pure extinction test.

Results and Discussion

All training and test data are presented in Figure 4. Acquisition of L1 pressing proceeded normally in all groups (Figure 4A), which performed a comparable number of presses in the final training session, F < 1, MSE = 23727. Analysis of the acquisition data revealed a reliable effect of training day, F(3, 78) = 79.71, MSE = 11865, but neither an effect of group, F < 1, MSE = 79811, nor an interaction between the factors, F < 1, MSE = 11865.

Figure 4. Results of Experiment 4.

A. Daily leverpressing means from Phase 1 acquisition of L1. B. Phase 2 extinction of L1. C. Phase 2 acquisition of L2. D. Phase 3 test of L1 (Note reduced y-axis scale). E. Phase 3 extinction of L2. Group labels and treatments are described in the text.

The results of Phase 2 suggest that availability of an alternative source of reward again facilitated extinction of L1 (Figure 4B). However, the ANOVA uncovered only a reliable effect of extinction day, F(2, 52) = 126.96, MSE = 2618; both the effects of group, F(3, 26) = 1.35, MSE = 7905, and the group x day interaction, F < 1, MSE = 2618, failed to reach the rejection criterion. Pairwise group x day analyses revealed day effects, Fs(2, 26) ≥ 51.04, but no group x day interactions, Fs(2, 26) ≤ 2.05, MSEs ≤ 3593. While Group EXT showed greater pressing than Group FR 10, F(1, 13) = 7.13, MSE = 4424, all other effects of group were not reliable, Fs(1, 13) ≤ 1.36, MSEs ≥ 7287. Reward earned on L2 in Group FR 10 once again accelerated extinction of L1, while L1 performance in both the Yoked-VI and Yoked-VT groups may have been intermediate between that of Groups FR 10 and EXT. All groups, however, exhibited comparable extinction by the third and final day of the phase, F(3, 26) = 1.21, MSE =2082.

Any differences in L1 extinction were not due to differential rates of pellet delivery in the three rewarded groups. The average interval between pellet deliveries in Phase 2 was necessarily identical in Group FR 10 and Group Yoked-VT (M = 15.4 sec, SD = 12.4) and was essentially the same in Group Yoked-VI (M = 15.8 sec, SD = 13.6). Despite the equated density of reward, the groups differed substantially in L2 pressing (Figure 4C). In particular, Yoked-VI animals pressed L2 substantially less than FR 10 animals. Analyses confirmed the difference in pattern, in that pairwise group x day ANOVAs showed that Groups Yoked-VI and FR 10 both increased performance over days, F(2, 28) = 45.06, MSE = 16834, with Group FR 10 doing so more rapidly, as seen in a reliable effect of group, F(1, 14) = 5.24, MSE = 277391, and a group x day interaction, F(2, 28) = 9.42, MSE = 16834. Groups EXT and Yoked-VT both pressed L2 less over days, F(2, 24) = 9.93, MSE = 409, and did so comparably, with neither a group effect, F < 1, MSE = 3060, nor an interaction, F < 1, MSE = 409.

In the resurgence test, Group EXT once again showed continued extinction of L1, but all other groups increased L1 presses (Figure 4D). An analysis comparing the final day of extinction with the test session in all groups revealed a reliable interaction between group and session, F(3, 26) = 6.11, MSE = 1645. As in the previous experiments, we assessed resurgence using both within- and between-subjects contrasts. Test L1 presses in Group FR 10 were reliably higher than in Group EXT, F(1, 13) = 9.99, MSE = 2791, and higher than during the final extinction day, F(1, 7) = 14.19, MSE = 2538. Group Yoked-VT's test-day L1 pressing was also elevated relative to Group EXT, F(1, 12) = 5.36, MSE = 2518, and marginally higher than on the last extinction day, F(1, 6) = 4.39, p < .10, MSE = 1445. Group Yoked-VI showed a similar pattern, in that its performance was reliably elevated relative to Group EXT, F(1, 13) = 7.72, MSE = 1835, and marginally higher during the resurgence test session than on the last day of extinction, F(1, 7) = 4.63, p < .10, MSE = 1797.

These data suggest that the resurgence effect does not depend on the nature of the reinforcement schedule used in Phase 2 when the rate of pellet delivery is matched. They also provide further evidence that the response prevention hypothesis alone cannot account for the resurgence effect. Animals performed fewer L1 presses in Group FR 10 than in Group Yoked-VI, in Group Yoked-VI than in Group Yoked-VT, and in Group Yoked-VT than in Group EXT, but resurgence occurred regardless of these differences. A common factor was present, however, in that all groups receiving pellets during Phase 2 would have experienced a change in context from L1 extinction in Phase 2 caused by the removal of pellet availability in Phase 3. Since the groups' rates of L2 pressing were substantially different during Phase 2 (in line with Dawson and Dickinson, 1990; Dickinson, Nicholas, & Adams, 1983), the similarity of resurgence in these groups suggests that from a contextual analysis, the animal's ongoing behavior might not be encoded as a major feature of the extinction context. In this experiment, experienced pellet distribution is the most obvious candidate for a contextual stimulus, as it was held explicitly constant in all groups that showed resurgence, although categorical sensitivity to changes in ongoing L2 behavior (rather than sensitivity to L2 pressing rate) could also provide such a mechanism.

General Discussion

The results of the present series of experiments demonstrated that the resurgence phenomenon is more general than would be predicted based upon previously considered mechanisms of action. Experiments 1 and 2 explored resurgence when employing much less dramatic changes in schedule of reinforcement than employed by others (e.g., Leitenberg et al. 1975; Rawson et al. 1977), finding resurgence whether there was an upshift, downshift, or no change in the schedule of reinforcement, and hence potential rate of reward delivery, between Phases 1 and 2. These experiments also demonstrated that the degree of suppression of L1 responding during extinction is not related to the degree of resurgence observed at test. Experiment 3 greatly diminished the possibility that the effect is entirely non-associative by finding that robust resurgence depends upon prior training of L1. Finally, Experiment 4 demonstrated that different types of schedule of reinforcement used in Phase 2 (i.e. FR, RI, or RT) produced similar amounts of resurgence when rate of reward was held constant and that resurgence can be detected after a wide variety of schedules of reinforcement in Phases 1 and 2.

Response prevention played little demonstrable role in these experiments. Animals in several experimental groups pressed L1 as much as, or more than, Group EXT animals during Phase 2 extinction; despite their high level of extinction leverpressing, they nevertheless showed resurgence, contrary to the predictions of the response prevention hypothesis. Even very different patterns of L2 pressing, as observed in Experiment 4, produced only slight effects on L1 pressing, which subsequently failed to translate into differential resurgence at Phase 3. In contrast, frustration remains a possible explanation for the observed resurgence, but only to the extent that it could selectively modulate extant leverpress - pellet associations. Animals with no history of reward on L1 reacted to the extinction of L2 with a slight increase in L1 pressing, in line with the possibility of frustration, but the increase observed when L1 presses previously delivered pellets was significantly greater.

Although resurgence appeared when the schedule was changed between Phases 1 and 2, the results of Experiments 1 and 2 showed that it was also quite robust when the schedule was the same across these phases. Thus, the type of schedule transition between Phases 1 and 2 appears to be relatively unimportant in generating resurgence. In contrast, during the transition from Phases 2 and 3, reward was always eliminated, and L2 pressing consequently began to decline. Either of these changes could have been detected and acted as a change of context in the transition from Phase 2 to Phase 3. It seems unlikely, however, that gross changes in behavior during the transition to extinction were the sole cause of resurgence, because resurgence was similar despite substantially different amounts of L2 behavior at the end of Phase 2 (Experiment 4). Since the distribution of pellets was held constant across Experiment 4's experimental groups, the data suggest that the change in pellet distribution between Phases 2 and 3 might be important in producing the resurgence effect. Such a conclusion would be consistent with previous research suggesting that pellet deliveries can be encoded as part of the context in which extinction takes place (Bouton et al., 1993).

From a contextual perspective, the resurgence demonstrated in these experiments is most readily characterized as an ABC renewal effect (e.g., Thomas, Larson, and Ayres, 2003). As a first approximation, removing reward of L2 at the start of the test (Phase 3) might appear to return the animal to the original Phase 1 context, and thus enable an ABA renewal effect. However, unlike Phase 1, no pellets were delivered in Phase 3, and in this important sense the test moved the animal to a new, never-experienced context instead of returning it to the original context. One could also ask whether resurgence might alternatively be described as an AAB renewal effect, particularly since we observed resurgence when there was no nominal change in reinforcement schedule between Phases 1 and 2 (Experiments 1 and 2). However, even in that case, the animal was required to make a new leverpress during Phase 2, and this change was detectable in that most animals successfully performed L2 rather than L1 during the phase. Thus, from a contextual perspective, the rat entered a series of different contexts (A, B, and C) across phases that were defined by both behavior and by pellet presentation.

Resurgence may have implications for understanding relapse after therapies designed to eliminate problematic instrumental behaviors such as drug abuse. Contingency Management, where a novel source of reward is used by a therapist to support abstinence, has been shown to prevent relapse during therapy for several kinds of substance abuse (e.g. Miller, Hersen, Eisler, & Watt, 1974; Stitzer, Bigelow, & Liebson, 1980; Budney, Higgins, Delaney, Kent, & Bickel, 1991), an effect that parallels the ability of L2 reward to accelerate the extinction of L1 pressing in the present studies. Follow-up measurements, however, suggest reduced abstinence once the novel reward is discontinued (e.g. Petry, Alessi, Carroll, Hanson, MacKinnon, Rounsaville, & Sierra, 2006), an effect that parallels the resurgence in L1 pressing observed when L2 reward is discontinued. According to the contextual analysis suggested here, these relapse effects may be critically dependent on the contextual change produced by termination of the novel reward source, and hence should be mitigated by the same kinds of manipulations that minimize resurgence.

In sum, resurgence is neither heavily dependent upon response suppression nor upon the nature of schedules of reinforcement employed in Phases 1 and 2. It is also greatest when the resurging behavior is an extinguished behavior that was originally rewarded. On the whole, the results are consistent with the hypothesis that resurgence is a renewal effect in which extinction of an instrumental behavior is shown to be specific to the context in which it is learned.