Effects of Thinning the Rate at Which the Alternative Behavior is Reinforced on Resurgence of an Extinguished Instrumental Response

Abstract

Three experiments with rats examined the effects of thinning the rate of reinforcement for the alternative behavior in the resurgence paradigm. In all experiments, pressing one lever (L1) was first reinforced and then extinguished while pressing a second, alternative, lever (L2) was now reinforced. When L2 responding was then itself extinguished, L1 responses “resurged.” Resurgence was always observed when L2 was reinforced on an unchanging reinforcement schedule during Phase 2. However, other rats received systematic decreases in the rate of L2 reinforcement before extinction of L2 began. Such a “thinning” procedure was predicted to reduce final resurgence by associating L1 extinction with longer and longer periods without a reinforcer. The procedure did reduce the resurgence effect observed when L2 was put on extinction (Experiment 3). However, in each experiment, thinned groups also returned to L1 responding, and continued to make L1 responses, while the reinforcement schedule for L2 was being thinned. Fine-grained analysis of behavior in time suggested that this early resurgence was not due to adventitious reinforcement of L1, occasion setting of L1 by reinforcer presentation, or the entrainment of L1 as a schedule-induced interim behavior. The results are overall consistent with the hypothesis that resurgence is a renewal effect in which extinguished L1 responding recovers when the context provided by the L2 reinforcement schedule is changed. Challenges for this view are also discussed.

It is now widely acknowledged that the extinction of behavior that occurs when a Pavlovian conditioned stimulus is presented repeatedly without a reinforcer does not reflect erasure of the original learning. Several recovery or “relapse” phenomena that occur after Pavlovian extinction suggest that the original learning is saved (e.g., Bouton, 2004; Bouton & Woods, 2008). Although research on extinction and recovery after operant conditioning has been less systematic, there is evidence that similar processes operate there as well (e.g., Bouton, 2011; Bouton & Swartzentruber, 1991; Bouton, Winterbauer, & Todd, 2012). For example, presentation of the reinforcer after a free operant response has been extinguished (as a consequence of letting it occur repeatedly without the reinforcer) can reinstate the operant response (e.g., Baker, Steinwald, & Bouton, 1991; Reid, 1958; Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011), either by setting the occasion for it or by conditioning the background context (e.g., Baker et al., 1991). In addition, a change of context after the extinction of operant behavior can renew responding (e.g., Welker & & McAuley, 1978; Nakajima, Tanaka, Urushihara, & Imada, 2000). Bouton, Todd, Vurbic, and Winterbauer (2011) recently demonstrated operant renewal when conditioning, extinction, and testing occurred in Contexts A, B, and A; A, B, and C; and A, A, and B. These phenomena suggest that extinction performance after operant extinction, like that after Pavlovian extinction, depends at least partly on the animal being in the context in which extinction was learned.

The present article is concerned with resurgence, a different relapse effect that has been studied primarily in operant extinction (e.g., Leitenberg, Rawson, & Bath, 1970; Podlesnik & Shahan, 2009; Winterbauer & Bouton, 2010). In this paradigm, an operant behavior (e.g., pressing one lever) is first reinforced and then extinguished. While the first behavior is undergoing extinction, a second, alternative behavior (e.g., pressing a second lever) is now trained and reinforced. When that alternative behavior is then extinguished, the first behavior returns or “resurges.” The phenomenon once again suggests that extinction is not erasure, and may be relevant to understanding why learning new behaviors that replace unwanted behaviors in clinical settings does not guarantee the prevention of relapse (e.g., Volpp, John, Troxel, Norton, Fassbender, & Loewenstein, 2008).

At least three explanations of resurgence have been proposed. Winterbauer and Bouton (2010; see also Bouton & Swartzentruber, 1991) suggested that resurgence might be another example of the renewal effect. That is, the original behavior is extinguished in the “context” of the new behavior being reinforced. When this context then changes (as it does when the reinforcer is omitted and the second behavior extinguishes), an ABC renewal effect may occur. Alternatively, Leitenberg, Rawson, and their colleagues suggested that performing the new response in extinction might suppress the first behavior and thus prevent the animal from learning about the extinction contingency (e.g., Leitenberg, Rawson, & Bath, 1970; Rawson, Leitenberg, Mulick, & Lefebvre, 1977). When the second behavior is extinguished, there is less competition with the first behavior, which now returns because it has not been fully extinguished. However, Winterbauer and Bouton (2010) found that the strength of resurgence was not related to response rate during response elimination. For example, resurgence was observed in groups that performed the first response at the same level as, or even higher than, extinction controls, indicating that response suppression is not necessary to produce the effect. Finally, in an extension of behavioral momentum theory (e.g., Nevin & Grace, 2000), Shahan and Sweeney (2011; see also Podlesnik & Shahan, 2009, 2010) suggested that resurgence of the first behavior occurs when the disruptive impact of reinforcing the new behavior is removed through extinction of that new behavior.

The present experiments were designed to test a direct implication of the context-change hypothesis. If resurgence depends on the animal's transition to a discriminable new context when the second behavior is extinguished, it should be possible to eliminate the effect by gradually thinning or fading the reinforcement schedule used to maintain the alternative behavior. Such a treatment would allow extinction of the first behavior to be gradually associated with longer and longer intervals between reinforcers that begin to approximate extinction. Consistent with such a possibility, Leitenberg et al. (1975, Experiment 3) found no resurgence in pigeons when the new alternative behavior was reinforced on a very lean VI 4-min schedule throughout the response elimination phase. However, a VI 4-min schedule is so lean that it seems little different from an extinction control condition. Winterbauer and Bouton (2010) found no difference in the level of resurgence when the alternative behavior was reinforced on Random Interval (RI) 10 s vs. 30 s schedules.

The present experiments addressed a related but different question. They asked whether exposure to incrementally leaner schedules during response elimination can reverse the effects of a schedule that would otherwise support resurgence. They used methods developed by Winterbauer and Bouton (2010), which were in turn based on those that Leitenberg et al. (1970) had used in rats: In the first phase, animals were reinforced for pressing one lever (L1) on a Random Interval (RI) 30 s schedule; in the second phase, pressing a second lever (L2) was reinforced while presses on L1 were not; and in the third phase, presses on neither lever were reinforced. When the schedule of reinforcement for L2 was changed from RI 20 s to 120 s or 180 s over several sessions, we found that final resurgence was in fact reduced. However, we also found that the animals began returning to and pressing L1 as soon as the L2 schedule was thinned. In what follows, we document and analyze both of these effects.

Experiment 1

Experiment 1 provided an initial test of the idea that resurgence could be reduced by thinning the reinforcement schedule for L2. Three groups of rats were trained to press L1 on an RI 30 s schedule and then given extinction. The critical treatments occurred during the extinction phase. Group Resurge received a resurgence treatment: L2 was inserted at the start of each extinction session, and presses on it were reinforced on an RI 20 schedule. Group Extinction also had L2 inserted in the chamber at the start of each extinction session, but presses on it were never reinforced. The new group, Group Thinning, began Phase 2 with L2 on the RI 20 schedule that Group Resurge received. However, beginning with the second session, the reinforcement schedule was made leaner midway through each session until a VI 160 s schedule was reached. Then, on the final day, all groups were given a Test session in which both levers were available but presses were never reinforced. Group Extinction was expected to continue to press L1 and L2 at low levels, and acted as a control for spontaneous recovery. Group Resurge was expected to show the resurgence effect, with extinction of L2 producing a return to L1 pressing. However, if thinning reinforcement on L2 resulted in a less salient transition from Phase 2 to the Test session, then Group Thinning should show less resurgence of L1 responding during the test.

Method

Subjects

The subjects were 32 naive female Wistar rats obtained from Charles River, Inc. (St. Constance, Quebec). They were approximately 85–95 days old at the start of the experiment and were individually housed in suspended stainless steel cages in a room maintained on a 16:8-hr light:dark cycle. Each rat was food deprived to 80% of its free-feeding weight and maintained at that level throughout the experiment by a single feeding following each day's second session.

Apparatus

Conditioning proceeded in two rooms, each with a different set of four standard conditioning boxes (Med-Associates model ENV-008-VP, St. Albans, VT) that were modified slightly for use as separate contexts in other experiments (box context was randomly allocated to groups in all of the present experiments). Boxes from both sets measured 31.5 × 25.4 × 24.1 cm (l × w × h), with side walls and ceilings made of clear acrylic plastic and front and rear walls made of brushed aluminum. Recessed 5.1 × 5.1-cm foodcups with infrared photobeams positioned approximately 1.2 cm behind the plane of the wall and 1.2 cm above the bottom of the cup were centered in the front wall about 3 cm above the grid. In one set of four boxes, the floor was composed of stainless steel rods (0.5 cm in diameter) in a horizontal plane spaced 1.6 cm center to center, while in the other set of four boxes, the floor was composed of identical rods spaced 3.2 cm apart in two separate horizontal planes, one 0.6 cm lower than the other and horizontally offset by 1.6 cm. The boxes with the planar floor grid had a side wall with black panels (7.6 × 7.6 cm) placed in a diagonal arrangement, and there were diagonal stripes on both the ceiling and back panel, all oriented in the same direction, 2.9 cm wide, and about 4 cm apart. The other boxes, with the staggered floor, were not adorned in any way. Retractable levers (1.9 cm when extended) were positioned approximately 3.2 cm to the right and to the left of the magazine and 6.4 cm above the grid. Both sets of boxes were housed in sound-attenuating chambers, and were illuminated by two 7.5-W incandescent light bulbs (houselights) mounted above the plexiglass box top on the chamber ceiling. Food reward consisted of 45-mg MLab Rodent Tablets (TestDiet, Richmond, IN).

Procedure

There were two experimental sessions a day, except as noted. (The rats were run twice in four squads composed of eight subjects.) Each day's first sessions began with approximately 10 hr of illuminated colony time remaining, as were sessions on single-session days, while second sessions began with about 7 hr of illuminated time remaining. Animals were placed in illuminated conditioning chambers, and the start of each session was indicated by the insertion of the lever(s) when appropriate. All sessions were 30 min in duration, and the end of the session was indicated by retraction of the lever(s).

Magazine training

On the first day of training, all animals received a single session of magazine training. Neither lever was inserted during this session. Approximately 60 food pellets were delivered at random intervals averaging 30 s. On this schedule, each second was terminated by delivery of a pellet with a uniform 1 in 30 probability.

L1 acquisition (Phase 1)

All animals then received 12 sessions in which they were reinforced for pressing the left lever (L1). In all sessions, presses on that lever delivered pellets on a Random Interval 30-sec (RI 30) schedule of reinforcement. On the RI 30 schedule, pellet availability in each second was randomly determined with a uniform 1 in 30 probability. Once the program indicated one (or more) pellet(s) was available, the animal's next L1 press would deliver one pellet and restart the schedule mechanism.

L1 extinction and L2 conditioning (Phase 2)

The rats were then randomly assigned to Groups Extinction (n = 10), Resurge (n = 11), and Thinning (n = 11) with the dual constraints that the groups did not differ reliably on response rate achieved and were not reliably unbalanced across daily runs (each of eight animals). All rats were then given four (twice-daily) sessions of extinction. In all rats, both the left (L1) and right (L2) levers were inserted throughout these sessions. In Group Extinction, presses on both levers had no programmed consequences, and pellets were never delivered. In Group Resurge, presses on L1 were not reinforced, but presses on the right lever (L2) earned pellets on an RI 20 schedule of reinforcement. In Group Thinning, presses on L1 were never reinforced, while presses on L2 were reinforced. In the first session, the schedule of reinforcement for L2 was RI 20, but in each of the subsequent sessions the schedule of reinforcement was doubled 15 min (i.e., halfway) into the session. The progression of schedule of reinforcement on L2 was therefore RI 20 in the first session, RI 20 to RI 40 in the second, RI 40 to RI 80 in the third, and RI 80 to RI 160 in the fourth and final session of Phase 2.

Resurgence test (Phase 3)

All animals were finally given a 1-hr test session with both levers inserted. In all groups, presses on both levers were not reinforced, and pellets were never delivered.

Statistical analysis

For this and following experiments, analysis of variance (ANOVA) with a rejection criterion of p < .05 was employed in all inferential tests unless otherwise specified.

Results

Acquisition proceeded normally, with all three groups showing a reliable increase in rate of L1 responding over sessions, F(11, 319) = 93.90, MSE = 216.80. Random assignment to groups was successful in that there was neither a Group effect nor a Group × Session interaction (Fs < 1). The rate of lever pressing in the last session averaged 43.3, 43.9, and 43.7 responses per min in Groups Extinction, Resurge, and Thinning, respectively.

Figure 1 presents data from Phase 2 and the test session. The results are shown as a function of 15-min (half-session) bins. Overall, the results indicated that L1 responding declined in the Resurgence group during Phase 2 (top left) at the same time L2 responding was acquired (bottom left). As expected, Group Resurge's L1 responding returned (resurged) when L2 was put on extinction (top right). However, the results in Group Thinning were dominated by the fact that Phase 2 responding on L1 first decreased and then increased as the L2 reinforcement schedule was thinned. In what follows, we report data analyses that focus on L1 and then L2 responding in Phase 2, and then response changes that occurred during resurgence testing.

Figure 1.

Results of Experiment 1. The upper panels describe L1 responding during 15-min (half-session) bins of Phase 2 extinction (left) and testing (right). The lower panels describe L2 responding during 15-min (half-session) bins of Phase 2 extinction (left) and testing (right).

First consider L1 responding in Phase 2 (top left). As suggested by the figure, L1 pressing in all three groups changed reliably over the eight 15-min Bins, F(7, 203) = 29.80, MSE = 56.33. However, a reliable Group × Bin interaction, F(14, 203) = 6.82, MSE = 56.33, indicated that responding changed differentially in the three groups. The Group main effect was also reliable, F(2, 29) = 5.41, MSE = 341.69. Pairwise comparison of the groups collapsing over bins showed that Group Resurge pressed less than either Group Thinning, F(1, 20) = 11.85, MSE = 269.40, or Group Extinction, F(1, 19) = 7.89, MSE = 28.41. Groups Thinning and Extinction, in contrast, did not differ, F < 1. Group Thinning showed significantly more responding on L1 than Group Resurge—the thinning procedure caused the rats to increase their L1 responding.

L2 responding (bottom left of Figure 1) also changed reliably over Bin, F(7, 203) = 7.84, MSE = 79.63, and a Group × Bin interaction indicated that it did so differently in the three groups, F(14, 203) = 11.15, MSE = 79.63.The groups again differed in their total amount of L2 responding during Phase 2, F(2, 29) = 49.31, MSE = 1780.00. All three pairwise Group contrasts showed reliable differences in the amount of L2 responding, Fs(1, ≥19) ≥ 11.92, MSE ≤ 2580.80. Group Thinning's leaner reinforcement schedules thus produced lower responding levels than the RI 20 schedule given Group Resurge.

During the test session (upper and lower right of Figure 1), L2 decreased and L1 increased in some of the groups. Analysis of L2 responding first revealed a significant decline in responding relative the last block of extinction, F(1, 29) = 48.86, MSE = 122.50. Both the Group effect, F(2, 29) = 19.11, MSE = 122.50, and the Group × Bin interaction, F(2, 29) = 136.55, MSE = 139.30, were reliable. Groups Resurge, F(1, 10) = 28.28, MSE = 326.80, and Thinning, F(1, 10) = 50.20, MSE = 28.30, reduced responding, while Group Extinction showed a slight but reliable increase in responding, F(1, 9) = 27.02, MSE = 0.14, from a rate of 0.11 Responses / min in the last bin of extinction to 0.62 Response / min in the first bin of resurgence testing. The rates were so low, however, that the increase was not considered consequential.

A final analysis focused on resurgence of L1 responding (upper right). Consistent with our earlier findings, Group Resurge increased its responding on L1 when L2 underwent extinction. A Group × Bin analysis comparing the final 15-min bin of extinction and the first bin of the test showed a reliable overall increase in pressing, F(1, 29) = 11.00, MSE = 20.89, and an overall effect of Group, F(2, 29) = 7.99, MSE = 62.77. Most importantly, however, the Group × Bin interaction was also reliable, F(2, 29) = 15.67, MSE = 20.89. Subsequent analysis of the change in responding between bins in each group revealed that while Group Resurge increased L1 responding from extinction to test, F(1, 10) = 35.20, MSE = 21.21, Group Thinning did not change, F(1, 10) = 2.24, MSE = 34.88. Group Extinction, however, also showed an increase from extinction to test, F(1, 9) = 12.00, MSE = 4.97, suggesting some spontaneous recovery of responding. However, an ANOVA including just Groups Resurge and Extinction revealed a Group × Bin interaction, F(1, 19) = 13.03, MSE = 13.52. Thus, the resurgence treatment produced a greater increase in responding than mere spontaneous recovery.

Discussion

The results of this experiment replicated the standard resurgence effect: Group Resurge increased its L1 responding from the end of Phase 2 to the test more than did Group Extinction. Consistent with our hypothesis, there was no comparable increase in responding in Group Thinning. However, interpretation of this effect is complicated by the fact that Group Thinning was also responding at a higher level at the end of Phase 2. Although the thinning procedure may have provided some protection against resurgence, the animals may have been at a functional response ceiling and hence unable to show a further increase. On the other hand, the most striking result of the experiment was that the thinning procedure caused a high level of L1 responding during Phase 2. One possibility is that Group Thinning's lower level of responding on L2 did not compete as much with L1 responding. However, a reduced level of response competition does not explain why L1 responding, which was on extinction throughout the phase, was so sustained over the sessions of Phase 2. Another possibility is that the doubling of the inter-reinforcer intervals that the Thinning group experienced in the middle of each session caused repeated exposure to noticeable contextual change. Other possibilities will be addressed in Experiments 2 and 3. For now, we note that the high level of L1 responding in Group Thinning at the end of response elimination makes it difficult to claim that thinning eliminated the final resurgence. Instead, the thinning procedure generated an early resurgence effect.

Experiment 2

Experiment 2 was designed to explore the effects of thinning L2 reinforcement further. Although many aspects of Experiment 1 might have led to the early resurgence in L1 responding during Phase 2, a likely candidate was the rapid change (a doubling) in the L2 reinforcement schedule in the middle of each session. In fact, the rats’ response to that change is similar to the typical response when the transition is to extinction; functionally, the rapid decrease in the L2 schedule may have been very much like a change to extinction. Experiment 2 was therefore designed to test the effects of two alternative thinning treatments that made the thinning more gradual.

Three groups first acquired L1 and then received extinction. At this time, L2 was also introduced and presses on it were reinforced. As before, Group Resurge earned pellets on a constant RI 20 schedule. Two thinned groups earned pellets on the same RI 20 schedule during the first session, but then received a thinning treatment that increased the active random interval by an average of only 20 s over each of the five sessions that followed. The groups differed in the abruptness of the transitions. In Group Stepped, the schedule was incremented by 20 s at the midpoint of each session, analogous to Group Thinning's treatment in Experiment 1. In Group Gradual, the interval was increased by a small and constant amount during each second within the session, so that a total of 20 s was gradually added to the average interval over the session as a whole.

Method

Subjects and Apparatus

The subjects were again 32 female Wistar rats from the same supplier, and with similar characteristics, as those used in Experiment 1. They were housed and maintained as described earlier, using the same light cycle and food deprivation procedure. Conditioning occurred in the same chambers used in Experiment 1.

Procedure

Twice-daily 30-min sessions were again employed throughout the experiment, except as noted, at the same point in the daily light-dark cycle. The rats were placed in illuminated conditioning chambers, and the session boundaries were again indicated by insertion and retraction of the lever(s) as appropriate. All animals received initial magazine training as in Experiment 1. They then received 12 sessions of instrumental conditioning initiated by insertion of L1. In half of the animals, L1 was the left lever, while in the other half it was the right lever. In all sessions, presses on L1 again delivered pellets on an RI 30 schedule of reinforcement. For Phase 2, the rats were randomly assigned to Group Resurge (n = 10), Group Stepped (n = 11), and Group Gradual (n = 11) with two constraints: groups did not reliably differ on level of performance achieved and groups were not reliably unbalanced across daily runs. All rats were given six sessions of Phase 2 conditioning with both levers inserted. If the left lever was L1 in Phase 1, the right lever was L2 in Phase 2, and vice versa. In Group Resurge, presses on L1 were not reinforced, but presses on the L2 earned pellets on an RI 20 schedule. In Group Stepped, presses on L1 were never reinforced, while presses on L2 were reinforced. In the first session, the L2 schedule of reinforcement was RI 20, but in each of the subsequent sessions, 20 s was added to the schedule's interval 15 min into the session. In Group Gradual, presses on L1 were never rewarded, while presses on L2 were reinforced. In the first session, the L2 schedule of reinforcement was RI 20, but in each of the subsequent sessions the schedule of reinforcement was incremented by 0.011... s each second, resulting in a 20-s increase in the schedule of reinforcement over the 30 mins of the session. The progression of schedule of reinforcement on L2 in both Thinning groups was therefore RI 20 throughout the first session, RI 20 to RI 40 in the second, RI 40 to RI 60 in the third, RI 60 to RI 80 in the fourth, RI 80 to RI 100 in the fifth, and RI 100 to RI 120 in the sixth and final session of Phase 2.

All animals were given a final 1-hr test session with both levers inserted. Presses on both levers were not reinforced, and pellets were never delivered.

Results

Responding over sessions

Acquisition proceeded uneventfully, with the rats showing a reliable increase in rate of L1 responding over sessions, F(11, 319) = 94.07, MSE = 129.60. Group assignment was also successful, showing neither a reliable effect of Group or a Group × Bin interaction (Fs < 1). The terminal rate of lever pressing averaged 32.3, 36.5, and 33.9 responses per min in Groups Gradual, Stepped, and Resurge, respectively, in the last session.

Figure 2 presents data from Phase 2 and the test session. As before, the results are shown in 15-min (half-session) bins. The Resurgence condition once again yielded resurgence in the final test. However, although there was evidence that the thinning procedure reduced the final amount of resurgence, both the Gradual and Stepped thinning conditions again allowed an early resurgence effect to occur.

Figure 2.

Results of Experiment 2. The upper panels describe L1 responding during 15-min (half-session) bins of Phase 2 extinction (left) and testing (right). The lower panels describe L2 responding during 15-min (half-session) bins of Phase 2 extinction (left) and testing (right).

The data analysis again focused on L1 and L2 responding in Phase 2 before turning to the results of resurgence testing. During extinction, L1 pressing in all groups reliably changed, F(11, 319) = 22.79, MSE = 21.68, but a Group × Bin interaction indicated that it did so differently in the three groups, F(22, 319) = 2.68, MSE = 21.68. The Group main effect was not reliable, F(2, 29) = 2.24, MSE = 329.43. Examination of the first two bins, where all groups were treated equivalently, did not reveal a group difference or a Group × Bin interaction, Fs < 1. The Bin effect, however, was significant, F(1, 29) = 64.80, MSE = 22.70. A different pattern emerged during differential treatment (Bins 3 – 12). Pairwise comparisons (with data collapsed over these bins) showed that although Groups Gradual and Stepped did not differ, F < 1, Groups Gradual, F(1, 19) = 8.77, MSE = 231.50, and Stepped, F(1,19) = 4.84, MSE = 276.80, both responded more than Group Resurge. The results thus replicated the early increase in L1 responding that was observed in Experiment 1 during the response elimination phase.

L2 pressing (bottom left of Figure 2) also changed over Bin, F(11, 319) = 10.34, MSE = 116.21, while the Group × Bin interaction showed that it did so differently in the three groups, F(22, 319) = 5.05, MSE = 116.21. The Group effect fell short of reliability, F(2, 29) = 2.75, MSE = 3794.50. Groups again failed to differ during the first two bins of equivalent treatment, F < 1, although they increased L2 responding over Bins, F(1, 29) = 33.09, MSE = 116.0, with no Group × Bin interaction, F < 1. During the thinning treatment (bins 3 – 12), Groups Gradual and Stepped pressed a comparable amount, F < 1, and both pressed less than Group Resurge. For Group Gradual, the difference was significant, F(1, 19) = 5.89, MSE = 3823.0; for Group Stepped, F(1, 19) = 3.23, MSE = 4479.0, p = .089, the difference fell short of reliability. The median reinforcement rates (per min) during Phase 2 were 2.73, 0.97, and 1.07 for Groups Resurge, Gradual, and Stepped. There was no difference in the rate of reinforcement between the Gradual and Stepped groups, F(1, 20) = 1.42, MSE = 0.12.

When L2 was extinguished in the Test session (bottom right of Figure 2), all three groups decreased L2 pressing compared with the final session of Phase 2, F(1,29) = 46.97, MSE = 158.80. The amount of decline was not different in all three groups as indicated by a nonsignificant Group × Bin interaction, F(2, 29) = 2.76, MSE = 158.80. The overall amount of L2 pressing, however, differed among groups, F(2, 29) = 8.13, MSE = 579.30. While the two thinning treatments led to similar rates of pressing, F < 1, they both led to reliably less pressing than in Group Resurge, Fs(1, 19) ≥ 7.38, MSE ≤ 524.00.

Test results with L1 (upper right) suggested differences in the amount of final resurgence. An ANOVA comparing the final bin of extinction and the first bin of testing showed a reliable overall increase in pressing over Bins, F(1, 29) = 8.08, MSE = 22.26, and an overall effect of Group, F(2, 29) = 3.72, MSE = 56.86. The Group × Bin interaction was not reliable, F(2, 29) = 2.08, MSE = 22.26. However, planned comparisons of each group's change in responding revealed a reliable increase in Group Resurge, F(1,9) = 20.37, MSE = 10.19, but none in Group Stepped, F < 1, or Group Gradual, F(1,10) = 1.95, MSE = 32.64.

Analysis of responding in time

Given Experiment 2's replication of the early resurgence of L1 responding during the thinning treatments, we undertook a finer-grained analysis of behavior occurring in real time. Such an analysis was possible because the computer time-stamped all events that were recorded in each experimental session. The analyses explored two possible explanations of the early resurgence observed in the thinned groups. First, one way to think of the effect of a thinning schedule is that it produces longer and longer periods of L1 extinction, when no reinforcers are presented for L2 responding, which are eventually followed by presentation of a pellet (earned for pressing L2). Presentation of a pellet after a period of extinction might “reinstate” L1 responding if, for example, pellet consumption had become part of a behavior chain (reinforced during Phase 1) that led to a return to the lever and more L1 responding. That is, during Phase 1 pellet presentation might have come to set the occasion for a return to an L1 response. To explore this possibility, we examined L1 responding as a function of time before and after each reinforcer delivered during the fifth session of Phase 2 (half-session bins 9 and 10), the session in which the difference among the group means was largest. The right hand part of Figure 3 shows responding 5 s before and 5 s after each pellet (excluding rare occasions when a second pellet was delivered within 5 s of the first). The vertical dashed line indicates the timepoint when a pellet was presented. The rate of L1 responding that occurred in each second after the pellet (positive numbers) and before the pellet (negative numbers) is also shown. The pattern lends little support to the idea that L1's recovery in the thinning groups was related to occasion setting by the pellet. Responding returned only gradually after pellet presentation, and although the thinning groups seemed to recover to a higher level than Group Resurge, the level reflected their generally higher level of L1 responding throughout the phase, which persisted and was still apparent during the seconds before the next pellet. These descriptions are consistent with statistical analysis. The data were put through a Group × Bin ANOVA that included the 5 1-s bins before and after each pellet presentation. The rate of L1 pressing in these bins, although showing a reliable dependence on Bin, F(9, 261) = 21.37, MSE = 0.0055, did not depend upon either Group, F(2, 29) = 1.58, MSE = 0.039, or on the Group × Bin interaction, F < 1. There was no support for the idea that occasion-setting of L1 responding by the pellet can explain the higher L1 responding observed in the thinning groups.

Figure 3.

Fine-grained analysis of events that occurred in the fifth session of Phase 2 in Experiment 2. Left: Pellet rate for each of the 5 s before (−5 to −1) and 5 s after (1 to 5) each L1 response. Right: L1 response rate during the 5 s before (−5 to −1) and 5 s after (1 to 5) each pellet earned for pressing L2.

A second possibility is that L1 responses in the thinned groups were somehow reinforced by chance pairings with a food pellet. The left-hand panel of Figure 3 suggests that this possibility is also unlikely. That panel examines the rate at which pellets were delivered as a function of their proximity to every L1 press in Session 5 (bins 9 and 10; we excluded occasions when another L1 response occurred within 5 s). The vertical dashed line now indicates the point in time at which an L1 response was emitted, with the rate of pellets occuring in each second after the response (positive numbers at right) and before the response (negative numbers at left) also being shown. The figure indicates that, if anything, pellets were less likely to occur soon after an L1 response in the thinned groups than in Group Resurge. This description was consistent with statistical analysis. A Group × Bin ANOVA revealed overall effects of Group, F(2, 29) = 20.34, MSE = 0.0012, Bin, F(9, 261) = 26.72, MSE = 0.00062, and the Group × Bin interaction, F(18, 261) = 7.00, MSE = 0.00062. The interaction was analyzed by comparing group means before and after the L1 press. Prior to the press, there was no effect of Group, F < 1. After the press, there was an effect of Group, F(2, 29) = 39.24, MSE = 0.0011. Group Resurge received more pellets following L1 presses than Group Stepped, F(1, 19) = 43.42, MSE = 0.0014, or Group Gradual, F(1, 19) = 45.27, MSE = 0.0016. The latter two groups did not differ, F(1, 20) = 1.18, MSE = 0.00040. The results therefore run counter to the idea that accidental reinforcement of L1 supports responding in the Thinning groups. The higher probability of a reinforcer following L1 in Group Resurge is presumably related to both its higher rate of reinforcement and the overall lower rate of L1 responding in this group. The main point, however, is that a finer-grained analysis of behavior failed to support the idea that L1 responding was somehow reinforced more often in the thinned groups than in Group Resurge.

Discussion

The results of Experiment 2 replicated Experiment 1's early resurgence effect. Despite the effort to decrease the abruptness of the change in reinforcement rate introduced by the thinning schedule, L1 responding recovered during the two new thinning treatments. Thus, the phenomenon appears to be replicable and robust. Although there was again some evidence that the thinning treatment reduced the final amount of resurgence caused by the transition of L2 to extinction (L1 responding did not resurge significantly in either thinned group when L2 was finally extinguished), once again the most striking finding was that thinning caused a recovery of responding soon after the interval of the RI schedule began to be increased. Both the stepped procedure, which increased the nominal inter-reinforcer interval by 20 s at the midpoint of each session, and the gradual procedure, which steadily incremented the interval throughout the session, produced the same result.

It is of course possible that the rat did perceive the changing reinforcement schedules, and that this created a context effect that would support a renewal of L1 responding. An analysis of responding in real time explored two other possibilities: occasion setting by pellet presentation and adventitious reinforcement of L1. There was little support for either mechanism; L1 responses were neither preceded nor followed by pellets in a way that made either of these mechanisms plausible.

Experiment 3

Experiment 3 was designed to examine another possible explanation of the early resurgence of L1 behavior in the groups that receive thinning schedules. While the high level of L1 responding in Phase 2 does not seem to be due to the pellets either reinstating or accidentally reinforcing L1 behavior, it seemed possible that L1 responding might increase due to another behavioral phenomenon that can develop when animals receive periodic reinforcers: so-called schedule-induced or “interim” behavior (e.g., Staddon, 1977; Staddon & Simmelhag, 1971). On this view, the rat might begin to fill the gaps between reinforcers with L1 responses, emitting them when the probability of the next reinforcer is low, i.e., the way schedule-induced behaviors are typically emitted in time (e.g., Staddon, 1977; Staddon & Simmelhag, 1971; Timberlake, 2001; Timberlake & Lucas, 1991). Such behaviors (e.g., schedule-induced polydipsia or wheel-running) can become excessive, and, although they are often natural behaviors (e.g., Timberlake, 2001), learned behaviors with a history of reinforcement can become targets for schedule induction (e.g., Cantor & Wilson, 1985; Staddon & Simmelhag, 1971). Since the temporal structure of reward in the RI schedule is unpredictable, the amount of interim behavior in the thinned groups in the previous experiments must have been limited. Experiment 3 was therefore designed to accentuate any possible schedule-induced effect by using Fixed Interval (FI) schedules on L2. If L1 develops as an interim behavior, a fixed schedule of reinforcement on L2 should promote not only higher levels of L1 pressing, but also a clear temporal pattern of behavior. To further capture the temporal dynamics of the behavior, we continued Phase 2 for a substantial number of sessions (eight) beyond the point at which the terminal reinforcement schedule on L2 (FI 120) was achieved.

The design of Experiment 3 was a 2 × 2 factorial. Half the rats received an unchanging interval schedule of reinforcement on L2 during Phase 2, and hence constituted a standard Resurgence condition. The other half received a gradually thinning schedule like that given Group Gradual in Experiment 2. An orthogonal half of the animals were rewarded for L2 pressing on an RI schedule of reinforcement, as before, while the other half were rewarded on an FI schedule. If early resurgence is due to L1 becoming an interim activity, we might find more of it with the FI schedules, and L1 responding should be especially probable during early parts of the inter-reinforcer interval.

Method

Subjects and Apparatus

The subjects were again 32 naive female Wistar rats from the source and with similar characteristics as those used in Experiment 1. They were housed and maintained as described earlier, using the same light cycle and food deprivation procedure. Conditioning occurred in the same chambers as in the previous experiments.

Procedure

Twice-daily 30-min sessions were again employed throughout the experiment, except as noted, at the same point in the daily light-dark cycle. Animals were placed in illuminated conditioning chambers, and session onset and offset were again indicated by insertion and retraction of the lever(s). All rats received initial magazine training as described above. They were then given 12 sessions of instrumental conditioning initiated by insertion of L1 as Phase 1. For half the rats, L1 was the left lever, while for the other half L1 was the right lever. In all sessions, presses on L1 again delivered pellets on an RI 30 schedule of reinforcement.

There was a total of 14 sessions in Phase 2. At the start, the rats were allocated into four groups (n = 8) representing all combinations of two factors in a balanced fashion, while maintaining a criterion of comparable acquisition performance. Throughout the phase, half the animals received training with FI schedules, and half received training on Random Interval (RI) schedules. An orthogonal half of the animals received a Resurgence treatment and half received a Thinning treatment. Again, if left lever was L1 in Phase 1, the right lever was L2 in Phase 2, and vice versa. In Phase 2, both levers were inserted in each session. All groups received extinction (no reinforcement) for pressing L1 throughout the phase. In Group Random Resurgence, presses on L2 earned pellets on an RI 20 schedule, and in Group Fixed Resurgence, presses on the L2 earned pellets on an FI 20-s schedule. In Group Random Thinning, the reinforcement schedule on L2 was changed over sessions in the following way. In the first session, the L2 schedule was RI 20, but in each of the next five sessions the schedule was incremented by 0.011... s each second, resulting in a 20-s increase in the schedule over the duration of each session. In the final eight sessions the schedule was held at RI 120. In Group Fixed Thinning, L2 was reinforced with schedule values matched to those in Group Random Thinning, but the random component was removed. Group Fixed Thinning therefore experienced a progressively increasing FI schedule of reinforcement of L2, followed by continued training with L2 under FI 120.

All animals were finally given a 1-hr test session with both levers inserted. Presses on both levers were not reinforced, and pellets were never delivered.

Results

Responding over sessions

Acquisition proceeded uneventfully, with all groups showing a reliable increase in rate of L1 responding over sessions, F(11, 308) = 60.90, MSE = 135.10. Group assignment was again successful in that there was no effect of Schedule, Thinning, or any of the interactions with Bin (Fs < 1) (these were dummy variables at this point). The terminal rate of lever pressing / min averaged 27.9, 29.0, 31.7, and 27.9 in Groups Fixed Resurge, Fixed Thinning, Random Resurge, and Random Thinning, respectively.

Figure 4 presents data from Phase 2 and the test session. Early resurgence of L1 was again observed, apparently equivalently, in the thinned groups, although it disappeared by the end of Phase 2 (while the reinforcement schedule was held at a constant RI 120). Importantly, the thinned groups both showed less final resurgence than the other groups during final testing.

Figure 4.

Results of Experiment 3. The upper panels describe L1 responding during 15-min (half-session) bins of Phase 2 extinction (left) and testing (right). The lower panels describe L2 responding during 15-min (half-session) bins of Phase 2 extinction (left) and testing (right). FI = Fixed Interval; RI = Random Interval.

As usual, we first report analyses focusing on L1 and L2 responding in Phase 2 before turning to changes in responding during resurgence testing. During extinction, L1 responding (upper left panel) changed reliably over Bins, F(27, 756) = 21.43, MSE = 15.24. Change depended upon the thinning treatment, F(27, 756) = 2.99, MSE = 15.24, but did not depend upon either the nature of the schedule (RI or FI), F(27, 756) = 1.09, MSE = 15.24, or the combination of thinning treatment and schedule, F < 1. There was a marginal overall effect of Thinning treatment, F(1, 28) = 3.56, MSE = 356.80, p = .07, but neither an effect of Schedule, F < 1, nor a Schedule × Treatment interaction, F < 1. In the first two bins (which occurred before thinning began), L1 responding declined in all groups, F(1, 28) = 99.57, MSE = 10.99, the decline did not interact with any other factor, Fs(1, 28) ≤ 1.85, MSE = 10.99, and there were no overall differences among the groups, Fs(1, 28) < 1; this was true despite the difference in L2 schedule type employed during these sessions. During the thinning procedure proper (bins 3 - 12), animals given either Thinning Treatment pressed L1 more than animals given the Resurgence Treatments, F(1, 28) = 6.57, MSE = 117.79, but neither the Schedule effect, F < 1, nor the Treatment × Schedule interaction, F < 1, was reliable. Finally, during bins 13 – 28, when the reinforcement schedule was held constant at either RI or FI 120 in the thinning groups, there was no overall effect of Schedule, F(1, 28) = 2.78, MSE = 251.43, and no Treatment or Treatment × Schedule interaction effects, Fs < 1. There was a reliable effect of Bin, F(15, 420) = 3.30, MSE = 9.28, but no Bin × Schedule, F(15, 420) = 1.17, MSE = 9.28, or Bin × Schedule × Treatment, F(15, 420) = 1.12, MSE = 9.28, effects. There was a reliable Bin × Treatment interaction, F(15, 420) = 3.55, MSE = 9.28, however, suggesting that the response rates of the groups tended to converge over time.

L2 showed a very different pattern of responding over groups during Phase 2 (bottom left of Figure 4). While there was no overall effect of Schedule, F(1, 28) = 1.41, MSE = 9653.00, there was both a Treatment effect, F(1, 28) = 15.68, MSE = 9653.00, and a Schedule × Treatment interaction, F(1, 28) = 4.26, MSE = 9653.00. Responding also depended upon Bin, with an overall Bin effect, F(26, 728) = 4.38, MSE = 130.98, as well as Schedule × Bin, F(26, 728) = 3.13, MSE = 130.98, Treatment × Bin, F(26, 728) = 6.42, MSE = 130.98, and Schedule × Treatment × Bin interactions, F(26, 728) = 4.69, MSE = 130.98. Performance among the groups was comparable over the first two bins (despite some animals getting Fixed and some getting Random schedules of reinforcement), as seen in no effects of Treatment, F(1,28) = 1.36, or a Schedule effect or Schedule × Treatment interaction, Fs < 1. Although animals increased responding over these Bins, F(1, 28) = 23.53, MSE = 48.07, there were no Schedule × Bin, F(1,28) = 1.32 < 1, Treatment × Bin, F < 1, or Schedule × Treatment × Bin interactions, F < 1. During bins 3 - 12, when some of the rats received thinning treatments, the thinned groups responded less than the groups getting the constant RI 20 schedule, F(1, 28) = 8.98, MSE = 2734.60, but neither the Schedule nor Schedule × Treatment interaction were reliable, Fs < 1. Finally, during the bins with L2 reinforcement held constant (13 – 28), there was no reliable effect of Schedule, F(1, 28) = 5.5, MSE = 8043.00, but there was a reliable effect of Treatment, F(1, 28) = 17.66, MSE = 8043.00, and most importantly a Schedule × Treatment interaction, F(1, 28) = 6.49, MSE = 8043.00. Pairwise comparison of the groups’ overall rate of L2 presssing showed that while animals in both Thinning conditions responded comparably, F(1, 14) = 1.02, MSE = 3196.20, animals in the Resurgence condition pressed more on the Random than on the Fixed schedule, F(1, 14) = 5.50, MSE = 12889.00. Although animals in the Fixed Resurgence group pressed reliably more than animals in the Random Thinning group, F(1, 14) = 5.40, MSE = 4854.7, groups Fixed Resurgence and Fixed Thinning did not differ, F(1, 14) = 1.67, MSE = 6584.70. The median reinforcement rates (reinforcers per min) throughout Phase 2 were 0.47, 0.57, 2.83, and 2.80 for groups Fixed Thinning, Random Thinning, Fixed Resurge, and Random Resurge, respectively.

When L2 was extinguished in the test session (bottom right panel of Figure 4), all four groups decreased L2 pressing relative to their performance in the final bin of Phase 2, F(1,28) = 38.82, MSE = 148.30. The decline did not depend on the schedule, the thinning treatment, or their combination, Fs(1,28) ≤ 3.57, MSE = 148.30. The total amount of L2 pressing depended upon the thinning manipulation, with less pressing in Thinning than in Resurgence animals, F(1, 28) = 8.54, MSE = 1135.20. No other overall effects were reliable, Fs(1, 28) ≤ 3.80, MSE = 11335.20.

Test results for L1 are shown in the top right panel of Figure 4. Analysis comparing the final bin of extinction with the first bin of test showed a reliable overall increase in pressing over Bins, F(1, 28) = 23.03, MSE = 18.89. Neither the Schedule nor Thinning Treatment had an overall impact on responding in these sessions, Fs(1, 28) ≤ 2.84, MSE = 46.44. Most important, however, the change from extinction to the test session depended on the thinning treatment, as reflected in a reliable Thinning Treatment × Bin interaction, F(1, 28) = 9.35, MSE = 18.89. No other interactions with Bin were reliable, Fs < 1. Although the increase between extinction and testing was reliable in animals given the Thinning treatments, F(1, 14) = 4.75, MSE = 6.02, the interaction indicates that the amount of change in the Resurgence treatment animals, F(1, 14) = 18.36, MSE = 31.77, was reliably greater. Although some of the increased responding in the Thinning groups might have been due to spontaneous recovery (see Experiment 1), the magnitude of any resurgence was clearly reduced by the thinning reinforcement schedules.

Analysis of responding in time

To further analyze the early resurgence of L1 responding observed during Phase 2, data collected in real time were again examined. The results once again indicated little role for occasion setting by the reinforcer, or accidental reinforcement of L1. They also produced no evidence that L1 had the temporal characteristics of an interim behavior during early resurgence. Figure 5 summarizes pellet rate (at left) as a function of time before and after an L1 response and L1 response rate (at right) as a function of time before and after a pellet presentation during Session 5 (half-sessions 9 and 10), when early resurgence was numerically strongest. As in Experiment 2, there was little evidence that the thinning groups received a reinforcer especially soon after an L1 response or made an L1 response especially soon after a pellet was presented. The early resurgence effect is not explained by the pellet either reinstating or reinforcing L1 responding.

Figure 5.

Fine-grained analysis of events that occurred in the fifth session of Phase 2 in Experiment 3. Left: Pellet rate for each of the 5 s before (−5 to −1) and 5 s after (1 to 5) each L1 response. Right: L1 response rate during the 5 s before (−5 to −1) and 5 s after (1 to 5) each pellet earned for pressing L2. Fixed = Fixed Interval; Random = Random Interval.

These conclusions were consistent with the results of statistical analyses. Concerning the rate of L1 pressing around a pellet delivery (Figure 5, right), a Treatment × Schedule × Bin ANOVA revealed an overall effect of Treatment, F(1, 28) = 13.64, MSE = 0.02, but not of Schedule, or a Treatment × Schedule interaction, Fs < 1. There were effects of Bin, F(9, 252) = 11.15, MSE = 0.0054, and Treatment × Bin, F(9, 252) = 6.23, MSE = 0.0054, Schedule × Bin interactions, F(9, 252) = 3.70, MSE = 0.0054, but no effect of the Treatment × Schedule × Bin interaction, F(9, 252) = 1.35, MSE = 0.0054. Separate analysis of the Schedule × Bin and Treatment × Bin interactions before and after pellet delivery showed that while there was no effect of Schedule before a pellet, F < 1, there was one following a pellet, F(1, 30) = 14.68, MSE = 0.0056, with the Fixed groups responding more than the Random groups. And while the Thinning animals pressed more than Resurge animals before a pellet, F(1, 30) = 10.94, MSE = 0.038, there was no effect of Treatment following a pellet, F(1, 30) = 2.15, MSE = 0.0078. Although the analyses reveal different levels of responding around pellet deliveries, they do not support the idea that L1 presses were being reinstated by pellet deliveries in the Thinning groups.

Concerning the rate of reinforcement around L1 responding (Figure 5, left), a Treatment × Schedule × Bin ANOVA revealed overall effects of Treatment, F(1, 28) = 34.95, MSE = 0.0011, but not Schedule, or a Treatment × Schedule interaction, Fs(1, 28) < 2.08, MSE = 0.0011. There were also effects of Bin, F(9, 252) = 9.91, MSE = 0.00064, and Treatment × Bin, F(9, 252) = 2.52, MSE = 0.00064, Schedule × Bin, F(9, 252) = 7.51, MSE = 0.00064, and Treatment × Schedule × Bin interactions, F(9, 252) = 2.80, MSE = 0.00064. The interactions were analyzed by comparing group means before and after the L1 response. Prior to the response, animals given Fixed schedules received more pellets than animals given Random schedules, F(1, 28) = 11.45, MSE = 0.001, and animals given the Thinning procedure received fewer pellets than animals given the Resurgence procedure, F(1, 28) = 8.36, MSE = 0.001. The two effects did not interact, F < 1. After the press, animals given Fixed schedules received fewer pellets than animals given Random schedules, F(1, 28) = 9.42, MSE = 0.0014, and rats given the Thinning procedure received fewer pellets than rats given the Resurgence procedure, F(1, 28) = 23.11, MSE = 0.0014. Treatment also interacted with Schedule, F(1, 28) = 6.77, MSE = 0.0014. Analysis comparing each of the groups with one another confirmed that Group Random Resurge received more pellets than the other groups, smallest F(1,14) = 8.31, MSE = 0.0026, and that the other groups did not differ from one another, largest F(1, 14) = 3.84, MSE = 0.0012, p = .07, for Groups Fixed Resurge vs. Group Random Thinning. Once again, there was no evidence that accidental reinforcement of L1 responding caused the Thinning groups to make more L1 responses than the Resurgence groups (Figure 4).

To examine the temporal relation between of L1 responding and the overall L2 reinforcement schedule, Figure 6 plots the mean rate of both L2 and L1 responding during the inter-reinforcer interval during Session 7 (half-sessions 13 and 14). We isolated Session 7 because it was the first post-thinning session in which the reinforcement schedules were held constant; a similar analysis of half-sessions 9 and 10, where the thinning groups’ intervals were changing from moment to moment but the early resurgence effect appeared numerically stronger, supported the same conclusions. Note that, since the Thinning groups reached a terminal interval of 120 s during training, and the Resurge groups were maintained at an interval of 20 s, the figure shows response rates over 120 s (20 6-s bins) and 20 s (20 1-s bins) in these respective groups. As expected, L2 responding on the Fixed schedules (bottom panels) tracked the increasing probability of reinforcement over time—L2 increased from a low rate at the start of the interval between reinforcers to a higher rate at the end of the interval. In contrast, although interim behaviors typically occur early in the interval between reinforcers, L1 responding was not distributed systematically. An ANOVA on L1 responding over bins in the two Thinning groups revealed a significant Bin effect, F(19, 266) = 2.92, MSE =0.015, but a nonsignificant Group × Bin interaction, F(19, 266) = 1.56, MSE = 0.015. Notice, however, that there was no indication of predominantly early L1 responses in Group Fixed Thinning. An ANOVA in the two Resurge groups revealed significant effects of both Bin, F(19, 266) = 3.29, MSE = 0.0012, and a Group × Bin interaction, F(19, 266) = 3.71, MSE = 0.0012. The interaction suggests that the FI Resurge group, which had received the reinforcer on FI 20 for six previous sessions, responded more on L1 earlier than later in the interval between successive reinforcers. But, as noted before, there was no such pattern in the Thinning groups. This, along with the failure to see extra L1 responding in the Fixed groups overall (see also Figure 5), suggests that the early resurgence of L1 responding in the Thinning groups does not conform to expectations based on the idea that it had become an interim behavior entrained on L2's schedule of reinforcement.

Figure 6.

Fine grained analysis of L1 and L2 responding during the inter-reinforcer intervals received by the Fixed Interval (Fixed) and Random Interval (Random) groups in Experiment 3. Different groups are shown in each panel; note the differences in the abscissa used to describe the results with the Thinning groups (in the left panels) and the Resurgence groups (in the right panels).

Discussion

In Experiment 3, the early resurgence of L1 responding observed in the previous experiments ultimately disappeared when training was extended on the final (120 s) reinforcement schedules. It is possible that the more sustained L1 behavior observed in the previous experiments thus depends on continued schedule change, although the rats that received extensive training with the final interval also received a greater amount of training overall than the thinned rats in the previous experiments. Importantly, because L1 responding in the various groups converged to a common level at the end of Phase 2, their vulnerability to resurge at the end of the experiment can be compared with greater confidence. Consistent with the original context-change hypothesis, thinning reinforcement for L2 clearly did reduce the resurgence that occurred when L2 was finally put on extinction.

Like the data of Experiment 2, a finer-grained analysis of responding in time suggested little support for the idea that L1 responding increased during thinning because of adventitious reinforcement or occasion setting by pellet presentation. In addition, our analysis of the distribution of L1 responding over the inter-reinforcer interval casts doubt on the hypothesis that the maintenance of L1 pressing during Phase 2 occurs because the L1 response becomes entrained as a kind of interim behavior. Although there was a suggestion that L1 responding was most likely early in the inter-reinforcer interval in a group that had received repeated exposure to reinforcers at 20 s intervals (Group FI Resurge), there was no evidence that the probability of L1 responding was distributed that way in the thinned groups. There was thus no support for the idea that early resurgence of L1 responding was due to L1 becoming an interim behavior.

It is also worth noting that animals in Group FI Resurge, which experienced unchanging FI 20 reinforcement of L2 pressing throughout Phase 2, showed less L2 pressing than did Group RI Resurge, and yet did not produce more L1 presses in this phase. This pattern continues to question a role for simple response competition between L2 and L1 in creating the results. If the increase in L1 responding during Phase 2 in the thinned groups merely reflected less competition from their lower L2 responding, then Group FI Resurge should have also shown more L1 responding than Group RI Resurge. But it did not. Moreover, the robust final resurgence eventually observed in both the Resurge groups also replicates our previous finding that the degree of resurgence does not depend upon the vigor of L2 responding in Phase 2 (Winterbauer & Bouton, 2010). Neither early resurgence (the emergence of L1 responding during Phase 2) nor resurgence itself (the emergence of L1 responding when L2 is put on extinction in Phase 3) seems to depend on the rate of L2 behavior.

General Discussion

The results of these three experiments suggest that procedures that thin the rate of reinforcement for L2 responding before extinction begins can reduce the size of the final resurgence effect. Although there were indications that thinning reduced final resurgence in Experiments 1 and 2, where the thinning groups and resurgence control groups entered testing with different rates of L1, the conclusion was most strongly supported by the results of Experiment 3, where the groups did not differ as they entered testing. The fact that thinning the L2 reinforcement schedule can reduce resurgence is directly predicted by the context-change hypothesis (e.g., Winterbauer & Bouton, 2010), which would assert that thinning allows extinction of L1 to become associated with long inter-reinforcer intervals that begin to approximate extinction. The finding that thinning also led to an “early resurgence” of L1 responding is also compatible with this approach, because detectable changes in the reinforcement “context” in which L1 extinction occurs would cause a renewal of responding earlier in Phase 2.

The elimination of final resurgence by thinning might also be anticipated by other accounts of resurgence. For example, behavioral momentum theories (Podlesnik & Shahan, 2009, 2010; Nevin & Grace, 2000) explain extinction with two factors. Animals experience generalization decrement from acquisition as a result of the changed circumstances of extinction, and responding over time is increasingly disrupted by termination of the response-reinforcer contingency. Shahan and Sweeney's (2011) quantitative model of resurgence extends this framework to include additional disruption due to the reinforcement presented during Phase 2. On this view, resurgence occurs when the latter source of disruption is eliminated by removing reinforcers for L2. Our simulations of the model indicate that thinned rates of reinforcement on L2 do cause less disruption of L1 responding than an unthinned control rate, and thus result in less final resurgence. The model also correctly predicts that thinning can increase L1 responding during Phase 2, provided reinforcement rate is entered into the model as reinforcers/hr rather than reinforcers/min (the unit was not specified by Shahan & Sweeney, 2011). The model is challenged, however, by our finding that a nine-fold increase in the number of Phase 2 sessions (from 4 to 36 sessions in which L1 was extinguished while L2 was reinforced on a Fixed Ratio 10 schedule) had no impact on the strength of the final resurgence effect (Winterbauer, Lucke, & Bouton, in press). According to the model, resurgence should decrease with increasing amounts of Phase 2 training. In its present form, the model can thus account for some, but not all, of the resurgence findings that have been obtained in this laboratory.

The effect of thinning on final resurgence might also be consistent with the response suppression hypothesis (e.g., Leitenberg et al., 1970), because the thinning treatments examined here all caused an early recovery of responding on L1—which could have given the animal more opportunity to learn about the L1 – no pellet contingency. However, the animals in the Resurge conditions actually had an ample opportunity to learn the L1-extinction contingency. For example, in Experiment 3 the Resurgence groups made an average total of 1,312 unreinforced L1 responses in extinction, and still showed a robust resurgence effect. In contrast, the Extinction group in Experiment 1 made an average total of only 872 L1 responses. Such results, coupled with previous data indicating that resurgence is not related to the rate of L1 responding (Winterbauer & Bouton, 2010), suggest that the elimination of final resurgence was not an artifact of more L1 responding in Phase 2.

Perhaps the most remarkable result of the present experiments was the early recovery of responding that was caused by each of the thinning procedures utilized here. In each experiment, the rats responded more on L1 when the L2 schedule was thinned, and L1 responding was sustained over several sessions. Our fine-grained analysis of responding over time cast doubt on the idea that L1 recovered because presentation of the reinforcer merely set the occasion for more L1 responding. The return to L1 was not especially rapid after each reinforcer, and once the response returned, it appeared merely to return to a level that was then fairly stable throughout the inter-reinforcer interval (see especially Figure 6). There was also no evidence that L1 responses were followed more frequently by presentation of the reinforcer. Finally, Fixed Interval schedules that were designed to exaggerate the possible role of schedule-induced behavior (Experiment 3) did not exaggerate the strength of the early resurgence effect, and there was no evidence that L1 responding was distributed over the inter-reinforcer interval in the manner typical of interim behaviors (e.g., Staddon, 1977). Thus, the early resurgence effect did not appear to follow from these possible effects of delivering the reinforcer periodically as a consequence of L2 responding.

Instead, the rat might have discriminated between the stimulus conditions that prevailed before and after the RI interval was incremented as the present reinforcement schedules were thinned. Any such change in stimulus conditions would be expected to produce an ABC renewal effect, and the results are thus further compatible with the context change hypothesis. One virtue of this approach is that, unlike the others described above, it explains and integrates resurgence with principles that account for other “relapse” effects after extinction, including renewal, spontaneous recovery, reinstatement, and rapid reacquisition (see Bouton & Woods, 2008, for a review). However, several features of the present data appear to challenge this account. For one thing, the increase in L1 responding seemed surprisingly rapid. We would note, however, that there is evidence that rats can sometimes discriminate changes in interval reinforcement schedules very rapidly. For example, Gallistel, Mark, King, and Latham (2001) found that under some conditions rats adjusted to schedule changes very quickly, and even optimally, so that they approximated an ideal Bayesian change detector. A second possible challenge for the context-change hypothesis is that different methods used to thin the L2 schedule seemed to do little to change the magnitude of the effect. For instance, the gradual and stepped procedures compared in Experiment 2 produced highly similar results. Thus, one would need to assume that the Gradual and Stepped procedures produced similar stimulus change. A third challenge for the context-change hypothesis is that, although the early resurgence of L1 responding eventually returned to baseline when training continued on the final reinforcement schedule (Experiment 3), it was not a transient effect. In all three experiments, the early resurgence of L1 responding continued for several sessions. In contrast, the ABC renewal effect can appear small and relatively transient under typical testing conditions (see Bouton et al., 2011; Todd, Winterbauer, & Bouton, 2012). Two considerations are worth noting here, however. First, in the present experiments, L1 responding was sustained under conditions in which the L2 schedule continued to be incremented; that is, the “context” continued to change. Second, in the case of early resurgence, but not the typical test of ABC renewal, renewed responding occurs against a background in which the reinforcer is still presented and expected. Contextual conditioning created by the reinforcer is known to augment extinguished operant responding (Baker et al., 1991). Both factors (continual contextual change and a background of contextual conditioning) could in principle contribute to the sustained L1 responding that was observed in Phase 2 in Experiments 1 and 2. Both resurgence and the early resurgence effect documented here are thus consistent with known principles of response recovery in extinction.