Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jun 1.
Published in final edited form as: Learn Behav. 2016 Jun;44(2):137–150. doi: 10.3758/s13420-015-0197-7

Role of the Discriminative Properties of the Reinforcer in Resurgence

Mark E Bouton 1, Sydney Trask 1
PMCID: PMC4838568  NIHMSID: NIHMS732107  PMID: 26486932

Abstract

Three experiments with rat subjects examined the effects of the discriminative effects of reinforcers that were presented during or after operant extinction. Experiments 1 and 2 examined resurgence, in which an extinguished operant response (R1) recovers when a second behavior (R2) that has been reinforced to replace it is also placed on extinction. The results of Experiment 1 suggest that the amount of R1’s resurgence is a decreasing linear function of the interreinforcement interval used during the reinforcement of R2. In Experiment 2, R1 was reinforced with one outcome (O1) and R2 was then reinforced with a second outcome (O2) while R1 was extinguished. In resurgence tests, response-independent (noncontingent) presentations of O2 prevented resurgence of R1, which otherwise occurred when testing was conducted with either no reinforcers or noncontingent presentations of O1. Experiment 3 then examined the effects of noncontingent O1 and O2 presentations after simple extinction in either the presence or absence of noncontingent presentations of O2. Overall, the results are consistent with a role for the discriminative properties of the reinforcer in controlling operant behavior. In resurgence, the reinforcer used during response elimination provides a distinct context that controls inhibition of R1. The results are less consistent with an alternative view emphasizing the disrupting effects of alternative reinforcement.

Resurgence is the return of an extinguished operant behavior that occurs when reinforcement for an alternative second behavior that has replaced it is discontinued (Leitenberg, Rawson, & Bath, 1970; Lieving & Lattal, 2003; Sweeney & Shahan, 2013a; Winterbauer & Bouton, 2010). In a typical study of resurgence, a target response (R1) is first reinforced in an initial phase (Phase 1). Following acquisition, a response-elimination phase (Phase 2) is conducted in which R1 is extinguished while a newly introduced response, R2, is reinforced. Finally, in a test phase (Phase 3), R2 reinforcement is discontinued (i.e. both responses are placed on extinction). The removal of reinforcement results in an increase, or resurgence, of R1 behavior. Resurgence is similar to several other forms of relapse that occur after the extinction of operant behavior (e.g. renewal, reinstatement, and spontaneous recovery) in suggesting that extinction does not erase the original learning (e.g., Bouton, 2014; Bouton, Winterbauer, & Todd, 2012; Bouton, Winterbauer, & Vurbic, 2012).

At least three explanations of resurgence have been suggested. Leitenberg et al. (1970) originally noted that resurgence might occur because reinforcement of R2 can cause this behavior to interfere with R1 behavior so much that the animal does not have the opportunity to learn that responding on R1 is no longer reinforced. Although such a mechanism might contribute to resurgence in some circumstances, animals often emit a large number of R1 responses in extinction, and resurgence can still occur under Phase 2 conditions that cause a higher level of R1 responding in a resurgence group than in an extinction control group that does not demonstrate resurgence during testing (Winterbauer & Bouton, 2010). Such a result seems incompatible with the idea that R1 resurges when it has received too little extinction.

Shahan and Sweeney (2011) proposed an alternative explanation of resurgence in the form of a quantitative model based on behavioral momentum theory (e.g., Nevin & Grace, 2000; Craig, Nevin, & Odum, 2014). The model assumes that adding reinforcement for R2 to the extinction of R1 has two fundamental effects on R1 performance. First, the alternative reinforcement disrupts R1 beyond that of a typical extinction procedure. Second, the alternative reinforcement also increases the strength of R1 through a process (contextual conditioning) that is assumed to strengthen all behavior. When the alternative reinforcement is discontinued during testing, the disruption of R1 behavior is removed, but the increased strength of R1 remains, resulting in resurgence. According to the model, schedules that include leaner programmed rates of R2 reinforcement during the final session of Phase 2 (and thus cause less final disruption and strengthening of R1 behavior) should result in lower levels of resurgence. Consistent with this prediction, resurgence is reduced by using very lean reinforcement rates during the response elimination phase (Leitenberg, Rawson, & Mulick, 1975; Sweeney & Shahan, 2013a). Resurgence (defined as an increase in responding from Phase 2 to testing) is also reduced by using schedules that are “thinned” from high rates to low rates across Phase 2 (Schepers & Bouton, 2015; Winterbauer & Bouton, 2012), although thinning itself can produce an increase in R1 responding (termed “early resurgence” by Winterbauer & Bouton, 2012). However, a reverse thinning procedure in which reinforcement rates start the phase at a lean rate and then increase during training also weakens resurgence (Bouton & Schepers 2014; Schepers & Bouton, 2015). Thus, contrary to the model, differences in resurgence have been demonstrated in experimental and control groups that receive the same rates of reinforcement for R2 in the final response elimination session. As a further example, Schepers and Bouton (2015, Experiment 3) gave one group of rats alternating response-elimination sessions in which R2 was reinforced (on VI 10-s) in odd-numbered sessions and extinguished in even-numbered sessions. After the final VI 10-s session, reinforcement was discontinued. No resurgence was observed in this group, but substantial resurgence was observed in a group that had received VI 10-s during every session. In addition, there was substantial resurgence in a group that received the alternating group’s overall average rate of reinforcement during each session of the phase (a VI 17.5-s reinforcement schedule). Such results suggest that, inconsistent with the model, reinforcement rate calculated either during the last session of response elimination (Shahan & Sweeney, 2011) or averaged over the response elimination phase (Sweeney & Shahan, 2013b) is not necessarily a good predictor of the strength of the resurgence effect.

A third account of resurgence emphasizes the discriminative properties of the reinforcer delivered for R2 responding in Phase 2. Winterbauer and Bouton (2010; see also Bouton & Swartzentruber, 1991) suggested that resurgence is a form of the renewal effect, in which extinguished responding recovers when the background context changes when alternate reinforcement is discontinued (e.g., Bouton, Todd, Vurbic, & Winterbauer, 2011). On this view, the discontinuation of reinforcement during the test creates a change of context from the one that prevailed during R1 extinction. Previous research in Pavlovian conditioning has established that the presence or absence of reinforcers (food pellets) can provide a “reinforcer context” that controls conditioned or extinction performance that is associated with it (Bouton, Rosengard, Achenbach, Peck, & Brooks, 1993). Thus, given the well-documented context-specificity of extinction learning, the animal in a resurgence experiment might learn to inhibit R1 in the “context” of reinforcement of R2; when that context is changed by discontinuing R2 reinforcement, response recovery will occur. Such a perspective is consistent with the finding, mentioned above, that alternating exposure to VI-10 reinforcement and extinction for R2 can abolish the resurgence effect (Schepers & Bouton, 2015). This is because the animal theoretically learns to inhibit R1 when there are few reinforcers in the background, conditions more like the extinction conditions that prevail during the resurgence test. The perspective is also consistent with the finding that reverse and forward thinning procedures both reduce resurgence (Bouton & Schepers, 2014; Schepers & Bouton, 2015), with forward thinning arguably being the more effective in reducing it (Schepers & Bouton, 2015). (Forward thinning may be more effective than reverse thinning because there is less abrupt of a change of context as the animal enters the test phase.) The contextual view importantly emphasizes the discriminative properties of reinforcers presented during Phase 2, rather than their reinforcing and disrupting effects. In doing so, it is consistent with a long theoretical tradition in learning theory (e.g., Ostlund & Balleine, 2007; Reid, 1958; Sheffield, 1949).

The purpose of the present experiments was to further contrast predictions of the behavioral momentum (Shahan & Sweeney, 2011) and contextual (Winterbauer & Bouton, 2010) accounts of resurgence. Experiment 1 parametrically explored the prediction, common to both views, that resurgence should be an inverse function of the rate of reinforcement delivered in Phase 2. Experiment 2 then pitted the two perspectives against one another more directly. In that experiment, rats earned one reinforcer (O1) for R1 in the first phase and a different reinforcer (O2) for R2 when R1 was extinguished in the second. We then examined the effects of response-independent presentations of either O1 or O2 during resurgence testing. Although the contextual view’s emphasis on the discriminative properties of the reinforcer suggests that O2 presentations (but not O1 presentations) should prevent resurgence, behavioral momentum theory does not make such a prediction. In Experiment 3, we examined the effects of adding noncontingent O2 reinforcers during a simple extinction procedure and tested its effects on reinstatement, the procedure in which extinguished responding returns after noncontingent reinforcement delivery (e.g., Baker, Steinwald, & Bouton, 1991; Franks & Lattal, 1976; Reid, 1958; Rescorla & Skucy, 1969). Overall, the results of Experiments 2 and 3 are inconsistent with the momentum-based resurgence model (Shahan & Sweeney, 2011) and point to a role for the discriminative properties of the reinforcer when it is presented during or following response elimination by extinction.

Experiment 1

The first experiment pursued the finding that a thinner reinforcement schedule during the response elimination phase would reduce or eliminate the resurgence effect. As noted earlier, response elimination procedures that have thinned the reinforcement schedule for R2 from either rich to sparse (Schepers & Bouton, 2015; Sweeney & Shahan, 2013a; Winterbauer & Bouton, 2012) or sparse to rich (Bouton & Schepers, 2014; Schepers & Bouton, 2015) have weakened the resurgence effect. Leitenberg et al. (1975) also found weaker resurgence when a second behavior was reinforced at a constant lean rate (VI 240-s) during extinction of the first behavior compared to resurgence seen when the second response is reinforced at a high rate (VI 30-s). Sweeney and Shahan (2013a) similarly found that while alternative reinforcement with a high rate of reinforcement (VI 10-s) resulted in resurgence, using a lean rate (VI 100-s) abolished the effect. However, Winterbauer and Bouton (2010) found no difference in resurgence between VI 10-s and VI 30-s. The parametric relationship between VI schedule and resurgence has not been firmly established.

Experiment 1 therefore explored the parameter space more systematically. After initial reinforcement of R1 on a VI 30-s reinforcement schedule, different groups received reinforcement of R2 on either a VI 30-s, VI 60-s, VI 90-s, or VI 120-s schedule during the extinction of R1. R1 and R2 were then tested in extinction. Given Winterbauer and Bouton’s (2012) finding that thinning to VI 120-s schedule can reduce resurgence, we expected little resurgence in the VI 120-s group. The question was how the strength of resurgence was parametrically related to R2’s rate of reinforcement.

Method

Subjects

The subjects were 32 female Wistar rats obtained from Charles River, Inc. (St. Constance, Quebec). The rats were approximately 85–95 days old at the start of the experiment and were individually housed in suspended stainless steel cages in a room maintained on a 16:8-h light:dark cycle. At the beginning of the experiment, all rats were food deprived to 80% of their free-feeding weight and maintained at that level throughout the experiment with a single feeding following each day’s session.

Apparatus

Conditioning proceeded in two sets of four standard conditioning boxes (Med-Associates Model Number: ENV-008-VP, St. Albans, VT) that were housed in different rooms of the laboratory. The sets had been modified as described below for use as separate contexts, although they were not used in that capacity here. Boxes from both sets measured 30.5 cm × 24.1 × 21.0 cm (l × w × h), with side walls and ceilings made of clear acrylic plastic and front and rear walls made of brushed aluminum. Recessed 5.1 cm × 5.1 cm foodcups with infrared photobeams positioned approximately 1.2 cm behind the plane of the wall and 1.2 cm above the bottom of the cup were centered in the front wall about 3 cm above the grid. In one set of four boxes, the floor was composed of stainless steel rods (0.5 cm in diameter) in a horizontal plane spaced 1.6 cm center to center, while in the other set of four boxes, the floor was composed of identical rods spaced 3.2 cm apart in two separate horizontal planes, one 0.6 cm lower than the other and horizontally offset by 1.6 cm. The boxes with the planar floor grid had a side wall with black panels (7.6 cm × 7.6 cm) placed in a diagonal arrangement, and there were diagonal stripes on both the ceiling and the back panel, all oriented in the same direction, 2.9 cm wide, and about 4 cm apart. The other boxes, with the staggered floor, were not adorned in any way. Retractable levers (1.9 cm when extended) were positioned approximately 3.2 cm to the right and to the left of the food cup and 6.4 cm above the grid. Both sets of boxes were housed in sound-attenuating chambers, and were continuously illuminated by two 7.5-W incandescent light bulbs mounted on the chamber ceiling. The apparatus was controlled by computer equipment located in an adjacent room. Food reinforcers consisted of 45-mg MLab Rodent Tablets (5-TUM: 181156; TestDiet, Richmond, IN).

Procedure

Following a day in which one session of magazine training occurred (see below), twice daily sessions were employed throughout the experiment. Each day’s first session began with approximately 15 h of illuminated colony time remaining. Each day’s second session began with approximately 12 h of illuminated time remaining. Animals were placed into illuminated conditioning chambers, and the start of each session was indicated by the insertion of the lever(s) as appropriate. All sessions were 30 min in duration, and the end of the session was indicated by retraction of the lever(s).

Magazine training

All animals received magazine training on the day immediately prior to the beginning of Phase 1. At this time, the rats received a single session with both levers retracted. An average of 60 food pellets were delivered during this session on a random time 30-s (RT 30-s) schedule of reinforcement.

R1 Acquisition (Phase 1)

All animals then received 12 sessions of instrumental conditioning initiated by insertion of the left lever in half animals and the right lever in the other half. In all sessions, presses on the lever (R1) delivered pellets on a VI 30-s schedule of reinforcement. No additional response shaping was necessary.

R1 Extinction and R2 Acquisition (Phase 2)

All animals then received ten sessions in which R1 presses were extinguished and presses to the second lever (R2) were reinforced. Both the left and the right levers were inserted throughout each session. R1 presses had no programmed consequences, but presses on the new second lever (R2) earned pellets on either a VI 30-s, VI 60-s, VI 90-s, or VI 120-s schedule of reinforcement for different groups (n = 8). All groups started with R2 reinforced on a VI 30-s schedule, but were quickly shaped to their terminal schedule (VI 60-s, VI 90-s or VI 120-s) over the first two sessions of the phase. For Group VI 60, shaping consisted of one session in which R2 presses were reinforced on VI 30-s schedule for 15 min and VI 60-s for the next 15 min. Groups VI 90 and VI 120 received a treatment identical to that in Group VI 60 in the first session of the phase. In the second session, R2 in Group VI 90 was reinforced on VI 60-s for 15 min and then VI 90-s for the last 15, whereas R2 in Group VI 120 was reinforced on VI 60-s for the first 10 min, VI 90-s during the second 10 min, and VI 120-s during the last 10 min. All rats were on their terminal schedules of reinforcement by the start of the third session of Phase 2.

Resurgence Test (Phase 3)

On the day following the conclusion of Phase 2, all groups received a final test session with both levers inserted. R1 and R2 presses were recorded, but neither had scheduled consequences.

Data Analysis

Analyses of variance (ANOVAs) were used to assess response rates throughout the experiment. The rejection criterion was p < .05 for all analyses.

Results

The results of the experiment are presented in Figure 1. Responding on R1 was readily learned in Phase 1 (top left) and extinguished in Phase 2 (top middle). Responding on R2 was also readily learned in Phase 2 (lower middle); while the rate of responding appeared most rapid in the VI 30 condition, R2 responding was statistically similar among the groups. However, resurgence of R1 during the final test (top right) was restricted to the groups that had received the richest schedules of reinforcement for R2 responding during Phase 2.

Figure 1.

Figure 1

Open in a new tab

Results of Experiment 1. Upper panels: Mean R1 responding (+ SEM) during acquisition (left), extinction (middle), and testing (right). Lower panels: Mean R2 responding (+ SEM) during its acquisition and testing. All groups received extinction of R1 at the same time R2 was introduced and reinforced on either a VI 30, VI 60, VI 90, or VI 120-s schedule of reinforcement. Note changes in the y axes between panels; error bars are only appropriate for between-group comparisons.

R1 Acquisition

A 4 (Group) × 12 (Session) ANOVA on R1 responding during acquisition revealed a main effect of session, F (11, 308) = 63.00, MSE = 42.04, p < .001, ηp2= .69, but no main effect of group or a group by session interaction, Fs < 1.

R1 Extinction and R2 Acquisition

R1 extinction was assessed with a 4 (Group) × 10 (Session) ANOVA. While there was a main effect of session, F (9, 252) = 38.78, MSE = 10.91, p < .001, ηp2= .58, there was no main effect of group or a group by session interaction, Fs < 1. A similar ANOVA performed on R2 acquisition found a main effect of session, F (9, 252) = 16.42, MSE = 34.18, p < .001, ηp2= .37, but neither a main effect of group, F (3, 28) = 2.79, MSE = 984.70, p > .05, nor a significant interaction, F < 1.

Resurgence Test

A 4 (Group) × 2 (Session) ANOVA compared the groups’ responding during the last 15 minutes of Phase 2 to the first 15 minutes of Phase 3. It revealed a significant overall effect of session, F (1, 28) = 24.29, MSE = 2.96, p < .001, ηp2= .46, suggesting resurgence, but no significant main effect of group, F < 1. Importantly, there was a significant group by session interaction, F (3, 28) = 4.81, MSE = 2.96, p < .01, ηp2= .34, suggesting that resurgence differed among the groups. Pairwise comparisons revealed that Group VI 30, F (1, 28) = 9.09, p < .01, ηp2= ,25, and Group VI 60, F (1, 28) = 28.11, p < .001, ηp2= .50, both increased responding between extinction and testing (resurgence), but that Groups VI 90 and VI 120 did not, Fs (1, 28) < 1.40. A trend analysis on the difference scores (first 15 min of the test - last 15 min of Phase 2) revealed a significant linear trend, F (1, 28) = 7.31, p < .05, ηp2= .21, suggesting that resurgence decreased as a linear function of VI schedule. The quadratic trend was not significant, F (1, 28) = 2.42, p = .13.

We would note that the increase in responding between the last extinction session and the resurgence test could have been due in part to spontaneous recovery. Indeed, responding increased as a function of time between the last two sessions of Phase 2. A similar 4 (Group) × 2 (Session) ANOVA on responding during the last 15 mins of Session 9 (the second-to-last Phase 2 session) and the first 15 minutes of Session 10 (the last session of Phase 2) revealed a significant effect of session, F (1, 28) = 16.08, MSE = 2.76, p < .001, ηp2= .37. But importantly, the group × session interaction did not approach reliability, F (3, 28) = 2.00, MSE = 2.76, p > .10. Thus, the differential effect of the group treatments did not appear until the resurgence test (Sessions 10 vs. 11). Mean responding on the last 15 mins of Session 9 and the first 15 mins of Session 10 were 4.2 and 5.9, respectively.

An ANOVA on R2 responding comparing the final 15 minutes of Phase 2 to the first 15 minutes of the resurgence test revealed a significant main effect of session, F (1, 28) = 47.45, MSE = 57.76, p < .001, ηp2= .63, but neither a group effect, F (3, 28) = 2.53, MSE = 239. 03, p > .05, nor a group by session interaction, F < 1. As before a trend analysis was conducted on R2 difference scores (first 15 of test – last 15 of extinction). Neither a linear (F < 1) nor a quadratic (F = 1.03) trend was significant.

Discussion

The results confirmed that resurgence decreases systematically as a function of the average inter-reinforcer interval (VI schedule) used in the response elimination phase. The fact that reinforcement of R2 on either a VI 90-s or a VI 120-s schedule while R1 was being extinguished prevented resurgence is consistent with the context hypothesis, which suggests that rats given infrequent reinforcers during Phase 2 might learn to inhibit R1 in a context that is increasingly similar to the (extinction) resurgence test context. It is also consistent with the momentum-based model (Shahan & Sweeney, 2011), which suggests that thinner schedules cause less conditioning of the background (which would otherwise serve to increase the strength of R1).

It is worth noting that the groups all received 30-min sessions in Phase 2, and that they consequently differed in the number of reinforcers they earned in Phase 2 as well as in their rates of reinforcement. We are not aware of any theory that predicts an effect of varying the number of reinforcers, however. It is also worth noting that the leanest reinforcement schedules used in Phase 2 (e.g., VI 120-s) differed most from the VI 30-s schedule used in Phase 1; the context hypothesis might therefore predict a faster loss of R1 responding due to differential generalization decrement. The fact that the groups did not differ in R1 responding in extinction could be due to the fact that the leanest reinforcement schedules might also generate the least response competition from R2 (although the groups also did not differ in the rate of R2). Regardless, the overall pattern of results may be consistent with the fact that although context change can have an impact on both operant responding and operant extinction, operant extinction appears to be more affected by context change (e.g., Bouton & Todd, 2014).

Experiment 2

The context hypothesis’s emphasis on the discriminative, rather than the reinforcing, properties of the reinforcer uniquely predicts that there might be effects of changing the qualitative properties of the reinforcer between phases in the resurgence design. In Experiment 2, three groups of rats therefore first received reinforcement of R1 on a VI 30-s schedule with one pellet outcome (O1; either a grain-based or sucrose food pellet, counterbalanced). Then, when R1 was extinguished in the next phase, R2 was reinforced (also on a VI 30-s schedule) with an alternate reinforcer (O2, the sucrose or grain-based pellet). According to the context hypothesis, R1 extinction would be learned in the presence of O2 presentations. To test this possibility, the groups received different treatments during testing. Group None was simply tested with both R1 and R2 in extinction; resurgence of R1 was expected in this group. In contrast, Group O2 received response-independent presentations of O2 during the test. Since O2 was presented noncontingently at the same rate that it was earned during response elimination, either the context hypothesis or the Shahan-Sweeney model would predict no resurgence in this group (see Lieving & Lattal, 2003, Experiment 3). However, the third group, Group O1, received response-independent presentations of O1, rather than O2, during testing. If the O1 pellet is sufficiently different from the O2 pellet that provided the context of R1’s extinction during Phase 2, the context hypothesis predicts response recovery in this group. In contrast, neither the Shahan-Sweeney (2011) model, nor the original behavioral momentum theory from which it was developed, distinguishes between different types of reinforcers (see Grimes & Shull, 2001; Shahan & Burke, 2004). Instead, the model would assume that delivery of O1 at the same rate as O2 in the response elimination phase would continue to suppress performance of R1.

Method

Subjects and Apparatus

The subjects were 24 naïve female Wistar rats of the same age and stock as those in Experiment 1. Their housing and maintenance conditions, as well as the apparatus, were also the same. Two reinforcers were used: the 45-mg grain-based rodent food pellet used in Experiment 1 as well as a 45-mg sucrose pellet (5-TUT: 1811251; TestDiet, Richmond, IN, USA).

Procedure

The animals received two sessions of training every day. The first session began with approximately 15 h of illuminated colony time remaining, while the second began with approximately 13 h of illuminated time remaining. As before, the start of each session was indicated by insertion of the lever(s). All sessions were 30 min in duration, and with the lever(s) being retracted at the end.

Magazine training

All animals received two magazine training sessions, one with each type of reinforcer, on the day immediately prior to beginning Phase 1. Pellets were delivered during each session (with both levers retracted) on an RT 30-s schedule, resulting in approximately 60 pellets delivered during each session. Pellet type was counterbalanced so that half the rats were trained first with their O1 pellet (further counterbalanced as grain or sucrose) and the other half were trained first with their O2 pellet (also counterbalanced as sucrose or grain).

R1 Acquisition (Phase 1)

All animals then received 12 sessions of instrumental conditioning of R1, which was the left lever for half the animals and the right lever for the other half. In all sessions, presses on R1 delivered O1 on a VI 30-s schedule of reinforcement.

R1 Extinction and R2 Acquisition (Phase 2)

All rats then received 12 sessions in which R1 was extinguished and R2 was reinforced. Both the left and the right levers were inserted throughout each session. R1 presses had no scheduled consequences, but presses on the newly-inserted second lever (R2) earned O2 on a VI 30-s schedule of reinforcement.

Resurgence Test (Phase 3)

The next day, all animals received a single test session with both levers inserted. For all groups, R1 and R2 responses had no programmed consequences (i.e., were extinguished). Group None received no other events during the test. Group O1 received noncontingent deliveries of O1 (delivered previously in Phase 1) on an RT 30-s schedule throughout the test. Group O2 received noncontingent delivery of O2 (previously delivered in Phase 2) on an RT 30-s schedule throughout the test.

Data Analysis

As in Experiment 1, analyses of variance (ANOVAs) were used to assess response rates throughout the experiment with a criterion of p < .05 for all analyses.

Results

The main results of the experiment are summarized in Figure 2. R1 responding was readily acquired in Phase 1, when O1 served as the reinforcer, and extinguished in Phase 2. R2 responding was also readily acquired in Phase 2, when O2 now served as the reinforcer. In the resurgence test, there was an increase in R1 responding in Groups None and O1, but Group O2 showed no change in R1 performance from extinction.

Figure 2.

Figure 2

Open in a new tab

Results of Experiment 2. Upper panels: Mean R1 responding (+ SEM) during acquisition (left), extinction (middle), and testing (right). Lower panels: Mean R2 responding (+ SEM) during its acquisition and testing. All groups received reinforcement of R1 with one outcome (O1) and then reinforcement of R2 with a second outcome (O2) while R1 was extinguished. During the test, rats received extinction of both R1 and R2 with either response-independent presentations of O1, O2, or no reinforcers (None). Note changes in the y axes between panels; error bars are only appropriate for between-group comparisons.

R1 Acquisition

The acquisition of R1 responding is shown in the upper left of Figure 2. A 3 (Group) × 12 (Session) ANOVA revealed a main effect of session, F (11, 231) = 54.61, MSE = 49.17, p < .001, ηp2= .72, but no main effect of group or a group by session interaction, Fs < 1.

R1 Extinction and R2 Acquisition

R1 extinction is shown in the upper middle of Figure 2. A 3 (Group) × 12 (Session) ANOVA uncovered a main effect of session, F (11, 231) = 19.81, MSE = 6.23, p < .001, ηp2= .49, but no group effect, F < 1, or interaction, F (22, 231) = 1.19, MSE = 6.23, p > .05. A similar ANOVA on R2 Acquisition (lower middle panel of the figure) also revealed a main effect of session, F (11, 231) = 41.75, MSE = 72.57, p < .001, ηp2= .67, but no group effect or interaction, Fs < 1.

Resurgence Test

As in Experiment 1, to assess resurgence, a 2 (Session) × 3 (Group) ANOVA compared the groups’ R1 responding on the last 15 min of Phase 2 and the first 15 min of the resurgence test (Phase 3). There was a main effect of session, F (1, 21) = 10.87, MSE = 11.58, p < .01, ηp2= .34, indicating an overall increase in responding, but no main effect of group, F < 1. The group by session interaction fell short of the conventional rejection criterion, F (2, 21) = 2.95, MSE = 11.58, p = .075, ηp2= .22. Planned pairwise comparisons revealed a significant increase in R1 responding in Group None, F (1, 21) = 10.64, p < .01, ηp2= .37, as well as Group O1, F (1, 21) = 6.12, p < .05, ηp2= .23. In contrast, Group O2 showed no change in performance, F < 1. No group differences were detected during the test, ps > .05, although there was a trend toward significance when comparing Group None and Group O2, p = .08. A supplemental analysis of R1 responding in the first 5 minutes of the test (and last 5 mins of extinction) supported the same conclusions. Here, Group None responded with rates of 0.2 and 5.3 responses/min in the last 5 mins of extinction and first 5 mins of testing, whereas Group O1 had corresponding rates of 1.1 and 6.8 and Group O2 had rates of 1.8 and 1.3. Once again, there was a main effect of session, F (1, 21) = 10.86, MSE = 13.09, p < .01, ηp2= .34, but no main effect of group, F (2, 21) = 1.53, MSE = 15.18, p > .05. Here the group by session interaction was significant, F (2, 21) = 3.63, MSE = 13.09, p < .05, ηp2= .26. As before, Group None, F (1, 21) = 8.10, ηp2= .28, p = .01, and Group O1, F (1, 21) = 9.93, p < .01, ηp2= .32, showed a significant increase in R1 responding between extinction and testing, whereas Group O2 did not, F < 1. Group O2 responded less than Group O1 throughout the first five minutes of the test, p < .05. No other group differences were detected.

As we noted in the previous experiment, the increase in responding between the last extinction session and the resurgence test could have been due in part to spontaneous recovery. A similar 3 (Group) × 2 (Session) ANOVA on responding during the last 15 mins of Session 11 (the second-to-last Phase 2 session) and the first 15 minutes of Session 12 (the last session of Phase 2) also revealed a significant effect of session, F (1, 21) = 10.29, MSE = 1.67, p < .01, ηp2= .33. But importantly, the group × session interaction did not approach reliability, F (2, 21) < 1. Thus, the differential effect of the group treatments was again restricted to the resurgence test (Sessions 12 vs. 13). Mean responding on the last 15 mins of Session 9 and the first 15 mins of Session 10 were 0.8 and 1.5, respectively.

Regarding R2 responding during the resurgence test (lower right of Figure 2), a 2 (Session) × 3 (Group) ANOVA run to asses R2 responding from the last 15 minutes of Phase 2 to the first 15 minutes of Phase 3 found a significant main effect of session, F (1, 21) = 27.95, MSE = 225.60, p < .001, ηp2= .57, but no main effect of group, F (2, 21) = 1.40, MSE = 624.33, p > .05, nor a group by session interaction, F < 1. A 2 (Session) × 3 (Group) ANOVA was run to assess responding on R2 from the last five minutes of extinction to the first five minutes of test. Mean response rates during the last five minutes of Phase 2 were 61.5, 62.7, and 48.6 responses per minute for Group None, Group O1, and Group O2 respectively, and 30.3, 35.6, and 27.0 responses per minute during the first five minutes of the test. While there was a significant main effect of session, F (1, 21) = 26.75, MSE = 318.83, p < .001, ηp2= .56, there was no main effect of group as well as no significant interaction, Fs < 1.

Discussion

As expected, removal of O2 during the test caused a resurgence of R1 responding in Group None. In contrast, the noncontingent presentations of O2 in Group O2, which theoretically served to maintain the “context” associated with R1’s extinction, prevented resurgence from occurring. Even more important, presentation of a different reinforcer in Group O1 allowed responding to recover. Evidently, specific presentation of O2 is necessary to inhibit or suppress R1 responding. The results are not anticipated by the Shahan-Sweeney model, and instead suggest a role for the discriminative properties of O2 in controlling the inhibition of R1 performance. The results further extend the work of Lieving and Lattal (2003), who found that noncontingent reinforcers presented in the test attenuated the resurgence effect. The current experiment suggests that this effect depends on the reinforcer being the one that was specifically presented during R1’s extinction.

One issue in Experiments 1 and 2 is whether the response recovery observed during resurgence testing was due to spontaneous recovery of the R1 response that might result from the passage of time between sessions rather than the removal of R2 reinforcement. To our knowledge, no experiment on resurgence has included a spontaneous recovery control that receives another session of the Phase 2 treatment during testing (see Bouton & Schepers, 2014; Cançado & Lattal, 2011; Kincaid, Lattal, & Spence, 2015; Lieving & Lattal, 2003; Schepers & Bouton, 2015; Sweeney & Shahan, 2013a; Winterbauer & Bouton, 2010, 2011, 2012). Our analyses of the results of both experiments indicated that while R1 responding did increase as a function of time between the last two sessions of Phase 2, the crucial differences between the groups that were present during resurgence testing were not present then. Thus, the effects of the groups’ treatments were linked to resurgence testing. That observation, along with the present finding that noncontingent O2 presentations during testing prevented response recovery, strongly suggests that the recovery observed during resurgence testing was at least partly due to the removal of the reinforcers delivered in Phase 2.

It is notable that presentations of O1 during testing (Group O1) did not increase R1 above the level observed in the group that received no reinforcers at all (Group None). Since O1 had been a feature of Phase 1, when R1 had been reinforced, it would have been possible to observe R1 at a level higher than that in Group None. Such a result, analogous to reinstatement after extinction (e.g., Baker et al., 1991; Franks & Lattal, 1976; Reid, 1958; Rescorla & Skucy, 1969), would have been expected if the rats had received simple extinction of R1, without additional reinforcement of R2 with O2 (Delamater, 1997). However, in the resurgence design used here, reinforcers are presented in both Phases 1 and 2, and this is not true of simple extinction. It is known that presenting the reinforcer in extinction can eliminate the reinstatement effect it produces when it is presented again after extinction (e.g., Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011). Consequently, if there is some degree of generalization between the present O1 and O2, which would not be surprising given that both were 45-mg food pellets with similar motivational attributes, then any stimulus features common to both O1 and O2 would be associated with the inhibition of R1 during Phase 2. Presentation of O1 should therefore have less impact when extinction involves O2 reinforcers than when it occurs without them. According to this analysis, the lack of reinstatement by O1 was due to O2’s presence in extinction. The next experiment was therefore designed to explore this possibility.

Experiment 3

In Experiment 3, rats again received reinforcement of R1 with O1 in an initial training phase. Then, in a second phase, R1 was simply extinguished, without reinforcement of a new behavior (R2). There were two groups. Group O2 received extinction of R1 in the presence of response-independent O2 presentations delivered on a VT 30-s schedule. Although O2 was presented noncontingently, it was presented at the same rate at which it was earned by all groups in the resurgence design of Experiment 2, and was expected to have a similar effect (see Winterbauer & Bouton, 2010, Experiment 4). The second group of the present experiment, Group Ext, received simple R1 extinction without any presentations of O2. In the test phase, both groups were tested for responding under three conditions presented in a counterbalanced order: one in which no pellets were delivered, one in which O2 was delivered noncontingently, and one in which O1 was delivered noncontingently. If O1 and O2 control behavior according to their discriminative properties, we should once again expect a recovery of R1 responding in the O2 group when O2 was removed. In contrast, presenting no reinforcers during the test should have little effect on performance in Group Ext. However, Group Ext should show reinstatement of extinguished R1 responding when O1 is presented during the test. And if the argument made above is correct, and there is indeed some generalization between O2 and O1, then there might be comparatively little reinstating effect of O1 presentations after extinction in Group O2.

Method

Subjects and Apparatus

The subjects were 36 naïve female Wistar rats of the same age and stock as those in Experiments 1 and 2, housed and maintained under the same conditions. The two reinforcers (grain-based and sucrose-based pellets) from Experiment 2 were also used in this experiment.

Procedure

Throughout magazine training and acquisition, subjects received two daily 30-min sessions, as in Experiments 1 and 2. During extinction, subjects received one daily 30-min session. All sessions began with left lever insertion after a two-minute delay following introduction to the experimental chamber and ended with lever retraction. The right lever remained retracted throughout the entire experiment.

Magazine Training

Magazine training proceeded exactly as in Experiment 2. Each animal was assigned to an experimental chamber and then given one session with its O1 reinforcer (grain or sucrose pellet, counterbalanced) delivered noncontingently on an RT 30-s schedule and another session with its O2 reinforcer (sucrose or grain pellet, counterbalanced) delivered the same way. Sessions were counterbalanced so that half the animals received O1 first and the other half received O2 first.

Acquisition

Over the next six days, the rats received 12 sessions in which responding on the left lever was reinforced with O1 according to a VI 30-s reinforcement schedule.

Extinction

Animals were then divided into two groups (ns = 18, a number that allowed the counterbalancing of test order, see below). For the next 8 daily sessions, responding on the left lever had no scheduled consequences (i.e., it was extinguished). For animals in Group O2, however, O2 reinforcers were delivered independent of responding according to an RT 30-s schedule throughout the session and during the two-minute delay before lever insertion. Animals in Group Ext received no O2 reinforcers. In order to control for exposure to O2, each animal in Group Ext was matched to an animal in Group O2 and received the same number of O2 reinforcers, but this time in a food cup in the homecage 3.5 – 5 hours following the day’s session. Rats in Group O2 were given equal exposure to an empty food cup in the homecage at this time.

Test

On the final day of the experiment, all rats were given three 5-min test sessions. In the O1 test, O1 was delivered on an RT 30-s schedule throughout the 5-min session and the 2-min delay. In the O2 test, O2 reinforcers were delivered in the same way. In the None (N) test, no reinforcers were delivered. Testing order was counterbalanced so that three animals in each group received tests in the order of O1O2N, O1NO2, O2O1N, O2NO1, NO1O2, and NO2O1. The interval between tests was approximately 60 min.

Data Analysis

As before, ANOVAs were conducted to assess responding with a rejection criterion of p < .05. Two animals in Group O2 failed to extinguish by the end of extinction compared to the other animals (final day of extinction Ms = 9.13, 8.33, Zs ≥ 3.10; Field, 2005) and were therefore excluded from all analyses. One animal in Group Extinction was also excluded from all analyses because she was a significant group outlier during the None Test (Z = 3.44, Field, 2005).

Results

The results of the experiment are summarized in Figure 3. As shown at top, both groups acquired R1 in the first phase and then decreased their performance in extinction. The presentation of noncontingent O2 reinforcers during extinction initially seemed to suppress, but then significantly enhanced, the rate of responding in extinction relative to Group Ext. During the final tests, only Group O2 showed a significant resurgence effect when tested without pellets in the None condition, and only Group Ext, which received simple extinction (without O2 presentations), showed a significant reinstatement effect when given the O1 pellet noncontingently in the O1 condition.

Figure 3.

Figure 3

Open in a new tab

Results of Experiment 3. A: Mean responding (+ SEM) during acquisition and extinction of the operant response. During acquisition, responding led to one outcome (O1). In extinction, one of the groups (O2) had response-independent presentations of a second outcome (O2) while the other group (Ext) received simple extinction. B: Mean responding (+ SEM) during testing. The three panels compare the groups’ responding during the last session of extinction (Ext 8) with responding during tests in the presence of response-independent presentations of O1 (O1 Test, left), O2 (O2 Test, center), or no reinforcers at all (None Test, right). Note changes in y axes between panels; error bars are only appropriate for between-group comparisons.

Acquisition

All animals increased responding throughout acquisition (shown in the top left panel of Figure 3), as confirmed by a 2 (Group) × 12 (Session) ANOVA that revealed a significant main effect of session, F (11, 241) = 79.93, MSE = 31.60, p < .001, ηp2= .78. Neither the main effect of group nor the interaction was significant, Fs < 1.

Extinction

Responding in extinction is shown in the top right panel of Figure 3. Both groups decreased responding throughout the extinction phase. However, the suppression of performance was slower in Group O2. This was confirmed by a 2 (Group) × 8 (Session) ANOVA which revealed a main effect of session, F (7, 217) = 68.07, MSE = 9.15, p < .001, ηp2= .69, and a significant group by session interaction, F (7, 217) = 4.46, MSE = 9.15, p < .001, ηp2= .13. There was no main effect of group, F (1, 31) = 1.72, p = .20. On the final day of extinction, Group O2 was still significantly elevated relative to Group Extinction, F (1, 31) = 6.68, MSE = 1.34, p < .05, ηp2= .18.

Test

The results of the test are shown across the bottom three panels of Figure 3. Separate ANOVAs were conducted to assess the change in responding from the last day of extinction to each test. In the O1 test (shown in the bottom left panel of Figure 3), a 2 (Group) × 2 (Session) ANOVA found both a significant main effect of session, F (1, 31) = 6.67, MSE = 2.56, p < .05, ηp2= .18, and a significant group by session interaction, F (1, 31) = 4.13, MSE = 2.56, p = .05, ηp2=,12. There was no main effect of group, F < 1. Pairwise comparisons revealed that while Group O2 did not change its performance from the last day of extinction, F < 1, Group Ext showed a significant increase in responding, F (1, 31) = 10.98, p < .01, ηp2= .26. In the O2 test (shown in the bottom middle panel of Figure 3), a similar ANOVA revealed no main effect of session, no main effect of group, and no interaction between the two, largest F = 2.46, p = .13. Follow-up comparisons found that neither Group O2, F < 1, nor Group Ext significantly different between sessions, although there was a trend in Group Ext, F (1, 31) = 3.52, p = .07, ηp2= .10. In the None test, where no reinforcers were delivered (right bottom panel of Figure 3), there was a significant main effect of session, F (1, 31) = 16.97, MSE = 3.39, p < .001, ηp2= .35, a significant main effect of group, F (1, 31) = 7.66, MSE = 8.65, p < .01, ηp2= .20, and a significant interaction between the two, F (1, 31) = 4.65, MSE = 3.29, p < .05, ηp2= .13. Pairwise comparisons found that while Group O2 showed a significant increase in responding from the last day of extinction to the None test, F (1, 31) = 19.12, p < .001, ηp2= .38, Group Ext did not, F (1, 31) = 1.98, p = .17. Additionally, animals in Group O2 responded significantly more than animals in Group Extinction during this test, F (1, 31) = 6.85, p = .01, ηp2= .18.

Discussion

After R-O1 training, rats in this experiment received either simple extinction (Group Ext) or extinction plus response-independent presentations of O2 (Group O2). The effect of presenting O2 in extinction was primarily to slow the loss of responding. However, in the tests that followed extinction, the main findings were (1.) that Group O2 showed an increase in responding when tested without pellets (and Group Ext did not), and (2.) that Group Ext showed an increase in responding when tested with O1 (whereas Group O2 showed less of this effect). These results are consistent with the idea that O1 and O2 control responding through their discriminative properties. In addition, other results suggest that there is some partial generalization between the pellets that served as O1 and O2 here. As noted above, noncontingent O2 in extinction slowed the loss of responding. It is already known that presentations of the same reinforcer slow response loss in extinction (Baker, 1990) and reduce the reinstating effect of that reinforcer (Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011). Such findings are often explained by acknowledging that the reinforcer sets the occasion for the next response (e.g. Reid, 1958; Ostlund & Balleine 2007); O2 might do so in the present experiment because it is not discriminated perfectly from O1. A second result suggesting generalization between O1 and O2 is that O2 caused a tendency (p = .07) to reinstate responding when it was presented noncontingently after extinction in Group Ext. Either result is consistent with the idea that there is some partial generalization between the two food pellets that provided O1 and O2. The results are not compatible with the idea that reinforcer presentations suppress or disrupt responding (e.g., Shahan & Sweeney, 2011). Indeed, in order to make the augmenting effects of noncontingent reinforcer presentations that we observed in both extinction (Group O2) and testing (Group Ext) more compatible with behavioral momentum theory, it would be necessary to implement a different model (Podlesnik & Shahan, 2010).

We would note that the response-enhancing effects of O1 and O2 presentations after extinction could be consistent with other views, e.g., that reinforcer presentations could increase the incentive motivation for operant responding (e.g., Stewart, de Wit, & Eikelboom, 1984). However, the challenge for any such view would be to explain simultaneously the response suppressing effects of O2 that were revealed during testing of Group O2 in the None condition, when O2 associated with extinction was removed, or in the basic resurgence design (e.g., Experiment 2). The simplest account of the overall pattern may be to accept a role for the discriminative properties of O2.

A somewhat surprising finding was that O1 did not cause as much recovery of R1 responding in Group O2 as it did in an analogous group (Group O1) in Experiment 2. In contrast to Experiment 2, in the present experiment O2 was delivered independently of an explicit response during R1 extinction, and the effects of O1, O2, and no reinforcer presentations were then tested within-subject. It is not clear why these differences would cause the discrepancy observed here. It is worth noting that O1 might have a neophobic effect when it is relatively unexpected; recall that the rats had been exposed to O2 but not O1 for the last eight days. The possibility that a relatively novel reinforcer can suppress behavior may also be consistent with the fact that O2 presentations initially appeared to suppress responding in Group O2 at the start of extinction, although that trend was not statistically significant.

General Discussion

The results of the present experiments provide new information that may be relevant for understanding resurgence. In Experiment 1, when R2 was reinforced during the response elimination phase with different VI schedules that varied systematically from VI 30-s to VI 120-s, the amount of resurgence was linearly related to reinforcement rate (and inversely related to the average interval between reinforcers). In Experiment 2, when R1 was reinforced with one reinforcer (O1) and R2 was reinforced with a different reinforcer (O2) during response elimination, resurgence was prevented when O2, but not O1, was presented noncontingently during testing. And in Experiment 3, the presence of O2 reinforcers during extinction reduced the ability of O1 to reinstate behavior above no reinforcer delivery, suggesting that there is some generalization between the O1 pellet context and the O2 pellet context. Overall, the results support the idea that in resurgence, R1 responding is influenced by the discriminative properties of reinforcers that are present during response elimination.

The results are generally consistent with the contextual account of resurgence (e.g., Bouton et al., 2012; Winterbauer & Bouton, 2010). That view accepts what may be the inherent context-specificity of extinction and suggests that resurgence might occur when the context created by reinforcement for R2 changes between response elimination and extinction testing. In Experiment 1, the approach predicted that leaner reinforcement schedules in the response elimination phase would allow the rat to learn R1 extinction under sparse reinforcement conditions that increasingly resemble the extinction conditions (context) of testing. In Experiment 2, it predicted that only noncontingent presentations of the reinforcer used during response elimination training (O2) would suppress R1 and hence reduce the resurgence effect. O1 was contrastingly ineffective because it had not been associated, as O2 was, with the extinction of R1. The results of Experiment 3 further suggest that there is some generalization between the O1 context and the O2 contexts, and less so to the “no pellet” (i.e., none) context. Overall, the results emphasize the role of the discriminative (rather than the reinforcing) properties of reinforcers in controlling extinguished R1 responding in the resurgence paradigm.

As we have noted throughout the article, the results are less consistent with the quantitative model proposed by Shahan and Sweeney (2011) that extended behavioral momentum theory. Although the model correctly predicts the results of Experiment 1, where the amount of resurgence was directly related to the rate of reinforcement for R2 during response elimination, it does not anticipate the results of Experiments 2 or 3. In Experiment 2, where noncontingent O2 but not O1 presentations during testing prevented resurgence, the model fell short because, in keeping with preceding versions of behavior momentum theory (e.g., Nevin & Grace, 2000), it does not discriminate between the effects of qualitatively different reinforcers (see Grimes & Shull, 2001; Shahan & Burke, 2004). In Experiment 3, reinforcers had an augmenting effect on operant responding, both when O2 was presented during the extinction phase in Group O2 and when O1 was presented during testing in Group Ext; neither finding is consistent with the disruption emphasized by the momentum-based model. It may be more parsimonious to accept that reinforcers have stimulus properties that can serve as discriminative cues controlling extinction or conditioned responding, and that such cues may play a role in causing resurgence during the transition between Phase 2 and testing. Without further expansion, the momentum-based resurgence model (Shahan & Sweeney, 2011) cannot implement such a mechanism.

How easy would it be to modify the momentum-based model to handle the results? Recall that the model emphasizes a release from disruption caused by reinforcers delivered at a high rate in Phase 2 in explaining resurgence. To accommodate the current results suggesting that the similarity between the stimulus conditions of Phase 2 and Phase 3 (testing) is important, the model could add a generalization parameter like the one it uses to describe performance in the transition between Phases 1 and 2. If such a parameter allowed the stimulus properties of the reinforcer to play a role (but see Grimes & Shull, 2001; Shahan & Burke, 2004), it could begin to explain resurgence the way the contextual account does (though only by accepting a new mechanism). But emphasizing generalization between Phases 2 and 3 would raise a new question: What is being generalized? The response suppression caused by reinforcement in Phase 2 is not learned—it is conceptualized as an unconditional disruption of performance. Why then should it generalize? Momentum theory gives no role to new learning in extinction, although new learning does occur (e.g., Bouton & Todd, 2014; Rescorla, 1993, 1997; Todd, Vurbic, & Bouton, 2014). Finally, as noted in the Introduction, other results have raised unresolved questions about how the model should calculate reinforcement rate in Phase 2 (Bouton & Schepers, 2014; see especially Schepers & Bouton, 2015). Several assumptions and aspects of the momentum-based model will thus need to be modified for it to provide a compelling account of resurgence.

Other aspects of the current results deserve further discussion. First, in Experiment 2, noncontingent presentations of O1 during testing did not reinstate R1 responding above the level seen in a group that received no reinforcers during testing, even though O1 had arguably been part of the context connected with R1’s original acquisition. This result was replicated in Experiment 3, where, using a simple extinction procedure, rats that were given free O2 pellets during extinction also failed to show reinstatement of responding by O1. At first glance, both results seem inconsistent with the context hypothesis, and indeed with studies of reinstatement, where presentations of reinforcers associated with conditioning readily reinstate extinguished responding. However, unlike simple extinction experiments, resurgence experiments involve reinforcer presentations during the sessions in which the target response (R1) is extinguished. Similarly presenting the reinforcer during simple extinction can abolish subsequent reinstatement (e.g., Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011) because it might undermine the reinforcer’s unique association with conditioning. Such a factor was potentially at work in the present experiments to reduce reinstatement by O1: As shown in Experiment 3, animals without O2 present during extinction (Group Ext) did show a significant reinstatement effect when tested with O1. In addition, it is worth noting that presentation of O1 during testing in Experiments 2 and 3 theoretically created a less complete or dramatic change of background context than the no-reinforcer conditions with which they were compared. This is because the common properties of O1 and O2 (they were both food pellets) would have decreased the contextual change from response elimination to testing relative to a group that received no pellets at all during testing. It thus may not be surprising to observe so little in the way of reinstatement-like results in Experiments 2 and 3 in animals given reinforcement during response elimination. The reinforcement conditions that prevail in extinction appear to play an especially important role in inhibiting R1 performance.

In summary, the present results provide further support for a role for the discriminative properties of the reinforcer in producing resurgence (see also Schepers & Bouton, 2015). When R1 is extinguished while R2 is being reinforced, the animal learns to inhibit R1 in the context of R2 reinforcement. When this context is changed, as it is when the R2 reinforcers are discontinued, or when the identity of the reinforcer is changed (Experiment 2), R1 may recover because extinction is relatively specific to the context in which it is learned. The results may have translational implication for reducing relapse after so-called contingency management (CM) treatments in which unhealthy behaviors such as smoking and drug taking are suppressed, in part, by the reinforcement of alternative behavior (e.g., Higgins, Heil, & Lussier, 2004; Petry, Martin, Cooney, & Kranzler, 2000). While CM seems to be effective at reducing the target behavior (Heil, Yoon, & Higgins, 2008; Petry et al., 2000), the behavior is susceptible to relapse when reinforcement is discontinued (Higgins, Silverman, & Heil, 2008; Silverman, Kaminski, Higgins, & Brady, 2011), perhaps resembling the resurgence effect. The present results, along with others, suggest that relapse may be prevented if one can make sure that the conditions of treatment generalize to those situations where relapse may occur. This is perhaps the most general implication of context-based accounts of extinction and voluntary response inhibition (e.g., Bouton, 2014).

Acknowledgments

This research was supported by NIH Grant RO1 DA 033123. We thank Scott Schepers and Eric Thrailkill for their comments.

References

  1. Baker AG. Contextual conditioning during free-operant extinction: Unsignaled, signaled, and backward-signaled nontongent food. Animal Learning & Behavior. 1990;18:59–70. [Google Scholar]
  2. Baker AG, Steinwald H, Bouton ME. Contextual conditioning and reinstatement of extinguished instrumental responding. The Quarterly Journal of Experiment Psychology Section B: Comparative and Physiological Psychology. 1991;43:199–218. [Google Scholar]
  3. Bouton ME. Why behavior change is difficult to sustain. Preventive Medicine. 2014;68:29–36. doi: 10.1016/j.ypmed.2014.06.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bouton ME, Rosengard C, Achenbach GG, Peck CA, Brooks DC. Effects of contextual conditioning and unconditional stimulus presentation on performance in appetitive conditioning. The Quarterly Journal of Experiment Psychology Section B: Comparative and Physiological Psychology. 1993;46:63–95. [PubMed] [Google Scholar]
  5. Bouton ME, Schepers ST. Resurgence of instrumental behavior after an abstinence contingency. Learning & Behavior. 2014;42:131–143. doi: 10.3758/s13420-013-0130-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bouton ME, Swartzentruber D. Sources of relapse after extinction in Pavlovian and Instrumental learning. Clinical Psychology Review. 1991;11:123–140. [Google Scholar]
  7. Bouton ME, Todd TP. A fundamental role for context in instrumental learning and extinction. Behavioural Processes. 2014;104:13–19. doi: 10.1016/j.beproc.2014.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bouton ME, Todd TP, Vurbic D, Winterbauer NE. Renewal after the extinction of free operant behavior. Learning & Behavior. 2011;39:57–67. doi: 10.3758/s13420-011-0018-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bouton ME, Winterbauer NE, Todd TP. Relapse processes after the extinction of instrumental learning: renewal, resurgence, and reacquisition. Behavioural Processes. 2012;90:130–141. doi: 10.1016/j.beproc.2012.03.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bouton ME, Winterbauer NE, Vurbic D. Context and extinction: Mechanisms of relapse in drug self-administration. In: Haselgrove M, Hogarth L, editors. Clinical applications of learning theory. East Sussex, UK: Psychology Press; 2012. pp. 103–134. [Google Scholar]
  11. Cançado CR, Lattal KA. Resurgence of temporal patterns of responding. Journal of the Experimental Analysis of Behavior. 2011;95:271–287. doi: 10.1901/jeab.2011.95-271. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Craig AR, Nevin JA, Odum AL. Behavioral momentum and resistance to change. In: McSweeney FK, Murphey ES, editors. The Wiley-Blackwell handbook of operant and classical conditioning. Oxford, UK: Wiley-Blackwell; 2014. pp. 249–274. [Google Scholar]
  13. Delamater AR. Selective reinstatement of stimulus-outcome associations. Animal Learning & Behavior. 1997;25:400–412. [Google Scholar]
  14. Field A. Discovering statistics using SPSS. Thousand Oaks, CA: Sage Publications; 2005. [Google Scholar]
  15. Franks GJ, Lattal KA. Antecedent reinforcement schedule training and operant response reinstatement in rats. Animal Learning & Behavior. 1976;4:374–378. [Google Scholar]
  16. Grimes JA, Shull RL. Response independent milk delivery enhances persistence of pellet-reinforced lever pressing by rats. Journal of the Experimental Analysis of Behavior. 2001;76:179–194. doi: 10.1901/jeab.2001.76-179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Heil SE, Yoon JH, Higgins ST. Pregnant and postpartum women. In: Higgins ST, Silverman S, Heil SH, editors. Contingency management in substance abuse treatment. New York, NY: Guilford Press; 2008. pp. 182–201. [Google Scholar]
  18. Higgins ST, Heil SH, Lussier JP. Clinical implications of reinforcement as a determinant of substance use disorders. Annual Review of Psychology. 2004;55:431–461. doi: 10.1146/annurev.psych.55.090902.142033. [DOI] [PubMed] [Google Scholar]
  19. Higgins ST, Silverman K, Heil SH, editors. Contingency management in substance abuse treatment. New York, NY: Guilford Press; 2008. [Google Scholar]
  20. Kincaid SL, Lattal KA, Spence J. Super-Resurgence: ABA Renewal Increases Resurgence. Behavioural Processes. 2015;115:70–73. doi: 10.1016/j.beproc.2015.02.013. [DOI] [PubMed] [Google Scholar]
  21. Leitenberg H, Rawson RA, Bath K. Reinforcement of competing behavior during extinction. Science. 1970;169:301–303. doi: 10.1126/science.169.3942.301. [DOI] [PubMed] [Google Scholar]
  22. Leitenberg H, Rawson RA, Mulick JA. Extinction and reinforcement of alternative behavior. Journal of Comparative and Physiological Psychology. 1975;88:640–652. [Google Scholar]
  23. Lieving GA, Lattal KA. Recent, repeatability, and reinforcer retrenchment: An experimental analysis of resurgence. Journal of the Experimental Analysis of Behavior. 2003;80:217–233. doi: 10.1901/jeab.2003.80-217. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Nevin JA, Grace RC. Behavioral momentum and the law of effect. Behavioral and Brain Sciences. 2000;23:73–130. doi: 10.1017/s0140525x00002405. [DOI] [PubMed] [Google Scholar]
  25. Ostlund SB, Balleine BW. Selective reinstatement of instrumental performance depends on the discriminative stimulus properties of the mediating outcome. Learning & Behavior. 2007;35:43–52. doi: 10.3758/bf03196073. [DOI] [PubMed] [Google Scholar]
  26. Petry NM, Martin B, Cooney JL, Kranzler HR. Give them prizes, and they will come: Contingency management for treatment of alcohol dependence. Journal of Counseling and Clinical Psychology. 2000;68:250–257. doi: 10.1037//0022-006x.68.2.250. [DOI] [PubMed] [Google Scholar]
  27. Podlesnik CA, Shahan TA. Extinction, relapse, and behavioral momentum. Behavioural Processes. 2010;84:400–411. doi: 10.1016/j.beproc.2010.02.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Reid RL. The role of the reinforcer as a stimulus. British Journal of Psychology. 1958;49:202–209. doi: 10.1111/j.2044-8295.1958.tb00658.x. [DOI] [PubMed] [Google Scholar]
  29. Rescorla RA. Inhibitory associations between S and R in extinction. Animal Learning & Behavior. 1993;21:327–336. [Google Scholar]
  30. Rescorla RA. Response inhibition in extinction. The Quarterly Journal of Experimental Psychology Section B: Comparative and Physiological Psychology. 1997;50:238–252. [Google Scholar]
  31. Rescorla RA, Skucy JC. Effect of response-independent reinforcers during extinction. Journal of Comparative and Physiological Psychology. 1969;67:381–389. [Google Scholar]
  32. Schepers ST, Bouton ME. Effects of reinforcer distribution during response elimination on resurgence of an instrumental behavior. Journal of Experimental Psychology: Animal Learning and Cognition. 2015;41:179–192. doi: 10.1037/xan0000061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Sheffield VF. Extinction as a function of partial reinforcementand distribution of practice. Journal of Experimental Psychology. 1949;39:511–526. doi: 10.1037/h0057242. [DOI] [PubMed] [Google Scholar]
  34. Silverman K, Kaminski BJ, Higgins ST, Brady JV. Behavior analysis and treatment of drug addiction. In: Fisher WW, Piazza CC, Roane HS, editors. Handbook of applied behavior analysis. New York, NY: Guilford Press; 2011. pp. 451–471. [Google Scholar]
  35. Shahan TA, Burke KA. Ethanol-maintained responding of rats is more resistant to change in a context with added non-drug reinforcement. Behavioural Pharmacology. 2004;15:279–285. doi: 10.1097/01.fbp.0000135706.93950.1a. [DOI] [PubMed] [Google Scholar]
  36. Shahan TA, Sweeney MM. A model of resurgence based on behavioral momentum theory. Journal of the Experimental Analysis of Behavior. 2011;95:91–108. doi: 10.1901/jeab.2011.95-91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Stewart J, de Wit H, Eikelboom R. Role of unconditioned and conditioned drugs effects in the self-administration of opiates and stimulants. Psychological Review. 1984;91:251–268. [PubMed] [Google Scholar]
  38. Sweeney MM, Shahan TA. Effects of high, low, and thinning rates of alternative reinforcement on response elimination and resurgence. Journal of the Experimental Analysis of Behavior. 2013a;100:102–116. doi: 10.1002/jeab.26. [DOI] [PubMed] [Google Scholar]
  39. Sweeney MM, Shahan TA. Behavioral momentum and resurgence: Effects of time in extinction and repeated resurgence tests. Learning & Behavior. 2013b;41:414–424. doi: 10.3758/s13420-013-0116-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Todd TP, Vurbic D, Bouton ME. Mechanisms of renewal after the extinction of discriminated operant behavior. Journal of Experimental Psychology: Animal Learning and Cognition. 2014;40:355–368. doi: 10.1037/xan0000021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Winterbauer NE, Bouton ME. Mechanisms of resurgence of an extinguished instrumental behavior. Journal of Experimental Psychology: Animal Behavior Processes. 2010;36:343–353. doi: 10.1037/a0017365. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Winterbauer NE, Bouton ME. Mechanisms of resurgence II: Response-contingent reinforcers can reinstate a second extinguished behavior. Learning and Motivation. 2011;42:154–164. doi: 10.1016/j.lmot.2011.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Winterbauer NE, Bouton ME. Effects of thinning the rate at which alternative behavior is reinforced on resurgence of an extinguished instrumental response. Journal of Experimental Psychology: Animal Behavior Processes. 2012;38:279–291. doi: 10.1037/a0028853. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES