Resurgence Following Higher or Lower Quality Alternative Reinforcement

Abstract

Resurgence is a temporary increase in a previously suppressed target behavior-following a worsening in reinforcement conditions. Previous studies have examined how higher rates or magnitudes of alternative reinforcement affect suppression of the target behavior and subsequent resurgence. However, there has been no investigation of the effects of higher versus lower qualities of alternative reinforcement on resurgence. Using a three-phase resurgence preparation with rats, the present experiments examined the effects of an alternative reinforcer that was of higher (Experiment 1) or lower (Experiment 2) quality than the reinforcer that had previously maintained the target behavior. The results of both experiments showed greater reductions in target behavior with a higher quality alternative reinforcer and larger increases in target responding when a higher quality alternative reinforcer was removed. Along with prior findings with higher rates and magnitudes of alternative reinforcement, these findings suggest that variations in reinforcer dimensions that increase the efficacy of alternative reinforcement also tend to increase resurgence when alternative reinforcement is removed. The results are discussed in terms of the resurgence as choice in context model and in terms of potential clinical implications.

Resurgence refers to a temporary increase in a previously suppressed operant behavior resulting from a relative worsening of reinforcement conditions (Lattal & Wacker, 2015; Shahan & Craig, 2017). In a typical experiment examining resurgence, reinforcers are delivered for a target behavior in Phase 1 (e.g., left lever press). In Phase 2, reinforcers for the target behavior are withheld (i.e., extinction) and instead delivered for an alternative response (e.g., right lever press). As a result, the target behavior typically decreases to low levels. In a final Phase 3 (i.e., resurgence test), reinforcers for the alternative behavior are also withheld, and the target behavior typically increases (i.e., resurgence).

There has been considerable recent interest in resurgence because of the apparent relevance of the phenomenon for the relapse of undesirable behavior following interventions based on differential reinforcement of alternative behavior (i.e., DRA; see Tiger et al., 2008; Petscher et al., 2009). In such interventions, the reinforcer maintaining a problem behavior is typically withheld and an alternative, appropriate behavior is reinforced instead. Although such interventions are highly effective at reducing problem behavior while in effect, problem behavior often increases when alternative reinforcement is not delivered or is reduced (e.g., Briggs et al., 2018; Falligant et al., 2022; Fuhrman et al., 2016; Haney et al., 2022; Lieving et al., 2004; Mitteer et al., 2022; Muething et al., 2021; Volkert et al., 2009; Wacker et al., 2011, 2013).

Previous research on resurgence has suggested that dimensions of the alternative reinforcer can affect both the efficacy of target suppression during Phase 2 and resurgence of target responding generated by the removal of the alternative reinforcer. With both nonhumans and humans, high-rate alternative reinforcement tends to generate greater suppression of target behavior than low-rate or no alternative reinforcement but removal of high-rate alternative reinforcement generates larger increases in target behavior than removal of low-rate alternative reinforcement (e.g., Bouton & Trask, 2015; Cançado et al., 2015; Craig & Shahan, 2016; Craig et al., 2016; Leitenberg et al., 1975; Nevin et al., 2016; Podlesnik et al., 2022; Pritchard et al., 2014; Smith et al., 2017; Sweeney & Shahan, 2013).¹ Similarly, with rats, larger magnitudes of alternative reinforcement produce more suppression of target responding in Phase 2 than smaller magnitudes, but removal of larger magnitudes generates larger increases in target responding in Phase 3 than removal of smaller magnitudes (Browning et al., 2022; Craig et al., 2017).²

Although the effects of both the rate and magnitude of alternative reinforcement on resurgence have been examined experimentally, little is known about how qualitative differences in alternative reinforcers affect resurgence. An increased understanding of any such effects could be of translational importance. A general finding in the clinic is that DRA is more effective at reducing aggression, self-injury, and other forms of problem behavior when the alternative response leads to a higher quality reinforcer (Horner & Day, 1991; Piazza et al., 1997; Van Camp et al., 2001, Weinsztok & DeLeon, 2022), and this approach is especially important when treatment procedures do not or cannot arrange extinction for problem behavior. Indeed, manipulating reinforcer quality is common across treatments without extinction when problem behavior is maintained by social forms of reinforcement (e.g., escape, attention, tangibles; Athens & Vollmer, 2010; Briggs et al., 2019) and also for treating automatically reinforced problem behavior when the reinforcing stimulus cannot be readily identified or withheld (e.g., Van Camp et al., 2001).

Some previous studies of resurgence with rats have demonstrated that removal of alternative reinforcement that is qualitatively different from the previous reinforcer for target behavior can generate resurgence. For example, Podlesnik et al. (2006) showed that removal of food reinforcement introduced during Phase 2 generated resurgence of extinguished target responding previously maintained by an alcohol solution (see also Nall et al., 2018). Similar effects have been noted following target responding previously maintained by cocaine infusions (e.g., Craig et al., 2016; Nall et al., 2018; Quick et al., 2011; Shahan et al., 2015). In addition, some studies have used different types of food pellets (e.g., grain versus sucrose) for target responding in Phase 1 and alternative responding in Phase 2 (e.g., Bouton & Trask, 2016; Winterbauer et al., 2013). However, none of these previous studies has examined how qualitatively different reinforcers of higher or lower value than the reinforcer for target behavior affect suppression in Phase 2 and resurgence in Phase 3. In the alcohol and cocaine studies, the drug reinforcer always served as the target reinforcer and food as the alternative reinforcer, and no assessment was ever made of the relative reinforcing value of the different reinforcers. Moreover, although Winter-bauer et al. (2013) examined resurgence generated by alternative reinforcers that were the same as or different from the target reinforcer, the two types of reinforcers appeared to be roughly equally valued, as they maintained similar rates of target behavior when available in Phase 1.

Thus, the present experiments examined the effects on resurgence of a qualitatively different alternative reinforcer that was of higher value (Experiment 1) or lower value (Experiment 2) than the reinforcer previously maintaining target behavior. To verify that we would be arranging qualitatively different reinforcers with the desired difference in reinforcing value, we first conducted a pilot study in which a group of 10 male Long-Evans rats chose between deliveries of single 45-mg grain-based food pellets (Bio Serv, Flemington, NJ) and 0.06-mL dippers of a 5% sucrose solution on concurrent variable-interval (VI) 30-s, VI 30-s schedules for 10 sessions. In addition, responding of the same rats was similarly examined when each reinforcer type was available alone on a single VI 30-s schedule. Figure 1 shows that in the last five sessions of exposure to these conditions, food-pellet-maintained response rates were roughly twofold (concurrent) to threefold (alone) higher than those maintained by deliveries of 5% sucrose solution (this was true for 10 of 10 rats in both conditions). These outcomes demonstrate that food pellets were a higher quality reinforcer than the 5% sucrose solution.

FIGURE 1.

Pilot data on reinforcer quality of food pellets and sucrose solution. Responses rates of 10 rats in a pilot experiment in which one lever produced food pellets and another produced dippers of a 5% sucrose solution, both on VI 30-s schedules. In separate conditions the levers were available either concurrently (left panel) or alone (right panel). Error bars reflect ± SEM.

EXPERIMENT 1

To investigate the effects of an alternative reinforcer that was the same as or of a higher quality than the target reinforcer, two groups of rats responded for 5% sucrose solution deliveries in Phase 1. In Phase 2, the two groups differed in terms of whether the same 5% sucrose solution or food pellets were available for an alternative behavior while the target behavior was extinguished. In Phase 3, the reinforcer for the alternative behavior was also removed.

Method

Subjects

Eighteen experimentally naïve male Long-Evans rats (Charles River, Portage, MI) served as subjects. The original sample size was 20, but one subject was removed for failing to acquire lever-pressing for sucrose during Phase 1, and an additional subject was removed due to failure of a liquid dipper during Phase 2. The rats were approximately 70–90 days old at the beginning of the experiment and were individually housed in a humidity- and temperature-controlled colony room on a 12 hr light: 12 hr dark cycle (lights on at 0700 hours). They were provided with ad libitum access to water in their home cages and were maintained at 80% of their free-feeding weights by supplemental postsession feeding. All experimental procedures were conducted in accordance with the university Institutional Animal Care and Use Committee.

Apparatus

Ten identical Med Associates operant chambers (30 × 24 × 21 cm) were used. They were housed in sound- and light-attenuating cubicles and included work panels on the right- and left-side walls, with a clear Plexiglas ceiling, front door, and back wall. On top of the left-side wall, a centered houselight provided general illumination. On the right-side wall, two retractable levers with stimulus lights above them were positioned on either side of a split receptacle that could dispense both food (45-mg grain-based food pellets; Bio Serv, Flemington, NJ) and liquid reinforcers (5% sucrose solution presented in 0.06 mL dipper cups). A computer running Med-PC IV software controlled all experimental events and data collection from an adjacent control room.

Procedure

Training

Experimental sessions were conducted 7 days per week at approximately the same time each day. All sessions lasted 30 min, excluding reinforcement deliveries, during which all experimental timers paused. Four sessions of pellet-magazine training and four sessions of liquid-dipper training were conducted in alternating order for a total of eight training sessions. During these sessions, all chamber lights were off and the levers were retracted. Reinforcers were delivered on a variable-time (VT) 60-s schedule using 10 intervals derived from the Fleshler and Hoffman (1962) distribution (as were all VI schedules below), and the split reinforcer receptacle was illuminated for 3 s for pellet deliveries or for the duration of the dipper presentation for liquid-dipper training. In the first session of liquid-dipper training, dippers were presented until the subject made a head entry, then lowered after 5 s. In the latter three sessions of dipper training, dippers were presented for 5 s and then lowered.

Phase 1: Baseline

This phase began the day following the last magazine-training session. No explicit lever-press training was required. The reinforcer for all rats was a 3-s delivery of a 5% sucrose solution presented in a 0.06-mL dipper. Sessions began with insertion of the target lever (left–right counterbalanced across rats) and illumination of the houselight and the stimulus light above the target lever. The first lever press on the target lever during the first session produced a sucrose delivery. During the remainder of the first session and during the second session, responses were reinforced on a VI 10-s schedule. During the third session, responses were reinforced on a VI 20-s schedule. Beginning with the fourth session, responses were reinforced on a VI 30-s schedule for the remainder of the phase. Phase 1 lasted 30 sessions for all subjects.

Phase 2: Treatment

At the end of Phase 1, rats were assigned to one of two groups: sucrose–sucrose (n = 8) and sucrose-pellet (n = 10) such that target response rates during the last 3 days of Phase 1 were similar between groups. Phase 2 sessions began with insertion of both levers and illumination of the houselight and of both lever stimulus lights. Target responding for all rats was placed on extinction, and presses of the alternative lever produced the same sucrose deliveries as in Phase 1 for the sucrose–sucrose group but single food-pellet deliveries for the sucrose-pellet group. The first response to the alternative lever during the first session of this phase produced a delivery of the assigned reinforcer. Alternative responses during the rest of the first session and in all subsequent sessions of the phase were reinforced on a VI 10-s schedule. A 3-s changeover delay was arranged between target and alternative responding such that the alternative response would not produce an arranged reinforcer for 3 s following a target response. This phase lasted 10 sessions for all rats.

Phase 3: Resurgence test

Alternative responding for all rats was also placed on extinction. Target responding remained on extinction. All other stimulus conditions remained as in Phase 2. This phase lasted seven sessions.

Data analysis

All analyses were conducted in R version 4.2.1 using the rstatix package. All statistical tests were conducted two-tailed with α = .05. Greenhouse–Geiser corrections were applied when violations of sphericity were detected for repeated measures in analysis of variance (ANOVA). Data for one rat in the sucrose–sucrose group were lost for Session 8 of Phase 2, as the rat escaped from the operant chamber during the session. Given the missing data for this session, this rat was excluded only from the statistical analyses of Phase 2 responding across sessions. This did not affect any analyses involving only the final session of Phase 2.

Results and discussion

The top-left panel of Figure 2 shows target response rates in the final three sessions of Phase 1 and across all sessions of Phase 2. As arranged by the matching of groups in Phase 1, mean response rates during the final three sessions of the phase did not differ for the sucrose–sucrose and sucrose-pellet groups, t(16) = −0.11, p = .912, d = −0.05. In Phase 2, target response rates for both groups decreased across sessions, with response rates slightly but reliably lower overall for the sucrose-pellet group that received a higher quality alternative reinforcer. The inset panel shows the same Phase 2 data plotted on a logarithmic y-axis in which this difference in response rates is more apparent. A 2 × 10 (Group × Session) mixed-model ANOVA conducted on target response rates across all sessions of Phase 2 revealed a significant main effect of group, F(1, 15) = 9.18, p = .008, ${\eta }_G^2=.16$, and session, F(1.78, 26.74) = 29.13, p < .001, ${\eta }_G^2=.57$, but no significant Group x Session interaction, F(1.78, 26.74) = 3.19, p = .063, ${\eta }_G^2=.13$.

FIGURE 2.

Mean target and alternative response rates from Experiment 1. The top-left panel shows target response rates in the final three sessions of Phase 1 and in all sessions of Phase 2 for Experiment 1. The inset panel shows the same Phase 2 data plotted with a logarithmic y-axis. The top-right panel shows target response rates in the last session of Phase 2 and in all sessions of Phase 3 (resurgence test). The bottom-left panel shows alternative response rates across all sessions of Phase 2. The bottom-right panel shows alternative response rates in the last session of Phase 2 and all sessions of Phase 3. Error bars reflect ± SEM.

The bottom-left panel of Figure 2 shows alternative response rates across Phase 2. Alternative response rates of both groups increased across sessions, but they increased more so for the sucrose-pellet group. A 2 × 10 (Group × Session) mixed-model ANOVA conducted on alternative response rates across Phase 2 revealed a significant main effect of session, F(2.9, 43.53) = 25.15, p < .001, ${\eta }_G^2=.23$, and a Group × Session interaction, F (2.9, 43.53) = 3.87, p = .016, ${\eta }_G^2=.04$, but no significant main effect of group, F(1, 15) = 4.03, p = .063, ${\eta }_G^2=.18$.

The top-right panel of Figure 2 shows target response rates in the last session of Phase 2 and across all sessions of the Phase 3 resurgence test for both groups. Response rates did not differ significantly for the two groups in the last session of Phase 2, t(16) = −1.53 p = .146, d = −.72. In the first session of Phase 3 when alternative reinforcement was removed, response rates for both groups increased relative to the last session of Phase 2, sucrose–sucrose: t(7) = −3.09, p = .018, d = −1.09; sucrose-pellet: t(9) = −4.24, p = .002, d = −1.34, but the increase was larger for the sucrose-pellet group than for the sucrose–sucrose group. A 2 × 2 (Group × Phase) mixed-model ANOVA revealed a main effect of phase, F (1, 16) = 20.52, p < .001, ${\eta }_G^2=.36$, and Group × Phase interaction, F(1, 16) = 7.68, p = .014, ${\eta }_G^2=.17$, but no main effect of group, F(1, 16) = 3.22, p = .092, ${\eta }_G^2=\mathrm{.10.}$ In addition, response rates in the first session of Phase 3 were significantly higher for the sucrose-pellet group than for the sucrose–sucrose group, t(16) = 2.30, p = .035, d = 1.15. A 2 × 7 (Group × Session) mixed-model ANOVA conducted on target response rates across all sessions of Phase 3 revealed no significant main effects of group, F(1, 16) = 2.19, p = .158, ${\eta }_G^2=.10$, or session, F(2.45, 39.25) = 2.15, p = .121, ${\eta }_G^2=.03$, and a marginal Group × Session interaction, F(2.45, 39.25) = 2.86, p = .059, ${\eta }_G^2=.04$, resulting from the fact that the initial difference in response rates for the two groups decreased across further exposure to extinction.

The bottom-right panel of Figure 2 shows alternative response rates in the last session of Phase 2 and all sessions of the Phase 3 resurgence test for both groups. Alternative response rates decreased for both groups across sessions, with the difference in response rates for the two groups decreasing with increasing sessions of extinction. A 2 × 7 (Group × Session) mixed-model ANOVA conducted on alternative response rates across all sessions of Phase 3 revealed a significant main effect of session, F(1.38, 22.06) = 63.66, p < .001, ${\eta }_G^2=.70$, a Group Session interaction, F(1.38, 22.06) = 5.79, p = .017, ${\eta }_G^2=.17$, and a marginal main effect of group, F(1, 16) = 4.31, p = .054, ${\eta }_G^2=.10$.

The top panels of Figure 3 show target response rates for individual rats in the sucrose–sucrose group in the final three sessions of Phase 1 and all sessions of Phase 2 (left panel) and for the last session of Phase 2 and all sessions of Phase 3 (right panel). The bottom panels show data presented similarly for individual rats in the sucrose-pellet group. As was apparent with the group mean data in Figure 2, rats in the sucrose-pellet group showed greater suppression of target behavior in Phase 2 and a larger increase in target responding in Phase 3 when alternative reinforcement was removed.

FIGURE 3.

Target response rates for individual rats from Experiment 1. The left column shows responding in the final three sessions of Phase 1 (i.e., P1) and across all sessions of Phase 2 (i.e., P2 Sessions). The right column shows responding in the last session of Phase 2 (i.e., Last P2) and across all sessions of the resurgence test in Phase 3 (i.e., P3 Sessions). Data are presented separately for the sucrose–sucrose and sucrose-pellet groups in the top and bottom rows, respectively.

In summary, the present experiment found that delivering a higher quality reinforcer (i.e., food pellets) than the reinforcer that had previously maintained the target behavior (i.e., 5% sucrose solution) produced slightly greater suppression of target responding during Phase 2 than delivering the same quality of reinforcer that had maintained the target behavior. Nevertheless, removal of that higher quality reinforcer in Phase 3 produced a larger temporary increase in target responding (i.e., resurgence) than removal of the same-quality reinforcer.

EXPERIMENT 2

This experiment examined the effects of an alternative reinforcer that was the same as or of lower quality than the target reinforcer on suppression and resurgence of a target response. Thus, two groups of rats responded for food-pellet deliveries in Phase 1 and either the same food pellets or a 5% sucrose solution for the alternative behavior in Phase 2. In Phase 3, the reinforcer for the alternative behavior was also removed for both groups.

Method

Subjects

Eighteen additional experimentally naïve male Long-Evans rats (Charles River, Portage, MI) served as subjects. The original sample size was 20, but two rats were removed in Phase 1 due to pellet-dispenser failures. All other details of the subjects and their care were as in Experiment 1.

Apparatus

The same apparatus was used as in Experiment 1.

Procedure

Training

As in Experiment 1.

Phase 1: Baseline

All details were as in Experiment 1, except that the reinforcer for all rats consisted of a single 45 mg food pellet.

Phase 2: Treatment

At the end of Phase 1, rats were assigned to one of two groups: pellet-pellet (n = 8) and pellet-sucrose (n = 10) such that target response rates during the last three sessions of Phase 1 were similar between groups. Target responding for all rats was placed on extinction, and presses of the alternative lever produced the same 45-mg food-pellet deliveries as Phase 1 for the pellet-pellet group but dippers of a 5% sucrose solution for the pellet-sucrose group. All other details were as in Experiment 1.

Phase 3: Resurgence test

As in Experiment 1, alternative responding for all rats was also placed on extinction and target responding remained on extinction. All other stimulus conditions remained as in Phase 2.

Data analysis

As in Experiment 1.

Results and discussion

The top-left panel of Figure 4 shows target response rates in the final three sessions of Phase 1 and across all sessions of Phase 2. As arranged by the matching of groups in Phase 1, mean response rates did not differ for the pellet-pellet and pellet-sucrose groups, t(16) = .26, p = .796, d = .12. In Phase 2, target response rates for both groups decreased across sessions, with response rates being considerably less suppressed for the pellet-sucrose group that received a lower quality alternative reinforcer. The inset panel shows the same Phase 2 data plotted on a logarithmic y-axis. A 2 × 10 (Group × Session) mixed-model ANOVA conducted on target response rates across Phase 2 revealed a significant main effect of group, F(1, 16) = 56.99, p < .001, ${\eta }_G^2=.60$, and session, F(2.57, 41.14) = 49.73, p < .001, ${\eta }_G^2=.64$, and a significant Group × Session interaction, F(2.57, 41.14) = 38.60, p < .001, ${\eta }_G^2=.58$.

FIGURE 4.

Mean target and alternative response rates from Experiment 2. The top-left panel shows target response rates in the final three sessions of Phase 1 and in all sessions of Phase 2 for Experiment 2. The inset panel shows the same Phase 2 data plotted with a logarithmic y-axis. The top-right panel shows target response rates in the last session of Phase 2 and in all sessions of Phase 3 (resurgence test). The bottom-left panel shows alternative response rates across all sessions of Phase 2. The bottom-right panel shows alternative response rates in the last session of Phase 2 and all sessions of Phase 3. Error bars reflect ± SEM.

The bottom-left panel of Figure 4 shows alternative response rates across Phase 2. Alternative response rates were higher for the pellet-pellet group than for the pellet-sucrose group, and the responding of both groups increased across sessions. A 2 × 10 (Group × Session) mixed-model ANOVA conducted on alternative response rates across Phase 2 revealed a significant main effect of group, F(1, 16) = 59.10, p < .001, ${\eta }_G^2=.72$, and session, F(3.24, 51.78) = 14.98, p < .001, ${\eta }_G^2=.21$, but no significant Group × Session interaction, F(3.24, 51.78) = 0.84, p = .488, ${\eta }_G^2=.02$.

The top-right panel of Figure 4 shows target response rates in the last session of Phase 2 and across all sessions of Phase 3 for both groups. Response rates were significantly higher for the pellet-sucrose than for the pellet-pellet group in the last session of Phase 2, t(16) = −3.91, p = .001, d = −1.85. In the first session of Phase 3 when alternative reinforcement was removed, response rates for both groups increased relative to those observed in the last session of Phase 2, pellet-pellet: t(7) = −5.72, p < .001, d = −2.02; pellet-sucrose: t(9) = −3.09, p = .013, d = −0.98. Response rates for the pellet-pellet group increased more than did those for the pellet-sucrose group with the removal of alternative reinforcement. A 2 × 2 (Group × Phase) mixed-model ANOVA revealed a significant main effect of phase, F(1, 16) = 39.75, p < .001, ${\eta }_G^2=.50$, and a Group × Phase interaction, F(1, 16) = 4.63, p = .047, ${\eta }_G^2=.10$, but no main effect of group, F(1, 16) = 0.48, p = .498, ${\eta }_G^2=.02$. Although response rates did not differ for the two groups in the first session of Phase 3, t(16) = 0.62, p = .546, d = 0.29, average response rates across all Phase 3 sessions were significantly higher for the pellet-pellet group. A 2 × 7 (Group × Session) mixed-model ANOVA conducted on target response rates across all sessions of Phase 3 revealed a significant main effect of group, F (1, 16) = 5.02, p = .040, ${\eta }_G^2=.13$, and session, F(3.29, 52.66) = 10.59, p < .001, ${\eta }_G^2=.25$, but no Group × Session interaction, F(3.29, 52.66) = 0.34, p = .812, ${\eta }_G^2=.01$.

The bottom-right panel of Figure 4 shows alternative response rates in the last session of Phase 2 and all sessions of Phase 3 for both groups. Alternative response rates decreased for both groups across sessions of Phase 3, with the difference in response rates for the two groups decreasing with increasing sessions of extinction. A 2 × 7 (Group × Session) mixed-model ANOVA conducted on alternative response rates across all sessions of Phase 3 revealed a significant main effect of group, F(1, 16) = 14.26, p = .002, ${\eta }_G^2=.31$, and session, F(1.66, 26.62) = 57.53, p < .001, ${\eta }_G^2=.65$, and a significant Group × Session interaction, F(1.66, 26.62) = 10.52, p < .001, ${\eta }_G^2=.25$.

The top panels of Figure 5 show target response rates for individual rats in the pellet-pellet group in the final three sessions of Phase 1 and all sessions of Phase 2 (left panel) and for the last session of Phase 2 and all sessions of Phase 3 (right panel). The bottom panels show data presented similarly for individual rats in the pellet-sucrose group. As was apparent with the group mean data in Figure 4, rats in the pellet-pellet group showed considerably greater suppression of target behavior in Phase 2 and tended to show higher target response rates across Phase 3 when alternative reinforcement was removed.

FIGURE 5.

Target response rates for individual rats from Experiment 2. The left column shows responding in the final three sessions of Phase 1 (i.e., P1) and across all sessions of Phase 2 (i.e., P2 Sessions). The right column shows responding in the last session of Phase 2 (i.e., Last P2) and across all sessions of the resurgence test in Phase 3 (i.e., P3 Sessions). Data are presented separately for the pellet-pellet and pellet-sucrose groups in the top and bottom rows, respectively.

In summary, the present experiment found that delivering a lower quality reinforcer (i.e., 5% sucrose solution) than had previously maintained the target behavior (i.e., food pellets) produced less suppression of target responding during Phase 2 than delivering the same higher quality reinforcer that had previously maintained the target behavior. Subsequent removal of that lower quality alternative reinforcer in Phase 3 produced a smaller initial increase and lower overall target response rates across continued exposure to extinction than did removal of the same higher quality reinforcer as had maintained the target behavior.

GENERAL DISCUSSION

Using a typical three-phase resurgence preparation, the present experiments examined the effects of an alternative reinforcer that was of higher (Experiment 1) or lower (Experiment 2) quality than the reinforcer that had previously maintained the target behavior. The results of both experiments suggest that a lower quality alternative reinforcer resulted in less suppression of target behavior in Phase 2 than did a higher quality reinforcer—although this effect appeared to be more pronounced when target response rates were higher as a result of previously producing the higher quality reinforcer in Phase 1 (i.e., in Experiment 2). In addition, Experiment 1 suggested that the removal of a higher quality alternative reinforcer in Phase 3 generated a larger temporary increase in target responding than did removal of the lower quality reinforcer that had previously maintained the target behavior. In Experiment 2, the removal of a lower quality alternative reinforcer than had maintained the target behavior produced a smaller initial increase and lower average target response rates across continued sessions of extinction in Phase 3 than did removal of a higher quality alternative reinforcer. Thus, both experiments suggest (1) greater reductions in target behavior with a higher quality alternative reinforcer and (2) more resurgence of target responding when a higher quality alternative reinforcer is removed. These outcomes are consistent with previously noted effects of higher rates and magnitudes of alternative reinforcement on suppression of target behavior in Phase 2 and resurgence in Phase 3 (e.g., Craig & Shahan, 2016; Craig et al., 2017). Together, this constellation of findings suggests that variations in reinforcer dimensions that increase the efficacy of alternative reinforcement also tend to increase resurgence when alternative reinforcement is removed. This outcome is generally consistent with the theoretical account of resurgence provided by resurgence as choice in context theory (i.e., RaC²; Shahan et al., 2020).

Resurgence as choice in context is an extension of the matching law (Herrnstein, 1961, 1970; Baum & Rachlin, 1969) and suggests that resurgence is a natural outcome of the processes that govern choice more generally. The theory characterizes target and alternative behaviors as two response options and calculates values of those options as longer-term repositories of their productivity over time and across varied reinforcement conditions, including extinction conditions. Resurgence results from an increase in the relative value of the target behavior following a worsening of conditions for the alternative behavior (e.g., alternative reinforcement removed or reduced). Moreover, incorporating some aspects of context theory (Bouton et al., 2012; Winterbauer & Bouton, 2010), RaC² also suggests that more local signaling effects of reinforcer deliveries or their absence influence the rates of target and alternative behaviors beyond the longer-term effects of changes in value across conditions.

Quantitatively, RaC² calculates the values of target and alternative options using a scaled version (see Shahan & Craig, 2017) of the temporal weighting rule (e.g., Devenport and Devenport, 1994; Devenport et al., 1997). The scaled temporal weighting rule provides a series of weightings (i.e., w_x) of previous experiences based on the relative recency of those experiences such that

$${\text{w}}_x=\frac{1/t_x^c}{\sum _{i=1}^{n}1/t_i^c},$$ (1)

where w_x is the weight applied to a session in the past and t_x is the time (i.e., number of sessions plus 1) between that session and the session of interest. Equation 1 generates a series of weightings that decreases hyperbolically into the past, with higher values of the scaling exponent c (i.e., >1) generating weightings that decrease relatively more steeply with the passage of time. The scaling exponent is calculated as

$$c=\lambda r+1,$$ (2)

where r is the average running reinforcement rate for a particular response across sessions and λ is a free parameter reflecting sensitivity to running reinforcement rate (see Shahan & Craig, 2017, for details).

To calculate the values of the two options, the appropriate series of weightings is applied to the reinforcement rates (i.e., R_x) experienced for each option in a series of past sessions under consideration and summed across those sessions such that

$$V=\sum _x{w}_x{R}_x.$$ (3)

The separate values obtained for the target (i.e., V_T) and alternative (i.e., V_Alt) options are then used to calculate absolute response rates (in responses/min) for the target (i.e., B_T) and alternative (i.e., B_Alt) behaviors:

$${\text{B}}_{\text{T}}=\frac{k{\text{V}}_{\text{T}}}{{\text{V}}_{\text{T}}+d_1\left({\text{V}}_{\text{Alt}}\right)+d_0\left(\frac{1}{A}\right)},$$ (4)

and

$${\text{B}}_{\text{Alt}}=\frac{k{d}_1\left({\text{V}}_{\text{Alt}}\right)}{{\text{V}}_{\text{T}}+d_1\left({\text{V}}_{\text{Alt}}\right)+d_0\left(\frac{1}{A}\right)},$$ (5)

where k is a free parameter scaling values into response rates and A represents the invigorating effects of the current values of the options:

$$A=a\left(V_T+V_{Alt}\right),$$ (6)

with a as a free parameter scaling value to invigoration. To incorporate more local signaling effects of reinforcement of the alternative behavior, d₁ reflects the biasing effects (toward the alternative behavior) of discriminating that the presence of alternative reinforcement signals the local absence of reinforcement for the target behavior and d₀ reflects similar biasing effects (toward not responding) of learning to discriminate that reinforcement is not available for either option. Acquisition of these discriminations about reinforcer availability or unavailability is assumed to occur as a result of exposure to the relevant conditions using a simplified version of the Gallistel et al. (2004) learning curve such that

$$d_1=d_m\left(1-e^{-x_{on}}\right),$$ (7)

and

$$d_0=d_m\left(1-e^{-x_{\mathit{\text{off}}}}\right),$$ (8)

where d_m is a free parameter for the shared asymptotic value of d₁ and d₀ and x_on and x_off represent sessions of exposure to alternative reinforcement present (i.e., on) or absent (i.e., off).

With respect to the present experiments, although qualitative differences in reinforcement are difficult to quantify, the matching law provides a preliminary means of doing so via the bias parameter (i.e., b; see Baum, 1974; Miller, 1976). As noted by Shahan and Craig (2017), because RaC² is based on the matching law, the effects of qualitatively different reinforcers can also be incorporated as a source of bias. Although the absolute values of two qualitatively different reinforcers may be unknown, the ratio of their values can be estimated by assessing choice between the two reinforcers with all else equal. For example, in the pilot experiment presented in Figure 1 above, rats chose between one lever that produced food pellets and another that produced dippers of a 5% sucrose solution. Response rates for food pellets were higher than those for the sucrose solution in a ratio of approximately 2:1, which would represent b = 2. Thus, a bias parameter allows the relative quality of the two reinforcers to be measured on the same scale (i.e., 1 food pellet ≈ 2 dippers of 5% sucrose solution). With bias thusly included, RaC² might provide a quantitative account of the present effects of qualitative differences in alternative reinforcement within a resurgence paradigm.

Across groups in the present experiments, both the higher (i.e., food pellet) and lower (i.e., sucrose solution) quality reinforcers served as the target reinforcer and/or as the alternative reinforcer. As a result, the most straightforward way to incorporate bias into the model is to scale reinforcement rates for one reinforcer (we have chosen sucrose here) prior to those reinforcer rates entering other calculations in the model. Thus, reinforcement rates for the four groups would be as follows for sessions in Phases 1 and 2, respectively: sucrose–sucrose, R_x/b, R_x/b; sucrose-pellet, R_x/b, R_x; pellet-pellet, R_x, R_x; pellet-sucrose, R_x, R_x/b, where R_x refers to obtained reinforcement rates in a given session. Scaling reinforcement rates for the sucrose reinforcer prior to other calculations has the effect of including b as necessary in the calculations of value in Equation 3 (with the values carried into the calculation of A in Equation 6) and in the calculation of c in Equation 2.³

Figure 6 shows simultaneous fits of RaC² to both target and alternative response rates for all groups from the present experiments. Best-fitting functions generated by the model after scaling obtained rates of sucrose delivery via b were obtained by applying least-squares regression (Microsoft Excel Solver) to the log-transformed data (see Shahan et al., 2020, for discussion). The bias parameter b was allowed to vary as a free parameter as were k, a, λ, dms, and dmp. The parameters dms and dmp reflect a different shared asymptote (i.e., dm in Equations 7 and 8) for d₁ and d₀ for sucrose and pellet alternative reinforcers, respectively.⁴ Thus, the model fit included six parameters and 140 data points. Overall, the fit accounts for 94% of the variance and captures many features of the data. Although the model provides a reasonably good description of the present data, there are a couple noteworthy shortcomings of the fit. First, during Phase 2, the model tended to overpredict target response rates for groups that experienced the same reinforcer between phases (i.e., the pellet-pellet and sucrose–sucrose groups) and underpredict target response rates for groups that experienced different reinforcers between phases (i.e., the pellet-sucrose and sucrose-pellet groups). Second, the model mispredicted alternative response rates, especially in Phase 2, in a variety of ways including underpredicting for the pellet-pellet group, overpredicting for the sucrose–sucrose and pellet-sucrose groups, and initially overpredicting and then underpredicting for the sucrose-pellet group. Related mispredictions of alternative response rates have been seen before (e.g., Shahan et al., 2020; Podlesnik et al., 2022; Nist & Shahan, 2023) and suggest obvious room for future model improvement. Nevertheless, it is important to note that despite the simplifying assumptions made to extend the model to the present data, the model did a good job overall at capturing ordinal differences in both target and alternative behavior simultaneously across the groups in Phases 2 and 3.

FIGURE 6.

Fit of RaC² to the data from both experiments from the end of baseline (presented on the y-axis above zero) and across all sessions of Phases 2 and 3. Solid lines represent means for each group, and dotted lines represent model predictions. Additional details in text.

One interesting aspect of the fit is that the estimated value of the b parameter (i.e., 4.74) is higher than the roughly twofold to threefold difference in responding obtained in the pilot experiment. It is important to note that in the resurgence preparation either one (Phase 2) or both responses are undergoing extinction, and as a result, the value estimate for one or both options is based on experienced reinforcement conditions that are no longer present. However, the reliability of this finding and the source of potential inflation of such biasing effects in a resurgence preparation remain unknown. Should it turn out that such biasing effects are reliably inflated in resurgence preparations, this would need to be considered in predictions about how different reinforcers might influence treatment efficacy and resurgence in situations employing DRA interventions. Additional research that directly compares reinforcer-quality induced bias inside versus outside of a resurgence preparation will be required.

In terms of further applications of RaC² to situations involving qualitatively different reinforcers, it is important to note that when it comes to potential interactions between reinforcers, the model inherits some of the shortcomings of the matching law upon which it is based. In applying the model to qualitatively different reinforcers, we have employed the bias parameter to scale the values of the two reinforcers. Such an approach that assumes the scalability of values of the reinforcers is only feasible because the two reinforcers employed likely were economic substitutes in the present context (see Green & Freed, 1993, for discussion). In behavioral-economic terms, if consumption of commodity A decreases with increases in its price (i.e., cost/benefit) but consumption of commodity B increases (despite no change in its price), then B can be said to be a substitute for A. Alternatively, if consumption of commodity A decreases with increases in its price (i.e., cost/benefit) and consumption of commodity B also decreases (despite no change in its price), then B can be said to be a complement for A (e.g., Hursh, 1980; Rachlin et al., 1976). The bulk of support for the matching law comes from situations in which the relative rates or magnitudes of two substitutes (usually identical reinforcers) are varied. However, the matching law does not straightforwardly apply to situations involving nonsubstitutable reinforcers (see Hursh, 1980; Green & Freed, 1993, for discussion). As a result, applying a choice-based model of resurgence to situations involving nonsubstitutable reinforcers will likely require a different approach that incorporates behavioral-economic considerations (e.g., Rachlin et al., 1981).

Similarly, situations involving qualitatively different reinforcers might also introduce other complexities for a scaling-based treatment of relative reinforcement value. For example, as a result of successive negative contrast effects, a higher quality reinforcer followed by a lower quality reinforcer could produce a lower valuation of the lower quality reinforcer than in a situation where the higher quality reinforcer was never experienced (see Papini, 2014, for review). Incorporation of such effects into a choice-based model of resurgence would require a considerably more complex scaling of the reinforcer values than was attempted here. Further development of such an approach would also require many more details about how such negative contrast effects influence valuations in operant choice situations.

The results of the present study emphasize the importance of obtaining information on reinforcer quality in the context of assessing and treating severe problem behavior, as a growing body of research suggests that such information can help to inform better treatment approaches. For example, using a human-operant preparation, Weinsztok and DeLeon (2022) recently showed that delivering a higher magnitude or higher quality reinforcer for alternative responding produced better protection against increasingly degraded levels of treatment integrity (i.e., errors of omission and commission) than did a control condition in which alternative responding produced a relatively lower magnitude or lower quality reinforcer, respectively. Although these findings differ from those of the present study, several procedural differences between the two studies limit a direct comparison. Potentially important differences between the two studies include (a) whether the alternative response was exposed to extinction rapidly or gradually, (b) whether the target response began to produce reinforcement as the schedule of alternative reinforcement was thinned, (c) whether subjects alternated between test and control conditions throughout the evaluation, and (d) whether response allocation was a primary measure of the maintenance of treatment effects. Despite these differences, the findings from both studies suggest that reinforcer quality affects both treatment efficacy and relapse susceptibility in more applied contexts. What precisely accounts for their seemingly discrepant findings will require additional investigation.

Applied research has generally supported the notion that reinforcer quality affects treatment efficacy, but comparatively little applied research has examined how reinforcer quality affects relapse susceptibility. However, Norris and Greer (2023) recently showed that individuals whose severe problem behavior is maintained by multiple reinforcers (e.g., access to attention and toys) often have strong preferences for one functional reinforcer over others. The extent to which the maintenance of problem behavior by qualitatively distinct reinforcers affects treatment efficacy and relapse susceptibility has yet to be explored. The results of the present study suggest that such differences in reinforcer quality are likely to affect both treatment efficacy and relapse susceptibility in clinically meaningful ways. Investigating these possibilities will be an important area for future applied research.