Mechanisms of renewal after the extinction of discriminated operant behavior

Abstract

Three experiments demonstrated, and examined the mechanisms that underlie, the renewal of extinguished discriminated operant behavior. In Experiment 1, rats were trained to perform one response (lever press or chain pull) in the presence of one discriminative stimulus (S; light or tone) in Context A, and to perform the other response in the presence of the other S in Context B. Next, each of the original S/response combinations was extinguished in the alternate context. When the S/response combinations were tested back in the context in which they had been trained, responding in the presence of S returned (an ABA renewal effect was observed). This renewal could not be due to differential context-reinforcer associations, suggesting instead that the extinction context inhibits either the response and/or the effectiveness of the S. Consistent with the latter mechanism, in Experiment 2, ABA renewal was still observed when both the extinction and renewal contexts inhibited the same response. However, in Experiment 3, previous extinction of the response in the renewing context (occasioned by a different S) reduced AAB renewal more than did extinction of the different response. Taken together, the results suggest at least two mechanisms of renewal after instrumental extinction. First, extinction performance is at least partly controlled by a direct inhibitory association that is formed between the context and the response. Second, in the discriminated operant procedure, extinction performance can sometimes be partly controlled by a reduction in the effectiveness of the S in the extinction context. Renewal of discriminated operant behavior can be produced by a release from either of these forms of inhibition.

Extinction of either Pavlovian or instrumental conditioning, produced by withholding the reinforcer, results in a decrease in the originally learned behavior (e.g., Pavlov, 1927, Mackintosh, 1974). Extinction is a fundamental process of behavior change that allows organisms to adapt to changes in their environment (Bouton, 1997, 2004). While some theories of associative learning assume that extinction results in a decrease or weakening of the underlying original learning (e.g., Rescorla & Wagner, 1972; cf. Pearce & Hall, 1980; Wagner, 1981), it is now widely understood that extinction results in new learning that is at least partly context-dependent (e.g., Bouton, 2002; Bouton, Westbrook, Corcoran, & Maren, 2006).

In Pavlovian conditioning, extinction takes the form of the conditioned response diminishing when the conditioned stimulus (CS) is repeatedly presented in the absence of the unconditioned stimulus (US). However, it is well known that responding to the CS will return if it is tested in a context that is different from the one in which extinction occurred. (In animal experiments, contexts are usually defined as the chambers in which conditioning occurs; they typically differ in visual, tactile, and olfactory characteristics.) This recovery of responding to the CS, known as renewal (Bouton & Bolles, 1979), occurs when conditioning takes place in Context A, extinction in Context B, and testing occurs in Context A (ABA renewal; Bouton & Bolles, 1979; Bouton & King, 1983; Bouton & Peck, 1989) or in a third, relatively neutral Context C (ABC renewal; Bouton & Bolles, 1979; Thomas, Larsen, & Ayres, 2003). Renewal has also been shown when conditioning and extinction both take place in Context A, and the CS is subsequently tested in Context B (AAB renewal; Bouton & Ricker, 1994; Laborda, Witnauer, & Miller, 2011; Tamai & Nakajima, 2000). The fact that mere removal from the context of extinction (as in AAB and ABC renewal) is sufficient for renewal to occur is consistent with the idea that extinction results in new inhibitory learning that is at least partly specific to the extinction context.

All three forms of renewal have also been demonstrated after the extinction of instrumental behavior, where the organism’s own behavior no longer produces the reinforcer (e.g., Bouton, Todd, Vurbic, & Winterbauer, 2011; Todd, 2013). In one experiment, Bouton et al. (2011) first trained rats to lever press for food pellets in Context A. Next, one group received extinction (where lever pressing was allowed but no food pellets were delivered) in Context B and a second group received extinction in Context A. In a final test, responding renewed when the rats were tested outside of the extinction context. That is, responding increased in Context A, when extinction had occurred in Context B (ABA renewal). Responding also increased in Context B, when extinction had occurred in Context A (AAB renewal; see also Todd, Winterbauer, & Bouton, 2012a). In a separate experiment, instrumental responding also renewed when acquisition, extinction, and testing all occurred in different contexts (ABC renewal; see also Todd, Winterbauer, & Bouton, 2012b). As in Pavlovian extinction, these three forms of renewal indicate that instrumental extinction does not completely eliminate the original learning, but instead results in new inhibitory learning that is partly context-dependent. Renewal occurs because of a failure to retrieve the new inhibitory association when testing occurs outside of the extinction context. It should be noted that ABA renewal has also been demonstrated with responses that have been reinforced with drugs (Bossert, Liu, Lu, & Shaham, 2004; Chaudhri, Sahuque, & Janak, 2009; Crombag & Shaham, 2002; Hamlin, Clemens, & McNally, 2008; Hamlin, Newby, & McNally, 2007; Zironi, Burattini, Aircardi, & Janak, 2006).

While much is known about the theoretical learning mechanisms that underlie Pavlovian extinction (see Bouton, 1993, 1994, 1997, 2002, 2004; Delamater, 2004; Laborda, McConnell, & Miller, 2011), less is known about the mechanisms that might underlie instrumental extinction (see Bouton et al., 2011; Todd, 2013). One possibility is that the extinction context might enter into a direct inhibitory association with the representation of the reinforcer (Polack, Laborda, & Miller, 2011; but see Bouton & King, 1983; Bouton & Swartzentruber, 1986, 1989). Mechanistically, during extinction the response predicts that the reinforcer will occur, but when the reinforcer is not delivered its surprising omission might cause the extinction context to form a direct inhibitory association with the representation of the reinforcer. Second, it is also possible that the context becomes a negative occasion setter for the response-reinforcer relationship. In this case, instead of inhibiting the reinforcer representation directly, the context signals that the response will not be followed by the reinforcer, perhaps by controlling an inhibitory association between them (Bouton & Nelson, 1994, 1998; see also Swartzentruber & Rescorla, 1993). A third possible mechanism is that the context might directly inhibit the response (Rescorla, 1993, 1997). For example, Rescorla (1993, Experiment 2) first trained two stimuli (light and noise) to signal that each of two operant responses (lever press or chain pull) would be reinforced. Each response was then extinguished in the presence of only one stimulus. During subsequent testing, when all four possible stimulus/response combinations were presented, responding was most depressed in the presence of the stimulus with which it had been extinguished. Importantly, although such results are compatible with the idea that an inhibitory association is formed between the stimulus and the response, they are equally compatible with the negative occasion-setting mechanism (Rescorla, 1993, p. 335; 1997, p. 249; Bouton, 2004).

Based on a recent series of experiments, Todd (2013) suggested that extinction of free-operant behavior may be best characterized by the formation of an inhibitory association between the context and the response. For example, in one experiment, rats were trained to either lever press or chain pull (R1) for food pellets in Context A, and to perform the other response (R2) in Context B. This ensured that both contexts were equally associated with reinforcement. Then, each response was extinguished in the alternate context, ensuring that both Contexts A and B were also equally associated with nonreinforcement. In a subsequent test, responding on R1 renewed in Context A and responding on R2 renewed in Context B. Thus, response-specific renewal (ABA) was observed when the contexts were equated for their histories of reinforcement and nonreinforcement, controlling for any differential context-reinforcer associations (excitation and/or inhibition). Using similar designs, AAB and ABC renewal were also both observed. The results of these experiments suggest that renewal does not require the extinction context forming a direct inhibitory association with the reinforcer during extinction. However, they could be interpreted in terms of either a direct inhibitory association between the context and the response, or a hierarchical association where the context signals that the response will not be followed by the reinforcer (i.e., negative occasion setting). Todd (2013, Experiment 4) also found that renewal was not reduced when testing occurred in a context that was previously associated with extinction of another response relative to a context that was not. Because negative occasion setting is known to transfer across suitable targets (e.g., Holland & Coldwell, 1993), this finding was not consistent with a negative occasion setting account. Thus, Todd (2013) suggested that the data were most consistent with the idea that a simple, direct inhibitory association is formed between the context and the response during extinction.

The present experiments examined extinction and renewal of discriminated operant behavior (see Nakajima et al., 2000; Vurbic, Gold, & Bouton, 2011). In the discriminated operant (or multiple schedule) procedure, the response-reinforcer contingency is only in place when a discriminative stimulus (S) is present (e.g., Skinner, 1938; Colwill & Rescorla, 1988). In a typical method, rats are allowed to lever press freely in the conditioning chamber, but lever presses are only reinforced during the presence of another stimulus (e.g., a light). The standard result is that rats learn to lever press at a high rate during the S (the light), and withhold responding when the S is not present.

Extinction and renewal of discriminated operant behavior are interesting both from a translational and a theoretical perspective. From a translational perspective, it can be noted that problematic operant behaviors such as drug-taking, gambling, and overeating are often under the control of reinforcer-associated discriminative stimuli (Conklin & Tiffany, 2002; McCarthy, Baker, Minami, & Yeh, 2011). Thus, discriminated operants might more closely model many behaviors in humans. From a theoretical perspective, the study of discriminated operant extinction is also interesting because little is known about the underlying mechanisms. Although Todd (2013) has shown that renewal of free-operant behavior does not depend upon differential context-reinforcer associations, it is possible that such associations do indeed play a role in renewal of a discriminated operant. Further, the addition of behavioral control by S increases the number of potential associations that might be learned and influence performance during extinction. Rescorla and Colwill have identified several associations in the control of discriminated operant responding (e.g., Colwill, 1994; Colwill & Rescorla, 1986; Rescorla, 1991), including an association between S and the reinforcing outcome (O), an association of the response (R) and the outcome, and a hierarchical association in which S signals the relation between R and O. It is conceivable that extinction could modulate any of these associations. Thus, the mechanisms controlling extinction of a discriminated operant might differ to some extent from those controlling extinction of an unsignaled free-operant response.

Finally, the use of a discriminated operant procedure allows us to test further predictions of the inhibitory context-response hypothesis. For example, if extinction of instrumental behavior results in the formation of such an inhibitory association, then renewal should be reduced if animals learn to inhibit the target response in the renewal context prior to renewal testing there. The current experiments were designed, in part, to test this prediction. The prediction could not be meaningfully tested in a free-operant procedure (e.g., Bouton et al. 2011; Todd, 2013), because extinction of a free operant in a renewing context would naturally and unsurprisingly create extinction performance and suppress renewal there. In contrast, in a discriminated operant method, inhibition of the response in the renewing context could take the form of extinction of responding in the presence of one S reducing renewal of the same response controlled by a different S.

Experiment 1

The purpose of the first experiment was to provide an initial test of ABA renewal of discriminated-operant behavior when the contexts were equated for their reinforcement histories (e.g., Bouton & Ricker, 1994; Harris, Jones, Bailey, & Westbrook, 2000; Lovibond, Preston, & Mackintosh, 1984; Rescorla, 2008; Todd, 2013). The design is presented in Table 1. Rats were first reinforced for performing R1 in the presence of S1 in Context A, and R2 in the presence of S2 in Context B. Lever pressing and chain pulling (counterbalanced) played the role of R1 and R2. Next, in the extinction phase, S1 was presented in Context B and S2 was presented in Context A, and R1 and R2 (respectively) were not followed by the reinforcer. This reduced both R1 and R2. In the final test, S1 was presented in Contexts A and B with R1 available, and S2 was presented in Contexts A and B with R2 available. Notice that the two contexts, R1 and R2, and S1 and S2 had all been treated equivalently. As in the free-operant experiments of Todd (2013), a renewal effect observed in Experiment 1 would not be attributable to differential context-reinforcer associations. Because each context is associated with reinforcement (acquisition) and nonreinforcement (extinction), any direct association between the contexts and the reinforcer should be equivalent. Thus, in this design, renewal cannot be due to one context being more excitatory or more inhibitory than the other.

Table 1.

Design of experiments 1–3

Renewal Type	Acquisition	Extinction	Test
Experiment 1
ABA	A: S1R1+	A: S2R2−	A: S1R1− v. B: S1R1−
	B: S2R2+	B: S1R1−	B: S2R2− v. A: S2R2−
Experiment 2
ABA (Same R)	A: S1R1+	A: S2R1−	A: S1R1− v. B: S1R1−
	B: S2R1+	B: S1R1−	B: S2R1− v. A: S2R1−
Experiment 3
			A: S1R1− v. B: S1R1−
	A: S1R1+, S2R2+	A: S1R1−, S2R2−	A: S2R2− v. B: S2R2−
AAB	B: S3R1+	B: S3R1−	and
	C: S4R2+	C: S4R2−	A: S1R1− v. C: S1R1−
			A: S2R2− v. C: S2R2−

Note. A, B, and C refer to counterbalanced contexts. R1 and R2 refer to lever press and chain pull, counterbalanced. S1 and S2 refer to discriminative stimuli (light and tone, counterbalanced), S3 and S4 (flashing light and click, counterbalanced). + designates reinforcement, − designates nonreinforcement (extinction).

Method

Subjects

The subjects were 16 naïve female Wistar (albino) rats purchased from Charles River Laboratories (St. Constance, Quebec). They were between 75 and 90 days old at the start of the experiment and were individually housed in suspended wire mesh cages in a room maintained on a 16:8-h light:dark cycle. Experimentation took place during the light period of the cycle. The rats were food-deprived to 80% of their initial body weights throughout the experiment.

Apparatus

Two sets of four conditioning chambers housed in separate rooms of the laboratory served as two contexts (counterbalanced). Each chamber was housed in its own sound attenuation chamber. All boxes were of the same design (Med Associates model: ENV-008-VP). They measured 30.5 cm × 24.1 × 21.0 cm (l × w × h). In one set of boxes, the sidewalls and ceiling were made of clear acrylic plastic, while the front and rear walls were made of brushed aluminum. The floor was made of stainless steel grids (0.48 cm diameter) staggered such that odd- and even-numbered grids were mounted in two separate planes, one 0.5 cm above the other. A recessed 5.1 cm × 5.1 cm food cup was centered in the front wall approximately 2.5 above the level of the floor. Retractable levers (Med Associates model: ENV-112CM) were positioned to the left and right of the food cup. These levers were 4.8 cm long and positioned 6.2 cm above the grid floor. The left lever protruded 1.9 cm when extended (the right lever remained retracted over the course of the experiment). A response chain (Med Associates model: ENV-111C) was suspended from a micro switch mounted on top (outside) of the ceiling panel. The chain hung 1.9 cm from the front wall, 3 cm to the right of the food cup, and 6.2 cm above the grid floor. Thus, the lever and chain were positioned symmetrically with respect to the food cup. A 28-V panel light (2.5 cm in diameter) was attached to the wall 10.8 cm above the floor and 6.4 cm both to the left and right of the food cup. The chambers were illuminated by one 7.5-W incandescent bulb mounted to the ceiling of the sound attenuation chamber, approximately 34.9 cm from the grid floor at the front wall of the chamber. Ventilation fans provided background noise of 65 dB. This set of boxes had no distinctive visual cues. A dish containing 5 ml of Rite Aid Lemon Cleaner (Rite Aid corporation, Harrisburg, PA) was placed outside of each chamber near the front wall.

The second set of boxes was similar to the lemon-scented boxes (e.g., also model ENV-008-VP, placement of levers, chain, panel lights), except for the following features. In each box, one sidewall had black diagonal stripes, 3.8 cm wide and 3.8 cm apart. The ceiling had similarly spaced stripes oriented in the same direction. The grids of the floor were mounted on the same plane and were spaced 1.6 cm apart (center-to-center). The chambers were illuminated by one 7.5-W incandescent bulb mounted to the ceiling of the sound attenuation chamber, near the back wall of the chamber. A distinct odor was continuously presented by placing 5 ml of Pine-Sol (Clorox Co., Oakland, CA) in a dish outside the chamber.

The reinforcer was a 45-mg grain-based rodent food pellet (TestDiet, Richmond, IN, USA). The auditory stimulus was a 30-s presentation of a 3000-Hz tone (80 dB) delivered through a 7.6-cm speaker mounted to the ceiling of the sound attenuation chamber. The visual stimulus was 30-s illumination of the left panel light; the right panel light was not used in this experiment. The apparatus was controlled by computer equipment located in an adjacent room.

Procedure

Magazine training

On the first day of the experiment, all rats were assigned to one box within each set of boxes. They then received a single 30-min session of magazine training in each context (A then B). Sessions were separated by approximately 1 hr. In each session, food pellets were delivered freely on a random time 30 s (RT 30s) schedule resulting in approximately 60 pellets being delivered. This schedule was programmed by delivering a pellet in a given second with a 1 in 30 probability. The levers and chains, and the tone and light, were not present during this training.

Acquisition

On each of the next two days, rats received two sessions, one of them in Context A and the other in Context B (order counterbalanced). For half the rats, the response trained in A (R1) was the lever and for half it was chain. The alternative response (R2) was trained in Context B. During each 30-min training session, responding was reinforced on a random interval (RI) 30-s schedule. This schedule was programmed by initiating pellet availability in a given second with a 1 in 30 probability. The pellet remained available until the next response, at which point the pellet was delivered and the schedule was restarted.

Over the next 12 days, rats received 24 sessions of discrimination training, half of them in Context A and the other half in Context B. In each Context A session, all rats received 16 trials in which R1 was only reinforced during the 30-s presentation of a discriminative stimulus S1 (panel light or tone, counterbalanced). In each session in Context B, all rats received 16 trials in which R2 was only reinforced during the 30-s presentation of S2 (tone or panel light, counterbalanced). During each S, responding was reinforced on a random interval (RI) 30-s schedule. The time between successive S presentations (intertrial interval: ITI) was variable, averaging 30 s on Day 1, 60 s on Day 2, and 90 s in all sessions thereafter. Daily sessions were separated by approximately 1 hr. The order of exposure to each context was double alternated, so that Contexts A and B were equally experienced in the first or second session of the day.

Extinction

On each of the next 4 days, rats received extinction of each response, with its respective S, in the context opposite to the one in which it was trained. Thus, the S1R1 combination was extinguished in Context B and S2R2 was extinguished in Context A. Extinction sessions were identical to acquisition sessions except that responses did not result in food pellets. There were two extinction sessions per day, one in each context, separated by approximately 1 hr. The order of exposure to each context was counterbalanced across days.

Renewal test

On the final day, each rat received two 8-min test sessions in each context, separated by approximately 30 min, resulting in a total of four test sessions. There were four presentations of S during each test session. The ITI was variable with a mean of 90 s. In each session, only one of the lever or chain was available and no reinforcers were delivered. Half the rats were first tested in the acquisition and then the extinction context for one S/response combination (e.g., S1R1), and then tested in the acquisition followed by the extinction context for the other S/response combination (e.g., S2R2). The other half of the rats were first tested in the extinction followed by the acquisition context for one S/response combination, and then the extinction followed by the acquisition context for the other S/response combination. The identities of S (light or tone) and the response (lever press or chain pull) were fully counterbalanced throughout this test procedure. The result of the design was that for each animal, each S/response combination was tested in the context in which it had been first reinforced and in the context in which it had been extinguished.

Data analysis

The computer recorded the number of responses made during the 30-s stimulus presentation as well as during the 30-s period just prior to the stimulus (the pre-S period). Elevation scores were calculated by subtracting the number of responses made prior to the stimulus (pre-S) from the number of responses made during the S for each trial. Mean elevation scores and mean pre-S scores were evaluated with analysis of variance (ANOVA) using a rejection criterion of p < .05.

For ANOVAs with more than one factor we report η_p² as a measure of effect size; for comparisons between two means we report η². For either measure, we also computed 95% confidence intervals using procedures described by Steiger (2004). First, we obtained the noncentrality parameter of the non-central F distribution. We then calculated the 95% confidence interval on the noncentrality parameter and used that to compute upper and lower boundaries of effect size estimates.

Results

Acquisition and Extinction

Elevation scores

The data from acquisition and extinction are presented in Figure 1. The figure shows the mean elevation scores per session for S1R1 and S2R2. There was a clear increase in discriminative responding over acquisition, followed by a decrease during extinction. In acquisition, a 2 (Response: S1R1 vs. S2R2) × 12 (Session) ANOVA revealed a main effect of Session, F(11, 165) = 35.04, MSE = 7.93, η_p² = .70, CI [.60, .74], p < .001. No other main effects or interactions were significant, Fs < 1. Further, S1R1 and S2R2 responding did not differ on the final day of acquisition, F(1, 15) = 1.26. Over the course of extinction, there was a main effect of Session, F(3, 45) = 62.96, MSE = 3.74, η_p² = .81, CI [.68, .86], p < .001. Although the main effect of Response was not significant (F < 1), there was an unexpected interaction between Response and Session, F(3, 45) = 2.88, MSE = 2.10, η_p² = .16, CI [.00, .31], p < .05. However, the effect of Response was not significant during any one session, largest F(1,15) = 2.35.

Figure 1.

Results of Experiment 1. Mean elevation scores, for S1R1 and S2R2, during each 30-min session of acquisition (left) and extinction (right).

Pre-S responding

Parallel analyses were conducted on the pre-S scores. Over the course of acquisition, a 2 (Response: S1R1 vs. S2R2) × 12 (Session) ANOVA revealed a main effect of Session, F(11, 165) = 2.22, MSE = 7.10, η_p² = .13, CI [.00, .17], p < .05. Although the effect of Response was not significant, F < 1, the interaction between Response and Session was, F(11, 165) = 3.72, MSE = 4.80, η_p² = .20, CI [.05, .25], p < .001. However, by the final session of acquisition training, there were no differences in pre-S responding for S1R1 and S2R2, F < 1. On the first day of training, pre-S responding averaged 6.9 and 8.2, for S1R1 and S2R2 respectively. By the final day of acquisition training, pre-S responding averaged 6.6 and 7.3, for S1R1 and S2R2.

Pre-S responding then decreased over the sessions of extinction, F(3, 45) = 34.10, MSE = 1.10, η_p² = .70, CI [.51, .77], p < .001. The effect of Response and the interaction between Response and Session were not significant, Fs < 1. During the first session of extinction, pre-S responding averaged 2.9 and 2.8 for S1R1 and S2R2, respectively. By the final day of extinction, pre-S responding averaged 0.5 and 0.7 for S1R1 and S2R2, respectively.

Renewal Test

Elevation scores

The data from the renewal test are presented in Figure 2, which depicts elevation scores for S1R1 and S2R2 in Context A and B. Note that the renewal context for S1R1 was Context A, which was the extinction context for S2R2, and that the renewal context for S2R2 was Context B, which was the extinction context for S1R1. There was clear evidence of renewal for both S1R1 and S2R2. A 2 (Response: R1 vs. R2) × 2 (Context: A vs. B) ANOVA confirmed this impression. Neither the main effect of Response, nor the main effect of Context were significant, larger F(1, 15) = 1.83, MSE = 11.52, p = .20. However, the interaction between these factors was highly reliable, F(1, 15) = 16.72, MSE = 10.55, η_p² = .53, CI [.13, .71], p < .01. S1R1 responding was higher in A than B, F(1, 15) = 8.06, MSE = 14.91, η² = .35, CI [.02, .60], p = .01 and S2R2 responding was higher in B than A, F(1, 15) = 10.55, MSE = 5.80, η² = .41, CI [.05, .64], p < .01.

Figure 2.

Results of Experiment 1 renewal test. Mean elevation scores during each test session.

Descriptively, the renewal effect was robust; 25 of the 32 possible renewal effects (2 effects each for 16 subjects) were observed (defined as r > e, where r is the mean elevation score in the renewal context and e is the mean elevation score in the extinction context). When the effect was averaged over the two responses, 14 of 16 rats showed a renewal effect.

Pre-S responding

As previously noted, R1 and R2 responding in the pre-S period had decreased during extinction. Interestingly, pre-S responding on its own also showed renewal. A 2 (Response: R1 vs. R2) × 2 (Context: A vs. B) ANOVA revealed a significant interaction between Response and Context, F(1, 15) = 10.26, MSE = 2.08, η_p² = .41, CI [.04, .63], p < .01. Neither the main effect of Response, nor the main effect of Context was significant, Fs < 1. Pre-S responding was higher on R1 in Context A (M = 1.73) than in Context B (M = 0.59), F(1, 15) = 4.54, MSE = 2.30, η² = .23, CI [.00, .51], p = .05, and slightly higher (although only approaching significance) on R2 in Context B (M = 1.83) than in Context A (M = 0.66), F(1, 15) = 4.39, MSE = 2.50, η² = .23, CI [.00, .50], p = .053.

Although the increase in pre-S responding was significant when averaged over all 4-test trials, the increase was primarily present early in the session. For example, during the last two-trial block, pre-S responding on R1 in Context A (M = 1.2) did not differ from Context B (M = .72), F < 1. Further, pre-S responding on R2 in Context B (M = 1.88) did not differ from Context A (M = .81), F(1, 15) = 2.10, MSE = 4.40, p = .17. Importantly, although pre-S responding was not significantly different during the final two-trial block, a strong renewal of responding was still observed in the S as judged by elevation scores. S1R1 responding was higher in A (M = 5.03) than B (M = 1.34), F(1, 15) = 5.30, MSE = 20.60, η² = .26, CI [.00, .53], p < .05 and S2R2 responding was higher in B (M = 3.40) than A (M = −.10), F(1, 15) = 10.30, MSE = 9.18, η² = .41, CI [.05, .63], p < .01.

Discussion

Experiment 1 demonstrated ABA renewal of extinguished discriminated operant behavior under conditions that equated the reinforcement and nonreinforcement histories of the extinction and renewal contexts. The results are consistent with the findings of Todd (2013; see also Cohen-Hatton & Honey, in press). Because the reinforcement and nonreinforcement histories of the extinction and renewal context were equated and controlled, ABA renewal with the discriminated-operant procedure cannot be due to differential context-reinforcer associations. The renewal observed in Experiment 1 is consistent with the idea that the extinction context forms an inhibitory association with the response (e.g., Todd, 2013). When testing occurs back in the acquisition context, the response is released from context-specific inhibition.

Alternatively, the renewal observed in Experiment 1 could be understood in terms of the extinction context reducing the effectiveness of the S extinguished there. Instead of, or in addition to, context-specific response inhibition, the extinction context may also acquire context-specific inhibition of the S. The final two experiments further investigated the role of these possible mechanisms in renewal of discriminated operant behavior. Experiment 2 examined if the extinction context modulates the effectiveness of the S, and Experiment 3 further examined the role of context-specific response inhibition.

Experiment 2

The purpose of the second experiment was to test the possibility that the extinction context reduces or modulates the effectiveness of the S extinguished there. To do so, renewal was tested under conditions where both the extinction and renewal contexts were previously trained to inhibit the same response. As shown in Table 1, a single group of rats was trained to perform R1 in the presence of S1 in Context A, and to perform R1 in the presence of S2 in Context B. Next, each S/response combination (S1R1 and S2R1) was extinguished in the alternative context (S1R1 in B, S2R1 in A). Notice that this treatment resulted in R1 being acquired and then extinguished in both Contexts A and B. In the final test, S1 was presented in Contexts A and B with R1 available, and S2 was presented in Contexts A and B with R1 available. Renewal would take the form of S1R1 responding to be higher in A than B, and S2R1 responding to be higher in B than A. As in Experiment 1, the reinforcement and nonreinforcement histories of the contexts, as well as the Ss, were equivalent. If renewal occurred under the present conditions, it would not be attributable to differences in direct context-reinforcer associations or to a release from context-specific response inhibition. Instead, it would suggest that renewal can result from release from some form of inhibition of the effectiveness of the S.

Method

Subjects and Apparatus

The subjects were 16 naïve female Wistar rats of the same age and purchased from the same vendor as those in the previous experiment, maintained under the same conditions. The apparatus was also the same.

Procedure

Magazine training was carried out in a manner identical to Experiment 1.

Acquisition

As in Experiment 1, on each of the next two days, rats received two sessions of free-operant training, one of them in Context A and the other in Context B (order counterbalanced). For half the rats, the response trained in A (R1) was the lever and for the other half it was chain. The same response (R1) was trained in Context B. During each 30-min training session, responding was reinforced on a random interval (RI) 30-s schedule.

Over the next 12 days, rats received 24 sessions of discrimination training, half of them in Context A and the other half in Context B (daily order counterbalanced). There was a session in each context on each day. In Context A sessions, all rats received 16 trials in which R1 was only reinforced during the 30-s presentation a discriminative stimulus, S1 (panel light or tone, counterbalanced). In each session in Context B, there were 16 trials in which R1 (lever-press or chain pull, counterbalanced) was only reinforced during the 30-s presentation of the S2 (panel light or tone, counterbalanced). During each S, responding was reinforced on a random interval (RI) 30-s schedule. As in Experiment 1, the ITI was variable, averaging 30 s on Day 1, 60 s on Day 2, and 90 s in all sessions thereafter. Daily sessions were separated by approximately 2 hrs and the order of exposure to each context was counterbalanced over days.

Extinction

On each of the next 4 days, rats received extinction of both of their trained S/response combinations in the context alternate to the one in which it had been trained. S1R1 was extinguished in Context B and S2R1 was extinguished in Context A. Extinction sessions were identical to acquisition sessions except that responses did not result in food pellets. There were two extinction sessions per day, one in each context, separated by approximately 2 hrs with the order of context exposure counterbalanced over days.

Renewal test

On the final day, each rat received two 8-min test sessions in each context, separated by approximately 30 min, resulting in 4 total test sessions. This phase of the experiment was carried out in a manner similar to Experiment 1. For each animal, each S/response combination was tested in the context in which it had been first reinforced and in the context in which it had subsequently been extinguished.

Results

Acquisition and Extinction

The data from acquisition and extinction are presented in Figure 3. The figure shows the mean elevation scores in each session for S1R1 and S2R1. There was a clear increase in discriminative responding over acquisition (left side of the figure), followed by a decrease in responding during extinction (right side of the figure). For acquisition, a 2 (Response: S1R1 vs. S2R1) × 12 (Session) ANOVA revealed a main effect Session, F(11, 165) = 26.75, MSE = 15.73, η_p² = .64, CI [.53, .68], p < .001. No other main effects or interactions were significant, Fs < 1. There was also no difference between S/response combinations during the final acquisition session, F < 1. Over the course of extinction, a 2 (Response: S1R1 vs. S2R1)× 4 (Session) ANOVA revealed effects of Session, F(3, 45) = 47.63, MSE = 11.96, η_p² = .76, CI [.61, .82], p < .001. No other main effects or interactions were significant, Fs < 1.

Figure 3.

Results of Experiment 2. Mean elevation scores, for S1R1 and S2R1, during each 30-min session of acquisition (left) and extinction (right).

Pre-S responding

A parallel analysis was conducted on the pre-S scores. Over the course of acquisition, a 2 (Response: S1R1 vs. S2R1) × 12 (Session) ANOVA revealed an interaction between Response and Session, F(11, 165) = 5.98, MSE = 4.20, η_p² = .26, CI [.13, .35], p < .01, although neither main effects were significant, Fs < 1. Pre-S responding averaged 9.4 and 10.7 during the first session of acquisition for S1R1 and S2R1, respectively. During the final session of acquisition, pre-S responding averaged 9.1 and 8.1 responses per minute for S1R1 and S2R1, respectively, and these did not differ from each other, F(1, 15) = 2.02, p = .18.

Over the course of extinction, a 2 (Response: S1R1 vs. S2R1) × 12 (Session) ANOVA revealed a main effect of Session, F(3, 45) = 38.79, MSE = 1.55, η_p² = .72, CI [.55, .79], p < .001, and a significant interaction between Response and Session, F(3, 45) = 3.27, MSE = 1.04, η_p² = .18, CI [.00, .33], p < .05. The main effect of Response approached significance, F(1, 15) = 3.92, p = .06. Pre-S responding in the first session of extinction averaged 2.7 and 4.0 for S1R1 and S2R1, respectively. Pre-S responding during the final session of extinction averaged 0.4 and 0.3 for S1R1 and S2R1, respectively, and this difference was not reliable, F(1, 15) = 2.17, p = .16.

Renewal Test

Elevation scores

The data from the renewal test are presented in Figure 4. The figure depicts responding for S1R1 in its extinction (B) and renewal (A) context as well as for S2R1 in its extinction (A) and renewal (B) context. Renewal occurred for both S1R1 and S2R1. A 2 (Response: S1R1 vs. S2R1) × 2 (Context: A vs. B) ANOVA confirmed this impression. Neither the main effect of Response, nor the main effect of Context were significant, Fs < 1. However, the interaction between these factors was reliable, F(1, 15) = 7.07, MSE = 3.19, η_p² = .32, CI [.01, .58], p = .018. S1R1 responding was higher in A than B, F(1, 15) = 4.70, MSE = 3.67, η² = .24, CI [.00, .51], p < .05 and S2R2 responding was higher in B than A, F(1, 15) = 3.08, MSE = 2.13, η² = .17, CI [.00, .46], p < .1, although it only approached significance. Nevertheless, when averaged over S1R1 and S2R2 the renewal effect was reliable, F(1, 15) = 7.07, MSE = 1.60, η² = .32, CI [.01, .58], p = .018.

Figure 4.

Results of Experiment 2 renewal test. Mean elevation scores during each test session.

Descriptively, 22 of the 32 possible renewal effects (2 effects each for 16 subjects) were observed. When the effect was averaged over the two responses, 11 of 16 rats showed a renewal effect.

Pre-S responding

As in Experiment 1, pre-S responding had also extinguished during the extinction phase. A 2 (Response: S1R1 vs. S2R1) × 2 (Context: A vs. B) ANOVA on pre-S responding did not reveal any significant main effects or interactions, largest F(1, 15) = 2.16, p = .16. Pre-S responding for S1R1 averaged .30 and .69, in the renewal (A) and extinction contexts (B), respectively. Pre-S responding for S2R1 averaged .30 and .52 in the renewal (B) and extinction contexts (A), respectively. Thus, there was no evidence of renewal in the pre-S period when the same response had been extinguished in both contexts.

Discussion

Experiment 2 demonstrated ABA renewal of discriminated operant responding when the same response was extinguished in the extinction and renewal context, presumably resulting in context-specific response inhibition (of the same response) in both contexts. The observation of renewal when inhibitory context response associations are equated suggests that, at least under some conditions, the extinction context may be modulating the effectiveness of the S extinguished there. Renewal occurs when the S is tested outside of the context it was extinguished in. Potential mechanisms by which the context could inhibit the effectiveness of the S are discussed in the General Discussion.

Experiment 3

As noted earlier, the results of Experiment 1 could be described in terms of the extinction context’s ability to inhibit either the response or the effectiveness of the S. Experiment 2 provided evidence for the latter, by demonstrating ABA renewal when the contexts were equated for their ability to inhibit the same response. However, the fact that Experiment 1 also found evidence of renewal in pre-S responding (that is, responding in the absence of S) suggests the presence of an additional mechanism that does not influence S. One candidate is a direct, inhibitory context-response association. The purpose of Experiment 3 was to assess this possibility.

There were actually two purposes of Experiment 3. The first was to demonstrate AAB renewal after the extinction of discriminated operant behavior. To our knowledge, AAB renewal has not been demonstrated with a discriminated operant procedure. In fact, there is at least one published report where AAB renewal was not observed (e.g., Nakajima et al., 2002). Demonstrating AAB renewal is itself important because it would indicate that removal from the context of extinction is sufficient for renewal to occur, and most directly implicate inhibitory learning during extinction (e.g., Bouton et al., 2011). The second purpose of the experiment was to provide a stronger test of the role of context-specific response inhibition. It employed the within-subject design illustrated in Table 1. A single group of rats received discriminated operant training with two different responses (R1 and R2) and four different discriminative stimuli (S1, S2, S3, S4) in three different contexts (A, B, and C). S1R1 and S2R2 were trained in Context A. S3R1 was trained in Context B and S4R2 was trained in Context C. Following acquisition, all S/response combinations underwent extinction in the contexts in which they had been trained. In the final renewal test, two target S/response combinations (S1R1 and S2R2) were tested in their extinction context (A) and in two renewal contexts (B and C). If context-specific response inhibition develops in extinction, renewal in Context B should be higher for S2R2 than S1R1, because R1 had previously been extinguished there (i.e., in S3R1). In a complementary way, renewal in Context C should be higher for S1R1 than S2R2 because R2 had previously been extinguished there (i.e., in S4R2). Because the design was within-subject, all rats had received equivalent training and extinction of R1 and R2, as well as S1, S2, S3, and S4.

Method

Subjects

The subjects were 24 naïve female Wistar rats purchased from the same vendor as those in the previous experiment and maintained under the same conditions.

Apparatus

The experiment employed three contexts, two of which were the contexts used in Experiments 1 and 2. The third context was of a type similar to those boxes (e.g., it was also model ENV-008-VP, with the same placement of levers, chains, and panel lights). The grids of the floor were mounted on the same plane and were spaced 1.6 cm apart (center-to-center) and there were no right levers in these chambers (in contrast, existing right levers had been retracted in the other two types of chambers). The chambers were illuminated by one 7.5-W incandescent bulb mounted to the ceiling of the sound attenuation chamber, near the front wall of the chamber. A distinct odor was continuously presented by placing 0.5 ml of vinegar (Heinz, Pittsburgh, PA) in a dish outside the chamber. The three sets of chambers were fully counterbalanced as Contexts A, B, and C. Two new stimuli (Ss) were used in addition to the tone and panel light stimuli used in Experiments 1 and 2. For one additional S, the right panel light flashed at a rate of 2 times per second (.4 s on, alternated with .1 second off). For the other S, a 70 dB(A) intermittent white noise was pulsed 4 times/s (i.e., “click”), through the speaker mounted to the back wall of the sound attenuation chamber. These stimuli were counterbalanced as S3 and S4; the usual tone and panel light were counterbalanced as S1 and S2.

Procedure

Magazine training

On the first day of the experiment a single session of magazine training was conducted in each of the three contexts. The method was the same as in the previous experiments. All rats received a session in Context A then B then C.

Acquisition

The next day, rats received two sessions of free-operant training in Context A. One session involved training with the lever and the other involved training with the chain (R1 and R2 counterbalanced). On the subsequent day, each rat received two-sessions of free-operant training again, one in Context B with R1 and one in Context C with R2. During each 30-min training session, responding was reinforced on a random interval (RI) 30-s schedule, programmed in the same way as in Experiments 1 and 2.

Over the next 12 days the rats received four sessions of discrimination training each day. On any day, there were two sessions in Context A, and one session each in Contexts B and C. In each Context B session, all rats received 16 trials in which R1 was only reinforced during the 30-s presentation of S3 (flashing light or click, counterbalanced). In each session in Context C, all rats received 16 trials per session in which R2 was only reinforced during the 30-s presentation of S4 (click or flashing light, counterbalanced). In one of the sessions in Context A, the rats received 16 trials in which R1 was only reinforced during the 30-s presentation of S1 (panel light or tone, counterbalanced); in the other A session, the alternative response (R2) was reinforced during the presentation of S2 (tone or panel light, counterbalanced). Thus, each day each rat received training of S1R1 and S2R2 in A, S3R1 in B, and S4R2 in C. As usual, during each S, responding was reinforced on a random interval (RI) 30-s schedule. The ITI was variable, averaging 30 s on Day 1, 60 s on Day 2, and 90 s in all sessions thereafter. Daily sessions were separated by approximately 1.5 hrs and daily training order for each stimulus (panel light, flashing light, tone, or click) was balanced in a quasi-random fashion. The 12 days of training were separated into 3, 4-session “blocks,” in each of which each stimulus was trained once as the first, second, third, or fourth stimulus on a given day.

Extinction

On each of the next 4 days, rats received extinction of each S/response combination in the context in which it had been trained. Extinction sessions were identical to acquisition sessions except that responses did not result in food pellets. There were four extinction sessions per day, separated by approximately 1.5 hrs with the order of context exposure balanced over days (using the same method as acquisition).

Renewal test

Two renewal tests were then conducted on the final two days. For the first test, all rats were tested in Context B (renewal) and A (extinction) with S1R1 and S2R2. Each rat received two 8-min test sessions in each context, separated by approximately 30 min, resulting in 4 total test sessions. In each test session, only one of the lever or chain was available and no reinforcers were delivered. There were 4 presentations of S during each session with a variable ITI averaging 90 s. Half of the rats were first tested in the acquisition and then the extinction context for one S/response combination (e.g., S1R1), and subsequently tested in the acquisition followed by the extinction context for the other S/response combination (e.g., S2R2). The other half of rats were first tested in the extinction followed by the acquisition context for one S/response combination, and then tested in the extinction followed by the acquisition context for the other S/response combination. The identity of the S (light or tone) and response (lever press or chain pull) was fully counterbalanced throughout this test procedure. The result of this design was that on the first test day both target S/response combinations were tested in the context in which they had been extinguished (Context A) and in the context that S3R1 had been extinguished (Context B). On the second test day, following the same procedure, S1R1 and S2R2 were tested in Contexts A and C (where S4R2 had been extinguished). Note that in each test, one stimulus/response combination was tested in a context in which it had previously been inhibited and the other was tested in a context in which it had not.

Results

Acquisition and Extinction

Elevation scores

The data from acquisition and extinction are presented in Figure 5. The figure shows the mean elevation scores in each session separately for each S/response combination. There was a clear increase in discriminative responding over acquisition (left side of the figure), followed by a decrease in responding during extinction (right side of the figure). For acquisition, a 4 (S/response combination) × 12 (Session) ANOVA revealed a main effect Session, F(11, 253) = 82.53, MSE = 17.62, η_p² = .78, CI [.73, .81], p < .001, and an interaction between Session and S/response combination, F(33, 759) = 1.73, MSE = 5.30, η_p² = .06, CI [.00, .07], p < .01. The main effect of S/response combination was not significant, F(3, 69) = 1.1. The effect of S/response combination was also not significant on the final day of acquisition, F(3, 69) = 1.45.

Figure 5.

Results of Experiment 3. Mean elevation scores, for S1R1, S2R2, S3R1, and S4R2 during each 30-min session of acquisition (left) and extinction (right).

Over the course of extinction, a 4 (S/response combination) × 4 (Session) ANOVA revealed effects of Session, F(3, 69) = 119.67, MSE = 8.20, p < .001, η_p² = .84, CI [.76, .86], p < .001, as well as a significant interaction between Session and S/response combination, F(9, 207) = 5.60, MSE = 3.83, p < .001, η_p² = .20, CI [.08, .26], p < .001. Inspection of Figure 5 suggests that during the first session of extinction there was a more rapid loss of responding for S3R1 and S4R2 compared to S1R1 and S2R2. This may have been due to the fact that the panel light and tone (which were counterbalanced as S1 and S2) happened to be the first two stimuli to be extinguished during the first day of extinction training. There was a significant effect of S/response combination on the first session of extinction, F(3, 69) = 4.72, MSE = 15.0, η² = .17, CI [.02, .30], p < .01, due to the higher levels of responding on S1R1 and S2R1. However, the effect of S/response combination was not significant during the final three sessions of extinction. Thus, on the final day of extinction, responding was not different across S/response combinations.

Pre-S responding

A parallel analysis was conducted on pre-S scores. Over the course of acquisition, a 4 (S/response combination) × 12 (Session) ANOVA revealed a main effect Session, F(11, 253) = 31.69, MSE = 10.61, η_p² = .58, CI [.49, .62], p < .001, and an interaction between Session and S/response combination, F(33, 759) = 2.54, MSE = 5.58, η_p² = .10, CI [.03, .10], p < .01. The main effect of S/response combination was not significant, F < 1. The effect of S/response combination was also not significant on the final day of acquisition, F < 1. On the first day of acquisition, pre-S responding averaged 6.9, 6.9, 8.9, and 9.8, for S1R1, S2R2, S3R1, and S4R2, respectively. By the final day of acquisition, pre-S responding averaged 5.0, 5.8, 4.7, and 5.2, for S1R1, S2R2, S3R1, and S4R2, respectively

Over the course of extinction, a 4 (S/response combination) × 4 (Session) ANOVA revealed effects of Session, F(3, 69) = 44.45, MSE = .99, p < .001, η_p² = .66, CI [.51, .73], p < .001, as well as a significant interaction between Session and S/response combination, F(9, 207) = 2.81, MSE = 3.56, p < .001, η_p² = .13, CI [.01, .16], p < .001. During the first session of extinction, pre-S responding averaged 2.1, 2.3, 1.2, and 1.1, for S1R1, S2R2, S3R1, and S4R2, respectively. During the final session of extinction, pre-S responding averaged 0.1, 0.2, 0.3, and 0.2, for S1R1, S2R2, S3R1, and S4R2, respectively.

Renewal Test

Elevation scores

The elevations scores from the renewal test are presented in Figure 6. They are averaged over the two renewal tests comparing A – B and A – C. (An initial ANOVA revealed no significant interactions between the Test factor and any other factors.) Elevation scores are presented for responding to S/response combinations in their extinction and renewal contexts depending on whether the response being tested had previously been extinguished in the renewal context (Same) or a different context (Diff). Inspection of Figure 6 suggests that AAB renewal was observed when a different response was previously extinguished in the renewal context (Diff), but not when the same response was previously extinguished in the renewal context (Same).

Figure 6.

Results of Experiment 3 renewal test. Mean elevation scores during each test session. “Same” = same response extinguished in both extinction and renewal context. “Diff” = different response extinguished in the extinction and renewal context.

This impression was confirmed by a 2 (Context: Ext vs. Renew) × 2 (Response: Same vs. Diff) ANOVA which revealed a main effect of Context, F(1, 23) = 4.64, MSE = 0.84, η_p² = .17, CI [.00, .41], p < .05 as well as an interaction between Context and Response, F(1, 23) = 9.38, MSE = 0.83, η_p² = .30, CI [.03, .52], p < .01. The main effect of Response was not significant, F(1, 23) = 2.7. Simple effect tests revealed that the effect of Context (Extinction vs. Renewal) was significant for the Diff response, F(1, 23) = 8.74, MSE = 1.30, η² = .28, CI [.02, .51], p < .01, but not for the Same response, F < 1. Thus, prior extinction of the same response in the renewing context abolished the AAB renewal effect. Moreover, the effect of Response (Same vs. Diff) was significant in the renewal context, F(1, 23) = 7.96, MSE = 1.15, η² = .26, CI [.02, .49], p = .01, but not in the extinction context, F(1, 23) = 1.69. This result confirms that responding in the renewal context was significantly greater for the Diff response than for the Same response.

Descriptively, of the 24 possible renewal effects, 17 were observed for Diff R, and only 12 were observed for Same R. Further, 18 of 24 rats had a mean elevation score in Ren-Diff that was higher than their mean elevation score in Ren-Same.

Pre-S responding

A similar pattern was observed for pre-S responding, which was also under the influence of extinction at the start of renewal testing. Pre-S responding averaged 0.3, and 0.3, for the Same response in the extinction and renewal context, respectively. Pre-S responding averaged 0.3, and 1.1, for the Diff response in the extinction and renewal context, respectively. A 2 (Context: Ext vs. Renew) × 2 (Response: Same vs. Diff) ANOVA revealed a main effects of Context, Response, and a significant interaction between the two, F(1, 23) = 8.93, MSE = 0.48, η_p² = .28, CI [.03, .51], p < .01. Simple effects revealed that the effect of Context was significant for the Diff response, F(1, 23) = 11.36, MSE = .68, η² = .33, CI [.05, .55], p < .01, but not for the Same response, F < 1. The effect of Response was significant in the renewal context, F(1, 23) = 8.15, MSE = .95, η² = .26, CI [.02, .50], p = .01, but not in the extinction context, F < 1. Thus, renewal was observed in pre-S responding for the Diff response, but not for the same response. This is consistent with the findings from Experiments 1.

Also consistent with the findings from Experiment 1, the renewal observed in pre-S responding was confined to the early part of the test session. During the final two-trial block, pre-S responding for Same averaged 0.16 and 0.27 in the extinction and renewal contexts, F < 1 and 0.13 and 0.42 for Diff in the two contexts, F(1, 23) = 3.46. Importantly, pre-S responding for Diff and Same were not reliably different in the renewal context at this time, F < 1. However, as in Experiment 1, although pre-S responding was not significantly different during the final two-trial block, renewal was still strongly observed in the elevation scores. Elevation scores for the Same response in the extinction (M = 0.71) context did not differ from the renewal context (M = 0.30), F(1, 23) = 3.08. However, Diff responding was higher in the renewal (M = 1.28) than extinction (M = 0.32) context, F(1, 23) = 6.20, MSE = 1.78, η² = .21, CI [.00, .45], p = .02. Further, in the renewal context, Diff responding (M = 1.28) was significantly higher than Same responding (M = 0.30), F(1, 23) = 8.57, MSE = 1.34, η² = .27, CI [.02, .50], p < .01. Thus, when the pre-S baselines were not significantly different, elevation to the S showed more renewal when a different response was extinguished in the renewal context relative to the same response.

Discussion

When rats were tested with a S/response combination in a context that differed from the one in which it had been conditioned and extinguished, an AAB renewal effect was observed. However, the main finding from Experiment 3 was that renewal was eliminated when the discriminated operant was tested in a context that was previously associated with extinction of the same response (occasioned by a different S). This result is consistent with the hypothesis that during extinction of instrumental behavior, an inhibitory association is formed between the context and the response (e.g., Todd, 2013). A direct inhibitory association between the context and response should reduce renewal of discriminated operant responding, even if the common response was occasioned by a discriminative stimulus that was not extinguished in the renewal context. Renewal was also evident in pre-S responding, which was also observed in Experiment 1, and is consistent with the results from free-operant procedures (e.g., Todd, 2013).

The observation of AAB renewal is itself noteworthy. To our knowledge this is the first demonstration of AAB renewal after extinction of a discriminated operant. Further, the results are not readily explained by differential context-reinforcer associations. Although in the current experiment the overall histories of the extinction and renewal contexts were not perfectly equated (there was twice as much conditioning and extinction in A relative to B or C), an explanation of the renewal observed here that relied solely upon differential context-reinforcer associations could not account for the different amounts of renewal observed in the Same and Diff conditions. Thus, like ABA renewal, it is unlikely that AAB renewal is solely dependent upon removal from a context that directly inhibits the representation of the reinforcer.

Finally, in the present experiment renewal was completely eliminated in the Same condition. This is in contrast to the findings from Experiment 2. One possible reason for this difference is that the overall rate of responding during testing in Experiment 3 (AAB renewal) was lower than the rate of responding during testing observed in Experiment 2 (ABA renewal). This may have influenced the ability to detect any residual renewal. However, regardless of the presence or absence of renewal in the Same condition, the important result was that renewal was weaker in the Same than in the Diff condition. That result is consistent with the idea that extinction involves learning to inhibit a particular response in the context of extinction.

General Discussion

The present experiments confirmed that discriminated operant responding is renewed when the context is changed after extinction. In Experiments 1 and 2, instrumental responding in the presence of a discriminative stimulus (S) was renewed when the S and its response were removed from the context of extinction and tested in the original context of acquisition (ABA renewal). In Experiment 3, such responding was also renewed when testing occurred outside the context in which it was conditioned and extinguished (AAB renewal). Renewal occurred even though the contexts were equated for their reinforcement and nonreinforcement histories (Experiments 1 and 2). Thus, like renewal of free-operant behavior (see Todd, 2013), renewal of discriminated operant performance is not merely due to differences in the contexts’ direct associations with the reinforcer.

Experiment 3 further suggested that an inhibitory association may be formed between the context and the response in operant extinction (see Todd, 2013). In that experiment, AAB renewal was observed when both the extinction and renewal contexts were associated with extinction of separate S/response combinations. Importantly, however, stronger responding was observed during renewal testing if the response extinguished in the renewal context was different from the one being tested there. When the response tested in the renewal context was the same as the one extinguished there, renewal was eliminated. Thus, previous inhibition of the target response in the renewal context attenuated subsequent renewal. When it occurs, renewal of extinguished instrumental behavior can thus be seen at least partly as a release from context-specific response inhibition.

It is important to note that, as expected from the perspective of context-response inhibition, responding during the absence of S (i.e., in the pre-S period) showed a similar pattern of results (Experiments 1 and 3). During acquisition, some pre-S responding was acquired, although its level was substantially less than that during S. During extinction, pre-S responding then diminished, and it was then renewed (released from inhibition) when it was tested in a different context in either the ABA or AAB arrangement. The findings are consistent with the results of Todd (2013), who also found that extinguished responding in the presence of the context alone is renewed when testing occurs in a different context. In fact, the observed differences in pre-S responding are consistent with, and indeed necessary to infer, the presence of direct context-response inhibition. According to this mechanism, inhibition learned in the extinction context would act directly upon the response, even in the absence of the S. We should emphasize that renewal of responding in the pre-S period does not challenge our interpretation of renewal of discriminative responding that was observed during the discriminative stimulus itself. The dependent measure used to index discriminative responding, the elevation score, is designed to control for pre-S responding by subtracting it from responding during the S. In all three experiments, discriminative responding in the S (as indexed by elevation scores) showed renewal over and above renewal observed in the pre-S period. Furthermore, late in the testing session (the final 2-trial block), pre-S responding did not differ between the extinction and renewal contexts, although responding did differ in the elevation scores (Experiments 1 and 3). Thus, renewed responding represented by an increase in the elevation evoked by S was not an artifact of differences in responding in the pre-S period. The finding that responding in the presence or absence of the S was diminished because of prior extinction of the same response in the renewal context indicates that inhibition between the context and the response can act in the presence or absence of the S.

The present results complement and extend recent findings of renewal after extinction of free-operant behavior (Todd, 2013). Todd suggested that extinction of free instrumental behavior may be best characterized by a simple, inhibitory association between the context and the response instead of a more hierarchical occasion-setting alternative in which the context modulates an inhibitory association between the context and the response. These mechanisms are illustrated, respectively, in Panels a. and b. of Figure 7. The main reason Todd’s results favored the inhibitory context-response association was that he did not observe transfer of negative occasion setting by the extinction context to a response that had been extinguished in a different context (Todd, 2013, Experiment 4). Because this type of transfer is considered a characteristic of negative occasion setters (e.g., Holland, 1992; Holland & Coldwell, 1993; Morell & Holland, 1993), the failure to detect it was taken as evidence against the hierarchical account. The results of the present Experiment 3 extended this hypothesis by producing positive confirmation of context-specific response inhibition. As noted above, in Experiment 3, renewal was reduced when the renewing context had been associated with prior inhibition of the same response.

Figure 7.

Roles for the context in extinction of free-operant and discriminated operant behavior. Arrows represent excitatory associations, “flat” lines represent inhibitory associations. “S” = discriminative stimulus, “R” = response, “O” = reinforcing outcome. See text for details.

The results thus favor the simple, binary association between the context and the response that is depicted in Figure 7 (Panel a.), instead of the more hierarchical mechanism (Panel b.). The hierarchical mechanism was rejected on the basis of Todd’s (2013) assumption that the power of negative occasion setters should transfer across suitable targets (e.g., Holland, 1992) (perhaps consistent with the relative ineffectiveness of extinguishing the different response in reducing renewal in the present Experiment 3). However, one could possibly claim that transfer of negative occasion setting might depend on the physical similarity between the trained and tested targets, and hence, that the failure of extinction with a different response in the renewal context to transfer and decrease renewal in Todd (2013) and perhaps Experiment 3 was due to a failure to generalize sufficiently between R1 and R2. However, there is no evidence that transfer of occasion setting requires physical similarity between targets; transfer has routinely been observed with physically-different targets provided they are both trained in a potentially occasion-setting relation. Moreover, Holland and Coldwell (1993) discussed and rejected a role for generalization between target responses based on the results of a previous study of transfer in negative occasion setting of operant behavior. The data reported here are most simply and parsimoniously explained by a direct inhibitory association between the response and the context. To state it less formally, in extinction the organism learns not to make a specific response in a particular context (e.g., Colwill, 1993).

In Experiment 2, a modest amount of renewal still occurred when the same response had been extinguished in the renewing context prior to renewal testing. In contrast, renewal was completely abolished in Experiment 3 when the extinction and renewing contexts inhibited the same response. The source of this difference is not clear, although as described earlier, the lower rate of test responding in Experiment 3 may have made it more difficult to detect residual renewal. Nevertheless, it is important to consider the possible mechanisms of renewal in Experiment 2. The result suggests that, in at least some cases, another mechanism in addition to an inhibitory context-response association might play a role in the renewal of discriminated-operant responding. It is therefore worth considering some of the associative structures that previous research suggests might underlie discriminated operant learning (e.g., Rescorla, 1991). The renewal observed in Experiment 2 could be due to the release from context-specific inhibition of any of the associative relations between S, O, and R that are sketched in Panel c of Figure 7. The first mechanism (1.) involves a direct association that is learned between the S and the reinforcing outcome during conditioning (Colwill & Rescorla, 1988). In this case, the S might be thought of as a conditioned stimulus that motivates instrumental behavior during acquisition (Rescorla & Solomon, 1967). Extinction learning might give the context the ability to inhibit or modulate that relation. This possibility might be especially attractive because it is consistent with a well-accepted model of extinction of Pavlovian conditioned stimuli (e.g., Bouton, 2004; Bouton & Ricker, 1994). The second possible mechanism (2.) involves an association between the S and the response (see Rescorla, 1993, 1997). During extinction, any inhibition that develops between S and R, perhaps analogous to the inhibition suggested here between the context and R, could be further modulated by the context. A third possible additional mechanism (3.) involves an associative structure that includes the S, the response (R), and the reinforcing outcome (O). Following acquisition the S might set the occasion for the response-reinforcer association (Colwill & Rescorla, 1990). Subsequently, extinction might somehow allow the context to modulate this three-term contingency. As an example, the extinction context might control the S’s ability to set the occasion for the response-reinforcer association.

Any of the three inhibitory mechanisms illustrated in Panel c would allow the S to elevate responding when tested in a non-extinction context. Further, it should be noted that the final two mechanisms might alone be capable of explaining the data from Experiment 3, independent of a direct inhibitory context-response association, if it is assumed that the inhibitory power of the extinction context generalizes to other S/R/O combinations based on a shared response (R). For example, if one extinction context modulated a three-term contingency including S, R, and O, renewal in that context might be reduced to a different S with the same R because of generalization mediated by the common R. However, this approach cannot capture all of the current data. As described earlier, responding in the absence of S (i.e., during the pre-S period) also showed extinction and renewal, indicating that the context must exert a direct inhibitory influence on the R without including S. Thus, the overall pattern of results may require at least two mechanisms of context inhibition: one that directly inhibits the R, as in panel “a” of Figure 7, and one that includes S, as in panel c.

In summary, the present experiments demonstrated both ABA and AAB renewal can occur after the extinction of discriminated operant behavior. Similar to the results of free-operant experiments (Todd, 2013; see also Cohen-Hatton & Honey, in press), the renewal effects reported here are not attributable to differential context-reinforcer associations in the renewal and extinction contexts. The present experiments also tested further implications of the idea that context-specific response inhibition develops during extinction. In Experiment 3, renewal was reduced when the renewing context had been associated with extinction of the same response, confirming a prediction of the inhibitory context-response account of instrumental extinction. However, renewal was still observed in Experiment 2 when the same response was previously extinguished in both the renewal context. This suggests the possible influence of an additional second mechanism, such as one of those described. A complete understanding of renewal after the extinction of discriminated-operant performance warrants further investigation.

Lastly, it is worth commenting on the clinical application of these findings. It is commonly believed that basic learning processes are involved in several forms of human psychopathology (see Haselgrove & Hogarth, 2012). Moreover, many problematic behaviors, such as drug addiction and overeating, clearly involve instrumental learning (e.g., Everitt & Robbins, 2005; Bouton, 2011). Instrumental extinction is a procedure that can be used to reduce these behaviors. However, as the current experiments have shown, any suppression of these behaviors via extinction may be context-specific and ultimately vulnerable to relapse following context changes. However, the findings of Experiment 3 suggest that protection from renewal, and the efficacy of clinical treatment, may be increased if a special emphasis is placed on extinction (inhibition) of the problematic response in a renewing context. For example, the results of Experiment 3 suggest that renewal of a discriminated operant can be reduced if the response is first inhibited in the renewing context in the presence of a separate discriminative stimulus. In instrumental extinction, extinction and inhibition of the specific response may be important even when it is occasioned by a different S.