Mechanisms of renewal after the extinction of instrumental behavior

Abstract

Four experiments with rats examined renewal of extinguished instrumental behavior when the reinforcement histories of the contexts were equated by giving complementary training and extinction of a different response (lever press and chain pull) in each context. In Experiments 1–3, renewal occurred when the response was tested in the acquisition context (ABA) or outside the extinction context (AAB and ABC). Further, in Experiments 1–3, when both responses were simultaneously available there was a clear preference for the response that was not in its extinction context. In Experiment 4, renewal was not reduced when testing occurred in a context that had been associated with extinction of the other instrumental response. The experimental designs rule out differential context-reinforcer associations being the only contributing mechanism of renewal, and also raise questions about configural and occasion-setting accounts. The results are consistent with the idea that during extinction an inhibitory association is formed between the context and the response.

Behavior that has been acquired either through Pavlovian or instrumental conditioning can be reduced by no longer presenting the reinforcer. This procedure is known as extinction (see Mackintosh, 1974; Pavlov, 1927). For example, after initial Pavlovian conditioning where the conditioned stimulus (CS) is repeatedly paired with the unconditioned stimulus (US), the conditioned response (CR) can be subsequently reduced by repeatedly presenting the CS alone. One approach to extinction assumes that the CR declines due to a reduction in the previously learned association between the CS and US (Rescorla & Wagner, 1972; cf. Pearce & Hall, 1980; Wagner, 1981). That is, the previous association is unlearned. However, there are a variety of empirical effects, particularly from research on Pavlovian conditioning, that suggest extinction does not erase the original learning (e.g., Bouton, 2002). Instead, extinction seems to result in new learning that is especially context-dependent (Bouton, 2002, 2004; Bouton, Westbrook, Corcoran, & Maren, 2006).

One line of support for the notion that extinction results in new, context-dependent learning comes from studies of the renewal effect (e.g., Bouton & Bolles, 1979). In Pavlovian conditioning, renewal is the return of responding to a previously extinguished CS when that CS is tested in a context that is different from the extinction context. (In animal experiments, contexts are usually defined as the chambers in which conditioning occurs; they typically differ in their olfactory, tactile, and visual characteristics). For example, after initial CS-US pairings in one context (Context A), and extinction trials (CS alone) in a different context (Context B), conditioned responding will return (renew) when the CS is once again presented in Context A (ABA renewal; Bouton & Bolles, 1979; Bouton & King, 1983; Bouton & Peck, 1989). Renewal also occurs if, after conditioning in Context A and extinction in Context B, testing occurs in a relatively novel, and neutral, third context (Context C, ABC renewal; Bouton & Bolles, 1979; Thomas, Larsen, & Ayres, 2003). Finally, there is evidence of renewal when conditioning and extinction both take place in Context A and testing occurs in Context B (AAB renewal; Bouton & Ricker, 1994; Laborda, Witnauer, & Miller, 2011; Tamai & Nakajima, 2000). AAB and ABC renewal suggest that removal from the extinction context alone is sufficient for extinguished responding to renew. Overall, these three forms of renewal (ABA, AAB, and ABC) indicate that post-extinction performance is at least partly dependent on the context.

There are several behavioral mechanisms that can give rise to renewal effects (see Bouton, 1993, 1994a; 1997; Laborda, McConnell, & Miller, 2011; Nelson, Sanjuan, Vadillo-Ruiz, Pérez, & León, 2011). For example, in ABA renewal, following excitatory conditioning in Context A, both the CS and the context can acquire separate excitatory associations with the US. During extinction in Context B, the CS is presented alone and the absence of the US can result in an inhibitory association between the context and the US. This inhibitory context-US association summates with the excitation of the CS resulting in a reduction of the CR. Importantly, this inhibitory association also protects the CS from full extinction (Lovibond, Davis, & O’Flaherty, 2000; Rescorla, 2003). Thus, when the CS is once again presented in Context A, residual excitation from both Context A and the CS manifests in a recovery of responding. This mechanism, in which the context forms a direct inhibitory association with the representation of the reinforcer, is predicted by elemental models of associative learning (e.g., Rescorla & Wagner, 1972; Wagner, 1980). An inhibitory context-US association can also contribute to ABC and AAB renewal.

A second possibility is that during Pavlovian extinction the context becomes a signal that modulates the original CS-US association. This type of learning is known as “occasion setting” (Holland, 1992). In occasion setting, one stimulus (in this case the context) modulates the ability of a second stimulus (the CS) to elicit a response. According to one version of this occasion-setting mechanism, during extinction the CS maintains its excitatory association with the US, but a new inhibitory association is also formed between the CS and US. Importantly, this inhibitory association is modulated by the context, meaning that it requires that the extinction context be present (Bouton, 1994b; 1997). Renewal occurs when testing takes place outside of the extinction context because the inhibitory association no longer receives this input. In this case, the extinction context is considered a “negative occasion setter”. This occasion-setting mechanism is supported by studies that have failed to detect any general inhibitory properties of the extinction context, which should be present if the context had formed a direct inhibitory association with the US (e.g., Bouton & King, 1983; Bouton & Swartzentruber, 1986).

Contemporary research on renewal has focused primarily on Pavlovian extinction. It is at least as important to understand the processes of instrumental extinction, however, because instrumental learning has a role in several types of unwanted behaviors, such as addiction (e.g., Conklin & Tiffany, 2002; Everitt & Robbins, 2005) and overeating (e.g., Bouton, 2011). Instrumental extinction is one procedure that can be used to reduce these behaviors. Although there is a growing parallel between Pavlovian and instrumental extinction, until recently the full extent of the parallel has been unclear. This has been true because while several reports have demonstrated instrumental ABA renewal with either food or drugs as the reinforcer (Bossert, Liu, Lu, & Shaham, 2004; Chaudri, Sahuque, & Janak, 2009; Crombag & Shaham, 2002; Hamlin, Clemens, & McNally, 2008; Hamlin, Newby, & McNally, 2007; Nakajima, Tanaka, Urushihara, & Imada, 2000; Welker & McAuley, 1978; Zironi, Burattini, Aircardi, & Janak, 2006), some reports have failed to demonstrate AAB renewal (see Bossert et al., 2004; Crombag & Shaham, 2002; Nakajima et al., 2000). Further, although Zironi et al. (2006) successfully demonstrated ABC renewal, unfortunately, in their experiment the test context was not counterbalanced, making interpretation difficult. Moreover, the effect was present when lever pressing was reinforced with sucrose, but absent when the reinforcer was ethanol.

A recent series of experiments has helped clarify this issue by demonstrating all three forms of renewal (ABA, AAB, and ABC) after extinction of instrumental responding (Bouton, Todd, Vurbic, & Winterbauer, 2011). In one experiment, rats were first trained to lever press for food pellets in Context A. One group of rats then received extinction (where lever presses no longer resulted in pellet delivery) in Context B and one group in Context A. Finally, both groups of rats were tested in both Context B and Context A (test order was counterbalanced). The results clearly showed both ABA and AAB renewal. Thus, for rats that received extinction in Context B, responding renewed when they were returned to the context of acquisition (ABA renewal). For rats that received extinction in Context A, responding also renewed when they were removed from the extinction context and tested in Context B (AAB renewal). In a separate experiment it was demonstrated that after acquisition in Context A and extinction in Context B, responding renewed when testing occurred in a third, relatively novel context (Context C, ABC renewal). Coupled with ABA renewal, these demonstrations of both AAB and ABC renewal suggest a strong parallel between Pavlovian and instrumental extinction. In both cases, removal from the context of extinction is sufficient for renewal to occur. This, in turn, suggests that extinction of instrumental behavior results in new learning that is more context-dependent than the original learning.

The empirical demonstrations of renewal after instrumental extinction (especially AAB and ABC renewal), suggest that the context of extinction somehow inhibits responding instead of erasing the originally acquired association. There are, however, several mechanisms that might be operating to produce this effect (see Bouton et al., 2011). One possibility is that the extinction context enters into a direct inhibitory association with the representation of the reinforcer (Laborda, McConnell, & Miller, 2011), as described above. Mechanistically, during extinction the response predicts that the reinforcer will occur, but when the reinforcer is not delivered the extinction context forms a direct inhibitory association with the representation of the reinforcer. Second, it is also possible that the context becomes a negative occasion setter (also described above) for the response-reinforcer relationship. In this case, instead of directly inhibiting the reinforcer representation, the context signals that the response will not be followed by the reinforcer. A third possible mechanism is that the context might directly inhibit the response (Rescorla, 1993, 1997). For example, Rescorla (1993, Experiment 2) first trained two stimuli (light and noise) to signal that each of two responses (lever press or chain pull) would be reinforced. Each response was then extinguished in the presence of only one stimulus. Subsequent testing, where all four possible stimulus/response combinations were presented, indicated that responding was most depressed in the presence of the stimulus with which it had been extinguished. Importantly, although such results are compatible with the idea that an inhibitory association is formed between the stimulus and the response, they are equally compatible with the negative occasion-setting mechanism described above (Rescorla, 1993, p. 335; 1997, p. 249; Bouton, 2004).

The aim of the present experiments was to determine which behavioral mechanism best describes renewal following instrumental extinction. To that end, the experiments drew from an experimental design that has been used to study renewal in Pavlovian conditioning (see Bouton & Ricker, 1994; Harris, Jones, Bailey, & Westbrook, 2000; Lovibond, Preston, & Mackintosh, 1984; Rescorla, 2008). The design controls and equates the associative histories of the extinction context and the testing (renewal) context. For example, in a pigeon autoshaping experiment, Rescorla (2008, Experiment 1a) first conditioned stimulus X in Context A and stimulus Y in Context B. This procedure ensured that both contexts had a stimulus reinforced in their presence. Then, X was extinguished in B and Y was extinguished in A. This symmetrical treatment ensured that both contexts had a stimulus nonreinforced in their presence. Finally, X and Y were tested in both Contexts A and B. The results clearly demonstrated a strong renewal effect: Responding to each stimulus was high in its renewal context, but low in its extinction context. Note again that this procedure controlled for both the conditioning and extinction histories of each context (i.e., both A and B had a stimulus conditioned and then extinguished in their presence), as well as matching the conditioned stimuli for their histories of reinforcement and nonreinforcement.

The current experiments extended previous research on renewal of instrumental behavior by controlling for the overall history, and thus the potential associative characteristics, of the acquisition and extinction contexts. By doing so, they begin to allow for an analysis of the mechanism of instrumental renewal. Specifically, if renewal is observed (ABA, AAB, and ABC), it is unlikely that the extinction context is directly inhibiting the representation of the reinforcer, because in each design both the extinction and renewal contexts are associated with nonreinforcement of a response and should therefore be equally inhibitory. Instead, observation of a renewal effect would suggest that each extinction context is specifically inhibiting the response that was extinguished in it, which would be consistent with either the inhibitory S-R or negative occasion-setting mechanisms. It is important to note that previous research on instrumental renewal (e.g., Bouton et al., 2011) does not choose between either of these theoretical mechanisms. This is because in all experiments, the contexts were differentially associated with conditioning, extinction, or nothing.

Experiment 1

The design of Experiment 1 is presented in Table 1. Rats were first trained to perform one instrumental response (R1; either lever pressing or chain pulling) for food pellet reinforcers in Context A and a different instrumental response (R2; lever pressing or chain pulling) in Context B. Next, R1 was extinguished in Context B and R2 was extinguished in Context A. Each response was then tested in both Contexts A and B using a within-subject procedure. An ABA renewal effect would be present if responding on R1 was high in Context A but low in B, and responding on R2 was high in B but low in A. After initial renewal testing, a second test, in which both responses were simultaneously available for the first time, was conducted in each context. Since the experimental design controlled for the associative histories of the contexts, observation of ABA renewal would suggest that renewal is due to the unique treatment of the particular responses in each context and not due to each context’s direct association with the reinforcer.

Table 1.

Experimental Designs

Group	Acquisition	Extinction	Test 1	Test 2
Experiment 1
ABA	A: R1+	A: R2−	A: R1−; B: R1−	A: (R1− v. R2−)
	B: R2+	B: R1−	B: R2−; A: R2−	B: (R2− v. R1−)
Experiment 2
ABA	A: R1+	A: R2−	A: R1−; B: R1−	A: (R1− v. R2−)
	B: R2+	B: R1−	B: R2−; A: R2−	B: (R2− v. R1−)
AAB	A: R1+	A: R1−	A: R1−; B: R1−	A: (R1− v. R2−)
	B: R2+	B: R2−	B: R2−; A: R2−	B: (R2− v. R1−)
Experiment 3
ABC	A: R1+	B: R1−	C: R1−; B: R1−	B: (R1− v. R2−)
	A: R2+	C: R2−	B: R2−; C: R2−	C: (R2− v. R1−)
Experiment 4
Ext-B	A: R1+	A: R2−	A: R2−; B: R2−	A: R2−; C: R2−
	B: R2+	B: R1−
	C:---	C:---
Ext-C	A: R1+	A: R2−	A: R2−; B: R2−	A: R2−; C: R2−
	B: R2+	B:---
	C:---	C: R1−

Note. A, B, and C refer to contexts. R1 and R2 refer to lever press and chain pull, counterbalanced. + designates reinforcement, − designates nonreinforcement (extinction), --- designates context exposure without the lever or chain present. Parentheses indicate both responses were simultaneously present.

Method

Subjects

The subjects were 16 naïve female Wistar rats purchased from Charles River Laboratories (St. Constance, Quebec). They were between 75 and 90 days old at the start of the experiment and were individually housed in suspended wire mesh cages in a room maintained on a 16:8-h light:dark cycle. Experimentation took place during the light period of the cycle. The rats were food-deprived to 80% of their initial body weights throughout the experiment.

Apparatus

Two sets of four conditioning chambers housed in separate rooms of the laboratory served as two contexts (counterbalanced). Each chamber was housed in its own sound attenuation chamber. All boxes were of the same design (Med Associates model: ENV-008-VP). They measured 30.5 cm × 24.1 × 21.0 cm (l × w × h). In one set of boxes, the sidewalls and ceiling were made of clear acrylic plastic, while the front and rear walls were made of brushed aluminum. The floor was made of stainless steel grids (0.48 cm diameter) staggered such that odd- and even-numbered grids were mounted in two separate planes, one 0.5 cm above the other. A recessed 5.1 cm × 5.1 cm food cup was centered in the front wall approximately 2.5 above the level of the floor. Retractable levers (Med Associates model: ENV-112CM) were positioned to the left and right of the food cup. These levers were 4.8 cm long and positioned 6.2 cm above the grid floor. The left lever protruded 1.9 cm when extended (the right lever remained retracted over the course of the experiment). A response chain (Med Associates model: ENV-111C) was suspended from a micro switch mounted on top (outside) of the ceiling panel. The chain hung 1.9 cm from the front wall, 3 cm to the right of the food cup, and 6.2 cm above the grid floor. Thus, the lever and chain were positioned symmetrically with respect to the food cup. A 28-V panel light (2.5 cm in diameter) was attached to the wall 10.8 cm above the floor and 6.4 cm both to the left and right of the food cup. The chambers were illuminated by one 7.5-W incandescent bulb mounted to the ceiling of the sound attenuation chamber, approximately 34.9 cm from the grid floor at the front wall of the chamber. Ventilation fans provided background noise of 65 dB. This set of boxes had no distinctive visual cues. A dish containing 5 ml of Rite Aid lemon cleaner (Rite Aid corporation, Harrisburg, PA) was placed outside of each chamber near the front wall.

The second set of boxes was similar to the Lemon-scented boxes (e.g., also model ENV-008-VP, placement of levers, chain, panel lights), except for the following features. In each box, one sidewall had black diagonal stripes, 3.8 cm wide and 3.8 cm apart. The ceiling had similarly spaced stripes oriented in the same direction. The grids of the floor were mounted on the same plane and were spaced 1.6 cm apart (center-to-center). The chambers were illuminated by one 7.5-W incandescent bulb mounted to the ceiling of the sound attenuation chamber, near the back wall of the chamber. A distinct odor was continuously presented by placing 5 ml of Pine-Sol (Clorox Co., Oakland, CA) in a dish outside the chamber.

The reinforcer was a 45-mg grain-based rodent food pellet (TestDiet, Richmond, IN, USA). The apparatus was controlled by computer equipment located in an adjacent room.

Procedure

Magazine training

On the first day of the experiment, all rats were assigned to one box within each set. They then received a single 30-minute session of magazine training in each context (A and B). Sessions were separated by approximately 1 hr. In each session, food pellets were delivered freely on a random time 30 s (RT 30s) schedule resulting in approximately 60 pellets being delivered. This schedule was programmed by delivering a pellet in a given second with a 1 in 30 probability. The levers and chains were not present during this training.

Acquisition

On each of the next 6 days, all rats received two daily sessions of response training, one in each context. The sessions were separated by approximately 3 hr. Only one of the lever or chain was available in a particular context. The assignment of the lever and chain response to each context was fully counterbalanced. Thus, R1 (lever or chain, counterbalanced) was trained in Context A and R2 (lever or chain, counterbalanced) was trained in Context B. During each 30-min session, responding was reinforced on a random interval (RI) 30-s schedule programmed by initiating pellet availability in a given second with a 1 in 30 probability. The pellet remained available until the next response, at which point the pellet was delivered and the schedule mechanism restarted. No additional response shaping was necessary. The order of exposure to each context was double alternated, resulting in each context being equally experienced as the first or second session of the day.

Extinction

On each of the next 4 days, rats received extinction of each response in the context that was different from the one in which it was trained. R1 was extinguished in Context B and R2 was extinguished in Context A. Extinction sessions were identical to acquisition sessions with the exception that responses did not result in food pellets. There were two extinction sessions per day, one in each context, separated by approximately 3 hr. Like acquisition, only one of the lever or chain was present at any time, and the order of exposure to each context was counterbalanced across days.

Renewal Test

On the next day, each rat received two 10-min test sessions in each context, separated by approximately 30 min, resulting in 4 total test sessions. In each test session, only 1 of the lever or chain was available and no reinforcers were delivered. Half of the rats were first tested in the acquisition and then the extinction context for one response, and subsequently tested in the acquisition followed by the extinction context for the other response. The other half of rats were first tested in the extinction followed by the acquisition context for one response, and then tested in the extinction followed by the acquisition context for the other response. The identity of the response (lever press or chain pull) was fully counterbalanced throughout this test procedure. The result of the design was that for each animal, each response (lever press and chain pull) was tested in the context in which it had been reinforced and in the context in which it had been extinguished.

Preference Test

On the next and final day of the experiment, each rat was tested in both contexts (order counterbalanced) with both the lever and chain simultaneously available for the first time. Each 10-min test session was separated by approximately 30 min. No reinforcers were delivered during this test.

Data Analysis

The results were evaluated with analysis of variance (ANOVA) using a rejection region of p < .05. Parametric statistics were chosen in order to evaluate critical interactions between factors. For consistency, follow-up pairwise comparisons were also made with parametric statistics. However, although not reported in the text, all of the pairwise comparisons were also significant when assessed with nonparametric statistics.

Results

Initial acquisition and extinction proceeded smoothly. On the final day of acquisition training, the mean number of responses per minute on the lever was significantly higher (27.8) than on the chain (20.1), F(1, 15) = 8.93, MSE = 53.61, p < .01. The course of acquisition and extinction, averaged over the type of response used as R1 and R2, are presented in Figure 1. A 2 (Response) × 6 (Session) ANOVA revealed a main effect of Session, F(5, 75) = 55.87, MSE = 23.17, p < .001. Neither the effect of Response nor the Response × Session interaction were significant, Fs < 1. The initial rise in responding during acquisition was followed by a rapid decrease during extinction (right side of Figure 1). A 2 (Response) × 4 (Session) ANOVA revealed a main effect of Session, F(3, 45) = 151.23, MSE = 2.07, p < .001. No other main effects or interactions were significant, Fs < 1.

Figure 1.

Results of Experiment 1. Mean responses per minute, for R1 and R2 (lever or chain, counterbalanced), during each 30-min session of acquisition (left) and extinction (right).

The data from the renewal test are presented in Figure 2. The figure shows responding for R1 and R2 in Contexts A and B. It is clear that responding on R1 was high in A and low in B, while responding on R2 was high in B but low in A. A 2 (Response) × 2 (Context) AVOVA did not reveal main effects of Response or Context, Fs < 1. However, the interaction between the two was highly significant, F(1, 16) = 29.10, MSE = 3.35, p < .001. Follow-up comparisons revealed that responding on R1 was significantly higher in Context A than in Context B, t(15) = 5.18, p < .001, and responding on R2 was significantly higher in Context B than in Context A, t(15) = 3.54, p < .01.

Figure 2.

Results of the Experiment 1 renewal test. Mean responses per minute during each 10-min test session.

Descriptively, the renewal effect was robust; 29 of the 32 possible renewal effects (2 effects each for 16 subjects) were observed (defined as r > e, where r is the rate of responding in the renewal context and e is the rate of responding in the extinction context).

When both the lever and the chain were simultaneously present for the first time, rats preferred the response that was currently in its renewal context. The mean response rates, averaged across the two tests, were 1.13 (renewal) and 0.37 (extinction). Although the overall response rates were relatively low (likely due to previous extinction testing), there was a clear preference for the response that had been extinguished in the other context, F(1, 15) = 26.48, MSE = 0.19, p < .001. That is, when allowed access to both responses simultaneously, rats preferred the response that was currently in its renewal context. Over the two tests, 15 of 16 rats showed a numerical preference (r > e) for the response that was in the renewal context.

Discussion

Although ABA renewal after instrumental extinction has been demonstrated previously (e.g., Bouton et al., 2011; Nakajima et al., 2000), it has not, until now, been demonstrated under conditions in which both the extinction and renewal contexts have equivalent reinforcement and nonreinforcement histories. After initial training of R1 in Context A and R2 in Context B, R1 was then extinguished in B and R2 in A. At test, responding on R1 renewed in Context A and R2 in Context B. Because the experimental design equated each context on their overall histories, the observed renewal effect could not result from differential context-US associations. Further, when both responses were simultaneously available for the first time during the preference test, the preferred response was the one that was currently in its non-extinction (renewal) context.

It is worthwhile to note that the numerical size of the ABA renewal effect observed is modest. It is important to keep in mind that the experiment was designed to control for factors (i.e., direct context-reinforcer associations) that might otherwise increase the size of the effect. Moreover, the size of the renewal effect is at least partly dependent upon the choice of experimental parameters (Todd, Winterbauer, & Bouton, 2012b). This issue is further discussed in the General Discussion.

The response-specific recovery of responding demonstrated here is consistent with the view that removal of the response from the extinction context results in renewal of responding. However, because the ABA renewal design involves a return to the original acquisition context, it is not clear if removal from the extinction context is sufficient for renewal to occur. Recently, Bouton et al. (2011; see also Todd, Winterbauer, & Bouton, 2012a) have demonstrated AAB renewal following instrumental extinction, suggesting that removal from the extinction context is, in fact, sufficient for a recovery of responding. The second experiment used a design similar to Experiment 1 to test for AAB renewal. It also compared AAB with ABA renewal.

Experiment 2

The design of Experiment 2 is shown in Table 1. Two groups of rats were trained to perform one response (either lever press or chain pull) in Context A and the other response in Context B. One group (Group ABA) then received extinction of each response in the alternative context. This group was a procedural replication of Experiment 1. A second group (Group AAB) received extinction of each response in the context in which it was originally trained. Specifically, after acquisition of R1 in Context A and R2 in Context B, Group AAB received extinction of R1 in Context A and R2 in Context B. Following extinction, renewal testing occurred for both groups. For Group AAB, a renewal effect would be present if responding on R1 was high in B but low in A, and responding on R2 was high in A but low in B. As in Experiment 1, there was a final test in each context where both responses were simultaneously available for the first time.

The design of Experiment 2 was meant to accomplish three goals. First, observation of AAB renewal would suggest that removal from the extinction context is sufficient for renewal to occur, even when contexts are equated for their overall histories. Second, since groups AAB and ABA received extinction in contexts that were either the same as or different from the context of conditioning, it was possible to determine if the context switch after conditioning had an effect on the rate of extinction. Bouton et al. (2011) demonstrated that extinction in Context B is more rapid than extinction in Context A (cf. Bossert et al., 2004; Crombag & Shaham, 2002; Nakajima et al., 2000, Experiment 1). This finding contrasts with research on Pavlovian conditioning, where there is often little effect of a context switch after initial conditioning (e.g., Bouton & King, 1983; Bouton & Peck, 1989). However, in the Bouton et al. (2011) study, Context B was relatively novel and less associated with the reinforcer, which may have contributed to the effect. In the current study, at the time of extinction, both Context A and B were equally familiar, equally associated with reinforcement, and had both had a response trained in their presence. This procedure resulted in a more controlled comparison between extinction in Context A versus Context B. Third, the relative size of ABA and AAB renewal can also be directly compared. In Bouton et al. (2011), ABA renewal was stronger than AAB renewal. But the corresponding groups had experienced very different learning histories. For instance, for Group ABA the renewal context had been extensively associated with reinforcement, whereas for Group AAB it had not. By controlling for the reinforcement histories of the contexts, it might be expected that the ABA and AAB renewal effects would be more similar in size.

Method

Subjects

The subjects were 32 naïve female Wistar rats purchased from the same vendor as those in the previous experiment and maintained under the same conditions.

Apparatus

The apparatus was the same as that in Experiment 1.

Procedure

All the phases were carried out in a manner identical to Experiment 1, except as noted. During extinction, 16 rats (Group ABA) received extinction of each response in the alternative context from the one in which it had been trained. This group provided a replication of the group studied in Experiment 1. The remaining 16 rats (Group AAB) received extinction of each response in the same context in which it had been previously trained. Following extinction, renewal testing occurred. For all subjects, one response was first tested in its renewal and extinction context (order counterbalanced) and then the other response was tested in both contexts. In this experiment, each test session was separated by approximately 1 hr. On the day following renewal testing, preference tests between R1 and R2 were conducted in a manner similar to Experiment 1 with the exception that test sessions were separated by approximately 1 hr.

Results

On the final day of acquisition, the mean number of responses per minute on the lever (31.5) was significantly greater than that of the chain (24.9), F(1, 31) = 15.63, MSE = 44.14, p < .001. The course of acquisition is plotted in the left part of Figure 3, collapsed over the lever and chain as R1 and R2. While responding increased steadily, the groups did not differ during initial acquisition. A 2 (Group) × 2 (Response) × 6 (Session) ANOVA revealed a main effect of Session, F(5, 150) = 102.74, MSE = 23.18, p < .001. No other main effects or interactions were significant, largest F(1, 30) = 2.96, MSE = 114.61, p < .10.

Figure 3.

Results of Experiment 2. Mean responses per minute during each 30-min session of acquisition (left) and extinction (right). For Group ABA, extinction of each response occurred in the context that the other response was trained in. For Group AAB, extinction of each response occurred in the context they were originally trained.

The extinction data are presented in the right part of Figure 3. Extinction proceeded more rapidly for Group ABA than Group AAB. However, by the last extinction session, the groups did not differ. A 2 (Group) × 2 (Response) × 4 (Session) ANOVA revealed a main effect of Session, F(3, 90) = 375.35, MSE = 3.13, p < .001, Group, F(1, 30) = 26.00, MSE = 6.19, p < .001, and a Session × Group interaction, F(3, 90) = 19.20, MSE = 3.13, p < .001. No other main effects or interactions were significant, largest F(1, 30) = 1.46, MSE = 3.38. To identify the source of the Session × Group interaction, separate oneway ANOVAs examined the effect of Group within each Session. Group ABA responded significantly less during extinction sessions 1, 2, and 3, minimum F(1, 30) = 6.32, MSE = 0.47, p = .018. The groups did not differ during the final session of extinction, F < 1.

The data from the renewal test are presented in Figure 4. As shown, both groups increased responding in the respective renewal context relative to the extinction context for each response. A 2 (Group) × 2 (Response) × 2 (Context) ANOVA did not reveal any significant main effects or interactions, largest F(1, 30) = 2.88, MSE = 1.20, p = .10, except for the crucial three-way interaction between Group, Response, and Context, F(1, 30) = 59.93, MSE = 1.68, p < .001. Thus, the level of responding on R1 and R2 depended upon Context, and this overall pattern depended upon Group membership.

Figure 4.

Results of the Experiment 2 renewal test. Mean responses per minute during each 10-min test session. Left panel = Group ABA, middle panel = Group AAB. Right panel = Groups ABA and AAB plotted to align “extinction” and “renewal” contexts.

To assess the source of the three-way interaction, separate 2 (Response) × 2 (Context) ANOVAs were conducted within each Group. For Group ABA (left panel of Figure 4), this analysis revealed a strong Response × Context interaction, F(1, 15) = 46.56, MSE = 1.52, p < .001. Neither the main effect of Response nor the main effect of Context approached significance, Fs < 1. Follow-up comparisons revealed the responding on R1 was higher in A than B, t(15) = 4.64, p < .001, and responding on R2 was higher in B than A, t(15) = 4.71, p < .001.

For Group AAB (middle panel of Figure 4), the parallel analysis revealed a significant main effect of Response, F(1, 15) = 6.15, MSE = 0.69, p = .025, and well as the crucial Response × Context interaction, F(1, 15) = 18.16, MSE = 1.84, p < .01. The main effect of Context was not significant, F < 1. For this group, follow-up comparisons revealed that responding on R1 was higher in B than in A, t(15) = 3.64, p < .01, and responding on R2 was higher in A than in B, t(15) = 4.29, p < .01. Overall, of the 64 possible renewal effects, 59 were observed (2 rats in Group ABA and 3 rats in Group AAB did not show renewal of one of their responses).

To more clearly examine the relative size of the renewal effects between Groups ABA and AAB, the data have been re-plotted in the right-most panel of Figure 4. This panel depicts the data by aligning Context A and B as “Extinction” and “Renewal” contexts for each group. What is clear from this figure is that the overall increase from extinction to renewal appears equal across groups. A 2 (Group) × 2 (Response) × 2 (Context) ANOVA, confirmed this impression, revealing only a significant main effect of Context, F(1, 30) = 59.93, MSE = 1.68, p < .001. No other main effects or interactions were significant, largest F(1, 30) = 2.88, MSE = 1.20, p = .10.

The data from the final preference test are presented in Figure 5. For both groups, there was a clear preference for the response that was currently in its renewal context. A 2 (Group) × 2 (Context) ANOVA revealed a main effect of Context, F(1, 30) = 37.73, MSE = 0.18, p < .001. No other main effects or interactions were significant, largest F(1, 30) = 1.13, MSE = .18, p = .30. Thus, as in Experiment 1, when rats were given access to both responses simultaneously, they preferred the response that was currently in its non-extinction context. Over the two tests, 27 of 32 rats showed a numerical preference (r > e) for the response that was in the renewal context (1 rat in Group ABA and 4 rats in Group AAB did not show the appropriate preference).

Figure 5.

Results of the Experiment 2 preference test. Mean responses per minute, averaged over each 10-min test session in the two contexts. In this test, both the lever and chain were simultaneously available. “Extinction” indicates the response was in the context where it had been extinguished. “Renewal” indicates the response was in the context where it had not been extinguished.

Discussion

Experiment 2 established four findings. First, the finding of ABA renewal replicated the results of Experiment 1. Second, AAB renewal was observed, for the first time, using a procedure that controlled for the reinforcement histories of Contexts A and B. Both ABA and AAB renewal were observed during initial renewal testing, when only one response option was available, and during preference testing when both responses were simultaneously available. During this second test, when given a choice, the preferred response was the one that was in its non-extinction context. Third, extinction occurred faster when it was conducted in a context different from the one in which it was trained: This finding replicates and extends the findings of Bouton et al. (2011) by showing that such a result occurs when the associative histories of the contexts are controlled. Finally, when the associative histories of the contexts are equated, the sizes of ABA and AAB renewal are remarkably similar.

The finding of AAB renewal implicates an inhibitory mechanism during extinction (Bouton et al., 2011), because removal from the extinction context is sufficient for responding to recover. However, the current results are not readily attributable to the extinction context forming a direct inhibitory association with the representation of the reinforcer. Given that both contexts (extinction and renewal) were equally associated with nonreinforcement, if the contexts were inhibitory then responding would be suppressed in both. While it is unlikely that renewal in this experiment is due to the extinction context becoming a conditioned inhibitor, either the inhibitory S-R or negative occasion-setting mechanisms (described above) are still able to account for the current data. According to the inhibitory S-R mechanism, each context would uniquely inhibit a specific response, and according to the negative occasion-setting mechanism, each context would uniquely signal that a particular response is not followed by the reinforcer. These mechanisms will be separated in Experiment 4.

Experiment 3 tested for ABC renewal after instrumental extinction. Like AAB renewal, ABC renewal also implicates inhibitory learning because renewal testing occurs in a context that has not been previously associated with reinforcement and thus has no basis for excitation.

Experiment 3

Although ABC renewal after instrumental extinction has been previously demonstrated (e.g., Bouton et al. 2011; Todd, Winterbauer, & Bouton, 2012b), it has not been demonstrated under conditions in which both Context B and Context C have equivalent histories and presumably equivalent associative properties. Like AAB renewal, ABC renewal indicates that removal from the extinction context is sufficient for renewal to occur, which suggests that renewal is due to the removal of some inhibitory influence learned during extinction.

The design of Experiment 3 is illustrated in Table 1. Rats were first trained to perform two instrumental responses (R1 and R2) in Context A during separate sessions. Next, R1 was extinguished in Context B, and R2 was extinguished in Context C. During renewal testing, R1 was tested in both B and C, and R2 was tested in both C and B. ABC renewal would take the form of responding on R1 being high in C but low in B, and responding on R2 being high in B but low in C. As in Experiments 1 and 2, a final test was conducted where both responses were simultaneously available for the first time.

Method

Subjects

The subjects were 24 naïve female Wistar rats purchased from the same vendor as those in the previous experiments and maintained under the same conditions.

Apparatus

This experiment employed 3 contexts, two of which were the used in Experiments 1 and 2. The third context was similar to the Pine and Lemon-scented boxes (e.g., also model ENV-008-VP, placement of levers, chains) except for the following features. The grids of the floor were mounted on the same plane and were spaced 1.6 cm apart (center-to-center). The chambers were illuminated by one 7.5-W incandescent bulb mounted to the ceiling of the sound attenuation chamber, near the front wall of the chamber. There were no panel lights or right lever (which remained retracted in the other boxes) on the inside of the chambers. A distinct odor was continuously presented by placing 5 ml of vinegar (Heinz, Pittsburgh, PA) in a dish outside the chamber. The three sets of chambers were fully counterbalanced as Contexts A, B, and C.

Procedure

Magazine training

On the first day of the experiment a single session of magazine training was conducted in each of the three contexts. The method was the same as in the previous experiments.

Acquisition

On each of the next 6 days, all rats received daily sessions of response training with each response (lever and chain) in Context A. The two daily sessions were separated by approximately 2 hours. Only one of the response, lever or chain, was available in any session. As noted, the identity of Context A was fully counterbalanced; that is, subsets of 8 rats received acquisition in the Lemon, Pine, or Vinegar scented boxes. During each 30-min training session, responding was reinforced on a random interval (RI) 30 s schedule. No additional response shaping was necessary. The response training of lever press and chain pull was double alternated, resulting in the training of each response occurring equally during the first or second acquisition session of the day.

Extinction

On each of the next 4 days, rats received two sessions of extinction, one in Context B and the other in Context C. During one session, either the lever press or chain pull response was extinguished and during the other session the remaining response was extinguished. The assignment of responses to contexts was balanced such that the lever and chain were equally represented in each physical chamber. R1 was extinguished in Context B and R2 was extinguished in Context C. Extinction sessions were identical to acquisition sessions with the exception that responses did not result in food pellets. The order of exposure to each context was counterbalanced across days. Like acquisition, these sessions were separated by approximately 2 hrs.

Renewal Test

On the next day, each rat received two 10-min test sessions in each of the extinction contexts (B and C), resulting in four total test sessions (each separated by approximately 30 min). In each test session, only one response (lever or chain) was available and no reinforcers were delivered. Half of the rats were first tested in the extinction and then the alternative context for one response, and subsequently tested in the extinction followed by the alternative context for the other response. The other half of rats were first tested in the alternative followed by the extinction context for one response, and then tested in the alternative followed by the extinction context for the other response. The identity of the response (lever press or chain pull) was fully counterbalanced throughout this test procedure. The result was that for each animal, each response (lever press and chain pull) was tested in the context in which it had been extinguished and the context in which the alternative response had been extinguished.

Preference Test

On the final day of the experiment, each rat was tested in both extinction contexts (order counterbalanced) with both responses (lever and chain) simultaneously available for the first time. Each 10-min test session was separated by approximately 1 hr. No reinforcers were delivered during this test.

Results

On the final day of acquisition, the mean number of responses per minute on the lever (18.6) was not reliably different from the chain (19.8), F < 1. The course of acquisition and extinction, collapsed over lever and chain as R1 and R2, are presented in Figure 6. A 2 (Response) × 6 (Session) ANOVA revealed a significant main effect of Session (left side of Figure 7), F(5, 115) = 107.98, MSE = 11.58, p < .001, and a significant Session × Response interaction, F(5, 115) = 6.54, MSE = 7.82, p < .001. The main effect of Response was not significant, F < 1. On the final day of acquisition, there was no difference in responding between R1 and R2, F(1, 23) = 2.77, MSE = 18.61, p = .11. Initial acquisition was followed by a rapid decrease in responding over the course of extinction (right side of Figure 6). A 2 (Response) × 4 (Session) AVOVA revealed a significant main effect of Session, F(3, 69) = 121.20, MSE = 1.72, p < .001. Although the main effect of Response was not significant, F(1, 23) = 2.72, MSE = 4.88, p = .11, there was an unexpectedly significant Session × Response interaction, F(3, 69) = 3.10, MSE = 1.24, p = .034. Follow-up comparisons revealed that responding on R2 was slightly lower than R1 for extinction session 2, t(23) = 2.54, p = .018, but no differences were detected for extinction sessions 1, 3, or 4, largest t(23) = 1.45, p = .16.

Figure 6.

Results of Experiment 3. Mean responses per minute during each 30-min session of acquisition (left) and extinction (right). Acquisition of R1 and R2 occurred in Context A. Extinction of R1 occurred in Context B and extinction of R2 occurred in Context C.

Figure 7.

Results of the Experiment 3 renewal test. Mean responses per minute during each 10-min test session.

The data from the renewal test are presented in Figure 7. It is clear from the figure that responding in the renewal context was higher than responding in the extinction context for each response. This impression was confirmed with a 2 (Response) × 2 (Context) ANOVA which revealed a strong interaction between Response and Context, F(1, 23) = 21.77, MSE = 5.50, p < .001. Neither the main effect of Response nor Context approached significance, Fs < 1. Responding on R1 was higher in C than in B, t(23) = 3.04, p < .01, and responding in R2 was higher in B than in C, t(23) = 4.09, p < .01. Of the 48 possible renewal effects, 38 were observed.

During the final preference test, the mean response rates were 1.62 (renewal) and 0.64 (extinction). Once again, although the overall response rate was low relative to the previous test, there was a clear preference for the response that was currently in its renewal context relative to the response that was in its extinction context, F(1, 23) = 16.24, MSE = 0.68, p < .001. Over the two tests, 18 of 24 rats showed a numerical preference (r > e) for the response that was in the renewal context.

Discussion

Experiment 3 demonstrated ABC renewal after instrumental extinction. The new finding from this experiment is that renewal was observed when the histories of the test and extinction contexts were equated. Context B and C were equally familiar, had equal histories of nonreinforcement, and were equally associated with extinction of an instrumental response. Like AAB renewal, ABC renewal indicates that removal from the extinction context is sufficient to cause renewal. Further, because there was no return to the original acquisition context (as in ABA renewal), ABC renewal also directly implicates inhibitory learning during extinction. It appears that Context B somehow inhibits responding during extinction, and that this inhibition is not present during testing in C. Just as in the case of AAB renewal, it is unlikely that this mechanism involves the context directly inhibiting a representation of the reinforcer, because both the extinction and renewal contexts shared equal histories of nonreinforcement. Thus, if extinction resulted in the context forming an inhibitory association with the representation of the reinforcer, this association would be present in both contexts and responding would be equally suppressed in B and C during testing. The nature of the inhibition was further examined in Experiment 4.

Experiment 4

The results of Experiments 1–3, which demonstrated ABA, AAB, and ABC renewal when the associative histories of the contexts were controlled, are consistent with two possible learning mechanisms. The first is that during extinction, the context forms an inhibitory association with the response extinguished in that context (Rescorla, 1993, 1997). As an example of how this mechanism could explain renewal after instrumental extinction, take the finding of AAB renewal in Experiment 2. After training of R1 in A and R2 in B, extinction of each occurs in the same context. According to this mechanism, extinction should result in an inhibitory association between Context A and R1, as well as an inhibitory association between Context B and R2. Renewal occurs when R1 is tested in Context B because that context specifically inhibits R2, not R1. The symmetrical argument holds for renewal of R2.

The second possibility is that the extinction context becomes a negative occasion setter and signals that the response will not be followed by the outcome. This type of mechanism has been proposed within the Pavlovian extinction literature (e.g., Bouton & King, 1983; Bouton & Swartzentruber, 1986; Nelson et al., 2011). According to this mechanism, during extinction an inhibitory association is formed between the response and the reinforcer, but this association is modulated by the context. Thus, when testing occurs outside the extinction context, renewal occurs because the inhibitory association is no longer active. To illustrate how this mechanism might operate, take again the AAB renewal observed in Experiment 2. After initial training of R1 in A and R2 in B, each response is then extinguished in its original context. According to the negative occasion-setting mechanism, during extinction of R1 an inhibitory association would form between R1 and the reinforcer, but this association would only be activated by Context A. Further, R2 would also form an inhibitory association with the reinforcer, but this association would be modulated by Context B. Renewal would occur when each response is removed from its respective extinction context because the inhibitory association between the response and the reinforcer (that is modulated by the context) would no longer be active.

One problem from the occasion-setting account, however, is that the effects of negative occasion setters have been shown to transfer to other targets of negative occasion setters (Holland & Coldwell, 1993; Morell & Holland, 1993). This might suggest that the occasion-setting ability of Context A would transfer to responding on R2 during renewal testing and vice versa, resulting in suppressed responding on both R1 and R2 in both contexts. However, with respect to the present experiments, it is important to note that even when transfer of occasion setting occurs, it is usually incomplete (Holland & Coldwell, 1993; Morell & Holland, 1993). Incomplete transfer of negative occasion setting would allow renewal effects in the present designs, even if both contexts acquired occasion-setting properties. Theoretically, the source of this transfer would be based on training history rather than simple generalization between occasion setters or targets. For example, there is generally little transfer of occasion setting to a new target unless the new target has been trained as a target of a different occasion setter (e.g., Holland, 1991; Morell & Holland, 1993).

Experiment 4 was designed to differentiate between the inhibitory S-R and negative-occasion setting mechanisms (see Table 1). Two groups of rats were trained on R1 in Context A and R2 in Context B. Next, for one group (Ext-B) R1 was extinguished in B, and R2 was extinguished in A. For a second group (Ext-C) R2 was also extinguished in A, but R1 was extinguished in Context C instead of B. If extinction results in the context acquiring negative occasion-setting properties, Context B should become a negative occasion setter for Group Ext-B. However, this cannot be true of Context B for Group Ext-C. In a renewal test, R2 was tested in Contexts A (its extinction context) and B (its renewal context). If Context B acquired negative occasion-setting properties, and if occasion setting does transfer to an equivalently trained R, then renewal in Context B should be weaker in Group Ext-B than Group Ext-C. This effect would occur because R2 itself was extinguished in Context A, making it a suitable target for transfer of negative occasion setting.

In contrast, the inhibitory S-R mechanism described above predicts little difference in renewal between the two groups, because during extinction each context would enter into an inhibitory association that is specific to the response that was extinguished in it. Renewal would then occur when that response was tested in a different context, irrespective of whether the renewal context had a different response previously extinguished in it or not.

A second renewal test was conducted with R2 in Contexts A and C. If extinction results in the context acquiring negative occasion-setting properties, then the pattern of results for this test should be much different from the first test. In this case, Group Ext-C should show weaker renewal than Ext-B, because R1 was extinguished in C for this group, potentially allowing Context C to acquire negative occasion-setting properties.

Method

Subjects

The subjects were 24 naïve female Wistar rats purchased from the same vendor as those in the previous experiments and maintained under the same conditions.

Apparatus

This experiment employed the same three sets of chambers used in Experiment 3. The three chambers were fully counterbalanced as Contexts A, B, and C.

Procedure

Magazine training

On the first day of the experiment a single session of magazine training was conducted in each of three contexts. The method was the same as in the previous experiments.

Acquisition

On each of the next 6 days, all rats received two daily sessions of response training, one in Context A (R1) and one in Context B (R2). Only one of the responses, lever or chain, was consistently available in a particular context. The assignment of the lever and chain response to each context was fully counterbalanced. During each 30-min training session, responding was reinforced on a random interval (RI) 30 s schedule. In addition to the two response training sessions, all subjects were exposed to Context C, without either of the responses (lever or chain) available, for 30 minutes each day. The rats were exposed to Context C to ensure equal exposure to all three contexts. As noted, the identity of Contexts A, B, and C, were fully counterbalanced. The daily order of exposure to each context was balanced by using the following sequences: ABC, BAC, ACB, BCA, CAB, CBA for all rats, on Day 1 to Day 6, respectively. Sessions were separated by approximately 1 hr.

Extinction

On each of the next 4 days, rats received two sessions of extinction. During one session, either the lever press or chain pull response was extinguished and during the other session the remaining response was. Extinction sessions were identical to acquisition sessions with the exception that responses did not result in food-pellet delivery. For all subjects, the response that was initially trained in Context B (R2) was extinguished in Context A. At this point, two groups of rats received different treatments. For half the rats (n = 12; Group Ext-B) the response that was initially trained in Context A (R1) was extinguished in Context B. For this group, there was a third session each day in which they were exposed to Context C, with no response manipulandum present, for 30 min. For the second group (n = 12; Group Ext-C), the response that was initially trained in Context A (R1) was subsequently extinguished in Context C. This group received additional exposure to Context B with no manipulandum present. The order of exposure to each context was counterbalanced across days. As in acquisition, these sessions were separated by approximately 1 hr.

Renewal Test 1

On the next day, each rat received two test sessions, one in Context A, and one in Context B, with the order of context exposure counterbalanced. Sessions were separated by approximately 1 hour. In each 10-min test session, only 1 response was available and no reinforcers were delivered. The response tested was the response that initially trained in Context B (R2) and subsequently extinguished in Context A. Thus, it was expected that ABA (BAB) renewal would be observed. Note however, that the history of Context B, and potentially the properties of this context, differed between groups. Group Ext-B had the alternative response extinguished in Context B, but Group Ext-C was only exposed to Context B. The identity of the response (lever press or chain pull) was fully counterbalanced throughout this test procedure.

Renewal Test 2

On the next day, a second renewal test was conducted. This test was identical to the first test, except now subjects were tested in Context A and Context C. Once again, rats were tested on the response that was trained in Context B and then extinguished in Context A. Thus, it was expected that ABC (BAC) renewal would be observed. But once again, the groups differed with respect to the history of Context C. Group Ext-B had only been exposed to Context C throughout the duration of the experiment. Group Ext-C had been exposed to Context C during acquisition, but during extinction the response that was initially trained in Context A was extinguished in Context C.

Results

On the final day of acquisition, the mean number of responses per minute on the lever (34.3) was not reliably different from the chain (31.4), F(1, 23) = 1.38, p = .25. The course of acquisition and extinction are presented in Figure 8. The figure presents responding for each group, separated by each response, R1 and R2 (lever and chain counterbalanced). A 2 (Group) × 2 (Response) × 6 (Session) ANOVA revealed a main effect of Session, F(5, 110) = 184.07, MSE = 19.60, p < .001, and a Session × Response interaction, F(5, 110) = 4.47, MSE = 12.96, p < .01. No other main effects or interactions were significant, Fs < 1. A 2 (Group) × 2 (Response) ANOVA on the final day of acquisition revealed no significant main effects or interactions, Fs < 1.

Figure 8.

Results of Experiment 4. Mean responses per minute, separated by Group and Response (R1 and R2; lever and chain counterbalanced), during each 30-min session of acquisition (left) and extinction (right). Ext-B = extinction of R1 in Context B, Ext-C = extinction of R1 in Context C.

The data from the extinction phase are presented in the right portion of Figure 8. A 2 (Group) × 2 (Response) × 4 (Session) ANOVA revealed significant main effects of Session, F(3, 66) = 114.79, MSE = 4.09, p < .001, and Response, F(1, 22) = 7.53, MSE = 5.17, p < .05. The effect of Session interacted both with Group, F(3, 66) = 2.83, MSE = 4.09, p < .05, and Response, F(3, 66) = 7.51, MSE = 2.45, p < .001. Extinction of R1 appeared slightly faster for Group Ext-C than Ext-B. No other main effects or interactions were significant, largest F(1, 22) = 2.61. A 2 (Group) × 2 (Response) ANOVA on the final day of extinction revealed no significant main effects or interaction, largest F(1, 22) = 2.96.

The data from the first renewal test are presented in Figure 9. As shown in the figure, there was a robust renewal effect in both groups. Importantly, the strength of this effect did not depend upon group membership. A 2 (Group) × 2 (Context) ANOVA revealed a significant effect of Context, F(1, 22) = 48.48, MSE = 3.77, p < .001. The effect of Group and the Group × Context interaction did not approach significance, Fs < 1. All of the 24 possible renewal effects were observed. Thus, in the current experiment there was no detectable effect of testing for renewal in a context in which a response was extinguished relative to a context that had not been associated with extinction.

Figure 9.

Results of the Experiment 4 renewal test 1. Responding on R2 during renewal test 1. “Extinction” = Context A, “Renewal” = Context B. Ext-B = extinction of R1 in Context B and exposure to Context C. Ext-C = extinction of R1 in Context C and exposure to Context B.

The data from the second renewal test are presented in Figure 10. Overall, responding was low, undoubtedly due to previous extinction testing. Although the overall increase in response rate was low relative to the previous test, there was still a robust renewal effect. A 2 (Group) × 2 (Context) revealed a significant main effect of Context, F(1, 22) = 16.77, MSE = .34, p < .001. Neither the main effect of Group, nor the Group × Context interaction were significant, largest F(1, 22) = 2.99. Of the possible 24 renewal effects, 21 were observed; 2 rats in Group Ext-C and 1 rat in Group Ext-B did not show an increase in responding from the extinction to the renewal context.

Figure 10.

Results of the Experiment 4 renewal test 2. Responding on R2 during renewal test 2. “Extinction” = Context A, “Renewal” = Context C. Ext-B = extinction of R1 in Context B and exposure to Context C. Ext-C = extinction of R1 in Context C and exposure to Context B.

Discussion

The results of the current experiment once again demonstrate ABA (renewal test 1) and ABC (renewal test 2) renewal after instrumental extinction. The main goal of the experiment, however, was to assess whether extinction resulted in the context acquiring negative occasion-setting properties or whether it resulted in an inhibitory association between the context and the response. To that end, renewal was compared between groups that either had a different response extinguished in the subsequent renewal context, or had not. In the first case, extinction of the different response should have endowed the context with negative occasion-setting properties, which would transfer to the target response during testing and weaken the renewal effect. Robust renewal was observed, and the groups did not differ on the overall strength of this effect. It should be noted that one cannot rule out the possibility that the extinction context acquires negative occasion-setting properties on the basis of a null result. Perhaps, under different conditions, transfer of negative-occasion setting might be easier to detect. Nevertheless, the results of this study are entirely consistent with the notion that the extinction context simply and directly inhibits the response extinguished in it.

The current experiment also casts doubt on the idea that instrumental extinction results in the context acquiring a direct inhibitory context-reinforcer association. For example, extinction of R1 in Context B for Group Ext-B could have made the context a simple inhibitor. The same would not be true of Context B for Group Ext-C because extinction of R1 occurred in an irrelevant context (Context C). If this set of circumstances were true, then Group Ext-B should have shown less renewal of R2 responding in that context during the renewal test; inhibition in B should have suppressed responding. Of course, it remains possible that a direct inhibitory context-reinforcer association might be detected under different conditions.

General Discussion

The present experiments demonstrated renewal after the extinction of instrumental behavior using a procedure that equated the overall histories of the extinction and renewal contexts by giving complementary training and extinction of a second response in them (e.g., Bouton & Ricker, 1994; Harris et al., 2000; Rescorla, 2008). Both the extinction and renewal contexts should thus have had similar associative properties, such as their overall excitatory or inhibitory value. Thus, the renewal of extinguished instrumental behavior observed here cannot be due to the contexts’ direct associations with the reinforcer. Such associations would not produce the response-specific renewal observed in Experiments 1–3.

Experiment 1 demonstrated ABA renewal when Contexts A and B had each been equally associated with acquisition and extinction. This effect was replicated and extended in Experiment 2, which also compared the strengths of ABA and AAB renewal. There were two important findings from the latter experiment. During the extinction phase, extinction was faster when it occurred in a context that was different from that of acquisition (e.g, Bouton et al., 2011), even though the contexts had equivalent histories of reinforcement. This effect of the context change contrasts with findings from the Pavlovian literature, where there is often little effect of a context switch after initial conditioning (e.g., Bouton & King, 1983; Bouton & Peck, 1989; Harris et al., 2000; cf. Hall & Honey, 1990; Rescorla, 2008). The finding that a context switch results in a decrement of instrumental responding (cf. Crombag & Shaham, 2002; Nakajima et al., 2000) makes the AAB and ABC renewal effects even more interesting. That is, even under conditions where initial conditioning does not transfer perfectly to a new context, AAB and ABC renewal both indicate that the initial learning still transfers more readily across contexts than extinction. If initial learning did not transfer better than extinction across contexts, renewal would not occur.

A second important finding of Experiment 2 is that the sizes of the ABA and AAB renewal effects were remarkably similar, which is in contrast to the findings of Bouton et al. (2011), where ABA was much larger than AAB renewal. However, in their design Contexts A and B had much different histories; Context A was uniquely associated with reinforcement and Context B with nonreinforcement. Although other results suggested that the overall differences in the size of these renewal effects was not due to any direct associations between the acquisition context and the reinforcer (Bouton et al., 2011, Experiment 4), it is possible that other differences, such as the overall familiarity with each context, did contribute. Nevertheless, the results reported here suggest that when the contexts’ reinforcement histories are equated, ABA and AAB renewal may be similar in size. Of course, under different conditions, or with a more sensitive test, it may be possible to observe differences between ABA and AAB renewal.

It should be recognized that while the renewal effects observed in the present experiments are statistically robust, the change in the rate of responding from the extinction to the non-extinction context was modest. There are several reasons for this pattern. First, as noted earlier, the size of the renewal effect is likely parameter specific. There is already evidence that ABA and ABC renewal can both be increased with extended initial acquisition (Todd et al., 2012b) and other factors. Although speculative, it is also possible that different schedules of reinforcement during initial acquisition might result in different levels of renewal. Second, the results of Experiment 2 show that initial acquisition does not transfer perfectly across contexts. A lack of perfect transfer of instrumental conditioning out of the training context would reduce the overall size of instrumental AAB and ABC renewal. Third, in instrumental learning experiments, responding consistently decreases rapidly during extinction (e.g., see Figures 1, 3, 6, and 8). This tendency makes it difficult to expect a high level of instrumental responding during testing under extinction conditions. Finally, the present experiments control for differential context-reinforcer associations. With a simpler experimental design, it might be possible that context-reinforcer associations can potentiate renewal, especially ABA renewal (cf. Bouton et al., 2011).

As in Pavlovian extinction, AAB and ABC renewal after instrumental extinction implicates inhibitory learning during extinction because testing occurs in a context that is different from acquisition. Thus, mere removal from the extinction context is sufficient to cause renewal. In Pavlovian situations, this inhibitory learning is often thought to take the form of negative occasion setting, where the extinction context activates an inhibitory association between the CS and US (e.g., Bouton, 1997). Essentially, the context signals that the CS will not be followed by the US. Support for this mechanism comes from studies that have failed to detect inhibition in the extinction context (e.g., Bouton & King, 1983; Bouton & Swartzentruber, 1986; Nelson et al., 2011; cf. Polack, Laborda, & Miller, 2011), which should be present if the context had formed a direct inhibitory association with the US. The current experiments also leave little room for a direct inhibitory context-reinforcer association being the sole mechanism of renewal after instrumental extinction. Since both contexts (extinction and renewal) were equally associated with nonreinforcement (and previous reinforcement), any direct inhibitory context-reinforcer association should be present in both and suppress responding equally.

It remains possible that some configural cue generated by the combination of the response and the context formed an inhibitory association with the reinforcer during extinction (e.g., Pearce, 1987; Pearce, 1994). However, careful consideration of such a configural account suggests that it cannot account for all aspects of the current data. First, while a configural account can explain both ABA and ABC renewal, it has difficulty with AAB. In the case of ABA and ABC renewal, the acquisition context and the response stimulus (e.g., presence of the lever) could be assumed to form a configural cue that gains excitation during acquisition. Then, a configuration of Context B and the response stimulus would become inhibitory during extinction. ABA renewal would occur upon a return to Context A due to the presence of the original excitatory configural cue. Importantly, due to the decrement in generalization from A to B demonstrated here, there would also be less inhibition accrued to the configural B cue than excitation to the A cue. Thus, when testing occurs in Context C (ABC) both excitation from A and inhibition from B would generalize to C, but there would be slightly less inhibition, allowing for renewal to occur. However, the configural account has more difficulty with AAB renewal because extinction in A would give excitation and inhibition equal values (there would be little generalization decrement between the conditions of acquisition and extinction). Thus, in order for renewal to occur in Context B, it would have to be further assumed that inhibition transfers less readily than excitation (cf. Rescorla, 2006). Second, the results of the second renewal test in Experiment 4 are also inconsistent with a configural perspective. Two groups were tested for renewal in Context C, which had never been associated with reinforcement. One group (Ext-B) had only been previously exposed to Context C, and the other group (Ext-C) had a different response extinguished there. If, during extinction, Context C had formed an inhibitory configural cue (a combination of the context and the response stimulus), then some of this inhibition should have generalized to the test response and reduced renewal. However, the overall level of renewal was equivalent for the two groups.

Finally, the configural account is challenged to explain the results of the preference tests in Experiments 1–3, where both response options were simultaneously available. In this test, generalization from excitatory (conditioning) and inhibitory (extinction) configurations would set the overall associative value of the current configuration of context and response stimuli (lever and chain), but would not provide information about which response to perform. It is thus unclear how the configural account would explain the appropriate response selection when both responses are simultaneously available. One possibility is that only one configuration, either the excitatory or inhibitory cue, might be sampled by the rat at a given time. If so, the excitatory cue might elicit approach and manipulation (e.g., Bolles, 1972), whereas the inhibitory cue would not, resulting in preference for the response in its non-extinction (renewal) context.

The data from Experiments 1–3 are consistent with the possibility that instrumental extinction is controlled by an occasion-setting mechanism like the one emphasized in the Pavlovian literature. The renewal effects observed in these experiments could be explained by the fact that while occasion setters tend to transfer to other targets of occasion setters, this transfer is typically incomplete (Holland & Coldwell, 1993; Morell & Holland, 1993). For example, in Experiment 3, Contexts B and C may have become negative occasion setters for their respective extinguished responses, R1 and R2. Given the known properties of negative occasion setters, during testing some transfer of control by each context would be expected when R1 was tested in C and R2 in B. However, as long as this transfer was not complete, renewal would still be expected. Negative occasion setting was further tested in Experiment 4. One group had a response extinguished in Context B, potentially making the context an occasion setter, while the other group was only exposed to Context B. Simple exposure to the context was not expected to result in occasion setting; Morrell and Holland (1993) have shown that simple exposure to a stimulus does not result in that stimulus acquiring negative occasion-setting properties. The finding of equivalent renewal between the two groups indicated that there was no detectable transfer of occasion setting, casting doubt on the occasion setting mechanism. The failure to observe transfer of negative-occasion setting does not rule out this mechanism completely, however, as it might be possible to detect negative-occasion setting under different conditions.

All of the results in the current experiments are consistent, however, with the idea that extinction results in an inhibitory association between the stimulus, in this case the context, and the response (e.g., Rescorla, 1993, 1997). According to this view, during extinction the context forms an inhibitory association with the response extinguished in that context. This mechanism readily accounts for ABA, AAB, and ABC forms of renewal; renewal occurs when the response is tested outside of its extinction context because the inhibitory influence on the response is no longer present. This mechanism also predicts the results of Experiment 4. Since the inhibitory S-R association formed during extinction would be response-specific, renewal should be unaffected by testing in a context previously associated with extinction of a different response.

The idea that extinction might result in a direct, inhibitory association between the context and the response is consistent with other experiments that have shown this type of association between discriminative stimuli and their responses. For example, there is evidence of inhibitory S-R associations between negative discriminative stimuli and their responses (e.g., Bonardi, 1989; Colwill, 1991; see also Colwill 1994) as well as between discriminative stimuli and responses extinguished in their presence (e.g., Rescorla, 1993, 1997). However, the extent to which these inhibitory associations are binary (S-R) or hierarchical (occasion setting) has never been firmly established (see Colwill, 1993). In fact, most of the evidence in favor of inhibitory S-R associations is also consistent with an occasion setting mechanism where the stimulus signals the response will not be followed by the reinforcer (e.g., Rescorla, 1993, p. 335). Experiment 4 attempted to separate these possibilities further, and is most consistent with the simple, inhibitory S-R mechanism.

In sum, the experiments described here demonstrate that context-reinforcer associations cannot be the only contributing mechanism of renewal after the extinction of instrumental behavior, because renewal was observed when context-reinforcer associations were equated and controlled. This does not mean, however, that the contexts have no associative relation with the reinforcer. Each context might be excitatory or inhibitory, but some other mechanism must be in place that allows for renewal when the context is switched after extinction. The mechanism suggested here is an inhibitory association between the context and the response.