Abstract
Four experiments with rat subjects examined whether D-cycloserine (DCS), a partial NMDA agonist, facilitates the extinction of operant lever-pressing reinforced by food. Previous research has demonstrated that DCS facilitates extinction learning with methods that involve Pavlovian extinction. In the current experiments, operant conditioning occurred in Context A, extinction in Context B, and then testing occurred in both the extinction and conditioning contexts. Experiments 1a and 1b tested the effects of three doses of DCS (5, 15, and 30 mg/kg) on the extinction of lever pressing trained as a free operant. Experiment 2 examined their effects when extinction of the free operant was conducted in the presence of non-response-contingent deliveries of the reinforcer (which theoretically reduced the role of generalization decrement in suppressing responding). Experiment 3 examined their effects on extinction of a discriminated operant, i.e., one that had been reinforced in the presence of a discriminative stimulus, but not in its absence. A strong ABA renewal effect was observed in all four experiments during testing. However, despite the use of DCS doses and a drug administration procedure that facilitates the extinction of Pavlovian learning, there was no evidence in any experiment that DCS facilitated operant extinction learning assessed in either the extinction or the conditioning context. DCS may primarily facilitate learning processes that underlie Pavlovian, rather than purely operant, extinction.
Keywords: Extinction, D-cycloserine, operant conditioning, instrumental learning, Operant learning, renewal, relapse
When either a Pavlovian conditional stimulus (CS) or an operant behavior (instrumental action) occurs repeatedly without a reinforcer, the strength of responding can decline. The reduction of responding, extinction, appears to result at least in part from new learning that inhibits expression of the original learning (e.g., Bouton, 2004, 2011; Bouton & Woods, 2008; Myers & Davis, 2007; Quirk & Mueller, 2008). For example, in Pavlovian fear conditioning, a CS is first paired with a footshock unconditional stimulus (US), and consequently comes to elicit fear when it is presented on its own. Fear elicited by the CS can then be reduced if the CS is repeatedly presented without the US. Although the extinction procedure eliminates the fear, it does not erase the original learning. For example, fear of the CS returns if the context is changed after extinction, a phenomenon known as the renewal effect (e.g., Bouton & King, 1983). Renewal, along with other results, suggests that extinction is due at least partly to new learning that depends on the context for retrieval. This idea has a number of implications for the success of anxiety-disorder therapies that rely on extinction (e.g., Bouton, 2002).
If extinction involves new learning, then it might be possible to enhance it if the animal is given a drug that facilitates learning. Consistent with this hypothesis, administration of D-cycloserine (DCS), a partial agonist of the NMDA receptor involved in long-term potentiation (a cellular model of learning), facilitates the extinction of fear conditioning (e.g., Walker, Ressler, Lu, & Davis, 2002; Ledgerwood, Richardson, & Cranney, 2003). Rats given DCS with a small number of extinction trials show less fear than control subjects that receive the same number of trials without DCS during tests of the CS conducted without the drug the next day. In a similar way, exposure therapy of anxiety disorders in humans can be enhanced by DCS administration during extinction exposure to feared cues (e.g., Guastella, Richardson, Lovibond, Rapee, Gaston, Mitchell, & Dadds, 2008; Hofmann, Meuret, Smits, Simon, Pollack, Eisenmenger, Shiekh, & Otto, 2006; Ressler, Rothbaum, Tannenbaum, Anderson, Graap, Zimand, Hodges, & Davis, 2004).
One important boundary condition of the effect of DCS on fear extinction, however, is that DCS does not abolish the renewal effect (Bouton, Vurbic, & Woods, 2008; Woods & Bouton, 2006). That is, although DCS combined with extinction exposure to the CS can increase the rate at which the fear response decreases, fear is still renewed when the CS is returned to and tested in the original context in which it was conditioned. Such results suggest that although DCS administration can benefit therapy, it does not qualitatively change the nature of fear extinction learning, which remains relatively context-specific and vulnerable to relapse.
Interestingly, DCS also facilitates extinction of CSs associated with drugs or drug effects in the conditioned place preference paradigm (cocaine: Botreau, Paolone, & Stewart, 2006; Paolone, Botreau, & Stewart, 2009; Thanos, Bermeo, Wang, & Volkow, 2009; ethanol: see Groblewski, Lattal, & Cunningham, 2009; opiate withdrawal: Myers & Carlezon, 2010). Although such results begin to suggest some generality to the effects of DCS on Pavlovian extinction, there has been relatively little systematic investigation of the effects of DCS on the extinction of operant, as opposed to Pavlovian, behavior. This is true even though operant conditioning, like Pavlovian conditioning, is strongly linked to a number of human behavior disorders, including but not limited to eating and overeating (e.g., Bouton, 2011; Epstein, Salvy, Carr, Dearing, & Bickel, 2010) and drug dependence (e.g., Bouton, Winterbauer & Vurbic, 2011; Higgins, Heil, & Lussier, 2004). An early free-operant experiment with rats suggested that DCS delivered 30 min before an extinction session increased the amount of lever-press responding in that session; that is, DCS slowed the rate of extinction (Port & Seybold, 1998). Although the results of several other studies begin to suggest that DCS can facilitate operant extinction under some conditions (Nic Dhonnchadha, Szalay, Achat-Mendes, Platt, Otto, Spealman, & Kantak, 2010; Shaw, Norwood, Sharp, Quigley, McGovern, & Leslie, 2009; Vengeliene, Kiefer, & Spanagel, 2008), all of those studies included extinction exposure to a Pavlovian cue that had been associated with the reinforcer in addition to extinction of the operant response itself (see General Discussion for more details). And none examined whether DCS has an effect on the renewal of operant responding.
The present experiments investigated the effects of DCS on extinction of an operant response using methods that we have begun to characterize in our laboratory (e.g., Bouton, Todd, Vurbic & Winterbauer, 2011; Winterbauer & Bouton, 2010, 2011). Lever pressing was reinforced by presentation of food pellets on a variable-interval 30-s (VI-30) schedule of reinforcement. In extinction, lever pressing no longer produced food pellets; there was no presentation at this time of a cue that had been associated with the reinforcer. Using this method, we have shown that extinction of the operant response depends on new learning that is relatively specific to its context: Extinction is vulnerable to ABA, ABC, and AAB forms of the renewal effect (Bouton et al., 2011) and to “resurgence,” in which the extinguished behavior recovers when an alternative behavior is reinforced and then extinguished (Winterbauer & Bouton, 2010, 2011). In all of the current experiments, DCS was prepared and administered using a procedure that deliberately replicated one that facilitates fear extinction in this laboratory (Bouton et al., 2008; Woods & Bouton, 2006). Experiments 1a and 1b studied the effect of DCS on extinction of free-operant responding; Experiment 2 studied extinction of a free-operant response that occurred when pellets were delivered noncontingently; and Experiment 3 studied extinction of a discriminated operant response in which lever pressing was only reinforced in the presence of a visual discriminative stimulus. The results produced no evidence that DCS had an impact on any of these examples of operant extinction.
Experiment 1
In the first experiment, rats received 5 daily 30-min sessions of VI-30 lever press training (see Bouton et al., 2011) and then extinction in which the food-pellet reinforcer was simply omitted. Different groups received extinction 15 min after injection of DCS in either 0, 5 mg/kg, 15 mg/kg, or 30 mg/kg doses. The injection-to-extinction interval was the same as that in our fear extinction experiments (Bouton et al., 2008; Woods & Bouton, 2006). The doses were also chosen based on our previous research with fear extinction (where 15 mg/kg and 30 mg/kg have been effective, Bouton et al., 2008), and that of Vengeliene et al. (2008), who found a 5 mg/kg dose effective with ethanol-reinforced operant methods. Acquisition was conducted in one set of boxes (Context A), and extinction was conducted in another (Context B, counterbalanced, see Bouton et al., 2011). There were then two drug-free test sessions conducted in a counterbalanced order. In one, lever pressing was tested in Context B, the context of extinction; in the other, lever pressing was tested in Context A, the context in which acquisition had occurred. Based on earlier research (Bouton et al., 2011), we expected renewed responding in Context A (see also Nakajima, Tanaka, Urushihara, & Imada, 2000; Nakajima, Urushihara, & Masaki, 2002). The design thus allowed us to examine the effects of DCS on the renewal effect in addition to its direct effect on extinction as assessed in the extinction context. In principle, if DCS facilitates operant extinction learning, it could (1.) reduce responding during the extinction sessions themselves, i.e., while the drug is active systemically, (2.) reduce responding 24 hrs later during the drug-free test in the extinction context (if it has a longer-term effect on extinction learning), and/or (3.) reduce responding during renewal testing in Context A (if its effect is potent enough to reduce relapse as represented in the renewal effect). Our previous results with fear extinction (Bouton et al., 2008; Woods & Bouton, 2006) suggest that DCS may affect responding when assessed 24 hrs later in a drug-free test (possibility 2, above) but not during extinction (possibility 1) or renewal testing (possibility 3). Interestingly, two recent reports with Pavlovian drug conditioning suggest that DCS effects might be more readily detected in relapse tests than extinction tests (Groblewski et al., 2009; Torregrossa, Sanchez, & Taylor, 2010).
Experiments 1a and 1b differed in the particulars of extinction. In Experiment 1a, extinction involved two 30-min sessions that were separated by a three-day interval. This procedure followed that of Shaw et al. (2008), who used rest intervals of several days between successive extinction sessions to prevent the desensitization of NMDA receptors by multiple exposures to DCS. Experiment 1b employed only a single extinction session before testing began the next day.
Experiment 1a
Method
Subjects
The subjects were 32 female Wistar rats purchased from Charles River Laboratories (St. Constance, Quebec). They were between 120 and 135 days old at the start of the experiment and were individually housed in suspended wire mesh cages in a room maintained on a 16:8-h light:dark cycle. The rats were food-deprived to 80% of their baseline body weights throughout the experiment. The rats had prior experience in two unrelated experiments; an appetitive Pavlovian conditioning experiment in which white noise and clicker CSs had been paired with a food pellet US, and a fear conditioning experiment in which a tone CS was paired with a footshock US. The previous experiments were performed in a different set of conditioning chambers.
Apparatus
The apparatus consisted of two unique sets of four conditioning chambers (Med Associates, St. Albans, VT) housed in separate rooms of the laboratory. Each chamber was housed in its own sound attenuation chamber. All boxes measured 30.5 cm × 24.1 × 23.5 cm (l × w × h). The side walls and ceiling were made of clear acrylic plastic, while the front and rear walls were made of brushed aluminum. The floor was made of stainless steel grids (0.48 cm diameter). A recessed 5.1 cm × 5.1 cm food cup was centered in the front wall approximately 2.5 above the level of the floor. In both sets of boxes, a retractable lever was positioned to the left of the food cup. A 28-V panel light (2.5 cm in diameter) was attached to the wall 10.8 cm above the floor and 6.4 cm to the left of the food cup. The chambers were illuminated by one 7.5-W incandescent bulbs mounted to the ceiling of the sound attenuation chamber, approximately 34.9 cm from the grid floor. Ventilation fans provided background noise of 65 dB.
The two sets of four boxes had unique features that allowed them to be used as different contexts (counterbalanced). In one set of boxes, one side wall had black diagonal stripes, 3.8 cm wide and 3.8 cm apart. The ceiling had similarly spaced stripes oriented in the same direction. A distinct odor was continuously presented by placing 5 ml Pine-Sol (The Clorox Co., Oakland, CA) in a dish outside the chamber. The grids of the floor were mounted on the same plane and were spaced 1.6 cm apart (center-to-center). The other set of boxes had no distinctive visual cues, and the grids of the floor were staggered such that odd- and even-numbered grids were mounted in two separate planes, one 0.5 cm above the other. The odor cue was provided by 5 ml of Lemon Cleaner (Rite Aid Corp., Harrisburg, PA). The reinforcer was a 45 mg food pellet (Traditional formula, Research Diets, New Brunswick, NJ). The apparatus were controlled by computer equipment located in an adjacent room.
Drugs
D-cycloserine (Sigma-Aldrich, St. Louis, MO) was mixed with chilled physiological saline (0.9%) and kept on ice until administered. Three concentrations (5, 15 and 30 mg/kg/ml) were prepared fresh on each day of extinction.
Procedure
Magazine Training
On Day 1, rats received a single 30-min session of magazine training in Context A in which approximately 60 pellets were delivered randomly on average every 30 s. On Day 2, all rats received magazine training in the other context (Context B). The levers were retracted during this phase.
Acquisition
On each of the next five consecutive days, rats received one session of lever-press training on a variable interval (VI) 30-s reinforcement schedule in Context A. The levers were inserted two minutes after the rats were placed in the chambers. No shaping was required. The session ended when the levers were retracted 30 min later.
Extinction
On the next day rats were randomly assigned to one of four groups (n = 8) with the restriction that boxes be balanced over the groups. The groups were approximately matched on lever pressing rates on the final day or acquisition. Two 30-min extinction sessions spaced three days apart were then conducted in Context B. The interval between the two sessions was intended to diminish the possible effects of receptor desensitization that may have occurred after the first DCS administration (Shaw et al., 2009). Rats were handled normally on the intervening days, but were not run in any experimental sessions. Both extinction sessions began 15 minutes after a subcutaneous injection in the colony room of either 5 mg/kg DCS, 15 mg/kg DCS, 30 mg/kg DCS, or vehicle. The behavioral procedure was exactly the same as the acquisition procedure, except that lever presses were not reinforced. Food pellets were not delivered at any time.
Renewal Test
On the final day the rats received a single 10-min test session in each context. The order of testing was counterbalanced such that half the rats in each group were first tested in Context A and the other half were first tested in Context B. The two sessions were separated by approximately 60 min. In either session, the levers were introduced two minutes after the rats were placed in the chambers and were retracted 10 min later. No pellets were delivered.
Data analysis
The results were evaluated with analyses of variance (ANOVAs) using a rejection criterion of p < .05.
Results
As shown on the left of Figure 1, the rats quickly acquired the lever press response and responding appeared to reach asymptote by the end of the acquisition phase. Since the groups were matched on response rates at the end of acquisition, it was not surprising that there were no group differences during this phase (DCS Dose × Session ANOVA: Session F(4,112) = 50.52; all other Fs < 1). For every rat, response rate then decreased over the course of extinction (middle panel). Responding across the extinction phase was analyzed in 5-min time bins. A Dose × Bin ANOVA found that response rates declined substantially from the first bin of Session 1 to the last bin of Session 2, F(1,28) = 231.86. There was no Dose effect or Dose × Bin interaction, largest F = 1.10. Since the rats were injected with DCS shortly before each extinction session, the latter statistics indicate that DCS did not unconditionally enhance responding (cf. Port & Seybold, 1998)
A separate ANOVA comparing the last bin of Session 1 and the first bin of Session 2 (bins 6 and 7 on the figure) revealed an increase in responding (i.e., a spontaneous recovery effect) between sessions, F(1,28) = 129.03. However, there was no evidence that DCS had any effect on this effect; neither the Dose effect nor Dose × Bin interaction was significant, largest F = 1.89.
The tests for renewal are presented on the right of Figure 1. As illustrated by the figure, there was a strong renewal effect, which took the form of a reliable increase in responding in Context A compared to Context B. A DCS Dose × Context × Test Order (Context A or B tested first) ANOVA confirmed this renewal effect, F(1,24) = 53.61. All but one rat responded more on the test in Context A than Context B. However, there was no evidence that DCS treatment influenced responding on either one of the tests. The analysis found no effect of DCS Dose or Dose × Context interaction, Fs < 1. No other effects or interactions approached significance. Separate ANOVAs on responding in each context additionally found no differences among the groups, Fs < 1.
Experiment 1b
Method
Subjects and Apparatus
Experiment 1b replicated the methods used above except as noted. The subjects were 32 female Wistar rats purchased from the same supplier and housed under the same conditions. They were approximately 90–105 days old at the start of food restriction. The rats had previous experience in a Pavlovian fear conditioning experiment in a different apparatus in which a tone CS was paired with a footshock US. The apparatus, drugs, and design of Experiment 1b were the same as Experiment 1a.
Procedure
Magazine training and acquisition were the same as in Experiment 1a. One 15-minute session of extinction was then conducted 24 hours after the last acquisition session. The extinction session began 15 minutes after drug injection. The renewal test was then conducted on the following day using the procedure used in Experiment 1a. There were no breaks between any sessions in the experiment that were greater than 24 hours.
Results and Discussion
The data from acquisition and extinction are shown in the left and center panels of Figure 2. As in the previous experiment, a DCS Dose x Session ANOVA found that the rats readily acquired the lever press response, Session F(4,112) = 75.94, but there were no group differences, Fs < 1. The response rates during the single extinction session declined in all rats, and as in Experiment 1A, a Dose × Bin ANOVA found that there was a significant decrease in response rate from the first 5-min to the last bin, F(1,28) = 389.05. There was also no Dose effect or Dose × Bin interaction, largest F = 1.67.
The results of the test sessions are shown on the right. Once again, there was a strong renewal effect with little evidence of an effect of DCS. A DCS Dose × Context × Test Order ANOVA confirmed a robust renewal effect, F(1,24) = 122.24. All 32 rats responded more in Context A than in Context B. Neither the Dose effect, F < 1, nor the Dose × Context interaction was significant, F(3,24) = 2.25, p = .11. Separate Dose x Order analyses in each test context found only a significant Order effect in Context B, F (1,24) = 6.02, which took the form of higher responding among the rats who were tested in B first. The Dose effect did not approach significance in either Context B, F(3,24) = 1.30 or Context A, F(3,24) < 1..
The behavioral methods used in Experiments 1a and 1b, which were based on those of Bouton et al. (2011), produced lawful conditioning, extinction, and ABA renewal during testing. However, despite the ability to observe these effects, there was no evidence that DCS had an effect on extinction learning at any point in either experiment. This was true even though the DCS injection procedure and timing were the same as that in previous experiments that demonstrated DCS effects on fear extinction (Bouton et al., 2008; Woods & Bouton, 2006).
Experiment 2
Theories of the extinction of instrumental behavior have often emphasized the role of generalization decrement in explaining the loss of responding that occurs when reinforcers are omitted (e.g., see Mackintosh, 1974, for one review). Such theories recognize that removal of the reinforcer can cause a decrement in responding because it introduces a change in the stimuli present during acquisition and extinction. There is evidence that delivery of the reinforcer in free-operant experiments (like Experiments 1a and 1b) can act as a discriminative stimulus that sets the occasion for responding. For example, presenting the reinforcer in a non-contingent manner during extinction slows the loss of leverpress responding compared to an extinction procedure in which the reinforcer is simply omitted (e.g., Baker, 1990; Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011). In addition, the mere presentation of the reinforcer after simple extinction can immediately lead the rat to lever press again (e.g., Baker, Steinwald, & Bouton, 1991; Reid, 1958; Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011). Thus, reinforcer presentations do provide stimulus support for operant responding, and their removal might easily reduce responding through generalization decrement.
Despite a possible role for generalization decrement, previous research with the methods used in Experiments 1a and 1b also indicates a role for active new extinction learning. For example, the observation of ABC and AAB renewal (Bouton et al., 2011) suggests that a form of inhibition learned in extinction actively suppresses performance in the extinction context. However, to the extent that generalization decrement does contribute, it might reduce the amount of new learning that occurs during extinction. The hypothesized effect of DCS is on new learning, rather than generalization decrement. Therefore, Experiment 2 sought to test its effects on extinction after minimizing generalization decrement. After lever-press training like that in Experiments 1a and 1b, rats received extinction in a session in which lever presses no longer produced food pellets, but pellets continued to be delivered independently of responding at an average of once every 30 s (as in the VI30 schedule). Noncontingent reinforcers should reduce generalization decrement, leaving any decrement in responding more likely under the control of new learning. There were two days of testing. On each day, responding was tested in Contexts A and B in a counterbalanced order. On one of the days both tests were conducted in the presence of noncontingent food pellets, and on the other, the tests occurred without pellets. The order of pellet and no-pellet testing was counterbalanced within each group.
Method
Subjects and Apparatus
The subjects were 32 female Wistar rats purchased from the same supplier and housed under the same conditions. They were approximately 90–105 days old at the start of food restriction. The rats had previous experience in a Pavlovian fear conditioning experiment in which a tone CS was paired with a footshock US. The apparatus and drugs were the same as Experiments 1a and 1b.
Procedure
Magazine training and acquisition were the same as in Experiments 1a and 1b. One 30-minute session of extinction was conducted in Context B 24 hours after the end of acquisition. Drug administration was the same as before and lever presses were not reinforced at any time. However, noncontingent food pellets were delivered on a variable time (VT) 30-s schedule throughout the session. Testing was then conducted over the next two days. Rats were given two 10-minute sessions on each day, one in Context A and one in Context B (counterbalanced such that half the rats received A-B and the other B-A; the order was kept consistent on both test days). One day of tests was performed using the same method as the extinction session; noncontingent pellets were delivered on a VT 30-s schedule, but responses were not reinforced. The other day of tests was conducted without any pellets. The test order (Pellet or No Pellet first) was also counterbalanced.
Results
The data from the acquisition and extinction sessions are shown in the left and center panels of Figure 3. As expected, acquisition and extinction went smoothly and there were no group differences in either phase. During extinction (with added noncontingent pellets), all rats showed a decline in response rate between the first and last 5-min bins of the session, F (1,28) = 161.28. Neither the Dose effect nor the Dose x Bin interaction was significant, Fs ≤ 1.24.
The tests for renewal under both the Pellet and No Pellet conditions are shown at right in Figure 3. These tests were analyzed in a DCS Dose × Context × Test Type (Pellet or No Pellet) × Pellet Test Order × Context Test Order ANOVA. The renewal effect (Context effect) was once again highly significant, F(1,16) = 43.74. Greater responding in Context A was observed in 29 of 32 rats in the Pellet test, and 28 out of 32 rats in the No Pellet test. Neither the DCS Dose effect nor the Dose x Context interaction approached significance, Fs < 1, once again indicating no effect of DCS on extinction learning. Pairwise comparisons also failed to find any significant Dose effects in any of the test sessions.
Overall, there was a trend toward more responding during the Pellet test than the No Pellet test, although the main effect of Test Type fell short of significance, F(1,16) = 3.64, p = .08. The fact that pellet presentations did not significantly enhance responding at this time is consistent with the fact that noncontingent pellets delivered in extinction reduce their “reinstating” effects when they are again administered after extinction (e.g., Rescorla & Skucy, 1969; Winterbauer & Bouton, 2011). There was a significant main effect of Pellet Test Order, F(1,16) = 5.15, such that the rats that received the Pellet test on the first day responded more overall than rats that received the Pellet test on the second day. However, this variable interacted with Test Type, F(1,16) = 12.39, such that Pellet-first rats responded more on the Pellet test than the No Pellet test. There was no such increase among the rats who received the Pellet test on the second day. There were additional interactions between Test Type, Test Order, and Context, F(1,16) = 10.16; Dose, Test Type, Test Order, and Context Order, F(3,16) = 3.36; and Dose, Test Order, Context Order, and Context, F(3,16) = 4.12. We do not have a straightforward explanation of the various interactions, but they do not appear to complicate interpretation of the main results.
In this experiment, a decrease in responding occurred when the rats received pellets in a noncontingent manner during the extinction session. The loss of responding was potentially slowed by the reduction of generalization decrement; it could also have been slowed by occasional chance pairings of the lever press response with a noncontingent pellet. Although the decline in responding must have been less dependent on simple generalization decrement than that in Experiments 1a and 1b, DCS once again had no detectable effect on extinction. Despite our effort to increase a role for new learning, we again found no evidence that DCS facilitated extinction of an operant response.
Experiment 3
In studies of fear extinction, where effects of DCS are relatively well established, the rat first learns that a CS is associated with a significant event (footshock); extinction takes the form of presenting the CS repeatedly, without shock, so that the conditioned response (fear or freezing) in the presence of the CS declines. A clear difference between Pavlovian learning and the free operant methods used in the preceding experiments is that no stimulus analogous to a CS was employed in Experiments 1a, 1b, and 2. It is conceivable that DCS facilitates a form of extinction learning in which the animal learns not to respond in the presence of a temporally-proximal stimulus. For example, it might facilitate the learning of an inhibitory stimulus-response association, a possible behavioral mechanism of extinction (e.g., Rescorla, 2001; see Bouton & Woods, 2008, for evaluation). In the preceding experiments, there was no stimulus that immediately preceded the response during extinction, perhaps making this form of extinction learning less available for influence by DCS.
Experiment 3 therefore arranged for the lever-press response to be associated with a temporally-proximal stimulus. It used a discriminated operant procedure in which the response was reinforced (again on a VI-30 reinforcement schedule) when a light discriminative stimulus was turned on, but not when the light was off. The rat therefore learned to respond in the presence of the light, and not in its absence. Theoretically, the light could acquire the ability to control operant responding during conditioning through at least three mechanisms. First, the stimulus could become a discriminative stimulus that “sets the occasion” for the response, for example, by signaling the response-reinforcer relationship (e.g., Rescorla, 1991). Second, the stimulus could be associated directly with the food pellet through Pavlovian conditioning and therefore invigorate the operant response either by motivating it (e.g., Corbit & Balleine, 2005; Rescorla & Solomon, 1967), by evoking an expectancy of the reinforcer that sets the occasion for the response (e.g., Trapold & Overmier, 1972), or by eliciting natural behaviors such as approach that might augment instrumental responding (e.g., Timberlake, 2001). Third, the stimulus might enter into a direct S-R association with the response. Notice that a fear response acquired through Pavlovian conditioning could share the second and third processes, but probably not the first (occasion setting) process. Although occasion setting has been studied in Pavlovian learning (e.g., Holland, 1992), it is a hierarchical process in which responding to one CS is controlled by the presence or absence of a second CS. It is not typically invoked to explain responding that develops through simple CS-US pairings.
After the conclusion of discrimination training, when lever pressing was clearly under control of the discriminative stimulus, a single extinction session was conducted. During the session, the light was presented over a series of trials (as before), but lever presses during it were no longer reinforced. Responding in the light therefore decreased. The question was whether DCS combined with this form of extinction would enhance the extinction process. The overall design was otherwise similar to that of Experiment 1b.
Method
Subjects and Apparatus
The subjects were 32 female Wistar rats purchased from the same supplier and housed under the same conditions. They were approximately 90–105 days old at the start of food restriction. The rats had previous experience in a Pavlovian fear conditioning experiment in which a tone CS was paired with a footshock US. The apparatus and drugs were the same as the previous experiments.
Procedure
Magazine Training
Two sessions of magazine training (one in each context) were conducted on the same day. The 30-min sessions were separated by two hours.
Acquisition
Over the next two days, rats were given two 30-min acquisition sessions on a VI-30s schedule of reinforcement, as in the previous experiments. Twelve daily sessions of discrimination training then began on the third day. In each session the rats received 16 trials in which lever pressing was only reinforced during the 30-s illumination of a panel light located above the lever. The VI-30s reinforcement schedule was in effect only during these trials, and no pellets were delivered during the intertrial interval (ITI). The ITI was variable, averaging 30 s in the first session, 60 s in the second session, and 90 s in all sessions thereafter.
Extinction
On the day that followed the final acquisition session, rats were assigned to four groups matched on response rate in the final session. They were then given one 30-min session of extinction 15 min after drug administration. The rats received 16 presentations of the light with a variable 90-s ITI. No pellets were delivered at any time during the session.
Renewal Test
On the last day the rats were given two test sessions; one in Context A and the other one in Context B (order counterbalanced). In each session they received 8 presentations of the light with a variable 90-s ITI. No pellets were delivered at any time.
Results and Discussion
To describe the elevation of responding that was occasioned by presentation of the light, elevation scores were calculated for each trial by subtracting the number of baseline responses made during a 30-s period immediately before each light stimulus from the number of responses made during the 30-s stimulus. The acquisition data are presented on the left of Figure 4. There was a steady acquisition of the discriminated operant response over the 12 sessions of discrimination training. On Session 1, the rats had a mean of 8.2 responses during the stimulus and 8.1 during the pre-stimulus periods; by Session 12, the corresponding means were 18.3 and 6.2. A DCS Dose × Session ANOVA on the elevation scores found only a Session effect, F(11,308) = 36.24, but no Dose effect or Dose × Session interaction, Fs < 1. In the extinction session shown in the middle panel, responding decreased in all animals. The trials were averaged into 4-trial blocks and analyzed with a Dose x Block ANOVA. As expected, there was a significant decrease in the response rate, F (3,84) = 31.80, but no Dose effect or Dose × Block interaction, largest F = 1.61.
The data from the renewal tests are displayed on the right of Figure 4. As seen in the figure, there was a strong renewal effect in all groups. A DCS Dose × Context × Test Order ANOVA found a significant effect of Context, F(1,24) = 83.20. Thirty-one out of 32 rats responded more in the light in Context A than in Context B. There were no other main effects or interactions, largest F(1,24) = 2.25, p = .15. Pairwise comparisons further confirmed the absence of significant group differences in either context. Analysis of pre-CS responding (Dose × Context × Order) found only a significant main effect of Context, F(1,24) = 8.70. Average pre-stimulus scores in the test in Context A were 1.17, 1.75, 1.59, and 2.56, for the 0, 5, 15 and 30 mg/kg groups, respectively. Pre-stimulus scores in Context B were 0.53, 0.36, 0.45, and 0.69 for the same groups.
Although the discriminated operant procedure introduced a stimulus whose onset, like that of a Pavlovian CS, stood in close temporal proximity to the response, there was once again no effect of DCS on extinction learning. Evidently, there is a difference between Pavlovian and discriminated operant learning. One possibility, as noted above, is that the present light discriminative stimulus worked primarily by setting the occasion for the response—rather than eliciting the response in the manner of a Pavlovian CS.
General Discussion
The present experiments used a range of behavioral methods to ask whether DCS facilitates the extinction of operant lever-pressing reinforced by food. There was no effect of DCS on extinction learning when the reinforcer was merely withheld in extinction (Experiments 1a and 1b), presented in a noncontingent manner in order to reduce the role of generalization decrement (Experiment 2), or withheld in a procedure that allowed the inhibition of responding to be associated with a co-present discriminative stimulus (Experiment 3). The experiments also found no impact of DCS when tests occurred in the conditioning context as well as the extinction context. If DCS has an effect on operant extinction learning, it must influence a mechanism that was not captured by the range of conditions that were investigated here.
Several aspects of these experiments should have helped to ensure detection of any DCS effect. First, the DCS administration procedure was exactly the same as one that has facilitated fear extinction in this laboratory (Bouton et al., 2008; Woods & Bouton, 2006): DCS was delivered via subcutaneous injections 15 min before the extinction session, and extinction training concluded within 45 min of DCS administration. In fact, the 30-min extinction sessions given in Experiments 1a, 2, and 3 were specifically arranged to occupy a position within the temporal post-injection window that was identical to the 30-min extinction session used in successful fear extinction experiments (Experiments 2a and 2b of Bouton et al., 2008). Second, the three DCS doses tested here (5, 15, and 30 mg/kg) were chosen because they have had an effect in other methods (Bouton et al., 2008; Vengeleine et al., 2009; Woods & Bouton, 2006). Third, in each experiment, all rats learned some extinction during the extinction session(s) as evidenced by the fact that their rates of lever pressing declined over the course of the session. This fact is worth noting, because the evidence suggests that the animal must learn some extinction while DCS is in its system for the drug to have an effect (Bouton et al., 2008; Weber, Hart, and Richardson, 2007). In summary, the present experiments met a variety of conditions that previous research suggests should have enabled the observation of an effect of DCS.
The current negative results contrast with those of other experiments suggesting that DCS can facilitate some aspect of operant extinction learning (Nic Dhonnchadha et al., 2010; Shaw et al., 2009; Vengeliene et al., 2008). Shaw et al. (2009) reported two experiments with mice suggesting that DCS delivered immediately after sessions in which food-reinforced lever pressing was extinguished can increase the rate of extinction (Shaw et al., 2009). And under at least some conditions, when oral ethanol (Vengeliene et al., 2008) or intravenous cocaine (Nic Dhonnchadha et al., 2010) have been used as reinforcers in rats, DCS administration before extinction sessions can immediately decrease the rate of responding, suggesting either an unconditional suppression of lever pressing (cf. Port & Seybold, 1998) or an increased rate of extinction learning. DCS before extinction sessions can also reduce the rate at which lever-pressing is reacquired when response-reinforcer pairings are resumed after extinction (Nic Dhonnchadha et al., 2010; Vengeliene et al., 2008).
It is relevant to note, however, that the experiments producing positive results all used methods in which a conditioned reinforcer associated with the primary reinforcer during training was also presented without the reinforcer during the extinction and test phases. Nic Dhonnchadha et al. (2010) presented a 2-s change in illumination whenever the reinforcer (cocaine) was delivered during acquisition, and then continued to present the stimulus as a consequence of the operant response as the response was being extinguished. Similarly, Shaw et al. (2009) retracted the lever during acquisition at the moment each reinforcer (food pellet) was presented, and continued to present the retraction stimulus during extinction. And Vengeliene et al. (2008) presented a brief auditory stimulus with each reinforcer (ethanol) during conditioning, and then continued to present it in extinction. In each case, extinction of the conditioned reinforcer would have engaged Pavlovian extinction, rather than extinction of the operant response itself. Extinction of a conditioned reinforcer alone can be sufficient to reduce operant responding, and can be enhanced by DCS (Torregrossa et al., 2010) (even when extinction is conducted in a context that differs from the test context). And Gabriele and Packard (2007) similarly found that Pavlovian extinction of goal box stimuli after food reinforcement of alley running was sufficient to decrease the strength of alley running (and was facilitated by DCS). In the present experiments, there was no role for extinction of a conditioned reinforcer. No explicit stimulus was associated with the food pellet in acquisition and then also presented in extinction. When the present results are combined with those in the rest of the literature, the pattern thus suggests that DCS may facilitate extinction when Pavlovian, but not purely operant, extinction processes can play a role.
The present results thus begin to suggest that DCS may influence the extinction of stimulus-outcome (Pavlovian) learning more than response-outcome (operant) learning.
Acknowledgments
This research was supported by Grant R01 MH064847 from the National Institute of Mental Health. BG was supported by a Summer Neuroscience Undergraduate Research Fellowship from the National Science Foundation. We thank Matt Campolattero, Travis Todd, and Neil Winterbauer for their comments, and Grace Bouton for help running the experiments. Send correspondence to Mark E. Bouton, Department of Psychology, University of Vermont, 2 Colchester Ave., Burlington, VT 05405.
Footnotes
Publisher's Disclaimer: The following manuscript is the final accepted manuscript. It has not been subjected to the final copyediting, fact-checking, and proofreading required for formal publication. It is not the definitive, publisher-authenticated version. The American Psychological Association and its Council of Editors disclaim any responsibility or liabilities for errors or omissions of this manuscript version, any version derived from this manuscript by NIH, or other third parties. The published version is available at www.apa.org/pubs/journals/bne
References
- Baker AG. Contextual conditioning during free-operant extinction: Unsignaled, signaled, and backward-signaled noncontingent food. Animal Learning & Behavior. 1990;18:59–70. [Google Scholar]
- Baker AG, Steinwald H, Bouton ME. Contextual conditioning and reinstatement of extinguished instrumental responding. Quarterly Journal of Experimental Psychology. 1991;43B:199–218. [Google Scholar]
- Botreau F, Paolone G, Stewart J. D-Cycloserine facilitates extinction of a cocaine-induced conditioned place preference. Behavioral Brain Research. 2006;172:173–178. doi: 10.1016/j.bbr.2006.05.012. [DOI] [PubMed] [Google Scholar]
- Bouton ME. Context, ambiguity, and unlearning: Sources of relapse after behavioral extinction. Biological Psychiatry. 2002;52:976–986. doi: 10.1016/s0006-3223(02)01546-9. [DOI] [PubMed] [Google Scholar]
- Bouton ME. Context and behavioral processes in extinction. Learning & Memory. 2004;11:485–494. doi: 10.1101/lm.78804. [DOI] [PubMed] [Google Scholar]
- Bouton ME. Learning and the persistence of appetite: Extinction and the motivation to eat and overeat. Physiology & Behavior. 2011 doi: 10.1016/j.physbeh.2010.11.025. in press. [DOI] [PubMed] [Google Scholar]
- Bouton ME, King DA. Contextual control of the extinction of conditioned fear: Tests for the associative value of the context. Journal of Experimental Psychology: Animal Behavior Processes. 1983;9:248–265. [PubMed] [Google Scholar]
- Bouton ME, Winterbauer NE, Vurbic D. Context and extinction: Mechanisms of relapse in drug self-administration. In: Haselgrove M, Hogarth L, editors. Clinical applications of learning theory. East Sussex, UK: Psychology Press; 2011. in press. [Google Scholar]
- Bouton ME, Vurbic D, Woods AM. D-Cycloserine facilitates context-specific fear extinction learning. Neurobiology of Learning & Memory. 2008;90:504–510. doi: 10.1016/j.nlm.2008.07.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bouton ME, Todd TP, Vurbic D, Winterbauer NE. Renewal after the extinction of free operant behavior. Learning & Behavior. 2011 doi: 10.3758/s13420-011-0018-6. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bouton ME, Woods AM. Extinction: Behavioral mechanisms and their implications. In: Byrne JH, Sweatt D, Menzel R, Eichenbaum H, Roediger H, editors. Learning and memory: A comprehensive reference. Vol. 1. Oxford: Elsevier; 2008. pp. 151–171. Learning Theory and Behaviour. [Google Scholar]
- Corbit LH, Balleine BW. Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian-instrumental transfer. Journal of Neuroscience. 2005;25:962–970. doi: 10.1523/JNEUROSCI.4507-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Epstein LH, Salvy SJ, Carr KA, Dearing KK, Bickel WK. Food reinforcement, delay discounting and obesity. Physiology & Behavior. 2010;100:438–445. doi: 10.1016/j.physbeh.2010.04.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gabriele A, Packard MG. D-Cycloserine enhances memory consolidation of hippocampus-dependent latent extinction. Learning & Memory. 2007;14:468–471. doi: 10.1101/lm.528007. [DOI] [PubMed] [Google Scholar]
- Groblewski PA, Lattal KM, Cunningham CL. Effects of D-cycloserine on extinction and reconditioning of ethanol-seeking behavior in mice. Alcoholism: Clinical and Experimental Research. 2009;33:772–782. doi: 10.1111/j.1530-0277.2009.00895.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guastella AJ, Richardson R, Lovibond PF, Rapee RM, Gaston JE, Mitchell P, Dadds MR. A randomized controlled trial of D-cycloserine enhancement of exposure therapy for social anxiety disorder. Biological Psychiatry. 2008;63:544–549. doi: 10.1016/j.biopsych.2007.11.011. [DOI] [PubMed] [Google Scholar]
- Higgins ST, Heil SH, Lussier JP. Clinical implications of reinforcement as a determinant of substance use disorders. Annual Review of Psychology. 2004;55:431–61. doi: 10.1146/annurev.psych.55.090902.142033. [DOI] [PubMed] [Google Scholar]
- Hofmann SG, Meuret AE, Smits JA, Simon NM, Pollack MH, Eisenmenger K, Shiekh M, Otto MW. Augmentation of exposure therapy with D-cycloserine for social anxiety disorder. Archives of General Psychiatry. 2006;63:298–304. doi: 10.1001/archpsyc.63.3.298. [DOI] [PubMed] [Google Scholar]
- Holland PC. Occasion setting in Pavlovian conditioning. In: Medin DL, editor. The Psychology of Learning and Motivation. Academic Press; New York: 1992. pp. 69–125. [Google Scholar]
- Ledgerwood L, Richardson R, Cranney J. Effects of D-cycloserine on extinction of conditioned freezing. Behavioral Neuroscience. 2003;117:341–349. doi: 10.1037/0735-7044.117.2.341. [DOI] [PubMed] [Google Scholar]
- Mackintosh NJ. The psychology of animal learning. London: Academic Press; 1974. [Google Scholar]
- Myers KM, Carlezon WA. D-cycloserine facilitates extinction of naloxone-induced conditioned place aversion in morphine-dependent rats. Biological Psychiatry. 2010;67:85–87. doi: 10.1016/j.biopsych.2009.08.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Myers KM, Davis M. Mechanisms of fear extinction. Molecular Psychiatry. 2007;12:120–150. doi: 10.1038/sj.mp.4001939. [DOI] [PubMed] [Google Scholar]
- Nakajima S, Tanaka S, Urushihara K, Imada H. Renewal of Extinguished Lever-Press Responses upon Return to the Training Context. Learning and Motivation. 2000;31:416–431. [Google Scholar]
- Nakajima S, Urushihara K, Masaki T. Renewal of operant performance formerly eliminated by omission or noncontingency training upon return to the acquisition context. Learning and Motivation. 2002;33:510–525. [Google Scholar]
- Nic Dhonnchadha BA, Szalay JJ, Achat-Mendes C, Platt DM, Otto MW, Spealman RD, Kantak KM. D-cycloserine deters reacquisition of cocaine self-administration by augmenting extinction learning. Neuropsychopharmacology. 2010;35:357–67. doi: 10.1038/npp.2009.139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Paolone G, Botreau F, Stewart J. The facilitative effects of D-cylcoserine on extinction of a cocaine-induced conditioned place preference can be long lasting and resistant to reinstatement. Psychopharmacology. 2009;202:403–409. doi: 10.1007/s00213-008-1280-y. [DOI] [PubMed] [Google Scholar]
- Port L, Seybold KS. Manipulation of NMDA-receptor activity alters extinction of an instrumental response in rats. Physiology & Behavior. 1998;64:391–393. doi: 10.1016/s0031-9384(98)00095-x. [DOI] [PubMed] [Google Scholar]
- Quirk GJ, Mueller D. Neural mechanisms of extinction learning and retrieval. Neuropsychopharmacology Reviews. 2008;33:5672. doi: 10.1038/sj.npp.1301555. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reid RL. The role of the reinforcer as a stimulus. British Journal of Psychology. 1958;49:202–209. doi: 10.1111/j.2044-8295.1958.tb00658.x. [DOI] [PubMed] [Google Scholar]
- Rescorla RA. Associative relations in instrumental learning: The eighteenth Bartlett Memorial Lecture. Quarterly Journal of Experimental Psychology. 1991;43B:1–23. [Google Scholar]
- Rescorla RA. Experimental extinction. In: Mowrer RR, Klein SB, editors. Handbook of contemporary learning theories. Mahwah, NJ: Erlbaum; 2001. pp. 119–154. [Google Scholar]
- Rescorla RA, Skucy JC. Effect of response-independent reinforcers during extinction. Journal of Comparative and Physiological Psychology. 1969;67:381–389. [Google Scholar]
- Rescorla RA, Solomon RL. Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. Psychological Review. 1967;74:151–182. doi: 10.1037/h0024475. [DOI] [PubMed] [Google Scholar]
- Ressler KJ, Rothbaum BO, Tannenbaum L, Anderson P, Graap K, Zimand E, Hodges L, Davis M. Cognitive enhancers as adjuncts to psychotherapy: use of D-cycloserine in phobic individuals to facilitate extinction of fear. Arch Gen Psychiatry. 2004;61:1136–1144. doi: 10.1001/archpsyc.61.11.1136. [DOI] [PubMed] [Google Scholar]
- Shaw D, Norwood K, Sharp K, Quigley L, McGovern SF, Leslie JC. Facilitation of extinction of operant behaviour in mice by D-cycloserine. Psychopharmacology. 2009;202:397–402. doi: 10.1007/s00213-008-1312-7. [DOI] [PubMed] [Google Scholar]
- Thanos PK, Bermeo C, Wang GJ, Volkow ND. D-cycloserine accelerates the extinction of cocaine-induced conditioned place preference in C57BL/c mice. Behavioral Brain Research. 2009;199:345–349. doi: 10.1016/j.bbr.2008.12.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Timberlake W. Motivational modes in behavior systems. In: Mowrer RR, Klein SB, editors. Handbook of contemporary learning theories. Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers; 2001. pp. 155–209. [Google Scholar]
- Torregrossa MM, Sanchez H, Taylor JR. D-cycloserine reduces the context specificity of Pavlovian extinction of cocaine cues through actions in the nucleus accumbens. The Journal of Neuroscience. 2010;30:10526–10533. doi: 10.1523/JNEUROSCI.2523-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Trapold MA, Overmier JB. The second learning process in instrumental learning. In: Black AH, Prokasy WF, editors. Classical Conditioning II: Current Theory and Research. New York: Appleton-Century-Crofts; 1972. pp. 427–452. [Google Scholar]
- Vengeliene V, Kiefer F, Spanagel R. D-Cycloserine facilitates extinction of conditioned alcohol-seeking behaviour in rats. Alcohol and Alcoholism. 2008;43:626–629. doi: 10.1093/alcalc/agn067. [DOI] [PubMed] [Google Scholar]
- Walker DL, Ressler KJ, Lu KT, Davis M. Facilitation of conditioned fear extinction by systemic administration or intra-amygdala infusions of D-cycloserine as assessed with fear-potentiated startle in rats. Journal of Neuroscience. 2002;15:2343–2351. doi: 10.1523/JNEUROSCI.22-06-02343.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weber M, Hart J, Richardson R. Effects of D-cycloserine on extinction of learned fear to an olfactory cue. Neurobiology of Learning and Memory. 2007;87:476–482. doi: 10.1016/j.nlm.2006.12.010. [DOI] [PubMed] [Google Scholar]
- Winterbauer NE, Bouton ME. Mechanisms of resurgence of an extinguished instrumental behavior. Journal of Experimental Psychology: Animal Behavior Processes. 2010;36:343–353. doi: 10.1037/a0017365. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Winterbauer NE, Bouton ME. Mechanisms of resurgence II: Response-contingent reinforcers can reinstate a second extinguished behavior. Learning and Motivation. 2011 doi: 10.1016/j.lmot.2011.01.002. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Woods AM, Bouton ME. D-cycloserine facilitates extinction but does not eliminate renewal of the conditioned emotional response. Behavioral Neuroscience. 2006;120:1159–1162. doi: 10.1037/0735-7044.120.5.1159. [DOI] [PubMed] [Google Scholar]