Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Sep 1.
Published in final edited form as: Neurobiol Learn Mem. 2019 Jul 15;163:107045. doi: 10.1016/j.nlm.2019.107045

Sign-tracking behavior is difficult to extinguish and resistant to multiple cognitive enhancers

Christopher J Fitzpatrick 1, Trevor Geary 2, Justin F Creeden 2, Jonathan D Morrow 1,2
PMCID: PMC6773616  NIHMSID: NIHMS1050822  PMID: 31319166

Abstract

The attribution of incentive-motivational value to drug-related cues underlies relapse and craving in drug addiction. One method of addiction treatment, cue-exposure therapy, utilizes repeated presentations of drug-related cues in the absence of drug (i.e., extinction learning); however, its efficacy has been limited due to an incomplete understanding of extinction and relapse processes after cues have been imbued with incentive-motivational value. To investigate this, we used a Pavlovian conditioned approach procedure to screen for rats that attribute incentive-motivational value to reward-related cues (sign-trackers; STs) or those that do not (goal-trackers; GTs). In Experiment 1, rats underwent Pavlovian extinction followed by reinstatement and spontaneous recovery tests. For comparison, a separate group of rats underwent PCA training followed by operant conditioning, extinction, and tests of reinstatement and spontaneous recovery. In Experiment 2, three cognitive enhancers (sodium butyrate, D-cycloserine, and fibroblast growth factor 2) were administered following extinction training to facilitate extinction learning. STs but not GTs displayed enduring resistance to Pavlovian, but not operant, extinction and were more susceptible to spontaneous recovery. In addition, none of the cognitive enhancers tested affected extinction learning. These results expand our understanding of extinction learning by demonstrating that there is individual variation in extinction and relapse processes and highlight potential difficulties in applying extinction-based therapies to drug addiction treatment in the clinic.

Keywords: Pavlovian conditioned approach, autoshaping, extinction, reinstatement, reinforcement, learning

1. Introduction

The attribution of incentive-motivational value to drug-related cues is believed to underlie craving and relapse in addiction (Robinson and Berridge, 1993). In support of this, reactivity to drug-related cues before and during treatment is associated with craving (Witteman et al., 2015) and relapse (Garland et al., 2012; Janes et al., 2010; Papachristou et al., 2014). Reducing the strength of drug-related memories, for example through extinction learning, is one way to decrease craving and facilitate addiction treatment. In the context of addiction, extinction is defined as a learning process that reduces the frequency or intensity of conditioned responses (CRs; e.g., craving) to drug-related cues. During extinction learning, the drug-related cue, or conditioned stimulus (CS; e.g., drug paraphernalia), is presented in the absence of the drug, or unconditioned stimulus (US; e.g., cocaine), which results in a new learning process that inhibits expression of the original learning (Bouton, 2004; Delamater, 2004; Myers and Davis, 2002).

Because patients suffering from addiction exhibit enduring cue reactivity even after treatment, a better understanding of extinction learning is required to increase its effectiveness in therapeutic settings. In the 1970s, individual variation was observed in abstinence patterns following behavioral treatment for addiction, and it was hypothesized that relapse was causally related to inadequate extinction learning established during treatment (Hunt et al., 1971; Litman et al., 1979). During the 1990s, cue-exposure therapy was advocated as a potential psychotherapy for addiction (Heather and Bradley, 1990), and initial studies appeared promising in decreasing cue reactivity (Franken et al., 1999) and relapse (Drummond and Glautier, 1994). Subsequent studies, however, did not replicate these findings, and a meta-analysis of cue-exposure studies showed no consistent evidence of efficacy (Conklin and Tiffany, 2002). However, animal extinction research upon which these treatments were based has advanced tremendously since that time, and exposure-based therapies might be informed and improved by novel discoveries in extinction learning. For example, little is known regarding how reward-related cues that are imbued with incentive-motivational value extinguish and relapse.

Our laboratory uses Pavlovian conditioned approach (PCA) procedures to investigate individual variation in the attribution of incentive-motivational value to reward-related cues (Robinson and Flagel, 2009). During PCA training, a conditioned stimulus (CS; e.g., a lever) response-independently predicts the delivery of an unconditioned stimulus (US; e.g., a food pellet). During training, three patterns of CRs develop: sign-tracking (CS-directed CRs), goal-tracking (US-directed CRs), and an intermediate response (both CRs). Previous research has shown that sign-trackers (STs), but not goal-trackers (GTs), attribute reward-related cues with incentive-motivational value, transforming them into powerful motivators of behavior (Flagel et al., 2009) that can promote addiction-like behaviors, such as cue-induced reinstatement of drug-seeking (Saunders and Robinson, 2010; Yager et al., 2015). It is also clear that sign-tracking behaviors directed toward reward-related cues imbued with incentive-motivational value are more resistant to extinction than goal-tracking behavior (Ahrens et al., 2016; Beckmann and Chow, 2015). It is unknown, however, (1) how long resistance to extinction of sign-tracking behavior persists, (2) whether PCA phenotypes differentially undergo relapse processes like reinstatement (recovery of conditioned responding as a result of US presentation) and spontaneous recovery (recovery of conditioned responding due to the passage of time), or (3) whether drugs currently being investigated as cognitive enhancers during extinction-based therapies are effective in promoting extinction of sign-tracking behavior (Quirk and Mueller, 2008).

To address this, rats received PCA training followed by extinction training and two tests for relapse-related processes: reinstatement and spontaneous recovery. In addition, a separate group of rats underwent PCA training followed by operant conditioning, extinction and tests for reinstatement and spontaneous recovery. Operant procedures were performed in parallel to determine whether differences in extinction learning are unique to Pavlovian procedures or a generalized feature of associative learning. Finally, three cognitive enhancers—sodium butyrate (NaB; a histone deacetylase inhibitor), D-cycloserine (DCS; an N-methyl-D-aspartate receptor [NMDAR] partial agonist), and fibroblast growth factor 2 (FGF2)—were administered during extinction training to facilitate the extinction of sign-tracking behavior. These drugs were selected because they have been previously shown to enhance extinction learning in animal models (Graham and Richardson, 2009; 2010; Paolone et al., 2009; Zhu et al., 2017).

2. Methods and Materials

2.1. Animals

One hundred sixty-eight adult male Sprague Dawley rats (275–300 g) were purchased from Charles River Laboratories and Envigo (formerly known as Harlan Laboratories). In Experiment 1, rats were used equally from Charles River Laboratories (Barriers C72, R04, and R09) and Envigo (202A, 202C, 217, and 206). In Experiment 2, rats were used exclusively from Charles River Laboratories (Barrier C72). Rats were maintained on a 12 h light/dark cycle, and standard rodent chow and water were available ad libitum. All procedures were approved by the University Committee on the Use and Care of Animals (University of Michigan; Ann Arbor, MI).

2.2. Drugs

Sodium butyrate (NaB; #sc-202341; Santa Cruz Biotechnology, Inc.; Dallas, TX), D-cycloserine (DCS; #C6880; Sigma-Aldrich, Inc.; St. Louis, MO), and fibroblast growth factor 2 (FGF2; #45103P; QED Bioscience, Inc.; San Diego, CA) were used. In Experiment 1, NaB (200 mg/kg, i.p.; 10 mL/kg) was dissolved in saline (pH = 7.34), which was also used as the vehicle control. In Experiment 2, DCS (30 mg/kg, i.p.; 1 mL/kg) was dissolved in saline (pH = 7.34) and FGF2 (20 μg/kg, i.p.; 2 mL/kg) was dissolved in 0.1% bovine serum albumin (Sigma Aldrich, Inc.) in 0.1 M phosphate-buffered saline (pH = 7.34). A combination of saline (1 mL/kg, i.p.) and 0.1% bovine serum albumin in 0.1 M phosphate-buffered saline (1 mL/kg, s.c.) was used as the vehicle control, and drug groups received drug and the opposite vehicle, so that each group had the same amount of injections. Doses of DCS (30 mg/kg; i.p.) and FGF2 (20 μg/kg; s.c.) were selected based upon previous studies showing that these doses enhance extinction training (Graham and Richardson, 2009; Vurbic et al., 2011). The dose of NaB (200 mg/kg; i.p.) was selected because it has been shown to enhance experience-dependent histone acetylation (Kumar et al., 2005).

2.3. Apparatus

Modular conditioning chambers (24.1 cm width × 20.5 cm depth × 29.2 cm height; MED Associates, Inc.; St. Albans, VT) were used for Pavlovian and operant conditioning. Each chamber was in a sound-attenuating cubicle equipped with a ventilation fan to provide ambient white noise. For Pavlovian conditioning, chambers were equipped with a pellet magazine, an illuminated retractable lever (counterbalanced on the left or right of the pellet magazine), and a red house light on the wall opposite to the pellet magazine. When inserted into the chamber, the retractable lever was illuminated by an LED light within the lever housing. A pellet dispenser delivered banana-flavored food pellets into the pellet magazine, and an infrared sensor inside the pellet magazine detected head entries. For operant conditioning, the lever was removed from the chamber and replaced with two nose-poke ports on either side of the pellet magazine.

2.4. Pavlovian Conditioned Approach: Procedure

For two days prior to pretraining, rats were familiarized with banana-flavored food pellets (45 mg; Bioserv; Frenchtown, NJ) in their home cages. Twenty-four hours later, rats were placed into the operant chambers and underwent one pretraining session during which the red house-light remained on, but the lever was retracted. Fifty food pellets were delivered on a variable time (VT) 30 schedule (i.e., one food pellet was delivered on average every 30 s, but actual delivery varied between 0–60 s). All rats consumed all the food pellets by the end of the pretraining session. Twenty-four hours later, rats underwent daily PCA training sessions over seven days. Each trial during a test session consisted of extension of the illuminated lever (the CS) into the chamber for 8 s on a VT 90 schedule (30–150 s). Retraction of the lever was immediately followed by the response-independent delivery of one food pellet (the US) into the pellet magazine. Each test session consisted of 25 trials of CS-US pairings, resulting in a total session length of approximately 40 min. All rats consumed all the food pellets that were delivered.

Following PCA training, approximately half of the rats underwent 29 daily PCA extinction sessions during which the CS was presented 25 times on the same schedule as in previous PCA sessions, but during extinction sessions CS presentation not did result in the delivery of a food pellet. Next, rats underwent a reinstatement test during which five food pellets were delivered at the beginning of the session followed immediately by an extinction session. Rats then underwent 10 more PCA extinction sessions followed by a spontaneous recovery test two weeks later. During the spontaneous recovery test, rats were placed back into chambers and behavior was recorded under extinction conditions. During Pavlovian extinction, reinstatement, and spontaneous recovery, the first three CRs were counted for statistical analysis. Probability of CRs (i.e. proportion of CS presentations in a given extinction session during which the animal engaged in the CR) was used as the dependent measure so that sign- and goal-tracking could be directly compared. For detailed analysis of extinction recall, the first and last CRs of sessions were counted for statistical analysis, and number of CRs was used as the dependent measure.

2.5. Operant Conditioning: Procedure

Following PCA training, approximately half of the rats underwent 12 operant conditioning sessions. Chocolate-flavored food pellets were used instead of banana-flavored pellets so that Pavlovian and operant procedures would not have the same US. Operant conditioning consisted of a 50-min session with three within-session phases: food pellet delivery on a fixed ratio (FR) 1 schedule for the first five food pellet deliveries (i.e., one active nose-poke response resulted in food pellet delivery), an FR 7 schedule for the next five food pellet deliveries (i.e., seven active nose-poke responses resulted in pellet delivery), then a variable ratio 20 (VR 20) schedule for all further food pellet deliveries (i.e., the number of active nose-poke responses required to result in pellet delivery varied randomly between 5 and 35). In our experience, this operant schedule produces behavior that is learned and stably expressed over approximately the same number and length of sessions as PCA behavior. Following operant conditioning, rats underwent six sessions of extinction training (50 min) during which active nose-poke responses no longer resulted in food pellet delivery. Next, rats underwent a reinstatement test during which five food pellets were delivered at the beginning of the session followed immediately by an extinction session. Rats then underwent two more operant extinction sessions followed by a spontaneous recovery test two weeks later. During the spontaneous recovery test, rats were placed back into chambers, and behavior was recorded under extinction conditions.

2.6. Statistical Analysis

PCA behavior was scored using an index that combines the number, latency, and probability of lever presses (sign-tracking CRs) and magazine entries (goal-tracking CRs) during CS presentations within a session. Briefly, we averaged together the response bias (i.e., number of lever presses and magazine entries for a session; [lever presses – magazine entries] / [lever presses + magazine entries]), latency score (i.e., average latency to perform a lever press or magazine entry during a session; [magazine entry latency – lever press latency]/8 s), and probability difference (i.e., proportion of lever presses or magazine entries; lever press probability – magazine entry probability). The index scores behavior from +1.0 (absolute sign-tracking) to −1.0 (absolute goal-tracking), with 0 representing no bias (Meyer et al., 2012). PCA index scores averaged over Sessions 6–7 were used to classify rats with the following cutoffs: STs (x ≥ 0.5), IRs (−0.5 < x < 0.5), and GTs (x ≤ −0.5).

SPSS (Version 22; IBM, Inc.) was used for all statistical analysis. For all linear mixed models, the covariance structure was selected based upon Akaike’s information criterion (i.e., the lowest number criterion represents the highest quality statistical model using a given covariance structure). Behaviors during PCA and operant training and extinction were analyzed using a linear mixed model, and tests of reinstatement and spontaneous recovery were analyzed using repeated measures analysis of variance (ANOVA). Extinction recall was analyzed using two-way ANOVA. With a significant ANOVA, multiple comparisons were performed using Fisher’s Least Significant Difference (LSD) post hoc test.

For analysis of latency to magazine entry following lever-CS retraction, the first magazine entry after the first CS period was used during the first four PCA extinction training sessions. Due to the variable nature of intertrial periods (30–150 s), latencies were capped at 30 s to standardize measurements across CS periods. Linear regressions were used to determine relationships between PCA index scores and variables related to Pavlovian conditioning (e.g. reinstatement and spontaneous recovery) or operant conditioning (e.g., extinction rate, reinstatement, and spontaneous recovery). To calculate extinction rate, a linear regression (for conditioning) or quadratic regression (for extinction) was selected based upon the shape of the data and performed for each individual rat, then the B coefficient of these regressions was used as a measure conditioning or extinction rate.

3. Results

3.1. Experiment 1: PCA training

Rats underwent seven daily sessions of PCA training and were classified as STs (n = 36), intermediate responders (IRs; n = 29), and GTs (n = 30) based upon their average PCA index scores over Sessions 6 and 7. Figure 1 shows that there was an effect of Phenotype on lever press number (Fig. 1A; F(2,130.74) = 95.2, p = 3.0 × 10−26), latency (Fig. 1C; F(2,114.31) = 38.4, p = 1.8 × 10−13), and probability (Fig. 1E; F(2,124) = 101, p = 1.1 × 10−26) as well as magazine entry number (Fig. 1B; F(2,123.63) = 51.3, p = 6.2 × 10−17), latency (Fig. 1D; F(2,127.70) = 70.9, p = 1.9 × 10−21), and probability (Fig. 1F; F(2,124.89) = 81.7, p = 2.1 × 10−23). Following PCA training, rats were divided into two groups. The first group underwent PCA extinction and testing for reinstatement and spontaneous recovery (ST, n = 18; IR, n = 17, GT, n = 12); the second group underwent operant training, extinction, and testing for reinstatement and spontaneous recovery (ST, n = 18; IR, n = 12, GT, n = 18).

Figure 1.

Figure 1.

Open in a new tab

Rats underwent Pavlovian conditioned approach training over seven daily sessions and were classified as sign-trackers (STs), intermediate-responders (IRs), or goal-trackers (GTs) based on the average lever press and magazine entry number, latency, and probability during Sessions 6 and 7. Data are mean and S.E.M. *** - p < 0.001.

3.2. Experiment 1: PCA extinction, reinstatement, and spontaneous recovery

Figure 2 shows extinction and extinction recall of PCA behavior in STs (n = 18), IRs (n = 17), and GTs (n = 12). Over 29 daily sessions of PCA extinction training, STs and GTs extinguished their conditioned responding (Figure 2A; effect of Session: F(28,407.14) = 10.1, p = 5.3 × 10−32); however, extinction of sign-tracking CRs in STs was weaker than that of goal-tracking CRs in GTs (effect of Phenotype: F(1,223.78) = 55.5, p = 2.0 × 10−12). Goal-tracking probability was very low in STs, but lever-pressing in GTs was high enough to discern a slow extinction curve, similar to the pattern observed in STs and IRs (Fig. S1). In addition, IRs extinguished their conditioned responding (Figure 2B; effect of Session: F(28,497.57) = 5.13, p = 5.6 × 10−15), but extinction of sign-tracking CRs was weaker than that of goal-tracking CRs in IRs (effect of Response: F(1,184.56) = 89.5, p = 1.5 × 10−17). Analysis of the first and last CS trials during the last PCA session and the first four PCA extinction sessions revealed that STs extinguished sign-tracking within sessions, but did not extinguish it between sessions (Figure 2C; effect of CS: F(1,170) = 132.27, p = 5.21 × 10−23). Post hoc comparisons revealed that STs sign-tracked less on the last CS trial compared to the first CS trial on every extinction session (ps < 0.05); however, sign-tracking on the first CS trial during extinction sessions was not different from the last PCA session (ps > 0.05). Similarly, IRs extinguished sign-tracking within-session, but did not extinguish it between-session (Figure 2E; effect of CS: F(1,110) = 24.90, p = 4.08 × 10−11). Post hoc comparisons revealed that IRs sign-tracked less on the last CS trial compared to the first CS trial on every extinction session (ps < 0.05), but sign-tracking on the first CS trial during extinction sessions was not different from the last PCA session (ps > 0.05). In contrast, GTs extinguished goal-tracking both within sessions and between sessions (Figure 2D; effect of CS: F(1,110) = 11.14, p = 0.001). Post hoc comparisons revealed that GTs goal-tracked less on the last CS trial compared to the first CS trial on the first extinction session (p < 0.001), and goal-tracking on subsequent sessions was low and similar between the first and last CS trials (ps > 0.05). In addition, goal-tracking on the first CS trial of Extinction Sessions 2–4 were significantly lower than the first CS trial of the last PCA session (ps < 0.01). Finally, IRs extinguished their goal-tracking within sessions, but not between sessions (Figure 2F; effect of CS: F(1,50) = 14.25, p = 2.21 × 10−4). Post hoc comparisons revealed that IRs goal-tracked less on the last CS trial compared to the first CS trial on the first extinction session (p < 0.01), and goal-tracking on subsequent sessions was low and similar between the first and last CS trials (ps > 0.05). Goal-tracking on the first CS trial of the last PCA session and extinction sessions, however, was too low to observe between-session extinction (ps > 0.05). Moreover, extinction recall was unaffected by behavior directed towards the magazine outside the CS period and all rats increasingly stopped checking the magazine for pellets (Fig. S2; effect of Session: F(3,122.44) = 3.9, p = 0.01; effect of Phenotype: F(2,61.62) = 1.3, p = 0.28).

Figure 2.

Figure 2.

Open in a new tab

Sign-trackers (STs), intermediate-responders (IRs), and goal-trackers (GTs) were tested for extinction of Pavlovian conditioned approach (PCA) behavior. (A-B) Rats underwent 29 PCA extinction sessions, and the probabilities of sign-tracking conditioned responses (lever press; LP) in STs, goal-tracking conditioned responses (magazine entry; ME) in GTs, and both conditioned responses in IRs were compared. (C-F) To investigate extinction recall of PCA behavior, the first and last conditioned stimulus (CS) within a session were compared between the last session of PCA training and the first four sessions of PCA extinction. Data are mean and S.E.M. * - p < 0.05, *** - p < 0.001, within-subjects comparisons; ## - p < 0.01, between-subjects comparison.

Figure 3 shows tests of reinstatement and spontaneous recovery following PCA extinction sessions. Behavior was compared against the last PCA extinction session (baseline). For the reinstatement test, the baseline was the last Pavlovian extinction session (i.e., pre-reinstatement extinction training); for the spontaneous recovery test, the baseline session was the last Pavlovian re-extinction session (i.e., post-reinstatement extinction session). For GTs and STs, exposure to the US did not reinstate behavior (Figure 3A; effect of Session: F(1,27) = 3.13, p = 0.088; effect of Phenotype: F(1,53) = 3.57, p = 0.064; interaction of Phenotype × Session: F(1,28) = 0.011, p = 0.92). For IRs, exposure to the US reinstated behavior (Figure 3B; effect of Session: F(1,32) = 22.48, p = 4.21 × 10−5). There were no differences, however, between the reinstatement of sign- and goal-tracking CRs (effect of Response: F(1,32) = 0.48, p = 0.49; interaction of Response × Session: F(1,32) = 0.12, p = 0.74). Following the reinstatement test, STs and GTs extinguished conditioned responding over 10 daily PCA extinction sessions (Fig. S3A; effect of Session: F(9,165.23) = 2.54, p = 0.009), and STs continued to extinguish their lever-press CRs slower than GTs extinguished their magazine-entry CRs (effect of Phenotype: F(1,73.87) = 13.2, p = 5.3 × 10−4; interaction of Phenotype × Session: F(1,165.24) = 0.45, p = 0.91). Moreover, IRs extinguished conditioned responding as well (Fig. S3B; effect of Session: F(1,192.47) = 1.97, p = 0.044), and they continued to extinguish their lever-press CRs slower than their magazine-entry CRs (effect of Response: F(1,70.16) = 13.8, p = 4.1 × 10−4; interaction of Response × Session: F(1,192.47) = 0.59, p = 0.80).

Figure 3.

Figure 3.

Open in a new tab

Following extinction of Pavlovian conditioned approach behavior, sign-trackers (STs), intermediate-responders (IRs) and goal-trackers (GTs) underwent tests for (A-B) reinstatement and (C-D) spontaneous recovery of lever press (LP) and magazine entry (ME) conditioned responses. Baseline refers to the most recent PCA extinction training session relative to the specific test session. Data are mean and S.E.M. ** - p < 0.01, *** - p < 0.001, within-subjects comparisons; ## - p < 0.009, between-subjects comparison.

Two weeks after PCA re-extinction training, rats underwent a spontaneous recovery test, and behavior was compared against the last PCA extinction session (baseline). For GTs and STs, conditioned responding spontaneously recovered (Figure 3C; effect of Session: F(1,27) = 39.21, p = 1.04 × 10−6), but CRs recovered differently (effect of Phenotype: F(1,27) = 6.41, p = 0.017; interaction of Phenotype × Session: F(1,27) = 5.05, p = 0.033). STs and GTs both spontaneously recovered their respective CRs; however, STs spontaneously recovered conditioned responding to a greater degree (p = 0.009). For IRs, conditioned responding spontaneously recovered (Figure 3D; effect of Session: F(1,34) = 11.83, p = 0.002). There were no differences, however, between the spontaneous recovery of sign- and goal-tracking CRs in the IRs (effect of Response: F(1,34) = 1.88, p = 0.18; interaction of Response × Test: F(1,34) = 0.02, p = 0.89).

3.3. Experiment 1: Operant conditioning, extinction, reinstatement, and spontaneous recovery

Figure 4A shows that over 12 sessions of operant conditioning rats (STs, n = 18; IRs, n = 12; GTs, n = 18) successfully acquired the operant response (effect of Session: F(11, 427.97) = 15.1, p = 8.3 × 10−25) and discriminated between active and inactive nose-poke ports (effect of Port: F(1, 129.97) = 219, p = 1.2 × 10−24). In addition, there were no differences in active nose-poke responses between phenotypes (effect of Phenotype: F(2,129.97) = 1.98, p = 0.14). Following operant training, rats underwent six operant extinction sessions, during which rats successfully extinguished their active nose-poke responses (Figure 4B; effect of Session: F(5, 298.32) = 66.4, p = 1.9 × 10−46). There was no difference in extinction between phenotypes (effect of Phenotype: F(2, 71.74) = 0.34, p = 0.72). Following operant extinction, rats underwent a reinstatement test, and their behavior was compared against the last operant extinction session (baseline). Exposure to the US reinstated behavior (Figure 4C; effect of Session: F(1,45) = 30.95, p = 1.38 × 10−6) without differences between phenotypes (effect of Phenotype: F(2,45) = 2.22, p = 0.12; interaction of Phenotype × Session: F(2,45) = 2.57, p = 0.087). Following the reinstatement test, rats were extinguished over two daily operant extinction sessions. After two weeks following operant extinction training, rats underwent a spontaneous recovery test, and behavior was compared against the last operant extinction session (baseline). Operant responding spontaneously recovered (Figure 4D; effect of Session: F(1,45) = 11.65, p = 1.37 × 10−3) without differences between phenotypes (effect of Phenotype: F(2,45) = 0.67, p = 0.52; interaction of Phenotype × Session: F(2,45) = 0.17, p = 0.85). In addition, linear regressions were used to compare PCA index scores with extinction rate, reinstatement, and spontaneous recovery of active nose-poke responses. There was a correlation between PCA index scores and reinstatement (Fig. S4C; r2 = 0.34, p = 0.02), but not conditioning rate (Fig. S4A; r2 = 0.05, p = 0.63), extinction rate (Fig. S4B; r2 = 0.01, p = 0.48) or spontaneous recovery (Fig. S4D; r2 = 0.16, p = 0.28).

Figure 4.

Figure 4.

Open in a new tab

Sign-trackers (STs), intermediate-responders (IRs), and goal-trackers (GTs) were tested for operant conditioning, extinction, reinstatement, and spontaneous recovery, and their active and inactive nose-poke (NP) responses were recorded during each session. Baseline refers to the most recent operant extinction training session relative to the specific test session. Data are mean and S.E.M.

3.4. Experiment 2: The effect of sodium butyrate on PCA extinction

Seven days following the spontaneous recovery test in Experiment 1, rats underwent three daily additional PCA training sessions to restore conditioned responding followed by four daily PCA extinctions. After each PCA extinction session, rats immediately received NaB or saline. Figure 5A–B shows that STs and GTs extinguished their conditioned responding (effect of Session: F(3,109.67) = 6.50, p = 4.4 × 10−4), but sign-tracking CRs in STs were more resistant to extinction than goal-tracking CRs in GTs (effect of Phenotype: F(1, 59.57) = 17.6, p = 1.1 × 10−4). However, NaB did not affect extinction training in either STs or GTs (effect of Drug: F(2, 49.45) = 0.16, p = 0.90; interaction of Drug × Phenotype: F(1, 49.47) = 0.16, p = 0.69; interaction of Drug × Phenotype × Session: F(6, 109.67) = 0.61, p = 0.61). Similarly, IRs extinguished their conditioned responding (effect of Session: F(3,95.62) = 6.40, p = 0.001), and sign-tracking CRs were more resistant to extinction than goal-tracking CRs in IRs (effect of Response: F(1,40.85) = 30.0, p = 2.4 × 10−6). However, NaB did not affect extinction training in IRs (effect of Drug: F(1, 40.86) = 3.02, p = 0.09; interaction of Drug × Response: F(1, 40.85) = 0.095, p = 0.76; interaction of Drug × Response × Session: F(3, 95.62) = 0.93, p = 0.43). Also, differences in prior behavioral training and duration of testing between the operant and Pavlovian training groups did not affect the conditioned responding during extinction training of sign-tracking in STs and goal-tracking in GTs (effect of Cohort: F(1,49.45) = 0.74, p = 0.39) or sign- and goal-tracking in IRs (effect of Cohort: F(1,34.02) = 0.67, p = 0.42).

Figure 5.

Figure 5.

Open in a new tab

Sign-trackers (STs), goal-trackers (GTs), and intermediate-responders (IRs) underwent four sessions of Pavlovian extinction. Immediately following each session, rats were administered (A-B) sodium butyrate (NaB; 200 mg/kg, i.p.), (C-D) D-cycloserine (DCS; 30 mg/kg, i.p.), or (E-F) fibroblast growth factor 2 (FGF2; 20 μg/kg, s.c.), and their respective vehicles. The last session of Pavlovian conditioned approach (PCA) training is shown for illustration and not statistical comparison. Data are mean and S.E.M.

3.5. Experiment 2: The effect of DCS and FGF2 on PCA extinction

In a new cohort, rats underwent seven daily sessions of PCA training and were classified as STs (n = 19), IRs (n = 26), and GTs (n = 19) based upon their average PCA index scores over Sessions 6 and 7. There was an effect of Phenotype on lever press number (F(2,85.77) = 52.8, p = 1.14 × 10−15), latency (F(2,86.35) = 71.1, p = 5.58 × 10−19), and probability (F(2,84.56) = 88.9, p = 1.6 × 10−21) as well as magazine entry number (F(2,118.64) = 38.0, p = 1.74 × 10−13), latency (F(2,88.59) = 42.0, p = 1.43 × 10−13), and probability (F(2,117.19) = 61.0, p = 6.99 × 10−19). Twenty-four hours following PCA training, rats were divided into two groups and underwent four daily sessions of PCA extinction, each followed immediately by a vehicle, DCS, or FGF2 injection. As mentioned previously, because DCS and FGF2 have different vehicle solutions, vehicle-treated rats received two vehicle injections and drug-treated rats received drug and the additional vehicle solution. Figure 5C–D shows that STs and GTs extinguished their conditioned responding (effect of Session: F(3,62.63) = 21.15, p = 1.4 × 10−9), but sign-tracking CRs in STs were more resistant to extinction than goal-tracking CRs in GTs (effect of Phenotype: F(1, 29.61) = 23.9, p = 3.29 × 10−5). In addition, DCS did not affect extinction training in either STs or GTs (effect of Drug: F(2,29.61) = 0.70, p = 0.41; interaction of Drug × Phenotype: F(1,29.61) = 0.40, p = 0.53; interaction of Drug × Phenotype × Session: F(6,62.63) = 0.62, p = 0.60). Similarly, IRs extinguished their conditioned responding (effect of Session: F(3,84.44) = 4.60, p = 4.99 × 10−3), and sign-tracking CRs were more resistant to extinction than goal-tracking CRs in IRs (effect of Response: F(1,39.59) = 40.12, p = 1.67 × 10−7). In addition, DCS did not affect extinction training of either CR in IRs (effect of Drug; F(2,39.59) = 0.40, p = 0.53: interaction of Drug × Response: F(1,39.59) = 0.016, p = 0.90; interaction of Drug × Response × Session: F(6,84.44) = 0.26, p = 0.86). Figure 5E–F shows that STs and GTs extinguished their conditioned responding (effect of Session: F(3,64.53) = 13.32, p = 7.25 × 10−6), but sign-tracking CRs in STs were more resistant to extinction than goal-tracking CRs in GTs (effect of Phenotype: F(1,30.51) = 30.06, p = 5.66 × 10−6). In addition, FGF2 did not affect extinction training in either STs or GTs (effect of Drug: F(2,30.51) = 1.41, p = 0.25; interaction of Drug × Phenotype: F(1,30.51) = 1.91, p = 0.18; interaction of Drug × Phenotype × Session: F(6,64.57) = 0.92, p = 0.44). Similarly, IRs extinguished their conditioned responding (effect of Session: F(3,85.08) = 7.21, p = 2.27 × 10−4), and sign-tracking CRs were more resistant to extinction than goal-tracking CRs in IRs (effect of Response: F(1,38.96) = 55.75, p = 4.97 × 10−9). In addition, FGF2 did not affect extinction training of either CR in IRs (effect of Drug; F(2,38.96) = 0.52, p = 0.47: interaction of Drug × Response: F(1,38.96) = 1.08, p = 0.31; interaction of Drug × Response × Session: F(6,85.08) = 0.84, p = 0.48).

4. Discussion

In the current study, we demonstrated individual differences in Pavlovian extinction and spontaneous recovery of reward-related cues that have or have not been imbued with incentive-motivational value. For example, sign-tracking behavior in STs was more resistant to extinction than goal-tracking in GTs, replicating the results of two previous studies that used similar procedures but did not test reinstatement or spontaneous recovery (Beckmann & Chow, 2015; Ahrens et al., 2016). The present study goes a step further by showing how much additional training is necessary to fully extinguish sign-tracking behavior. Additionally, the data presented here show that sign-tracking behavior in IRs was more resistant to extinction than goal-tracking behavior, suggesting that the observed differences lie in the behaviors as opposed to the individual animals. This finding is novel, as previous studies did not examine sign- and goal-tracking behaviors in IRs. These extinction deficits in sign-tracking are the result of between-session deficits in extinction recall whereas within-session extinction is relatively unimpaired. In addition, sign-tracking behavior in STs exhibits higher spontaneous recovery than goal-tracking behavior in GTs following extinction. In contrast, operant conditioning, extinction, reinstatement, and spontaneous recovery were not different between phenotypes, which supports previous findings that they do not differ in operant learning (Saunders and Robinson, 2010; 2011). Thus, extinction deficits in STs are specific to Pavlovian procedures during which reward-related cues are imbued with incentive-motivational value (Ahrens et al., 2016; Beckmann and Chow, 2015).

A number of limitations apply to the interpretation of this study, or indeed any study that attempts to directly compare the psychological properties of sign- and goal-tracking. Though sign- and goal-tracking may develop under identical training conditions, these two conditioned behaviors are phenomenologically quite distinct. For example, outcome measures like number of lever presses may not be reflective of the same level of learning and motivation as number of magazine entries because the way animals physically interact with those two stimuli are so different. Sign-tracking often involves biting and grabbing of the CS, which is measured by mechanical deflections of the lever, whereas goal-tracking involves investigation of the US location, measured by infrared beam breaks within the magazine cavity. We tried to focus our analysis on the probability of a sign- or goal-tracking response on any given trial because this measure is probably the least affected by simple physical differences between the two behaviors, but this can only partially address the inherent problem of non-equivalence. Even more problematic in this regard is any attempt to directly compare operant to Pavlovian behaviors. We therefore did not attempt to match many of the parameters between the Pavlovian and operant tasks, other than general session length and overall rate of acquisition. Instead, we focused these experiments on comparing individual differences between sign- and goal-trackers in aspects of operant learning. It is entirely possible that testing under different operant conditions would reveal differences that were not apparent in this study. For example, the data depicted in Figure 4A might indicate a trend toward faster acquisition of operant tasks in sign-trackers. However, the lack of any major differences observed in operant learning under these conditions, and particularly in extinction of operant behavior, gives us more confidence in concluding that sign- and goal-trackers do not differ because of any general deficits in associative learning processes per se. The consistent detection of differences in extinction of sign- and goal-tracking even among IRs, where both behaviors are measured in the same animal, gives further credence to the interpretation that STs and GTs do not differ in general learning ability.

Significant individual differences in extinction learning have been observed in rodents (Shumake et al., 2014) and humans (Gershman and Hartley, 2015; Lommen et al., 2013). Contemporary learning theory posits that extinction learning likely involves the formation of new CS-no US associations that inhibit the original CS-US associations (Konorski, 1948; Rescorla, 1993; Rescorla, 2001; Hall, 2002). One understudied aspect of extinction, however, is whether it uniformly influences different associations between the CS and US (i.e., sensory and motivational properties; Konorski, 1967; Delamater and Oakeshott, 2007), which would have important implications for learning theory and the study of individual differences. Certain theories of extinction (i.e., AESOP theory) have acknowledged the utility of understanding these differences (Wagner and Brandon, 1989) and PCA procedures can separate at least two associations between the CS and US: incentive-motivational (sign-tracking CR) and sensory-predictive (goal-tracking CR).

In the present study, the extinction of goal-tracking generally supports the classical view of Pavlovian extinction using the Rescorla-Wagner model (Rescorla and Wagner, 1972) and negative prediction errors (Delamater and Westbrook, 2014; Schultz, 1998; Schultz et al., 1997). The extinction of sign-tracking, however, is not as consistent with the assumptions of the Rescorla-Wagner model and negative prediction errors. Based on these assumptions, Pavlovian extinction causes negative prediction errors (i.e., an absence of reward is poorer than the predicted outcome) that directly result in decreased strength of the original CS-US association and allow for the formation of new CS-no US associations. It is possible that the incentive-motivational value of the CS somehow interferes with processing of negative prediction errors, resulting in persistent behavior directed towards the CS during Pavlovian extinction training. Although it is currently unknown what behavioral and neurobiological mechanisms underlie the extinction of sign-tracking, our results suggest deficient memory consolidation, because sign-tracking extinguishes within- but not between-session. One possibility is that the attribution of incentive-motivational value to a CS impairs extinction learning by increasing attention to the CS, because extinction is commonly viewed as reduced processing of and attention to a CS undergoing nonreinforcement(Robbins, 1990).

There are several brain regions that could be responsible for resistance to extinction of sign-tracking behavior. First, it may result from hypoactivity in the prefrontal cortex (PFC), because it has previously been shown that temporary inactivation or NMDAR antagonism of the nidopallium caudolaterale (the functional analogue of the mammalian PFC) impairs the extinction of sign-tracking behavior in pigeons (Lengersdorf et al., 2015; Lengersdorf et al., 2014). Within-session extinction does not require the infralimbic (IL) cortex of the PFC; however, consolidation of within-session extinction does require the IL (Quirk et al., 2000). Because sign-tracking rats have decreased levels of BDNF in the PFC (Morrow et al., 2015), and BDNF in the IL is both necessary and sufficient for Pavlovian extinction (Rosas-Vidal et al., 2014), it is possible that decreased levels of BDNF in the IL underlie extinction deficits in STs. Second, resistance to extinction could result from persistent cue-induced activation of the basolateral amygdala (BLA), which has increased neural activity in STs relative to GTs under acute extinction conditions (Yager et al., 2015). Alternatively, abnormal activity during extinction learning of sign-tracking behavior may occur in both regions, because it is known that a BLA-IL circuit contributes to the formation and extinction of reward-related memories (Rosen et al., 2015; Xin et al., 2014). Interestingly, IRs showed similar resistance to extinction of sign- but not goal-tracking behavior, suggesting that these circuits may be differentially engaged between the two behaviors in IRs and supporting the notion that sign- and goal-tracking behaviors are learned and extinguished through unique and dissociable systems even in the same animal (Clark et al., 2012).

As previously mentioned, increased spontaneous recovery of sign-tracking in STs compared to goal-tracking in GTs may result from the same processes that cause individual variation in resistance to extinction. Spontaneous recovery results from the fact that extinction memories inhibit, rather than erase, conditioning memories and are susceptible to weakening over time (Todd et al., 2014). Previously, it has been demonstrated that extinction is associated with increased excitability of IL neurons, which is reversed during spontaneous recovery (Cruz et al., 2014); therefore, resistance to extinction and enhanced spontaneous recovery may stem from similar impairments in synaptic plasticity in the IL. Moreover, and as previously described, STs have increased neural activity in the BLA relative to GTs under acute extinction conditions, and BLA activity during extinction predicts the strength of subsequent spontaneous recovery (Courtin et al., 2014) whereas its inactivation decreases spontaneous recovery (Peters et al., 2008).

Recently, there has been a great interest in using cognitive enhancers to facilitate extinction learning (Kaplan and Moore, 2011). In the current study, three promising cognitive enhancers were given during the extinction of sign- and goal-tracking behaviors, yet none of these compounds facilitated extinction learning. Interestingly, all three were unsuccessful even though they enhance extinction learning through separate mechanisms. Each compound either increases synaptic plasticity through glutamate receptor modulation (DCS), histone acetylation (NaB), or growth factor-mediated intracellular signaling (FGF2).

There are several important considerations as to why these cognitive enhancers were ineffective at promoting extinction of sign-tracking. First, the vast majority of research investigating Pavlovian extinction learning and its pharmacological enhancement have stemmed from fear conditioning in animals (Todd et al., 2014) and anxiety disorders in humans (Hofmann et al., 2015). Although it is known that there is an overlap in the circuits (Peters et al., 2009) and signaling pathways (Heldt et al., 2014) underlying appetitive and aversive learning, it is also known that there are important differences in these processes (Hebart and Glascher, 2015; Knapska et al., 2006; Reichelt and Lee, 2013). Indeed, most behavioral studies of extinction stem from appetitive conditioning procedures whereas most neuropharmacological studies of extinction stem from aversive conditioning procedures (Delamater, 2004). Second, there is a possibility that these drugs might have been effective if given before Pavlovian extinction training. In previous studies, DCS facilitates extinction both when administered before (Woods & Bouton, 2006; Bouton et al., 2008) and after (Weber et al., 2007; Toth et al., 2012) Pavlovian conditioning (for a review on the identities and timing of other pharmacological manipulations, see Fitzgerald et al., 2014). In the current study, we targeted the consolidation, rather than the acquisition, phases of extinction learning due to our observation that sign-tracking behavior shows within-session extinction but impaired recall of extinction learning. Third, although doses were based on extinction studies in the existing literature, it is possible that different concentrations of the drugs used in the current study are necessary to facilitate the extinction of sign-tracking behavior. Further research is needed to determine the exact mechanisms underlying impaired extinction of sign-tracking behavior, investigate dose-response relationships of the drugs administered in the current study, and explore other pharmacological treatments targeting different signaling pathways (e.g., cannabinoid, opioid, noradrenergic, etc.) that might more effectively weaken resistance to extinction of sign-tracking behavior (Singewald et al., 2015).

5. Conclusion

Our results demonstrate that when reward-related cues are attributed with incentive-motivational value they become enduringly resistant to extinction and more prone to spontaneous recovery. Accordingly, current models of extinction should be updated to support individual variation in extinction of different CRs in response to the same CS. In addition, our results provide insight into several translationally relevant questions regarding the poor efficacy of exposure therapy for addiction (Kantak and Nic Dhonnchadha, 2011). For example, the poor efficacy of cue exposure therapy may not solely be the result of drug-induced neurocognitive impairments in drug-addicted patients, but rather a fundamental difference in the extinction sensitivity of cues imbued with incentive-motivational properties. Moreover, and based upon our results, caution should be taken when broadly applying research results from studies of Pavlovian fear conditioning to appetitive processes, particularly pharmacological investigations using cognitive enhancers. In the future, studies of appetitive Pavlovian extinction would benefit from the incorporation of PCA procedures to identify individual differences in the attribution of incentive-motivational value.

Supplementary Material

Supplemental Material

Acknowledgements

We would like to thank Dr. Terry Robinson for material support of a portion of these studies and for contributions to the initial study design. This work was funded by the University of Michigan Department of Psychiatry (U032826 [JDM]) and Rackham Graduate School (Predoctoral Fellowship, CJF), the Department of Defense (DoD) National Defense Science and Engineering Graduate (NDSEG) Fellowship (CJF), the Brain & Behavior Research Foundation (NARSAD 20829 [JDM]), and the National Institute on Drug Abuse (NIDA; K08 DA037912–01 [JDM]; T32 DA007281 [CJF]).

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

  1. Ahrens AM, Singer BF, Fitzpatrick CJ, Morrow JD, & Robinson TE (2016). Rats that sign-track are resistant to Pavlovian but not instrumental extinction. Behav Brain Res, 296, 418–430. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Beckmann JS, & Chow JJ (2015). Isolating the incentive salience of reward-associated stimuli: value, choice, and persistence. Learn Mem, 22, 116–127. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bouton ME (2004). Context and behavioral processes in extinction. Learn Mem, 11, 485–494. [DOI] [PubMed] [Google Scholar]
  4. Bouton ME, Vurbic D, & Woods AM (2008). D-cycloserine facilitates context-specific fear extinction learning. Neurobiol Learn Mem, 90(3): 504–510. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Clark JJ, Hollon NG, & Phillips PE (2012). Pavlovian valuation systems in learning and decision making. Curr Opin Neurobiol, 22, 1054–1061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Conklin CA, & Tiffany ST (2002). Applying extinction research and theory to cue-exposure addiction treatments. Addiction, 97, 155–167. [DOI] [PubMed] [Google Scholar]
  7. Courtin J, Karalis N, Gonzalez-Campo C, Wurtz H, & Herry C (2014). Persistence of amygdala gamma oscillations during extinction learning predicts spontaneous fear recovery. Neurobiol Learn Mem, 113, 82–89. [DOI] [PubMed] [Google Scholar]
  8. Cruz E, Lopez AV, & Porter JT (2014). Spontaneous recovery of fear reverses extinction-induced excitability of infralimbic neurons. PLoS One, 9, e103596. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Delamater AR (2004). Experimental extinction in Pavlovian conditioning: behavioural and neuroscience perspectives. Q J Exp Psychol B, 57, 97–132. [DOI] [PubMed] [Google Scholar]
  10. Delamater AR, & Oakeshott S (2007). Learning about multiple attributes of reward in Pavlovian conditioning. Ann N Y Acad Sci, 1104, 1–20. [DOI] [PubMed] [Google Scholar]
  11. Delamater AR, & Westbrook RF (2014). Psychological and neural mechanisms of experimental extinction: a selective review. Neurobiol Learn Mem, 108, 38–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Drummond DC, & Glautier S (1994). A controlled trial of cue exposure treatment in alcohol dependence. J Consult Clin Psychol, 62, 809–817. [DOI] [PubMed] [Google Scholar]
  13. Fitzgerald PJ, Seemann JR, & Maren S (2014). Can fear extinction be enhanced? A review of pharmacological and behavioral findings. Brain Res Bull, 105: 46–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Flagel SB, Akil H, & Robinson TE (2009). Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction. Neuropharmacology, 56 Suppl 1, 139–148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Franken IH, de Haan HA, van der Meer CW, Haffmans PM, & Hendriks VM (1999). Cue reactivity and effects of cue exposure in abstinent posttreatment drug users. J Subst Abuse Treat, 16, 81–85. [DOI] [PubMed] [Google Scholar]
  16. Garland EL, Franken IH, & Howard MO (2012). Cue-elicited heart rate variability and attentional bias predict alcohol relapse following treatment. Psychopharmacology (Berl), 222, 17–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Gershman SJ, & Hartley CA (2015). Individual differences in learning predict the return of fear. Learn Behav, 43, 243–250. [DOI] [PubMed] [Google Scholar]
  18. Graham BM, & Richardson R (2009). Acute systemic fibroblast growth factor-2 enhances long-term extinction of fear and reduces reinstatement in rats. Neuropsychopharmacology, 34, 1875–1882. [DOI] [PubMed] [Google Scholar]
  19. Graham BM, & Richardson R (2010). Fibroblast growth factor-2 enhances extinction and reduces renewal of conditioned fear. Neuropsychopharmacology, 35, 1348–1355. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Hall G (2002). Associative structures in Pavlovian and instrumental conditioning In Pashler H & Gallistel R (Eds.), Steven’s handbook of experimental psychology, Vol. 3: Learning, motivation, and emotion (3rd ed., pp. 1–45). New York: John Wiley & Sons, Inc. [Google Scholar]
  21. Heather N, & Bradley BP (1990). Cue exposure as a practical treatment for addictive disorders: why are we waiting? Addict Behav, 15, 335–337. [DOI] [PubMed] [Google Scholar]
  22. Hebart MN, & Glascher J (2015). Serotonin and dopamine differentially affect appetitive and aversive general Pavlovian-to-instrumental transfer. Psychopharmacology (Berl), 232, 437–451. [DOI] [PubMed] [Google Scholar]
  23. Heldt SA, Zimmermann K, Parker K, Gaval M, Weinshenker D, & Ressler KJ (2014). BDNF deletion or TrkB impairment in amygdala inhibits both appetitive and aversive learning. J Neurosci, 34, 2444–2450. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hofmann SG, Mundy EA, & Curtiss J (2015). Neuroenhancement of Exposure Therapy in Anxiety Disorders. AIMS Neurosci, 2, 123–138. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hunt WA, Barnett LW, & Branch LG (1971). Relapse rates in addiction programs. J Clin Psychol, 27, 455–456. [DOI] [PubMed] [Google Scholar]
  26. Janes AC, Pizzagalli DA, Richardt S, de B. FB, Chuzi S, Pachas G, Culhane MA, Holmes AJ, Fava M, Evins AE, & Kaufman MJ (2010). Brain reactivity to smoking cues prior to smoking cessation predicts ability to maintain tobacco abstinence. Biol Psychiatry, 67, 722–729. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kantak KM, & Nic Dhonnchadha BA (2011). Pharmacological enhancement of drug cue extinction learning: translational challenges. Ann N Y Acad Sci, 1216, 122–137. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kaplan GB, & Moore KA (2011). The use of cognitive enhancers in animal models of fear extinction. Pharmacol Biochem Behav, 99, 217–228. [DOI] [PubMed] [Google Scholar]
  29. Knapska E, Walasek G, Nikolaev E, Neuhausser-Wespy F, Lipp HP, Kaczmarek L, & Werka T (2006). Differential involvement of the central amygdala in appetitive versus aversive learning. Learn Mem, 13, 192–200. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Konorski J (1948). Conditioned reflexes and neuron organization. New York: Cambridge University Press. [Google Scholar]
  31. Konorski J (1967). Integrative activity of the brain. Chicago: University of Chicago Press. [Google Scholar]
  32. Lengersdorf D, Marks D, Uengoer M, Stuttgen MC, & Gunturkun O (2015). Blocking NMDA-receptors in the pigeon’s “prefrontal” caudal nidopallium impairs appetitive extinction learning in a sign-tracking paradigm. Front Behav Neurosci, 9, 85. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Lengersdorf D, Stuttgen MC, Uengoer M, & Gunturkun O (2014). Transient inactivation of the pigeon hippocampus or the nidopallium caudolaterale during extinction learning impairs extinction retrieval in an appetitive conditioning paradigm. Behav Brain Res, 265, 93–100. [DOI] [PubMed] [Google Scholar]
  34. Litman GK, Eiser JR, & Taylor C (1979). Dependence, relapse and extinction: a theoretical critique and a behavioral examination. J Clin Psychol, 35, 192–199. [DOI] [PubMed] [Google Scholar]
  35. Lommen MJ, Engelhard IM, Sijbrandij M, van den Hout MA, & Hermans D (2013). Pre-trauma individual differences in extinction learning predict posttraumatic stress. Behav Res Ther, 51, 63–67. [DOI] [PubMed] [Google Scholar]
  36. Morrow JD, Saunders BT, Maren S, & Robinson TE (2015). Sign-tracking to an appetitive cue predicts incubation of conditioned fear in rats. Behav Brain Res, 276, 59–66. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Myers KM, & Davis M (2002). Behavioral and neural analysis of extinction. Neuron, 36, 567–584. [DOI] [PubMed] [Google Scholar]
  38. Paolone G, Botreau F, & Stewart J (2009). The facilitative effects of D-cycloserine on extinction of a cocaine-induced conditioned place preference can be long lasting and resistant to reinstatement. Psychopharmacology (Berl), 202, 403–409. [DOI] [PubMed] [Google Scholar]
  39. Papachristou H, Nederkoorn C, Giesen JC, & Jansen A (2014). Cue reactivity during treatment, and not impulsivity, predicts an initial lapse after treatment in alcohol use disorders. Addict Behav, 39, 737–739. [DOI] [PubMed] [Google Scholar]
  40. Peters J, Kalivas PW, & Quirk GJ (2009). Extinction circuits for fear and addiction overlap in prefrontal cortex. Learn Mem, 16, 279–288. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Peters J, Vallone J, Laurendi K, & Kalivas PW (2008). Opposing roles for the ventral prefrontal cortex and the basolateral amygdala on the spontaneous recovery of cocaine-seeking in rats. Psychopharmacology (Berl), 197, 319–326. [DOI] [PubMed] [Google Scholar]
  42. Quirk GJ, & Mueller D (2008). Neural mechanisms of extinction learning and retrieval. Neuropsychopharmacology, 33, 56–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Quirk GJ, Russo GK, Barron JL, & Lebron K (2000). The role of ventromedial prefrontal cortex in the recovery of extinguished fear. J Neurosci, 20, 6225–6231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Reichelt AC, & Lee JL (2013). Memory reconsolidation in aversive and appetitive settings. Front Behav Neurosci, 7, 118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Rescorla RA, and Wagner AR (1972). A theory of Pavlovian conditioning variations in the effectiveness of a reinforcement and nonreinforcement In Black AHP & Prokasy WF (Eds.), Classical conditioning II: Current research and theory (pp. 64–99). New York: Appleton-Century-Crofts. [Google Scholar]
  46. Rescorla RA (1993). Inhibitory associations between S and R in extinction. Animal Learning & Behavior, 21(4), 327–336. [Google Scholar]
  47. Rescorla RA (2001). Experimental extinction In Mowrer RR & Klein SB (Eds.), Handbook of contemporary learning theories (pp. 119–154). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. [Google Scholar]
  48. Robbins SJ (1990). Mechanisms underlying spontaneous recovery in autoshaping. Animal Behavior Processes, 16(3), 235–249. [Google Scholar]
  49. Robinson TE, & Berridge KC (1993). The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res Brain Res Rev, 18, 247–291. [DOI] [PubMed] [Google Scholar]
  50. Robinson TE, & Flagel SB (2009). Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences. Biol Psychiatry, 65, 869–873. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Rosas-Vidal LE, Do-Monte FH, Sotres-Bayon F, & Quirk GJ (2014). Hippocampal--prefrontal BDNF and memory for fear extinction. Neuropsychopharmacology, 39, 2161–2169. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Rosen LG, Sun N, Rushlow W, & Laviolette SR (2015). Molecular and neuronal plasticity mechanisms in the amygdala-prefrontal cortical circuit: implications for opiate addiction memory formation. Front Neurosci, 9, 399. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Saunders BT, & Robinson TE (2010). A cocaine cue acts as an incentive stimulus in some but not others: implications for addiction. Biol Psychiatry, 67, 730–736. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Saunders BT, & Robinson TE (2011). Individual variation in the motivational properties of cocaine. Neuropsychopharmacology, 36, 1668–1676. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Schultz W (1998). Predictive reward signal of dopamine neurons. J Neurophysiol, 80, 1–27. [DOI] [PubMed] [Google Scholar]
  56. Schultz W, Dayan P, & Montague PR (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599. [DOI] [PubMed] [Google Scholar]
  57. Shumake J, Furgeson-Moreira S, & Monfils MH (2014). Predictability and heritability of individual differences in fear learning. Anim Cogn, 17, 1207–1221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Singewald N, Schmuckermair C, Whittle N, Holmes A, & Ressler KJ (2015). Pharmacology of cognitive enhancers for exposure-based therapy of fear, anxiety and trauma-related disorders. Pharmacol Ther, 149, 150–190. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Todd TP, Vurbic D, & Bouton ME (2014). Behavioral and neurobiological mechanisms of extinction in Pavlovian and instrumental learning. Neurobiol Learn Mem, 108, 52–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Toth I, Dietz M, Peterlik D, Huber SE, Fendt M, Neumann ID, Flor PJ, & Slattery DA (2012). Pharmacological interference with metabotropic glutamate receptor subtype 7 but not subtype 5 differentially affects within- and between-session extinction of Pavlovian conditioned fear. Neuropharmacology, 62(4): 1619–1626. [DOI] [PubMed] [Google Scholar]
  61. Vurbic D, Gold B, & Bouton ME (2011). Effects of D-cycloserine on the extinction of appetitive operant learning. Behav Neurosci, 125, 551–559. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Wagner AR, & Brandon SE (1989). Evolution of a structured connectionist model of a Pavlovian conditioning (AESOP) In Klein SB & Mowrer RR (Eds.), Contemporary learning theories: Pavlovian conditioning and the status of learning theory (pp. 149–189). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. [Google Scholar]
  63. Weber M, Hart J, & Richardson R (2007). Effects of D-cycloserine on extinction of learned fear to an olfactory cue. Neurobiol Learn Mem, 87(4): 476–482. [DOI] [PubMed] [Google Scholar]
  64. Witteman J, Post H, Tarvainen M, de Bruijn A, Perna Ede S, Ramaekers JG, & Wiers RW (2015). Cue reactivity and its relation to craving and relapse in alcohol dependence: a combined laboratory and field study. Psychopharmacology (Berl), 232, 3685–3696. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Woods AM, & Bouton ME (2006). D-cycloserine facilitates extinction but does not eliminate renewal of the conditioned emotional response. Behav Neurosci, 120(5): 1159–1162. [DOI] [PubMed] [Google Scholar]
  66. Xin J, Ma L, Zhang TY, Yu H, Wang Y, Kong L, & Chen ZY (2014). Involvement of BDNF signaling transmission from basolateral amygdala to infralimbic prefrontal cortex in conditioned taste aversion extinction. J Neurosci, 34, 7302–7313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Yager LM, Pitchers KK, Flagel SB, & Robinson TE (2015). Individual variation in the motivational and neurobiological effects of an opioid cue. Neuropsychopharmacology, 40, 1269–1277. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Zhu J, Zhao N, Chen Y, Zhu L, Zhong Q, Liu J, & Chen T (2017). Sodium butyrate modulates a methamphetamine-induced conditioned place preference. J Neurosci Res, 95, 1044–1052. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material
NIHMS1050822-supplement-Supplemental_Material.docx (280.8KB, docx)

RESOURCES