Skip to main content
NCBI home page
Learning & Memory logoLink to Learning & Memory
. 2010 Mar;17(3):148–154. doi: 10.1101/lm.1687310

A molecular dissociation between cued and contextual appetitive learning

Mazen A Kheirbek 1,4,5, Jeff A Beeler 2, Wanhao Chi 2, Yoshihiro Ishikawa 3, Xiaoxi Zhuang 2
PMCID: PMC2832921  PMID: 20189959

Abstract

In appetitive Pavlovian learning, animals learn to associate discrete cues or environmental contexts with rewarding outcomes, and these cues and/or contexts can potentiate an ongoing instrumental response for reward. Although anatomical substrates underlying cued and contextual learning have been proposed, it remains unknown whether specific molecular signaling pathways within the striatum underlie one form of learning or the other. Here, we show that while the striatum-enriched isoform of adenylyl cyclase (AC5) is required for cued appetitive Pavlovian learning, it is not required for contextual appetitive learning. Mice lacking AC5 (AC5KO) could not learn an appetitive Pavlovian learning task in which a discrete signal light predicted reward delivery, yet they could form associations between context and either natural or drug reward, which could in turn elicit Pavlovian approach behavior. However, unlike wild-type (WT) mice, AC5KO mice could not use these Pavlovian conditioned stimuli to potentiate ongoing instrumental behavior in a Pavlovian-to-instrumental transfer paradigm. These data suggest that AC5 is specifically required for learning associations between discrete cues and outcomes in which the temporal relationship between conditioned stimulus (CS) and unconditioned stimulus (US) is essential, while alternative signaling mechanisms may underlie the formation of associations between context and reward. In addition, loss of AC5 compromises the ability of both contextual and discrete cues to modulate instrumental behavior.

In Pavlovian learning, animals form associations between discrete or contextual stimuli in their environment to shape their behavior and make appropriate responses. In discrete cue appetitive Pavlovian conditioning, a single cue with a defined onset and offset that typically activates one sensory modality is provided, immediately followed by reward delivery (Hall 2002; Domjan 2006; Ito et al. 2006). Alternatively, behavior can be driven by context, an assortment of stimuli activating a number of sensory modalities that contribute to the representation of environmental space (Balsam 1985; Rudy and Sutherland 1995; Smith and Mizumori 2006). Collectively, these stimuli make up a context that is paired with reward delivery in contextual appetitive learning. One important distinction between these two forms of learning is that in cued conditioning, there is a discrete temporal relationship between conditioned stimulus (CS) and unconditioned stimulus (US). Thus, an animal can effectively anticipate timing of reward delivery from onset and offset of CS. In vivo studies of dopamine (DA) neuron activity have suggested this discrete temporal relationship can be encoded by DA neurons (Schultz et al. 1997; Schultz 1998a). In contrast, in many contextual Pavlovian conditioning tasks, US delivery is not predicted, it is delivered as the animal explores the environment; thus, the temporal relationship between contextual stimuli and reinforcement is not an essential component of the learned associations (Fanselow 2000). These two types of environmental stimuli may be encoded differently and mediated by different neural substrates.

Lesion studies have elucidated the anatomical dissociations between cued and contextual appetitive learning. Using a modified Y-maze procedure, it has been suggested that contextual appetitive learning is hippocampus- and nucleus-accumbens (NAc) dependent, while cued learning is dependent on the basolateral nucleus of the amygdala (BLA) and the NAc (Ito et al. 2005, 2006). In addition, as the NAc processes glutamatergic inputs from the amygdala and the hippocampus (Groenewegen et al. 1999; Goto and Grace 2008), recent studies have indicated that disconnecting the hippocampus from the NAc shell can disrupt contextual appetitive conditioning (Ito et al. 2008). In addition to glutamatergic inputs, the NAc, as part of the ventral striatum, receives dense dopaminergic input from midbrain nuclei (Groenewegen et al. 1999). Temporal shifts in phasic DA release in striatal regions has been correlated with appetitive Pavlovian learning (Day et al. 2007), and models of striatal function suggest that DA-dependent modification of glutamatergic transmission in the striatum may underlie reinforcement learning (Reynolds et al. 2001; Reynolds and Wickens 2002).

The cAMP pathway has been implicated in plasticity and learning in a number of neuronal structures (Abel et al. 1997; Ferguson and Storm 2004; Pittenger et al. 2006). Adenylyl cyclase (AC), the enzyme that makes cAMP, has nine membrane-bound isoforms, each with different expression patterns and regulatory properties (Hanoune and Defer 2001). AC5 is highly enriched in the striatum, with very low levels of expression in other regions of the brain (Mons et al. 1998; Iwamoto et al. 2003; Kheirbek et al. 2008, 2009), and genetic deletion of AC5 (AC5KO) severely compromises DA's ability to modulate cAMP levels in the striatum (Iwamoto et al. 2003). Previous studies have shown that AC5KO mice were severely impaired in acquisition of a cued appetitive Pavlovian learning task, while formation of action–outcome contingencies in instrumental learning was intact (Kheirbek et al. 2008). Yet, it remains unknown whether the cAMP pathway in the striatum underlies all forms of appetitive Pavlovian learning, or how it contributes to the ability of Pavlovian cues to modulate instrumental behavior.

In this study, we asked if genetic deletion of AC5 selectively impairs cued or contextual appetitive learning. In addition, we tested whether loss of AC5 affects the ability of conditioned cues or contexts to modulate instrumental behavior. Our data indicate that although loss of AC5 abolishes cued appetitive learning, contextual learning is spared. Although contextual stimuli could elicit approach behavior in AC5KO mice, they could not potentiate an ongoing instrumental response, highlighting the importance of this isoform of AC in Pavlovian–instrumental interactions.

Results

AC5KO mice are impaired in discrete cue appetitive conditioning

Previously, we have shown that AC5KO mice lack the ability to use a discrete auditory Pavlovian CS to predict reward delivery (Kheirbek et al. 2008). To extend these findings, we tested whether AC5KO mice could use short, explicit visual cues to predict reward delivery. Mice were presented with either a 10-sec illumination of the left signal light and extension of the left lever (CS+), followed by reward delivery, or a 10-sec illumination of the right signal light and extension of the right lever (CS−), which was not followed by reward (Fig. 1). In each of the 14 daily sessions, mice were presented with 20 CS+ trials and 20 CS− trials with a 1-min variable intertrial interval (ITI). Formation of the Pavlovian contingency was assessed by recording head entries into the feeder immediately following CS presentation. Wild-type (WT) but not AC5KO mice learned to predict reward delivery based on CS+ (n = 7 WT, 8 AC5KO, genotype × stimulus interaction, F(1,24) = 9.79, P = 0.0046; genotype × session × stimulus interaction, F(13,312) = 4.164, P < 0.0001; Fig. 1A,C). We also analyzed each group separately. WT mice acquired a conditioned approach response to CS+ presentation and withheld approach after CS− presentation, suggesting formation of the Pavlovian CS–US contingency (ANOVA, stimulus effect F(1,10) = 9.47, P = 0.01; session × stimulus interaction F(13,130) = 7.27, P < 0.0001; Fig. 1). Unlike WT mice, AC5KO mice did not learn to discriminate between CS+ and CS− presentation (ANOVA, stimulus effect F(1,14) = 0.031, P = 0.86; session × stimulus interaction F(13,182) = 0.324, P = 0.9877; Fig. 1A). As a consequence of this inability to use the CS+ to predict reward delivery, unlike WT, AC5KO mice did not learn to withhold head entries during the ITI (ANOVA, genotype effect F(1,13) = 10.1, P = 0.01; day × genotype interaction F(13,169) = 2.86, P = 0.001; Fig. 1B). Analysis of head entry behavior around reward delivery revealed that WT mice learned to distribute their head entries to the 2-sec bin immediately following reward delivery, suggesting that the offset of CS+ and pellet drop served as a discrete cue eliciting a head entry response (stimulus effect F(1,10) = 8.66, P = 0.01; Fig. 1C). This was not seen in AC5KO mice, as they did not learn to differentiate between CS+ and CS− and did not distribute head entries to the CS+ like WT littermates (stimulus effect F(1,14) = 0.84, P = 0.37; genotype × stimulus effect F(9,106) = 7.98, P < 0.0001; Fig. 1C). Although a lack of peak responding in AC5KO could potentially arise from the mice placing their head in the hopper at the onset of the CS+ and waiting for the reward, this is unlikely, as their pattern of responding did not differentiate between the CS+ and CS−. Moreover, if this were the case we would expect to see a decrease in head entries while the mice kept their head in the hopper waiting for an expected reward, and we would expect them to distribute their head entries at the onset of the CS+, with an increase in this distribution across days as they learned the task, which we did not see. Finally, this deficit was not due to an approach to the CS, as there were barely any lever presses during the CS (Supplemental Fig. 1B). The lack of conditioned approach in AC5KO mice was not due to a suppression of responding as a consequence of CS presentation, as there was no significant difference in the rate of responding between CS and ITI times (stimulus effect F(2,21) = 0.045, P = 0.96; stimulus × day interaction F(26,273) = 0.383, P = 0.99; Fig. 1D). These data suggest an inability of AC5KO mice to use discrete visual cues to predict reward delivery.

Figure 1.

Figure 1.

Open in a new tab

AC5KO mice are impaired in discrete cue Pavlovian conditioning. (A) AC5KO mice did not differentiate between CS+ and CS− as revealed by head entries in the 2-sec bin following CS+ and CS− presentation. (Inset) Experimental design. (B) ITI head entry rate. (C) Head entries in WT (above) and AC5KO (below) in 2-sec bins during CS presentation, and 10 sec after reward delivery. (Inset) Data presentation. Data points represent head entries in 2-sec bins around reward delivery. (D) Head entry rate in AC5KO mice during CS presentation and ITI. (E) Second-order conditioning in AC5KO and WT mice. All error bars,±SEM.

To determine if a discrete CS could be used as a secondary reward in an instrumental conditioning paradigm, mice were tested for responding in a second-order schedule of reinforcement (Di Ciano and Everitt 2005). Mice were trained to press a lever for sucrose reward, in which each rewarded lever press was followed by the onset of the signal light above the lever for 1 sec, contingent on reward delivery (see Materials and Methods), and stable responding was established at a fixed interval (FI) 180-sec schedule of reinforcement. Then, mice were tested to determine if the CS acquired reinforcing properties by providing the CS after each lever press, noncontingent on reward delivery (FI180:FR1CS). Comparison of response rates during the first 15 min of the FI180 and the FI180:FR1CS sessions suggested a trend toward blunted potentiation of responding in AC5KO mice (genotype effect F(1,24) = 14.7, P = 0.0008; schedule effect F(1,24) = 5.56, P = 0.024; genotype × schedule interaction F(1,24) = 2.28, P = 0.1; Fig. 1D). Although genotype × schedule interaction was not statistically significant based on the conventional criterion, analysis of schedule effects in WT and AC5KO mice separately indicated a significant potentiation of lever pressing as a consequence of CS presentation in WT mice only (WT, t(12) = −2.2, P = 0.04; AC5KO, t(12) = −0.9, P = 0.38). These data suggest that AC5KO mice were impaired in the ability for CS to modulate their lever pressing behavior, consistent with a deficit in cued appetitive learning.

Normal appetitive conditioning to a long CS in AC5KO mice

In the conditioning paradigm described above, there existed a tight temporal relation between CS and US, as reward was delivered immediately after CS presentation. As AC5KO mice could not distribute head entry responses to CS offset and reward delivery like WT mice, we next asked whether AC5KO mice could learn a task where there was not a strict temporal predictability of the US. To test this we trained mice using a 2-min CS where reward was delivered at variable time points during CS presentation. Recent studies have suggested that in appetitive Pavlovian learning using a long cue CS rather than a discrete, short cue, CS can modulate different motivational and learning processes and engage different neurobiological substrates (Crombag et al. 2008). To test this, we used 2-min illumination of the testing chamber (dark→light) as a long cue CS for reward availability. Mice were tested for 14 d, four trials a day with each trial consisting of chamber illumination for 2 min (houselights and both signal lights), during which three sucrose pellets were delivered on a variable interval (VI) 15-sec schedule of reinforcement. Thus, unlike the discrete cue conditioning procedure, where pellets were delivered immediately after offset of CS in a temporally precise fashion, in this contextual learning task the illumination of the chamber indicated a reward rich context. Head entries into the feeder were measured in the 2 min before CS onset (pre-CS) and during CS presentation. Both AC5KO and WT mice increased their CS head entries across days, suggesting both genotypes could learn the context–US association (n = 8 WT, 8 AC5KO, genotype effect F(1,14) = 0.224, P = 0.64; session effect F(1,13) = 19.9, P < 0.0001; session × genotype interaction F(13,182) = 0.133, P = 0.19; Fig. 2A). In addition, both groups learned to reduce their pre-CS head entries as they learned the task (F(1,14) = 2.1 genotype effect P = 0.16; session effect F(1,13) = 14.3, P < 0.0001; session × genotype interaction F(13,182) = 1.7, P = 0.06; Fig. 2B). These data suggest AC5KO mice can form normal Pavlovian associations to long CSs.

Figure 2.

Figure 2.

Open in a new tab

Normal contextual Pavlovian conditioning for sucrose in AC5KO mice. (A) Experimental design. (B,C) Across sessions, like WT mice, AC5KO mice increased responding to the contextual CS (B) and reduced responding in the pre-CS period (C). Error bars,±SEM.

Normal appetitive contextual learning in AC5KO mice

We have construed the long stimulus as contextual due to the lack of a precise temporal relationship between the cue and the outcome(s), indicating an increased density of reward without specific temporal information about when reward will be delivered (Crombag et al. 2008). However, context is more broadly viewed as a configural set of cues that together comprise a composite cue indicating increased availability of reward, again without specific temporal information. These data indicate that AC5KO mice can learn contextual cues, as they could learn long CSs; and total head entries in the discrete cue procedure showed a drastic increase from the first to the second day of testing, suggesting they quickly learned the location of food delivery, as well as the context of the box being a reward-rich environment (Supplemental Fig. S1A). To confirm normal appetitive contextual learning in AC5KO mice, we tested mice in a cocaine-induced conditioned place preference (CPP) task. In this task, mice form a Pavlovian association between an environmental context and drug reward (Bardo and Bevins 2000). Acquisition of this association is tested in a drug-free condition for preference of the context previously paired with drug. Mice were given 10 conditioning sessions, alternating between cocaine- (20 mg/kg) and saline-paired chambers, with injections given immediately before placing mice in the chamber for 30 min. At this dose, both AC5KO and WT mice showed an acute locomotor response to cocaine injection on the first day of training, (n = 8 per genotype, drug effect F(1,14) = 60.9, P < 0.0001; genotype effect F(1,14) = 3.3, P = 0.09; drug × genotype interaction F(1,14) = 4.0, P = 0.06; Fig. 3A) and during the repeated injections on subsequent days training (ANOVA, genotype effect F(1,14) = 3.7, P = 0.073; drug effect F(1,14) = 65.4, P < 0.0001; session effect, F(4,56) = 3.19, P = 0.02; genotype × session effect F(4,56) = 0.8, P = 0.53; genotype × drug × session effect F(4,56) = 0.12, P = 0.98; Fig. 3A). On the preference test day, mice were placed in the apparatus drug-free and allowed access to both chambers. Both WT and AC5KO mice spent significantly more time in the chamber previously paired with cocaine, indicating similar acquisition of context–drug association (day 1, treatment effect WT t(14) = 4.7, P = 0.0003; AC5KO t(14) = 6.3. P < 0.0001; ANOVA genotype effect F(1,14) = 3.7, P = 0.07; treatment effect F(1,14) = 31.4, P < 0.0001; genotype × treatment interaction F(1,14) = 1.6, P = 0.23; Fig. 3B). In addition, both groups retained place preference up to 4 wk after the last pairing session, suggesting a similar ability of both genotypes to retain context–drug Pavlovian associations (days 7–28, ANOVA, treatment effect F(1,14) = 5.8, P = 0.03; genotype × treatment interaction, F(1,14) = 0.96, P = 0.76; Fig. 3B).

Figure 3.

Figure 3.

Open in a new tab

AC5KO mice acquire conditioned place preference for cocaine. (A) Locomotor response to 20 mg/kg cocaine in the repeated 30-min pairing sessions. (B) Percentage of time spent in drug- and saline-paired chambers at days indicated after conditioning. Error bars,±SEM.

Impaired Pavlovian-to-instrumental transfer in AC5KO mice

As AC5KO mice were able to learn context–reward associations, we next asked if a contextual cue previously paired with reward could increase the rate of ongoing instrumental responding for the same reward. In this Pavlovian–instrumental transfer (PIT) paradigm, mice that were trained in the 2-min chamber illumination paradigm described above were trained to acquire a novel lever press response to receive a sucrose reward. Mice acquired the lever press response at a FI 20 sec (FI20) schedule of reinforcement and then were trained for 2 d on random interval (RI) 30 and 3 d on a RI60 schedule of reinforcement. As we have previously reported (Kheirbek et al. 2008), lever press rate in AC5KO mice did not significantly differ from WT littermates on the last day of training (AC5KO 1.5 LP/min ± 0.46 SD, WT 1.8 LP/min ± 0.8, genotype effect t(12) = −0.81, P = 0.43). Twenty-four hours after the last day of RI60, mice were tested for PIT. Mice were given eight presentations of the 2 min CS, with a 2 min ITI. Transfer was measured as a change in response rate to the CS from the pre-CS period (Yin et al. 2006). Interestingly, while WT mice increased their responding during CS presentation, AC5KO mice decreased their responding, suggesting a deficit in PIT in AC5KO mice (CS−pre-CS, single sample t-test, hypothesized mean = 0, WT t(6) = 2.78, P = 0.03; AC5KO t(7) = 2.8, P = 0.02; ANOVA F(1,13) = 15.8, cue × genotype effect P = 0.0016; Fig. 4A). This difference in responding in AC5KO mice was not due to an increase in conditioned approach during CS presentation compared with that of WT, as cued head entries did not differ between the genotypes (ANOVA, genotype effect F(1,13) = 0.75, P = 0.4; genotype × cue interaction F(1,13) = 0.06, P = 0.81; Fig. 4B). This suggests impaired ability of a previously trained CS to stimulate ongoing lever pressing for reward in AC5KO mice. That the CS decreased responding in AC5KO mice is similar to our earlier studies in which we found that in WT mice, a CS decreased responding before Pavlovian training but increased responding after Pavlovian training (Sanders et al. 2007).

Figure 4.

Figure 4.

Open in a new tab

Impaired Pavlovian-to-instrumental transfer in AC5KO mice. (A) Potentiation of lever press responses in WT but not AC5KO during extinction test for transfer. Difference score calculated as (responses during CS presentation) − (responses during pre-CS period). (B) CS presentation did not alter head entry behavior in either group. Error bars,±SEM.

Discussion

The results presented here indicate that contextual and cued appetitive learning does not share the same molecular substrates within the striatum. Genetic deletion of AC5, the primary link between DA and cAMP in the striatum, selectively abolishes cued appetitive learning, while leaving contextual learning intact. In addition, deletion of AC5 disrupts the ability of Pavlovian stimuli, whether they are contextual or cued, to modulate instrumental behavior, suggesting a shared neural substrate between cued Pavlovian learning and Pavlovian–instrumental interactions.

The anatomical substrates underlying cued and contextual fear conditioning have been studied extensively (Phillips and LeDoux 1992; Fanselow 2000; LeDoux 2000). Lesion and inactivation studies have indicated that the hippocampus and amygdala play a significant role in contextual fear learning, while the amygdala is essential for cued fear learning (Phillips and LeDoux 1992). Recently, a similar dissociation has been proposed for cued versus contextual appetitive learning. Using a modified Y-maze procedure, these studies showed that lesions of this hippocampus rendered rats unable to learn information related to context–reward, while lesions of the BLA inhibited the ability to use a discrete visual cue to guide approach behavior (Ito et al. 2005, 2006). In addition, it has been demonstrated that connections between the hippocampus and the NAc shell form a circuit that underlies this form of contextual appetitive learning (Ito et al. 2008). It has also been shown that lesions of the core of the NAc inhibit discrete cue Pavlovian learning (Parkinson et al. 1999). As the NAc receives convergent input from amygdala and the hippocampus (Groenewegen et al. 1999; French and Totterdell 2002, 2003), it is possible that information about the nature of the reward-related cues carried by each of these inputs could be processed differently in the NAc.

In addition to the afferent inputs from the amygdala and the hippocampus, the NAc also receives DA input from the ventral tegmental area, which is directly modulated by reward-related cues during the learning process (Schultz 1998a) and modulates plasticity in striatal regions (Reynolds and Wickens 2002). Thus, the NAc provides an attractive target for study of molecular mechanisms that may be differentially modulated by either contextual or discrete conditioned cues. Here we show that AC5, an enzyme expressed throughout the NAc that regulates DA-mediated cAMP production (Iwamoto et al. 2003), is not required for contextual appetitive learning but is required for discrete cue learning. Short, discrete visual cues that immediately preceded reward delivery were unable to elicit conditioned approach behavior in AC5KO mice. Yet, AC5KO mice could learn that contexts were paired with rewards, and in the case of conditioned place preference for drug reward, preference for reward-related context could be detected 4 wk following conditioning. These findings imply that, at the molecular level, discrete and contextual cues are processed differently within the NAc. Although it is likely that loss of AC5 within the NAc accounts for these deficits, it should be noted that in a recent report, very low levels of AC5 were detected in cortical and hippocampal regions (Kheirbek et al. 2009). Future studies using more elegant, tissue-specific, genetic designs will be required to rule out the contribution of AC5 in these regions to discrete cue appetitive learning.

The finding that AC5KO mice lack of PIT suggests shared molecular substrates for discrete cue Pavlovian learning and PIT. A number of anatomical structures have been proposed to underlie PIT, including the ventral tegmental area, amygdala, and the shell of the NAc, a region with high AC5 expression (Corbit et al. 2001, 2007). This is supported by data suggesting that increasing local DA concentrations in the shell of the NAc can potentiate PIT (Wyvell and Berridge 2000). Thus, as hypothesized above for discrete cue appetitive Pavlovian learning, AC5 may be required for processing of the DA signal associated with the motivational effects of reward related stimuli on instrumental behavior. Alternatively, the houselight cue in the PIT paradigm may have both contextual and discrete cue properties. The contextual aspect signals a reward-rich environment, giving rise to increased exploratory approach behavior, which is spared in the AC5KO. Though the contingency is less precise, the same cue does contain temporal information that during the next 2 min pellets will be available. This temporal predictive association, impaired in the AC5KO, may be critical for PIT.

In addition to dissociating two forms of appetitive Pavlovian learning, these results may provide insight into how the temporal profile of DA release may underlie these two forms of learning. In vivo, it is known that DA neurons exhibit two modes of firing: tonic and phasic (Grace and Bunney 1984a,b; Overton and Clark 1997). Recordings of DA cell activity and measurements of DA release indicate that conditioned discrete cues elicit phasic activity in DA neurons leading to a phasic increase in DA in terminal regions (Schultz 1998b; Day et al. 2007). The regulatory properties of AC5 suggest it may be crucial for processing the phasic DA signal. AC5 is negatively regulated by protein kinase A (Iwami et al. 1995), a direct downstream target of activation of AC5, providing a short negative feedback loop constraining cAMP production. Thus, one would expect that very robust stimulation would be required to overcome this constraint. Phasic DA activation by discrete cues could fulfill this role, preferentially activating the lower affinity and extrasynaptically located D1 receptors (Richfield et al. 1989; Gonon 1997). Consistent with this, D1 receptor activation has been shown to be required for initial acquisition of a Pavlovian approach response to a discrete cue (Eyny and Horvitz 2003), and electrophysiological studies indicate that long-term potentiation of glutamatergic inputs in the striatum require D1 receptor activation (Reynolds et al. 2001). Applying this framework, we hypothesize that during acquisition of discrete cue appetitive Pavlovian conditioning, presentation of the US causes phasic release of DA, strengthening those inputs activated by the discrete stimulus that preceded the US. High levels of DA will activate D1 type receptors, stimulating AC5 and increasing cAMP levels permissive for LTP induction. In the absence of AC5, D1 receptor stimulation is decoupled from cAMP, blocking the ability of DA to strengthen appropriate synapses, thus weakening the discrete CS–US association.

Therefore, the data presented here suggest that a different mechanism may underlie contextual appetitive learning in the striatum. Although blockade of DA receptors inhibits contextual learning (Acquas et al. 1989; Ettenberg 1989), the underlying specific DA receptor signaling mechanisms remain unknown. It is possible that other AC isoforms with low expression in the striatum mediate contextual learning, while the role of AC5 is specific for discrete cue conditioning. This is supported by data indicating that global inhibition of PKA in the NAc inhibits acquisition of CPP for amphetamine (Gerdjikov et al. 2007). In addition, genetic deletion of AC1 and AC8 has been reported to decrease locomotor sensitization to cocaine (DiRocco et al. 2009). However, it is also possible that contextual learning, while being DA-dependent, is not mediated by the phasic DA signal but by increases in tonic levels of DA activating different downstream signaling pathways independent of AC and cAMP. Non-cAMP-dependent signaling pathways have been described that mediate some DA-dependent behaviors, including the β-arrestin 2-mediated pathway, which is a slower pathway than the cAMP pathway (Beaulieu et al. 2005), making it an attractive candidate for this form of learning. Alternatively, D1-type receptors can signal via a phospholipase-C (PLC)-dependent mechanism (Yu et al. 1996; Jin et al. 2003), and recent studies using optogenetic techniques suggest that stimulation of PLC-dependent pathways are sufficient for supporting contextual appetitive conditioning (Airan et al. 2009).

Taken together, our data suggest a very specific discrete cue Pavlovian learning deficit rather than a general deficit in reinforcement learning in AC5KO mice, as mice were able to learn a contextual conditioning task and an action–outcome association-mediated lever pressing task. Therefore, phasic DA signaling via the cAMP pathway may not be required for all forms of reinforcement learning. Future experiments elucidating the non-AC5-dependent signals that contribute to acquisition of action–outcome contingencies and contextual appetitive Pavlovian learning will provide insight into the molecular mechanisms of these forms of learning.

Materials and Methods

Mice

AC5KO mice, generated as previously described (Iwamoto et al. 2003), were backcrossed to C57BL/6 for eight generations. All mice tested were 8–12 wk of age. All animals were group-housed (four to five per /cage) in a temperature- and humidity-controlled barrier facility, with lights on/off at 06:00/18:00 h. All testing was conducted during the light phase. With the exception of conditioned place preference for drug reward, all mice were food restricted for behavioral experiments. Mice were fed regular chow ad libitum for 2 h each day following testing. All experiments were approved by the Institutional Animal Care and Use Committee of the University of Chicago.

Behavioral procedures

Cued appetitive conditioning

Cued appetitive conditioning was conducted in mouse operant conditioning chambers that have two retractable levers, a houselight, two signal lights above levers, a signal light, a nosepoke hole on the back wall, and a feeder with photobeam (Med Associates). Naïve, food-restricted mice were placed in the chamber, and sessions began with the onset of the fan. Each session consisted of 40 trials, 20 CS+ and 20 CS− in random order, with a VI 60-sec ITI. CS+ trials consisted of 10-sec illumination of the left signal light and extension of the left lever. After 10 sec, the signal light was turned off and the lever retracted, and a single sucrose pellet was delivered. In CS− trials, the right signal light was illuminated and right lever was extended for 10 sec, and no reward was delivered. Conditioned responses were measured as head entries into the feeder.

Contextual appetitive conditioning for sucrose

Naïve, food-restricted mice were placed in operant conditioning chambers, and sessions began with the onset of the fan, with the houselight off. The contextual CS was a 2-min presentation of the houselight and both signal lights. Each session consisted of four contextual CS presentations delivered at VI 5 min. During contextual CS presentation, three rewards were delivered at a VI 15-sec schedule of reinforcement during CS presentation, excluding the first 7 sec and last 30 sec of CS presentation to reduce the likelihood that mice would form associations between CS onset or offset with reward delivery. Head entries into the feeder were measured during the CS presentation and 2 min preceding CS presentation.

Conditioned place preference for cocaine

Tests were conducted as previously described (Beeler et al. 2009). Briefly, mice were placed in a conditioned place preference apparatus (MedAssociates) which consisted of two separate chambers with a removable gate for access to both sides of the chamber. The flooring differed on either side of the chamber, with wire mesh on one side and bars on the other. Mice were provided with five conditioning sessions for each chamber, alternating between saline and cocaine (20 mg/kg) for a total of 10 sessions. Mice were counterbalanced to eliminate any chamber bias. After training, mice were tested for preference by removing the gate between the chambers and allowing exploration of both sides drug-free for 30 min.

Pavlovian-to-instrumental transfer

After contextual appetitive conditioning for sucrose, mice were trained to acquire a lever press response for sucrose at a FI20 schedule of reinforcement. When mice reached learning criterion of 30 rewards in a 1-h session, they were trained for 2 d at RI30 and 2 d at RI60. Twenty-four hours after the last day of RI60 training, mice were trained for PIT. After an 8-min extinction period to reduce responding, mice were presented with eight 2-min CS presentations with a 2-min ITI. Transfer was calculated as the difference in responding between CS and pre-CS (Yin et al. 2006).

Second-order conditioning

Naïve, food-restricted mice were placed in operant conditioning chambers and trained to press a lever for sucrose pellets at FI20. During training, each rewarded lever press was followed by a 1-sec presentation of the signal light above the lever, followed by reward delivery. After mice reached learning criterion of 30 rewards in a 1-h session, mice were for trained for one session at each of the following schedules: FI40, FI60, FI90, FI120, and FI150, followed by 10 d at FI180 to acquire stable responding and the association between the CS and reward delivery. Twenty-four hours after the last FI180 session, mice were given a test session for second-order responding where the CS was given after each lever press, and the reward given at an FI180 schedule (FR1CS:FI180). Lever press response rates were analyzed for the first 15 min of the session and compared with the first 15 min of the FI180 session to control for extinction of the CS–US contingency.

Acknowledgments

We thank Shana Augustin for assistance with behavioral experiments. This work was supported in part by NIH F31MH076422 (M.A.K.), NIDA DA25875 (J.A.B.), HL059139 and GM067773 (Y.I.), and DA022269, MH66216, and the Edward Mallinckrodt, Jr. Foundation (X.Z.).

Footnotes

[Supplemental material is available online at http://www.learnmem.org.]

References

  1. Abel T, Nguyen PV, Barad M, Deuel TA, Kandel ER, Bourtchouladze R. Genetic demonstration of a role for PKA in the late phase of LTP and in hippocampus-based long-term memory. Cell. 1997;88:615–626. doi: 10.1016/s0092-8674(00)81904-2. [DOI] [PubMed] [Google Scholar]
  2. Acquas E, Carboni E, Leone P, Di Chiara G. SCH 23390 blocks drug-conditioned place-preference and place-aversion: Anhedonia (lack of reward) or apathy (lack of motivation) after dopamine-receptor blockade? Psychopharmacology. 1989;99:151–155. doi: 10.1007/BF00442800. [DOI] [PubMed] [Google Scholar]
  3. Airan RD, Thompson KR, Fenno LE, Bernstein H, Deisseroth K. Temporally precise in vivo control of intracellular signalling. Nature. 2009;458:1025–1029. doi: 10.1038/nature07926. [DOI] [PubMed] [Google Scholar]
  4. Balsam PD. The functions of context in learning and performance. In: Balsam PD, Tomie A, editors. Context and learning. Hillsdale, NJ: Erlbaum; 1985. pp. 1–21. [Google Scholar]
  5. Bardo MT, Bevins RA. Conditioned place preference: What does it add to our preclinical understanding of drug reward? Psychopharmacology. 2000;153:31–43. doi: 10.1007/s002130000569. [DOI] [PubMed] [Google Scholar]
  6. Beaulieu JM, Sotnikova TD, Marion S, Lefkowitz RJ, Gainetdinov RR, Caron MG. An Akt/β-arrestin 2/PP2A signaling complex mediates dopaminergic neurotransmission and behavior. Cell. 2005;122:261–273. doi: 10.1016/j.cell.2005.05.012. [DOI] [PubMed] [Google Scholar]
  7. Beeler JA, Cao ZF, Kheirbek MA, Zhuang X. Loss of cocaine locomotor response in Pitx3-deficient mice lacking a nigrostriatal pathway. Neuropsychopharmacology. 2009;34:1149–1161. doi: 10.1038/npp.2008.117. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Corbit LH, Muir JL, Balleine BW. The role of the nucleus accumbens in instrumental conditioning: Evidence of a functional dissociation between accumbens core and shell. J Neurosci. 2001;21:3251–3260. doi: 10.1523/JNEUROSCI.21-09-03251.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Corbit LH, Janak PH, Balleine BW. General and outcome-specific forms of Pavlovian-instrumental transfer: The effect of shifts in motivational state and inactivation of the ventral tegmental area. Eur J Neurosci. 2007;26:3141–3149. doi: 10.1111/j.1460-9568.2007.05934.x. [DOI] [PubMed] [Google Scholar]
  10. Crombag HS, Galarce EM, Holland PC. Pavlovian influences on goal-directed behavior in mice: The role of cue-reinforcer relations. Learn Mem. 2008;15:299–303. doi: 10.1101/lm.762508. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Day JJ, Roitman MF, Wightman RM, Carelli RM. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci. 2007;10:1020–1028. doi: 10.1038/nn1923. [DOI] [PubMed] [Google Scholar]
  12. Di Ciano P, Everitt BJ. Neuropsychopharmacology of drug seeking: Insights from studies with second-order schedules of drug reinforcement. Eur J Pharmacol. 2005;526:186–198. doi: 10.1016/j.ejphar.2005.09.024. [DOI] [PubMed] [Google Scholar]
  13. DiRocco DP, Scheiner ZS, Sindreu CB, Chan GC, Storm DR. A role for calmodulin-stimulated adenylyl cyclases in cocaine sensitization. J Neurosci. 2009;29:2393–2403. doi: 10.1523/JNEUROSCI.4356-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Domjan M. The principles of learning and behavior. 6th ed. Florence, KY: Wadsworth; 2006. [Google Scholar]
  15. Ettenberg A. Dopamine, neuroleptics and reinforced behavior. Neurosci Biobehav Rev. 1989;13:105–111. doi: 10.1016/s0149-7634(89)80018-1. [DOI] [PubMed] [Google Scholar]
  16. Eyny YS, Horvitz JC. Opposing roles of D1 and D2 receptors in appetitive conditioning. J Neurosci. 2003;23:1584–1587. doi: 10.1523/JNEUROSCI.23-05-01584.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Fanselow MS. Contextual fear, gestalt memories, and the hippocampus. Behav Brain Res. 2000;110:73–81. doi: 10.1016/s0166-4328(99)00186-2. [DOI] [PubMed] [Google Scholar]
  18. Ferguson GD, Storm DR. Why calcium-stimulated adenylyl cyclases? Physiology. 2004;19:271–276. doi: 10.1152/physiol.00010.2004. [DOI] [PubMed] [Google Scholar]
  19. French SJ, Totterdell S. Hippocampal and prefrontal cortical inputs monosynaptically converge with individual projection neurons of the nucleus accumbens. J Comp Neurol. 2002;446:151–165. doi: 10.1002/cne.10191. [DOI] [PubMed] [Google Scholar]
  20. French SJ, Totterdell S. Individual nucleus accumbens-projection neurons receive both basolateral amygdala and ventral subicular afferents in rats. Neuroscience. 2003;119:19–31. doi: 10.1016/s0306-4522(03)00150-7. [DOI] [PubMed] [Google Scholar]
  21. Gerdjikov TV, Giles AC, Swain SN, Beninger RJ. Nucleus accumbens PKA inhibition blocks acquisition but enhances expression of amphetamine-produced conditioned activity in rats. Psychopharmacology. 2007;190:65–72. doi: 10.1007/s00213-006-0590-1. [DOI] [PubMed] [Google Scholar]
  22. Gonon F. Prolonged and extrasynaptic excitatory action of dopamine mediated by D1 receptors in the rat striatum in vivo. J Neurosci. 1997;17:5972–5978. doi: 10.1523/JNEUROSCI.17-15-05972.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Goto Y, Grace AA. Limbic and cortical information processing in the nucleus accumbens. Trends Neurosci. 2008;31:552–558. doi: 10.1016/j.tins.2008.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Grace AA, Bunney BS. The control of firing pattern in nigral dopamine neurons: Burst firing. J Neurosci. 1984a;4:2877–2890. doi: 10.1523/JNEUROSCI.04-11-02877.1984. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Grace AA, Bunney BS. The control of firing pattern in nigral dopamine neurons: Single spike firing. J Neurosci. 1984b;4:2866–2876. doi: 10.1523/JNEUROSCI.04-11-02866.1984. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Groenewegen HJ, Wright CI, Beijer AV, Voorn P. Convergence and segregation of ventral striatal inputs and outputs. Ann N Y Acad Sci. 1999;877:49–63. doi: 10.1111/j.1749-6632.1999.tb09260.x. [DOI] [PubMed] [Google Scholar]
  27. Hall G. Associative structures in Pavlovian and instrumental conditioning. In: Pashler HE, editor. Steven's handbook of experimental psychology. 3rd ed. New York: Wiley; 2002. pp. 1–46. [Google Scholar]
  28. Hanoune J, Defer N. Regulation and role of adenylyl cyclase isoforms. Annu Rev Pharmacol Toxicol. 2001;41:145–174. doi: 10.1146/annurev.pharmtox.41.1.145. [DOI] [PubMed] [Google Scholar]
  29. Ito R, Everitt BJ, Robbins TW. The hippocampus and appetitive Pavlovian conditioning: Effects of excitotoxic hippocampal lesions on conditioned locomotor activity and autoshaping. Hippocampus. 2005;15:713–721. doi: 10.1002/hipo.20094. [DOI] [PubMed] [Google Scholar]
  30. Ito R, Robbins TW, McNaughton BL, Everitt BJ. Selective excitotoxic lesions of the hippocampus and basolateral amygdala have dissociable effects on appetitive cue and place conditioning based on path integration in a novel Y-maze procedure. Eur J Neurosci. 2006;23:3071–3080. doi: 10.1111/j.1460-9568.2006.04883.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Ito R, Robbins TW, Pennartz CM, Everitt BJ. Functional interaction between the hippocampus and nucleus accumbens shell is necessary for the acquisition of appetitive spatial context conditioning. J Neurosci. 2008;28:6950–6959. doi: 10.1523/JNEUROSCI.1615-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Iwami G, Kawabe J, Ebina T, Cannon PJ, Homcy CJ, Ishikawa Y. Regulation of adenylyl cyclase by protein kinase A. J Biol Chem. 1995;270:12481–12484. doi: 10.1074/jbc.270.21.12481. [DOI] [PubMed] [Google Scholar]
  33. Iwamoto T, Okumura S, Iwatsubo K, Kawabe J, Ohtsu K, Sakai I, Hashimoto Y, Izumitani A, Sango K, Ajiki K, et al. Motor dysfunction in type 5 adenylyl cyclase-null mice. J Biol Chem. 2003;278:16936–16940. doi: 10.1074/jbc.C300075200. [DOI] [PubMed] [Google Scholar]
  34. Jin LQ, Goswami S, Cai G, Zhen X, Friedman E. SKF83959 selectively regulates phosphatidylinositol-linked D1 dopamine receptors in rat brain. J Neurochem. 2003;85:378–386. doi: 10.1046/j.1471-4159.2003.01698.x. [DOI] [PubMed] [Google Scholar]
  35. Kheirbek MA, Beeler JA, Ishikawa Y, Zhuang X. A cAMP pathway underlying reward prediction in associative learning. J Neurosci. 2008;28:11401–11408. doi: 10.1523/JNEUROSCI.4115-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Kheirbek MA, Britt JP, Beeler JA, Ishikawa Y, McGehee DS, Zhuang X. Adenylyl cyclase type 5 contributes to corticostriatal plasticity and striatum-dependent learning. J Neurosci. 2009;29:12115–12124. doi: 10.1523/JNEUROSCI.3343-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. LeDoux JE. Emotion circuits in the brain. Annu Rev Neurosci. 2000;23:155–184. doi: 10.1146/annurev.neuro.23.1.155. [DOI] [PubMed] [Google Scholar]
  38. Mons N, Decorte L, Jaffard R, Cooper DM. Ca2+-sensitive adenylyl cyclases, key integrators of cellular signalling. Life Sci. 1998;62:1647–1652. doi: 10.1016/s0024-3205(98)00122-2. [DOI] [PubMed] [Google Scholar]
  39. Overton PG, Clark D. Burst firing in midbrain dopaminergic neurons. Brain Res Brain Res Rev. 1997;25:312–334. doi: 10.1016/s0165-0173(97)00039-8. [DOI] [PubMed] [Google Scholar]
  40. Parkinson JA, Olmstead MC, Burns LH, Robbins TW, Everitt BJ. Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive Pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by D-amphetamine. J Neurosci. 1999;19:2401–2411. doi: 10.1523/JNEUROSCI.19-06-02401.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Phillips RG, LeDoux JE. Differential contribution of amygdala and hippocampus to cued and contextual fear conditioning. Behav Neurosci. 1992;106:274–285. doi: 10.1037//0735-7044.106.2.274. [DOI] [PubMed] [Google Scholar]
  42. Pittenger C, Fasano S, Mazzocchi-Jones D, Dunnett SB, Kandel ER, Brambilla R. Impaired bidirectional synaptic plasticity and procedural memory formation in striatum-specific cAMP response element-binding protein-deficient mice. J Neurosci. 2006;26:2808–2813. doi: 10.1523/JNEUROSCI.5406-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Reynolds JN, Wickens JR. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 2002;15:507–521. doi: 10.1016/s0893-6080(02)00045-x. [DOI] [PubMed] [Google Scholar]
  44. Reynolds JN, Hyland BI, Wickens JR. A cellular mechanism of reward-related learning. Nature. 2001;413:67–70. doi: 10.1038/35092560. [DOI] [PubMed] [Google Scholar]
  45. Richfield EK, Penney JB, Young AB. Anatomical and affinity state comparisons between dopamine D1 and D2 receptors in the rat central nervous system. Neuroscience. 1989;30:767–777. doi: 10.1016/0306-4522(89)90168-1. [DOI] [PubMed] [Google Scholar]
  46. Rudy JW, Sutherland RJ. Configural association theory and the hippocampal formation: an appraisal and reconfiguration. Hippocampus. 1995;5:375–389. doi: 10.1002/hipo.450050502. [DOI] [PubMed] [Google Scholar]
  47. Sanders AC, Hussain AJ, Hen R, Zhuang X. Chronic blockade or constitutive deletion of the serotonin transporter reduces operant responding for food reward. Neuropsychopharmacology. 2007;32:2321–2329. doi: 10.1038/sj.npp.1301368. [DOI] [PubMed] [Google Scholar]
  48. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol. 1998a;80:1–27. doi: 10.1152/jn.1998.80.1.1. [DOI] [PubMed] [Google Scholar]
  49. Schultz W. The phasic reward signal of primate dopamine neurons. Adv Pharmacol. 1998b;42:686–690. doi: 10.1016/s1054-3589(08)60841-8. [DOI] [PubMed] [Google Scholar]
  50. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
  51. Smith DM, Mizumori SJ. Hippocampal place cells, context, and episodic memory. Hippocampus. 2006;16:716–729. doi: 10.1002/hipo.20208. [DOI] [PubMed] [Google Scholar]
  52. Wyvell CL, Berridge KC. Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: Enhancement of reward “wanting” without enhanced “liking” or response reinforcement. J Neurosci. 2000;20:8122–8130. doi: 10.1523/JNEUROSCI.20-21-08122.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Yin HH, Zhuang X, Balleine BW. Instrumental learning in hyperdopaminergic mice. Neurobiol Learn Mem. 2006;85:283–288. doi: 10.1016/j.nlm.2005.12.001. [DOI] [PubMed] [Google Scholar]
  54. Yu PY, Eisner GM, Yamaguchi I, Mouradian MM, Felder RA, Jose PA. Dopamine D1A receptor regulation of phospholipase C isoform. J Biol Chem. 1996;271:19503–19508. doi: 10.1074/jbc.271.32.19503. [DOI] [PubMed] [Google Scholar]

Articles from Learning & Memory are provided here courtesy of Cold Spring Harbor Laboratory Press

RESOURCES