Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Nov 1.
Published in final edited form as: Neurobiol Learn Mem. 2020 Aug 25;175:107299. doi: 10.1016/j.nlm.2020.107299

CaMKII antagonism in the ventral tegmental area impairs acquisition of conditioned approach learning in rats

Rudolf Nisanov a, Eva Schelbaum b, Debra Morris b, Robert Ranaldi a,b,*
PMCID: PMC7655631  NIHMSID: NIHMS1626734  PMID: 32853813

Abstract

This study investigated the role of calcium2+/calmodulin-dependent protein kinase II (CaMKII), a protein in the second messenger pathway of NMDA receptors, in the ventral tegmental area (VTA) in the acquisition and performance of conditioned approach learning. Male Long-Evans rats (N = 79) were exposed to 3 (to test acquisition) or 7 (to test performance) conditioning sessions in which they received 30 paired presentations of a light stimulus (CS) and a food pellet (US) on a random time schedule. These conditioning sessions were then followed by one 30-min session without the CS or US and lastly by a CS-only test session, where only the light stimulus was presented (without food) according to the same schedule as the conditioning sessions. Bilateral intra-VTA injections of KN93 (vehicle, 3.0, 4.5 or 6.0 μg/0.5 μL), a CaMKII inhibitor, were administered prior to each conditioning session to test effects on the acquisition of conditioned approach or prior to the CS-only test session to test effects on the performance of conditioned approach. KN93, when given prior to conditioning sessions, significantly reduced the number of conditioned approach responses emitted during CS presentations in the CS-only test. When KN93 was given prior to the CS-only test it had no effect. These results suggest that CaMKII activation in the VTA is necessary for the acquisition, but not the performance, of reward-related learning.

Keywords: Reward-related learning, Conditioned approach, Ventral tegmental area, CaMKII inhibitor, CaMKII

1. Introduction

Reward-related learning entails learned associations between primary rewards (unconditioned stimuli; USs), such as food, water and drugs, and reward-related stimuli. Stimuli associated with primary rewards acquire the ability to elicit behavioral responses similar to the primary rewards themselves, including the ability to elicit conditioned approach responses. Such stimuli are referred to as conditioned stimuli (CSs) and the mechanisms whereby stimuli become CSs are not fully known and constitute the focus of the present paper. We propose that the ability of CSs to elicit conditioned approach involves the acquisition by these reward-associated stimuli (i.e., CSs) of the ability to activate the same incentive motivational neural systems activated by primary rewards.

Mesocorticolimbic dopamine (DA) neurons have been implicated in reward-related learning, specifically in mediating the behavioral effects of primary reward, conditioned reward and conditioned approach (Fiorino, Coury, Fibiger, & Phillips, 1993; Ranaldi, 2014; Robinson & Berridge, 1993; R. A. Wise, 2004; Zellner & Ranaldi, 2010). The presentation of primary rewards has been shown to activate VTA DA neurons and trigger DA release in terminal regions of the mesocorticolimbic DA system (Day, Roitman, Wightman, & Carelli, 2007; Hernandez & Hoebel, 1988; Pfaus, Damsma, Wenkstern, & Fibiger, 1995; Yoshida et al., 1992). Antagonism of DA neurotransmission in terminal regions results in significant reductions in the rewarding effects of USs such as food, brain stimulation and drugs of abuse (R. A. Wise, 2004; Zellner & Ranaldi, 2010). Similarly, the presentation of CSs has also been shown to activate and modulate VTA DA neurons (Kiyatkin & Rebec, 2001; Kosobud, Harris, & Chapin, 1994; Miller, Sanghera, & German, 1981; Wilson & Bowman, 2004), as well as increase DA release in nucleus accumbens (NAcc) and caudate nucleus in rats (Phillips, Atkinson, Blackburn, & Blaha, 1993). These findings indicate that CSs gain the capacity to activate VTA DA neurons, but the mechanisms involved in this process remain unclear.

We have proposed a model stipulating that concurrent activation of the CS- and US-relevant receptors on DA neurons in the VTA is necessary for CSs to gain the capacity to activate these DA neurons (Zellner & Ranaldi, 2010). We propose that the US input is comprised of acetylcholine (ACh) stimulation of muscarinic receptors (mACh) on DA neurons (Sharf and Ranaldi 2006; Sharf et al. 2006; Zellner et al. 2009; Ranaldi et al. 2011; Kest et al. 2012; Nisanov et al. 2016; Galaj et al. 2017). The major sources of ACh to the VTA are the pedunculopontine tegmentum (PPTg) and laterodorsal tegmentum (LDTg) hindbrain nuclei (Garzón, Vaughan, Uhl, Kuhar, & Pickel, 1999; Holmstrand & Sesack, 2011; Oakman, Faris, Kerr, Cozzari, & Hartman, 1995; Omelchenko & Sesack, 2006). Extracellular concentrations of ACh in the VTA increase during eating, drinking and self-stimulation (Rada, Mark, Yeomans, & Hoebel, 2000). Stimulation of mACh receptors in the VTA enhances brain stimulation reward (BSR) while antagonism reduces BSR (Kofman & Yeomans, 1988; Yeomans, Mathur & Tampakeras, 1993; Yeomans, Kofman, & McFarlane, 1985) as well as eating and approach to food (Ikemoto & Panksepp, 1996; Rada et al., 2000). These findings demonstrate that VTA ACh plays a role in primary reward and signaling of the US. We also propose that the CS signal is comprised of glutamate released from cortical and/or tegmental afferents to the VTA that stimulates N-methyl-D-aspartate (NMDA) receptors on VTA DA neurons (Galaj & Ranaldi, 2018; Galaj, Seepersad, Dakmak, & Ranaldi, 2018; Hachimine, Seepersad, Babic, & Ranaldi, 2016; Kest et al., 2012; Ranaldi et al., 2011; Zellner et al., 2009). Tegmental neurons respond to independent visual and auditory stimuli (Pan & Hyland, 2005) as well as those associated with rewards (Okada, Toyama, Inoue, Isa, & Kobayashi, 2009; Pan & Hyland, 2005) and control the activity of VTA DA cells (Lodge & Grace, 2006) and DA release in the NAc (Forster & Blaha, 2000; Steidl, O’Sullivan, Pilat, Bubula, Brown, & Vezina, 2017), suggesting that encounters with environmental stimuli, such as those associated with primary rewards, cause glutamate signaling in the VTA that affects DA neuronal activity. In our model, the concurrent stimulation of NMDA and mACh receptors results in NMDA-mediated intracellular events that lead to the neural/synaptic changes whereby an originally weak (neutral reward-associated stimulus) glutamate activation of DA neurons (reward substrate) is transformed into a stronger (now CS) glutamate activation of DA neurons (reward substrate).

We have previously shown that non-novel, neutral stimuli do not have the capacity to significantly activate VTA DA neurons whereas food (a US) and food associated stimuli (CSs) do (Kest et al., 2012). We have also shown that the number of VTA DA neurons activated is significantly correlated with the capacity of a CS to elicit conditioned approach responses (Galaj & Ranaldi, 2018). Further, we have shown that the antagonism of mACh receptors in the VTA (Sharf and Ranaldi 2006; Sharf et al. 2006) or systemically (Nisanov et al. 2016) impairs the acquisition of reward-related learning. Likewise, antagonism of NMDA receptors in the VTA impairs the acquisition of reward-related learning (Kest et al., 2012; Ranaldi et al., 2011; Zellner, Kest, & Ranaldi, 2009). Given the crucial role of VTA NMDA receptor stimulation for the acquisition of this type of learning, the present study further investigated the intracellular processes that are associated with NMDA-mediated plasticity – namely, the role of calcium2+/calmodulin protein kinase II (CaMKII) – in reward-related learning.

CaMKII is a protein kinase in the second messenger pathway of NMDA receptors. The activation of CaMKII by calcium ions results in synaptic insertion and increased single-channel conductance of AMPA (a-amino-3-hydroxy-5-methyl-4-isoxazole propionic acid) receptors (Derkach, Barria, & Soderling, 1999; Hayashi et al., 2000; Rongo & Kaplan, 1999) among other long-lasting neural changes (Bayer, De Koninck, Leonard, Hell, & Schulman, 2001; Lisman, Malenka, Nicoll, & Malinow, 1997) believed to underlie the synaptic plasticity implicated in learning and memory. Based on our model, we predict that, like NMDA receptor antagonism, inhibition of CaMKII in the VTA will impair the acquisition of conditioned approach learning but have no effect on the expression or performance of this behavior after it is learned. We tested this hypothesis.

2. Materials and Methods

2.1. Animals

The protocols used in the present experiments were in accordance with the National Institutes of Health Guide for Care and Use of Laboratory Animals and were approved by the Queens College Institutional Animal Care and Use Committee.

Subjects consisted of 79 male Long Evans rats that were facility-bred from males and females obtained from Charles River Laboratories (Raleigh, NC). All rats were housed individually post-surgery and maintained on a 12-hour light:12-hour dark cycle (lights off at 9 AM) in a temperature-controlled (70°F) vivarium. Experimental sessions took place during the animals’ dark phase, in order to test them during their active cycle. The animals received an unlimited amount of food (LabDiet chow) and water, until a week prior to the start of the experimental sessions. Surgery was performed when they weighed approximately 375–400 g (i.e., at about three months of age). After three to four days of recovery, the animals were placed on a food restriction regimen that reduced their weights to 85% of their free feeding values. Once they reached their target weight, rats were given daily rations of food to maintain their target weights throughout the rest of the experiment.

2.2. Drugs

KN93 phosphate, a water-soluble calcium2+-calmodulin protein kinase II inhibitor (Tocris: R&D Systems, Minneapolis, MN), was dissolved in distilled water to produce doses of 3.0, 4.5 and 6.0 μg/0.5 μL and delivered bilaterally into the VTA. Drug doses were chosen based on the literature indicating behavioral effects after intracranial injections (Dunn, Inderwies, Licata, & Pierce, 2005; Licata, Schmidt, & Pierce, 2004; Faccidomo, Reid, Agoglia, Ademola, & Hodge, 2016).

2.3. Apparatus

Testing took place in eight individual conditioning chambers, each measuring 30 cm x 21 cm x 18 cm (L x W x H). Each chamber was enclosed in a sound-attenuating outer box equipped with a ventilating fan. The conditioning chambers were equipped with a food trough (5 cm x 5 cm) in which food pellets (45 mg each; Bio-Serv, Inc., Frenchtown, NJ) were dropped by a dispenser located outside the chamber. A white stimulus light was mounted 1 inch above and 2 inches to the left of the food trough. A photo-sensor in the food trough detected head entries. The floor of each chamber consisted of aluminum rods. All events in the conditioning chambers were programmed with MED-PC IV software.

2.4. Surgery

All rats received an intraperitoneal (IP) injection of atropine sulfate (0.54 μg/0.1 ml of distilled water). After approximately 8 minutes, rats were deeply anesthetized with an IP injection of sodium pentobarbital (65 mg/kg). An incision was made on the scalp to expose the skull and two holes were drilled into the skull to allow for cannula implantation. Stainless steel guide cannulas (22 gauge/15 mm long) were bilaterally lowered into the brain using the following coordinates from the Paxinos and Watson stereotaxic atlas (Paxinos & Watson, 1982): −5.6 mm caudal to bregma, ±2.0 mm from the midline at a 10° angle toward the midline and −8.6 mm below the surface of the skull. The dorsal to ventral coordinates were adjusted to −7.1 mm for the anatomical control groups. The cannulas were then permanently fixed to the skull using dental acrylic that formed around four stainless steel screws, placed into four proportional quadrants on the skull. Obturators that measured 16 mm precisely, were inserted into the guide cannulas and extended 1 mm beyond the cannula. Obturators remained inserted there at all times throughout the protocol with the exception of their removal during microinjections.

2.5. Experimental Procedures

2.5.1. Acquisition of conditioned approach learning

Rats received 8–10 BioServ food pellets in their home cages prior to their magazine training session. Rats began with a 20-min magazine training session, in which 20 food pellets were delivered on a random time schedule in their conditioning chamber food troughs. This allowed the rats to become familiar with the delivery of food pellets. Beginning the next day, rats received three consecutive 60-min conditioning sessions with a day off between sessions. During each conditioning session rats were presented with thirty presentations of the light stimulus (CS) under a random time schedule with an average interstimulus interval (ISI) of 150 s. Each light presentation was immediately followed by the delivery of a food pellet (US). Before each conditioning session, the obturators were removed, and each rat received bilateral microinjections of KN93 (3.0, 4.5 or 6.0 μg/0.5 μl) or vehicle (distilled water). The microinjectors measured 16 mm long, extending 1 mm past the guide cannulas. Injections were delivered using a 10 μl Hamilton gas-tight syringe. For each dose of KN93 or vehicle, injections were administered over the course of 60 seconds and then kept in place for an additional 30 seconds, to allow the drug to diffuse. After the microinjections were completed, the obturators were re-inserted into the cannulas and the rats were immediately placed into their conditioning chambers. Two days after their last conditioning session, rats returned to the conditioning chambers for a 30-min session with no light or food presentations. The next day, rats were exposed to a CS-only test session, in which they received 30 presentations of the light stimulus only (no food) under the same schedule as during conditioning sessions.

2.5.2. Performance/Expression of Conditioned Approach Learning

The performance/expression of conditioned approach procedure was similar to that of the acquisition experiment. In this experiment rats were exposed to seven consecutive conditioning sessions, and intra-VTA microinjections of KN93 (3.0, 4.5 or 6 μg) or vehicle were not administered prior to the conditioning sessions but were administered prior to the CS-only test session.

2.6. Histology

After the CS-only test, all rats were perfused for brain extraction and cannula placement verification. Rats were anesthetized using an overdose of sodium pentobarbital. Once the rats were deeply anesthetized, they were perfused using 120 ml of 0.9% saline solution followed immediately by 60 ml of 4.0% formalin solution and decapitated for brain extraction. Each brain was stored in 4.0% formalin in a refrigerator for 2–3 days, then placed in a 30% sucrose solution for 3–4 days. Using a cryostat, brains were sliced in coronal sections, which were placed on specimen slides and later inspected under a MicroProjector for cannula placement.

2.7. Statistical and Data Analysis

For each conditioning session and the CS-only test session, the data consisted of the number of head entries made during the following 3 periods; 1) Pre-CS, a 6-s period starting 6 s prior to the onset of the CS, 2) CS, a 6-s period starting with the onset of the CS and 3) Non-CS, all other times outside of the Pre-CS and CS periods.

For each session, the difference score was calculated by subtracting the number of head entries made during the Pre-CS periods from the number made during the CS periods (CS-Pre-CS entries). The difference score is used to measure the degree of conditioned approach learning. A one-way analysis of variance (ANOVA) was used to analyze the difference scores during the CS-only test session. A separate mixed ANOVA, with session as a repeated measures factor and KN93 dose as a between-groups factor was conducted on the difference scores during the conditioning phase. Statistically significant results were further analyzed using Tukey and Dunnett’s post-hoc tests. Mauchly’s test was carried out to determine violation of the sphericity assumption. Greenhouse-Geisser corrections for sphericity are reported where appropriate.

3. Results

3.1. Cannula verification

The data included in our analyses were restricted to that collected from rats with confirmed cannula placements and injection sites in the VTA or just dorsal to the VTA (dorsal control). The VTA microinjection sites spanned much of the rostral-caudal length of the VTA but mostly were localized in the caudal portion of the VTA (−5.8 to −6.04 mm posterior to bregma) with some injections occurring in the central portion (−5.2 to −5.6 mm posterior to bregma) (see Fig. 1).

Figure 1.

Figure 1.

Open in a new tab

Histological reconstruction of injection sites adapted from Paxinos and Watson for (A) acquisition experiment and (B) expression experiment. Black circles represent intra-VTA injections. Gray circles represent anatomical dorsal control injections. The numbers to the right of each section indicate the distance (mm) posterior to bregma.

3.2. Experiment 1: The effects of intra-VTA injections of KN93 on the acquisition of conditioned approach learning

Figure 2 illustrates the difference scores (CS minus Pre-CS head entries) of rats grouped by dose of KN93, across the three conditioning and CS-only test sessions. There were similar increases in the difference scores for all groups across the three conditioning sessions. A mixed ANOVA revealed a significant main effect of session [F2,56 = 51.33, p < .001], but no significant dose effect nor session x dose interaction (p >.05).

Figure 2.

Figure 2.

Open in a new tab

Mean (± SEM) CS minus Pre-CS difference scores for groups receiving VTA microinjections of KN93 [vehicle (n=8), 3 (n=8), 4.5 (n=7) or 6 μg/kg (n=9)] prior to each conditioning session. * denotes statistically significant difference from the vehicle group, p < 0.05.

In contrast to conditioning, difference scores from the CS-only test session were different across groups, showing dose-related decreases with the 6 μg group demonstrating the lowest scores. A one-way ANOVA on the data from the CS-only test session confirmed these observations, revealing a significant dose effect [F3,28 = 4.17, p < 0.05]. Tukey’s post hoc analyses revealed that the 6 μg group was significantly different from the vehicle group, p < .01.

Figure 3 shows mean total number of head entries across the three conditioning sessions and the CS-only test session. The total head entries decreased for all groups across sessions. The 4.5 μg group appeared to have a higher number of head entries than the other dose groups during conditioning, but not during the CS-only test session. A mixed-ANOVA on the total head entries made across conditioning sessions revealed a significant main effect of session, [F1.58, 56 = 13.19, p < .001] and a significant main effect of dose [F3, 28 = 4.29, p < .05] but no interaction (p >.05). Tukey’s post hoc comparisons showed that the total head entries for the 4.5 μg group was significantly higher than for the 3 μg and 6 μg groups (p < .05). During the CS-only test session, however, the total head entries were similar across dose groups. A separate one-way ANOVA did not reveal a significant dose effect (p >.05).

Figure 3.

Figure 3.

Open in a new tab

Mean (± SEM) total head entries for groups receiving VTA microinjections of KN93 [vehicle (n=8), 3 (n=8), 4.5 (n=7) or 6 μg/kg (n=9)] prior to each conditioning. * denotes statistically significant difference from the vehicle group, p < 0.05.

In Figure 4, the difference scores for the anatomical dorsal control groups (vehicle and 6 μg/0.5 μL) showed a similar pattern across the three conditioning sessions and the CS-only test session. Difference scores were lower during the first conditioning session, after which they plateaued for the remaining sessions. A mixed-ANOVA on the difference scores across the three conditioning sessions showed a significant main effect of session [F2, 20 = 6.72, p < .01] but no significant dose effect nor session x dose interaction (p >.05). During the CS-only test session, the group treated with 6 μg of KN93 prior to each conditioning session exhibited similar difference scores to that of the vehicle group. A one-way ANOVA confirmed that these groups were not significantly different (p >.05).

Figure 4.

Figure 4.

Open in a new tab

Mean (± SEM) CS minus Pre-CS difference scores for groups receiving 1mm dorsal to VTA microinjections of KN93 [vehicle (n=6) or 6 μg/kg (n=6)] prior to each conditioning session. * denotes statistically significant difference from the vehicle group, p < 0.05.

Figure 5 shows mean total number of head entries across the three conditioning sessions and the CS-only test session in the dorsal control groups. The total head entries decreased for both groups in a similar fashion across sessions. The 6 μg dose group appeared to have a slightly higher number of head entries than the control group across sessions but this was not significant. A mixed ANOVA on the total head entries made across conditioning sessions revealed a significant main effect of session, [F1.22, 20 = 11.31, p < .05], but no significant dose effect nor session x dose interaction (p >.05). During the CS-only test session the total head entries were similar for both groups, and this observation was confirmed by a separate one-way ANOVA that did not reveal a significant dose effect (p >.05).

Figure 5.

Figure 5.

Open in a new tab

Mean (± SEM) total head entries for groups receiving 1mm dorsal to VTA microinjections of KN93 [vehicle (n=6) or 6 μg/kg (n=6)] prior to each conditioning session. * denotes statistically significant difference from the vehicle group, p < 0.05.

3.3. The effect of intra-VTA injections of KN93 on the expression of conditioned approach learning

Figure 6 shows the difference scores across seven conditioning sessions and the CS-only test session, during which rats received intra-VTA injections of KN93 (3.0, 4.5 or 6.0 μg/0.5 μl) or vehicle prior to the CS-only test session and not prior to each conditioning session. Generally, all dose groups had similar increases in difference scores across the conditioning sessions. Although the 6 ug dose group also showed increases across sessions, the difference scores for this group tended to be lower than those for the other groups. A mixed-ANOVA on the difference scores across the seven conditioning sessions revealed a significant main effect of session [F3.095, 186 = 21.00, p < .001] but no session x dose interaction (p >.05). During the CS-only test session, prior to which all groups received their respective intracranial injections, the difference scores were similar across groups. A one-way ANOVA confirmed this observation and revealed no significant dose effect (p >.05).

Figure 6.

Figure 6.

Open in a new tab

Mean (± SEM) CS minus Pre-CS difference scores for groups receiving intra-VTA injections of KN93 [vehicle (n=9), 3 (n=8), 4.5 (n=9) or 6 μg/kg (n=9)] prior to the CS-only test session. * denotes statistically significant difference from the vehicle group, p < 0.05.

The mean total head entries made by groups in the expression experiment consisted of similar decreases across the seven conditioning sessions across groups and for many of these sessions the 6 μg group showed total head entries that tended to be higher than for the other groups (Figure 7). A mixed-ANOVA revealed a significant main effect of session [F3.43, 186 = 38.36, p < .001], but no significant dose effect nor session x dose interaction (p >.05). The total head entries during the CS-only test, prior to which all groups were treated, was similar among all groups. A one-way ANOVA on these data did not reveal a significant dose effect (p >.05).

Figure 7.

Figure 7.

Open in a new tab

Mean (± SEM) total head entries for groups receiving intra-VTA injections of KN93 [vehicle (n=9), 3 (n=8), 4.5 (n=9) or 6 μg/kg (n=9)] prior to the CS-only test session. * denotes statistically significant difference from the vehicle group, p < 0.05.

4. Discussion

In Experiment 1, inhibition of CaMKII occurred prior to each of the three conditioning sessions by intra-VTA microinjections of KN93, to assess the role of CaMKII in the acquisition of conditioned approach learning. The outcome was a dose-related, significant reduction in differences scores emitted during the CS-only test session, suggesting a reduction in conditioned approach learning. When similar KN93 microinjections were made just dorsal to the VTA they produced no effect on the acquisition of conditioned approach learning. The lack of effect with dorsal injections rules out the possibility of unintentional dispersed drug effects of KN93 in dorsal areas, where some drug diffusion may occur after intra-VTA injections (Wise & Hoffman, 1992), and argues strongly that the effect of intra-VTA injections were caused by the localized pharmacological action of KN93, specifically in the VTA, during learning. In Experiment 2, we found that when rats were not treated with KN93 prior to each of the conditioning sessions but rather just prior to the CS-only test, they showed similar performance of the learned conditioned approach response across groups. Finally, an analysis of total head entries, which can be seen as an indicator of non-specific treatment effects and general capacity to make the head entry response, did not reveal dose differences between any dose and vehicle in any of the experiments, eliminating the possibility that significant difference scores resulted from incapacitation. Altogether, these experiments show that inhibition of CaMKII in the VTA before conditioning, but not before the CS-only test, results in a significant impairment of conditioned approach learning.

The impairment in the acquisition of conditioned approach observed in Experiment 1 is best explained as a direct result of the localized pharmacological action of KN93, namely, the inhibition of CAMKII in the VTA. All animals consumed the same number of food pellets during conditioning sessions across all experiments, arguing against the possibility that the reduced conditioned approach occurred because of reduced food consumption or, perhaps, fewer stimulus-reward pairings. Furthermore, the results from both experiments indicate that there was no significant difference in the total numbers of head entries across groups and across sessions. Given that the total number of head entries during a session is a measure of noncontingent activity this result would argue against the possibility that treatment resulted in motoric incapacitation. The results of these experiments indicate that inhibition of CaMKII in the VTA impairs the acquisition of conditioned approach learning but not the performance (i.e., expression) of the learned response.

The results of the current study add further support for our neuronal model of reward-related learning (Zellner and Ranaldi 2010; Ranaldi 2014). This model stipulates that acquisition of conditioned approach depends on the capacity of the CS to stimulate DA neurons in the VTA, in the absence of the US. For the CS to gain the capacity to activate the reward substrate there must be concurrent stimulation of NMDA and mACh receptors on this reward substrate (proposed to be DA neurons) in the VTA. The mACh receptor stimulation causes depolarization of DA cells, removal of the magnesium block from the NMDA receptor and NMDA receptor stimulation triggers an influx of calcium ions into DA neurons. The rise in intracellular calcium ions activates several enzyme protein kinases including CaMKII, initiating the intracellular processes that lead to the strengthening of CS-related synapses and the CS signal itself. Thus, according to this model, inhibition of CaMKII should prevent the underlying neural plasticity from occurring, impairing reward-related learning. The present results support this model by providing evidence that CaMKII activity in the VTA indeed plays a critical role in the acquisition of reward-related learning.

Our finding that VTA CaMKII activity is involved in the acquisition of learning but not in the performance (expression) of an already learned response is in line with the current body of literature. While our study focused on conditioned approach learning, similar findings have been produced elsewhere in conditioned avoidance learning. The blockade of CaMKII activation in the amygdala resulted in dose-dependent impairment of the acquisition, but not expression, of auditory and contextual fear conditioning (Rodrigues, Farb, Bauer, LeDoux, & Schafe, 2004). Moreover, both the activation of NMDA receptors and the autophosphorylation of CaMKII with genetic disruptions were found to be necessary for the establishment of a CS, in Pavlovian CS-US fear conditioning (Frankland et al., 2004). Interestingly, CaMKII activity has also been found to be involved in expectations of future reward in computational model for reward-guided network plasticity and Hamiltonian synaptic sampling framework (Yu et al., 2018). While manipulations of CaMKII activity in the striatum had no effect on the acquisition of instrumental learning, it did reduce the motivational control of instrumental behavior elicited by Pavlovian CSs (Wiltgen, Law, Ostlund, Mayford, & Balleine, 2007).

CaMKII has been shown to be involved in reward-related learning based on other classes of primary rewards such as drugs of abuse. Inhibition of CaMKII activity in the VTA impaired the acquisition, but not the expression, of cocaine conditioned place preference and cocaine induced synaptic plasticity in the nucleus accumbens shell (Liu et al., 2014). The activity of CaMKII in the nucleus accumbens shell plays a role in amphetamine-seeking behavior (Loweth et al., 2010), cocaine-seeking behavior (Anderson et al., 2008) and morphine-seeking behavior (Liu, Zhang, Liu, & Yu, 2012).

5. Conclusion

In summary, the present study is the first demonstration that inhibition of CaMKII in the VTA attenuates acquisition of conditioned approach while having no effect on the performance of this behavior after it is learned. These results suggest that CaMKII activity in the VTA is critically involved in the acquisition by an initially neutral, reward-associated stimulus of CS properties. We propose that the acquisition of CS properties happens when the reward-associated stimulus acquires the ability to stimulate the primary reward neural substrates and that this process depends critically on stimulation of CaMKII in the VTA. These findings add to our understanding of the basic neural processes underlying reward-related learning and incentive motivation and suggest possible mechanisms that may be targeted to modulate reward-related pathologies such as substance use disorders.

Supplementary Material

1

Acknowledgments

This work was supported by National Institute of General Medical Science of the National Institutes of Health under award number 1SC3GM130430-01 to R.R. The authors have no conflict of interest in the manuscript.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

6. References

  1. Anderson SM, Famous KR, Sadri-Vakili G, Kumaresan V, Schmidt HD, Bass CE, … Pierce RC (2008). CaMKII: a biochemical bridge linking accumbens dopamine and glutamate systems in cocaine seeking. Nature Neuroscience, 11(3), 344–353. 10.1038/nn2054 [DOI] [PubMed] [Google Scholar]
  2. Bayer KU, De Koninck P, Leonard AS, Hell JW, & Schulman H (2001). Interaction with the NMDA receptor locks CaMKII in an active conformation. Nature, 411(6839), 801–805. 10.1038/35081080 [DOI] [PubMed] [Google Scholar]
  3. Day JJ, Roitman MF, Wightman RM, & Carelli RM (2007). Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nature Neuroscience, 10(8), 1020–1028. 10.1038/nn1923 [DOI] [PubMed] [Google Scholar]
  4. Derkach V, Barria A, & Soderling TR (1999). Ca2+/calmodulin-kinase II enhances channel conductance of α-amino-3-hydroxy-5-methyl-4-isoxazolepropionate type glutamate receptors. Proceedings of the National Academy of Sciences of the United States of America, 96(6), 3269–3274. 10.1073/pnas.96.6.3269 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Dunn JM, Inderwies BR, Licata SC, & Pierce RC (2005). Repeated administration of AMPA or a metabotropic glutamate receptor agonist into the rat ventral tegmental area augments the subsequent behavioral hyperactivity induced by cocaine. Psychopharmacology, 179(1), 172–180. 10.1007/s00213-004-2054-9 [DOI] [PubMed] [Google Scholar]
  6. Faccidomo S, Reid GT, Agoglia AE, Ademola SA, & Hodge CW (2016). CaMKII inhibition in the prefrontal cortex specifically increases the positive reinforcing effects of sweetened alcohol in C57BL/6J mice. Behavioural Brain Research, 298, 286–290. 10.1016/j.bbr.2015.11.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Fiorino DF, Coury A, Fibiger HC, & Phillips AG (1993). Electrical stimulation of reward sites in the ventral tegmental area increases dopamine transmission in the nucleus accumbens of the rat. Behavioural Brain Research, 55(2), 131–141. 10.1016/0166-4328(93)90109-4 [DOI] [PubMed] [Google Scholar]
  8. Forster GL, & Blaha CD (2000). Laterodorsal tegmental stimulation elicits dopamine efflux in the rat nucleus accumbens by activation of acetylcholine and glutamate receptors in the ventral tegmental area. European Journal of Neuroscience, 12(10), 3596–3604. 10.1046/j.1460-9568.2000.00250.x [DOI] [PubMed] [Google Scholar]
  9. Frankland PW, Josselyn SA, Anagnostaras SG, Kogan JH, Takahashi E, & Silva AJ (2004). Consolidation of CS and US representations in associative fear conditioning. Hippocampus, 14(5), 557–569. 10.1002/hipo.10208 [DOI] [PubMed] [Google Scholar]
  10. Galaj E, Nisanov R, & Ranaldi R (2017). Blockade of muscarinic acetylcholine receptors in the ventral tegmental area blocks the acquisition of reward-related learning. Behavioural Brain Research, 329, 20–25. 10.1016/j.bbr.2017.04.037 [DOI] [PubMed] [Google Scholar]
  11. Galaj E, & Ranaldi R (2018). The strength of reward-related learning depends on the degree of activation of ventral tegmental area dopamine neurons. Behavioural Brain Research, 348(April), 65–73. 10.1016/j.bbr.2018.04.012 [DOI] [PubMed] [Google Scholar]
  12. Galaj Ewa, Seepersad N, Dakmak Z, & Ranaldi R (2018). Blockade of NMDA receptors blocks the acquisition of cocaine conditioned approach in rats. European Journal of Pharmacology, 818, 480–485. 10.1016/j.ejphar.2017.11.029 [DOI] [PubMed] [Google Scholar]
  13. Garzón M, Vaughan RA, Uhl GR, Kuhar MJ, & Pickel VM (1999). Cholinergic axon terminals in the ventral tegmental area target a subpopulation of neurons expressing low levels of the dopamine transporter. Journal of Comparative Neurology, 410(2), 197–210. [DOI] [PubMed] [Google Scholar]
  14. Hachimine P, Seepersad N, Babic S, & Ranaldi R (2016). Concurrent antagonism of NMDA and AMPA receptors in the ventral tegmental area reduces the expression of conditioned approach learning in rats. Behavioural Brain Research, 298, 142–149. 10.1016/j.bbr.2015.10.054 [DOI] [PubMed] [Google Scholar]
  15. Hayashi Y, Shi SH, Esteban JA, Piccini A, Poncer JC, & Malinow R (2000). Driving AMPA receptors into synapses by LTP and CaMKII: Requirement for GluR1 and PDZ domain interaction. Science, 287(5461), 2262–2267. 10.1126/science.287.5461.2262 [DOI] [PubMed] [Google Scholar]
  16. Hernandez L, & Hoebel BG (1988). Food reward and cocaine increase extracellular dopamine in the nucleus accumbens as measured by microdialysis. Life Sciences, 42(18), 1705–1712. 10.1016/0024-3205(88)90036-7 [DOI] [PubMed] [Google Scholar]
  17. Holmstrand EC, & Sesack SR (2011). Projections from the rat pedunculopontine and laterodorsal tegmental nuclei to the anterior thalamus and ventral tegmental area arise from largely separate populations of neurons. Brain Structure and Function, 216(4), 331–345. 10.1007/s00429-011-0320-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Ikemoto S, & Panksepp J (1996). Dissociations between appetitive and consummatory responses by pharmacological manipulations of reward-relevant brain regions. Behavioral Neuroscience, 110(2), 331–345. 10.1037/0735-7044.110.2.331 [DOI] [PubMed] [Google Scholar]
  19. Kest K, Cruz I, Chen DH, Galaj E, & Ranaldi R (2012). A food-associated CS activates c-Fos in VTA DA neurons and elicits conditioned approach. Behavioural Brain Research, 235(2), 150–157. 10.1016/j.bbr.2012.07.044 [DOI] [PubMed] [Google Scholar]
  20. Kiyatkin EA, & Rebec GV (2001). Impulse activity of ventral tegmental area neurons during heroin self-administration in rats. Neuroscience, 102(3), 565–580. 10.1016/S0306-4522(00)00492-9 [DOI] [PubMed] [Google Scholar]
  21. Kofman O, & Yeomans JS (1988). Cholinergic antagonists in ventral tegmentum elevate thresholds for lateral hypothalamic and brainstem self-stimulation. Pharmacology, Biochemistry and Behavior, 31(3), 547–559. 10.1016/0091-3057(88)90229-8 [DOI] [PubMed] [Google Scholar]
  22. Kosobud AEK, Harris GC, & Chapin JK (1994). Behavioral associations of neuronal activity in the ventral tegmental area of the rat. Journal of Neuroscience, 14(11 II), 7117–7129. 10.1523/jneurosci.14-11-07117.1994 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Licata SC, Schmidt HD, & Pierce RC (2004). Suppressing calcium/calmodulin-dependent protein kinase II activity in the ventral tegmental area enhances the acute behavioural response to cocaine but attenuates the initiation of cocaine-induced behavioural sensitization in rats. European Journal of Neuroscience, 19(2), 405–414. 10.1111/j.0953-816X.2003.03110.x [DOI] [PubMed] [Google Scholar]
  24. Lisman J, Malenka RC, Nicoll RA, & Malinow R (1997). Learning mechanisms: The case for CaM-KII. Science, 276(5321), 2001–2002. 10.1126/science.276.5321.2001 [DOI] [PubMed] [Google Scholar]
  25. Liu X, Liu Y, Zhong P, Wilkinson B, Qi J, Olsen CM, … Liu QS (2014). CaMKII activity in the ventral tegmental area gates cocaine-induced synaptic plasticity in the nucleus accumbens. Neuropsychopharmacology, 39(4), 989–999. 10.1038/npp.2013.299 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Liu Z, Zhang JJ, Liu XD, & Yu LC (2012). Inhibition of CaMKII activity in the nucleus accumbens shell blocks the reinstatement of morphine-seeking behavior in rats. Neuroscience Letters, 518(2), 167–171. 10.1016/j.neulet.2012.05.003 [DOI] [PubMed] [Google Scholar]
  27. Lodge DJ, & Grace AA (2006). The hippocampus modulates dopamine neuron responsivity by regulating the intensity of phasic neuron activation. Neuropsychopharmacology : Official Publication of the American College of Neuropsychopharmacology, 31(7), 1356–1361. 10.1038/sj.npp.1300963 [DOI] [PubMed] [Google Scholar]
  28. Loweth JA, Singer BF, Baker LK, Wilke G, Inamine H, Bubula N, … Vezina P (2010). Transient overexpression of α-Ca2+/calmodulin-dependent protein kinase II in the nucleus accumbens shell enhances behavioral responding to amphetamine. Journal of Neuroscience, 30(3), 939–949. 10.1523/JNEUROSCI.4383-09.2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Miller JD, Sanghera MK, & German DC (1981). Mesencephalic dopaminergic unit activity in the behaviorally conditioned rat. Life Sciences, 29(12), 1255–1263. 10.1016/0024-3205(81)90231-9 [DOI] [PubMed] [Google Scholar]
  30. Nisanov R, Galaj E, & Ranaldi R (2016). Treatment with a muscarinic acetylcholine receptor antagonist impairs the acquisition of conditioned reward learning in rats. Neuroscience Letters, 614, 95–98. 10.1016/j.neulet.2015.12.064 [DOI] [PubMed] [Google Scholar]
  31. Oakman SA, Faris PL, Kerr PE, Cozzari C, & Hartman BK (1995). Distribution of pontomesencephalic cholinergic neurons projecting to substantia nigra differs significantly from those projecting to ventral tegmental area. Journal of Neuroscience, 15(9), 5859–5869. 10.1523/jneurosci.15-09-05859.1995 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Okada KI, Toyama K, Inoue Y, Isa T, & Kobayashi Y (2009). Different pedunculopontine tegmental neurons signal predicted and actual task rewards. Journal of Neuroscience, 29(15), 4858–4870. 10.1523/JNEUROSCI.4415-08.2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Omelchenko N, & Sesack SR (2006). Cholinergic axons in the rat ventral tegmental area synapse preferentially onto mesoaccumbens dopamine neurons. Journal of Comparative Neurology, 494(6), 863–875. 10.1002/cne.20852 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Pan WX, & Hyland BI (2005). Pedunculopontine tegmental nucleus controls conditioned responses of midbrain dopamine neurons in behaving rats. Journal of Neuroscience, 25(19), 4725–4732. 10.1523/JNEUROSCI.0277-05.2005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Paxinos G, & Watson C (1982). The Rat Brain in Stereotaxic Coordinates. The Rat Brain in Stereotaxic Coordinates, 52 10.1016/c2009-0-63235-9 [DOI] [PubMed] [Google Scholar]
  36. Pfaus JG, Damsma G, Wenkstern D, & Fibiger HC (1995). Sexual activity increases dopamine transmission in the nucleus accumbens and striatum of female rats. Brain Research, 693(1–2), 21–30. 10.1016/0006-8993(95)00679-K [DOI] [PubMed] [Google Scholar]
  37. Phillips AG, Atkinson LJ, Blackburn JR, & Blaha CD (1993). Increased extracellular dopamine in the nucleus accumbens of the rat elicited by a conditional stimulus for food: An electrochemical study. Canadian Journal of Physiology and Pharmacology, 71(5–6), 387–393. 10.1139/y93-059 [DOI] [PubMed] [Google Scholar]
  38. Rada PV, Mark GP, Yeomans JJ, & Hoebel BG (2000). Acetylcholine release in ventral tegmental area by hypothalamic self-stimulation, eating, and drinking. Pharmacology, Biochemistry, and Behavior, 65(3), 375–379. https://doi.org/S0091-3057(99)00218-X [pii] [DOI] [PubMed] [Google Scholar]
  39. Ranaldi R (2014). Dopamine and reward seeking: the role of ventral tegmental area. Reviews in the Neurosciences, 25(5), 1–10. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/24887956 [DOI] [PubMed] [Google Scholar]
  40. Ranaldi R, Kest K, Zellner MR, Lubelski D, Muller J, Cruz Y, & Saliba M (2011). The effects of VTA NMDA receptor antagonism on reward-related learning and associated c-fos expression in forebrain. Behavioural Brain Research, 216(1), 424–432. 10.1016/j.bbr.2010.08.026 [DOI] [PubMed] [Google Scholar]
  41. Robinson TE, & Berridge KC (1993). The neural basis of drug craving: An incentive-sensitization theory of addiction. Brain Research Reviews, 18(3), 247–291. 10.1016/0165-0173(93)90013-P [DOI] [PubMed] [Google Scholar]
  42. Rodrigues SM, Farb CR, Bauer EP, LeDoux JE, & Schafe GE (2004). Pavlovian Fear Conditioning Regulates Thr286 Autophosphorylation of Ca2+ /Calmodulin-Dependent Protein Kinase II at Lateral Amygdala Synapses. Journal of Neuroscience, 24(13), 3281–3288. 10.1523/JNEUROSCI.5303-03.2004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Rongo C, & Kaplan JM (1999). CaMKII regulates the density of central glutamatergic synapses in vivo. Nature, 402(6758), 195–199. [DOI] [PubMed] [Google Scholar]
  44. Sharf R, McKelvey J, & Ranaldi R (2006a). Blockade of muscarinic acetylcholine receptors in the ventral tegmental area prevents acquisition of food-rewarded operant responding in rats. Psychopharmacology, 186(1), 113–121. [DOI] [PubMed] [Google Scholar]
  45. Sharf R, McKelvey J, & Ranaldi R (2006b). Blockade of muscarinic acetylcholine receptors in the ventral tegmental area prevents acquisition of food-rewarded operant responding in rats. Psychopharmacology, 186(1), 113–121. 10.1007/s00213-006-0352-0 [DOI] [PubMed] [Google Scholar]
  46. Sharf R, & Ranaldi R (2006a). Blockade of muscarinic acetylcholine receptors in the ventral tegmental area disrupts food-related learning in rats. Psychopharmacology, 184(1), 87–94. [DOI] [PubMed] [Google Scholar]
  47. Sharf R, & Ranaldi R (2006b). Blockade of muscarinic acetylcholine receptors in the ventral tegmental area disrupts food-related learning in rats. Psychopharmacology, 184(1), 87–94. 10.1007/s00213-005-0235-9 [DOI] [PubMed] [Google Scholar]
  48. Steidl S, O’Sullivan S, Pilat D, Bulbula N, Brown J, Vezina P (2017). Operant responding for optogenetic excitation of LDTg inputs to the VTA requires D1 and D2 dopamine receptor activation in the NAcc Stephan. Behavioural Brain Research, 333, 161–170. 10.1016/j.bbr.2017.06.045. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Wilson DIG, & Bowman EM (2004). Nucleus accumbens neurons in the rat exhibit differential activity to conditioned reinforcers and primary reinforcers within a second-order schedule of saccharin reinforcement. European Journal of Neuroscience, 20(10), 2777–2788. 10.1111/j.1460-9568.2004.03747.x [DOI] [PubMed] [Google Scholar]
  50. Wiltgen BJ, Law M, Ostlund S, Mayford M, & Balleine BW (2007). The influence of Pavlovian cues on instrumental performance is mediated by CaMKII activity in the striatum. European Journal of Neuroscience, 25(8), 2491–2497. 10.1111/j.1460-9568.2007.05487.x [DOI] [PubMed] [Google Scholar]
  51. Wise RA (2004). Dopamine, learning and motivation. Nature Reviews. Neuroscience, 5(6), 483–494. 10.1038/nrn1406 [DOI] [PubMed] [Google Scholar]
  52. Wise RA, & Hoffman DC (1992). Localization of drug reward mechanisms by intracranial injections. Synapse, 10(3), 247–263. 10.1002/syn.890100307 [DOI] [PubMed] [Google Scholar]
  53. Yeomans JS, Mathur A, Tampakeras M (1993). Rewarding brain stimulation: role of tegmental cholinergic neurons that activate dopamine neurons. Behavioral Neuroscience, 107(6), 1077–1087. [DOI] [PubMed] [Google Scholar]
  54. Yeomans JS, Kofman O, & McFarlane V (1985). Cholinergic involvement in lateral hypothalamic rewarding brain stimulation. Brain Research, 329(1–2), 19–26. 10.1016/0006-8993(85)90508-6 [DOI] [PubMed] [Google Scholar]
  55. Yoshida M, Yokoo H, Mizoguchi K, Kawahara H, Tsuda A, Nishikawa T, & Tanaka M (1992). Eating and drinking cause increased dopamine release in the nucleus accumbens and ventral tegmental area in the rat: Measurement by in vivo microdialysis. Neuroscience Letters, 139(1), 73–76. 10.1016/0304-3940(92)90861-Z [DOI] [PubMed] [Google Scholar]
  56. Yu Z, Kappel D, Legenstein R, Song S, Chen F, & Maass W (2018). CaMKII activation supports reward-based neural network optimization through Hamiltonian sampling. Retrieved from http://arxiv.org/abs/1606.00157
  57. Zellner MR, Kest K, & Ranaldi R (2009a). NMDA receptor antagonism in the ventral tegmental area impairs acquisition of reward-related learning. Behavioural Brain Research, 197(2), 442–449. 10.1016/j.bbr.2008.10.013 [DOI] [PubMed] [Google Scholar]
  58. Zellner MR, Kest K, & Ranaldi R (2009b). NMDA receptor antagonism in the ventral tegmental area impairs acquisition of reward-related learning. Behavioural Brain Research, 197(2), 442–449. [DOI] [PubMed] [Google Scholar]
  59. Zellner MR, & Ranaldi R (2010a). How conditioned stimuli acquire the ability to activate VTA dopamine cells: A proposed neurobiological component of reward-related learning. Neuroscience and Biobehavioral Reviews, 34(5), 769–780. 10.1016/j.neubiorev.2009.11.011 [DOI] [PubMed] [Google Scholar]
  60. Zellner MR, & Ranaldi R (2010b). How conditioned stimuli acquire the ability to activate VTA dopamine cells: A proposed neurobiological component of reward-related learning. Neuroscience and Biobehavioral Reviews. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1626734-supplement-1.xml (307B, xml)

RESOURCES