Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jan 22.
Published in final edited form as: J Psychopharmacol. 2010 Nov 18;25(7):944–951. doi: 10.1177/0269881110388330

Increased expression of the 5-HT6 receptor by viral mediated gene transfer into posterior but not anterior dorsomedial striatum interferes with acquisition of a discrete action–outcome task

Daniel Eskenazi 1, John F Neumaier 2
PMCID: PMC4723119  NIHMSID: NIHMS752049  PMID: 21088040

Abstract

Serotonin plays a role in reinforcement learning; however, it is not known which serotonin receptors mediate these effects. Serotonin 6 (5-HT6) receptors are abundant in the striatum, a brain area that is involved in reinforcement learning. We previously found that 5-HT6 receptors in the dorsomedial striatum (DMS) affect reinforcement learning or consolidation over several days. We use viral-mediated gene transfer to discern the role that 5-HT6 receptors play in mediating post-synaptic responses in anterior versus posterior DMS. Male Long-Evans rats were used to study learning acquisition during a single session of 100 trials on a fixed interval of 20 seconds. In a discrete action–outcome learning task, rats had 10 seconds to press a lever to induce lever retraction and sucrose pellet delivery. In another group of rats, the task had a lever that was continuously extended but only active every 20 seconds, allowing for repetitive, mostly non-reinforced, lever pressing. Results demonstrate that increased expression of 5-HT6 receptors in the posterior DMS interferes with earning sucrose pellets in only the former task. We take this to indicate that 5-HT6 receptor signaling in the posterior DMS interferes with acquisition of discrete action–outcome responding.

Keywords: 5-HT(6) receptor, dorsomedial striatum, operant conditioning, serotonin, rat

Introduction

Study of the role of serotonin (5-hydroxytryptamine [5-HT]) in reward-oriented and reinforcement-learning behaviors is complicated by the diversity of serotonergic receptors (Millan et al., 2008). Broadly speaking, low serotonergic states (e.g. serotonin depletion induced by 5,7-dihydroxytryptamine serotonergic-specific lesions) are associated with enhanced operant responding (Fletcher et al., 1999) and high serotonergic states (e.g. induced with the serotonin releaser and reuptake inhibitor d-fenfluramine or with intra-cerebral injections of 5-HT) with diminished responding (Fletcher, 1995, 1996). One method of separating the roles of individual receptor subtypes is via receptor-specific pharmacological agents, but doing so substitutes information that may be encoded by the pattern of serotonin release dictated by the serotonergic neuron firing rate with a fairly constant level of receptor activity while the drug occupies the targeted receptor. Because computational modeling suggests that the serotonergic neuron firing rate has an influence on reinforcement learning behaviors (Daw et al., 2002) we employ viral-mediated gene transfer (VMGT) to enhance receptor expression of specific receptor subtypes in order to modulate neurotransmission, without disrupting information encoded by the temporal pattern of serotonin release. We focused on the serotonin-6 (5-HT6) receptor because it modulates reward and reinforcement learning, as we and others have shown (Ferguson et al., 2008; Frantz et al., 2002; Mitchell et al., 2007; Perez-García and Meneses, 2005).

The 5-HT6 receptor is densely distributed in the striatum, a brain region critical for operant learning. Previous work from our lab demonstrated that increased expression of 5-HT6 receptors in the dorsomedial striatum (DMS) interfered with the learning acquisition of a simple operant task (Mitchell et al., 2007). However, the multi-day experimental design used in that study was not leveraged to address whether encoding during the sessions (i.e. acquisition) or consolidation between days was affected. Acquisition and consolidation are dissociable mechanisms indicating different cellular and molecular mechanisms on different time scales (Abel and Lattal, 2001), as has been shown in corticostriatal systems (Laubach, 2005), and with regard to serotonin signaling (Sossin, 2008). We chose behavioral tasks that focus on the acquisition stage because our previous study strongly suggested, but did not definitely establish, that increased 5-HT6 receptor signaling in DMS induces a deficit in acquisition (Mitchell et al., 2007). In order to allow for a single session of reliable operant conditioning, the duration of the session was lengthened, the number of trials increased, and the inter-trial interval shortened.

In order to distinguish learning acquisition of a discrete action–outcome association from what may be characterized as the beginning of more habitual or compulsive responding, we employed two behavioral tasks that differ in how they allow rats to respond. In both tasks, performed on separate groups of animals, there is a single session of 100 trials of lever pressing for sucrose pellet reinforcement on a fixed interval, 20-second reinforcement schedule. In the inserting–retracting (IR) task, adapted from Mitchell et al. (2007), lever availability is indicated by its insertion and lever presses result in the lever’s retraction. Thus rats must attend to the lever and there are no non-reinforced lever presses. In contrast, during the continuously extended (CE) task, the lever remains extended throughout the entire session but there is no signal indicating whether a lever press will be reinforced with sucrose pellet delivery. Thus, many lever presses are not reinforced, allowing for a more repetitive manner of responding. At face value, the IR task is seemingly easier, given that lever insertion serves as a cue, and lever retraction prohibits the compulsive-like responding seen on the CE task. By comparing performance on these two tasks, using the appropriate controls, one can distinguish between acquisition of a discrete action–outcome process (as captured by the IR task) and the associative learning between lever and reward (as captured by the CE task).

Anatomically, the striatum integrates information received from many brain regions (including efferents from cortex, thalamus, amygdala, and hippocampus). Based on these striatal afferents, a dorsolateral-to-ventromedial gradient of functional organization has been proposed (Voorn et al., 2004). As reviewed elsewhere (Yin and Knowlton, 2006), the striatum can be divided based on anatomical and behavioral data into three main areas, each serving a different role in memory and learning: the limbic striatum, the associative striatum, and the sensorimotor striatum. The limbic striatum, made up of the nucleus accumbens and part of the olfactory tubercle, is involved in appetitive conditioning and Pavlovian classical conditioning, namely stimulus–outcome associations. The associative striatum, composed of the DMS in rodents and the caudate in primates, participates in action–outcome learning. The sensorimotor striatum, composed of the dorsolateral striatum in rodents or the putamen in primates; has its role in stimulus–response learning and habitual responding. Thus, while dorsal and ventral subdivisions of the striatum have been well studied in the rat, little is known about the anatomical and functional differences along an antero-posterior axis despite well-studied differences in humans and nonhuman primates, especially in the head versus tail regions of the caudate (Grahn et al., 2008).

The DMS contains abundant levels of 5-HT6 receptors (Roberts et al., 2002) on both types of medium spiny neurons (Ward and Dorsa, 1996) and at least one type of interneuron (Bonsi et al., 2007). In addition, increased expression of 5-HT6 receptors via VMGT in the DMS impairs performance on an operant task (Mitchell et al., 2007) suggesting a role for this receptor type in instrumental learning. VMGT of 5-HT6 receptors leads to an increase in the number of receptors on neuronal cell membranes thereby increasing the post-synaptic receptor response to serotonin released from dorsal raphe efferents (Mitchell et al., 2007).

In this study, we sought to explore first whether increased 5-HT6 receptor signaling affects learning acquisition within a single session, and second whether there are differences along an anterio-posterior axis within the DMS. Our findings indicate that increased expression of 5-HT6 receptors in posterior DMS interferes with acquisition of a discrete action–outcome task. This finding helps to better understand striatal learning mechanisms which are relevant to neuropsychiatric illnesses such as obsessive-compulsive disorder, Tourette’s syndrome, and drug addiction.

Methods and materials

Animal use and overview

Experimental procedures were approved by the University of Washington Institutional Animal Care and Use Committee and were conducted in accordance with National Institutes of Health guidelines.

Male Long-Evans rats (Charles River, 250–275 g at arrival, 121 rats in total were used) were habituated to the colony room (12-hour light/dark cycle, lights on at 06:00, temperature set to 21°C, ad libitum rat chow except as indicated) for 5 days before being handled. For the next 5 days, rats were handled and then fed once daily with 18 g of rat chow at 17:30 (this mildly restricted diet produced ~5% reduction in weight gain compared to ad libitum fed rats). Next, rats underwent stereotaxic surgery (at ~300 g, see detailed methods below) to infuse either the 5-HT6/eGFP or the eGFP-only viral vector (described below). Three days after viral infusion, rats were habituated to the operant chamber during a pre-test in which they received non-contingent sucrose pellets. The following day they underwent a single test session of either two tasks after which they were euthanized and transcardially perfused so that injection accuracy could be determined via fluorescent microscopy.

Viral vectors

Herpes simplex virus (HSV) viral vectors were used in this study, as previously described by our lab (Ferguson et al., 2008; Mitchell et al., 2007). This HSV viral vector system infects neurons and not glia and expresses transgene for about 7 days (Barot et al., 2007). The experimental viral vector contains two cassettes, one that expresses the hemagglutinin (HA) epitope-tagged 5-HT6 receptor driven by the HSV IE2/3 promoter and another that expresses enhanced green fluorescent protein (eGFP) driven by the cytomegalovirus (CMV) promoter, which serves as a way to locate virally infected neurons (Mitchell et al., 2007). Thus, all neurons that express eGFP using this vector also express the HA-tagged 5-HT6 receptor. In addition, a time course showing increased expression of 5-HT6 receptors after intracerebral infusion of this viral vector in rats demonstrated that peak expression of transgenic 5-HT6 receptors occurs at day 4 post-infusion (Ferguson et al., 2008). The control viral vector expresses eGFP but not 5-HT6 receptors. Non-specific effects of transgene expression were not found in previous studies, since eGFP controls learned tasks at the same rate as sham-operated controls, and the learning effects of this 5-HT6 receptor-expressing vector were reversed by a selective 5-HT6 receptor antagonist (Ferguson et al., 2008; Mitchell et al., 2007).

Surgical procedures

Rats were removed from food restriction the night before surgery and returned to food restriction 24 hours post-operatively. Rats were anesthetized with isoflurane gas (1–3%) throughout the stereotaxic surgery and given buprenorphine analgesia (0.1 mg/kg, s.c.) pre-operatively. Bilateral bore holes were drilled above the coordinates (either aDMS, coordinates from bregma: A/P + 1.2 mm, M/L ± 1.6 mm, D/V − 4.0 mm; or pDMS: A/P − 0.8 mm, M/L ± 3.2 mm, D/V − 3.4 mm). Using a 27-gauge long dental needle, 2 µL of viral particles (108 infective units/mL) per side were infused over 10 minutes at a rate of 200 nL/min, and the needle was left in place for an additional 5 minutes to minimize backflow before being slowly removed. This amount of viral vector was chosen based on previous studies to produce discrete infection at the target region (Mitchell et al., 2007; Neumaier et al., 2002).

Transcardial perfusions, tissue preparation and exclusion criteria

Rats were euthanized with an intraperitoneal injection of pentobarbital sodium and phenytoin sodium. The procedure proceeded once the rat was unresponsive to paw pinch and upon absence of corneal reflex. Perfusions were performed with 100 mL phosphate-buffered saline (PBS) solution followed by 200 mL of 4% paraformaldehyde, both at pH 7.4 and kept on ice. Brains were removed and left in 4% paraformaldehyde for 4 hours, then placed into PBS. Tissue sections were made on a Leica VT1000S vibrating blade microtome and mounted on slides and cover-slipped with Vectashield mounting medium (Vectorlabs, Burlingame, CA). Slides were visualized with a Nikon Eclipse E600 with HyQ FITC, HyQ TRITC, and DAPI epi-fluorescence filters.

Given that the nature of the study was to compare distinct anatomical regions, strict criteria were used to determine injection accuracy. Accuracy was determined for each rat in a blinded fashion using the following criteria: for aDMS, A/P + 1.2 mm, lateral to ventricle but not past a plane parallel to midline going through the apex of the corpus callosum (approximately M/L 2.0 mm), ventral to the corpus callosum but not more ventral than D/V − 6.0 mm from skull surface; for pDMS A/P − 0.8 mm, lateral to ventricle but not past approximately M/L 3.0 mm, ventral to corpus callosum but not more ventral than D/V − 5.0 mm from skull surface (Figure 1). If either viral vector entered the ventricle this was readily apparent (fluorescence in periventricular neurons away from the injection site) and the rat was excluded from the study. Surgical precision was lower in pDMS given the wider size and shape of the ventricle. (Overall accuracy is comparable to other studies for these targets given the strict criteria used and the requirement that injections in both hemispheres must be accurate, i.e. rats with unilateral hits were excluded from the study.) For the IR task, the following number of rats per total were excluded: aDMS eGFP 5/13; aDMS 5-HT6 4/12, pDMS eGFP 4/20, pDMS 5-HT6 8/21. For the CE task: aDMS eGFP 4/12, aDMS 5-HT6 4/11, pDMS eGFP 8/16, pDMS 5-HT6 8/16. This gives 45 excluded out of 121 total.

Figure 1.

Figure 1

Open in a new tab

Rat atlas figures showing injection targets as shaded areas in (A) anterior dorsomedial striatum (aDMS) and (B) posterior dorsomedial striatum (pDMS). (C) A typical injection site in the DMS shows expression of enhanced green fluorescent protein in neuron somas and processes. Scale bar = 200 µm.

Behavioral procedures

Behavioral testing was conducted in standard rat modular test chambers (Med Associates, Georgia, VT, USA) equipped with two levers and two lights on either side of a sucrose pellet receptacle on one wall and a house-light on the opposite wall. All chambers were kept in sound-attenuating boxes (Med Associates) equipped with fans providing temperature regulation and white noise. Individual rats underwent testing on only one of two behavioral tasks. Both tasks consisted of 100 trials on a fixed interval of 20 seconds with the house-light on throughout the session. Animals were placed in the operant box for 5 minutes before beginning the session.

Pre-test

Three days post-operatively, the day before the test, rats were habituated to the testing chamber for 30 minutes during which they were given non-contingent 45-mg sucrose pellets (Bioserv, Frenchtown, NJ, USA) in the pellet receptacle inside the chamber with the house-light on. Any pellets remaining were placed into the home cage of the rat at the end of the session. Sucrose pellets were used to allow comparisons with previous studies from our lab and others (Mitchell et al., 2007; Perez-Garcia and Meneses, 2005).

Inserting/retracting task (IR)

This paradigm was adapted from Mitchell et al. (2007), which was in turn based on previous learning studies investigating 5-HT6 receptors (Perez-Garcia and Meneses, 2005), modified to capture learning acquisition. Each trial consisted of the presentation of a single lever, which inserted into the chamber on a fixed interval of 20 seconds after which it remained extended for 10 seconds before retracting. A successful lever press resulted in delivery of one pellet and the retraction of the lever for 20 seconds. The session ended once 100 trials had occurred. House-light and light above lever were continuously on throughout the session.

Continuously extended task (CE)

The house-light was turned on and the two levers were extended and remained so during the entire session. One lever was never reinforced. Presses on the other lever were reinforced with one 45-mg sucrose pellet; but after a pellet was delivered there was a 20-second un-signaled timeout during which lever presses upon the reinforced lever did not result in pellet delivery. After the 20-second timeout period, the reinforced lever was then once again primed until pressed. A light remained on continuously above the reinforced lever, though there was no cue indicating the state of the lever (primed or on timeout) to prevent the possibility of incentive salience forming toward the cue itself. For this task, head entries into the pellet receptacle were recorded. The session ended once 100 pellets were obtained or 90 minutes lapsed, whichever occurred first.

Data analysis

Data from the behavioral sessions was collected using Med PC IV software. Sucrose pellet consumption during pre-test was analyzed using a one-way analysis of variance (ANOVA). Student’s t-test was used to compare head entries during the CE task and pellets earned on both the IR and CE tasks. Generalized estimating equation (GEE) analysis was used to analyze performance over time in groups with significantly different total sucrose pellets earned. GEE analysis was used to compare the rates of acquisition between the treatment groups.

Student’s t-test and one-way ANOVA were performed using GraphPad Prism version 5.00 for Windows, GraphPad Software, San Diego, CA, USA and the rate of acquisition learning was analyzed with GEE analysis with repeated measures on 5-minute bins of data using SPSS version 18, Chicago, IL, USA.

Results

Stereotaxic injection accuracy

Illustrations of the targeted regions for viral vector infusion are shown in Figure 1(A) and (B) and a representative micrograph of eGFP positive cells in the area of infusion is shown in Figure 1(C).

Sucrose consumption during pre-test and head entries into the pellet receptacle during the CE task

Sucrose consumption during the pre-test was not affected by increased expression of 5-HT6 receptors in aDMS or pDMS compared with eGFP-only controls (Figure 2(A)) F3,72 = 1.096, p = 0.3563. One advantage of the CE task is that groups of rats earned similar numbers of pellets, allowing us to examine the orientation toward the pellet receptacle during learning. Head entries into the pellet receptacle were not different between groups (Figure 2(B) and (C)), for aDMS p = 0.9522, for pDMS p = 0.7510.

Figure 2.

Figure 2

Open in a new tab

Increased expression of 5-HT6 receptors in the dorsomedial striatum (DMS) does not affect reward learning. There were no differences in rats’ sucrose consumption in the operant chamber during the pre-test in any of the four groups (group sizes included rats from both tasks). (A) aDMS: enhanced green fluorescent protein (eGFP) n = 16, 5-HT6 n = 24; pDMS: eGFP n = 15, 5-HT6 n = 21). Orientation to the pellet receptacle during the continuously extended task was unaffected by treatment. (B) aDMS: eGFP n = 8, 5-HT6 n = 7; (C) pDMS (both groups n = 8). All error bars represent SEM.

Performance on the IR task

In the IR task, increased expression of 5-HT6 receptors in pDMS (Figure 3(B)) decreased the total number of pellets earned; p = 0.0242. This was not the case in the aDMS p = 0.9767 (Figure 3(A)). Since there was an overall difference between groups after gene transfer into pDMS, we compared the rates of IR task acquisition between 5-HT6 versus eGFP treatment; animals with increased 5-HT6 receptor expression in pDMS failed to acquire the IR task at the rate of eGFP controls (Wald Chi Square = 6.92, p = 0.009).

Figure 3.

Figure 3

Open in a new tab

Increased expression of 5-HT6 receptors in the posterior dorsomedial striatum (pDMS) impairs the ability to acquire a discrete action–outcome task using the inserting/retracting (IR) task. (A) Performance on the IR task is not affected by increased expression of 5-HT6 receptors in the aDMS as compared to enhanced green fluorescent protein (eGFP) controls (n = 8 per group). (B) However, increased expression of 5-HT6 receptors in the pDMS significantly altered rats’ ability to earn sucrose pellets (eGFP n = 16, 5-HT6 n = 13). (C) The impaired performance of rats with increased expression of 5-HT6 receptors in the pDMS is seen over the course of the session. *p < 0.05; **p < 0.01. All error bars represent SEM.

Performance on the CE task

Increased expression of 5-HT6 receptors via viral-mediated gene transfer in neither aDMS nor pDMS affected the number of pellets earned (p = 0.78 and p = 0.53 respectively; Figure 4(A) and (C)), even though all rats did distinguish the active lever from the inactive lever (see below) (Figure 4(B) and (D)). This indicates that the motor skills necessary to perform lever-pressing tasks are not impaired by increased expression of 5-HT6 receptors in DMS. Rats in all groups were able to distinguish the active lever from the inactive lever. Two-way ANOVA with viral vector treatment and lever as factors for aDMS: interaction F1,26 = 0.009688, p = 0.9223, viral vector F1,26 = 0.08290, p = 0.7757, lever F1,26 = 18.12, p = 0.0002. For pDMS: interaction F1,28 = 0.3138, p = 0.5798, viral vector F1,28 = 0.2714, p = 0.6065, lever F1,28 = 8.299, p = 0.0075.

Figure 4.

Figure 4

Open in a new tab

Motor skills necessary to perform a lever-pressing task are not impaired by increased expression of 5-HT6 receptors in anterior or posterior dorsomedial striatum (aDMS/pDMS). The number of sucrose pellets earned is not affected by increased expression of 5-HT6 receptors in (A) aDMS (eGFP n = 8, 5-HT6 n = 7) or (C) pDMS (n = 8 per group). Rats in all groups were able to distinguish the active lever from the inactive lever, aDMS (B), pDMS (D). **p < 0.01; ***p < 0.001. All error bars represent SEM.

Discussion

In this study we found that increased expression of 5-HT6 receptors in the pDMS interferes with the acquisition of a discrete action–outcome task. This was apparent in the IR task, in which rats must attend to the availability of lever insertion and in which all lever presses are reinforced, but not when the lever was continuously extended and only some of the responses were reinforced thereby demonstrating that the ability to make discrete action–outcome associations was impaired by increased 5-HT6 receptor activity.

Increased expression of 5-HT6 receptors in aDMS or pDMS did not alter sucrose consumption during the pretest compared with controls. In addition, head entries made into the sucrose pellet receptacle during the CE task were not different between groups. Taken together, these data suggest that both orientation to the sucrose pellets and goal-directedness of the task were unaltered by the manipulation; therefore, increased dorsomedial striatal 5-HT6 receptor signaling did not affect reward learning per se. We did not detect any other general behavioral changes in 5-HT6-receptor/eGFP-treated rats versus eGFP-only controls. 5-HT6-expressing rats did not display yawning, stretching or chewing (as has been reported after administration of 5-HT6 receptor antagonists and 5-HT6 receptor antisense oligonucleotides; see Bourson et al., 1995; Sleight et al., 1998). Similarly, there was no difference in the growth curves between treatment groups (data not shown), whereas systemic administration of 5-HT6 ligands affected weight gain in some previous studies, but not others (Heal et al., 2008).

Given that the IR task requires the coordination of movement to execute the lever press, any impairment of motor output would confound the findings of this study. However, motor impairment seems unlikely, since the CE task, which had an equal or greater motor output requirement, was not affected by increased 5-HT6 receptor expression. In the CE task, rats could earn sucrose pellets from one lever on a fixed interval 20-second schedule of reinforcement while responses on the other lever were never reinforced. The CE task served as a control to determine rats’ motor ability to perform the IR task and their ability to make associations to distinguish between levers. However, because rats did not have to attend to the availability of the lever presentation and because they made numerous lever presses that were not reinforced, they were able to learn the task with mechanisms other than making a strict action–outcome association. Differences found in the performance of rats on these different tasks (cf. Figures 3(B) and 4(C)) demonstrates that each task requires a different set of cognitive skills. During the CE task many lever presses on the reinforced lever do not trigger sucrose pellet delivery (because they occur during the 20-second timeout period) whereas during the IR task each lever press is reinforced and causes the lever to retract, preventing habit-like responding. Given the repetitive lever pressing seen during the CE task, it may be that this task captures the beginnings of what may become habitual responding (Graybiel, 2008) and is thus more likely to be mediated by the sensorimotor dorsolateral striatum (Yin and Knowlton, 2006).

Consolidation of memory, including procedural memory, entails molecular events underlying longer term changes in the brain than occur in between sessions and are facilitated by sleep (Diekelmann et al., 2009) and thus are better captured in a multi-day study design. So even though a previous study from our lab demonstrated that 5-HT6 receptor signaling interfered with operant learning over several days (Mitchell et al., 2007), the short sessions and long inter-trial interval used made it unclear whether learning acquisition or memory consolidation was disrupted as a result of increased expression of 5-HT6 receptors. Increasing 5-HT6 receptor signaling after the task was already acquired and performed at high levels did not disrupt expression of that behavior, and 5-HT6 receptor antagonists given to rats with increased expression of 5-HT6 receptors before but not after the sessions improved task performance. Both of these experiments strongly suggested that acquisition rather than consolidation was affected by 5-HT6 receptor signaling. The current study clarified this matter by using single session tests of the IR and CE tasks.

In a previous study we examined the effects of increased 5-HT6 receptor expression in DMS on several learning and memory tasks. While the learning of the IR task was impaired by increased 5-HT6 receptor expression in DMS in an antagonist-sensitive manner, learning and memory in the Morris Water Maze task, previously learned lever-pressing, and extinction learning was unchanged, as was approach to and motivation to consume sucrose pellets in a familiar environment (Mitchell et al., 2007).

The dorsal striatum is a large, heterogeneous brain region. There are clear differences, anatomically and behaviorally, between the associative (DMS) and the sensorimotor striatum (dorsolateral striatum) (Voorn et al., 2004; Yin and Knowlton, 2006). Furthermore, within DMS, posterior regions have been demonstrated to be more critical for the acquisition of goal-directed behaviors (Yin et al., 2005). Increased 5-HT6 receptor signaling in posterior but not anterior DMS disrupts acquisition of a discrete operant task. One anatomical difference between the aDMS and pDMS is that the latter receives projections from the basolateral amygdala (Kelley et al., 1982), a brain region that, when activated, can decrease emotional and motivational processes toward conditioned stimuli (Everitt et al., 2003). Interestingly, systemically administered 5-HT6 receptors antagonists reverse deficits in behavioral assays of emotional learning (Mitchell and Neumaier, 2008) but immediate early gene activation localizes these effects to the amygdala itself (where 5-HT6 receptors are also found) not the pDMS. Therefore, it is unlikely that the action of lever insertion is somehow startling rats with increased 5-HT6 receptor expression in the pDMS and that this is interfering with acquisition of the simple operant task. Lesions of other brain regions, such as the dorsal hippocampus, also impair the ability of rats to demonstrate action–outcome learning (Corbit and Balleine, 2000), suggesting these brain regions are likely to be involved in these tasks. However, we chose to focus on the striatum, given the abundance of 5-HT6 receptor expression there, and for comparison with our previous results.

The current study identified the location where 5-HT6 receptors in striatum can strongly affect action–outcome learning; future studies will need to examine the cellular mechanism of the effect in more detail. However, a limitation of the current study is that we did not address the neurochemical mechanisms behind how increased expression of 5-HT6 receptors in pDMS may be impairing learning acquisition. A previous study found that 5-HT6 receptor agonists increase g-amino butyric acid levels in the striatum (Schechter et al., 2008), providing a clue to the mechanism by which 5-HT6 signaling interferes with the striatum’s role in mediating action–outcome associations. However, in that study the agonists were administered systemically, thus effects could not be ascribed solely to the striatum, let alone to specific subregions therein. It is also possible that 5-HT6 receptors acting in medium spiny neurons (Ward and Dorsa, 1996) affect action–outcome learning. For example, indirect pathway medium spiny neurons express both 5-HT6 receptors and dopamine D2 receptors that are, respectively, positively and negatively coupled to adenylate cyclase providing a potential cellular stage for opposing interactions between serotonin and dopamine signaling within the striatum. While dopamine facilitates reward and reinforcement learning, as supported by a large body of evidence (Dayan and Balleine, 2002; Everitt and Robbins, 2005; Wickens et al., 2007), serotonin might play an opposing role in striatum as our data and computational modeling suggest (Daw et al., 2002). Future studies, using neuron-subtype-specific viral vectors will help to clarify this matter. It is also possible that the temporal pattern of 5-HT6 receptor activation interferes with dopaminergic mechanisms that promote reinforcement learning.

In summary, this study establishes the importance of striatal 5-HT6 receptor signaling in the posterior DMS in interfering with the acquisition of a discrete action–outcome task. This finding will help in understanding the role of individual serotonin receptor subtypes in the underlying learning mechanisms relevant to neuropsychiatric disorders of the striatum.

Acknowledgments

Thanks are due to Dr Michele Kelly and Dr Sunila Nair for helpful comments and review of the manuscript and to Evan Carlos, Hannah DeMeritt and Aaron Mesnik-Greene for technical support.

Funding

This work was supported by the NIH Medical Scientist Training Program (grant number 5 T32 GM07266 to DE), an NIH Institutional Grant for Neurobiology (grant number 5 T32 GM07108 to DE), an Achievement Rewards for College Scientists (to DE), and NIH NIDA (grant number DA021273 to JFN).

Footnotes

Conflict of interest statement

None declared.

References

  1. Abel T, Lattal KM. Molecular mechanisms of memory acquisition, consolidation and retrieval. Curr Opin Neurobiol. 2001;11:180–187. doi: 10.1016/s0959-4388(00)00194-x. [DOI] [PubMed] [Google Scholar]
  2. Barot SK, Ferguson SM, Neumaier JF. 5-HT(1B) receptors in nucleus accumbens efferents enhance both rewarding and aversive effects of cocaine. Eur J Neurosci. 2007;25:3125–3131. doi: 10.1111/j.1460-9568.2007.05568.x. [DOI] [PubMed] [Google Scholar]
  3. Bonsi P, Cuomo D, Ding J, et al. Endogenous serotonin excites striatal cholinergic interneurons via the activation of 5-HT 2C, 5-HT6, and 5-HT7 serotonin receptors: implications for extrapyramidal side effects of serotonin reuptake inhibitors. Neuropsychopharmacology. 2007;32:1840–1854. doi: 10.1038/sj.npp.1301294. [DOI] [PubMed] [Google Scholar]
  4. Bourson A, Borroni E, Austin RH, Monsma FJ, Jr, Sleight AJ. Determination of the role of the 5-HT6 receptor in the rat brain: a study using antisense oligonucleotides. J Pharmacol Exp Ther. 1995;274:173–180. [PubMed] [Google Scholar]
  5. Corbit LH, Balleine BW. The role of the hippocampus in instrumental conditioning. J Neurosci. 2000;20:4233–4239. doi: 10.1523/JNEUROSCI.20-11-04233.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Daw ND, Kakade S, Dayan P. Opponent interactions between serotonin and dopamine. Neural Netw. 2002;15:603–616. doi: 10.1016/s0893-6080(02)00052-7. [DOI] [PubMed] [Google Scholar]
  7. Dayan P, Balleine BW. Reward, motivation, and reinforcement learning. Neuron. 2002;36:285–298. doi: 10.1016/s0896-6273(02)00963-7. [DOI] [PubMed] [Google Scholar]
  8. Diekelmann S, Wilhelm I, Born J. The what’s and when’s of sleep-dependent memory consolidation. Sleep Med Rev. 2009;13:309–321. doi: 10.1016/j.smrv.2008.08.002. [DOI] [PubMed] [Google Scholar]
  9. Everitt BJ, Cardinal RN, Parkinson JA, Robbins TW. Appetitive behavior: impact of amygdala-dependent mechanisms of emotional learning. Ann N Y Acad Sci. 2003;985:233–250. [PubMed] [Google Scholar]
  10. Everitt BJ, Robbins TW. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci. 2005;8:1481–1489. doi: 10.1038/nn1579. [DOI] [PubMed] [Google Scholar]
  11. Ferguson SM, Mitchell ES, Neumaier JF. Increased expression of 5-HT6 receptors in the nucleus accumbens blocks the rewarding but not psychomotor activating properties of cocaine. Biol Psychiatry. 2008;63:207–213. doi: 10.1016/j.biopsych.2007.02.018. [DOI] [PubMed] [Google Scholar]
  12. Fletcher PJ. Effects of d-fenfluramine and metergoline on responding for conditioned reward and the response potentiating effect of nucleus accumbens d-amphetamine. Psychopharmacology (Berl) 1995;118:155–163. doi: 10.1007/BF02245834. [DOI] [PubMed] [Google Scholar]
  13. Fletcher PJ. Injection of 5-HT into the nucleus accumbens reduces the effects of d-amphetamine on responding for conditioned reward. Psychopharmacology (Berl) 1996;126:62–69. doi: 10.1007/BF02246412. [DOI] [PubMed] [Google Scholar]
  14. Fletcher PJ, Korth KM, Chambers JW. Selective destruction of brain serotonin neurons by 5,7-dihydroxytryptamine increases responding for a conditioned reward. Psychopharmacology (Berl) 1999;147:291–299. doi: 10.1007/s002130051170. [DOI] [PubMed] [Google Scholar]
  15. Frantz KJ, Hansson KJ, Stouffer DG, Parsons LH. 5-HT(6) receptor antagonism potentiates the behavioral and neurochemical effects of amphetamine but not cocaine. Neuropharmacology. 2002;42:170–180. doi: 10.1016/s0028-3908(01)00165-4. [DOI] [PubMed] [Google Scholar]
  16. Grahn JA, Parkinson JA, Owen AM. The cognitive functions of the caudate nucleus. Prog Neurobiol. 2008;86:141–155. doi: 10.1016/j.pneurobio.2008.09.004. [DOI] [PubMed] [Google Scholar]
  17. Graybiel AM. Habits, rituals, and the evaluative brain. Annu Rev Neurosci. 2008;31:359–387. doi: 10.1146/annurev.neuro.29.051605.112851. [DOI] [PubMed] [Google Scholar]
  18. Heal DJ, Smith SL, Fisas A, Codony X, Buschmann H. Selective 5-HT6 receptor ligands: progress in the development of a novel pharmacological approach to the treatment of obesity and related metabolic disorders. Pharmacol Ther. 2008;117:207–231. doi: 10.1016/j.pharmthera.2007.08.006. [DOI] [PubMed] [Google Scholar]
  19. Kelley AE, Domesick VB, Nauta WJ. The amygdalostriatal projection in the rat – an anatomical study by anterograde and retrograde tracing methods. Neuroscience. 1982;7:615–630. doi: 10.1016/0306-4522(82)90067-7. [DOI] [PubMed] [Google Scholar]
  20. Laubach M. Who’s on first? What’s on second? The time course of learning in corticostriatal systems. Trends Neurosci. 2005;28:509–511. doi: 10.1016/j.tins.2005.07.008. [DOI] [PubMed] [Google Scholar]
  21. Millan MJ, Marin P, Bockaert J, la Cour CM. Signaling at G-protein-coupled serotonin receptors: recent advances and future research directions. Trends Pharmacol Sci. 2008;29:454–464. doi: 10.1016/j.tips.2008.06.007. [DOI] [PubMed] [Google Scholar]
  22. Mitchell ES, Neumaier JF. 5-HT6 receptor antagonist reversal of emotional learning and prepulse inhibition deficits induced by apomorphine or scopolamine. Pharmacol Biochem Behav. 2008;88(3):291–298. doi: 10.1016/j.pbb.2007.08.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Mitchell ES, Sexton T, Neumaier JF. Increased expression of 5-HT6 receptors in the rat dorsomedial striatum impairs instrumental learning. Neuropsychopharmacology. 2007;32:1520–1530. doi: 10.1038/sj.npp.1301284. [DOI] [PubMed] [Google Scholar]
  24. Neumaier JF, Vincow ES, Arvanitogiannis A, Wise RA, Carlezon WA., Jr Elevated expression of 5-HT1B receptors in nucleus accumbens efferents sensitizes animals to cocaine. J Neurosci. 2002;22:10,856–10,863. doi: 10.1523/JNEUROSCI.22-24-10856.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Perez-Garcia G, Meneses A. Oral administration of the 5-HT6 receptor antagonists SB-357134 and SB-399885 improves memory formation in an autoshaping learning task. Pharmacol Biochem Behav. 2005;81:673–682. doi: 10.1016/j.pbb.2005.05.005. [DOI] [PubMed] [Google Scholar]
  26. Perez-García G, Meneses A. Oral administration of the 5-HT6 receptor antagonists SB-357134 and SB-399885 improves memory formation in an autoshaping learning task. Pharmacol Biochem Behav. 2005;81:673–682. doi: 10.1016/j.pbb.2005.05.005. [DOI] [PubMed] [Google Scholar]
  27. Roberts JC, Reavill C, East SZ, et al. The distribution of 5-HT(6) receptors in rat brain: an autoradiographic binding study using the radiolabelled 5-HT(6) receptor antagonist [(125)I]SB-258585. Brain Res. 2002;934:49–57. doi: 10.1016/s0006-8993(02)02360-0. [DOI] [PubMed] [Google Scholar]
  28. Schechter LE, Lin Q, Smith DL, et al. Neuropharmacological profile of novel and selective 5-HT6 receptor agonists: WAY-181187 and WAY-208466. Neuropsychopharmacology. 2008;33:1323–1335. doi: 10.1038/sj.npp.1301503. [DOI] [PubMed] [Google Scholar]
  29. Sleight AJ, Boess FG, Bos M, Levet-Trafit B, Riemer C, Bourson A. Characterization of Ro 04-6790 and Ro 63-0563: potent and selective antagonists at human and rat 5-HT6 receptors. Br J Pharmacol. 1998;124:556–562. doi: 10.1038/sj.bjp.0701851. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Sossin WS. Defining memories by their distinct molecular traces. Trends Neurosci. 2008;31:170–175. doi: 10.1016/j.tins.2008.01.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Voorn P, Vanderschuren LJ, Groenewegen HJ, Robbins TW, Pennartz CM. Putting a spin on the dorsal–ventral divide of the striatum. Trends Neurosci. 2004;27:468–474. doi: 10.1016/j.tins.2004.06.006. [DOI] [PubMed] [Google Scholar]
  32. Ward RP, Dorsa DM. Colocalization of serotonin receptor subtypes 5-HT2A, 5-HT2C, and 5-HT6 with neuropeptides in rat striatum. J Comp Neurol. 1996;370:405–414. doi: 10.1002/(SICI)1096-9861(19960701)370:3<405::AID-CNE10>3.0.CO;2-R. [DOI] [PubMed] [Google Scholar]
  33. Wickens JR, Budd CS, Hyland BI, Arbuthnott GW. Striatal contributions to reward and decision making: making sense of regional variations in a reiterated processing matrix. Ann N Y Acad Sci. 2007;1104:192–212. doi: 10.1196/annals.1390.016. [DOI] [PubMed] [Google Scholar]
  34. Yin HH, Knowlton BJ. The role of the basal ganglia in habit formation. Nat Rev Neurosci. 2006;7:464–476. doi: 10.1038/nrn1919. [DOI] [PubMed] [Google Scholar]
  35. Yin HH, Ostlund SB, Knowlton B, Balleine BW. The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci. 2005;22:513–523. doi: 10.1111/j.1460-9568.2005.04218.x. [DOI] [PubMed] [Google Scholar]

RESOURCES