Deconstructing 5-HT₆ receptor effects on striatal circuit function

Abstract

Medium spiny neurons (MSNs) constitute 95% of neurons in dorsal striatum subdivided into direct (striatonigral) and indirect (striatopallidal) pathways. Whereas D₁ and D₂ receptors and several neuropeptides, including dynorphin and enkephalin, are differentially expressed in these neurons, 5-HT₆ receptors are expressed in both pathways. Previous results demonstrate that concurrent 5-HT₆ receptor overexpression in MSNs of both pathways in dorsomedial striatum interferes with instrumental learning and that 5-HT₆ overexpression in dorsolateral striatum relieves rats from inflexible habitual behaviors. We hypothesized that 5-HT₆ receptor-mediated co-activation of both pathways interferes with the differential activation/inhibition of direct/indirect pathways by dopamine. To test this idea, we cloned novel viral vectors to selectively overexpress 5-HT₆ receptors in direct or indirect pathway MSNs to deconstruct their role in modulating instrumental learning and habitual responding. We found that increasing 5-HT₆ receptor expression in either direct or indirect pathway MSNs of the posterior dorsomedial striatum selectively enhanced or impaired initial acquisition of a discrete instrumental learning task, respectively, though all rats were ultimately able to learn the task. In a separate set of experiments, 5-HT₆ receptor overexpression in indirect pathway MSNs of the dorsolateral striatum facilitated behavioral flexibility in rats overtrained on a repetitive pressing task using a variable interval schedule of reinforcement, during an omission contingency training session and subsequent probe testing. Together these findings further the notion that 5-HT₆ signaling causes balanced activation of opposing MSN pathways by serotonin in sub-regions of dorsal striatum allowing for more reflective modalities of behavior.

1. INTRODUCTION

Multiple lines of evidence point toward a role for striatal serotonin in the symptoms from, and treatments for, several neuropsychiatric conditions including addiction, obsessive compulsive disorder, and Tourette syndrome (Di Matteo et al., 2008, Goddard et al., 2008, Nic Dhonnchadha and Cunningham, 2008, Roessner et al., 2013). Treatment options for these disorders are limited due to the nonspecific actions of current medications in relation to multiple serotonin receptors. By clarifying the receptor-specific mechanisms of serotonergic signaling in the striatum we hope to facilitate the direction of development of more effective therapeutic approaches to these disorders. 5-HT₆ receptors make excellent candidates as a therapeutic target because they are abundantly expressed throughout striatum (Hirst et al., 2000, Roberts et al., 2002), modulate striatal-based behaviors (Mitchell et al., 2007, Ferguson et al., 2008, Eskenazi and Neumaier, 2010) and show high-affinity for several psychiatric medications used in the treatment of the aforementioned disorders (Roth et al., 1994, Dupuis et al., 2008).

The role of striatum on various behaviors such as reward motivated learning and habitual or compulsive behavior are sub-region specific (Yin and Knowlton, 2006) with dorsomedial striatum (DMS) being more involved in instrumental learning acquisition and with dorsolateral striatum (DLS) playing a larger role in habitual responding. Extracellular serotonin levels are increased from baseline in the striatum of rats during operant conflict procedures, but not in hippocampus or cortex (Beaufour et al., 2001). Striatal 5-HT₆ receptor signaling acts in region-specific ways to either decrease reward valence in classical conditioning in ventral striatum (Ferguson et al., 2008), interfere with instrumental learning in DMS (Mitchell et al., 2007, Eskenazi and Neumaier, 2010) or impede habitual responding after variable interval training in DLS (Eskenazi and Neumaier, 2011). In high responder rats bred to be more active in novel environments and have increased drug-taking behavior there are lower endogenous levels of striatal 5-HT₆ receptors compared to low responders (Ballaz et al., 2007). Therefore, it appears that 5-HT₆ receptors play a role in these behaviors that serves to temper goal-directed and habitual actions; therefore, 5-HT₆ receptors’ activity may be therapeutic in overdriven states. However, it is unclear which cells underlie these effects.

5-HT₆ receptors are found on neurons of both main efferent pathways of striatum: the striatonigral (direct) and striatopallidal (indirect) medium spiny neurons (MSN) which comprise 95% of the neurons in the striatum (Ward and Dorsa, 1996). Both direct and indirect pathway MSNs (dMSN and iMSN respectively) participate in cortico-striato-thalamo-cortical loops; in general, dMSNs facilitate and iMSNs inhibit action selection and habitual responding (Gerfen, 1992). Since these neurons comingle throughout striatum, it has been technically challenging to discriminate the function of each type of MSN separately as with, for example, focal electrical stimulation or lesion studies which would either activate or inhibit both pathways simultaneously. Some degree of selectively has been achieved with pharmacologic agents, and even more recently, technical advances have allowed researchers to separate the functions of these pathways either by using dissociation and cell sorting methods (Lobo et al., 2006), or using viral vectors or germline genetic mutations to express transgenes selectively in one pathway or the other. Recent methods such as differential Cre recombinase expression of tetanus toxin (Hikida et al., 2010), designer receptors exclusively activated by designer drugs (DREADDs) (Ferguson et al., 2011) and optogenetic (Kravitz et al., 2010) have shown the functional opposition of the two striatal pathways. These studies support the view of dMSNs and iMSNs as generally activating and inhibiting behavioral output, respectively. For the purposes of this study, we developed viral vectors that utilize the preprodynorphin or preproenkephalin promoters to manipulate rat dMSNs and iMSNs selectively, based on known well-characterized viral vector promotors (Ferguson et al., 2011, Anderson et al., 2013, Ferguson et al., 2013, Michaelides et al., 2013).

The mechanism underlying serotonin’s role in modulating these behaviors most likely involves excitation of GABAergic MSNs by 5-HT₆ receptors via positive coupling to adenylate cyclase (Sebben et al., 1994) as demonstrated by 5-HT₆ selective agonist-induced elevation of striatal GABA levels (Schechter et al., 2008) and electrophysiological recordings (Bonsi et al., 2007). Therefore, our hypothesis is that using viral-mediated gene transfer to overexpress 5-HT₆ receptors selectively in iMSNs will be sufficient to attenuate instrumental learning when infused into DMS and will interfere with habitual behaviors when infused into DLS. This strategy differs from pharmacological approaches because the viral-vector mediated increase in receptor density is activated by endogenously released serotonin, preserving the encoding of information inherent to the temporal and spatial dynamics of serotonin release in striatum; this information encoded by 5-HT release has been shown to be responsive to reinforcement and anticipation of reward (Miyazaki et al., 2012). Recent findings demonstrate that activation of serotonergic dorsal raphe neurons positively reinforces behaviors and signals reward (Liu et al., 2014), providing further evidence of a role for serotonin in modulating instrumental and habitual behaviors.

2. EXPERIMENTAL PROCEDURES

2.1. Viral vector construction

Two Herpes Simplex Virus (HSV) vectors were constructed for the purposes of these studies based on vectors that we and others have extensively characterized with regard to their time course and selectivity of expression (Ferguson et al., 2011, Anderson et al., 2013, Ferguson et al., 2013, Michaelides et al., 2013). HSV is an enveloped, double-stranded DNA virus that selectively enters neurons via glycoprotein interactions. We use a replication-deficient strain that displays a predilection for anterograde transport resulting in transfection of neurons at the injection site (i.e. the strain is not substantially retrogradely transported) (Fink et al., 1996). The first utilizes preprodynorphin promoter (pDYN) to drive transgene expression in dMSNs while the second uses the preproenkephalin promoter (pENK) to target iMSNs; both are >90% selective (Ferguson et al., 2011, Anderson et al., 2013, Ferguson et al., 2013, Michaelides et al., 2013). To allow us to distinguish from endogenous 5-HT₆ receptors, a double hemagglutinin (2xHA) epitope-tag was introduced to the N terminal of the rat 5-HT₆ receptor gene by cloning an oligonucleotide in-frame upstream of the 5-HT₆ sequence using pBlueScript as this does not interfere with receptor functionality as shown previously (Mitchell et al., 2007). Correct sequences were confirmed using high-quality automated DNA sequencing technology. The 2xHA-5-HT₆ gene was then inserted into the final vector plasmids: pDYN-5-HT₆ and pENK-5-HT₆. These plasmids were then used to package the final vectors at titers of approximately 1×10⁸ infectious particles/ml as previously described (Clark et al., 2002). Unlike transgene expression driven from the HSV promotor which has a peak expression around day four, transgene expression from pDYN and pENK promotors are maximally expressed by day seven post-infusion and remain elevated for at least 30 days after (Ferguson et al., 2011), regardless of whether packaged inside HSV virions. As controls, we used vectors previously described that drive enhanced green fluorescent protein (eGFP) expression from the same promotors, i.e. pENK-eGFP and pDYN-eGFP (Ferguson et al., 2011, Ferguson et al., 2013, Michaelides et al., 2013).

2.2. Immunohistochemical verification of viral vector selectively in vivo

The selectivity of the pDYN and pENK vectors to drive expression of genes selectively in either dMSNs or iMSNs has been characterized elsewhere using co-localization of protein expression with pathway-specific markers as well as retrogradely-transported FluroGold tracer into substantia nigra and globus pallidus respectively (Ferguson et al., 2011). Verification of overexpression of 5-HT₆ receptors using the vectors created for this study were confirmed by immunohistochemical (IHC) colocalization of the HA tag with markers of the direct and indirect pathway, substance P (SP) and enkephalin (ENK) respectively, as the HA tag is exclusive to virally expressed receptors and is not found on endogenous receptors. In addition to rats used exclusively for IHC verification which were euthanized seven days post infusion of viral vector into dorsal striatum, rats used for behavioral studies to were also used in order to obtain time points of expression spanning the duration of behavioral studies and minimize number of animals used. Thus, rats from Experiment 2 (see below) were euthanized twenty-two days after viral vector infusion into dorsal striatum. Tissue was prepared as described in methods below. For staining, floating sections were washed in 0.5% Triton-X/phosphate buffered saline (PBS) for 10 min, then blocked in 5% normal goat serum (NGS)-0.25% Triton X/PBS for one hour. Sections were then incubated in 2.5% NGS-0.25%Triton-X/PBS containing primary antibody to HA (1:200, Chemicon/Millipore) and either SP (1:400, Chemicon/Millipore, Temecula, CA) or ENK (1:100, Immunostar, Hudson, WI) with gentle agitation at 4°C for 24 to 72h. Next, sections were rinsed four times in PBS and incubated in species-appropriate Alexa Fluor 488 (green) and Alexa 568 (red)-conjugated goat secondary antibodies (1:500, Invitrogen, Carlsbad, CA) for one hour. Sections were washed two times in PBS, mounted on slides and cover-slipped with Vectashield (Vector Labs, Burlingame, CA). Images were captured with a Bio-Rad Radiance 2000 confocal system and an associated Nikon fluorescence microscope using an argon/krypton laser and red laser diode. Images were analyzed using ImageJ (Schneider et al., 2012) to compile z-stacks, enhance and smooth processes.

2.3. Animal use

Experimental procedures were approved by the University of Washington Institutional Animal Care and Use Committee and were conducted in accordance with National Institutes of Health guidelines.

Male Long-Evans rats (Charles River, 250–275g at arrival) were habituated to the colony room (12-hour light/dark cycle, lights on at 06:00 AM, temperature set to 21°C, ad libitum rat chow except as indicated) for five days before being handled. For the next five days, rats were handled and then fed once daily at the end of the light cycle with 18g of rat chow (this mildly restricted diet produced ~5% reduction in weight gain compared to ad libitum fed rats). Behavioral experiments were performed during the light cycle in order to allow comparisons to previous experiments.

Behavioral testing was conducted in sound-attenuating boxes containing standard rat modular test chambers equipped with two levers, two cue lights, a house light, a food receptacle, and fans to provide temperature regulation and white noise (Med Associates, Georgia, VT, USA). Each individual rat was testing in the same chamber throughout all sessions in order to reduce variability between sessions.

2.3.1. Surgical procedures

Our surgical procedures have been previously published (Eskenazi and Neumaier, 2010, 2011). Briefly, rats were removed from food restriction the night before surgery and returned to food restriction 24 hours post-operatively. Rats were anesthetized with isoflurane gas (1–3%) before and throughout the stereotaxic surgery and meloxicam analgesia (0.1cc at 0.4mg/ml sc) was administered pre-operatively. Choice of infusion coordinates were based on previous studies: posterior DMS: A/P −0.8mm from bregma, M/L ±3.2 from midline, D/V −3.4mm from brain surface (Yin et al., 2005, Eskenazi and Neumaier, 2010) and DLS: A/P + 0.7mm from bregma, M/L ± 3.8mm from midline, D/V −4.0mm from brain surface (Yin et al., 2006, Eskenazi and Neumaier, 2011). Bilateral bore holes were drilled above the coordinates. Using a 27-G long dental needle, 2μL of viral particles (10⁸ infective units/mL) per side were infused over 10min at a rate of 200nL/min, and the needle was left in place for an additional 5min to minimize backflow before being slowly removed. This amount of viral vector was chosen based on previous studies to produce discrete infection at the target region; representative areas of infusion are shown in figures from cited references (Neumaier et al., 2002, Eskenazi and Neumaier, 2010, 2011). Effects of pathway specific 5-HT₆ receptor overexpression in medium-spiny neurons of the DMS on learning were compared to pathway-specific eGFP expressing control vectors. Prior work demonstrated that rats receiving eGFP control vector were no different than those receiving sham surgery or no surgery using this same behavioral procedure (Mitchell et al., 2007).

2.3.2. Transcardial perfusions, tissue preparation and exclusion criteria

Rats were euthanized with an intraperitoneal injection of pentobarbital sodium and phenytoin sodium. Perfusion only proceeded once rats were unresponsive to paw pinch and upon absence of corneal reflex. Transcardiac perfusion used 100mL phosphate-buffered saline solution followed by 200mL of 4% paraformaldehyde (both solutions titrated to pH 7.4, kept on ice and prepared within 24hours of perfusion). Brains were removed and left in 4% paraformaldehyde for 4 hours, then stored in PBS. Tissue sections were made on a Leica VT1000S vibrating blade microtome and mounted on slides at 40μm thickness. Accuracy of injection coordinates was confirmed by visualization of eGFP or hemagglutinin immunofluorescence or by cresyl violet staining of the injection needle tracts. Rats with injection sites outside of the targeted brain region were excluded from the experiments.

2.4. Experiment 1: Effect of 5-HT₆ receptor overexpression in iMSNs versus dMSNs of DMS on instrumental learning

In our previous studies increased expression of 5-HT₆ receptors in the posterior DMS using a viral vector specific to neurons, but not specific to neuronal subtypes, interfered with instrumental learning (Eskenazi and Neumaier, 2010). For this reason, we targeted the posterior DMS for these studies and decided to test the pENK and pDYN vectors to drive 5-HT₆ receptor overexpression selectively in either iMSNs or dMSNs on separate groups of rats using a three-day learning paradigm. This paradigm was adapted from Eskenazi & Neumaier 2010, which was in turn based on previous learning studies investigating 5-HT₆ receptors from our lab and others (Perez-Garcia and Meneses, 2005, Mitchell et al., 2007). Briefly, prior studies demonstrated that DMS 5-HT₆ receptor overexpression interfered with an instrumental learning auto-shaping task consisting of three training sessions of 50 trials on a 50 second fixed interval (FI50) schedule over three days (Mitchell et al., 2007). Subsequent work using a single training session of 100 trials on a FI20 schedule confirmed that DMS 5-HT₆ receptors impaired within session learning acquisition as distinct from between session learning consolidation (Eskenazi and Neumaier, 2010). This work also parametrically delineated two different behavioral procedures in separate groups of rats. DMS 5-HT₆ receptor overexpression affected performance on a task with cued trials of an inserting/retracting lever in which rats had to attend to lever insertion and each press was reinforced causing lever retraction but did not alter performance on a continuously extended lever that permitted rats to employ a repetitive pressing strategy. This suggested that the different modality of responding may result from different underlying neural structures. In these prior studies lever-pressing behavior underwent extinction after three more sessions without reinforcement comparably in rats receiving eGFP vs 5-HT₆ viral vectors bolstering support for this as a goal-directed instrumental learning task.

For the present set of experiments, rats habituated to the colony for five days, were handled for five days, then put on food restriction of rat chow 18g daily at the end of the light cycle, handled another five days before undergoing stereotaxic surgery for infusion of viral vectors. Seven days post-infusion rats underwent a short pre-test to habituate the rat to the testing chamber and sucrose pellets. The following day began training sessions on consecutive days between the hours of 2:00 – 6:00 PM. Each of three sessions comprised of 100 trials that consisted of the presentation of a single lever that was inserted into the chamber on a FI20 schedule. The lever remained extended for ten seconds; then retracted either if pressed or at the end of ten seconds. Successful lever-presses led to the delivery of one sucrose pellet into the food receptacle and lever retraction for the twenty second inter-trial interval. This thereby prevented rats from employing a repetitive lever-pressing modality. House-light and the light above the lever were continuously on throughout the session. After completion of the 100 trials the rat was returned to the home cage. Initially, eight rats per group were included, then numbers added according to statistical stopping rules (Fitts, 2010). Final groups (and sizes) consisted of pENK-5-HT₆ (n=12), pENK-eGFP (n=8), pDYN-5-HT₆ (n=6), and pDYN-eGFP (n=7), after exclusion of surgical misses (pDYN-5HT₆ 2 rats, pDYN-eGFP 1 rat).

Main outcomes were time to initial lever press (interval from session start until first successful response) across sessions and further analysis as a survival curve. These were used as main outcomes because averaging of data to make learning curves can sacrifice important individual data as discussed elsewhere (Gallistel et al., 2004). Total sucrose pellets earned as well as latency to respond after lever extension on completed trials were also analyzed over all three sessions.

2.5. Experiment 2: Effect of 5-HT₆ receptor overexpression in DLS-iMSNs on inflexible habitual responding

Directed in part by the results from Experiment 1 (described below) and our interest in finding potential signaling mechanisms that decrease habitual responding, we decided to focus on indirect pathway MSNs using the pENK driven vectors and the omission learning procedure (Yin et al., 2006, Eskenazi and Neumaier, 2011) for this experiment. In all phases of Experiment 2, there were two levers flanking the pellet receptacle that were continuously extended, one that was associated with pellet delivery based on the schedule of reinforcement or omission contingency for that session (active lever) and another lever (inactive lever) on which presses were recorded but had no impact on any other component of the session (e.g. pellet delivery or session termination). The presence of an inactive lever demonstrates the association with a specific lever (i.e. the lever associated with pellet delivery) rather than simply lever pressing per se. For a given rat, the active lever never varied (e.g. if the lever on the right of the food receptacle was active this held true across all sessions). By being continuously extended, the lever allowed for repetitive pressing (rather than discrete pressing as in Experiment 1). After arrival to the colony room, rats were given five days to habituate to the colony, handled for five days, then food restricted to rat chow 18g daily at end of light cycle and handled another five days. Then, rats underwent stereotaxic infusion of viral vectors and recovered for seven days before behavioral studies began.

2.5.1. Training

Due to the longer duration of gene expression from the pENK promotor compoared to HSV promoter (Barot et al., 2007, Ferguson et al., 2011) we were able to evaluate how 5-HT₆ receptor overexpression affected both habit formation and habit expression. After bilateral stereotaxic infusion of either pENK-5-HT₆ or pENK-eGFP vectors in the DLS training began seven days later, using the training schedule described by our group and others (Yin et al., 2004, Eskenazi and Neumaier, 2011). First, rats underwent two sessions of 30min each on the first day consisting of non-contingent pellet delivery (i.e. levers retracted throughout session, no press required) on a variable interval (VI) 60 seconds schedule to acclimate to chambers. Lever-press training was then as follows: four days on a fixed interval (FI) 20 seconds schedule, then one day of VI30 followed by six days VI60.

2.5.2. Omission training

After the training above, rats underwent an omission contingency training session. Rats were assigned to either omission or yoked conditions based on number of active lever presses in their last variable interval training session to create balanced groups. A session consisted of an omission group rat and a yoked group rat concurrently behaving in separate chambers. Rats in the omission group were reinforced for omitting lever presses (also referred to as differential reinforcement of other, DRO). Pellet delivery for both rats was controlled by a timer that counted down from 20sec before delivering a pellet. Each time the omission group rat pressed the lever, the timer reset to 20sec; however, if the omission group rat omitted pressing the lever for 20sec, one pellet was delivered to each rat, omission and yoked. Lever presses performed by the yoked group rat did not affect pellet delivery but were recorded.

2.5.3. Final probe session

The day following omission training rats underwent their final ten min probe test under extinction conditions to test learning of the contingency switch the day before without confound of within session pellet delivery.

Group sizes were initially 20 rats per treatment (pENK-5-HT₆ and pENK-eGFP) divided into groups balanced for behavioral contingency (i.e. omission and yoked); thus, after exclusion of one rat based on surgical criteria (see above), group sizes were pENK-5-HT₆-omission (n=10), pENK-5-HT₆-yoked (n=10), pENK-eGFP-omission (n=10) and pENK-eGFP-yoked (n=9).

2.6. Statistical analyses

All analyses were performed using GraphPad Prism (Version 6.01).

Experiment 1 data for seconds to initial lever press were analyzed using two-way repeated measures analysis of variance (ANOVA) across sessions 1–3 and Tukey’s multiple comparisons test. Time to initial lever press from session one were also analyzed as a Kaplan-Meier survival plot and log-rank Mantel Cox test. Analysis of total pellets earned and per press latency during instrumental learning across all sessions was performed using two-way repeated measures ANOVA.

Experiment 2 data were analyzed using two-way repeated measures ANOVA for training lever press rate across training sessions 1–11. Within-session data for the omission contingency training session were also analyzed with two-way repeated measures ANOVA. Final probe session data were also analyzed with two-way ANOVA (viral vector x behavioral contingency). Analysis of relationship between final probe responsivity (final probe session over training press rate) and degree of training (i.e. active lever press rate in the final variable interval training session) was performed with linear regression.

Significance was determined with alpha value of 0.05. Binned data are shown in tenths of a session (e.g. omission session is 30min and subdivided into 3min bins).

3. RESULTS

3.1. Results of immunohistochemical verification of viral vector selectivity and surgical accuracy

After pENK-5-HT₆ vector infusion into rat striatum, anti-HA immunostaining colocalized with anti-enkephalin immunostaining in iMSNs (Fig 1C–E), whereas anti-HA immunostaining did not colocalize with substance-P staining (Fig 1F–H), a marker for dMSNs. In analogous comparisons, we found that the pDYN-5-HT₆ vector demonstrated colocalization of HA and substance-P immunostaining in infected dMSNs (Fig 1I–K) whereas anti-HA and anti-enkephalin immunostaining did not colocalize (Fig 1L–N). Using these same vectors to express eGFP and DREADD receptors, we previously showed that these vectors require about one week to reach steady levels of transgene expression that then persists for at least several weeks (Ferguson et al., 2011). Indeed, half of the sections used for IHC staining verification were from rats euthanized seven days post infusion and the other half were from rats used in Experiment 2 that were euthanized twenty-two days after viral vector infusion, confirming that these vectors expressed 5-HT₆ receptors during the entire duration of the behavioral experiments (Experiment 1 behavioral tasks were run on days 8–11 post-infusion and Experiment 2 behavior tasks were run on days 8–21 post-infusion).

Figure 1. Viral vector plasmid maps and immunohistochemistry.

Plasmid maps for both experimental vectors expressing hemagluttinin (HA)-tagged 5-HT₆ receptors via either the pENK promoter (A) or the pDYN promoter (B). The top six photomicrographs (panels C–H) are of sections from rats infused in dorsal striatum with the pENK-5-HT₆ vector, the bottom six photomicrographs (panels I–N) with pDYN-5-HT₆. Immunohistochemical staining was as follows: leftmost column (C, F, I, L) with primary anti-HA and secondary conjugated Alexa Fluor 488; (D, M) with primary anti-enkephalin and secondary conjugated Alexa Fluor 568; (G, J) with primary anti-substance P and secondary conjugated Alexa Fluor 568. The rightmost column (E, H, K, N) are merges of the two preceding panels. Magnification of figures C–E/I–K is 60x and F–H/L–N is 40x.

3.2. Experiment 1: Effects of pathway specific 5-HT₆ receptor overexpression in dorsomedial striatum on a simple instrumental conditioning task targeting dMSNs and iMSNs using pDYN and pENK vectors respectively

During the first session, the pENK-5-HT₆ group took the most time until initial lever press (8min 22sec) and the pDYN-5-HT₆ group took the least amount of time (22sec). All groups demonstrated significantly decreased time until initial lever press across sessions, with a main effect of time (F ₆ = 3.246, P = 0.0464). Furthermore, post-hoc analysis demonstrated a significant difference between pDYN-5-HT₆ and pENK-5-HT₆ during session one (95% confidence interval −870.6 to −78.02), though no other within-session pairwise comparisons were statistically significant, as shown in Figure 2A. Further analysis taking into account only data from session one for initial lever press viewed as a survival plot separately revealed a statistically significant difference between groups (χ² = 18.92 with three degrees of freedom and P = 0.0003, Figure 2B).

Figure 2. Experiment 1 data: DMS dMSN and iMSN vectors effects on instrumental learning.

(A) Times until initial lever press (interval between beginning of session and first successful lever press) in all groups across three sessions. (B) Further analysis of session one with a Kaplan-Meier survival plot. (C & D) Pellets earned by group across three sessions; note: data were analyzed jointly but are shown separately for clarity. Group sizes: pENK-5-HT₆ (n=12), pENK-eGFP (n=8), pDYN-5-HT₆ (n=6), pDYN-eGFP (n=7). * P < 0.05, *** P < 0.0001. All error bars represent standard error of the mean; error bars not seen are smaller than the symbol for that data point.

There was a main effect of time across sessions for total sucrose pellets earned (F_2,58 = 33.20, P < 0.0001) though no main effect of viral vector (F_3,29 = 0.7858, P = 0.5116). However, as is evident from the graphs, there is a trend for pENK-5-HT₆ rats to earn fewer pellets than pENK-eGFP controls and a trend for pDYN-5-HT₆ rats to earn more pellets than pDYN-eGFP controls. Pellets earned per session were jointly analyzed for all groups but were graphed separately for direct/indirect pathway vectors for clarity (Figure 2C and 2D). Latency to respond on completed trials (time from lever extension until lever press per trial) was also significantly improved across sessions (F_2,58 = 11.31, P < 0.0001) but there was no main effect of viral vector (F_3,29 = 0.1537, P = 0.9265, data not shown).

3.3. Experiment 2: Effects of 5-HT₆ receptor overexpression in iMSNs of the dorsolateral striatum using pENK vectors

Rats in both the pENK-5-HT₆ and the pENK-eGFP groups reached high lever press rates over eleven training sessions of Experiment 2 (main effect time F_10,370 = 79.10, P < 0.0001) and there was no difference between groups (main effect vector F_1,37 = 0.0372, P = 0.8482), Figure 3A. After training, each group was further subdivided into balanced subgroups assigned to either omission contingency training or yoked control. From omission contingency training session data, there was an interaction between vector and contingency assignment on active lever presses (F_1,35 = 5.325, P = 0.0271), accounted for by a significant difference between pENK-5-HT₆ omission and pENK-5-HT₆ yoked groups (Tukey’s post-hoc test 95% confidence interval −365.8 to −12.22); (main effect behavioral contingency F_1,35 = 2.940, P = 0.0953; main effect viral vector F_1,35 = 0.0596, P = 0.8086). These data are shown in two different graphs for clarity though data were analyzed together (Figure 3B and C). In all eleven training sessions and the omission contingency training session there was a robust preference for the active lever over the inactive lever (data not shown).

Figure 3. Experiment 2 data: DLS-iMSN increased 5-HT₆ receptors expression effects on training, omission contingency and final probe.

(A) Training press rate on the active lever for both pENK-5-HT₆ and pENK-eGFP groups during the first eleven training sessions with contingency shown below x-axis. (B & C) Within session data in ten three-minute bins from the omission contingency training session, note: data analyzed jointly for all four groups but shown separately in B and C for clarity. (D) Final probe session with press rates normalized to training rates. (E) Secondary analysis of probe over training press rate ratio by training press rate as measure of omission susceptibility. Group sizes pENK-eGFP n=19 (10 omission, 9 yoked), pENK-5-HT₆ n=20 (10 omission, 10 yoked). * P < 0.05, *** P < 0.0001. All error bars represent standard error of the mean; error bars not seen are smaller than the symbol for that data point. FI: fixed interval schedule of reinforcement, PR: press rate, VI: variable interval.

Data from the final probe session demonstrated a statistically significant interaction between viral vector and behavioral contingency (interaction F_1,35= 4.265, P = 0.0464) though no main effect from either behavioral contingency (omission vs yoked, F_1,35=2.190, P=0.1478) or viral vector (eGFP vs 5-HT₆, F_1,35=0.1467, P=0.7040). This effect is primarily driven by differences between 5-HT₆ omission and yoked groups by post-hoc testing (Sidak 95% confidence interval −0.4368 to −0.01835), as no other pairwise comparisons were significant, Figure 3D. Data for each rat were normalized to final training press rate.

Secondary analysis to assess a potential relationship between degree of pre-omission training and final probe session performance revealed by linear regression analysis a statistically significant difference between groups (F_3,31 = 4.1212, P = 0.0143). Both eGFP groups (omission and yoked) and the 5-HT₆ omission group demonstrated inverse correlation of final probe performance and pre-omission training; however, the 5-HT₆ yoked group showed a positive correlation (Figure 3E).

4. DISCUSSION

These experiments deconstruct the impact of 5-HT₆ signaling in striatum by selectively overexpressing 5-HT₆ receptors in either direct or indirect pathway medium-spiny neurons (dMSN and iMSN, respectively) of the dorsomedial or dorsolateral striatum (DMS and DLS) using vectors cloned for this study. Overall, we observed that increased 5-HT₆ expression in iMSNs had a particularly significant impact in DMS by slowing instrumental learning and in DLS by facilitating behavioral flexibility after habitual responding.

The DMS is important in goal-directed behavior such as reward motivated learning, and differential plasticity in dMSNs and iMSNs is an important contributor to such learning (Shan et al., 2014). While the differential activation and inhibition of dMSNs and iMSNs by dopamine acting via D₁ and D₂ receptors has received great attention, it is possible that other neurotransmitters, including serotonin, also impact striatal learning by their balanced or imbalanced activity differentially activating these pathways (Kravitz et al., 2012). We used a simple instrumental learning task to examine the role of 5-HT₆ receptors in this region in this and previous studies. Using an HSV vector that was not selective for either MSN pathway, we previously found that increasing 5-HT₆ receptors in DMS interfered with instrumental learning (Mitchell et al., 2007, Eskenazi and Neumaier, 2010). Here, we demonstrate that pathway-specific 5-HT₆ receptor overexpression in the DMS differentially affected lever pressing patterns during instrumental learning on a simple sucrose-reinforced operant task. Increased expression of 5-HT₆ receptors in DMS-iMSNs significantly delayed initiation of responding during a simple operant task during the first session. Additionally, there was a trend for slower acquisition of the task over the three days that were tested. Conversely, there was a trend for rats with increased expression of 5-HT₆ receptors in dMSNs to make their initial lever press sooner and thereby acquire this task more quickly. However, the control rats learned the task fairly quickly and this may have reduced sensitivity to detect an accelerated acquisition rate. Perhaps in states of impaired striatal function, such as with reduced dopamine function as with Parkinson Disease, dMSN 5-HT₆ receptors might reveal such an effect, but this is outside the scope of the present study.

The DLS plays an important role in stimulus-response conditioning and is important for the maintenance of habit-based patterns of operant responding (Featherstone and McDonald, 2004, Yin et al., 2004, 2006). The omission procedure is sensitive to alterations in DLS function (Yin et al., 2006) so it was used to examine the role of 5-HT₆ receptors in habitual responding. Previously we found that increased 5-HT₆ receptor expression in DLS increased the sensitivity of rats to a change in contingency, showing a high rate of habit-like lever pressing using a non-selective vector (expressed in all striatal neurons in which it enters at the infusion site) (Eskenazi and Neumaier, 2011). We predicted that increasing 5-HT₆ receptors selectively in only indirect pathway MSNs would be sufficient to replicate our earlier result as they would oppose dopamine’s actions mediated by D₂ receptors in these neurons. Indeed, increasing 5-HT₆ receptor signaling in iMSNs of DLS was sufficient to enhance sensitivity to a change in reinforcement contingency. Since all rats were able to press at similarly high rates during the initial eleven days of training, there was unlikely to be any impairment in motor responses or instrumental learning due to 5-HT₆ receptor overexpression. Since there was no abatement of lever-pressing in the 5-HT₆-yoked controls, activation of 5-HT₆ receptors alone was insufficient to impair habit expression. Thus, we conclude that the combination of increased 5-HT₆ signaling in iMSNs and changed contingency was necessary to alter behavior. This is intriguing because it indicates that a combination of a biological and a behavioral intervention was needed to facilitate a behavioral change (as indicated by the reduced lever pressing during the probe session performed under extinction conditions). This is relevant to clinical problems such as obsessive-compulsive disorder, in which either biological or behavioral interventions alone may be insufficient to produce meaningful behavioral changes whereas a combination of these interventions may be more effective.

Dorsolateral striatum is a critical region supporting stable, inflexible patterns of response which may be adaptive in some cases but are insensitive to changes in environmental contingency, such as is encountered after compulsive habits are formed, in which case the continued expression of those behaviors is maladaptive. Dopamine release in striatum differentially affects the two output pathways by increasing dMSN activity and decreasing iMSN activity (Tritsch and Sabatini, 2012) which is thought to cause dopamine-dependent changes in the DLS, especially during habit formation more so than subsequent habitual responding (Wickens et al., 2007). On the contrary, 5-HT₆ receptors are expressed in both pathways (Ward and Dorsa, 1996) and serotonin release in striatum would act on these receptors to activate both striatal output pathways in a balanced fashion, thereby opposing the differentially-activating role of striatal dopamine or by re-patterning underlying DLS circuitry plasticity. Therefore, while 5-HT₆ signaling in DMS interferes with instrumental learning, it can facilitate behavioral flexibility in DLS—in both cases by opposing the differential activation of the MSN output pathways by dopamine. This is consistent with several previous observations. Increased 5-HT₆ receptor expression in nucleus accumbens disrupted the development of conditioned place preference for cocaine while a selective 5-HT₆ antagonist had the opposite effect; however, most importantly, in both cases 5-HT₆ manipulations did not alter associative learning in the conditioned place preference model in the absence of cocaine (Ferguson et al., 2008). Perhaps symmetrical stimulation of both dMSNs and iMSNs by 5-HT₆ receptor activation tends to preserve the stability of current behavioral strategies unless paired with new behavioral contingencies or psychoactive substances. The presence of several additional serotonin receptors in striatum certainly makes this more complex, but to our knowledge there is no evidence that any of the serotonin receptors are differentially expressed in dMSNs and iMSNs, suggesting that in all cases, serotonin may interfere with dopamine’s differential activation of these pathways.

Furthermore, the present data tie in to substantial work performed by others on the role of serotonin in impulsivity. In states of global serotonin depletion, the resulting behavioral effects can be fractionated into increases in impulsive choice as well as impulsive action (e.g. perseveration of non-rewarded responses) (Winstanley et al., 2004); which appear to be dependent on striatal serotonin-dopamine interactions (Winstanley et al., 2005). Experiment 1 data showed that increased expression of 5-HT₆ receptors in DMS-iMSNs decelerated initial acquisition of a lever-pressing task; this may relate to decreased impulsive exploration in the novel environment of the operant chamber, especially since there were no differences between groups in subsequent sessions. Similarly, in Experiment 2 DLS-iMSN 5-HT₆ overexpression in the omission group interfered with perseverative lever pressing during the omission session and final probe test. This could be interpreted as 5-HT₆ receptors decreasing impulsivity when faced with a contingency switch or extinction conditions. This view incorporates the finding that states of increased serotonin in rodents and humans facilitate waiting for reward (Schweighofer et al., 2008, Miyazaki et al., 2011, 2012) and that specifically striatal serotonin levels are elevated during operant conflict (Beaufour et al., 2001).

There were several limitations of the present study. The instrumental learning design did not distinguish between action-outcome and stimulus-response learning, which would have required another probe test such as reinforcer-devaluation (Dickinson and Balleine, 1994). However, our present data are nonetheless of interest especially since the learning impairment shown in session one by the DMS-iMSN group dissipates by sessions two and three. We also did not test increased 5-HT₆ expression in dMSNs of DLS, as we focused on the pathway that we expected would be the most informative and because we reasoned that there was no way to increase the inflexible responding observed in control animals further using the omission procedure.

5. CONCLUSIONS

In line with our hypothesis, 5-HT₆ receptors activate both dMSN and iMSN pathways to differentially impact instrumental learning and habitual responding. Together, these results suggest that striatal serotonin mediated 5-HT₆ receptor activation, when combined with switched-contingency training, may be a treatment option for compulsive disorders involving overlearned, stimulus-elicited actions. Further investigation of the role of 5-HT₆ receptors in the striatum will not only clarify how serotonin influences striatal functions but may also lead to new potential therapeutics for disorders involving compulsive behaviors.

HIGHLIGHTS.

• Striatal pathway-specific viral vectors expressing 5-HT₆ receptors are demonstrated

• 5-HT₆ overexpression within indirect MSNs in DMS slows instrumental learning

• Habit-like responding is attenuated by 5-HT₆ overexpression in indirect MSNs in DLS

ABBREVIATIONS

5-HT

5-hydroxytryptamine (serotonin)

5-HT₆

5-hydroxytryptamine 6 receptor

DLS

Dorsolateral striatum

DMS

Dorsomedial striatum

dMSN

Direct pathway medium spiny neuron (striatonigral)

DYN

Dynorphin (marker of striatonigral/dMSN pathway neurons)

eGFP

Enhanced green fluorescent protein

ENK

Enkephalin (marker of striatopallidal/iMSN pathway neurons)

FI

Fixed-interval reinforcement schedule

HSV

Herpes-simplex virus

iMSN

Indirect pathway medium spiny neuron (striatopallidal)

MSN

Medium spiny neuron

pDYN

Promoter region of the prodynorphin gene (expressed primarily in dMSNs

pENK

Promotore region of the preproenkephalin gene (expressed primarily in iMSNs)

PR

Lever-press rate

VI

Variable-interval reinforcement schedule