Serotonergic regulation of the dopaminergic system: implications for reward-related functions

Abstract

Serotonin is a critical neuromodulator involved in development and behavior. Its role in reward is however still debated. Here, we first review classical studies involving electrical stimulation protocols and pharmacological approaches. Contradictory results on the serotonergic’ involvement in reward emerge from these studies. These differences might be ascribable to either the diversity of cellular types within the raphe nuclei or/and the specific projection pathways of serotonergic neurons. We continue to review more recent work, using optogenetic approaches to activate serotonergic cells in the Raphe to VTA pathway. From these studies, it appears that activation of this pathway can lead to reinforcement learning mediated through the excitation of dopaminergic neurons by serotonergic neurons co-transmitting glutamate. Finally, given the importance of serotonin during development on adult emotion, the effect of abnormal early-life levels of serotonin on the dopaminergic system will also be discussed. Understanding the interaction between the serotonergic and dopaminergic systems during development and adulthood is critical to gain insight into the specific facets of neuropsychiatric disorders.

1. Introduction

“We would like provisionally to name it serotonin, which indicates that its source is serum and its activity is one of causing constriction” (Rapport et al., 1948). Serotonin (5-hydroxytryptamine, 5-HT), first named after the study of Rapport et al. in 1948, is now a classic and perhaps one of the most described neuromodulators. The functions of this crystalline substance go well beyond the vasoconstriction effect described in 1948. Indeed, although secreted to a minor extent in the brain compared to the massive amount found in the gastrointestinal tract (Bertrand and Bertrand, 2010), 5-HT has a major role in neuronal function.

Primarily found in the raphe nuclei (RN), subdivided into the Dorsal Raphe Nucleus (DRN) and Median Raphe Nucleus (MRN), 5-HT neurons compose 30–50% of these nuclei (Belin et al., 1983; Descarries et al., 1982; Huang et al., 2019). The function of this monoaminergic system starts early in development and 5-HT has been shown to have a key role in neural development (Khozhai and Otellin, 2012; Lavdas et al., 1997; Shah et al., 2018; Vichier-Guerre et al., 2017). In addition, the massive distribution of 5-HT fibers to practically all parts of the brain combined with the diversity of receptors (over 14 subtypes) explain the involvement of 5-HT in a long list of physiological and behavioral functions (Berger et al., 2009; Linley et al., 2013; Linley et al., 2017; Rodriguez et al., 2011; Vasudeva et al., 2011; Vertes, 1991; Vertes et al., 2010; Zhou and Azmitia, 1986) including respiration, reproduction, cardiovascular regulation, sleep-wake cycle, locomotion, emotion, learning, and memory (Charnay and Leger, 2010; Gervasoni et al., 2000; Jacobs and Fornal, 1993; King et al., 2008; Lucki, 1998; Teixeira et al., 2018; Teran et al., 2014; Urbain et al., 2006).

Among all the roles of the 5-HT system, one that is still being dissected is its role in reward (For review: (Hayes and Greenshaw, 2011; Hu, 2016; Kranz et al., 2010; Luo et al., 2015)). In animals, it was notably shown that electrical stimulation of the RN can be reinforcing (Miliaressis, 1977; Miliaressis et al., 1975; Van Der Kooy et al., 1978). In humans, using fMRI, the DRN is activated during a reward-related task (Tanaka et al., 2004).

The RN has strong connections to brain areas involved in reward processing including the nucleus accumbens (NAc), the amygdala, the medial prefrontal cortex and the olfactory tubercle (Beart and McDonald, 1982; Berridge and Kringelbach, 2008; Brown and Molliver, 2000; Macoveanu, 2014; Moore et al., 1978; Paton et al., 2006; Steinbusch et al., 1981; Vertes, 1991). But the most important interconnection linking the serotonergic system with reward function is the one with the dopaminergic system, dopamine (DA) being the well-recognized neurotransmitter involved in reward processing (Schultz, 1997; Schultz et al., 1997; Wise, 1996). The RN project massively to the ventral tegmental area (VTA) and to the substantia nigra (Herve et al., 1987; Miller et al., 1975; Vertes, 1991; Watabe-Uchida et al., 2012). The RN also receive, although to a lesser extent, projections back from the dopaminergic system (Kalen et al., 1988) (Ogawa et al., 2014; Vertes and Linley, 2008). Different views have emerged on the respective role of DA and 5-HT in reward functions either opposing them or making them allies (Boureau and Dayan, 2011; Cools et al., 2011; Daw et al., 2002). The complexity of this relationship notably lies in the fact that there is a heterogeneous expression of 5HT receptors in the VTA on both DA and non-DA neurons (De Deurwaerdere and Di Giovanni, 2017). The heterogeneous expression of receptors also holds true in other structures involved in reward processing (Hayes and Greenshaw, 2011). That may explain why manipulation of 5-HT levels has complex effects on reward processing. For example, in humans, there is a disparity in the effects of acute tryptophan depletion on reward-related processes and on the activation of reward-related brain areas (Cools et al., 2005; Cools et al., 2008; Faulkner and Deakin, 2014; Finger et al., 2007; Macoveanu, 2014; Rogers et al., 2003; Tanaka et al., 2007).

To clearly decipher the role of the serotonergic system in reward and on the dopaminergic system, there is a need to target specific circuits. This can notably be achieved through the use of recent tools including opto- and chemo-genetics allowing to specifically tag different neuronal subsets. In this review, we will first describe the evidence linking 5-HT to reward, discuss the conflicting evidence emerging on the effects of the serotonergic system over the dopaminergic system and then focus on recent studies specifically tagging the RN to VTA pathway in the context of reward.

2. The serotonergic system and reward: a complex relationship

In this section, we will describe some of the evidence linking 5-HT to reward. We will start with the “classical” involvement of 5-HT in reinforcing paradigms, specifically intracranial self-stimulation (ICSS) and conditioned place preference (CPP). In Table 1, we summarize ICSS and pharmacological studies and its effects on behavior and dopaminergic activity. We will then discuss the encoding of reward in RN neurons and recent studies specifically tagging and stimulating RN 5-HT neurons during different reward-related tasks.

Table 1. Effect of ICSS and pharmacological manipulations on behavior and neural activity.

Abbreviation: Intracranial self-stimulation: ICSS; Self-stimulation: SS; intra-peritoneal injection: i.p.; intravenous i.v; Conditioned place preference: CPP; p-chlorophenylalanine: PCPA; 8-hydroxy-2-(di-n-propylamino):8-OH-DPAT; Tryptophan hydroxylase: TPH; 5-methoxy-6methyl-2aminoindane: MMAI; itrifluoromethylphenylpiperazine: TFMPP; m-chlorophenylpiperazine: mCPP;((3-alpha-tropanyl) 1H-benzimidazolone-3-carboxamide chloride): DAU 6215; dopamine: DA; Nucleus accubems: NAc; Ventral tegmentar área: VTA; Dorsal Raphe Nuclei: DRN; Medial Raphe Nuclei: MRN; Selective serotonin reuptake inhibitors: SSRI.

Manipulation	Effect on 5-HT transmission	Drug and target	Electrode location (stimulation or recording)	Effects on behavior	Biomarker effect	References
ICSS
ICSS			DRN and MRN	↑SS		[1-6]
ICSS + Pharmacological (inhibitor of TPH)	↓5-HT	PCPA (400 mg/kg i.p [3, 4] ;400 mg/kg intragastric injection [6])	DRN and MRN	↓SS		[3, 4, 6]
ICSS + Pharmacological (inhibitor of TPH)	↓5-HT	PCPA (350 mg/kg i.p)	DRN	↑SS		[7]
ICSS + pharmacological (inhibitor of TPH)	↓5-HT	PCPA (316 mg/kg i.p.)	DRN	SS not altered		[2]
Pharmacological (GABA receptor agonist)	↓5-HT	Baclofen (0.1–2.5mM intracranial injection [8])/Muscimol (50 and 100 μM intracranial injection [9])	DRN and MRN	↑SS and Place preference	Tonic inhibition of the DA dependent reward circuitry	[8, 9]
ICSS		Downregulation of GluN2C NMDA subunit (siRNA in VTA)	DRN	↓Nose poke		[10]
ICSS + Pharmacological (5HT1A receptor agonist)	↓5-HT	8-OH-DPAT (0.2–5 ug intracranial injection) in MRN	Lateral hypothalamus stimulation	↑SS		[11]
ICSS + Pharmacological(inhibitor of TPH)	↓5-HT	PCPA (500 mg/kg i.p.)	Medial forebrain and VTA	↑SS		[12]
ICSS + Pharmacological (inhibitor of TPH)	↓5-HT	PCPA (400 mg/kg intragastric injection. [13, 14]; 500 mg/kg i.p. [12])	Lateral hypothalamus	↑SS		[12-14]
ICSS+ Pharmacological (inhibitor of TPH)	↓5-HT	PCPA intragastric (400mg/kg)	Hippocampus	↓SS		[14]
ICSS+ Pharmacological (inhibitor of TPH)	↓5-HT	PCPA intragastric (400mg/kg)	Neostriatum	↓SS		[13]
Pharmacological (5HT)	↑5-HT	5-HT (0–3.0 mg/ml intracranial injection) in ventral mesencephalon	Medial forebrain	↑SS		[15]
CPP
Pharmacological (5HT1A receptor agonist)	↓5-HT	8-OH-DPAT (125 µg/kg i.p. and 0.1, 0.5 and 1 µg intracranial injection [16]. 20 ng intracranial injection[17])	MRN and DRN	↑Place preference		[16, 17]
Pharmacological (5HT release agent)	↑5-HT	MMAI (10 and 20 mg/kg i.p.)		↓Place preference		[18]
Pharmacological (SSRI)	↑5-HT	Zimelidine (20 mg/kg i.p)		Does not modify place preference induced by morphine and does not induce place aversion		[19]
Pharmacological (SSRI followed by monoamine agent)		Zimelidine followed by d-amphetamine (5mg/kg i.p.)		Blocked place preference induced by d-amphetamine		[19]
Pharmacological (5-HT receptor antagonist and monoamine release agent)		Ritanserin (1 or 2.5 mg/kg i.p.) or Ritanserin followed by d-amphetamine (1.5mg/kg i.p.)		Ritanserin does not induce place preference but decreases place preference induced by d-amphetamine		[20]
Pharmacological (SSRI)	↑5-HT	Fluoxetine (10 and 15 mg/kg s.c.) in DA-deficient mice		↑Place preference		[21]
Impact of pharmacological manipulation of 5HT or electrical stimulation of DRN on VTA activity
Pharmacological (SSRI and Beta blocker)	↑5-HT	Paroxetine (20–1280 µg/kg i.v.), sertraline (20–1280 µg/kg i.v), and fluvoxamine (20–1280 µg/kg i.v.), and tertatolol (1 mg/kg i.v.)	VTA		↓Firing rate	[22]
Pharmacological (5HT receptor agonist)	↓5-HT	8-OH-DPAT (1.25–80 µg/kg i.v.)	VTA		↑Basal firing rate	[23]
Pharmacological (5HT release agent)	↑5-HT	TFMPP (1.25–160 µg/kg i.v.) and mCPP (1.25–320 µg /kg i.v.)	VTA		↓Firing rate	[23]
Pharmacological (5HT3 receptor antagonist )	↑5-HT	Chronic DAU 6215 (15 ug/kg s.c.)	VTA		↓Spontaneous activity of DA neurons; effect reversed by apomorphine	[24]
Pharmacological (5-HT2 receptor antagonist)	↓5-HT	Ritanserin (0.5–2.0 mg/kg i.v.)	VTA		↑Firing rate and bursting–this effect can be prevented by PCPA	[25]
Pharmacological (5HT)	↑5-HT	5-HT (2 µg intracranial injection)	VTA		↑Dopamine release in the NAc	[26]
Pharmacological (5-HT and 5-HT2 receptor agonist)	↑5-HT	5-HT (3–100 µM in vitro), ketanserin (100 µM in vitro)	VTA		Excitatory effect of 5-HT on VTA DA neuron, blocked by ketanserin	[27]
Electrical stimulation of DRN			Substantia nigra VTA		DA cells 63% initially excited followed by inhibition and 34% inhibition followed by excitation	[28]

Ref-erences.

^1.Gratton, A., Time course analysis of para-chlorophenylalanine induced suppression of self-stimulation behavior. Pharmacol Biochem Behav, 1982. 17(4): p. 597–602.

^2.Margules, D.L., Noradrenergic rather than serotonergic basis of reward in the dorsal tegmentum. J Comp Physiol Psychol, 1969. 67(1): p. 32–5.

^3.Miliaressis, E., A. Bouchard, and D.M. Jacobowitz, Strong positive reward in median raphe: specific inhibition by para-chlorophenylalanine. Brain Res, 1975. 98(1): p. 194–201.

^4.Miliaressis, E., Serotonergic basis of reward in median raphe of the rat. Pharmacol Biochem Behav, 1977. 7(2): p. 177–80.

^5.Rompre, P.P. and S. Boye, Localization of reward-relevant neurons in the pontine tegmentum: a moveable electrode mapping study. Brain Res, 1989. 496(1–2): p. 295–302.

^6.Van Der Kooy, D., H.C. Fibiger, and A.G. Phillips, An analysis of dorsal and median raphe self-stimulation: effects of parachlorophenylalanine. Pharmacol Biochem Behav, 1978. 8(4): p. 441–5.

^7.Simon, H., M. Le Moal, and B. Cardo, Intracranial self-stimulation from the dorsal raphe nucleus of the rat: effects of the injection of para-chlorophenylalanine and of alpha-methylparatyrosine. Behav Biol, 1976. 16(3): p. 353–64.

^8.Shin, R. and S. Ikemoto, The GABAB receptor agonist baclofen administered into the median and dorsal raphe nuclei is rewarding as shown by intracranial self-administration and conditioned place preference in rats. Psychopharmacology (Berl), 2010. 208(4): p. 545–54.

^9.Liu, Z.H. and S. Ikemoto, The midbrain raphe nuclei mediate primary reinforcement via GABA(A) receptors. Eur J Neurosci, 2007. 25(3): p. 735–43.

^10.Hernandez, G., et al., Glutamate NMDA receptors containing GluN2C subunit relay the reward signal of the ventral tegmental area upon dorsal raphe stimulation. bioRxiv, 2020: p. 2020.07.26.222125.

^11.Fletcher, P.J., M. Tampakeras, and J.S. Yeomans, Median raphe injections of 8-OH-DPAT lower frequency thresholds for lateral hypothalamic self-stimulation. Pharmacol Biochem Behav, 1995. 52(1): p. 65–71.

^12.Poschel, B.P. and F.W. Ninteman, Intracranial reward and the forebrain’s serotonergic mechanism: studies employing para-chlorophenylalanine and para-chloroamphetamine. Physiol Behav, 1971. 7(1): p. 39–46.

^13.Phillips, A.G., D.A. Carter, and H.C. Fibiger, Differential effects of para-chlorophenylalanine on self-stimulation in caudate-putamen and lateral hypothalamus. Psychopharmacology (Berl), 1976. 49(1): p. 23–7.

^14.van der Kooy, D., H.C. Fibiger, and A.G. Phillips, Monoamine involvement in hippocampal self-stimulation. Brain Res, 1977. 136(1): p. 119–30.

^15.Redgrave, P. and R.I. Horrell, Potentiation of central reward by localised perfusion of acetylcholine and 5-hydroxytryptamine. Nature, 1976. 262(5566): p. 305–7.

^16.Fletcher, P.J., Z.H. Ming, and G.A. Higgins, Conditioned place preference induced by microinjection of 8-OH-DPAT into the dorsal or median raphe nucleus. Psychopharmacology (Berl), 1993. 113(1): p. 31–6.

^17.Higgins, G.A., et al., Behavioural and biochemical consequences following activation of 5HT1-like and GABA receptors in the dorsal raphe nucleus of the rat. Neuropharmacology, 1988. 27(10): p. 993–1001.

^18.Marona-Lewicka, D., et al., Reinforcing effects of certain serotonin-releasing amphetamine derivatives. Pharmacol Biochem Behav, 1996. 53(1): p. 99–105.

^19.Kruszewska, A., S. Romandini, and R. Samanin, Different effects of zimelidine on the reinforcing properties of d-amphetamine and morphine on conditioned place preference in rats. Eur J Pharmacol, 1986. 125(2): p. 283–6.

^20.Nomikos, G.G. and C. Spyraki, Effects of ritanserin on the rewarding properties of d-amphetamine, morphine and diazepam revealed by conditioned place preference in rats. Pharmacol Biochem Behav, 1988. 30(4): p. 853–8.

^21.Hnasko, T.S., B.N. Sotak, and R.D. Palmiter, Cocaine-conditioned place preference by dopamine-deficient mice is mediated by serotonin. J Neurosci, 2007. 27(46): p. 12484–8.

^22.Di Mascio, M., et al., Selective serotonin reuptake inhibitors reduce the spontaneous activity of dopaminergic neurons in the ventral tegmental area. Brain Res Bull, 1998. 46(6): p. 547–54.

^23.Prisco, S., S. Pagannone, and E. Esposito, Serotonin-dopamine interaction in the rat ventral tegmental area: an electrophysiological study in vivo. J Pharmacol Exp Ther, 1994. 271(1): p. 83–90.

^24.Prisco, S., et al., Chronic treatment with DAU 6215, a new 5-HT3 receptor antagonist, causes a selective decrease in the number of spontaneously active dopaminergic neurons in the rat ventral tegmental area. Eur J Pharmacol, 1992. 214(1): p. 13–9.

^25.Ugedo, L., J. Grenhoff, and T.H. Svensson, Ritanserin, a 5-HT2 receptor antagonist, activates midbrain dopamine neurons by blocking serotonergic inhibition. Psychopharmacology (Berl), 1989. 98(1): p. 45–50.

^26.Guan, X.M. and W.J. McBride, Serotonin microinfusion into the ventral tegmental area increases accumbens dopamine release. Brain Res Bull, 1989. 23(6): p. 541–7.

^27.Pessia, M., et al., Actions of 5-hydroxytryptamine on ventral tegmental area neurons of the rat in vitro. Brain Res, 1994. 654(2): p. 324–30.

^28.Gervais, J. and C. Rouillard, Dorsal raphe stimulation differentially modulates dopaminergic neurons in the ventral tegmental area and substantia nigra. Synapse, 2000. 35(4): p. 281–91.

2.1. Reinforcing paradigms implicating the serotonergic system in reward

2.1.1. Intracranial Self-stimulation

ICSS has been widely used to determine the brain areas involved in reward function and drug addiction. In this operant paradigm, the animals perform an action (e.g., nose poke, lever press) to receive a brief intra-cerebral electrical stimulation. The electrical stimulation of a brain area involved in reward will induce higher rates of “action”. Mapping of brain areas involved in reward using this approach had a massive bump in the seventies. Notably, stimulation of the VTA and the lateral hypothalamus stimulations was found to be reinforcing (Miliaressis and Cardo, 1973). Many studies demonstrated the critical importance of the dopaminergic system in reward function through this paradigm (Phillips and Fibiger, 1978; Routtenberg and Malsbury, 1969).

Regarding the serotonergic system, several studies did demonstrate increased rates of self-stimulation (SS) with electrodes located in the DRN and the MRN (Gratton, 1982; Margules, 1969; Miliaressis, 1977; Miliaressis et al., 1975; Rompre and Boye, 1989; Van Der Kooy et al., 1978). However, there were some discrepancies regarding the neurochemical basis of this effect. Margules (1969) observed that SS was increased by D-amphetamine (reuptake inhibition increasing DA and Noradrenaline), blocked by chlorpromazine (DA₂ receptor antagonist), and not altered by the administration of para-chlorophenylalanine (PCPA; potent tryptophan hydroxylase inhibitor), leading the author to conclude that SS from the DRN was due to the stimulation of noradrenergic fibers. However, high rates of SS were later reported with electrodes located in the MRN or DRN and this effect was decreased following PCPA, suggesting the involvement of 5-HT (Miliaressis, 1977; Miliaressis et al., 1975; Van Der Kooy et al., 1978). On the contrary, Simon et al. (1976) reported that PCPA increased the SS rate with electrodes located in the DRN. This result aligns with experiments showing that muscimol infusion (GABAa agonist) in the MRN and DRN induces higher rates of lever press, suggesting that the RN neurons act on the reward circuit through a tonic-inhibition of the DA-dependent reward circuit (Liu and Ikemoto, 2007).

To conclude, several bodies of evidence point to higher rates of RN SS, however the neurochemical basis of this effect seems to be complex. The complexity and contradictory effects are even more prominent when looking at 5-HT implication in brain-stimulation paradigms. For example, depletion of brain 5-HT, using PCPA (intragastric route), with electrodes located in the lateral hypothalamus increases the rate of SS (Phillips et al., 1976; Poschel and Ninteman, 1971; van der Kooy et al., 1977). Similarly, another drug reducing the 5-HT level, 5-HT1A agonist 8-hydroxy-2-(di-n-propylamino) tetralin injection in the MRN, increases lateral hypothalamus stimulation (Fletcher et al., 1995). This increase was also found with electrodes located in the medial forebrain bundle and VTA (Gibson et al., 1970; Poschel and Ninteman, 1971). These results suggest that a decrease of 5-HT potentiates sensitivity to rewarding stimuli. However, the effect is not ubiquitous as it was found that PCPA decreases SS with electrodes located in the hippocampal formation (van der Kooy et al., 1977) and the neostriatum (Phillips et al., 1976). In addition, local infusion of 5-HT in the ventral midbrain increases SS with electrodes located in the medial forebrain bundle (Redgrave and Horrell, 1976).

2.1.2. Conditioned place preference

Another behavioral paradigm testing the reward function is CPP. CPP is a behavior combining Pavlovian learning and motivation, where an animal is placed in a box with two compartments. An unconditioned stimulus (e.g. drug, stimulation/inhibition of a brain area) is associated with only one of the compartments. The amount of time spent in each compartment is measured to assess preference. The rewarding or aversive value of the unconditioned stimulus is then assessed as a function of preference. Different authors have tested either the value of 5-HT as an unconditioned stimulus or how 5-HT was able to modulate CPP induced by other drugs.

Fletcher et al. (1993) injected the 8-hydroxy-2-(di-n-propylamino) tetralin in the MRN and DRN, leading to a decrease of 5-HT transmission. They demonstrated a preference for the compartment where the animals were injected with this selective agonist, and the effect was more robust when the drug was injected in the DRN compared to the MRN. CPP was also observed when injecting baclofen, a GABAb receptor agonist leading to an inhibition of 5-HT neurons, in the DRN and MRN, or muscimol in the MRN (Liu and Ikemoto, 2007; Shin and Ikemoto, 2010). These reports, as well as others (Higgins et al., 1988; Invernizzi et al., 1991) lead to the view that decreasing 5-HT activity facilitates the reward circuitry. In this line of evidence, Marona-Lewicka et al. (1996) showed that the selective 5-HT releasing agent, 5-methoxy-6methyl-2aminoindan, induces place aversion in the CPP test. However, and again, the picture is not so simple. Indeed, CPP induced with amphetamine was blocked by a pretreatment with a selective 5-HT uptake blocker [zimelidine- (Kruszewska et al., 1986)] but was also blocked with a treatment decreasing 5-HT transmission [ritanserin, (Nomikos and Spyraki, 1988)]. In addition, it was shown that fluoxetine, increasing the level of 5-HT, can induce CPP in DA-deficient mice (Hnasko et al., 2007).

Why is it so difficult to get a unified picture on the role of the serotonergic system in reward functions? The broad distribution of serotonergic fibers in the brain, the heterogenous properties and distribution of 5-HT receptors, and therefore, the electrode location, the specificity, dose and route of administration of the pharmacological agents used are some of the factors that can account for the contradictory results observed in the literature (Kranz et al., 2010; Muller and Homberg, 2015; Van Der Kooy et al., 1978). Importantly, these inconsistent results are also observed in human studies, notably using acute tryptophan depletion in reward-related tasks. Indeed, acute tryptophan depletion has been shown to either leave reward processes unaltered [e.g. reward prediction; choice between immediate and delayed reward; (Cools et al., 2008; Crean et al., 2002; Tanaka et al., 2007)] or to have an impact on reward processes such as discrimination and response to different magnitudes of expected rewards or delayed reward discounting (Aquili, 2020; Cools et al., 2005; Faulkner and Deakin, 2014; Finger et al., 2007; Rogers et al., 2003; Schweighofer et al., 2008). In addition, the administration of anti-depressants has been shown to have distinct effects on brain regions responding to reward (Abler et al., 2011; Del-Ben et al., 2005; Macoveanu, 2014; Macoveanu et al., 2013; Marutani et al., 2011; McCabe et al., 2010; Vollm et al., 2006). For example, the ventral striatal response to an appetitive reward is decreased following citalopram administration (McCabe et al., 2010) while striatal response in a monetary reward task was increased following two-week administration of duloxetine (Ossewaarde et al., 2011).

It clearly appears from these studies that the serotonergic system is involved in reward processing however, there is a need to specifically tag 5-HT neurons (leaving the other neuromodulatory system such as noradrenergic fibers unaffected), as well as specific pathways. To go further in the understanding of the contribution of 5-HT neurons in reward processing, we will next review the encoding of reward in putative and genetically identified 5-HT neurons.

2.2. Encoding of reward processing in the DRN

The encoding of reward has been well described in the dopaminergic system, especially in the VTA (Cohen et al., 2012; Schultz, 1997; Schultz et al., 1997). In humans, the DRN is activated during a reward-related task using fMRI, specifically when participants learn to obtain large future rewards while sustaining small immediate losses (Tanaka et al., 2004). DRN neurons were also recorded in awake animals performing reward-related tasks. Most of the studies classified putative DRN 5-HT neurons based on their electrophysiological properties (Aghajanian and Vandermaelen, 1982). However, these criteria might be insufficient to clearly identify 5-HT neurons as demonstrated by juxtacellular experiments (Allers and Sharp, 2003; Hajos et al., 2007; Kocsis et al., 2006). We will thus also highlight recent evidence obtained with recordings of genetically identified 5-HT neurons during reward-related tasks.

One of the first studies evaluating the activity of DRN neurons in a reward-related task was performed in monkeys (Nakamura et al., 2008). This study, using saccade tasks with a biased reward schedule, revealed that putative DRN 5-HT neuron activity is modulated during the delay period (before reward) but also after reward onset. In rats performing a two-alternative odor discrimination task, DRN neurons are also significantly modulated during the task with a majority responding during the reward phase of the task (Ranade and Mainen, 2009). This effect was further confirmed in studies recording genetically identified 5-HT neurons (Cohen et al., 2015; Li et al., 2016; Liu et al., 2014; Ren et al., 2018). The common features of all these reports were that DRN neuron activities are diverse and modulated during reward-related tasks with their activity locked to different events (e.g., nose poke, moving to a reward port, the different reward-related periods).

Response to reward-predicting cues:

Responses to reward-predicting cues were observed in a subset of 5-HT neurons and correlated to the stimulus value/expected reward size (Cohen et al., 2015; Hayashi et al., 2015; Li et al., 2016; Liu et al., 2014; Matias et al., 2017). In this line of results, it was also shown, in humans with acute tryptophan depletion, an altered ability in providing adaptive response to incentive-motivational cues signaling reinforcement certainty (Cools et al., 2005).

Delay period:

During the delay period, most 5-HT neurons increase their firing rate (Li et al., 2016; Miyazaki et al., 2011a). In a sequential food-water navigation task in rats, 5-HT firing increases during waiting for delayed reward (Miyazaki et al., 2011a; Miyazaki et al., 2018). Confirming the contribution of 5-HT neurons during the delay period, Zhong et al. (2017) demonstrated a ramp-up activity pattern in response to the conditioning stimulus during the delay to reward period and increasing across learning. This result paralleled the result obtained by DRN recordings showing tonic spike firing during the delay period (Bromberg-Martin et al., 2010; Li et al., 2016; Liu et al., 2014; Nakamura et al., 2008). These observations lead some of the authors to conclude that putative 5-HT neurons were involved in waiting (Miyazaki et al., 2011a; Miyazaki et al., 2011b). Interestingly, in humans it was found that dietary tryptophan depletion increases the delayed reward discounting with a preference for small immediate rewards rather than larger but delayed rewards (Schweighofer et al., 2008).

Response to reward:

Several studies have observed a response of DRN neurons to the reward itself (Bromberg-Martin et al., 2010; Hayashi et al., 2015; Inaba et al., 2013; Nakamura et al., 2008). Interestingly, Li et al. (2016) demonstrated that DRN 5-HT neurons display a change of activity in response to natural/ecological rewards: when the mouse looks for and acquires sucrose, food, sex, and social interaction. This finding thus extends the previous results to different qualities of reward.

Importantly, activity of DRN neurons is modulated as a function of the received reward magnitude, and these neurons can keep track of the reward value with an activity dependent on reward probability (Bromberg-Martin et al., 2010; Hayashi et al., 2015; Inaba et al., 2013; Nakamura et al., 2008). This effect was further confirmed with single-unit/fiberphotometry recordings of identified DRN 5-HT neurons (Li et al., 2016). Finally, Zhong et al. (2017) demonstrated that 5-HT activity tracks reward availability and it has also been proposed that DRN neurons have a role in evaluating future reward and tracking progress toward future delayed reward (Bromberg-Martin et al., 2010; Inaba et al., 2013). Indeed, Bromberg-Martin et al. (2010) showed that most of DRN neurons have their firing activity at task onset correlated with their response to the reward cues and outcomes. Paralleling the encoding of reward magnitudes in the DRN, it was shown in humans, with dietary tryptophan depletion, a diminished discrimination between different magnitudes of expected rewards (Rogers et al., 2003). In addition, in humans, tryptophan depletion alters the representation of reward outcome value (Seymour et al., 2012) and impairs response to rewarded stimuli (Finger et al., 2007).

Activity in the absence of reward:

It was observed by Ranade and Mainen (2009), using reward omission, that a very small population of DRN neurons (4/32) changed their firing rate around the expected time of reward, however, this modulation of DRN activity when the reward was omitted was not found by (Miyazaki et al., 2011a). This difference might be ascribable to the small percentage of units displaying a significant change of activity in the absence of reward and that this subpopulation was not recorded in Miyazaki et al. (2011a). This discrepancy was still present when recording genetically identified 5-HT neurons. Zhong et al. (2017) demonstrated that reward omission led to a progressive decrease of 5-HT neuron response to reward predicting cues. However, Matias et al. (2017) observed increased 5-HT activity when a fully predicted reward was omitted. Indeed, using a reversal, a more sudden alteration of the cue-outcome, they showed that 5-HT neurons increase activity to positive and negative reward prediction error which corresponds to unsigned prediction error (Matias et al., 2017).

Encoding of aversive stimulus- Timescale view:

Different studies found phasic response to aversive events during a conditioning task (Cohen et al., 2015; Matias et al., 2017), however, this response to aversive events was not observed by Li et al. (2016). Nonetheless, an interesting, and perhaps key, point raised by the study of Cohen et al. (2015) is that there are different timescales of 5-HT response during the reward/punishment conditioning task, with a tonic firing variation depending on state value (long-lasting), and a phasic/transient activity in response to reward-predicting cues, reward or punishment. This important notion of timescale was also found by Hayashi et al. (2015) demonstrating temporal firing rate variation between appetitive and aversive contexts. Notably, they found that 5-HT neurons encode the aversive blocks via a tonic activity but rather on a shorter time scale during the appetitive blocks. These results are paralleled with the ones performed in humans where it was suggested and then demonstated that 5-HT is involved in reward prediction at different timescales (Doya, 2002; Kranz et al., 2010; Tanaka et al., 2007). Indeed, Tanaka et al. (2007) demonstrated that 5-HT, through dietary manipulation of tryptophan level, differentially controls the time scale of reward prediction within the striatum.

To conclude, all these studies point to the role of DRN 5-HT neurons in reward, however, there is a lot of diversity in neuronal response. A critical point highlighted by those different studies is the importance of the timescale and the possibility of separate DRN circuits and pathways for different reward functions (McDannald, 2015). Specifically, McDevitt et al. (2014) revealed different patterns of projection of DRN 5-HT neurons vs. DRN non-5-HT neurons. In addition, in mice trained to lever press for a sucrose solution, Ren et al. (2018) elegantly demonstrated that 5-HT neurons projecting to the orbitofrontal cortex and 5-HT neurons projecting to the central amygdala have similar responses to reward (activation after the onset of licking), however, their activity differs prior to lever press and they have opposite response to foot shock (inhibition of 5-HT neurons projecting to OFC and activation of 5-HT neurons projecting to central amygdala). Therefore, these results highlight the importance of not only recording identified 5-HT neurons, but also recordings from projection-specific populations (see section 3.3).

To gain insight into the functions of DRN 5-HT neurons in reward, the next section will focus on the consequences of activation of genetically-tagged 5-HT neurons during reward-related tasks.

2.3. Specific DRN 5-HT neurons stimulations: to be or not to be rewarding?

The development of tools including opto- and chemo-genetics allows to specifically tag different neuronal subsets, to test the role of DRN 5-HT neurons in reward-related tasks and to have good temporal resolution. In Table 2, we summarize recent studies using optogenetic stimulation to assess the interaction between the serotonergic system, reward and the dopaminergic system.

Table 2. Effect of optostimulation on behavioral expression and neural modulation.

Abbreviation: Self-stimulation: SS; excitatory postsynaptic potential: EPSP; dopamine: DA; Nucleus accubems: NAc; Ventral tegmental area: VTA; Dorsal Raphe Nuclei: DRN;

Opto or chemo-genetic stimulation (neuronal population targeted)	Location of optical fiber	Effects on reward-related behaviors	Biomarker effect	References
5-HT neurons (Tph2-tTA promoter[1, 2]; SERT promoter [3]; Pet-1 [4])	DRN	NOT reinforcing but Enhances patience for future rewards		[1-4]
5-HT neurons (Pet1 promoter)	DRN	↑SS	Firing activity during reward mediated by 5-HT and glutamate	[5]
5-HT neurons (Tph2 promoter)	DRN	↑SS and Place preference		[6]
5-HT neurons (Pet1 promoter)	DRN	↓Place Preference		[7]
5-HT neurons projection to VTA (Pet1 promoter) citalopram (5 or 10 mg/kg i.p.).	VTA	↓Operant response		[8]
DRN glutamatergic projection to VTA (Vglut3 promoter)	VTA	↑Operant response		[4, 9]
5-HT neurons (SERT promoter) and glutamatergic projections to VTA (Vglut3 promoter)	VTA	↑Place preference	Release of dopamine in the NAc mediated through AMPA and 5-HT3 receptors	[10]
DRN 5-HT projection to VTA (Pet1 promoter)	VTA	↑Locomotion	EPSPs on DA neurons blocked by AMPA receptor antagonist	[7]
DRN 5-HT projecting to amygdala (SERT promoter)		Anxiety like behavior	Activation by reward and punishment	[11]
DRN 5-HT projecting to frontal cortex (SERT promoter)		Increase active coping in face of challenge	Activation during reward and inhibited by punishment	[11]

Ref-erences.

^1.Miyazaki, K., et al., Reward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience. Nat Commun, 2018. 9(1): p. 2048.

^2.Miyazaki, K.W., et al., Optogenetic activation of dorsal raphe serotonin neurons enhances patience for future rewards. Curr Biol, 2014. 24(17): p. 2033–40.

^3.Fonseca, M.S., M. Murakami, and Z.F. Mainen, Activation of dorsal raphe serotonergic neurons promotes waiting but is not reinforcing. Curr Biol, 2015. 25(3): p. 306–315.

^4.McDevitt, R.A., et al., Serotonergic versus nonserotonergic dorsal raphe projection neurons: differential participation in reward circuitry. Cell Rep, 2014. 8(6): p. 1857–1869.

^5.Liu, Z., et al., Dorsal raphe neurons signal reward through 5-HT and glutamate. Neuron, 2014. 81(6): p. 1360–74.

^6.Nagai, Y., et al., The Role of Dorsal Raphe Serotonin Neurons in the Balance between Reward and Aversion. Int J Mol Sci, 2020. 21(6).

^7.Cunha, C., et al., Perinatal interference with the serotonergic system affects VTA function in the adult via glutamate co-transmission. Mol Psychiatry, 2020.

^8.Browne, C.J., et al., Dorsal raphe serotonin neurons inhibit operant responding for reward via inputs to the ventral tegmental area but not the nucleus accumbens: evidence from studies combining optogenetic stimulation and serotonin reuptake inhibition. Neuropsychopharmacology, 2019. 44(4): p. 793–804.

^9.Qi, J., et al., A glutamatergic reward input from the dorsal raphe to ventral tegmental area dopamine neurons. Nat Commun, 2014. 5: p. 5390.

^10.Wang, H.-L., et al., Dorsal Raphe Dual Serotonin-Glutamate Neurons Drive Reward by Establishing Excitatory Synapses on VTA Mesoaccumbens Dopamine Neurons. Cell Reports, 2019. 26(5): p. 1128–1142.e7.

^11.Ren, J., et al., Anatomically Defined and Functionally Distinct Dorsal Raphe Serotonin Sub-systems. Cell, 2018. 175(2): p. 472–487 e20.

Liu et al. (2014) demonstrated that photo-stimulation of DRN 5-HT neurons induces operant reinforcement for self-administration and, preference for a place or a less-preferred drink (with Channelrhodopsin 2 being expressed under the control of the Pet-1 promoter). This evidence points to DRN 5-HT neurons involvement in reinforcing paradigms. This reinforcing effect was confirmed in another study using the tryptophan hydroxylase 2 promoter, DRN 5-HT neuron stimulation induced self-stimulation and conditioned place preference (Nagai et al., 2020). In addition, Cunha et al. (2020) also confirmed that stimulation of RN 5-HT neurons can elicit place preference and hyperlocomotion similar to what was obtained when stimulating VTA DA neurons. Importantly, Liu et al. (2014) demonstrated that 5-HT neurons also release glutamate. Using different knock-out mice combined with optogenetics and pharmacological agents, they showed that both 5-HT and glutamate are critical for reward signaling in DRN 5-HT neurons. This study strongly supports RN serotonergic neural activity as a key signal for reward. At the same time and using different paradigms as well as targeting the 5-HT population with a different promoter (tryptophan hydroxylase 2 promoter), it was shown that activating 5-HT neurons rather than being rewarding by itself, promotes waiting for future rewards (Miyazaki et al., 2018; Miyazaki et al., 2014). Notably, the authors used a sequential tone-food waiting task, and showed that DRN-5-HT photostimulation 1) is not rewarding as it does not reinforce spontaneous nose poking, 2) decreases the tone wait errors while the mice are waiting for the tone, 3) decreases the reward wait errors while the mice are waiting during the variable delay period for a reward, this is specifically true for high expectation reward (75% vs 25% or 50%) and when the timing of the reward is more uncertain and 4) increases the waiting time when 5-HT neurons were activated during the reward omission trials. To disentangle the possible contribution of DRN 5-HT neurons as being rewarding and/or being involved in waiting (i.e. patience), Fonseca et al. (2015) activated the DRN 5-HT (using the SERT promoter) in both a waiting-task as in Miyazaki et al. (2014), in a CPP, a real-time place preference, and in a probabilistic choice task similar to the reinforcing tasks used by Liu et al. (2014). They demonstrated that activation of DRN 5-HT neurons 1) increases the waiting time in a delayed response task as Miyazaki et al. (2018); Miyazaki et al. (2014), and 2) does not induce CPP, real-time place preference nor choice bias in their choice task, disagreeing with the results obtained by Liu et al. (2014). In the same line of evidence, McDevitt et al. (2014) demonstrated that photostimulation of DRN 5-HT neurons (using the Pet1 promotor) was not reinforcing in an operant task. Nevertheless, they found that activating DRN non-serotonergic cells (with a combination of knockout/stimulation protocol) reinforces operant task and real-time place preference. Finally, it was demonstrated that DRN 5-HT photostimulation alone was not sufficient to impact in any direction the operant response to saccharine. However, the association of low dose of citalopram to DRN 5-HT stimulation significantly reduced the operant response to saccharine (Browne et al., 2019).

As suggested by Ranade et al. (2014) as well as Fonseca et al. (2015), the discrepancy of 5-HT neurons stimulation as being rewarding or not might be ascribable to the behavioral tests used, different signaling modes of DRN 5-HT neurons (tonic vs. phasic) and importantly to different population of 5-HT targeted (different promoters and AAV subtype used…). As highlighted by Ren et al. (2018) projection-specific populations (e.g. DRN 5-HT to orbitofrontal cortex versus DRN-5-HT to amygdala) may have distinct behavioral functions (table 2). In the rest of the review, we will focus on a specific pathway, namely the RN to VTA to disentangle the effect of the serotonergic system in reward function.

3. The serotonergic system and the dopaminergic system : a love/hate relationship

The VTA is an integral piece of the reward puzzle. Its contribution to reward processing has been well described using single-unit recordings and pharmacological approach (Morales and Margolis, 2017; Schultz, 1997; Schultz et al., 1997; Wise, 2004). Optogenetics manipulations have also revealed that phasic photoactivation of VTA DA neurons promotes place preference in the CPP test and positive reinforcement (Kim et al., 2012; Tsai et al., 2009).

The anatomical connection between the RN and the VTA, the impact of RN on VTA DA activity as well as the specific stimulation of the RN to VTA pathway will be discussed in this section.

3.1. Anatomical relationship between the RN AND VTA

It has been demonstrated that the RN project massively to the VTA (Herve et al., 1987; Miller et al., 1975; Vertes, 1991; Watabe-Uchida et al., 2012). Both the MRN and DRN project to the VTA and substantia nigra but the DRN projections are more intense than the MRN (Watabe-Uchida et al., 2012).

Importantly, RN to VTA projecting neurons have been found to release not only 5-HT, but also glutamate. In fact, different populations have been identified that transmit 5-HT only, glutamate-only or co-transmit 5-HT and glutamate (Qi et al., 2014; Wang et al., 2019); possibly having different implications in behavioral expression.

Critically, there are different 5-HT receptors present in the VTA and their expression depends on the neuronal type (i.e. DA vs non-DA neurons; for extensive reviews, see (De Deurwaerdere and Di Giovanni, 2017; Hayes and Greenshaw, 2011)). 5-HT1A, 5-HT2A, 5-HT2C and 5-HT3 are mostly expressed in DA neurons while 5HT1A; 5-HT1B; 5-HT2A; 5HT2C; 5HT4–3; 5HT 6 are expressed on non-DA cells, presumably GABAergic cells in the VTA. Interestingly, the expression of either different receptors in DA and non DA-cells or the expression of 5-HT2A receptors on both DA and non-DA cells offer various way of regulation of the serotonergic system over the VTA activity (Doherty and Pickel, 2000).

The RN also receive, although to a lesser extent, projections back from the dopaminergic system: DRN and MRN 5-HT neurons receive projections from the substantia nigra, the VTA as well as the retrorubral field (Kalen et al., 1988) (Ogawa et al., 2014; Vertes and Linley, 2008). Ogawa et al. (2014) demonstrated that VTA and substantia nigra project strongly to DRN 5-HT neurons. Interestingly, the majority of VTA neurons projecting to DRN 5-HT neurons are non-dopaminergic. MRN 5-HT neurons also receive input from the VTA but much more moderate inputs from substantia nigra. It is interesting to note that DRN 5-HT neurons and VTA DA neurons share many monosynaptic inputs, especially from the hypothalamus, amygdala, and basal ganglia (Beier et al., 2015; Ogawa et al., 2014; Watabe-Uchida et al., 2012).

To conclude, hardwire between the serotonergic system and the dopaminergic system is heavy, specifically there are intense projections from the DRN to the VTA and back and those two areas share many similar inputs from a wide variety of brain regions. Therefore, understanding this specific pathway is critical to gain insight in reward-related functions given the major role of VTA DA in reward.

Importantly, a recent work, using rabies virus-based and cell-type-specific monosynaptic tracing techniques, in addition to whole brain trans-synaptic tracing experiments was able to reveal the input-output relationship of VTA DA neurons (Beier et al., 2015). This study has uncovered a strong disynaptic circuit linking the DRN, VTA, and NAc with DRN input to NAc projecting VTA DA neurons. This is particularly interesting in the context of reward and drug addiction, reviewed in (Day and Carelli, 2007).

3.2. Suppressing or activating? Impact of DRN 5-HT neurons on the VTA dopaminergic system

It has long been known both in-vitro and in-vivo that 5-HT-like drugs can modulate activity of dopaminergic neurons and the concentration of DOPAC in the VTA (Beart and McDonald, 1982). This supports the mechanism by which anti-depressants can modulate mesolimbic dopaminergic circuits in humans (McCabe et al., 2010; Ossewaarde et al., 2011).

The effect of 5-HT on DA neuronal activity seems to be complex, depending on the type of the receptors targeted and the specificities of administration of the pharmacological agent (route, localization, acute or chronic). De Deurwaerdere and Di Giovanni (2017) thoroughly reviewed the effects of different 5-HT modulation (raphe lesion; receptor agonist or antagonist) on DA neuronal activity [for a full list see table 2 of (De Deurwaerdere and Di Giovanni, 2017)].

Along with the longstanding view that increasing 5-HT level inhibits reward circuits, it was shown that various selective 5-HT reuptake inhibitors induce a significant decrease of the basal firing of VTA DA cells (Di Mascio et al., 1998; Prisco et al., 1994; Prisco et al., 1992), leading to the interpretation that VTA DA neurons are under the tonic inhibitory control of DRN 5-HT terminals (Prisco et al., 1994). Moreover, in-vivo systemic injection of 5-HT1a receptor agonist induces an increase of VTA neuron firing rate, and specifically their bursting, but this was not observed when the agonist was infused directly in the VTA. In addition, it was shown that ritanserin (5-HT2 receptor antagonist, systemic administration) increases the firing rate and the bursting of VTA and substantia nigra DA neurons (Ugedo et al., 1989). Congruent with this observation, it was shown that lesions of the DRN AND MRN increase the dopaminergic metabolites (DOPAC and HVA) but the level of DA level remained unchanged in the striatum (Juorio and Greenshaw, 1986). Consistently, Morelli et al. (2011) demonstrated in a model of mice with chronic 5-HT transporter blockade that DA signaling (DOPAC and HVA metabolite level) is decreased in the nigrostriatal system.

However, once again there are ambiguities (table 1). Indeed, it was shown by Guan and McBride (1989) that infusion of 5-HT directly in the VTA increases the DA release in the NAc via 5-HT1B receptors leading the authors to conclude that 5-HT innervations in the VTA have an excitatory action. An excitatory effect of 5-HT (highly concentrated) on VTA DA neurons was also found in vitro (Pessia et al., 1994). The authors showed that 5-HT depolarized almost half of the DA cells recorded in vitro. This depolarization was blocked by ketanserin and reproduced by local application of a 5-HT2 receptor agonist, meaning that the increase of DA neuron firing rate was mostly mediated through 5-HT2 receptors.

There is an interesting alternative route by which 5-HT can regulate DA activity - via modulation of GABAergic cells - adding another layer of complexity to the DA-5-HT interaction. As mentioned earlier, 5-HT2 receptors are expressed in the VTA in both DA and GABAergic neurons (Doherty and Pickel, 2000; Nocjar et al., 2002). These receptors have been shown to mediate a modest depolarization leading to increased DA activity and DA release (Pessia et al., 1994). However, 5-HT2A/C receptor activation in substantia nigra GABAergic neurons may indirectly inhibit DA neuron activity (Demireva et al., 2018; Esposito, 2006; Tepper et al., 1995).

It is clear that 5-HT impacts VTA DA activity, however, the effects are complex (De Deurwaerdere and Di Giovanni, 2017; De Deurwaerdère and Di Giovanni, 2020). Another report in-vivo in anesthetized rats notably highlighted this complexity. Gervais and Rouillard (2000) electrically stimulated the DRN while recording extracellular single unit activity of DA neurons in the substantia nigra and VTA, and they revealed a complex response to DRN stimulation. They showed two populations of VTA DA cells; 63% that were initially excited followed by an inhibition, and 34% displayed an inhibition-excitation response. Importantly, this former inhibition-excitation response was abolished by PCPA but not the excitation-inhibition responses. This suggests that the DRN-5-HT inputs to VTA DA neurons are mainly inhibitory but also that some DRN non-5-HT inputs can impact the activity of VTA DA cells. To go further in the understanding of the impact of DRN 5-HT neurons and non-5-HT neurons on VTA, it is necessary to specifically tag each cellular type and record VTA DA neuron activity.

3.3. Activation of the DRN to VTA pathway in reinforcing paradigms: contribution of serotonin and glutamate

As described in section 2.3, whether or not the stimulation of DRN 5-HT neurons was reinforcing is still under debate with some studies showing no reinforcing effect (Fonseca et al., 2015; McDevitt et al., 2014; Miyazaki et al., 2018; Miyazaki et al., 2014) and others demonstrating a reinforcing effect (Cunha et al., 2020; Liu et al., 2014; Nagai et al., 2020; Wang et al., 2019).

To refine the analysis of the role of DRN 5-HT neurons in reward functions, the DRN 5-HT neurons to VTA DA pathway can be specifically tagged and modulated (see table 2). Consistent with previous studies showing either that DRN 5-HT stimulation was not reinforcing or that DRN stimulation reinforcing effect was not 5-HT dependent (Fonseca et al., 2015; Miyazaki et al., 2014; Simon et al., 1976), it was shown that photo-stimulation of DRN 5-HT neuron projections to VTA when associated to a low dose of citalopram decreases operant response to saccharine, similar to what the authors found when stimulating 5-HT neurons in the DRN (Browne et al., 2019). Therefore, this report shows that DRN 5-HT projections to VTA have an inhibitory effect. In addition, McDevitt et al. (2014) did not observe that DRN 5-HT fibers to VTA stimulation was reinforcing. However, Nagai et al. (2020) found the opposite effect. Indeed, they showed that activation of DRN 5-HT neurons to VTA pathway increases self-stimulation behavior and conditioned place preference. In addition, these authors also showed that inhibiting the DRN 5-HT neurons to VTA pathway induced conditioned place aversion. The authors conclude that DRN 5-HT to VTA pathway is thus a key element in the balance between reward and aversion. Interestingly, Cunha et al. (2020) demonstrated that optogenetic stimulation of 5-HT terminals in the VTA results in place preference and hyperlocomotion similar to what was obtained when directly stimulating the VTA-DA neurons.

These discrepancies might come from the fact that the DRN is composed of different cellular types: serotonergic, GABAergic, as well as non-serotonergic glutamatergic neurons projecting to the VTA. In addition, it has also been demonstrated that DRN 5-HT neurons, notably projecting to the VTA, can also co-express vesicular glutamate transporter type 3 (Amilhon et al., 2010; Gras et al., 2002; Ren et al., 2018). Importantly, McDevitt et al. (2014) found that while DRN 5-HT fibers to VTA stimulation was not reinforcing, stimulating DRN non-serotonergic glutamatergic projections to the VTA was. They also showed that stimulation of this non-serotonergic DRN to VTA pathway induces monosynaptic glutamatergic currents and action potentials in DA cells. Congruently, Qi et al. (2014) also demonstrated DRN glutamatergic input to VTA and the expression of VGlut3. They showed that stimulation of the DRN VGLUT3 to VTA pathway induces preference to the laser associated chamber, AMPA-mediated excitatory currents in VTA DA neurons and release of DA in NAc. Recently, an NMDA component in the DRN to VTA connection has also been proposed. Indeed, it was shown that a downregulation of GluN2C NMDA subunit (siRNA injection in the VTA) leads to a reduction of nose pokes in rats with stimulating electrodes implanted in the DRN (Hernandez et al., 2020).

Altogether, these studies let McDannald (2015) to propose that DRN serotonergic cells were involved in patience while the glutamatergic DRN to VTA pathway was mediating reward (Fonseca et al., 2015; McDevitt et al., 2014; Miyazaki et al., 2018; Miyazaki et al., 2014). But, as always with the serotonergic system, this conclusion must be put in perspective with a recent cohort of studies.

Different groups have recently demonstrated that stimulation of 5-HT cells can be reinforcing and that this effect is mediated through activation of VTA-DA cells via glutamatergic co-transmission (Cunha et al., 2020; Liu et al., 2014; Luo et al., 2015; Wang et al., 2019). Indeed, it was demonstrated that DRN neurons express VGlut3 and that both 5-HT only and 5-HT-glutamatergic neurons from DRN project to VTA neurons (Cunha et al., 2020; Hioki et al., 2010; Liu et al., 2014; Wang et al., 2019). Paralleling these results, a co-transmission of 5-HT and glutamate in the basal amygdala and orbitofrontal cortex following 5-HT fibers stimulation was demonstrated (Ren et al., 2018; Sengupta et al., 2017).

In-vitro recordings of VTA DA neurons combined with specific stimulation of DRN-5-HT neurons revealed that these neurons release 5-HT and glutamate on VTA DA neurons (Liu et al., 2014). Optical stimulation of RN 5-HT fibers can evoke EPSP in VTA DA neurons that are blocked by AMPA receptor antagonists (Cunha et al., 2020). In addition, DRN 5-HT-glutamatergic neurons make excitatory synapses on VTA DA neurons targeting the NAc, and this effect is mediated through both AMPA and 5-HT3 receptors (Wang et al., 2019). Moreover, stimulation of 5-HT fibers in the VTA led to a release of DA in the NAc mediated through AMPA and 5-HT3 receptors (Wang et al., 2019). The results from these three laboratories lead to the conclusion that DRN 5-HT projections to the VTA can be excitatory through a co-transmission of glutamate and led to a release of DA in the NAc. This mechanistic and hardwire evidence explains the results obtained during reinforcing paradigms. Indeed, activation of DRN-5-HT fibers in the VTA led to a strong reinforcement effect in place preference mediated through AMPA and 5-HT3 receptors (Cunha et al., 2020; Liu et al., 2014; Wang et al., 2019). Finally, Wang et al. (2019) made a clever comparison by stimulating either the DR SERT+ fibers, which can co-transmit 5-HT and glutamate, or the DR glutamatergic (VGlut3 - only) fibers to the VTA. They showed that both induce CPP, but the effect is weaker but lasts longer when stimulating SERT+ fibers compared to DR VGLUT3-only fibers. This is particularly interesting in the light of different timescales encoding found in DRN 5-HT neurons in rodents (Cohen et al., 2015; Hayashi et al., 2015) as well as the involvement of 5-HT, in humans, in the control of the time scale of reward prediction within different subparts of the striatum (Tanaka et al., 2007).

The different results obtained by the stimulation of the DRN to VTA pathway and whether the reinforcing effect is mediated through DRN glutamatergic only or DRN serotonergic and glutamatergic co-transmission are probably not exclusive from each other and are probably dependent on the context, the tasks used, their timeline, the cellular type tagged as well as the network of cells recruited during each specific task. In our opinion, the critical point from all these studies is that stimulation of DRN to VTA inputs can be rewarding, either by 5-HT via 5-HT3 receptor or/and glutamate (Figure 1).

Figure 1:

RN to VTA projections can be serotonergic, glutamatergic or can co-transmit glutamate and 5-HT. Serotonergic release via 5-HT3 receptors, glutamatergic transmission (Glut) and glutamatergic-serotonergic co-transmission have been associated with reward in operant (either by nose poke or lever pressing) and place preference tasks and increased DA release into the nucleus accumbens. 5-HT3R: activation via 5-HT3 receptors, GlutR: activation via glutamatergic receptors. These papers show effects on behavior of activation of the following pathways: (1) DRNSerotonergic only : Nagai et al., 2020; Wang et al., 2020 ; (2) DRN-Glutamatergicserotonergic co-transmission: Liu et al., 2014; Wang et al., 2020; Cunha et al., 2020; (3) DRN-glutamatergic only: McDevitt et al., 2014; Qi et al., 2014.

Finally, in addition to the routes or neurotransmitters by which RN can modulate DA activity, a loop can also take place between the RN and the VTA. Indeed, the VTA projects to the DRN (Ogawa et al., 2014) and this circuit, as well as the feedback activity it may convey, can also be critical for reward-related functions. Recently, it was found that VTA inputs to the DRN were primarily GABAergic. Importantly, there were spatially segregated population-projections with the rostral VTA projecting to DRN GABAergic neurons and the caudal VTA projecting to DRN 5-HT neurons (Li et al., 2019). The functions of those two sub-circuits are distinct, with the rostral VTA GABAergic projection to DRN GABAergic neurons leading to a disinhibition of 5-HT neurons, while the caudal VTA GABAergic neurons led to an inhibition of 5-HT neurons. This was further tested in real-time place preference or real-time place aversion tasks. Photoactivation of the rostral VTA fibers to the DRN lead to an avoidance of the chamber, while the activation of the caudal VTA fibers to the DRN led to a preference for the stimulated-paired chamber (Li et al., 2019). The results of this study are in line with results demonstrating that DRN 5-HT neurons are not reinforcing (Browne et al., 2019; McDevitt et al., 2014) . But as we highlighted, the complexity of the serotonergic system and the numerous subpopulations of the RN, as well as the subpopulations from the dopaminergic systems might give rise to different behavioral outcomes when stimulated. In any case, this last study highlights the importance of studying sub-circuits, neuronal types, and different timescales in a large cohort of reward-related behaviors. For example, it has recently shown a contribution of DRN DA neurons in encoding the motivational salience of cues and unsigned prediction errors (Cho et al., 2020).

4. RN to VTA sensitivity to high levels of 5-HT during development

In rodents, 5-HT neurons appear on embryonic day 12 (E12) (Lauder and Bloom, 1974). 5-HT begins to be released by day 13 [E13 (Lambe et al., 2000; Lidov and Molliver, 1982a, b)], reaching adult levels by P15 (Hohmann et al., 1988). Compared to the serotonergic system, the dopaminergic system develops more slowly. Dopaminergic neurons emerge between E12 and E15 (Birnie et al., 2020; Olson and Seiger, 1972), but dopaminergic innervation gradually increases until P60 (Kalsbeek et al., 1988). The earlier developing serotonergic system sends strong projections to the dopaminergic system, enabling the serotonergic system to modulate dopaminergic development (Suri et al., 2014). Given the earlier apparition of the 5-HT system and its strong inputs to the dopaminergic system, the question arises: How does change in 5-HT levels during the perinatal period alter the dopaminergic system?

It is now well documented that abnormal 5-HT levels during perinatal development can have a strong impact on emotional and cognitive functions in adulthood (Shah et al., 2018). Interestingly, Yu et al. (2014) artificially increased 5-HT signaling during adolescence by blocking either the 5-HT transporter or monoamine oxidase A. They demonstrated that increased 5-HT signaling during development lead to decreased aggression during adulthood. Moreover, they showed that aggression in the adult was correlated with dopaminergic activity, together suggesting a possible blunting of the dopaminergic activity resulting from postnatal/adolescent exposure to high levels of 5-HT.

Different studies have demonstrated the importance of the DRN to VTA control in adult animals (see section 3.3). Notably, it has been shown that the DRN 5-HT neurons modulate VTA DA activity through glutamatergic co-transmission (Cunha et al., 2020; Liu et al., 2014; Wang et al., 2019). Given the strong impact of abnormal 5-HT levels during development on adult behaviors, a recent study explored the effect of high-levels of 5-HT on this specific pathway [i.e.: RN→VTA; (Cunha et al., 2020)]. In this study, pups (P2 to P11) received a daily treatment of fluoxetine (a selective 5-HT reuptake inhibitor (SSRI)). The authors found that modulating 5-HT during early life has an effect on the serotonergic to VTA dopaminergic pathway with a decrease of excitatory input through glutamate co-transmission. Importantly, this early life manipulation had long lasting consequences in dopaminergic function and in adult behavior (Cunha et al., 2020). The question that is still ongoing is how the effect of early life abnormal 5-HT levels on the RN to VTA pathway can impact reward-related functions in the adult. A recent study suggests that early-life manipulation of the serotonergic system may result in the disruption of reward mechanisms. Indeed, adult mice exposed to a SSRI from postnatal day 2 to 11 presented reduced effort-related motivation in the progressive ratio in operant conditioning, associated with dopaminergic deficits (Menezes et al., 2021). This reduction could be reversed by the administration in adulthood of a DA/norepinephrine reuptake inhibitor (Menezes et al., 2021).

5. Conclusion

Certainty is a concept that is difficult to reach with the serotonergic system and its contribution to different functions, especially reward-related. It seems clear that the serotonergic system is involved in reward processing and recent evidence clarifies its contribution but there are still many steps to reach a conclusion on all the mechanisms possibly involved. Many points of divergence between the studies looking at the involvement of the serotonergic system in reward-related functions can probably be explained by the variety, heterogeneity and expression of 5-HT receptors combined with the specificity of the DRN cellular type activated or/and the use of 5-HT agents directly. From our point of view, there are some points of convergence of all these studies that differ from the classical view that DRN neurons inhibit the dopaminergic system: projections from the DRN either coming from non-serotonergic neurons or/and from serotonergic neurons co-transmitting glutamate (Cunha et al., 2020; Liu et al., 2014; McDevitt et al., 2014; Wang et al., 2019) can excite VTA DA neurons which in turn release DA in the NAc leading to a reinforcement learning in reward tasks (Figure 1). Critically, we also described that abnormal levels of 5-HT during development can have lifelong effect on the specific RN to VTA pathway and possibly implication in reward-related functions.

More studies are necessary to fully dissect this circuit. Understanding how 5-HT and DA interact in the context of reward is critical to comprehend certain aspects of neuropsychiatric disorders such as deficits in motivation, anergia and apathy (Kapur and Remington, 1996; Salamone et al., 2016).

Highlights.

• Serotonin (5-HT) plays an important role in the reward circuits. However, its exact role has been debated.

• Several studies have shown that serotonin inhibits dopaminergic activity.

• Recent research, using optogenetic manipulations of genetically identified cells, shows that activation of serotonergic cells can activate dopaminergic neurons, via glutamatergic co-transmission and 5-HT3 receptors, leading to reward.

• Early-life manipulation of the serotonergic system can modulate the glutamatergic-co-transmission of serotonergic cells.