Interaction between the 5-HT system and the basal ganglia: functional implication and therapeutic perspective in Parkinson's disease

Abstract

The neurotransmitter serotonin (5-HT) has a multifaceted function in the modulation of information processing through the activation of multiple receptor families, including G-protein-coupled receptor subtypes (5-HT₁, 5-HT₂, 5-HT_4–7) and ligand-gated ion channels (5-HT₃). The largest population of serotonergic neurons is located in the midbrain, specifically in the raphe nuclei. Although the medial and dorsal raphe nucleus (DRN) share common projecting areas, in the basal ganglia (BG) nuclei serotonergic innervations come mainly from the DRN. The BG are a highly organized network of subcortical nuclei composed of the striatum (caudate and putamen), subthalamic nucleus (STN), internal and external globus pallidus (or entopeduncular nucleus in rodents, GPi/EP and GPe) and substantia nigra (pars compacta, SNc, and pars reticulata, SNr). The BG are part of the cortico-BG-thalamic circuits, which play a role in many functions like motor control, emotion, and cognition and are critically involved in diseases such as Parkinson's disease (PD). This review provides an overview of serotonergic modulation of the BG at the functional level and a discussion of how this interaction may be relevant to treating PD and the motor complications induced by chronic treatment with L-DOPA.

Serotonergic innervation in the brain originates from the raphe nuclei. Both, the medial and the dorsal raphe nucleus (DRN), project to common areas implicated in motor control, such as the thalamus. Nevertheless, the basal ganglia (BG) nuclei receive serotonergic afferences coming prevalently from the DRN (reviewed in Di Matteo et al., 2008). The BG contain serotonin (5-HT) and its metabolite 5-hydroxy-indolacetic acid (5-HIAA) (Palkovits et al., 1974; Saavedra, 1977; Lavoie and Parent, 1990), 5-HT transporter (SERT) and serotonergic receptors (from 5-HT₁ to 5-HT₇). These serotonergic receptors are unevenly expressed along the BG, and their distribution also differs between species. Here, we will review the evidences supporting the serotonergic system as a modulator of the BG functionality. Both physiological and pathological conditions will be analyzed from the basic and clinical point of view.

Physiological serotonergic modulation of the basal ganglia

In accordance with its neuroanatomical distribution (as summarized in Table 1), 5-HT physiologically modulates BG nuclei activity by acting on serotonergic receptors.

Table 1.

Localization and expression density of serotonergic receptors in the basal ganglia of healthy brains of rodents, monkeys and humans.

	GPe/GPi(EP)	Striatum	STN	SNc/SNr	References
5-HT1A	+r	+r	+r	+r	rLanfumey and Hamon, 2000
	+m	+m(matrix)	+m	+m	mFrechilla et al., 2001; mHuot et al., 2012a
	+h	++m(striosome)
		+h			hHuot et al., 2012b
5-HT1B	+++r	++r	+++r	+++r	rBruinvels et al., 1993
	+++h	++h		+++h	hVarnas et al., 2004a
5-HT2A	++r	++r	+h	+r	rPazos et al., 1985
	+m	+m		+m	mHuot et al., 2012c
	++/+h	++h		++/+h	hHoyer et al., 1986; hPazos et al., 1987; hHall et al., 2000; hVarnas et al., 2004a
5-HT2C	++r	++r	+++r	+++r(c)	rMengod et al., 1990; rPompeiano et al., 1994; rAbramowski et al., 1995; rClemett et al., 2000
	+++h	+++h		+++h	hPazos et al., 1987; hLopez-Gimenez et al., 2001
5-HT3		+r		+r	rKilpatrick et al., 1987; rGehlert et al., 1993
		+++h		++h	hBufton et al., 1993
5-HT4	+++r	+++r		+++r	r,mJakeman et al., 1994; rNirogi et al., 2013
	+++m	+++m		+++m
	+++h	+++h		+++h	hBonaventure et al., 2000; hVarnas et al., 2003, 2004a
5-HT5A	+r	+r	++r	+++r	rOliver et al., 2000
5-HT6		+++r		++r	rGerard et al., 1997
		+++h		++h	hKohen et al., 1996
5-HT7	+r	+r	+h	+r	rHorisawa et al., 2013
		+h		+h	r,hMartin-Cora and Pazos, 2004
					hVarnas et al., 2004b

+++, strong; ++, moderate; +, weak/r, rodent; m, monkey; h, human. EP, entopeduncular nucleus; GPe, external segment of the globus pallidus; GPi, internal segment of the globus pallidus; STN, subthalamic nucleus; SNc, substantia nigra pars compacta; SNr, substantia nigra pars reticulata.

Striatum

The striatum is the main input nucleus of the BG and a key neural substrate for motor function. Several studies have shown that 5-HT affects striatal function. In fact, both DRN stimulation and local administration of 5-HT into the striatum inhibit the vast majority of the striatal cells (Olpe and Koella, 1977; Davies and Tongroach, 1978; Yakel et al., 1988). However, by performing intracellular recordings, some researchers have reported striatal excitatory postsynaptic potentials after DRN stimulation, as well as a 5-HT-induced increase in firing rate of medium spiny neurons (MSN) (Vandermaelen et al., 1979; Park et al., 1982; Stefani et al., 1990; Wilms et al., 2001). Stimulation of presynaptic 5-HT_1A and 5-HT_1B receptors inhibits striatal 5-HT release (Gerber et al., 1988; Knobelman et al., 2000), and these receptors also control the release of other neurotransmitters in the striatum. Accordingly, 5-HT_1A receptor activation decreases glutamate release from corticostriatal projections (Antonelli et al., 2005; Mignon and Wolf, 2005; Dupre et al., 2011, 2013). On the other hand, activation of 5-HT_1B receptors indirectly stimulates the substantia nigra pars compacta (SNc) by decreasing GABA release from the substantia nigra pars reticulata (SNr), what consequently leads to increasing striatal dopamine levels (Gerber et al., 1988).

The 5-HT₂ receptor family produces an inhibitory action on striatal neuron activity, mainly by modulating MSN (el Mansari et al., 1994; el Mansari and Blier, 1997). Moreover, Rueter et al. (2000) have shown that 5-HT_2C receptors exert tonic inhibitory control over MSN membrane excitability. Other in vivo studies, however, have shown contradictory results suggesting that the effect of serotonergic drugs depends on the area of the striatum analyzed (Wilms et al., 2001). 5-HT₂ receptor activation indirectly reduces the activity of striatal MSN by enhancing the inhibitory tone of cholinergic interneurons over these output neurons. The increased release of acetylcholine is due to activation of cholinergic interneurons mainly through 5-HT_2C receptors, although the involvement of 5-HT₆ and 5-HT₇ receptors has also been demonstrated (Bonsi et al., 2007; Blomeley and Bracci, 2009). In addition, the activation of 5-HT_2C receptors located on fast-spiking interneurons increases their excitability, causing an enhancement of GABAergic postsynaptic inhibition that also decreases the activity of striatal projecting neurons (Blomeley and Bracci, 2009).

Subthalamic nucleus

5-HT exerts a complex effect in the subthalamic nucleus (STN) that is considered to be a powerful excitatory drive in the BG motor circuit. Both pharmacological lesion of the DRN and 5-HT depletion increase STN firing frequency and burst activity in vivo (Liu et al., 2007; Aristieta et al., 2013). Decreased and increased excitability have been reported with the activation of 5-HT_1A and 5-HT_2C, and 5-HT₄ receptors, respectively (Flores et al., 1995; Stanford et al., 2005; Xiang et al., 2005; Shen et al., 2007; Aristieta et al., 2013). In addition, activation of 5-HT_1B receptors inhibits synaptic activity of STN neurons (Barwick et al., 2000; Shen and Johnson, 2008).

Globus pallidus

The globus pallidus (GP) has two segments, the external GP (GPe), which has a central position in the BG loop, and the internal GP (GPi/EP), which, together with the SNr, form the output structures of the BG. In the GPe, 5-HT depletion decreases the firing frequency and increases the proportion of bursty and irregular neurons (Delaville et al., 2012b). In contrast, local application of 5-HT or selective serotonin reuptake inhibitor (SSRI) administration excites most of GPe neurons (Querejeta et al., 2005; Zhang et al., 2010; Wang et al., 2013). These findings have been further confirmed by a patch-clamp recording study in which 5-HT perfusion produced a reversible depolarization of the GP neuron membrane potential, thereby increasing the firing rate of these neurons (Chen et al., 2008). In vivo studies indicate that the stimulatory effect of 5-HT on GPe neurons is mediated by the activation of 5-HT₄ or 5-HT₇ postsynaptic receptors, but not 5-HT_2C and 5-HT₃ receptors (Bengtson et al., 2004; Kita et al., 2007; Chen et al., 2008; Hashimoto and Kita, 2008). In contrast, 5-HT can decrease the presynaptic release of glutamate and GABA from the subthalamopallidal and striatopallidal terminals, respectively, through 5-HT_1B receptors (Querejeta et al., 2005). In addition, 5-HT has been proposed to modulate the inhibitory and excitatory responses in GPe electrical stimulation of the motor cortex in awake monkeys (Kita et al., 2007). In fact, 5-HT suppresses GABAergic inhibitory responses to cortical stimulation through presynaptic 5-HT_1B receptors and glutamatergic excitatory responses involving presynaptic or postsynaptic 5-HT_1A receptors (Kita et al., 2007).

Few studies have been conducted to investigate the effects of 5-HT on the GPi/EP nucleus. Recently, it has been shown that intra-EP administration of a 5-HT₂ receptor agonist promotes oral movements and inhibits EP neuronal activity in dopamine-depleted rats (Lagiere et al., 2013).

Substantia nigra

Together with the GPi, the SNr constitutes the principal output nucleus of the BG and plays a relevant role in movement initiation. In this nucleus, 5-HT induces mostly an inhibitory effect in vivo (Dray et al., 1976; Collingridge and Davies, 1981), while 5-HT depletion decreases firing rate and increases burst activity of SNr neurons (Delaville et al., 2012a). Electrophysiological studies carried out in brain slices indicate that 5-HT not only excites SNr neurons acting directly on 5-HT_2C receptors (Rick et al., 1995; Stanford and Lacey, 1996; Stanford et al., 2005) but also disinhibits SNr neurons by reducing GABA release from striatonigral terminals via presynaptic 5-HT_1B receptor stimulation (Stanford and Lacey, 1996). A recent electrophysiological study reveals that presynaptic 5-HT_1B receptor activation gates STN excitatory inputs to the SNr and reduces burst firing activity of the SNr, and therefore may be critically involved in movement control (Ding et al., 2013).

The role of 5-HT transmission in modulating the activity of dopaminergic SNc neurons is still unclear. Although the effect of 5-HT input seems to be inhibitory (Sinton and Fallon, 1988; Arborelius et al., 1993), chemical lesion of the DRN does not significantly alter SNc activity and DRN electrical stimulation only inhibits spontaneous activity in a subset of neurons (Kelland et al., 1990). Further, SSRI administration does not modulate SNc activity (Prisco and Esposito, 1995), and 5-HT depletion has been shown to either decrease or have no significant effect on SNc neuron excitability (Kelland et al., 1990; Minabe et al., 1996). Non-selective 5-HT₂ receptor antagonists stimulate SNc neurons (Ugedo et al., 1989), whereas 5-HT₄ receptors selectively prevents the stimulatory effect induced by haloperidol in this brain area (Lucas et al., 2001).

Implication of the serotonergic system in Parkinson's disease

In the parkinsonian state and subsequent replacement therapy with L-DOPA, the serotonergic system adapts to the lack of dopamine by adopting anatomical and functional transformations.

Serotonergic system in Parkinson's disease and parkinsonian animal models

Parkinson's disease (PD) is a neurodegenerative disease typified by loss of dopaminergic neurons in the SNc and subsequent dopamine depletion in the striatum. In patients with PD, it is generally supported that serotonergic neurotransmission decreases in advanced stages of the disease (Haapaniemi et al., 2001; Kerenyi et al., 2003) since the DRN, in addition to other nuclei, undergoes degeneration (Halliday et al., 1990; Jellinger, 1990). Moreover, 5-HT and 5-HIAA concentrations, as well as SERT expression, are reduced in several BG nuclei (Scatton et al., 1983; Raisman et al., 1986; D'Amato et al., 1987; Chinaglia et al., 1993; Kerenyi et al., 2003; Guttman et al., 2007; Kish et al., 2008; Rylander et al., 2010). Regarding receptor expression, 5-HT_1A is decreased and 5-HT_2C is increased in some BG nuclei (Fox and Brotchie, 2000; Ballanger et al., 2012) (Figure 1). Other serotonergic receptor (5-HT_1B/D, 5-HT₃, and 5-HT₄) densities are however not modified by the dopaminergic loss (Steward et al., 1993; Reynolds et al., 1995; Wong et al., 1996; Castro et al., 1998). Overall, this dysfunctional serotonergic neurotransmission can indeed be linked to the high prevalence of depressive symptoms in parkinsonian patients (Reijnders et al., 2008).

Figure 1.

Simplified diagram of the basal ganglia circuits and altered serotonergic receptor expression in pathological states. Changes found in serotonergic receptor density in parkinsonian (left boxes) and dyskinetic (right boxes) patients or animals models compared to control conditions. Each nucleus and its modifications in receptor expression are encoded with the same color. GABAergic inhibitory pathways are represented in dark blue and glutamatergic excitatory pathways in red. Modulatory dopaminergic connections are indicated in green and serotonergic pathways in brown. DRN, dorsal raphe nucleus; GPi (EP), internal segment of the globus pallidus (entopeduncular nucleus); GPe, external segment of the globus pallidus; STN, subthalamic nucleus; SNc, substantia nigra pars compacta; SNr, substantia nigra pars reticulata. r, rodent; m, monkey; h, human.

In animal models of parkinsonism, the changes occurring after dopaminergic lesion have not been equally reproduced by different research groups. The discrepancies between these studies may be due to different protocol paradigms used for inducing the parkinsonian state, including the age of the animals, site of injection, concentration of the toxin, and the time between surgery and performing the studies. Several researchers have reported hyperinnervation (Zhou et al., 1991; Rozas et al., 1998; Balcioglu et al., 2003; Maeda et al., 2003), while others found no sprouting (Prinz et al., 2013), or even a decrease in striatal serotonergic fibers after dopaminergic damage (Takeuchi et al., 1991; Rylander et al., 2010). Along the same lines, striatal 5-HT levels have been found to be increased (Commins et al., 1989; Zhou et al., 1991; Karstaedt et al., 1994; Balcioglu et al., 2003), unchanged (Breese et al., 1984; Carta et al., 2006), or decreased (Frechilla et al., 2001; Aguiar et al., 2006, 2008). As detailed in Figure 1, studies performed in different animal models report unequal modification in serotonergic receptor expression along the BG nuclei. On the other hand, the DRN also suffers adaptative changes after the dopaminergic degeneration, such as increased 5-HT_1A expression in MPTP monkeys (Frechilla et al., 2001) or weaker inhibitory effects of 5-HT_1A agonists on neuron activity in rats (Wang et al., 2009). Electrophysiological studies using different 6-hydroxydopamine (6-OHDA) lesion models have shown increased basal firing rate of serotonergic cells in the parkinsonian state (Zhang et al., 2007a; Kaya et al., 2008; Wang et al., 2009; Prinz et al., 2013), while others show decreases (Guiard et al., 2008) or no changes (Miguelez et al., 2011).

In spite of the disparity of results, it seems clear that to varying extents, the serotonergic system is affected in parkinsonian conditions. More clinical and preclinical studies using the same experimental models and a greater amount of samples would help to clarify the role of the serotonergic system in each stage of PD.

Serotonergic system in L-DOPA induced dyskinesia

The dopamine precursor L-DOPA is the most effective pharmacological treatment for PD, but it does not stop the progression of the disease. Moreover, long-term administration of L-DOPA induces motor complications, known as L-DOPA induced dyskinesias (LID), which have been related to adaptive changes of the serotonergic system. For example, a recent publication revealed that patients who had developed dyskinetic movements showed significant serotonergic hyperinnervation in the GPe and caudate, in comparison to non-dyskinetic individuals (Rylander et al., 2010). Such sprouting was directly correlated with the severity of motor complications. In contrast, other studies have shown that striatal postmortem content of 5-HT and SERT levels did not differ significantly between dyskinetic and non-dyskinetic cases (Calon et al., 2003; Kish et al., 2008), and chronic L-DOPA treatment did not influence SERT expression (Politis et al., 2010). As for serotonergic receptors, a study performed in PD patients that followed L-DOPA treatment showed increased 5-HT_1A expression in several cortical areas, while no modification in the striatum, GP, SN, or thalamus was reported (Huot et al., 2012b). In the SNr, 5-HT_2C expression has also been observed to be raised in those patients (Fox and Brotchie, 2000).

The use of animal models has provided valuable data to better understand the physiopathological mechanisms of LID. The most used models include non-human primates injected with MPTP and rodent-models with hemilateral dopaminergic loss chronically treated with L-DOPA. Although differences may arise from the methodological protocols, such models are considered to reproduce resembling symptoms and molecular changes to those observed in PD patients and efficiently respond to antidyskinetic therapy (Iderberg et al., 2012). It is now well known that exogenously administered L-DOPA can be stored, transformed into dopamine, and released from serotonergic terminals to multiple brain regions, including the striatum, in an uncontrolled manner, producing a non-physiological stimulation of sensitized dopaminergic receptors (Arai et al., 1995; Carta et al., 2007; Yamada et al., 2007; Navailles et al., 2010b, 2013). Lesions of the DRN consistently prevent the expression of dyskinesia (Carta et al., 2007; Eskow et al., 2009) or dopamine release after an acute L-DOPA injection (Navailles et al., 2010b). This interaction between serotonergic and dopaminergic systems is reciprocal, as 5-HT levels also decrease after L-DOPA administration, and L-DOPA itself can antagonize the effect of serotonergic agents (Bartholini et al., 1968; Everett and Borcherding, 1970; Commissiong and Sedgwick, 1979; Borah and Mohanakumar, 2007; Navailles et al., 2010a; Riahi et al., 2011; Miguelez et al., 2013). In dyskinetic animals, SERT expression has been found to be up-regulated (Rylander et al., 2010), not modified (Prinz et al., 2013), or decreased (Nevalainen et al., 2011). Serotonergic receptor expression in the BG is unevenly modified with L-DOPA treatment: 5-HT_2A and 5-HT_1B receptor expression is increased (Zhang et al., 2008; Riahi et al., 2011, 2013; Huot et al., 2012c), while 5-HT_1A receptor expression is increased (Huot et al., 2012a) or does not change (Riahi et al., 2012) (Figure 1). The primary modifications occurring in the serotonergic system are thought to take place at terminal levels because no changes in the number of serotonergic neurons (Rylander et al., 2010; Inden et al., 2012) or 5-HT or dopamine levels in the DRN of dyskinetic rats have been reported (Bishop et al., 2012).

Clinical relevance

Although motor complications appear in the majority of the patients that receive chronic treatment with L-DOPA, an effective pharmacological tool for avoiding or treating LID expression is still missing. In this sense, 5-HT_1A/1C receptors, which are involved in the regulation of the ectopic dopamine release, are envisaged as promising targets. In 6-OHDA-lesioned rats and MPTP monkeys chronically treated with L-DOPA, 5-HT_1A/1C receptor agonists reduce expression of LID without impairing L-DOPA improvement in motor performance (Bibbiani et al., 2001; Ba et al., 2007; Dupre et al., 2007). Furthermore, administration of the 5-HT_1A agonist, 8-OH-DPAT, also prevents L-DOPA-induced increment of extracellular dopamine (Nahimi et al., 2012). Other drugs that modulate 5-HT neurotransmission have shown efficacy over LID. Thus, a recent study has revealed that the treatment with the precursor of 5-HT, 5-hydroxytryptophan reduces the appearance of LID in L-DOPA-primed rats (Tronci et al., 2013). The 5-HT_2A receptor inverse agonist ACP-103 reduces tremor in rodents and LID in MPTP monkeys (Vanover et al., 2008). Acute and prolonged SSRI treatment attenuates the severity and development of LID in L-DOPA-primed and naive rats without interfering with motor improvement, which may be mediated in part by 5-HT_1A receptors (Bishop et al., 2012; Conti et al., 2014). In contrast, in PD patients, while buspirone, a partial 5-HT_1A agonist, ameliorates dyskinesia (Kleedorfer et al., 1991; Bonifati et al., 1994), sarizotan, another 5-HT_1A receptor agonist, failed to improve it compared with placebo (Goetz et al., 2008) and significantly increased off time (Goetz et al., 2007).

Concluding remarks

The effects of 5-HT in the BG depend on the specific nucleus and its receptor distribution. 5-HT induces an inhibition of MSN in the striatum using either direct or indirect activation of serotonergic receptors, as well as in the STN and SNr in vivo. In contrast, in the GPe the overall effect of 5-HT is excitatory. In other nuclei such as the EP or SNc the net effect is still not well understood.

The serotonergic physiological modulation may be modified in pathological conditions where the BG nuclei are highly affected. Here, we provide data regarding the alteration of the serotonergic system in PD, pointing out important discrepancies about the relationship between the serotonergic and dopaminergic systems in pathological states. In this concern, key methodological differences such as the use of different animal species and models, pharmacological treatments or stage of the disease in PD patients may explain these inconsistencies.

In summary, the serotonergic system is implicated in the modulation of the BG activity and in the etiopathology of PD and LID. However, although in preclinical studies results indicate that serotonergic drugs may be suitable for treating LID, this fact has yet to be supported by clinical trials. Accordingly, further investigation is required to determine the most suitable serotonergic target to treat these motor disturbances.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.