Serotonin as a Modulator of Glutamate- and GABA-Mediated Neurotransmission: Implications in Physiological Functions and in Pathology

Abstract

The neurotransmitter serotonin (5-HT), widely distributed in the central nervous system (CNS), is involved in a large variety of physiological functions. In several brain regions 5-HT is diffusely released by volume transmission and behaves as a neuromodulator rather than as a “classical” neurotransmitter. In some cases 5-HT is co-localized in the same nerve terminal with other neurotransmitters and reciprocal interactions take place. This review will focus on the modulatory action of 5-HT on the effects of glutamate and γ-amino-butyric acid (GABA), which are the principal neurotransmitters mediating respectively excitatory and inhibitory signals in the CNS. Examples of interaction at pre-and/or post-synaptic levels will be illustrated, as well as the receptors involved and their mechanisms of action. Finally, the physiological meaning of neuromodulatory effects of 5-HT will be briefly discussed with respect to pathologies deriving from malfunctioning of serotonin system.

INTRODUCTION

The neurotransmitter serotonin (5-hydroxy-tryptamine, 5-HT) is involved in the regulation of basic physiological functions such as hormone secretion [69], sleep-wake cycle [181], motor control [85], immune system functioning [133], nociception [47], food intake [122] and energy balance [74]. In addition, 5-HT participates to higher brain functions, such as cognition and emotional states, by modulating synaptic plasticity [52] and, as recently discovered, neurogenesis [42, 62, 83].

Serotonergic neurons of the CNS are localized in clusters within the raphe nuclei, central gray and reticular formation [68, 84] and have been classified into nine groups named B1-B9 [29] (Fig. 1). Nerve fibers arising from the caudal groups of serotonergic neurons (B1-B4) form a descending system directed to the spinal cord and also project to cerebellum, pontine and midbrain structures, whereas ascending fibers originate from the rostral groups of serotonergic neurons (B5-B9) and innervate almost all brain areas.

Fig. (1).

Serotonin system. Principal groups of serotonergic neurons in the CNS and their projection sites.

Serotonin receptors were initially divided in two main classes named 5-HT₁ and 5-HT₂, displaying respectively nanomolar and micromolar affinity for 5-HT [150]. Five other 5-HT receptor types have since been characterized and named 5-HT₃ [16], 5-HT₄ [45], 5-HT₅, 5-HT₆ and 5-HT₇ [64, 176] and within each receptor family different subtypes exist. To date, fourteen subtypes of serotonin receptors have been cloned; the localization, pharmacological profile [9, 79], intracellular signaling pathways [79, 140, 149], modulatory effects on membrane ion currents [12] and physiological functions [9, 193] of the principal serotonin receptor subtypes are summarized in Table 1. With the exception of the 5-HT₃ receptor, which is a ligand-gated ion channel, all the other 5-HT receptor subtypes are metabotropic G-protein-coupled receptors and modulate an intracellular second messenger system. In particular, subtypes belonging to the 5-HT₁ family are coupled to a G-protein of the G_i/o type and reduce adenylate cyclase activity; 5-HT₂ receptor subtypes activate phospholypase C, thus stimulating PI hydrolysis and intracellular Ca²⁺ release; 5-HT₄, 5-HT₆ and 5-HT₇ are instead positively coupled to adenylate cyclase (through a G_s) and increase cAMP levels. The final effect on membrane potential is hyperpolarizing for 5-HT_1A receptors and depolarizing for most other subtypes, with a fast depolarization mediated by 5-HT₃ receptors and a slow depolarization mediated by 5-HT_2A-B, 5-HT₄ and 5-HT₇ receptors. 5-HT_2C were shown to exert either hyperpolarizing [25, 37] or depolarizing effects [48, 190] in distinct areas.

Table 1.

Serotonin Receptor Subtypes. 5-HT receptors have been grouped into seven principal classes, named 5-HT₁ to 5-HT₇; for each subtype, the table indicates the pharmacological characteristics, the localization, the intracellular action mechanism and final effect on neuronal excitability, the physiological function in which the receptor is involved and the pathologies deriving from its malfunctioning.

Receptor	Localization	Agonists	Antagonists	Intracellular second messenger	Membrane effects	Physiological function	Malfunctioning pathologies
5-HT1A	Dorsal raphe; Hippocampus	8-OH-DPAT buspirone gepirone	NAN 190 MDL 73005EF WAY 100635	Gi/o ↑↓ camp ↑ PI turnover*	Hyperpolarization (increase in gK; decrease in gCa)	Autoreceptor; modulation of release of other neurotransmitters modulation of anxiety	Anxiety; depression
5-HT1B	Hippocampus; Striatum; Substantia nigra; Raphe nuclei; Cerebellum; Frontal cortex; Cerebral arteries	Sumatriptan	GR55562 SB 216641 SB 272183	-	-	Nerve terminal autoreceptor; modulation of release of other neurotransmitters	migraine
5-HT1D	Dorsal raphe; Human heart	Sumatriptan PNU 109291	BRL 15572	-	-	autoreceptor	migraine
5-HT2A	Cortex; Basal ganglia; Peripheral tissues	DOI DOB	Ketanserin MDL 100907 Cinanserin Mianserin Methysergide	Gq/11 ↑ PI turnover	Depolarization (decrease in gK)	Possibile role in learning and memory	Psychiatric disorders
5-HT2B	Cerebellum; lateral septum; hypothalamus; amigdala; cardiac valves	BW 723C86	SB 200646 SB 204741	Gq/11 ↑ PI turnover	-	Food intake; behaviour	Anxiety; feeding disorders; cardiac valvulopathies
5-HT2C	Choroid plexus; Hippocampus Habenula Substantia nigra Raphe nuclei	Ro 600175	Mianserin Methysergide Mesulergine	Gq/11 ↑ PI turnover	Depolarization or hyperpolarization in distinct neurons	Food intake; neuroendocrine regulation	Feeding disorders; Cognitive impairment
5-HT3	Olfactory bulb Cerebral cortex Hippocampus Amygdala Hypothalamus Solitary tract nucleus	2-methyl-5-HT SR 57227	ICS 205930 Zacopride Ondansetron Granisetron Tropisetron	None: direct gating of channel	Fast depolarization (increase in gNa and gK)	Pre-synaptic modulation of transmitter release	Anxiety; Schizophrenia; Cognitive impairment
5-HT4	Colliculi Hippocampus Peripheral tissues	Renzapride BIMU 8 RS 67506 ML 10302	GR 113808 SB 204070	Gs ↑ cAMP	Slow depolarization (decrease in gK)	Modulation of transmitter release; memory enhancement	Neurodegenerative diseases; Cardiac arrhythmia
5-HT6	Striatum Amygdala N. accumbens Hippocampus Cortex Olfactory tubercle		Ro 630563 SB 271046 SB 357134	Gs ↑ cAMP	-	Modulation of acetylcholine transmission	Cognitive disfunctions (Alzheimer)
5-HT7	Cerebral cortex; Thalamic nuclei; hypothalamus; limbic structures	8-OH-DPAT	SB 258719 SB 269970	Gs ↑ cAMP	Slow depolarization (increase of Ih)	Control of circadian rhythms; Thermoregulation Mood and behaviour	Affective disorders; Migraine; Nociception

Notes: * in transfected cells; ↑ increase; ↓ decrease. Abbreviations: I_h hyperpolarization-activated nonselective cation current; g_Na sodium conductance; g_K potassium conductance.

References: [9, 12, 17, 27, 64, 79, 176, 193]

Some of the cloned 5-HT receptors (5-ht_1E, 5-ht_1F, 5-ht_5A, and 5-ht_5B) are indicated in lower case to denote that their endogenous expression and physiological function still have to be found (Table 2); no selective ligand for these receptors is yet available, but the mRNA coding for each of them has been localized in discrete brain areas by in situ hybridization [79].

Table 2.

Putative Serotonin Receptor Subtypes. 5-HT_1E, 5-HT_1F and 5-HT₅ (5A and 5B) receptors have been cloned and their mRNA has been localized in various CNS ares by in situ hybridization; the existence of native counterparts of these receptors and their physiological function still has to be proved.

Receptor	Ligands	Localization (mRNA)	Second messenger	Hypotheses on function
5-ht1E	-	human frontal cortex; putamen	Gi/o ↓cAMP*	-
5-ht1F	LY 334370 sumatriptan	Dorsal raphe Hippocampus Cortex Striatum Thalamus Hypothalamus	Gi/o ↓cAMP*	autoreceptor
5-ht5A	-		Gi/o
5-ht5B	-		None identified

References: [9, 79]

In many brain regions 5-HT receptors have been localized on neurons that do not receive a direct serotonergic innervation; in parallel, 5-HT fibers often lack typical synaptic contacts with post-synaptic neurons [35, 180]. Such a mismatch between serotonergic fibers and post-synaptic 5-HT receptors has suggested that 5-HT in the CNS may preferentially use volume transmission and behave as a neuromodulator rather than as a “classical” neurotransmitter [3, 20, 21, 158]. As a matter of fact, a large number of studies report that 5-HT is able to modulate excitatory and inhibitory effects respectively mediated by glutamate and GABA; some examples will be illustrated below.

MODULATORY ACTION OF SEROTONIN ON GLUTAMATE- AND GABA- MEDIATED TRANSMISSION

Modes of Interaction

5-HT differently modifies glutamate- and GABA- mediated effects, acting on distinct 5-HT receptor subtypes. At a pre-synaptic level, 5-HT can modulate neurotransmitter release: for example, in various brain regions glutamate release is reduced by 5-HT_1A [162, 164, 177], 5-HT_1B [13, 130, 151, 167] and 5-HT₆ receptors [31]. Similarly, pre-synaptic inhibition of GABA release is mediated by 5-HT_1A [90, 95, 96] and 5-HT_1B receptors [88, 117]. By activation of 5-HT₃ receptors, instead, 5-HT stimulates the release of either glutamate [7, 57, 185] or GABA [90, 96, 179]. Also 5-HT₂ receptors were shown to stimulate GABA release [2] and to either increase [2, 72, 177] or reduce [118] glutamate release in distinct structures.

Concerning GABA release, another action of 5-HT consists in modulating the excitability of GABAergic interneurons; on this aspect, detailed studies have been performed in the hippocampus. Serotonergic fibers from dorsal raphe selectively target a subset of hippocampal interneurons specifically involved in GABA_B-mediated feedforward inhibition [55, 56]. In interneurons from stratum lacunosum-molecolare of the CA1 region, the amplitude of a T-type low threshold voltage-dependent Ca²⁺ current, which is responsible for rhythmic oscillations of membrane potential, is enhanced by 5-HT and reduced by a GABA_B agonists [54]. Concerning the role of different subtypes of 5-HT receptors, it has been shown that GABA release from CA1 interneurons is inhibited by 5-HT_1A receptors [90, 163] and enhanced by 5-HT₂ [166] and 5-HT₃ receptors [90, 121, 157, 179]. An important issue is that a very large heterogeneity exists among hippocampal interneurons [148]: as a matter of fact, as many as 16 different types have been characterized based on their location and morphology; besides, hippocampal interneurons also differ with respect to colocalization of GABA with various peptides, expression of calcium binding proteins, expression of membrane ion channels, discharge patterns and responsiveness to neurotransmitters. In fact, the activity of hippocampal interneurons is differentially modulated by 5-HT, noradrenaline, acetylcholine acting on muscarinic receptors and glutamate acting on metabotropic receptors, following at least 25 different response patterns. Since each type of interneuron probably controls a peculiar function (for example the generation of action potentials from pyramidal neurons, the integration of input signals or the modulation of local inhibitory networks), the authors postulated that the role of modulating neurotransmitters, among which 5-HT, is to switch between functions and thus change hippocampal computation [148].

Similarly to the hippocampus, modulation of GABA release from interneurons (mainly inhibition by 5-HT_1A receptors and stimulation by 5-HT₂ and/or 5-HT₃ receptors) has also been observed in other areas, among which dentate gyrus [132, 152], enthorinal cortex [40, 165], piriform cortex [115], frontal and anterior cingulate cortex [192].

Glutamate- and GABA-mediated effects can also be modulated by 5-HT at a post-synaptic site; the mechanisms of interaction have been elucidated in some cases and include: 1) recruitment of receptors on the post-synaptic membrane [104]; 2) modulation of receptor function by promoting its phosphorylation [49, 103, 191]; 3) effects converging on a common signaling pathway, such as a G protein [6, 144], adenylate cyclase [169] or phospholipase C [147]; 4) modulation of a common membrane ion channel [54, 169]; 5) effects on distinct ion channels, reciprocally influencing membrane potential and neuronal excitability [2, 46, 154, 155, 161].

The following paragraphs will illustrate in more details some examples of 5-HT modulation on glutamate- and GABA-mediated transmission, especially with respect to the physiological functions of the brain areas where such modulation occurs.

Modulation of Glutamate Transmission

Glutamate is the most widely diffused excitatory amino acid in the central nervous system, activating three major types of ionotropic receptors, namely N-methyl-D-aspartate (NMDA), α-amino-3-hydroxy-5-methyl-4-isoazolepropionic acid (AMPA) and 2-carboxy-3-carboxymethyl-4-isopropenylpyrrolidine (kainate) receptors, and three main groups of metabotropic receptors (counting eight subtypes named mGluR1-8) [93].

A recent article points out that co-transmission of glutamate and monoamines is a very frequent phenomenon in the CNS [178]. This view is supported by the finding that raphe neurons, the main source of serotonergic fibres projecting to almost all brain regions, are immunopositive for glutamate [138, 146] as well as for phosphate-activated glutaminase (PAG), an enzyme involved in glutamate metabolism [89], and contain the vescicular glutamate transporter VGLUT3 [65]. When grown in microcultures, approximately 60% of serotonergic raphe neurons evoke AMPA/kainate-mediated excitatory post synaptic potentials (EPSPs), indicating that most of these neurons in addition to 5-HT use glutamate as a co-transmitter [87].

Modulation of glutamate transmission by 5-HT has been described in several CNS regions, especially in those controlling cognition, nociception and motor functions.

Among the structures involved in learning and memory, in the hippocampus 5-HT-glutamate interaction has been extensively explored. One of the first observations was that the effects mediated by NMDA and 5-HT₂ receptors converge on the same signal transduction mechanism (phosphatidylinositol breakdown) [59]. Later, it was reported that 5-HT suppresses long term potentiation (LTP) in hippocampal slices by preventing the activation of NMDA receptors and the enhancement of AMPA-mediated currents that lead to LTP induction [171]. The hippocampus contains several subtypes of serotonin receptors, namely 5-HT_1A [160, 174], 5-HT_1B [4, 130], 5-HT₂ [17, 28], 5-HT₃ [116, 132], 5-HT₄ [5, 160], 5-HT₆ and 5-HT₇ [73], many of which modulate glutamate-mediated transmission acting at different levels. In the CA1 region, pre-synaptic 5-HT_1A receptors reduce glutamate release from Schaffer collaterals to CA1 pyramidal neurons [162] (Fig. 2). 5-HT_1B receptors are instead located on the axons terminals of CA1 pyramidal neurons [4]; their activation inhibits local glutamate release, depressing especially the AMPA/kainate component of excitatory transmission to neighboring CA1 pyramidal neurons and to local interneurons [130, 131]. 5-HT_1B receptors also reduce glutamate release from CA1 fibers in the subicular cortex [13]. Based on these data, it was proposed that the memory impairment observed after a treatment with 5-HT_1B agonists is probably caused by a reduction of excitatory neurotransmission in circuits involving the hippocampus [129].

Fig. (2).

Modulation of glutamate- and GABA-mediated transmission by 5-HT in the hippocampus. In the CA1 region, pre-synaptic 5-HT_1A receptors on Schaffer collaterals reduce glutamate release to pyramidal neurons [162]. 5-HT_1B receptors, located on axon terminals from pyramidal neurons and on their recurrent collaterals, inhibit glutamate release to neighboring pyramidal neurons and to local interneurons [130, 131]. The release of GABA from CA1 inhibitory interneurons is stimulated by 5-HT₂ [101, 166] and by 5-HT₃ receptors [90, 121, 157, 163, 179] and inhibited by 5-HT_1A receptors [90, 157, 163]. 5-HT and GABA_B receptors respectively increase and decrease T-type Ca²⁺ current on interneurons from stratum lacunosum-moleculare [54]. In dentate gyrus, 5-HT₄ receptors inhibit glutamate-mediated transmission [97], whereas 5-HT₃ receptors stimulate GABA release from interneurons [121, 152]. In CA3 pyramidal neurons, 5-HT inhibits GABA_B -mediated IPSCs acting both pre- and post-synaptically [145]; 5-HT and GABA_B receptors cooperate in increasing a hyperpolarizing outward potassium (K+) current [169]. A decrease in glutamate release mediated by 5-HT₆ receptors has been measured by microdialysis in the whole dorsal hippocampus [31]. Note: this figure represents a simplified scheme and does not account for the different subtypes of hippocampal interneurons.

On the other side, 5-HT_2A receptors stimulate glutamate release from fibers arising in the CA1 and CA3 regions and directed to dorsolateral septal nucleus [72].

Concerning 5-HT₄ receptors, various modulatory effects on excitatory transmission have been observed in dentate gyrus by two distinct research groups using a similar experimental protocol (recording of evoked field potentials in dentate gyrus of freely moving rats): 5-HT₄ receptor activation induced either a decrease [97] or no effect [114] on basal synaptic transmission. Both studies concur however that 5-HT₄ receptor activation modulated LTP as well as depotentiation, a low-frequency stimulation-induced reversal of LTP, suggesting their role in metaplasticity, a high order level of synaptic plasticity related to activation of NMDA receptors and subsequent Ca²⁺ entry prior to stimulation protocols that induce LTP or LTD [1].

Another 5-HT receptor modulating hippocampal glutamate transmission is the 5-HT₆ subtype, the activation of which tonically inhibits glutamate release in dorsal hippocampus, as shown by microdialysis experiments in vivo [31].

In addition, hippocampal circuits are modulated by 5-HT through enthorinal cortex, the region sending one of the main inputs to hippocampus, where pre-synaptic 5-HT_1A receptors reduce glutamate release [164].

A reciprocal interaction between serotonin and glutamate also occurs in frontal cortex, a brain area responsible for working memory and selective attention, thus playing a primary role in cognition. In this region, similarly to results obtained in dorsal hippocampus, glutamate release is tonically inhibited by 5-HT₆ receptors [31]. At the same time, 5-HT is able to counteract a number of effects mediated by NMDA receptors: for example, NMDA-induced stimulation of 5-HT release from fibers originating in raphe nuclei is inhibited by 5-HT autoreceptors belonging to the 5-HT₁ family [51]. Activation of NMDA receptors in frontal cortex by microdialysis stimulates the release of glutamate in the striatum while reducing local glutamate release, and such effects are inhibited respectively by 5-HT_1A and 5-HT_2A receptors [26, 41]. NMDA-mediated NO-synthase activation and cGMP formation in cultured cortical neurons is inhibited by 5-HT₂ receptor agonists [60]; the same effect was observed in cortical slices after application of the 5-HT_1B-D agonists sumatriptan and zolmitriptan [172].

Differing from frontal cortex, where many glutamatemediated effects are contrasted by 5-HT, in prefrontal cortex 5-HT enhances glutamate transmission by both pre- and post-synaptic mechanisms: 5-HT_2A receptor activation increases glutamate release and enhances the amplitude of glutamatergic EPSCs by inducing a sub-threshold sodium current in the apical dendrites of pyramidal neurons [2].

One of the best known functions of 5-HT in the nervous system is the control of nociception, exerting either anti- or pro-nociceptive effects depending on the type of 5-HT receptor and on its site of action [8, 50, 58, 63, 125]. With respect to glutamate-mediated transmission, modulatory effects of 5-HT have been observed in the thalamus, a structure involved in the integration of sensory inputs including pain stimuli, and in the dorsal horn of spinal cord, an important gating region for nociceptive transmission. In neurons from ventrobasal thalamus, 5-HT enhances both NMDA- and non-NMDA-mediated effects; such action of 5-HT, however, is indirectly mediated by an increase of Ih, a hyperpolarization-activated Na⁺/K⁺ current contrasting inhibitory inputs [46, 161].

In spinal dorsal horn, descending serotonergic fibers arising from raphe nuclei exert a strong modulation on the input from primary sensory afferents, inducing either inhibition or enhancement of glutamate transmission through distinct 5-HT receptors. As a matter of fact, a reduction of synaptic currents is mediated by 5-HT_1A receptors [78], which depress the responsiveness of dorsal horn neurons to NMDA [110, 135], whereas an enhancement is induced by 5-HT₂ receptors by switching silent glutamatergic synapses into functional ones [105]. In particular, 5-HT₂-receptor activation stimulates the recruitment of post-synaptic AMPA receptors; such effect is strictly dependent on a PDZ protein (Post-synaptic density 95/Disc-large/Zonula occludens-1) [104], a membrane protein involved in various functions among which the targeting of receptors to a specific subcellular compartment [63, 71]. 5-HT₂–mediated enhancement of synaptic currents in spinal dorsal horn has been predominantly observed during development. Vice versa, in adult animals 5-HT exclusively inhibits synaptic responses; however, 5-HT inhibitory effect can be turned into a stimulation by elevating intracellular cAMP levels; again, 5-HT action mechanism is based on recruiting functional AMPA responses in previously pure NMDA (silent) synapses [184]. Thus, in spinal dorsal horn of adult animals 5-HT₂ receptors can mediate either facilitating or inhibitory effects on glutamate-mediated transmission depending on the intracellular level of cAMP, a condition that can be modified by a number of other neurotransmitters modulating adenylate cyclase activity.

5-HT-glutamate interaction in the spinal cord, besides controlling pain transmission, plays a role in motor control. At a cellular level, 5-HT is able to increase glutamate-induced excitability of spinal motoneurons [77, 82, 168, 187]. Consistently, an experimentally-induced decrease in 5-HT levels in ventral horn was found to suppress postural muscle tone, an effect that is likely to contribute to the muscle atonia occurring during rapid eye movement (REM) sleep [98]. In addition, activation of spinal 5-HT receptors modulates some typical NMDA-receptor mediated behaviors [128].

5-HT is able to enhance glutamate-mediated excitation of motoneurons in some cranial motor nuclei, among which facial [119] and trigeminal nucleus [80]. In substantia nigra, another region controlling voluntary movement, post-synaptic 5-HT_2A and 5-HT₄ receptors located on dopaminergic neurons selectively inhibit the mGluR-mediated component of glutamate response; since mGluR effect consisted in a slow inhibitory outward current, the action of 5-HT shifted neuronal response to glutamate towards excitation [147].

In our laboratories, we have studied the modulatory action of 5-HT in vestibular nuclei and in the red nucleus of mesencephalon, both structures involved in motor functions. Microiontophoretic applications of 5-HT in the lateral vestibular nucleus increased neuronal firing rate; the response to 5-HT was attenuated when neuronal background firing was increased by glutamate co-application, suggesting that 5-HT- and glutamate- mediated effects, occluding each other, might converge on a common mechanism [109]. The interplay between 5-HT and glutamate in vestibular nuclei involved different receptor subtypes: the prevailing action of serotonin was a depression of the NMDA component of glutamate response and such effect was mediated by 5-HT₂ receptors. Conversely, an enhancement of both NMDA and non-NMDA receptor-mediated responses was observed in a minority of cases and was mediated by 5-HT_1A receptors [106]. As to the functional consequences of 5-HT effect, changing the responsiveness of vestibular neurons to glutamate might affect postural balance and ocular movements, the main functions controlled by vestibular nuclei.

In the red nucleus 5-HT depressed glutamate-induced excitation; 5-HT effect was mediated by 5-HT_1A receptors and was mostly directed to non-NMDA responses [107].

Still concerning motor function, inhibition of glutamate-mediated effects by serotonin has been described the cerebellum, where 5-HT depresses glutamate-induced excitation of Purkinje cells [75, 102]. Consistently, another study shows that activation of 5-HT₂ receptors reduces glutamate release from synaptosomes derived from mossy fibers [118]. In neurons of deep nuclei from cerebellar slices, 5-HT decreased responses to glutamate, quisqualate and NMDA, probably acting at a post-synaptic level [61].

Modulation of glutamate transmission by 5-HT has also been observed in other brain areas involved in different functions. For example, in tractus solitarius nucleus [7], dorsal vagal motor nucleus [185] and area postrema [57] 5-HT stimulates glutamate release by activation of pre-synaptic 5-HT₃ receptors. In suprachiasmatic nucleus, pre-synaptic 5-HT_1B receptors inhibit retinal input [151] whereas 5-HT₇ receptors reduce glutamate effects at a post-synaptic level [155]; by these effects, serotonin changes neuronal responsiveness to light stimuli and participates to the regulation of circadian rhythms.

Modulation of GABA-Mediated Transmission

γ-Amino-butyric acid (GABA) is the principal inhibitory neurotransmitter in the CNS and activates three main receptor families: GABA_A and GABA_C receptors are ligand-gated ion channels carrying a chloride current [14], whereas GABA_B are metabotropic G-protein coupled receptors that reduce neuronal excitability mainly by modulating K⁺ and/or Ca²⁺ channels [139]. Immunohistochemical data indicate the presence of GABA and serotonin in the same neurons in raphe nuclei [10] and in neurons of medulla oblongata projecting to the spinal cord [126], although colocalization is limited to a small extent [127, 170].

GABA-mediated inhibitory transmission can be strongly modulated by 5-HT: such modulation, in parallel with 5-HTglutamate interactions, has been observed in structures involved in learning and memory, sensory processing, nociception and motor control.

In the hippocampus, as already mentioned, 5-HT is able to modulate GABA release from specific subsets of interneurons; besides, post-synaptic interactions between 5-HT and GABA_B receptors have been described (Fig. 2). In CA3 pyramidal neurons, 5-HT selectively depresses the GABA_B component of GABA-mediated inhibitory post synaptic potential (IPSP) [145]; in particular, 5-HT and GABA_B receptors interact by increasing a hyperpolarizing inwardly rectifying potassium current on CA3 neurons [6, 144]. The mechanism of this interaction (also involving mGluR, adenosine and somatostatin receptors) has been investigated in details [169]: when 5-HT and the GABA_B receptor agonist baclofen were used at saturating concentration, co-application induced occlusive effects, indicating a common action mechanism; however at sub-saturating concentrations effects were supra-additive, suggesting cooperation through distinct mechanisms. The authors proposed that the effects mediated by 5-HT and GABA_B receptors probably converge on the same target, such as a G-protein or a common pool of potassium channels; in addition, the existence of separate pools of potassium channels, selectively modulated by each neurotransmitter, might account for cooperation [169].

In basolateral amygdala [96], another brain region involved in cognition and mood, 5-HT₃ receptor activation enhances GABA release from interneurons.

In pyramidal neurons from prefrontal cortex, both 5-HT₂ and 5-HT₄ receptors modulate post-synaptically GABA_A-mediated effect [49]. In particular, 5-HT₂ receptors, by activation of protein kinase C (PKC) and of its anchoring protein RACK1 (receptor for activated C kinase), promote a phosphorylation of GABA_A receptors which ultimately reduces GABA_A–mediated Cl^- currents. Conversely, 5-HT₄ receptors are able to exert a dual modulation on GABA_A-mediated current depending on protein kinase A (PKA) activation level: 5-HT₄ receptors either enhance or depress GABA_A current respectively with a low or high basal PKA level. As a possible explanation, it was suggested that different degrees of PKA activity may induce phosphorylation of distinct β subunits of the GABA_A receptor, namely a β₃ subunit (reducing GABA_A receptor function) at low PKA levels and additionally a β₁ subunit (leading to an increase of GABA_A-mediated current) at high PKA level [49]. The authors also showed that the amount of PKA-mediated phosphorylation increased with neuronal depolarization, providing an example of how 5-HT₄ receptors can dynamically regulate GABA-ergic transmission in an activity-dependent manner: an increase in neuronal firing, increasing PKA levels, would enhance GABA_A-mediated inhibition, providing a negative feedback control; however when 5-HT₄ receptors are also activated, 5-HT₄-mediated modulation of GABA_A current is switched from enhancement to reduction, which creates a “locked” loop reinforcing neuronal activity [22]. The 5-HT-mediated modulation of GABA transmission described above can be prolonged by corticotrophin releasing factor (CRF), a neuropeptide regulating physiological reactions to stress, which once again links the neuromodulatory role of serotonin to stress-related cognitive and emotional disorders [175].

5-HT modulates the action of GABA in the thalamus, in periaqueductal grey and in spinal dorsal horn, all structures involved at different levels in sensory and pain transmission. In thalamic nuclei, activation of 5-HT₂ receptors stimulates GABA release from dendrites of interneurons with a novel mechanism of action: 5-HT effect involved PLC and an increase in intracellular Ca²⁺ concentration that did not depend on Ca²⁺ release from stores nor on membrane voltage-gated Ca²⁺ channels, but was instead critically dependent on the TRPC4 channel [134], a membrane cation channel belonging to the recently discovered family of transient receptor potential (TRP) channels whose function, still largely unknown, seems to be either store-operated or receptor-operated Ca²⁺ channels [182]. By increasing GABA release from dendrites of thalamic interneurons, 5-HT can strengthen local GABAergic inhibition and ultimately modulate thalamic processing of sensory signals [134].

In periaqueductal grey 5-HT exerts a dual control on inhibitory interneurons: GABA release is inhibited by activation of 5-HT_1A receptors [95] and is probably stimulated by 5-HT_2A receptors, which have been localized on GABAergic neurons [66].

In neurons from spinal dorsal horn, 5-HT₂ receptor activation enhances GABA_A–induced Cl^- current acting through a protein kinase-dependent pathway [103, 183, 191] and activation of 5-HT₃ receptors evokes GABA release [91], both effects reinforcing GABA-mediated inhibition.

5-HT-GABA interactions have also been described in structures involved in motor control. In the cerebellum, Lugaro cells, a particular class of inhibitory interneurons exclusively found in cerebellar cortex [99, 123], are generally silent unless in the presence of 5-HT, which to date seems to be the only neurotransmitter activating them. In fact, 5-HT induces a release of GABA from Lugaro cells to Purkinje neurons [38, 39], which are subsequently hyperpolarized by an inhibitory current mediated by GABA_A receptors with an unusual pharmacology [33]. By this mechanism, 5-HT indirectly reduces the excitability of Purkinje neurons, thus affecting also the output from cerebellum and its influence on motor function [38].

In the red nucleus (RN), a work from our laboratories shows that 5-HT enhances GABA responses in neurons mostly located in the rostral part of the nucleus and this effect is mediated by activation of 5-HT_1A receptors; vice-versa, activation of 5-HT_2A receptors depresses GABA-mediated inhibition of rubral neurons mostly located in the caudal part of the nucleus [108]. Thus 5-HT is able to exert a dual control on GABA-ergic inhibition in the RN: in particular, inhibition is reduced by 5-HT in the RN area giving rise to rubro-spinal fibers, and is instead reinforced in RN areas involved in a neural circuit delivering information from substantia nigra to reticular substance. By regulating GABAergic inhibition in the RN, 5-HT modulates rubral output and the functions regulated by the RN, which concern not only motor execution but also, according to some authors [81, 142], the acquisition of conditioned reflexes.

PHYSIOLOGICAL MEANING OF 5-HT MODULATION AND ROLE IN PATHOLOGY

As a general view, the reports above cited indicate that in many brain regions 5-HT induces a decrease of glutamate transmission and a parallel increase in GABA transmission; such a pattern is particularly evident in the hippocampus (Fig. 2), [31, 90, 97, 101, 121, 130, 131, 157, 162, 163, 166, 179], in frontal cortex [31, 192] and in the cerebellum [38, 39, 75, 102]. These data together suggest that, at least in the above mentioned districts, the modulatory action of 5-HT may serve as a “brake” on neuronal excitability.

As above discussed, 5-HT modulates glutamate- and GABA-mediated effects in nervous structures mostly deputed to cognitive functions, pain transmission and motor control. In view of this, it is plausible to speculate that a malfunctioning of 5-HT modulation may participate to the pathogenesis of various diseases, some of which will be briefly illustrated.

Role of 5-HT-Mediated Modulation in Learning and Memory

It is well known that psychiatric disorders such as depression and schizophrenia, causing emotional and cognitive disorders, are related to an alteration of the serotonin system [32, 113, 153, 178]; in these pathologies, some responsibility might be attributed to a disruption of 5-HT-mediated control over glutamate- and GABA-mediated transmission in the hippocampus and frontal cortex [32, 113, 136]. In particular schizophrenia is characterized by changes in synaptic connectivity in the hippocampus, affecting mainly glutamate transmission but also the function of GABAergic neurons [70]. Positive symptoms of schizophrenia have been described as a state of “overattention” due to removal of sensory gating in the hippocampus, a physiological mechanism to which different 5-HT receptors participate, mainly by reducing glutamate-mediated transmission while reinforcing GABA-mediated inhibition (Fig. 2).

Consistently, some drugs effective as anti-psychotics as well as cognition enhancers behave as either antagonists or partial agonists of 5-HT_1A receptors [159]. Activation of hippocampal 5-HT_1B receptors induces a memory impairment [18], anxiety and a strong behavioral inhibition [17]. In line with this, 5-HT_1B knockout mice when confronted with a novel environment showed a less anxious and more explorative behavior [17, 112] and in the Morris water maze acquired a hippocampus-dependent spatial reference memory task more easily than wild-type littermates [19], suggesting that 5-HT_1B antagonists might be useful in the treatment of anxiety and improve hippocampus-dependent memory.

Also 5-HT₆ receptors, inhibiting glutamate release in hippocampus and frontal cortex [31], have become a promising target for pharmacological treatment of cognitive deficits [159].

5-HT_1A and 5-HT_2A-C agonists resulted effective in preventing glutamate-induced neurotoxicity on cultured frontal cortex neurons [60]; accordingly, in frontal cortex of freely moving animals an experimentally induced enhancement of glutamate release, which would induce excitotoxic effects and impair attentional performance, can be prevented by blockade of 5-HT_2A receptors [26, 60]. 5-HT_2A antagonists were found to enhance cognition in schizophrenia [159] and in memory deficits resulting from reduction of glutamate transmission [124]; for these reasons, a pharmacological manipulation of 5-HT_2A receptors has been suggested in therapies of cognitive disorders associated with either increased or decreased glutamate-mediated transmission [24, 26].

An interesting review [23] put forward the hypothesis that infantile autism is a hypoglutamatergic disorder based on the following observations: first of all, autism is associated with alterations in brain regions (frontal cortex, amygdala, hippocampus, cerebellum) mostly containing glutamatergic neurons. Secondly, administration of glutamate antagonists (especially NMDA antagonists) induces a series of symptoms very similar to those of autism, such as distorted sensory perception, memory defects, mood fluctuations, social withdrawal and repetitive movements. A relevant fact is that the typical features of autism are mimicked not only by NMDA antagonists but also by 5-HT_2A agonists. As the author suggested, the interplay between glutamate and 5-HT receptors offers new possibilities in the treatment of a glutamate deficit: as a matter of fact, the use of glutamate agonists would induce neurotoxicity and convulsions, whereas it might prove a good strategy to act on 5-HT_2A receptors which in turn modulate glutamate-mediated effects [23].

A serotonergic hypothesis to explain the cause of autism is indicated in another just outcome review [186]. Autism is in fact characterized by high levels of blood serotonin, and when this condition is mimicked in laboratory animals most of the typical symptoms of human autism are reproduced. The author suggested that, at an early stage of development (during the first two years of life for humans), circulating 5-HT can penetrate the immature blood brain barrier and diffuse into the brain, inducing a loss of serotonergic fibers and/or an altered production of oxytocin and calcitonin-gene related peptide (CGRP), both involved to some extent in social behavior. In light of the negative control exerted by 5-HT_2A receptors over glutamate transmission [23], it is plausible to speculate that chronically high levels of 5-HT in the developing brain might also lead to a hypoglutamatergic condition.

Alzheimer disease (AD), another pathology causing severe cognitive disorders, besides the well-known cholinergic deficits is characterized by altered levels of glutamate and serotonin, as well as of their receptors, in brain regions involved in learning and memory [30, 43, 86, 141]. Again it has been shown that glutamate transmission, which is defective in AD, can be pharmacologically modulated using antagonists of 5-HT_1A receptors in cerebral cortex [15, 53] and in the hippocampus [11], suggesting that also AD patients may benefit from a treatment with serotonergic drugs.

Role in Analgesia

5-HT is one of the neurotransmitters used by a descending system controlling pain transmission. In spinal dorsal horn, besides modulating glutamate- and GABA-mediated effects, 5-HT exerts a very complex control, also involving interactions with a large number of other neurotransmitters and peptides [125]. The final effect of 5-HT can be either pro- or anti-nociceptive depending on the receptor type activated [173]. In some cases, various and opposing effects have been described for a single 5-HT receptor subtype; for example activation of spinal 5-HT_1A receptors can induce either antinociception or hyperalgesia, an excessive pain response to noxious stimuli [125]. A novel selective agonist of 5-HT_1A receptors named F13640 was found to induce hyperalgesia in normal animals, whereas in rats with spinal cord injury the same substance is able to prevent allodynia [189], a condition in which innocuous sensory stimuli are perceived as painful [173].

For 5-HT₂ receptors, enhancing glutamate transmission in spinal dorsal horn [105], various effects on pain perception have been reported [125]; however, in general they seem to mediate hyperalgesia and might be implicated in chronic pain [63].

5-HT₃ receptor-mediated stimulation of GABA release [91] would suggest for them an antinociceptive function. 5-HT₃ receptors, however, are also located on a subpopulation of small-diameter afferent fibers and stimulate excitatory neurotransmission; thus their final effect seems rather to be pro-nociceptive [173]. In fact, the 5-HT₃ receptor antagonist ondansetron was shown to reduce allodynia both in an animal model with spinal cord injury [143] and in human patients suffering from neuropathic pain [120].

5-HT-glutamate interactions also take place in the pathogenesis of migraine. As already cited, the 5-HT_1B/1D agonists sumatriptan and zolmitriptan in brain cortex contrast NMDA-induced synthesis of NO, one of the main factors triggering headache [172]. In addition, 5-HT_1B/1D/1F receptor agonists modulate glutamate release in trigeminal ganglia, [111] and alleviate pain in patients suffering from cutaneous allodynia associated with migraine by reducing nociceptive transmission pre-synaptically [100]. These results suggests that at least part of the anti-migraine action of triptan serotonin agonists is based on a blockade of glutamate effects.

Role in Motor Control

The modulatory action of 5-HT in cerebellum, vestibular nuclei, basal ganglia, red nucleus and spinal cord participates to motor control and postural balance. As already cited, in the motor system 5-HT either enhances [80, 82, 106, 119, 168, 187] or depresses [61, 75, 102, 106, 107, 118] glutamate-mediated transmission in distinct structures. Also for GABA-mediated transmission either enhancement [39, 44, 108] or decrease [36, 108] by 5-HT have been observed in structures controlling movement.

With respect to glutamate, 5-HT modulation is often directed to NMDA-mediated effects [61, 77, 80, 106], whose function is related to the inhibitory control of movement [188]. Interestingly, recent studies show that impulsive-type behaviours (such as hyperlocomotion and stereotypy) induced by blockade of NMDA receptors can be attenuated by administration of 5-HT_2A antagonists [24, 76], suggesting a possible use of 5-HT_2A receptor antagonists to normalise NMDA receptor hypofunction.

Some observations also indicate an involvement of the serotonin system in Parkinson disease, a pathology characterized by severe motor deficit due to a disruption of the strio-nigral pathway. In basal ganglia, for example, 5-HT exerts a phasic inhibitory control over GABA synthesis and release [36]. Parkinson patients display a reduced level of striatal serotonin binding sites [30] and transporters [92]; accordingly, the selective 5-HT uptake inhibitor citalopram, associated to L-DOPA treatment, was reported to alleviate both bradykinesia and depressive symptoms [34, 156]. Therefore, a more detailed investigation for the precise role of 5-HT receptors controlling GABA-mediated transmission in basal ganglia might offer new perspectives in the therapy of Parkinson disease.

Other Functions

The modulatory control of 5-HT over glutamate- and/or GABA-mediated transmission also plays a role in other functions, among which the regulation of circadian rhythms [151, 155] and may be involved in pathological states linked to circadian system malfunctioning, such as depression, shift work or jet lag.

5-HT_1A and GABA_A receptors interact in the ventromedial nucleus of female rat hypothalamus to regulate lordosis and thus modulate sexual behavior [67].

Finally, it should also be mentioned that 5-HT, by interacting with NMDA receptors, is able to enhance ethanol tolerance [94], a condition induced by chronic alcohol consumption and characterized by increased GABA_A-mediated effects and decreased glutamate transmission. Evidence exists that alcohol addicts display a decreased level of 5-HT receptors, whereas pharmacological agents activating the serotonin system can effectively reduce ethanol consumption [137].

CONCLUDING REMARKS

In the CNS, 5-HT exerts a very complex modulatory control over glutamate- and GABA-mediated transmission, involving many subtypes of 5-HT receptors and a large variety of effects. Such action is made more complex by the fact that 5-HT interacts with many other neurotransmitters, including the two other monoamines noradrenaline and dopamine which also behave as neuromodulators.

In many cases, the use of combined techniques has revealed in details the action mechanism of neuromodulation by 5-HT. Since a malfunctioning of 5-HT-mediated modulation has been associated with a number of pathologies among which schizophrenia, childhood autism, cognitive and motor deficits, migraine and drug abuse, a deeper knowledge of the 5-HT receptor types involved and of their action mechanism(s) will provide useful strategies in the therapy of such diseases.

ABBREVIATIONS

5-HT

5-hydroxy-tryptamine, serotonin

CNS

Central nervous system

GABA

γ-amino-butyric acid

cAMP

Cyclic adenosine-3′,5′-monophosphate

NMDA

N-methyl-D-aspartate

AMPA

α-amino-3-hydroxy-5-methyl-4-isoazolepropionic acid

PAG

Phosphate-activated glutaminase

VGLUT3

Vescicular glutamate transporter 3

EPSP

Excitatory post synaptic potential

LTP

Long term potentiation

LTD

Long term depression

NO

Nitric oxyde

cGMP

Cyclic guanosine-3′,5′-monophosphate

PDZ

Post-synaptic density 95/Disc-large/Zonula occludens-1

Ih

Hyperpolarization-activated ion current

REM

Rapid eyes movement

mGluR

Metabotropic glutamate receptor

IPSP

Inhibitory post synaptic potential

PKC

Protein kinase C

RACK1

Receptor for activated C kinase

PKA

Protein kinase A

CRF

Corticotrophin releasing factor

TRP

Transient receptor potential

TRPC4

Transient receptor potential channel 4

CGRP

Calcitonin-gene related peptide

L-DOPA

L-Dihydroxyphenylalanine