Neuropharmacology of 5-hydroxytryptamine

Abstract

This review outlines the history of our knowledge of the neuropharmacology of 5-hydroxytryptamine (5-HT; serotonin), focusing primarily on the work of U.K. scientists. The existence of a vasoconstrictive substance in the blood has been known for over 135 years. The substance was named serotonin and finally identified as 5-HT in 1949. The presence of 5-HT in the brain was reported by Gaddum in 1954 and it was Gaddum who also demonstrated that the action of 5-HT (in the gut) was antagonised by the potent hallucinogen lysergic acid diethylamide. This provoked the notion that 5-HT played a pivotal role in the control of mood and subsequent investigations have generally confirmed this hypothesis. Over the last 50 years a good understanding has been gained of the mechanisms involved in control of the storage, synthesis and degradation of 5-HT in the brain. Knowledge has also been gained on control of the functional activity of this monoamine, often by the use of behavioural models. A considerable literature also now exists on the mechanisms by which many of the drugs used to treat psychiatric illness alter the functional activity of 5-HT, particularly the drugs used to treat depression. Over the last 20 years the number of identified 5-HT receptor subtypes has increased from 2 to 14, or possibly more. A major challenge now is to utilise this knowledge to develop receptor-specific drugs and use the information gained to better treat central nervous system disorders.

General introduction

Accepting the challenge of writing an historical overview is always a risky business, particularly in an area going back over 50 years. The risk of missing significant advances is substantial and it is also unwise to ascribe too much importance to recent publications. Time is required to put new findings into perspective. Albert Sjoerdsma undertook the task of reviewing the history of 5-hydroxytryptamine (5-HT; serotonin) in 1989 at the New York Academy of Sciences meeting entitled ‘The Neuropharmacology of Serotonin' and received barbed ‘questions' at the end of his presentation from other senior scientists who felt he had not given sufficient weight (or worse had ignored) particular scientific advances that held significance to them. As also tends to happen when any group of like-minded scientists get together, small cohorts were subsequently seen muttering over their coffee that they did not share his view on the importance of some specific study.

At least I do not have to expose myself to a live audience in that way when presenting my views in this article. However, the size of the task is formidable. There is a major body of information on the neuropharmacology of 5-HT encompassing its involvement in functions such as diverse as mood, appetite, sleep, sex and temperature and an equally large library on its peripheral actions in modulating cardiovascular function, the gastrointestinal system and peripheral secretory mechanisms. Clearly an overview on all of this was going to be impossible in the designated length and major omissions can doubtless be detected even though I have restricted myself to the neuropharmalogical aspects of 5-HT. This is an area I have been involved with, and enjoyed working on, since I started as a Ph.D. student with Gerald Curzon at the Institute of Neurology in 1966. Furthermore, given the remit of this special issue, the focus has, of necessity, been primarily on the work of U.K. scientists and, where possible, their publications in the British Journal of Pharmacology.

A brief overview on the history of 5-HT

The key work on the isolation and characterisation of serotonin, and its final identification as 5-HT, took place between 1946 and 1949. This final identification allowed the compound to be synthesised by chemists at the laboratories of both the Upjohn and Abbott companies in the U.S.A. and the pure substance (as its salt) made available to pharmacologists around the world. Since that time over 90,000 papers on 5-HT have appeared in print. Since the ‘birth' of the British Journal of Pharmacology occurred in 1946 and publications on serotonin started appearing at approximately the same time, it is not surprising that essentially all the major advances in our knowledge of 5-HT can be followed by reading back issues of this prestigious journal.

Interest in what finally became known as 5-HT had started many years earlier. As long ago as 1868 it was known that the blood contained a vasoconstrictive substance. This substance, released in serum by platelet breakdown, proved to be a problem to Irvine Page in his studies on malignant hypertension so he, together with Arda Green and a recently qualified postdoctoral student Maurice Rapport, isolated and characterised this interfering substance, and named it for its vasoconstrictor properties – serotonin. However, it was Rapport alone who finally identified the substance as 5-HT (Rapport, 1949). It was therefore a particular pleasure to recently meet Maurice Rapport at the Serotonin Club meeting held in Porto as a satellite to the EPHAR (2004) meeting. He gave an elegant lecture (see Figure 1) outlining the discovery of serotonin and it was gratifying to see many young pharmacologists in the audience enjoying learning about the history of a substance on which they were working and for them to perhaps understand that not all major scientific discoveries have been made in the last few years.

Figure 1.

Maurice Rapport presenting to the Serotonin Club satellite conference to the 4th EPHAR meeting, Porto, Portugal, 18th July 2004.

However, Page and colleagues working at the Cleveland Clinic were not the only persons trying to identify 5-HT in the 1940s. In Italy, Erspamer had, since the late 1930s, been investigating a constituent of gastric and enteric mucosa of mammals, and salivary glands of octopus. He had named the compound enteramine and this substance was finally demonstrated also to be 5-HT (Erspamer & Asero, 1952).

Interest in 5-HT in the U.K. rapidly followed publication of its identification and Rapport sent a small sample to John Gaddum in 1950 thereby doubtless helping in the studies that led to the major observation that 5-HT was present in the brain (Amin et al., 1954). Crucially, Gaddum demonstrated that the action of 5-HT in the gut was antagonised by the recently discovered hallucinogen lysergic acid diethylamide or LSD (Gaddum & Hameed, 1954). This fact, coupled with the realisation that the structure of 5-HT is ‘contained' within that of LSD (Figure 2) resulted in a fascination with the role of 5-HT in controlling mood that has never abated. Similarly, its role in peripheral cardiovascular and gastrointestinal systems also continues to be explored with dedication and enthusiasm.

Figure 2.

The structure of 5-HT and lysergic acid diethylamide (LSD).

By the mid-1950s B.B. (Steve) Brodie had established the Laboratory of Chemical Pharmacology at NIH and his genius and drive, coupled with good resources and outstanding staff, resulted in a flood of papers which have proved to be the basis of much of our current knowledge not only about drug metabolism, but also monoamine neuropharmacology. Brodie collected a group of scientists who led the world in monoamine research from the 1950s through to the 1970s, and when Julius Axelrod ‘broke away' from Brodie and established his own section at NIH, he likewise attracted many of the best neuropharmacologists from around the world. Names of those working in these laboratories included Costa, Udenfriend, Sjoesdma, Shore, Snyder, Wurtman and Spector from the U.S.A. and Iversen, Glowinski, Carlsson and Thoenen from Europe. Sweden also established itself as a considerable force in 5-HT research with Carlsson, Andén, Fuxe, Dahlstrom, Hillarp, Hökfelt and Ross not only publishing a substantial number of seminal biochemical pharmacology studies on 5-HT but also, crucially, mapping the 5-HT pathways in the brain. In the U.K. during the 1960s and beyond, Bradley, Grahame-Smith and Curzon established and headed laboratories in which research on 5-HT played a major part. The MRC Brain Metabolism Unit in Edinburgh was also well established at that time and this group, led by George Ashcroft and Donald Eccleston, was producing major preclinical and clinical data on the role of 5-HT in psychiatric disorders. This research area was strong in the U.K., with Merton Sandler, Mike Pare and Alec Coppen also providing much of the evidence that we now take for granted when talking about the pivotal role that 5-HT plays in mental health, and the mechanisms by which antidepressant drugs alter both 5-HT function and mental state. During that period one certainly gained the impression that U.S.A. researchers were concentrating on the involvement of noradrenaline in affective disorders while the U.K. researchers championed the role of 5-HT (see also Iversen, this issue).

In 1971 Grahame-Smith published a major paper demonstrating that a complex behavioural syndrome (now generally called the serotonin syndrome) could be produced in rats by injection of tryptophan and a monoamine oxidase (MAO) inhibitor (Grahame-Smith, 1971a) which was quickly followed by another in the British Journal of Pharmacology reporting that this syndrome was also produced by administering the 5-HT agonist 5-methoxy N,N-dimethyltryptamine (Grahame-Smith, 1971b). This work provoked the realisation that it was not enough to measure 5-HT levels or even the synthesis or turnover rate of the transmitter, because these parameters were not always associated with the functional activity of the amine. The serotonin syndrome established one approach to examine the effect of drugs on the functional activity of 5-HT in the brain, since the effect of compounds on the severity of the syndrome could be quantified. Other behavioural syndromes linked to the function of specific 5-HT receptor subtypes in the brain were subsequently identified and these continue to be used by investigators (see later).

By the late 1970s, however, interest in 5-HT had waned in the U.K. with the ‘hot' neuropharmacological area being dopamine and schizophrenia, assisted by studies on cyclic AMP and ligand–receptor binding. Nevertheless, in 1984 Philip Bradley organised a 1 day symposium on 5-HT as part of the British Pharmacological Society (BPS) meeting in Birmingham (in those days symposia were unusual events at BPS meetings and a full day symposium was rare). Given the apparent low level of research activity at that time I was very surprised to find a large lecture theatre packed with an attentive audience and therefore asked Les Iversen why he thought there was such a sudden upsurge in interest in 5-HT. He professed that he did not know (although, in retrospect, I wonder if the fact that he was just establishing new research laboratories for Merck, Sharp and Dohme meant that he was just not letting on). Bizarrely I should have known, since my talk had touched on the reason without my realising it. The humorous ‘social minutes' of the Birmingham meeting read by Tony Birmingham, the General Secretary, said it all: “We thought it (the meeting) got off to a good start with a provocative symposium on 5-HT. There was plenty of disagreement about how to classify 5-HT receptors. It was not actually made any easier when Richard Green tried to kid us there were actually A, B and C subtypes of 5-HT₁ receptor, but we weren't having any”. During the late 1970s and into the 1980s ligand–receptor-binding studies had indicated the existence of several 5-HT receptor subtypes and by the time of the meeting most of the major pharmaceutical companies were working hard to identify and characterise these subtypes and develop receptor-specific drugs. Major players in the U.K. were Glaxo, Beecham and Wellcome (all then different companies of course) and Sandoz and Merrell Dow in Europe. So that explained the audience size.

Over the following years many papers appeared on the identification and pharmacology of these receptor subtypes and several drugs resulting from this research have found success in the market. At present 5-HT research is in a quieter phase again, although knowledge still increases as new receptors or specific compounds become available and functional correlations of the receptor subtypes are identified. We have seen two explosive increases in interest in 5-HT during the late 1950–1960s and during the 1980s. Are we perhaps now ready for another?

In the following sections, I have produced headings which make for ease of writing. However, it should be remembered that discoveries in different areas were often being made in parallel, so work arbitrarily placed in one section doubtless assisted in the development of research ideas reported in another section.

On the measurement of 5-HT

Early research on 5-HT used bioassay for analysis, primarily the isolated oestrus uterus (Amin et al., 1954). John Vane developed a new method using the rat stomach strip which was both more reliable and more sensitive (Vane, 1957) and he later used this method to examine the relative potency of tryptamine analogues (Vane, 1959).

However, bioassay is not amenable to the rapid measurement of the 5-HT concentration in the brain of a large number of rats pretreated with psychoactive compounds. Robert Bowman, working with the American Instrument Company, improved the instrument previously used by Brodie and Udenfriend during the war to measure Atabrine and the result was the famous and commercially available Aminco–Bowman spectrophotofluorimeter. This instrument allowed Brodie, Shore and Udenfriend to develop an assay for cerebral 5-HT based on the fact that the indole fluoresced in a strongly acid solution. This method was used extensively until the mid-1960s when a far more sensitive method was published by Snyder, Axelrod and Zweig which measured the very highly fluorescent complex produced by reacting 5-HT with ninhydrin.

In 1969 Roger Maickel and colleagues published a new fluorescent method for measuring 5-HT in the brain which detected the fluorescent product of 5-HT reacted with o-phthaldehyde. This paper was soon followed by a report by Jacob Korf describing the measurement of the 5-HT metabolite 5-HIAA in urine, also by reacting it with o-phthaldehyde. Gerald Curzon arrived at my bench one morning with the Korf paper and said, “Since the Maickel method extracts only 5-HT from the organic phase into the acidic aqueous phase in order to measure it, it stands to reason that the 5-HIAA is left behind. If we shake the organic phase again with a neutral buffer we should be able to get the 5-HIAA out and therefore be able to measure that in the same tissue. Why don't we try?” We did, and found it was possible to accurately measure both 5-HT and 5-HIAA in small regions of rat brain. This work was written up as a short communication (three pages) for the British Journal of Pharmacology (Curzon & Green, 1970). A satisfying, but nevertheless derivative, piece of work. It was therefore an unexpected pleasure about 8 years later to find the paper reviewed in the front of Current Contents as a ‘Citation Classic' having been cited (and therefore presumably used) in nearly 1000 papers in the intervening period. However, by the late 1970s a new technique was rapidly gaining acceptance, that of high-performance liquid chromatography (HPLC) and Charles Marsden was able to demonstrate that this method not only gave similar results to the fluorimetric method but did so with both greater speed and sensitivity (Marsden, 1981). This method has reigned supreme ever since, and is still regarded as the standard approach for analysing brain monoamines.

The synthesis and metabolism of 5-HT

5-HT was known to be synthesised from tryptophan early on in its existence and the sequence of steps shown to be hydroxylation followed by decarboxylation. However, there was argument as to whether it was hydroxylated by tyrosine hydroxylase or a specific tryptophan hydroxylase enzyme, a problem that had engaged Udenfriend and colleagues at NIH for some time. It was of great surprise to them therefore when an ‘unknown' U.K. scientist David Grahame-Smith published a full characterisation of the specific tryptophan hydroxylase enzyme, having extracted it from both carcinoid tumour tissue and brain (Grahame-Smith, 1964). David recalls being asked by one of the NIH researchers “How the hell did you do this when we have not been able to?” Shortly thereafter, Koe & Weissman (1966) published a seminal paper on the specific tryptophan hydroxylase inhibitor p-chlorophenylalanine (PCPA). Since the hydroxylation step is rate limiting, administration of PCPA can decrease the cerebral 5-HT content by 80% or more. However, giving this compound resulted in little change in the overt behaviour of rats. This resulted in Udenfriend, on hearing these results in a lecture by Ken Koe at the 1966 FASEB meeting, going up to Maurice Rapport and saying “You know, serotonin doesn't do anything”. The Koe and Weissman data produced the same response in many others. For example, Gerald Curzon returned from a major meeting on 5-HT in New York at which Koe had presented data on PCPA, and remarked to me “It's amazing, these people have given rats a new compound which depletes brain 5-HT by 70% and there is no change in the animals at all”. Of course it wasn't quite that simple and further studies did reveal both subtle changes and also evidence that a small pool of 5-HT remains functionally active; but at the time the data were a considerable shock to all pharmacologists working on 5-HT.

The realisation that tryptophan hydroxylase is not saturated with its substrate and that if a tryptophan load is given to a rat much more 5-HT is synthesised in the brain resulted in two further major branches of 5-HT research. The first branch was a series of studies, primarily by Wurtman and his group at M.I.T. and Curzon in London on the control of cerebral tryptophan concentration. It was found that large neutral amino acids (tyrosine, phenylalanine, leucine, isoleucine and valine) competed with tryptophan for entry. Since tryptophan is the least abundant available amino acid, these observations led to the realisation that a dietary drink rich in tyrosine and low in tryptophan could be used to rapidly decrease the concentration of 5-HT in the rat brain. The rapid tryptophan depletion technique has been used extensively to investigate the action of 5-HT and drugs on mood particularly by Simon Young at McGill University and Philip Cowen at Oxford University (Smith et al., 1997).

The second branch was that of investigating how 5-HT function is controlled in the brain. Despite the fact that a tryptophan load markedly increases cerebral 5-HT content and synthesis rate, animals administered tryptophan show few overt behavioural changes. However, administration of a MAO inhibitor with the tryptophan results in a complex behavioural syndrome, often called the ‘serotonin syndrome' with several clear behaviours being seen, including head weaving and reciprocal forepaw treading. This work was first detailed by Grahame-Smith (1971a, 1971b) in his studies on the serotonin syndrome and from these he deduced that 5-HT function was probably controlled, at least in part, by intracellular metabolism of 5-HT by MAO. Crucially he realised that this indicated that measuring 5-HT concentration or synthesis rate was not sufficient to deduce whether function is altered. He also realised that we now had a ‘handle' on examining how psychoactive drugs altered 5-HT function by examining the effect of drugs on the syndrome. This led to a series of investigations in his laboratory in Oxford in the 1970s and beyond (Green & Grahame-Smith, 1976).

The existence of an enzyme in the body that degrades amines had been known for nearly 100 years (see also Youdim & Bakhle, this issue). The actual characterisation of MAO was also first performed a surprisingly long time ago – 1928, by Mary Hare (Bernheim) as part of her graduate studies at Cambridge University. However, a good understanding of its role in metabolising monoamines is the result of the work of one of the great U.K. biochemical pharmacologists, Hugh Blaschko. In the 1970s the discovery that there were two forms of the enzyme (MAO_A and MAO_B) with partial substrate specificity led to the development of clorgyline (a compound with MAO_A specificity) and deprenil (MAO_B specific) and the hope that deprenil administration would not induce the ‘hypertensive crisis' or ‘cheese reaction' which results from the inhibition of tyramine metabolism. This proved to be the case, but the antidepressant activity of the selective drugs is weak, possibly because inhibition of one form of the enzyme results in the amine continuing to be metabolised, at least in part, by the other form of the enzyme. When Moussa Youdim and I presented these data at a Ciba Foundation meeting (Green & Youdim, 1976) Alfred Pletcher (then head of research at Hoffman La Roche) remarked, “Wonderful, the industry spends millions making selective drugs and now you show that they work best when we mix them back together”.

Receptor subtype identification and classification

The first definitive identification of 5-HT receptor subtypes was by Gaddum & Picarelli (1957), although the proposal that 5-HT possessed two receptor subtypes was first suggested in an earlier paper by Gaddum & Hameed (1954) and based on some of their experimental observations. Gaddum's classification was the D- and M-receptor, named because of the sensitivity of the receptor subtypes to dibenzyline and morphine as blockers. However, his proposal was generally accepted as being relevant only to peripheral receptors rather than those in the brain. In the 1970s ligand–receptor-binding techniques were developing rapidly and in 1976 a seminal paper was presented to a meeting of the BPS by Gilles Fillion that formed the basis of a major new research initiative on 5-HT receptor identification and function. This study examined the high-affinity binding of [³H]-5-HT and [³H]-LSD to brain synaptosomes and found the binding characteristics differed (Fillion et al., 1976). Full details of this work was subsequently published elsewhere (Fillion et al., 1977) and work by this group and others, including Nelson, Hamon, Yamamura and Snyder finally led to the work of Peroutka et al. (1981) which demonstrated that there was a reasonable correlation between the potency of drugs that bind to the 5-HT₂ receptor (³H-LSD binding) and their ability to inhibit 5-HT-induced head shake behaviour in mice. This study also confirmed a correlation between the ability of drugs to inhibit 5-HT₁ receptor (³H-5-HT) binding and the inhibition of 5-HT-sensitive adenylate cyclase activity.

The 5-HT₁ receptor was then rapidly further subdivided into 5-HT_1A, 5-HT_1B, 5-HT_1C and 5-HT_1D. The situation rapidly became somewhat anarchic, with further 5-HT receptor subtypes being identified and named by groups of workers, often with little regard to the findings of others. Even the naming schemes became diverse; for example, 5-HT₂ and S₂ both being used at one time.

The symposium held by the BPS in Birmingham in 1984 (see earlier) identified some of these problems and a group of interested individuals, all members of the Society, made the first attempt to bring together the classification of Gaddum and Picarelli and that of Fillion, Peroutka and Snyder. They also tried to accommodate the new data that were appearing and integrate all the information into one coherent and accurate whole. This first attempt at classification was published about 2 years later (Bradley et al., 1986). This equated the D-receptor with the 5-HT₂ receptor site, introduced the term 5-HT₃ receptor (which was suggested to be the old M-receptor) and further recognised subgroups within the 5-HT₁ site. This classification rapidly became out of date as the speed with which further papers on 5-HT receptor subtypes appeared gathered pace. A year later, following an initiative by the BPS, IUPHAR took up the problem and established a nomenclature subcommittee to deal with the classification problem of 5-HT receptor subtypes.

This subcommittee involved both the BPS and several of its members and also members of the newly formed ‘Serotonin Club'. In 1987, at its first meeting in Australia, several members of the Serotonin Club who had been involved in the Bradley et al. (1986) report met and formed a ‘Nomenclature Committee' (Figure 3). This Committee was one of the first subcommittees recognised by IUPHAR and Figure 3 shows the group making estimates as to how many subtypes the members felt might exist. Ewan Mylecharane with 10 was nearest, since this number was identified around 1990. The number has subsequently risen to 14. This nomenclature sub-committee still exists, although its activities have lessened substantially since the heady and fast moving days of the late 1980s and early 1990s. During that time, for example, the 5-HT_1C receptor was ‘moved' to become a 5-HT₂ receptor subtype since it is coupled positively to phospholipase C, as are other 5-HT₂ receptors, rather than coupled negatively to adenylate cyclase like the rest of the 5-HT₁ family. Interestingly, all the 5-HT receptors are G-protein-coupled receptors with the single exception of the 5-HT₃ receptor which is a ligand gated ion channel receptor (Figure 4).

Figure 3.

Some members of the plenary ‘Workshop on 5-HT Receptor Classification' chaired by Pat Humphrey and Brian Richardson following dinner at the first Serotonin Club satellite meeting to the 1987 IUPHAR Congress. The satellite meeting was held on Heron Island, Queensland, Australia. Pictured (left to right) are: Günter Engel (Sandoz, Basle), Toshiro Shibano (Daiichi Pharmaceutical, Tokyo), Brain Richardson (Sandoz, Basle), Ewan Mylecharane (University of Sydney), Stephen Peroutka (Stanford University, Ralph Purdey (University of California, Irvine), John Fozard (Sandoz, Basle) and Pat Humphrey (Glaxo, Ware). Missing is Pramod Saxena. Members are indicating the number of 5-HT receptor subtypes they believed existed (Purdey is suggesting an action at the α-adrenoceptor). Company names are given as they then existed. The elected ‘Nomenclature Committee' was Patrick Humphrey (Chairman), Jack Peter Green (Vice-Chairman), George Aghajanian, Philip Bradley, Marlene Cohen, John Fozard, Josée Leysen, Ewan Mylecharane, Stephen Peroutka, Brian Richardson and Pramod Saxena. I thank Ewan Mylecharane for providing this photograph.

Figure 4.

Current 5-HT receptor subtype classification scheme. Lower case designates that receptors that have not been demonstrated to definitively function in native systems. cAMP, 3′–5′ cycle adenosine monophosphate; PLC; phospholipase C; −ve, negative; +ve, positive. Reprinted from Hoyer et al. (2002) with permission of Elsevier Ltd.

In 1993 a new classification system was published by the Serotonin Club Nomenclature Committee (Humphrey et al., 1993) and this work was later presented in detail elsewhere (Hoyer et al., 1994). It is worth noting that many of the discussions of this group were conducted in the ‘Friend in Hand' and other pubs near Russell Square in London during the Winter meetings of the BPS which were being held nearby at the Institute of Education.

The nomenclature system was later modestly adapted further in order to align it with the human genome (Hoyer & Martin, 1997). Where ‘molecular' identification had been made without functional identity a lower case designation was used (e.g. 5-ht_5A). Historically, receptor classification had been functional (the acetylcholine receptor and the adrenergic receptor are both good examples). Not only was this not easily possible with all 5-HT receptor subtypes in the absence of selective agonists and antagonists and, crucially, a defined functional response, but the advent of knowledge about transduction systems and genomics was rapidly producing other information. Hoyer and Martin therefore followed the precedent of earlier attempts to classify the receptors by utilising operational, transactional and structural characteristics, none of which had precedence.

Heroic attempts to bring all the receptor classification and function into a coherent whole have recently been published by both Barnes & Sharp (1999) and Hoyer et al. (2002). The latter review elegantly covers all the key findings in only 20 pages, although this is perhaps not surprising as Daniel Hoyer has been at the forefront of receptor subtype identification and classification for over 20 years. The current classification scheme was also recently published in convenient ‘shorthand' form in the British Journal of Pharmacology (Alexander et al., 2004).

New receptors, new drugs

The fact that tricyclic antidepressants inhibited monoamine uptake was established in the late 1960s and structure activity relationships reviewed soon after (Horn & Trace, 1974). Langer and colleagues, using [³H]-imipramine binding demonstrated that this drug bound to a presynaptic site on the 5-HT nerve ending and presented these data to meetings of the BPS in the early 1980s. This ligand was soon succeeded by the more specific [³H]-paroxetine. However, this research resulted in the identification of a new receptor that was being acted on by an established drug and while it was seen as an important finding, it was clearly the hope of many researchers that the recently discovered 5-HT receptor subtypes would allow the discovery and synthesis of novel selective compounds to provide new therapeutic approaches. In fact the discovery of possible new receptors rapidly outpaced both the ability of investigators to ascribe new functions to them and also the skills of the medicinal chemists to synthesise selective compounds which would assist in characterising their function. Nevertheless, the discovery process was assisted by the realisation that certain existing compounds did have selectivity for the newly discovered receptor subtypes. For example, the characterisation of the 5-HT_1A receptor was enhanced markedly by the realisation that 8-OH-DPAT acted at this receptor (Middlemiss & Fozard, 1983) and that this compound produced functional changes in both behaviour and temperature (Goodwin & Green, 1985). Since the 5-HT_1A receptors are somatodendritic and act as autoreceptors and are also located postsynaptically, drugs acting on this receptor can markedly alter 5-HT function. At present only the partial 5-HT_1A agonist buspirone has achieved clinical use (as an anxiolytic). However, a pure 5-HT_1A antagonist (WAY 100635) was subsequently discovered at the Wyeth Laboratories in the U.K. and the labelled version of this compound has proved valuable in clinical PET studies.

The pharmacology of the 5-HT_1B receptor is complicated in that there is a rat and human homologue. It is an autoreceptor and probably also a heteroceptor. It is currently the subject of research as modification of its activity could prove useful in the treatment of certain psychiatric conditions and SmithKline Beecham (now GlaxoSmithKline) scientists including Blackburn, Middlemiss and Price have played major roles in these studies.

The triptans, with sumatriptan being the prototypical compound, are primarily 5-HT_1B/5-HT_1D agonists and have achieved notable success clinically in treating migraine. However, again the groundwork for this success was performed over many years before final receptor identification, notably by Pat Humphrey and colleagues in Glaxo. His team had been publishing (primarily in the British Journal of Pharmacology) on vascular receptors over many years prior to receptor characterisation (Humphrey et al., 1988).

The 5-HT₂ receptor was reported to be associated with head twitch behaviour in mice by Peroutka et al. (1981) and Janssen Pharmaceutica research made available the first 5-HT₂ antagonists including pirenperone, ketanserin and ritanserin, which confirmed this association. Following the identification of 5-HT₂ receptor subtypes it was further shown that it was the 5-HT_2A receptor that was the D-receptor as described by Gaddum & Picarelli (1957). Recently there had been considerable research on all of the 5-HT₂ receptor subtypes; particularly attempts to understand their pharmacology and their involvement in feeding behaviour, anxiolysis and schizophrenia (the efficacy of the new antipsychotics such as quetiapine may involve antagonism at 5-HT_2A and dopamine D₂ receptors).

The 5-HT₃ receptor is the M-receptor of Gaddum & Picarelli (1957) and is present in both brain and periphery. Interestingly, Rocha e Silva et al. (1953) first showed that the action of 5-HT in the gut could be antagonised by cocaine, and over 20 years later studies by John Fozard, who combined this observation with others on the antagonist action of metaclopramide, resulted in Merrell Dow producing MDL 72222, the first potent and selective 5-HT₃ antagonist. Research at Beecham (BRL24924 and granisetron) and Sandoz (ICS205–930, tropisetron) also resulted in compounds of somewhat related structure. In contrast, the Glaxo compound ondansetron, which was produced and characterised by a team led by Mike Tyers, had a totally unrelated molecular structure. Tropisetron was the first compound to be shown to be antiemetic in clinical studies and this important property has resulted in these drugs being used extensively as adjuncts to chemotherapy or radiation therapy. The development of ondansetron also involved work by Robert Naylor, Brenda Costall and their colleagues at the University of Bradford (Costall et al., 1987) and for a while their research suggested that 5-HT₃ antagonists might also prove valuable in therapeutic areas totally unrelated to the antiemetic action of the drug. Much research was presented to meetings of the BPS on the effect of this compound in animal models of anxiety, cognitive impairment and drug abuse. The clinical trials however failed to confirm the preclinical promise of the 5-HT₃ antagonists in these therapeutic areas.

The predominant interest in the 5-HT₄ receptor has been in modification of gastro-intestinal function. The 5-HT₄ agonist cisapride is a gastro-prokinetic agent and the more selective compound tegaserod is used for the treatment of irritable bowel syndrome and constipation. The possibility of modifying the activity of the 5-HT₄ receptor for the treatment of neurodegenerative diseases has also been proposed in a report on the distribution of this receptors in the human brain (Reynolds et al., 1995).

The 5-HT₆ and 5-HT₇ receptors are still being characterised although evidence is evolving that suggests a possible role for 5-HT₆ receptors in cognition and 5-HT₇ in sleep. The activities of researchers at Glaxo SmithKline at Harlow who have been publishing on receptor selective drugs in the British Journal of Pharmacology over the last few years should assist in further clarification of the role of these receptors.

5-HT and behaviour

As noted above, a variety of behavioural effects can be monitored following administration of 5-HT agonist compounds and these actions have been usefully exploited over the years to examine the effect of drugs on 5-HT function (Goodwin et al., 1985). However, equally important is information about the role of 5-HT in various animal models of psychopathology such as anxiety, stress and depressive-like behaviour. Interpretation of data was strengthened if 5-HT function was either modified or monitored. The discovery of the selective neurotoxin 5,7-dihydroxytryptamine by Baumgarten and colleagues in the early 1970s undoubtedly assisted in this process since lesions of 5-HT terminals could be placed in select regions of the brain. The development of techniques to measure extracellular 5-HT in the freely moving animal added a further dimension and Charles Marsden and colleagues in Nottingham University pioneered these techniques in the U.K., using in vivo voltammetry and subsequently in vivo microdialysis. This latter technique has the advantage of allowing absolute identification of the released neurotransmitter. More recently this group has been combining microdialysis with behavioural measures to provide a powerful broad spectrum of information about the action of drugs on 5-HT function in the brain (Mundey et al., 1996).

The role of 5-HT in the action of drugs used in psychiatry

The development of the idea that 5-HT has a major role in the action of antidepressant treatments is an interesting amalgam of information being obtained from pharmacology, clinical biochemistry and psychiatry together, it must be admitted, with a generous helping of serendipity.

During the 1950s reports appeared that iproniazid, a drug used to treat tuberculosis was producing behavioural activation, and in 1955 Zeller showed that the drug to be a MAO inhibitor. Crane proposed that this effect of iproniazid, while undesirable in the treatment of tuberculosis, might be valuable in the treatment of depression. It had also been noted that reserpine, a drug used to treat hypertension, could precipitate depression. Following evidence that this compound depleted brain monoamines, the simple idea gained credence that increased monoamine levels (including 5-HT) in the brain produced euphoria and decreased levels produced depression.

At the same time imipramine (a drug structurally related to chlorpromazine) was being investigated in psychiatric patients (schizophrenics and depressives) and it was noted that it was only the latter patients who improved. Todrick and colleagues had noted the opposite actions of imipramine and iproniazid in altering platelet 5-HT levels (Marshall et al., 1960) and (subsequent to the large number of structurally related tricyclic compounds being developed and assessed) evidence accumulated on the fact that most tricyclics inhibited 5-HT uptake at the nerve ending (Horn & Trace, 1974). This indicated that 5-HT function was increased by the tricyclics because of their ability to increase the synaptic cleft concentration of the neurotransmitter. In turn this fitted in with the idea of increased 5-HT levels being antidepressant and decreased levels being mood lowering. Further credence to this idea was gained by clinical evidence for the antidepressant efficacy of zimeldine and fluoxetine, two potent and selective 5-HT uptake inhibitors. The idea that 5-HT ‘levels' in the brain control mood can now be read frequently in articles in newspapers and popular magazines, despite the fact that evidence over the last 25 years has provided no support for such a simple notion. However, the heuristic value of the original concept cannot be overestimated as all subsequent research has continued to focus on the involvement of 5-HT function in affective disorders.

Problems arose with the original idea with the development of drugs that did not obviously alter monoamine ‘levels' and the realisation that 5-HT uptake was inhibited very rapidly (minutes, hours) while a clear antidepressant effect often took weeks to manifest itself. From this developed ideas that adaptation was occurring at 5-HT receptors either because of the raised monoamine concentration or a direct action of some of the drugs or that there was a complex interplay between some of the receptor subtypes. All these ideas are probably true as is the idea that focussing on one neurotransmitter may have limitations. This last point has resulted in a new generation of drugs that have moved from the selective serotonin reuptake inhibitors (SSRI) like fluoxetine back to the value of the joint serotonin, noradrenaline reuptake inhibitors (SNRI) compounds such as venlaxafine. It is notable that the top 5 selling CNS drugs in 2000 all modulate 5-HT function (Jones & Blackburn, 2002).

What has been valuable for research in this area has been the way that many of the current senior clinical psychopharmacologists in the U.K. have undertaken preclinical research either during their earlier training or in parallel with their clinical studies. Consequently research studies on 5-HT in laboratory animals have often resulted in hypotheses then being examined in clinical studies, and sometimes back again to the preclinical laboratory. One example, given earlier, is the way that studies on the role of tryptophan in 5-HT synthesis and function developed into investigations on the effect on mood of rapid tryptophan depletion (Smith et al., 1997). Another example is the studies conducted on the role of the 5-HT_1A receptor. Early studies established the biochemical action of 8-OH-PAT at the 5-HT_1A autoreceptor and in producing certain 5-HT mediated behaviour in rodents. Certain nonselective β-adrenoceptor antagonists such as propranol and pindolol had been shown to block 5-HT-mediated behaviour (Costain & Green, 1978), including behaviour induced by 8-OH-DPAT. The hypothesis that the use of pindolol to block the autoreceptor to enhance the action of SSRI was developed, and electrophysiology and microdialysis data in rats confirmed the validity of the proposal (Gartside et al., 1999). However, clinical trials on depressed patients treated with SSRI and pindolol produced conflicting data on efficacy and the use of PET and [¹¹C]-WAY1000635 demonstrated that the dose of pindolol given was unlikely to have produced significant 5-HT_1A receptor occupancy in the human brain (Rabiner et al., 2000).

A role for 5-HT has not only been indicated in affective disorders. Its role in schizophrenia, anxiety disorders and obsessive compulsive disorders has also been recently reviewed (Jones & Blackburn, 2002) although the involvement of other neurotransmitter systems makes interpretation of data difficult and clear evidence for the efficacy of drugs acting solely on 5-HT functions is lacking.

The future of 5-HT research

The speed with which biomedical research is advancing makes any attempt to predict the future a foolhardy exercise. In just over 10 years we went from knowing about perhaps two 5-HT receptors in the brain to 12–14 and much of this was due to molecular biological techniques. However, it seems unlikely that such an explosion of information will occur again and we still await a good understanding of the action of many of these receptors. However, this knowledge will come with the availability of receptor-selective drugs. What is perhaps demoralising is our inability to use all the new information to treat psychiatric illness, particularly depression more effectively. The key involvement of 5-HT in this illness had long been known, but our pharmacological treatment of it has improved only modestly over the last 40 years. The current drugs are much better than the old tricyclic compounds in terms of producing far fewer adverse effects (although sometimes one would not think so when reading newspapers, particularly when they focus on problems caused by use of the SSRI compounds). The newer compounds are also much safer in overdose. However, in terms of the percentage of patients who are resistant to treatment, this figure stubbornly refuses to decrease markedly, particularly when treating the elderly. One has to hope that consolidation of existing knowledge and the new drugs that will come out of that knowledge will improve the treatment of psychiatric patients in the next few years.

Glossary

BRL24924

renzapride

5-HIAA

5-hydroxyindole acetic acid

LSD

lysergic acid diethylamide

MDL 72222

3-tropanyl-3,5-dichlorobenzoate

8-OH-DPAT

8-hydroxy-2-(di-n-propylamino)tetralin

PCPA

p-chlorophenylalanine

SNRI

serotonin noradrenaline reuptake inhibitor

SSRI

selective serotonin reuptake inhibitor

WAY100635

N-[2-[4-(2-methoxyphenyl)-1-piperazinyl]ethyl]-N-(2-pyridinyl) cyclohexane carboxamide.3HCl