Structural studies of serotonin receptor family

Abstract

Serotonin receptors, also known as 5-HT receptors, belong to the G protein-coupled receptors (GPCRs) superfamily. They mediate the effects of serotonin, a neurotransmitter that plays a key role in a wide range of functions including mood regulation, cognition and appetite. The functions of serotonin are mediated by a family of 5-HT receptors including 12 GPCRs belonging to six major families: 5-HT₁, 5-HT₂, 5-HT₄, 5-HT₅, 5-HT₆ and 5-HT₇. Despite their distinct characteristics and functions, these receptors’ subtypes share common structural features and signaling mechanisms. Understanding the structure, functions and pharmacology of the serotonin receptor family is essential for unraveling the complexities of serotonin signaling and developing targeted therapeutics for neuropsychiatric disorders. However, developing drugs that selectively target specific receptor subtypes is challenging due to the structural similarities in their orthosteric binding sites. This review focuses on the recent advancements in the structural studies of 5-HT receptors, highlighting the key structural features of each subtype and shedding light on their potential as targets for mental health and neurological disorders (such as depression, anxiety, schizophrenia, and migraine) drugs.

INTRODUCTION

G protein-coupled receptors (GPCRs) are the largest family of cell surface receptors in the human genome with more than 800 members. The general structure of the GPCRs consists of seven hydrophobic transmembrane (TM) domains connected by alternating extracellular and intracellular loops. The N-terminus is located on the extracellular side of the cell while the C-terminus is located on the intracellular side. There are five different classes of GPCRs in humans based on the sequence phylogeny of a conserved seven transmembrane domain (7TM): class A (rhodopsin), class B1 (secretin), class B2 (adhesion), class C (glutamate), and class F (frizzled/smoothed) (1-3). Upon ligand binding, GPCRs initiate the transmission of signals across the cell membrane, leading to intracellular responses through the activation of intracellular heterotrimeric G proteins, G protein-receptor kinases, arrestins and other signaling pathways (4).

Serotonin is a neurotransmitter responsible for modulating mood, cognition, perception, memory, sleep, appetite, anxiety, gut motility and blood clotting. It is primarily present in the central nervous system (CNS), gut and blood platelets (5). Its functions are mediated by a family of serotonin receptors (5-HTR), including 12 GPCRs, and one ligand-gated cation channel (Table 1). With the exception of 5-HT₃R, which is a ligand-gated ion channel, all the other 5-HT receptors (5-HT₁, 5-HT₂, 5-HT₄, 5-HT₅, 5-HT₆ and 5-HT₇) are members of GPCRs superfamily. The human CNS contains all the serotonin receptors except 5-HT_2BR (6). A comparison of human G-protein coupled 5-HT receptors’ sequences shows strong conservation, with 33 residues fully conserved and 27 residues conserved up to 80%. These conserved residues are primarily located in the regions of the receptors that participate in ligand binding and G-protein interaction. This conservation highlights the critical role of these regions in mediating ligand binding and facilitating the signaling process through G-protein coupling (7).

Table 1.

Serotonin receptor structure

Serotonin receptor family	G protein coupling	PDB	Method	Ligand	Reference
5-HT1A	Gi/Go	7E2X	Cryo-EM	Apostate	(24)
		7E2Y	Cryo-EM	5-HT
		7E2Z	Cryo-EM	Aripiprazole
5-HT1B	Gi/Go	4IAQ	X-ray	Dihydroergotamine	(25)
		4IAR	X-ray	Ergotamine
		5V54	X-ray	Methiothepin	(27)
		6G79	Cryo-EM	Donitriptan	(26)
		7C61	X-ray	Ergotamine	(57)
5-HT1D	Gi/Go	7E32	Cryo-EM	5-HT	(24)
5-HT1E	Gi/Go	7E33	Cryo-EM	BRL-54443
5-HT1F	Gi/Go	7EXD	X-ray	Lasmiditan	(29)
5-HT2A	Gq	6A93	X-ray	Risperidone	(36)
		6A94	X-ray	Zotepine	(36)
		7WC4	X-ray	5-HT	(58)
		7WC5	X-ray	Psilocin
		7WC6	X-ray	LSD
		7WC7	X-ray	Lisuride
		7WC8	X-ray	Lumateperone
		7WC9	X-ray	IHCH-7086
		6WHA	Cryo-EM	25-CN-NBOH	(59)
		7RAN	Cryo-EM	(R)-69	(60)
5-HT2B	Gq	5TUD	X-ray	Ergotamine	(61)
		7TVN	X-ray	LSD	(39)
		6DRX	X-ray	Lisuride	(40)
		6DRY	X-ray	Methylergonovine
		6DRZ	X-ray	Methysergide
		6DS0	X-ray	LY266097
		4IB4	X-ray	Ergotamine	(16)
		7SRQ	Cryo-EM	LSD	(17)
		7SRR	Cryo-EM	LSD
		7SRS	Cryo-EM	LSD
5-HT2C	Gq	6BQG	X-ray	Ergotamine	(38)
		6BQH	X-ray	Ritanserin
		8DPF	Cryo-EM	Lorcaserin	(41)
		8DPG	Cryo-EM	Psilocin
		8DPH	Cryo-EM	Lorcaserin
		8DPI	Cryo-EM	Lorcaserin
5-HT4	Gi	7XTA	Cryo-EM	5-HT	(14)
	Gs	7XT8	Cryo-EM	5-HT
		7XT9	Cryo-EM	5-HT
5-HT5A	Gi/Go	7UM4	X-ray	AS2674723	(47)
		7UM5	Cryo-EM	5-CT
		7UM6	Cryo-EM	Lisuride
		7UM7	Cryo-EM	Methylergometrine
		7X5H	Cryo-EM	5-CT	(46)
5-HT6	Gs	7XTB	Cryo-EM	5-HT	(14)
		8JLZ	Cryo-EM	ST1936	(62)
5-HT7	Gs	7XTC	Cryo-EM	5-CT	(14)

Studying the structure, functions and effects of serotonin receptors is essential for understanding how serotonin functions in the brain and developing treatments for mental health disorders such as depression, anxiety, schizophrenia, and migraines (8, 9). However, developing drugs that selectively target specific receptor subtypes is challenging due to the structural similarities in their orthosteric binding sites. There are ongoing extensive research efforts to elucidate the precise mechanisms of action of each receptor subtype and explore their potential as drug targets for improved management of mental health and neurological disorders. The goal of this review is to present a detailed summary of the current understanding of the serotonin receptors’ structure based on available studies. It will discuss the structural characteristics of different serotonin receptor subtypes, the binding sites for their ligands, G-protein interactions, and the changes that occur in their conformation when the receptors are activated.

STRUCTURES OF 5-HT RECEPTOR FAMILY

The general architecture of 5-HT receptors follows the canonical class A GPCR structure characterized by an amino terminus located externally, seven TM helices with hydrophobic regions, three extracellular and three intracellular domains, and a carboxy terminus positioned intracellularly (4). Residues within the TM domains bind to the receptors, whereas the intracellular domains interact with various cytoplasmic proteins to initiate the downstream signaling pathways (10). Despite the TM domains being fairly comparable, with minor variations in tilt and twist, there is significant diversity observed in the loops, particularly in extracellular loop 2 (ECL2). The conserved residues are identified using the Ballesteros-Weinstein numbering system which serves as an indicator of the correct residue position in the transmembrane of the GPCR family (11). The 3D structures of all 12 serotonin receptors have been determined using X-ray crystallography and cryo-electron microscopy (cryo-EM) (Table 1 and Fig. 1).

Fig. 1.

Structures of all 12 G-protein coupled serotonin receptors. Cryo-EM maps of (A) 5-HT1 family: aripiprazole bound 5-HT_1A (PDB ID: 7E2Z), donitriptan bound 5-HT_1B (PDB ID: 6G79), 5-HT bound 5-HT_1D (PDB ID: 7E32), BRL-54443 bound 5-HT_1E (PDB ID: 7E33), Lasmiditan bound 5-HT_1F (PDB ID: 7EXD), (B) 5-HT₂ family: 25-CN-NBOH bound 5-HT_2A (PDB ID: 6WHA), LSD bound 5-HT_2B (PDB ID: 7SRR), LSD bound 5-HT_2B in complex with β-arrestin (PDB ID: 7SRS), Lorcaserin bound 5-HT_2C (PDB ID: 8DPI) and (C) 5-HT_4/5/6/7 family: 5-HT bound 5-HT₄ (PDB ID: 7XT8), 5-CT bound 5-HT₅ (PDB ID: 7UM5), 5-HT bound 5-HT₆ (PDB ID: 7XTB), and 5-CT bound 5-HT₇ (PDB ID: 7XTC).

The classification of 5-HT receptors can also be based on their coupling with specific G proteins (Table 1). For instance, the 5-HT₁R and 5-HT₅R subtypes couple with G_i/o protein, leading to the inhibition of adenylyl cyclase (AC) activity and a reduction in cyclic adenosine monophosphate (cAMP) levels (12). On the other hand, 5-HT₂R couples with G_q/11 protein, stimulating phospholipase C, which leads to elevated levels of inositol triphosphate and calcium ions (Ca²⁺) (13). The 5-HT₄, 5-HT₆ and 5-HT₇ receptors couple with the G_s protein to enhance AC activity and elevate cAMP levels. Notably, 5-HT₄R can bind to both G_s and G_i proteins with preference for G_s over G_i. Each subtype has unique characteristics and functions, although they share common structural features and signaling mechanisms (14). Apart from the typical coupling with G proteins, GPCRs have been discovered to participate in alternative signaling pathways through the recruitment of arrestins which results in GPCR desensitization. GPCR desensitization involves GPCR kinases (GRKs) being recruited to the receptor, subsequently phosphorylating it, and then β-arrestin binding, which sterically hinders further G protein coupling and leads to desensitization (15). Biased signaling is usually observed in 5-HT₂R, where certain ligands selectively activate specific signaling pathways (16, 17).

5-HT₁ RECEPTOR SUBFAMILY

The 5-HT₁ receptor subfamily comprises 5-HT_1A, 5-HT_1B, 5-HT_1D, 5-HT_1E and 5-HT_1F receptor subtypes. The subtypes are predominantly inhibitory in nature and are involved in regulating neurotransmitter release and neuronal activity. The 5-HT₁ receptors play a role in mood regulation, anxiety, pain perception and are a target for migraine, depression, and schizophrenia therapeutic agents (6). In particular, the 5-HT_1AR regulates mood, anxiety, cognition, sleep and pain perception in the brain, making it a potential target for therapeutic agents for conditions such as mood disorders, anxiety, cognitive issues, sleep disorders, and chronic pain (18). The 5-HT_1BR, found predominantly in the CNS and peripheral vasculature, regulates neurotransmitter release and vasoconstriction (19, 20). It shares high sequence homology (77% within the TM regions) with 5-HT_1DR (21). They are associated with migraine physiology and are a primary target of antimigraine drugs (22). 5-HT_1ER is primarily expressed in the hippocampus and hypothalamus. Similarly, 5-HT_1FR is expressed in the cortical and hippocampal areas of the brain and acts as a target for antimigraine drugs (23).

Three structures of G_i-coupled 5-HT_1AR have been reported; the apo state, bound to the ligand 5-HT, and bound to aripiprazole (Table 1, Fig. 1A). Intriguingly, all three structures feature a lipid molecule, phospholipid phosphatidylinositol 4-phosphate (PtdIns4P), found in the pocket created by TM3, TM6 and TM7 of the receptor along with the α5 helix of the G_i protein. PtdIns4P interacts with 5-HT_1AR and Gα_i, leading to enhanced G-protein coupling and GTPase activity. Consequently, acting as a positive allosteric modulator for the receptor, and playing a crucial role in regulatory function. Moreover, cholesterol molecules surrounding the TM domain shape the ligand-binding pocket, thereby influencing ligand specificity for aripiprazole. The unbound (apo) form of the 5-HT_1AR closely resembles its structure when bound to 5-HT. Within the orthosteric binding pocket (OBP), three water molecules are positioned in a spatial arrangement similar to that of 5-HT. These water molecules have been shown to exhibit similarities to the polar functional groups of 5-HT, suggesting their potential role in activating the receptor (24).

The ergoline class of drugs although target 5-HT_{1A/1B/1D/1E/1F} and 5-HT_2A/2B/2C receptors, they exert their antimigraine effects through the activation of 5-HT_1B, 5-HT_1D and 5-HT_1F receptors. Ergotamine primarily binds to the 5-HT receptor family with varying affinities, likely due to variations in the EBP, as OBP remains highly conserved. For instance, ergotamine-bound 5-HT_1BR adopts a typical active state, regulating the usual pathway, while 5-HT_2BR combines active and inactive features, showing strong β-arrestin signaling selectivity (16, 25). In a crystal structure, ergotamine and dihydroergotamine revealed a similar binding mode to the human 5-HT_1BR. The ligands bind to the OBP located deep within the 7TM core and an extended binding pocket (EBP) near the extracellular loops (Fig. 1A). The ergoline scaffold of the ligand forms a salt bridge interaction with D129^3.32 (superscripts indicate the Ballesteros-Weinstein numbering for GPCR residues) (11), which is present in all the 5-HT receptors. Additionally, a narrow hydrophobic cleft formed by the side chains of C133^3.36, I130^3.33, W327^6.48, F330^6.51 and F331^6.52 tightly interacts with the nearly planar ergoline ring system (25). Compared to ergot, donitriptan, an agonist of 5-HT_1BR, binds differently. It engages residues from helices 3, 5, 6 and 7, extending into the extracellular region where it interacts with helices 6 and 7 and ECL2 in the receptor’s binding pocket. Its binding primarily relies on van der Waals interactions and has limited polar interactions, resembling ergotamine’s mode of binding. However, donitriptan and ergotamine occupy opposite sides of the binding pocket, with distinct orientations of F351^7.65 and M337^6.58 accommodating these ligands (26). On the contrary, the inverse agonist methiothepin (MT) predominantly binds to the lower OBP and shows minimal interactions with the EBP residues, which accounts for its non-selectivity towards other members of the 5-HT receptor family. In comparison to agonist-bound structure, MT causes structural changes by pushing some key residues (W327^6.48, F330^6.50 and F331^6.51) within the binding pocket, leading to an inward shift in TM6, a common feature in inactive GPCR structures (27).

A study of the structure of 5-HT_1ER bound to BRL-54443, a selective agonist for 5-HT_1ER and 5-HT_1FR, highlighted the significance of the 6.55 residue in the ligand binding pocket by influencing receptor selectivity. In 5-HT_1ER, E311^6.55 forms an additional hydrogen bond with BRL-54443’s hydroxyl group, which is absent in 5-HT_1AR and 5-HT_1DR due to different amino acids at that position (Fig. 2A). This interaction, along with the presence of identical E313^6.55 and K312^6.54 residues in 5-HT_1FR, explains why BRL-54443 exhibits selectivity towards both the 5-HT_1ER and 5-HT_1FR. However, the replacement of the hydroxyl group by the carboxamide group in 5-carboxamidotryptamine (5-CT) shows a higher affinity towards HT_1A, HT_1B and 5-HT_1D receptors than 5-HT_1E and 5-HT_1F receptors supporting the role of residue 6.55 as the determinant of subtype selectivity (24).

Fig. 2.

Structural difference in the ligand binding pocket of all serotonin receptors were superimposed and compared the ligand binding pose in the same direction. Close up view of the ligand binding site of (A) 5-HT1 family: aripiprazole bound 5-HT_1A (PDB ID: 7E2Z), donitriptan bound 5-HT_1B (PDB ID: 6G79), 5-HT bound 5-HT_1D (PDB ID: 7E32), BRL-54443 bound 5-HT_1E (PDB ID: 7E33), lasmiditan bound 5-HT_1F (PDB ID: 7EXD), (B) 5-HT₂ family: 25-CN-NBOH bound 5-HT_2A (PDB ID: 6WHA), LSD bound 5-HT_2B (PDB ID: 7SRR), lorcaserin bound 5-HT_2C (PDB ID: 8DPI) and (C) 5-HT_4/5/6/7 family: 5-HT bound 5-HT₄ (PDB ID: 7XT8), 5-CT bound 5-HT₅ (PDB ID: 7UM5), 5-HT bound 5-HT₆ (PDB ID: 7XTB), and 5-CT bound 5-HT₇ (PDB ID: 7XTC).

Triptan, a class of anti-migraine drugs, caused vasoconstrictive side effects by targeting 5-HT_1B and 5-HT_1D receptors, prompting the development of lasmiditan, a more selective anti-migraine drug targeting 5-HT_1FR (28). The cryo-EM structure of lasmiditan bound to the 5-HT_1FR (Fig. 2A) revealed differences in the EBP unlike the 5-HT_1BR, while the OBP remained conserved. Structural differences are also observed in the extracellular region, with TM4 and TM5 shifting away in the 5-HT_1FR. Notably, lasmiditan interacts with E313^6.55 and N317^6.59 residues in the TM6 of 5-HT_1FR through hydrogen bonds unlike the S334^6.55 and P338^6.59 in 5-HT_1BR, which cannot form similar interactions. Similarly, lasmiditan interaction resulted in distinct conformations of ECL2 in both the 5-HT_1F and 5-HT_1B receptors. Despite its high sequence homology with 5-HT_1ER, lasmiditan is selective for 5-HT_1FR due to differences in TM4, TM5 and ECL2 conformations. These structural distinctions explain lasmiditan’s specificity for 5-HT_1FR (29).

5-HT₂ RECEPTOR FAMILY

The 5-HT₂ receptor family consists of 5-HT_2A, 5-HT_2B and 5-HT_2C subtypes. Although primarily G_q/11-coupled, 5-HT_2AR and 5-HT_2BR also recruit β-arrestin-2 upon lysergic acid diethylamide (LSD) and ergotamine binding. These receptors are widely distributed in the brain and peripheral tissues, with 5-HT_2AR contributing to vascular tone regulation, mood, cognition and hallucinogenic effects. Selective modulation of 5-HT_2AR is a potential treatment option for various neuropsychiatric conditions and neurological disorders (30). 5-HT_2BR are found in the smooth muscle cells, cardiac tissue and endothelial cells, playing a role in cardiovascular function. 5-HT_2CR are present in the gastrointestinal tract and are involved in gut motility and appetite regulation. Consequently, they are a pharmacological target for anti-obesity drugs (31, 32). 5-HT_2AR and 5-HT_2CR share nearly 80% sequence homology in their TM regions and have similar pharmacological features (33). The subfamilies show differences in their structures in the ICL2. The differences observed between 5-HT_2BR and 5-HT_2AR, particularly in the outward shifts of TM5 and TM6 and the inward shift of TM3, highlight the structural divergence between these two receptors. Additionally, the longer ECL2 in 5-HT_2BR, containing six additional residues, results in a shorter TM4 compared to 5-HT_2AR, influencing the conformation of F^5.38 and contributing to the distinctive side-extended cavity formation in 5-HT_2AR (34).

The 5-HT_2AR is notable for its significance in regulating essential brain processes (35). Psychedelic substances, renowned for their strong psychoactive effects, are thought to primarily interact with the 5-HT_2AR (5). The crystal structures of 5-HT_2AR and antipsychotics, risperidone, zotepine and lumateperone, complex exhibit the interaction of these ligands with I163^3.40 and F332^6.44 in the PIF motif and the toggle switch W336^6.48 on the lower part of the ligand-binding pocket, which stabilizes the receptor in its inactive state (Fig. 1B). Additionally, located between TM4 and TM5 is a side-extended cavity that connects the orthosteric site and the plasma membrane, facilitating interaction with ligands. This cavity is stabilized by G238^5.42 residue present at the entrance of the cavity. The residues on ECL2 adopt different conformations upon binding to these ligands and the entrance of the ligand binding pocket is wider in risperidone bound structure compared to zotepine bound structure. The ligands’ binding sites overlap with those of inverse agonists in other receptors (ritanserin in 5-HT_2CR, risperidone in D₂R and doxepin in H₁R). The structure of risperidone-bound 5-HT_2AR is similar to that of the 5-HT_2CR bound to ritanserin, but 5-HT_2AR has a unique side-extended cavity near the OBP. Additionally, 5-HT_2AR differs from D₂R in ECL1 and ECL2 conformation, suggesting a potential for designing selective 5-HT_2AR antagonists. The structures of serotonin and psilocin-bound 5-HT_2AR showed glycerol groups of monoolein inserted deep into the OBP, which led to the activation of 5-HT_2AR. The glycine residue at position 5.42 in the OBP is conserved only in the 5-HT₂ family, which allows for the extension of the side extended pocket (SEP) from the OBP. In 5-HT_2BR and 5-HT_2CR, phenylalanine disrupts the SEP despite the conserved glycine. Lipids did not induce robust G protein signaling in 5-HT receptors lacking conserved glycine. These findings explain the lipid modulation of 5-HT_2AR signaling (36-38).

The structure of LSD and ergotamine-bound 5-HT_2BR demonstrated the mechanisms responsible for G protein and β-arrestin signaling (Fig. 1B). LSD bound structure showed that residue I453 and L464 in the C-tail are important for β-arrestin-1 recruitment. β-arrestin-1 coupling of 5-HT_2BR is mediated by hydrophobic and electrostatic interactions between the finger loop of β-arrestin-1 and the cytoplasmic core of 5-HT_2BR. Additionally, the hydrophobic interactions between I161^34.51 residue of 5-HT_2BR and the β-arrestin-1 residues play a significant role in β-arrestin-1 recruitment by 5-HT_2BR. The transducer coupling mode of 5-HT_2BR is determined by residue N384^8.47 present at helix 8. In G_q bound state it forms a hydrogen bond with α5 residue N244^5.22 of G_q, whereas the residue rotates 90° in the β-arrestin-1 state with no significant interaction (17). Notably, LSD-bound 5-HT_2BR showed that the diethylamide moiety of LSD interacts with TM3 and TM7 within the EBP and the recognition of LSD in this region was stereoselective. Only the (S,S)-azetidide stereoisomer of LSD showed LSD’s potent agonism (39). A comparison of the structures of 5-HT_2BR bound to lisuride and LSD showed differences in the EBP, indicating that the interaction of ligands with TM7, specifically with L362^7.35, triggers a supplementary mechanism of agonist activation through the EBP (40). Another important residue for receptor activation is T140^3.37 and A225^5.46, which form hydrogen bond and van der walls interactions, respectively, with the indole group of ergolines. Methylergonovine acts as an agonist at 5-HT_2BR, whereas methysergide that differs from methylergonovine in one methyl group acts as an antagonist due to the lack of interactions with the T140^3.37 and A225^5.46 residues (40).

The structures of 5-HT_2CR have been solved in complex with agonist ergotamine, lorcaserin, and psilocin and inverse agonist ritanserin. Compared to the active ERG-bound structure, ritanserin binds approximately one helical turn deeper into the TM bundle forming interactions with the toggle switch and PIF motif and prevents the conformational changes in these microswitches as observed in inverse agonists bound 5-HT_2AR (38). The structures of different isoforms of 5-HT_2CR and lorcaserin and psilocin complex revealed that the active state structure of 5-HT_2CR is stabilized by a hydrogen bond network in the ICL2 region of the receptor (Fig. 2B). Such interactions favored the binding of the α5 helix of Gα_q to the receptor and thus activating the receptor (41).

5-HT₅ RECEPTOR FAMILY

The 5-HT₅ receptor subfamily consists of two receptors, 5-HT_5A and 5-HT_5B. The two receptors have a 69% similarity in their amino acid composition and a 23-34% similarity with the remaining 5-HT receptors (42). 5-HT_5BR are specific to the brain and are not present in humans (43). 5-HT_5AR are mainly located in the brain regions responsible for memory and learning, and they play a role in neurotransmitter release and synaptic plasticity (44). The human 5-HT_5AR receptor remains largely mysterious and poorly understood compared to other serotonin receptors (45).

Four structures of 5-HT_5AR have been determined in the presence of two full agonists, 5-CT and methylergometrine, one partial agonist lisuride and antagonist AS2674723 (Table 1, Fig. 1C). Compared to the 5-CT and methylergometrine bound 5-HT_5AR-G_i structure, lisuride forms weaker interactions with TM3 and TM6 resulting in the pan agonism of lisuride. The inactive state structure of 5-HT_5AR bound to antagonist AS2674723 revealed that the tetrahydroisoquinoline ring, guanidine group, and a trifluorophenyl ring in the AS2674723 interacts with the OBP and EBP by hydrophobic interactions, π−π interactions and salt bridge. Particularly, the formation of a salt bridge by guanidine with E101^2.65 is accompanied by a shift in the extracellular end of TM2 compared to the active state structures. Two less conserved residues in the 5-HT receptors, the L324^7.39 and E101^2.65, play a role in determining the selectivity of AS2674723 for 5-HT_5AR. 5-HT₇R has aspartate in place of glutamate at position 2.65 which can form similar electrostatic interactions with the guanidine group of AS2674723, explaining the off-target effect of this ligand for 5-HT₇R. The inactive 5-HT_5AR structure also showed a kinked structure in the intracellular end of TM5, which may inhibit signaling activation through the blockage of TM6 movement. This type of interface is present only in 5-HT_1ER, 5-HT_1FR, 5-HT_5AR and 5-HT₇R among all serotonin receptors offering potential opportunities for designing inactive state structures of these receptors (46, 47).

5-HT₄, 5-HT₆, 5-HT₇ RECEPTOR SUBFAMILIES

5-HT₄R are found in various regions of the CNS and peripheral tissues, including GIT, heart and smooth muscle cells. In the GIT, they regulate motility, secretion and visceral sensitivity. Consequently, they are therapeutic targets for 5-HT₄R agonists for managing gastrointestinal disorders like irritable bowel syndrome (IBS) and constipation (48). In contrast, 5-HT₆R are primarily found in the CNS, and therefore are an interesting target for therapeutic agents for enhancing cognitive function (49). The 5-HT₇R have a broader distribution in the CNS, encompassing regions such as the spinal cord, hippocampus and thalamus. These regulate circadian rhythm, mood and cognitive processes, making them potential targets for treating conditions like depression, anxiety, schizophrenia and cognitive impairments (50). 5-HT₆R and 5-HT₇R are the targets of several typical and atypical antipsychotics (51).

The cryo-EM structures of 5-HT_4/6/7 receptors and ligands complex reveal differences in ECL2 conformation (Fig. 2C). A structural study of 5-HT₇R highlighted the significance of the 6.55 residue in determining ligand selectivity. This residue has a smaller side chain in 5-HT_7/1A/1B/1D receptors, whereas other 5-HT receptors have a bulkier side chain. A comparison of 5-HT₇R and 5-HT_1ER structures suggests that the bulkier E311^6.55 in 5-HT_1E may lead to unfavorable interactions with the amide group of 5-CT, resulting in reduced binding affinity for 5-HT_1ER. Additionally, a comparison of 5-HT₄R, 5-HT₆R and other serotonin-bound receptors revealed some unique residues in their ligand-binding pockets such as residues at positions 3.28, 5.46 and 6.55 in 5-HT^4/6/1A/1D receptors. For instance, 5-HT₄R has a unique R96^3.28 residue that is involved in a hydrogen-bonding network, 5-HT₆R has a distinctive T196^5.46 residue that forms a hydrogen bond with indole nitrogen of serotonin and 5-HT_1AR lacks a polar group at position 6.55 that are present in 5-HT₄R, 5-HT₆R and 5-HT_1DR. These unique binding site residues is a potential for optimizing receptor selectivity in drug design (14).

LIGAND BINDING SITES IN SEROTONIN RECEPTORS

The ligand binding region of 5-HT receptors is formed by residues from TM3, TM5, TM6, TM7 and ECL2. The OBP is located deep within the core of the seven TM structure and is a primary site where ligands bind to activate the receptor (52). Analyses of all human 5-HT GPCR sequences demonstrated that the residues found within the OBP exhibit a higher level of conservation suggesting that there is likely an evolutionary drive to maintain the structural integrity of the OBP, which is crucial for the recognition of the native ligand, 5-HT (16, 53). The conserved Asp^3.32 residue within the TM3 interacts with the amine-containing ligands through a salt bridge (16, 25, 34, 38). Moreover, within the binding pocket of all 5-HT receptor family, there are conserved aromatic residues W^6.48, F^6.51, F^6.52 and Y^7.43 (Fig. 2). These residues create a confined hydrophobic crevice that facilitates crucial π–π stacking interactions with ligands containing aromatic ring structures (6, 54). On the other hand, the EBP is a supplementary site situated close to the receptor’s extracellular entrance. Although not directly involved in ligand binding and activation, it can modulate ligand selectivity and receptor function. Compared to the OBP, the residues in the EBP are less conserved and vary among the receptor subtypes, thus allowing for receptor-specific interactions with the ligands (6). The superimposition of all ligands bound 5-HT receptors showed that the ligands bind to the different pockets. The OBP ligands are mostly the tryptamine family such as 5-HT and 5-CT. The ligands aripiprazole and donitriptan have quite different characters. Both ligands share the same OBP, but they bind different EBP (Fig. 2). The residues 6.55 and 6.58 may affect the ligand specificity and binding pose with the two ligands.

STRUCTURAL ANALYSIS OF G-PROTEIN COUPLING PROFILE

The G-protein coupling mode of 5-HT receptor family is similar to class A GPCRs involving the rearrangement of residues in the conserved motifs and microswitches (55). In the active state of the GPCRs, the binding of agonist induces an outward shift of the cytoplasmic end of TM6. This conformational shift is caused by the rearrangement of residues in the PIF and DRY motifs and the downward press of W^6.48 in the toggle switch, resulting in the breaking of the conserved salt bridge between TM3 and TM6. Additionally, TM7 undergoes an inward movement, opening the cytoplasmic pocket for binding to G_i protein (24). Despite elucidating the structures of all the 12 5-HT receptors, our knowledge regarding the specific G protein preferences of these receptors is still lacking due to insufficient research in this area. To understand the variations in G protein interactions, we conducted a superimposition of the 12 solved 5-HT receptor structures using the structure of the receptor as a template. This allowed us to identify and compare the differences among the structures in their interactions with G proteins. The G protein demonstrates a rotation around the axis of the I residue (G.H5.15), resulting in more significant rotations of both the ɑ5 helix and ɑN helix (Fig. 3A). Afterward, we conducted a deeper examination of the positioning of the ɑN helix in different G proteins, including G_i, G_s and G_q proteins. Surprisingly, we found that each G protein could be classified based on the location of its ɑN helix. Specifically, the G_q subunit’s ɑN helix was positioned on the right side; counterclockwise part in Fig. 3A, the G_i subunit’s helix was in the middle, and the G_s subunit’s helix was situated on the left and bottom, the clockwise part in Fig. 3B. The salt bridge between R^3.50 and D^3.49 in the DRY motif determines the functional selectivity of the receptor for β-arrestin signaling. In the crystal structure of 5-HT_1BR, the salt bridge between R^3.50 and D^3.49 is broken, but the bond is preserved in 5-HT_2BR suggesting that ergolines bound to these receptors showed characteristics of an intermediate active state for the 5-HT_1BR and β-arrestin biased activation state for 5-HT_2BR (16). The conserved residues within the TM5 and TM6 in class A GPCRs determine the receptor’s selectivity towards G proteins. 5-HT₄R, which can couple to both G_s and G_i showed a noticeable change in the intracellular end of TM5, which is extended by 12 residues in the G_s-coupled state in contrast to the G_i-coupled state (14).

Fig. 3.

The binding of G protein and receptor exhibit distinct binding orientations by G protein subunit. (A) Comparison of the Gα conformation among the structures of different G-protein coupled serotonin receptors. The superimposition of G protein using the structure of the serotonin receptor as a template shows the structural difference of the αN helix and α5 helix between G alpha subunits such as G_s, G_i and G_q subunit. (B) Detailed view of the conformation of α helix in complex with serotonin receptors.

DISCUSSION

The structural features of the 5-HT family, in complex with various ligands and effectors provide valuable insights into their ligand recognition and downstream signaling pathways. Conserved residues within their TM domains play a crucial role in ligand binding and interaction with G proteins. Diversity in extracellular loops, particularly ECL2, suggests potential variations in ligand selectivity and receptor function among subtypes. The OBP and EBP modulate ligand selectivity and receptor function, while specific G protein coupling profiles determine intracellular responses. Additionally, arrestins’ involvement adds complexity to GPCR regulation. 5-HT receptors have implications for mental health disorders and neurological diseases (56). However, despite considerable progress in studying the structure of GPCRs, specific agonists and antagonists for subfamilies are lacking. Consequently, during treatment, there is polypharmacology and side effects. Therefore, further research on the structure of 5-HT receptors is needed to gain a comprehensive understanding of their function and enable the development of novel therapeutics that are highly selective to specific receptor subtypes based on their structural characteristics.

Funding Statement

ACKNOWLEDGEMENTS This work was supported by the National Research foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2023-00214187). This work was supported by the Yonsei University Research Fund of 2022 (2022-22-0138).