Activation of 5-HT6 Receptors Modulates Sleep–Wake Activity and Hippocampal Theta Oscillation

Abstract

The modulatory role of 5-HT neurons and a number of different 5-HT receptor subtypes has been well documented in the regulation of sleep–wake cycles and hippocampal activity. A high level of 5-HT₆ receptor expression is present in the rat hippocampus. Further, hippocampal function has been shown to be modulated by both 5-HT₆ agonists and antagonists. In the current study, the potential involvement of 5-HT₆ receptors in the control of hippocampal theta rhythms and sleep–wake cycles has been investigated. Hippocampal activity was recorded by intracranial hippocampal electrodes both in anesthetized (n = 22) and in freely moving rats (n = 9). Theta rhythm was monitored in different sleep–wake states in freely moving rats and was elicited by stimulation of the brainstem reticular formation under anesthesia. Changes in theta frequency and power were analyzed before and after injection of the 5-HT₆ antagonist (SAM-531) and the 5-HT₆ agonist (EMD386088). In freely moving rats, EMD386088 suppressed sleep for several hours and significantly decreased theta peak frequency, while, in anesthetized rats, EMD386088 had no effect on theta power but significantly decreased theta frequency, which could be blocked by coadministration of SAM-531. SAM-531 alone did not change sleep–wake patterns and had no effect on theta parameters in both unanesthetized and anesthetized rats. Decreases in theta frequency induced by the 5-HT₆ receptor agonist correspond to previously described electrophysiological patterns shared by all anxiolytic drugs, and it is in line with its behavioral anxiolytic profile. The 5-HT₆ antagonist, however, failed to potentiate theta power, which is characteristic of many pro-cognitive substances, indicating that 5-HT₆ receptors might not tonically modulate hippocampal oscillations and sleep–wake patterns.

The 5-HT6 receptor is a relatively new member of the serotonergic receptor family with almost exclusive localization in the brain.¹⁻³ It has from the start been implicated in cognitive and affective disorders, mainly because of its preferential distribution in cortical and limbic structures^1,4,5 and because of the affinity of some antidepressants and antipsychotics to this receptor.^3,6 In the hippocampus, 5-HT6 receptors are expressed by GABA interneurons (7) and can thus participate in the control of hippocampal activity by modulating the excitatory/inhibitory balance and acting on generators of various oscillations. Rhythmic synchronized activity at different frequencies is a prominent feature of hippocampal networks and is essential for functioning of these networks. Theta rhythm is the most studied of hippocampal oscillations. This 5–10 Hz rhythm is associated in the rat with awake exploration and rapid eye movement (REM) sleep, that is, with behavioral states related to different stages of memory formation.⁸ Hippocampal theta rhythm co-occurs with increased cortical gamma activity⁹ and represents a key signal establishing transient hippocampo-cortical coupling through the mechanism of cross-frequency modulation.^10,11 Thus, theta rhythm also appears in structures functionally related to the hippocampus during specific tasks. For example, theta modulation of gamma activity in the prefrontal cortex is prominent during select cognitive tasks¹² and coherent theta rhythm links hippocampus with the amygdala during processing of emotional information.^13,14 Hippocampal theta activity has also been demonstrated in humans in similar conditions, for example, during spatial memory tasks as well as during REM sleep.¹⁵⁻¹⁷

Theta rhythm is controlled by ascending systems which arise from the brainstem and target the oscillatory circuits in the hippocampus directly and indirectly, through theta pacemaker circuits in the medial septum (MS) and supramammillary nucleus.^18,19 The median raphe nucleus (MRN), the origin of serotonergic innervation of the hippocampus, is a key component of this ascending control. Serotonin exerts its modulating effect on neuronal elements of theta generators through multiple types of receptors which are coupled to different G-proteins, are located in different types of neurons, and have different anatomical distributions. These factors also determine their distinct role in serotonergic modulation of theta rhythm. Previous studies focusing on 5-HT1A and 5-HT2 receptors, for example, revealed opposite effects of these receptors, that is, a decrease in theta amplitude by 5-HT2c mediated activation of MS GABA interneurons²⁰ and lasting theta activation by 5-HT1A mediated inhibition of serotonergic firing in the MRN.^21,22

The possible contribution of the recently identified family of G_s-protein coupled 5-HT receptors, including 5-HT6, to hippocampal oscillations has not been addressed to date; their electrophysiological characterization remains limited to in vitro studies²³ and sleep studies using surface cortical electrodes.^24,25 These receptors are absent from the raphe nuclei, but their expression patterns at cortical and hippocampal serotonergic projections^1,4,5 suggest a possible involvement in modulating oscillatory networks. Extensive behavioral investigations during the past years are in line with this hypothesis, as they primarily implicated 5-HT6 receptors in cognitive²⁶ and affective processes²⁷ which require effective functioning of theta networks in the hippocampus and functional theta coupling between hippocampus and cortical and limbic structures. These studies defined two major lines of possible therapeutic applications of 5-HT6 receptor-active compounds as potential pro-cognitive and anxiolytic drugs. Preclinical studies showed that 5-HT6 receptor antagonist improved cognition^26,28 but also showed antidepressant- and anxiolytic-like activity,²⁹⁻³¹. Furthermore, there are sporadic data indicating that 5-HT6 agonists may have similar effects.^32,33 The present study used electrophysiological markers to further clarify this issue based on earlier research that established the changes of theta rhythm parameters in anesthetized and freely moving rats as promising biomarkers of the pro-cognitive and anxiolytic potential of pharmacological compounds.³⁴

Results

The Effect of 5-HT6 Receptor Activation on Sleep–Wake States

The 5-HT6 receptor agonist EMD386088 drastically reduced sleep time; after injection, all rats were awake for several hours (Figure 1A). The effect was dose dependent, and continuous uninterrupted waking started immediately after the injection of 10 mg/kg EMD386088 and lasted for 11–13 h. After 5 mg/kg injection, wake time increased to 48 ± 23% (compared with 24 ± 15% in preinjection control or with 14 ± 4% after saline injection) and then progressively increased reaching a level of continuous wakefulness by the fourth hour after injection and lasted for 7–10 h (Figure 1D). The 5-HT6 antagonist SAM had no effect on the sleep–wake cycle (Figure 1B and D); the hourly averages of time spent in wake were 24 ± 9% and 16 ± 5% after 3 and 17 mg/kg injection, respectively. Wake time increased to 45 ± 11% and 60 ± 10% after 17 and 3 mg/kg injection, respectively, by the end of the light period following the normal circadian pattern, also observed after vehicle control (Figure 1C–F; 57 ± 11%). Statistical analysis compared hourly averages of sleep and wake time during the light period of the day, that is, 4 h before and 6 h after drug administration. The differences relative to vehicle control were highly significant (two-way ANOVA, drug: F[4,216] = 42.56, P < 0.001; posthoc Bonferroni p < 0.05 for all veh vs EMD and SAM vs EMD pairs; repeated measures ANOVA for time F[10] = 12.61 for 10 mg/kg and F[10] = 13.37 for 5 mg/kg p < 0.05; Bonferroni p < 0.05 for each postinjection time point after 10 mg/kg and 1 and 4–6 h after 5 mg/kg).

Figure 1.

Effect of 5-HT6 receptor agonist EMD386088 and antagonist SAM-531 on the sleep–wake cycle. (A–C) Hourly averages of time (%) spent in wake, slow wave sleep and REM sleep before and after injection of 10 mg/kg EMD386088 (A), 17 mg/kg SAM-531 (B), and saline (C). (D–F) Dose response effect of EMD386088 (5 and 10 mg/kg) and SAM-531 (3 and 17 mg/kg) on waking (D), slow wave sleep (E), and REM sleep (F). The experiments started in the morning, drug administration took place after 4 h control recording, at noon (data are shown for the light period of the day; start of recording at ZT1). On the abscissa −4, −3, −2, −1 indicates hourly averages during the fourth, third, second, and first hour before injection, 1, 2, ... indicates the first, second, ... hour after injection.

The Effect of 5-HT6 Receptor Activation on Theta Rhythm in Freely Moving Rats

The 5-HT6 receptor agonist EMD386088 significantly reduced theta frequency (Figure 2A). After 10 mg/kg injection, the frequency dropped by more than 1 Hz during the first 2 h and remained about ∼0.5 Hz below the preinjection and vehicle control levels until the end of the light period (Figure 2D). Administration of 5 mg/kg EMD386088 also reduced theta frequency to a similar extent (>1 Hz), but the effect only lasted for 3 h, after which theta rhythm returned to normal (Figure 2D). The effect was significant during the first 3 h for the 5 mg/kg and for 2 h after the 10 mg/kg dose. 5-HT6 receptor antagonist SAM had no effect on theta frequency (drug: F[4,213] = 16.34, p < 0.001; time: F[6,213] = 6.82, p < 0.001, posthoc Bonferroni p < 0.05 for all veh vs EMD and SAM vs EMD pairs; repeated measures ANOVA for time F[5] = 10.03 for 10 mg/kg and F[5] = 13.86 for 5 mg/kg p < 0.05; Bonferroni p < 0.05 for 2 h postinjection time compared with preinjection time points; Figure 2A and D).

Figure 2.

Effect of 5-HT6 receptor agonist EMD386088 and antagonist SAM-531 on hippocampal theta oscillations. (A) Theta frequency (during theta states) before and after injection of EMD386088 (5 and 10 mg/kg), SAM-531 (3 and 17 mg/kg), or saline. (B, C) Variations in theta power during REM sleep (B) and waking exploration (C) before and after drug administration, relative to 4 h preinjection. (A–C: 1 h before and after injection.) (D) Time course of changes in the theta frequency (during theta states) before and after drug administration. Inset shows theta frequency during active wake and REM sleep for 5 mg/kg EMD386088 (on identical scale). (E) Time course of changes in spectral power in the 5–10 Hz frequency range (total power irrespective of behavior). (D: theta frequency calculated in theta states; E: theta power calculated irrespective of state or behavior; D, E: hourly averages; “0 h”: immediate reaction, i.e. first 30 min; start of recording at ZT1).

Neither activation nor blockade of the 5-HT6 receptor had a direct effect on the power of theta oscillations (two-way ANOVA, drug: F[4,216] = 2.12, p = 0.08; time: F[6,216] = 1.28, p = 0.70). Figure 2B and C compares pre- and postinjection theta power during wake and REM sleep and shows that the variations remained within 2–3% for all four drug/dose combinations as well as for vehicle. In this analysis, calculation of theta power was limited to theta states (i.e., segments in which theta/delta ratio was higher than 4, see Methods and, e.g., ref (21)) to separate the effect on theta from those on behavior. Indeed, total power calculated within the theta frequency band, irrespective of behavioral state, showed an increase after EMD386088 administration (Figure 2E) (two-way ANOVA, drug: F[4,216] = 10.65, p < 0.001; time: F[6,216] = 1.90, p = 0.08, posthoc Bonferroni p < 0.05 for all veh vs EMD and SAM vs EMD pairs) which could thus be explained by the dominance (increased occurrence and length) of waking theta states induced by these compounds.

The Effect of 5-HT6 Receptor Activation on Theta Rhythm under Urethane Anesthesia

The effect of 5-HT6 receptor activation was further verified in urethane anesthetized rats using the higher doses of the agonist and antagonist and their combinations. In urethane anesthetized rats, theta rhythm is reliably and reversibly elicited by high-frequency (100 Hz) electrical stimulation of the pontine reticular formation (RPO³⁵) and the parameters of the theta oscillation are experimentally controlled by changing the level of the stimulus intensity; that is, stronger stimulations will lead to faster hippocampal oscillations and will also increase theta wave amplitude. Theta frequency has a strong linear relationship with RPO stimulus intensity, whereas the changes in theta power are less consistent and show more variability.^36,37 The stimulus intensity versus theta frequency characteristic is built individually in each experiment, using a scale of stimulus intensities from the threshold, at which theta is first elicited, to the maximum, at which theta frequency saturates. The modulatory effect of various substances can thus be quantified by measuring the shifts of the characteristics of stimulus intensity versus theta frequency and power or by comparing the effect at intensities, functionally equivalent in different rats. As shown in a representative experiment in Figure 3, the frequency characteristic did not show significant alterations over time during the control period and was not affected by administration of 5-HT6 receptor antagonist (Figure 3C). In contrast, 5-HT6 receptor agonist reduced theta frequency at all stimulus intensities and also changed (flattened) the intensity versus frequency characteristic (Figure 3B).

Figure 3.

Effect of 5-HT6 receptor agonist EMD386088 and antagonist SAM-531 on RPO stimulation elicited theta rhythm under urethane anesthesia, in representative experiments. (A) Traces of hippocampal recording during 10 s RPO stimulation (see time marker) at high intensity before and after EMD386088 administration (10 mg/kg) eliciting, respectively, 8.6 and 5.8 Hz theta rhythm. (B, C) Change in theta frequency during standard sequences of RPO stimulations at 5 equidistant levels of intensity between threshold and maximum repeated every 15 min, before and after administration of EMD386088 (B) or SAM-531 (C).

On the group average, EMD386088 (10 mg/kg) decreased theta frequency at each level of stimulation intensity (Figure 4A; n = 5, paired two-tailed t test, p < 0.05), although the effect was stronger at suppressing faster theta components (Figure 4C). In contrast, administration of SAM (17 mg/kg) did not change theta frequency (Figure 4B; n = 8, paired two-tailed t test, p > 0.05). When administered in combination with EMD386088, however, it blocked the effect of the agonist (Figure 4C; n = 9, paired two-tailed t test, p > 0.05). Neither compound had any significant effect on theta power (Figure 4D–F).

Figure 4.

Changes in theta frequency and power after administration of the 5-HT6 receptor agonist EMD386088 and antagonist SAM-531 in urethane anesthetized rats. A and B. Frequency of theta rhythm induced by stimulations of RPO at different intensities before and after administration of 10 mg/kg EMD386088 (A) or 17 mg/kg SAM-531 (B). C. Changes in theta frequency (drug-vehicle difference) after injection of EMD386088 (n = 5), SAM-531 (n = 8), or coadministration of the two compounds (10 mg/kg EMD386088 and 17 mg/kg SAM-531, n = 9). (D, E) The effect of EMD386088 and SAM-531 on theta power (4–10 Hz). (F) Drug-vehicle differences in theta power after EMD386088, SAM-531, and their coadministration.

Discussion

Ascending serotonergic projections play a key role in the regulation of hippocampal activity¹⁸ exerted through a variety of 5-HT receptors in the septo-hippocampal system. This study examined the involvement of 5-HT6 receptors in this control. The major finding was that 5-HT6 receptor activation decreased theta peak frequency without changing its power. In freely moving rats, the 5-HT6 receptor agonist EMD386088 reduced theta frequency in a dose dependent manner during awake theta states in which theta rhythm is associated with specific behaviors. EMD386088 also drastically suppressed sleep, including REM sleep, for several hours. In anesthetized rats, the 5-HT6 receptor agonist had no effect on theta power but significantly decreased theta frequency, which could be blocked by coadministration with the receptor antagonist SAM-531. SAM-531 alone did not change sleep–wake pattern (during the light period of the day) and had no effect on theta parameters in either experiment. The decrease in theta frequency induced by 5-HT₆ agonist corresponds to previously described electrophysiological pattern shared by all tested anxiolytic drugs,³⁴ and it is in line with its behavioral anxiolytic profile. The 5-HT6 antagonist, however, did not show potentiation of theta power characteristic for many pro-cognitive substances,³⁴ indicating that 5-HT6 receptors in normal rats might not tonically modulate hippocampal oscillation.

The effect of 5-HT6 receptor activation on theta differs from other types of serotonin receptors previously investigated, indicating that different receptors make specific contributions to forebrain activity which may not be obvious from the overall effect of serotonergic activation. The midbrain raphe is part of a group of brainstem nuclei which send positive or negative modulatory input to septo-hippocampal networks. It occupies a unique position among them as the most powerful pathway with an inhibitory role in hippocampal theta generation.^18,19 Electrical stimulation of the MRN disrupts hippocampal theta rhythm and replaces it with low amplitude high frequency activity^35,38−41 whereas lesioning or pharmacological blockade of MRN produces increased theta activity.⁴²⁻⁴⁵ The majority of 5-HT neurons are silent during REM sleep when the strongest and clearest theta oscillations occur.⁴⁶⁻⁴⁸ However, increase or decrease in the serotonin level by reuptake inhibitors or blockade of 5-HT synthesis, respectively, do not have such global effects on the occurrence of theta rhythm,^49,50 suggesting a more delicate serotonergic involvement in the regulation of forebrain oscillations. The complexity of this regulation is supported by a significant diversity of 5-HT neurons⁵¹⁻⁵⁴ and by the wide variety of 5-HT receptors (55) their diverse anatomical distribution, and neuronal expression which may induce nonuniform serotonergic actions at different targets and during different behaviors.

The exact mechanism of how systemic administration of 5-HT6 receptor agonist leads to decrease in theta frequency is not clear, but several plausible explanations exist. The first, for which supporting histological evidence is available, is direct action on the hippocampal theta generator. 5-HT6 receptors are expressed in limbic system, in the hippocampus in particular.^1,4,5 Co-localization of 5-HT6 receptors with GAD67 indicates prevalent expression by GABA cells.⁷ Through G_s-protein, positively coupled to adenylyl cyclase (or possibly by other subcellular pathways⁵⁶), they can facilitate GABA neurons,³⁰ which are critical components of hippocampal rhythm generators.⁸ 5-HT6 receptor mechanisms can also exert an indirect modulatory role by controlling acetylcholine release,⁵⁷ which is particularly important in generating theta oscillations. Combination of 5-HT6 effects on glutamatergic, GABAergic, and cholinergic mechanisms may underlie the nonuniform 5-HT6 effects on low and high frequency theta components under anesthesia (Figure 4A and C) and might also explain theta slowing in freely moving rats (Figure 2A and D). High and low-frequency components of RPO-elicited theta in anesthetized rats show different sensitivity to atropine,⁵⁸ just as the faster type 1 and the slower type 2 theta in freely moving rats.^19,59 Thus, theta slowing might indicate a predominance of type 2 theta in the hippocampus and/or the associated behaviors, which however could not be differentiated due to technical limitations of this study.

Alternatively, 5-HT6 active compounds may act on subcortical structures which participate in the control of hippocampal activity. Although detailed histological data is still missing, this mechanism should be considered given that other 5-HT receptors which are also expressed in the hippocampus modulate theta acting at subcortical targets. For example, the 5-HT1A agonist 8-(OH)-DPAT was shown to increase theta rhythm when injected systemically⁶⁰ just as its local MRN administration induces lasting theta at short latency,²⁰ similar to MRN lesion or its pharmacological suppression by local injection of GABA agonists^42,43 or excitatory acid receptor antagonists.⁶¹ This indicates a dominant action on autoreceptors in the MNR, although it does not exclude an additional direct 5-HT1A effect on the hippocampal theta generator. 5-HT2 receptors are also expressed in the hippocampus and midbrain raphe,^62,63 but the effect of systemic injection of 5-HT2A and 5-HT2C compounds on theta rhythm^21,22 is most consistent with their action on the MS theta pacemaker (the effect could be replicated by local MS agonist and antagonist administration; Nguy, Wang, Hajos, Kocsis, unpublished). In the MS, serotonergic projections terminate on GABA neurons⁶⁴ which provide theta rhythmic input to basket and chandelier GABA cells in the hippocampus.⁶⁵ 5-HT6 receptors are absent in the MRN, and their expression in the MS is relatively low.^4,5 Similar to the 5-HT2 receptors, the primary location of 5-HT6 receptors is on GABA neurons and both receptors positively modulate GABA cells, although through different G-proteins and different intracellular pathways. Yet they affect different theta parameters and thus are unlikely to use a common target. The supramammillary nucleus is a possibility as a target, which was proposed to control theta frequency as opposed to MS controlling theta amplitude⁶⁶ and, thus, could explain the primary effects of 5-HT2 and 5-HT6 agonists to decrease theta amplitude and theta frequency, respectively. 5-HT6 receptor expression was shown in several hypothalamic nuclei, although the supramammillary nucleus was not specifically investigated.⁴ Hypothalamic targets may also be involved in the strong wake-promoting effect of 5-HT6 agonist. Two recent sleep studies^24,25 using 5-HT6 receptor antagonists, RO-4368554 and SB-399885, reported significant effects on sleep–wake behavior which was however relatively modest compared with 5-HT2 compounds (see also ref (21)) and somewhat contradictory, as the same compound was found to promote sleep in one study²⁴ during the dark period of the day and promoting wake (during the light period) or no effect (during night) in the other.²⁵ It was proposed that the effect may involve 5-HT6 receptors which facilitate histamine and hypocretin cells in the hypothalamus.²⁴ Although speculative at this time, the wake-promoting effect of EMD386088 found in our study would be consistent with this suggestion.

The findings of this study add a new dimension to pharmacological characterization of compounds acting on the 5-HT6 receptor and help to identify their therapeutic potential. We have recently reviewed evidence that anxiolytics and pro-cognitive drugs induce characteristic changes in hippocampal theta oscillations and proposed that these alterations can be used as effective biomarkers.³⁴ Thus, all traditional and novel anxiolytics were shown to decrease the frequency of brainstem stimulation-induced theta rhythm independent of whether they act on GABAA receptors as barbiturates and benzodiazepines,⁶⁷⁻⁶⁹ on glutamatergic transmission, for example, group II metabolic glutamate agonists,⁷⁰ on serotonergic transmission as buspirone⁶⁷ and fluoxetine,⁵⁰ or on membrane channels as phenytoin⁷¹ and pregabalin.⁷² On the other hand, many pro-cognitive compounds were shown to increase theta amplitude, including histamine H3 receptor antagonists,³⁷ norepinephrine reuptake inhibitor,^73,74 and alpha7 nicotinic agonists,⁷⁵ whereas drugs inducing cognitive deficits, for example, 5-HT2c²¹ and CB1 agonists⁷⁶ or NMDA antagonists,⁷⁷ are also decreasing theta power.

Behavioral studies implicated 5-HT6 receptors both as potential anxiolytics and cognitive enhancers. In particular, several 5-HT6 agonist showed antidepressant and/or anxiolytic effects^29,30 including EMD386088,³¹ the anxiolytic potential of which was corroborated in the present study based on the strong predictive value of the change in theta frequency. Some 5-HT6 antagonists showed similar effects in behavioral tests,^32,33 but the findings of this study, at least for the antagonist SAM-531, failed to provide supporting evidence.

Despite strong evidence in a variety of preclinical studies showing cognitive improvement,^7,26 neither the 5-HT6 agonist nor the antagonist tested in this study enhanced theta oscillations. Although theta band power increased after EMD386088 injection in freely moving rats, this effect was due to increased waking activity rather than a direct effect on the theta generator. It should be noted, however, that we only used acute administration of 5-HT6 receptor compounds in normal, drug-free rats. A detailed review of the role of 5-HT6 receptor in cognitive function by Fone²⁸ emphasized that the outcome of behavioral experiments strongly depended on the task and concluded that these compounds do not improve memory under normal conditions, but can reverse cognitive deficits induced by pharmacological interventions that attenuate cholinergic or glutamatergic mechanisms. Recent studies have reported that 5-HT6 receptor antagonists can also reverse cognitive deficits in chronic models of pathological conditions with impaired cholinergic and glutamatergic function, such as Alzheimer’s disease⁷⁸ and schizophrenia^56,79 and that this may involve complex molecular mechanisms beyond the adenylyl cyclase cascade.⁵⁶ Since impairment of theta oscillations was shown in such models,^77,80 future studies might reveal enhanced theta associated with cognitive enhancement by 5-HT6 antagonists under these conditions.

Methods

Animals and Surgery

This study included three series of experiments, one conducted in freely behaving rats and the other two under urethane anesthesia. Male Sprague–Dawley rats (body weight 350–450 g) were kept under standard temperature and humidity controlled laboratory conditions with food and water ad libitum in a 12 h light/12 h dark cycle. All experiments were performed in accordance with National Institute of Health guidelines and were approved by the Institutional Animal Care and Use Committee of Beth Israel Deaconess Medical Center.

Rats for survival surgery were given a mixture of ketamine and xylazine (35–45 and 5 mg/kg, respectively, i/p), and those for acute experiments were anesthetized using urethane (750 mg/mL, 1.5 g/kg body weight, i/p). In each rat, pairs of stainless steel wires were implanted for recording hippocampal field potentials on both sides at AP −3.7 mm, Lat ±2.2 mm, DV −3.5 mm, relative to bregma, and fixed to the skull with dental acrylic. The tips of the electrodes were separated by ∼1 mm and positioned so the deeper electrode reached below the hippocampal fissure and the shorter would be located in the CA1 oriens/pyramidal layer. Stainless steel screws were fixed in the skull over the cerebellum and olfactory bulb to serve as ground and reference electrodes. In anesthetized rats, two additional pairs of wires were implanted for electrical stimulation in the nucleus reticularis pontis oralis (RPO, AP −7.8 mm, Lat ±1.5 mm, DV −8.0 mm) on either side, through guide cannulas, which were 2 mm shorter than the electrodes. Rats for freely moving recordings did not receive RPO electrodes but were equipped instead with an additional screw electrode to record EEG over the frontal cortex and two EMG wires used for polysomnography. The electrodes were connected to two 6-channel connectors (Plastics One, Inc.), and the assembly was fixed to the skull using dental cement.

Drugs

Two compounds were used in this study. EMD386088 (purchased from Tocris Bioscience) is a potent 5-HT6 agonist with selectivity (EC₅₀ = 1.0 nM) over other 5-HT receptor agonists (IC₅₀ = 7.4 for 5-HT 6 compared with 110, 180, 240, 450, 620, 660, 3000 nM for 5-HT1D, 5-HT1B, 5-HT2A, 5-HT2C, 5-HT4, 5-HT1A, 5-HT7, respectively⁸¹). The second compound, SAM-531 (provided by Pfizer, Inc.) is a 5-HT6 antagonist with >200-fold selectivity over more than 80 targets.⁸² EMD386088 was dissolved in saline, and doses of 5 and 10 mg/kg were injected in freely moving rats (n = 4 and n = 3, respectively), while urethane anesthetized rats (n = 5) received 10 mg/kg injections. SAM-531 was dissolved in 95% PEG200 (Pfizer) and 5% DMSO (Sigma). Rats in chronic experiments were injected with 3 and 17 mg/kg SAM-531 (n = 9 and n = 8, respectively), whereas urethane anesthetized rats received a single dose of 17 mg/kg SAM-531 (n = 8). Nine urethane anesthetized rats received a combination of 10 mg/kg EMD386088 and 17 mg/kg SAM-531. Saline and PEG200 and DMSO solution were used as vehicle controls. All drugs and vehicles were injected subcutaneously. Nine rats were used for chronic recordings, all of which received vehicle and SAM-531 injection (9 rats received 17 mg/kg and 8 received 3 mg/kg). Five rats of this group also received EMD386088 (n = 2 received both 5 and 10 mg/kg, 2 rats received 5 mg/kg, and one rat received 10 mg/kg only). At least 3 days were left between injections to allow washout. Anesthetized rats (total of n = 22) only received one treatment each, that is, either agonist (n = 5) or antagonist (n = 8), or a combination of the two (n = 9).

Electrophysiological Recordings

In chronic experiments, daily recordings started 7–10 days after surgery. In the morning, 1 h after light on (ZT1). The rats were placed in their own dedicated recording boxes and connected to a 12-channel slip-ring commutator, through a 50 cm cable (Plastics One, Inc.), which allowed free turning and moving around in the box. The recording boxes were made of white opaque plastic, had relatively small horizontal dimensions of 19.5 cm × 29 cm and larger than normal height of 28.5 cm to allow rearing without compromising electrophysiological recordings. The boxes had regular bedding and standard water and food containers. Cortical and hippocampal activities (filtered between 0.1 and 100 Hz) and neck muscle EMG were continuously recorded for 24 h. Injections started about a week later when the normal sleep–wake proportions were restored. Injections were made after 4 h of control recording so the injection occurred between noon and 1 pm (ZT5-ZT6). Statistical analysis was performed on daytime recordings when the rats spend most of their time sleeping or in quiet waking.

In the experiments using anesthetized rats, hippocampal field potentials were recorded throughout the experiment. The signals were amplified and filtered between 0.15 and 100 Hz (model 3500, A-M Systems). RPO stimulation started after a 30–40 min control recording of spontaneous activity. For electrical stimulation of the RPO, 0.1 ms square waves were used at 100 Hz, for 10 s (AMPI Master 8 stimulator with IsoFlex constant current unit). The stimulus intensity varied between animals and was set in each individual experiment using the well-known linear characteristics between stimulus intensity and theta frequency (see, e.g., refs (36, 37, and 58)). Thus, RPO was stimulated first at different intensities to identify the threshold to elicit theta rhythm in the hippocampus and the intensity necessary to elicit the largest response, that is, above which the frequency no longer increases. Test stimulations then used 4 or 5 stimulus intensities equally spaced from threshold to maximum (usually between 0.1 and 1.5 mA). Such sequences of stimuli were applied at least 5 times, in the control period and then repeated every 15–20 min for 90 min after drug injection.

Data Analysis

In chronic experiments, all signals were sampled at 256 Hz and power spectra were calculated using fast Fourier transform on 16 s long windows (i.e., 4096 data points). Sleep–wake states were identified for each 16 s segment using the level of delta power in the frontal cortex (1–4 Hz) and theta in the hippocampus (5–10 Hz) along with the root-mean-square value of the EMG, according to common practice. REM sleep was identified by atonia and dominance of hippocampal theta rhythm. Identification of slow wave sleep was based on large delta waves in both cortex and hippocampus along with the occurrence of sleep spindles and low EMG activity. Awake state was detected primarily by high level of motor activity (active waking) and/or low amplitude cortical EEG, whereas hippocampal recording could include theta and nontheta segments, but not delta waves. To specifically quantify the effect on the theta generators rather than on the circuits switching between sleep–wake or between theta and nontheta states, theta segments, associated with awake exploratory behavior or REM sleep, were identified when theta power was at least 4 times higher than delta and theta power in the 5–10 Hz frequency range was calculated separately for waking and REM sleep episodes and averaged over 1 h periods. Theta power for analysis of drug effect was calculated as total power in the 5–10 Hz band irrespective of behavior, as well as during specific theta segments of REM sleep and active waking. Theta frequency was only calculated in theta states as the local maximum in the theta band. For statistical analyses, two-tailed paired Student’s t test and two-way ANOVA (drug/dose and time as main factors) and one-way repeated measures ANOVA for time with post hoc Bonferroni pairwise comparisons were used when analysis of variance indicated statistical significance.

In anesthetized rats, electrophysiological signals were sampled at 1000 Hz and power spectra were calculated on 1 s windows (chronic recordings). Theta power was calculated between 4 and 10 Hz, that is, in the range where spontaneous theta oscillations appear in this preparation. Elicited theta was calculated as average of five 10 s segments. Theta was also calculated in 60 s baseline segments before drug injection and used to normalize the signal amplitude between experiments; that is, power was expressed relative to this baseline average. Theta frequency and power before and after drug injection were compared using two-tailed paired Student’s t test.

This study was supported by NIH (MH087777, HL095491) and a research grant from Pfizer, Inc.

The authors declare no competing financial interest.

Funding Statement

National Institutes of Health, United States