The Role of the Serotonergic System in REM Sleep Behavior Disorder

Abstract

Study Objectives:

REM sleep behavior disorder (RBD) can be induced by antidepressants, especially serotonin reuptake inhibitors (SSRI), thus a role of the serotonergic system in the pathogenesis of RBD has been proposed. However, the serotonergic system integrity in idiopathic RBD (iRBD) is still unknown. We aimed to study brain stem serotonergic system integrity, by means of ¹²³I-FP-CIT-SPECT, in a group of iRBD patients as compared to normal subjects.

Design:

Single-center, prospective observational study.

Setting:

University hospital.

Patients or Participants:

Twenty iRBD outpatients and 23 age-matched normal controls.

Measurements and Results:

The diagnosis of RBD was determined clinically and confirmed by means of overnight, laboratory-based video-polysomnography. Both iRBD patients and normal subjects underwent ¹²³I-FP-CIT-SPECT as a marker of dopamine transporter (DAT) at basal ganglia level and of serotonin transporter (SERT) at brainstem and thalamus levels. ¹²³I-FP-CIT-SPECT images were analyzed and compared between iRBD patients and controls by means of both region of interest analysis at basal ganglia, midbrain, pons and thalamus levels, and voxel-based analysis, taking into account age and the use of SSRI as confounding factors. No difference in ¹²³I-FP-CIT-SPECT specific to nondisplaceable binding ratios (SBR) values was found between iRBD and normal subjects at brainstem and thalamus levels while iRBD patients showed lower SBR values in all basal ganglia nuclei (P < 0.0001) compared to controls.

Conclusions:

These results suggest that the serotonergic system is not directly involved in RBD pathogenesis while confirming nigro-striatal dopaminergic deafferentation in iRBD.

Citation:

Arnaldi D, Famà F, De Carli F, Morbelli S, Ferrara M, Picco A, Accardo J, Primavera A, Sambuceti G, Nobili F. The role of the serotonergic system in REM sleep behavior disorder. SLEEP 2015;38(9):1505–1509.

INTRODUCTION

Acute REM sleep behavior disorder (RBD) can be induced by the use of antidepressants, especially serotonin reuptake inhibitors (SSRI),^1–7 suggesting a role of the serotonergic system in the pathogenesis of RBD. Overall, serotonin promotes the wake state and inhibits REM sleep.⁸ The cholinergic neurons in the pons are under the inhibitory control of brainstem serotonergic and noradrenergic neurons and they trigger REM sleep by activating the glutamatergic sublaterodorsal nucleus.⁹ Then, the glutamatergic pathway activates glycinergic and GABAergic neurons, inhibiting motoneurons as well as brainstem serotonergic and noradrenergic neurons.⁹ Thus, the physiological reduction in serotonin release during REM sleep reinforces REM atonia by reducing motoneuron activation,¹⁰ while an abnormal increase in serotonergic tone (possibly due to SSRI) might induce REM sleep without atonia (RSWA). An animal study is in agreement with this hypothesis by showing that serotonin cells in the dorsal raphe fail to switch off during REM sleep in cats with experimentally induced RSWA.¹¹ According to this hypothesis, an increased serotonergic tone would be expected in RBD patients, compared to normal subjects. However, the serotonin system integrity in idiopathic RBD (iRBD) patients has not been evaluated yet. With the hypothesis of an altered serotonin system at brainstem level in iRBD patients compared to normal subjects, we performed ¹²³I-FP-CIT single photon emission computed tomography (SPECT) scans to assess serotonin transporter (SERT) brainstem level in a group of consecutive iRBD patients and we compared findings with a group of normal subjects. In fact, ¹²³FP-CIT-SPECT is widely used as a marker of dopamine transporter (DAT) binding at basal ganglia level,^12–14 but it has also been used as a marker of SERT binding at brainstem level, assuming that tracer binding at this level is predominantly related to SERT.^15–17

METHODS

Subjects

Twenty-four consecutive iRBD outpatients were recruited at the sleep unit of our University Department. The diagnosis of iRBD was made according to the second edition of the International Classification of Sleep Disorders (ICSD-2) criteria¹⁸ by a sleep disorders expert (DA) based on the results of both video polysomnography (PSG) findings and clinical interviews with patients and bed partners. All patients underwent brain magnetic resonance imaging (MRI), or computed tomography (CT) in the case MRI was unfeasible, to rule out other brain diseases. Patients with brain infarcts on MRI/CT or with a history of stroke or transient ischemic attacks were excluded, whereas the presence of small white matter hyperintensities on MRI was not an exclusion criterion if they did not involve the basal ganglia and the pons-mesencephalon. Dementia was excluded by means of clinical interview and questionnaires for activities of daily living (ADL) and instrumental ADL. The Mini-Mental State Examination (MMSE) was used as a measure of global cognition. The Beck depression inventory-II (BDI-II) was administered to rate depression. Patients with any abnormal finding suggestive of parkinsonism, other neurological or psychiatric disorder, or showing moderate or severe sleep apnea (apnea-hypopnea index ≥ 15) were excluded. Twenty iRBD patients matched these criteria and were enrolled in the study. ¹²³I-FP-CIT-SPECT was performed in all iRBD patients in order to explore the DAT binding at basal ganglia level and the SERT binding at brainstem level. A group of 23 normal subjects in the same age range as patients served as controls for ¹²³I-FP-CIT-SPECT comparison with iRBD patients. Their healthy condition was carefully checked by reviewing their general medical history and clinical examination; the exclusion criteria were the same as for iRBD patients. None of the controls had a history of dream-enacting behaviors. MMSE was performed and only subjects with a score ≥ 28 were enrolled. The study protocol met the approval of the local Ethics Committee and an informed consent form was signed by all participants, in compliance with the Helsinki Declaration of 1975.

Polysomnographic Recording

All iRBD patients underwent overnight video-PSG (BE-Plus LTM, EBNeuro, Florence, Italy), performed by technicians with expertise in the field and the sleep scoring was performed following current criteria.¹⁹ Polysomnographic derivations were placed according to recommended rules of standard criteria¹⁹ in order to evaluate sleep scoring, respiratory, cardiac, and limb events. In order to support the diagnosis of iRBD, EMG was evaluated by means of the cutoff value established by Montplaisir et al.²⁰ All patients were asked to withdraw melatonin and/or clonazepam for 2 weeks before the recording.

¹²³I-Ioflupane Single Photon Emission Computed Tomography (¹²³I-FP-CIT-SPECT)

Both iRBD patients and controls underwent ¹²³I-FP-CITSPECT. Scans were performed no more than 6 months apart from PSG. Images were acquired 3–4 h after intravenous administration of about 185 MBq of ¹²³I-FP-CIT (DaTSCAN, GE Healthcare, Little Chalfont, Buckinghamshire, UK) by means of a dual-head gamma camera equipped with low-energy, high-resolution collimators (Millenium VG, GE Healthcare, Little Chalfont, Buckinghamshire, UK), according to the European Association of Nuclear Medicine (EANM) guidelines.¹⁴ Six iRBD patients and 6 controls were taking SSRI for mild depressive trait, not fulfilling the DSM-IVr criteria for the diagnosis of depression, at the time of the scan, and they were all asked to hold the medication the day of the examination. Further technical data for SPECT acquisition and image reconstruction are detailed elsewhere.^{21 123}I-FP-CIT-SPECT scans were visually reported by a nuclear medicine physician (SM).

Image Analysis

Since DAT is a presynaptic membrane protein responsible for the reuptake of dopamine into dopaminergic nerve terminals and its density in extrastriatal regions is negligible,²² the ¹²³I-FP-CIT uptake at basal ganglia level reflects the nigrostriatal pathway integrity. The reconstructed ¹²³I-FP-CITSPECT images were exported in analyze format and processed by the automatic BasGan algorithm²³ based on a high-definition, three-dimensional (3D) striatal template, derived from Talairach's atlas (details in Calvini et al.²³) in order to evaluate the DAT binding at basal ganglia level. Background uptake was subtracted by putamen and caudate uptake as follows: (Caudate- or Putamen-uptake – Background-uptake)/ Background-uptake to compute specific to non-displaceable binding ratios (SBR). Subsequently, gamma camera-calibrated SBR values were corrected for age and gender based on the European normative database of the ENC-DAT study.²⁴ Since SERT is a presynaptic selective transporter of serotonin, it could be hypothesized that changes in SERT availability are seen when either external stimuli, such as drugs, disturb the homeostasis to maintain the serotonergic tone or in the presence of alteration of the serotonergic system integrity. Thus, studying SERT availability at brainstem and thalamus levels where the highest density of SERT was observed²⁵ should provide an indirect measure of the serotonin system, as DAT density at those levels is negligible.²² In order to study SERT binding, ¹²³I-FP-CIT-SPECT images, exported in analyze format, were analyzed using SPM8 package (Wellcome Department of Cognitive Neurology, London, UK) implemented in Matlab 7.5 (MathWorks, Natick, Massachusetts, USA). A customize brain ¹²³I-FP-CIT-SPECT template was built from 20 brain MRI and SPECT scans of healthy subjects that are not included in the control group of the present study. Template editing was performed according to published procedures.^{26,27 123}I-FP-CIT-SPECT scans were subjected to affine and nonlinear spatial normalization into the MNI space, using the ¹²³I-FP-CIT-SPECT template. Next, a set of 3D regions of interest (ROIs) defining the pons, the midbrain, and the thalamus was identified by means of the SPM8 WFU-PickAtlas toolbox. An occipital cortex (OC) ROI served as reference region. Mean counts per voxel were extracted for each ROI by means of the SPM8 MarsBaR toolbox. SBR in each brainstem ROI was computed according to the formula SBR = (target ROI-OC)/OC.

Statistics

Unpaired t-test was performed for each ROI (right and left caudate and putamen, pons, midbrain, and thalamus) to compare SBR values between iRBD and control groups. Since the use of SSRI could affect SERT binding at brainstem nuclei,^28,29 ROI analysis was repeated considering only subjects not taking SSRI, including 14 patients and 17 controls. To further check for between-group differences, voxel-based analysis was also performed using SPM8. The spatially normalized set of images was smoothed with an isotropic gaussian kernel of FWHM 12 mm to blur individual variations in gyral anatomy and to increase the signal-to-noise ratio. Resulting counts were normalized to OC values. As a first step, the “whole brain” comparison was performed in order: (i) to verify the expected nigrostriatal deafferentation in iRBD patients,^30–32 and (ii) to explore other differences between groups. SPM t-maps were thresholded using a P < 0.005 at voxel level, uncorrected for multiple comparisons. Only clusters containing > 100 significant voxels were considered for subsequent analysis. The selected threshold is accepted as a reasonable choice, as it takes into account the need to balance between type I and type II errors, and also considering the relatively low sensitivity of SPECT in the lack of repeated measures.³³ The significance of identified regions was established by means of a 2-sample t-test design at a P < 0.05, corrected for multiple comparisons with the false discovery rate (FDR) option at the cluster level and taking into account age and the use of SSRI as “nuisance” variables. The comparison was performed in both ways: (1) Controls minus iRBD and (2) iRBD minus Controls. In order to locate significant clusters, correction of SPM coordinates to match the Talairach coordinates was achieved by using the Lancaster transform.³⁴ As a last step, in order to explore between-group differences in SERT binding at brainstem and thalamus levels, we performed the same SPM comparisons, using an ROI analysis by means of the PickAtlas toolbox implemented in the SPM 2-sample t-test design, at pons, mid-brain, and thalamus levels independently, and taking age and the use of SSRI into account as “nuisance” variables. IBMSPSS Statistics for Windows (Version 19.0. Armonk, IBM Corp., NY, USA) and SPM8 implemented in Matlab 7.5 were used for statistical analysis.

RESULTS

Main PSG features are shown in Table 1. SBR values were significantly lower in iRBD patients than controls in all basal ganglia nuclei (P < 0.0001). SBR at pons, midbrain, and thalamus levels did not differ between groups, even when the comparison was limited to the subjects not taking SSRI. SBR mean values, standard deviation, and relative P values are shown in Table 2.

Table 1.

Main polysomnographic features of iRBD patients.

Table 2.

Main clinical, demographic and ¹²³I-FP-CIT-SPECT SBR features (mean ± SD) of the two groups.

The voxel-based “whole brain” analysis showed a significant reduction in ¹²³I-FP-CIT uptake in iRBD group compared to controls at basal ganglia level of both hemispheres (FDR-corrected P < 0.0001 at cluster level). No significant difference was found with the SPM-implemented ROI analysis at brainstem and thalamus levels.

DISCUSSION

Our data show that the SERT availability at brainstem level, as evaluated by means of ¹²³I-FP-CITSPECT, is not significantly different in a group of iRBD patients in comparison with normal subjects in the same age range. This finding does not confirm the hypothesis of an altered SERT availability in iRBD and is in line with recent data showing no difference in SERT binding at brainstem level, as evaluated by means of [¹¹C]-DASB positron emission tomography, between PD patients either with or without RBD symptoms.³⁵ In that study the patients had mild to moderate PD, with mean disease duration of 6 years. Serotonergic raphe complex neurons are degenerated in PD, even in a relatively early stage of the disease, and serotonergic deficit is involved in several non-motor symptoms.³⁶ Thus, in PD patients any further subtle changes in serotonergic functioning, possibly responsible for RBD symptoms could be blurred by the neurodegenerative process due to PD pathology. A confirmation of the finding of Kotagal et al.³⁵ in a group of patients with RBD but without PD was needed, and our data seem to provide it, as our patients had iRBD, thus without parkinsonism or other neurological disease, and they all have a PSG-confirmed diagnosis of RBD. Indeed, Kotagal et al. evaluated the presence of RBD by means of a questionnaire, and RBD was not confirmed by PSG. Thus, the present data and the data of Kotagal suggest that the serotonergic system, at least at brainstem level, is not involved in RBD. Several data have shown that antidepressant drugs, in particular SSRI, may induce or exacerbate RSWA as well as RBD.^{1,3,6,7,37–41} Moreover, RBD is strongly associated with depression and antidepressant use.^{2,4,5,42–44} However, Postuma et al.⁴⁵ recently suggested that antidepressants unveil a subclinical RBD, triggering an early clinical presentation of RBD nonetheless due to underlying neurodegeneration. In fact, even if acute RBD has been reported in patients taking SSRI drugs, the prevalence of dream enactment in patients under antidepressant therapy is only about 6%, with slightly higher incidence in older subjects.^2,4,5,43 If the increased serotonergic tone provoked by antidepressant therapy induced RBD symptoms by itself, a higher prevalence of RBD in patients taking antidepressants should be expected. An alternative explanation is that antidepressant drugs induce RBD because of their anticholinergic effect. Although the effect of acetylcholinesterase inhibitors on RBD symptoms is still controversial,¹ an involvement of the cholinergic system in RBD pathophysiology has been suggested.^35,46,47 However, in a recent neuropathological study no difference has been found in brainstem cholinergic nuclei between Lewy body disease patients either with or without RBD.⁴⁸ Our finding suggest that the serotonergic system is not directly involved in the pathogenesis of RBD, even if an increased serotonergic tone could unveil acute RBD in predisposed subjects. This notion, translated into clinical practice, suggests that an antidepressant-induced RBD could reveal subjects with high risk to develop a neuro-degenerative disease. A limitation of the present study is that ¹²³I-FP-CIT has not a selective affinity to the SERT. DAT/SERT selectivity for ¹²³I-FP-CIT is 2.8:1.⁴⁹ However, DAT and SERT show different distributions in subcortical cerebral structures. DAT is mostly expressed at basal ganglia level while SERT levels are highest in the thalamus and midbrain, where DAT is extremely low.^50–52 Thus, it is reasonable to assume that ¹²³I-FPCIT uptake at basal ganglia level mainly reflects DAT availability whereas at midbrain and thalamus level it reflects SERT availability. FP-CIT has also shown some, albeit lower, sensitivity to the noradrenaline molecular transporters,⁵³ even if its extrastriatal binding has been primarily attributed to SERT binding.¹⁵ Moreover, the noradrenergic system might also be involved in RBD pathogenesis¹⁰ since noradrenergic antagonist drugs have been reported to induce RBD.^54,55 However, both dorsal and medial raphe nucleus could exhibit serotonin and the connections involved in the control of sleep of both nuclei are found in both midbrain and pons,⁵⁶ while the noradrenergic locus coeruleus is the pons. We independently evaluated ¹²³I-FP-CIT uptake at both levels, and no differences were found at either level. It has been suggested that approximately 70% of FP-CIT thalamic uptake reflects SERT binding¹⁵; thus, modifications of FP-CIT thalamic uptake could at least partially reflect SERT availability. However, no FP-CIT uptake difference between iRBD and controls has been found in the thalami. Finally, ¹²³I-FP-CIT binding could be influenced by several factors such as age^57–59 and SSRI use.^28,29 Thus, we also performed ROI analysis taking into account only patients not taking SSRI; moreover, SBR values at basal ganglia level were corrected for age and gender based on the European normative database of the ENC-DAT study.²⁴ Finally, all the voxel-based analysis were performed using both age and the use of SSRI as “nuisance” covariates. In conclusion, we found that the SERT availability, evaluated by means of ¹²³I-FP-CIT-SPECT, in a group of PSG-confirmed iRBD patients is comparable to the SERT availability in normal subjects. This finding suggests that the serotonergic system is not directly involved in the pathogenesis of RBD. Further, longitudinal studies, possibly with more serotonin-selective tracers, are needed to confirm these data.

DISCLOSURE STATEMENT

This was not an industry supported study. Dr. Nobili has received fees from Piramal for lectures and from Eli-Lilly for board participation. The other authors have indicated no financial conflicts of interest. The work was performed at Clinical Neurology, Dept. of Neuroscience (DINOGMI), University of Genoa, IRCCS AOU San Martino-IST, Genoa, Italy.