REM Sleep Characteristics in Narcolepsy and REM Sleep Behavior Disorder

Abstract

Study Objectives:

To assess the presence of polysomnographic characteristics of REM sleep behavior disorder (RBD) in narcolepsy; and to quantify REM sleep parameters in patients with narcolepsy, in patients with “idiopathic” RBD, and in normal controls.

Design:

Sleep laboratory study

Participants:

Sixteen patients with narcolepsy and cataplexy matched for age and sex with 16 patients with “idiopathic” RBD and with 16 normal controls were studied.

Measurements and Results:

Higher percentages of REM sleep without atonia, phasic electromyographic (EMG) activity, and REM density were found in patients with narcolepsy than normal controls. In contrast, RBD patients had a higher percentage of REM sleep without atonia but a lower REM density than patients with narcolepsy and normal controls. Based on a threshold of 80% for percentage of REM sleep with atonia, 50% of narcoleptics and 87.5% of RBD patients had abnormal REM sleep muscle activity. No significant behavioral manifestation in REM sleep was noted in either narcoleptics or controls. We also found a higher frequency of periodic leg movements during wake (PLMW) and during sleep (PLMS) in narcoleptic patients compared to controls.

Conclusions:

The present study demonstrates abnormalities in REM sleep motor regulation with an increased frequency of REM sleep without atonia, phasic EMG events and PLMS in narcoleptic patients when compared to controls. These abnormalities were seen more prominently in patients with RBD than in narcoleptics, with the exception of the PLMS index. We proposed that dysfunctions in hypocretin/dopaminergic system may lead to motor dyscontrol in REM sleep that results in dissociated sleep/wake states.

Citation:

Dauvilliers Y; Rompré S; Gagnon JF et al. REM sleep characteristics in narcolepsy and REM sleep behavior disorder. SLEEP 2007;30(7):844-849.

INTRODUCTION

NARCOLEPSY IS CHARACTERIZED BY THE PRESENCE OF EXCESSIVE DAYTIME SLEEPINESS AND CLINICAL MANIFESTATIONS OF DISSOCIATED RAPID EYE MOVEMENT (REM) sleep including cataplexy, hypnagogic and hypnopompic hallucinations, and sleep paralysis.¹ Normal REM sleep is characterized by tonic features, including cortical EEG desynchronization and muscle atonia, and phasic events, including bursts of rapid eye movements, phasic activities of both chin and limb EMG, and cardiorespiratory variability.² Abnormalities in REM sleep motor regulation have been described in narcolepsy, including persistence of muscle tone, excessive twitching, and periodic leg movements during sleep (PLMS).^3–5

The presence of REM sleep without atonia associated with prominent motor behavioral manifestations associated with dreaming during REM sleep is the key feature of a condition called REM sleep behavior disorder (RBD).⁶ RBD may occur alone and is then called “idiopathic” RBD. However, it is often associated with other neurological diseases such as lesions of the brainstem, neurodegenerative disease, especially synucleinopathies, and narcolepsy.^7–9 In these latter cases, it is termed “secondary RBD.”

Patients with narcolepsy and those diagnosed with RBD share several common polygraphic features, especially signs of REM sleep dysregulation. Both have increased PLMS at night with a specific increase of PLMS in REM sleep^10–11; both have dissociated manifestations of REM sleep with a loss of REM sleep muscle atonia in RBD and the inappropriate occurrence of atonia during wakefulness (cataplexy) in narcolepsy.^1,3–5 The loss of hypocretin neurons within the lateral hypothalamus causes narcolepsy in humans.¹² Recent results based on detailed anatomy and lesion experiments in rats have identified independent pathways in brainstem that mediate the atonia and EEG phenomena of REM sleep.¹³ As hypocretin neurons are excitatory and active during wakefulness with strong projections to the brainstem structures implicated in REM sleep motor regulation,¹⁴ decreased hypocretinergic tone may cause REM sleep without atonia and RBD in humans.

The aim of the present study was to assess in narcoleptic patients the REM sleep characteristics known to be impaired in RBD: REM sleep without atonia, REM sleep chin EMG phasic activities, and REM density, and to compare these results with results obtained in patients with RBD and normal controls.

METHODS

Subjects

Sixteen patients with narcolepsy and cataplexy (11 men and 5 women; mean age at PSG ± SD, 59.0 ± 6.5 years) were included. Each patient had a clinical evaluation by a sleep specialist followed by at least one night of polysomnographic recording. Inclusion criteria for narcolepsy were the presence of both excessive daytime sleepiness and cataplexy, the presence of the HLA-DR2 antigen, and at least one sleep onset REM period (SOREMP) during the multiple sleep latency test (MSLT).¹⁵ The standard criterion for the diagnosis of narcolepsy is the presence of 2 SOREMPs⁶; however, we reported recently that the number of SOREMPs decreases with age, and a substantial number of narcoleptics of more than 50 years of age had only 1 SOREMP.¹⁶ Mean age at onset of narcolepsy was 24.2 ± 14.1 years, and mean duration of the disease was 36.3 ± 15.7 years. Results of the MSLT showed a mean sleep latency of 2.7 ± 1.7 min. All patients had sleep latency <7 min on the MSLT. The frequency of cataplectic attacks was assessed on a scale from 1 to 5, in which 1 represents one or fewer cataplectic attack per year, 2 represents more than one cataplectic attack per year but fewer than one per month, 3 represents more than one cataplectic attack per month but fewer than one per week, 4 represents more than one cataplectic attack per week but fewer than one per day, and 5 represents severe cases with at least one cataplectic attack per day.^16,17 The mean frequency of cataplexy was 3.9 ± 1.0. The Epworth sleepiness scale (ESS)¹⁸was administered to assess daytime sleepiness, and the mean score was 19.5 ± 2.1. The presence of hypnagogic hallucinations and sleep paralysis was also noted.

Narcoleptic patients were randomly selected from our narcoleptic population in order to be sex- and age-matched with patients with “idiopathic” RBD. The selection process was carried out blind with regard to the presence of abnormal behavior during nocturnal sleep recording.

Sixteen patients with “idiopathic” RBD (11 men and 5 women; mean age at PSG ± SD, 59.9 ± 7.9 years) were included. The diagnosis of RBD was made on the basis of the presence of a clinical history of agitated motor activity occurring repeatedly during sleep for at least 6 months, associated with dream mentation, and the presence of at least one documented episode of abnormal motor behavior occurring during REM sleep in the sleep laboratory. Subjects were excluded if they had a history of head trauma, seizure, encephalopathy of any cause, cerebrovascular disease, or conditions known to be associated with RBD, including Parkinson disease, multiple system atrophy, and dementia with Lewy bodies.^7–9

Results obtained in these 2 groups of patients were compared to those of 16 normal controls matched for age and sex. Exclusion criteria for normal controls were excessive daytime sleepiness, cataplexy, nocturnal behavioral manifestations, or any other condition known to be associated with narcolepsy or RBD. None of the patients or controls had any psychiatric disorder based on the DSM-IV criteria, a diagnosis of RLS, or any other condition known to be associated with PLMS such as anemia, renal insufficiency, rheumatoid arthritis, diabetes, polyneuropathies, based on medical history and clinical examination. Finally, none of patients or controls was taking any medication known to influence sleep or muscle activity at the time of the sleep laboratory investigation.

Polysomnography Recordings

All subjects were recorded for one night in the sleep laboratory. In addition, narcoleptic patients were recorded during 5 naps scheduled at 2-hour intervals (MSLT) during the following day. The recording montage included left and right electro-oculograms, submental muscle EMG, and central (C3-A2) and occipital (O2-A1) EEG. Subjects were also monitored with infrared video to detect movements during REM sleep.

Sleep stages 1 to 4 were scored according to the method of Rechtschaffen and Kales,¹⁹ using 20-second epochs. According to this method,¹⁹ muscle atonia is a mandatory criterion to score REM sleep, therefore we could not use it for scoring REM sleep in patients with REM without muscle atonia. REM sleep was scored on the basis of EEG and electro-oculograms only.²⁰ The occurrence of the first REM epoch was used to determine the onset of a REM sleep period. The termination of REM sleep periods was identified by the occurrence of an EEG feature indicative of another stage (K complex, sleep spindle, or EEG sign of arousal) or by the absence of rapid eye movements during 3 consecutive minutes. The same method was used to score REM sleep in narcoleptic, RBD, and control subjects. REM sleep muscle atonia was scored according to a method described elsewhere.²⁰ Each 20-second epoch during REM sleep was scored as tonic or atonic depending on whether tonic chin EMG activity was present for ≥50% or <50% of the epoch. A baseline EMG signal for atonia was determined for each subject; it was found to be between 3 and 7 μV. Any EMG signal that was twice the amplitude measured during atonia and >10 μV was considered to be tonic. As previously described,²¹ REM sleep without atonia in patients is considered when there is tonic REM submental EMG activity for more than 20% of the total REM sleep duration. The threshold of abnormal REM sleep without atonia was also determined with the Received Operating Characteristics (ROC) curve methodology, based on data from RBD patients and controls.

Two types of phasic activity characteristics of REM sleep were analyzed. Phasic EMG density was scored from the submental EMG recording and represented the percentage of 2-second miniepochs of REM sleep containing phasic EMG events defined as any burst of activity lasting from 0.3 to 5 seconds with an amplitude exceeding 4 times the baseline EMG signal.²⁰ Rapid eye movement (REM) density was defined as the percentage of 2-second mini-epochs of REM sleep containing at least one rapid eye movement. The threshold abnormality of phasic EMG density in REM sleep was also determined with the Received Operating Characteristics (ROC) curve methodology.

Surface EMG from right and left anterior tibialis muscles were recorded to quantify periodic leg movements in sleep (PLMS). The PLMS were recorded and scored according to the method developed by Coleman.²² Only movements lasting 0.5 to 5 sec, separated by intervals of 4 to 90 sec, and occurring in series of ≥4 consecutive movements were counted as PLMS. Indices of PLMS were calculated separately in REM and NREM sleep in subjects with a PLMS index >5 per hour of sleep, a minimum requirement to have a sufficient number of movements to calculate indices in various sleep stages. The same criteria were used to score PLM while awake (PLMW) during the nocturnal sleep recording, but allowing for movement duration up to 10 seconds. The amplitude criterion for both PLMS and PLMW was at least 25% of the EMG potential recorded at the time of a voluntary movement performed prior to nocturnal PSG recording.

Recordings of oral and nasal airflow, thoracic and abdominal movements, and oximetry were performed to detect apneas and hypopneas. None of the patients or controls had an apnea index >5 or a respiratory event index >10.

Statistical Analyses

Data were presented as means ± SD. One-way analyses of variance (ANOVA) were performed to evaluate between-group difference on the polysomnography (PSG) data listed on Tables 1 and 2. Post hoc analyses were done using the Tukey HSD test. A Kruskall-Wallis test was performed to compare the ratio of REM/NREM PLMS indices in all 3 groups. Continuous variables were evaluated using t-tests, and dichotomous variables were assessed using Pearson χ² to evaluate differences between narcoleptic patients with REM sleep atonia and without atonia. Pearson correlation coefficients were calculated to assess the relationship between PLMS and PLMW indices, and between the percentages of REM sleep atonia, chin phasic EMG activity, and REM density in REM sleep.

Table 1.

Polysomnographic Characteristics of Patients with Narcolepsy, with “Idiopathic” RBD, and Controls

	Narcoleptics N=16	RBD N=16	Controls N=16	P
Men/women	11/5	11/5	11/5	Ns
Age at PSG	59.0 ± 6.5	59.9 ± 7.9	59.6 ± 9.5	Ns
Total sleep time	399.3 ± 57.0	376.7 ± 55.0	423.9 ± 39.3	0.04c
Sleep latency	5.4 ±3.9	21.7 ±19.7	12.6 ± 6.4	0.002a
REM sleep latency	28.6 ± 12.7	91.2 ±14.2	129.5 ± 24.7	0.001ab
Sleep efficiency	78.8 ±11.1	82.7 ± 9.9	87.4 ± 6.6	0.04b
% Stage 1	15.9 ±5.4	12.4 ± 7.9	12.9 ±8.0	Ns
% Stage 2	52.7 ± 9.1	65.3 ± 8.6	60.9 ± 8.9	0.008ab
% SWS	6.3 ± 5.3	4.7 ± 4.6	5.6 ± 7.3	Ns
% REM	25.1 ± 7.9	17.6 ±7.0	20.6 ±3.8	0.007a
REM efficacy	85.1 ±9.1	87.5 ± 10.2	87.8 ± 9.7	Ns
PLMW index	56.0 ± 34.4	33.7 ± 26.2	13.6 ±17.2	0.0002b
PLMS index	35.5 ±31.7	29.6 ± 32.8	9.7 ±16.3	0.03b
PLMS Index ≥5 (%)	87.5 (n=14)	81.2 (n=13)	37.5 (n=6)	0.004bcd
REM/NREM PLMS indexe	0.9 ± 0.8	22.1 ± 39.6	0.17 ±0.2	0.0006c
PLMS Index ≥10 (%)	81.2 (n=13)	68.8 (n=11)	25.0 (n=4)	0.003bcd

^aNarcoleptics vs RBD

^bNarcoleptics vs controls

^cRBD vs controls

^dChi-square test

^eREM/NREM PLMS index: Ratio of PLM index in REM sleep/PLM index in NREM sleep

Table 2.

REM Sleep Atonia, Chin Phasic EMG Density, and REM Density in REM Sleep in Narcoleptics, “Idiopathic” RBD, and Controls.

	Narcoleptics N=16	RBD N=16	Controls N=16	P
REM sleep atonia, %	70.5 ± 21.4	41.4 ± 27.4	89.2 ± 9.9	0.0000001a,b,c
Phasic EMG density, %	23.0 ± 11.9	29.3 ± 11.7	8.7 ± 4.2	0.000002b,c
REM density, %	28.4 ± 7.5	17.8 ± 9.8	21.0 ± 6.9	0.002a,b

^aNarcoleptics vs RBD

^bNarcoleptics vs controls

^cRBD vs controls

The ROC curves were calculated using the percentages of REM sleep without atonia and phasic EMG activity in REM sleep in both RBD patients and controls. We therefore determined the optimally efficient cut-point to minimize false-positive and false-negative errors.

RESULTS

Clinical Data and PLM Features in NREM and REM Sleep

All patients with RBD reported numerous episodes of behavioral manifestations during sleep associated with dream mentation and showed behavioral manifestations such as jerking, kicking, or gesturing during REM sleep (a mandatory inclusion criterion for the diagnosis of RBD). In contrast, none of the 16 patients with narcolepsy and none of the 16 normal controls reported nocturnal behavioral manifestations associated with dreaming at home. Also, according to experienced PSG technicians, none of these patients presented any behavioral manifestations in REM sleep during the sleep laboratory investigation. However, videotape recordings were not available to further assess the presence of abnormal behaviors during REM sleep.

Polysomnographic data of patients with narcolepsy, patients with RBD, and controls are shown in Table 1. PLMW and PLMS indices were higher in patients with narcolepsy and in patients with RBD than in controls, but the difference reached significance only in narcoleptics (Table 1). A strong correlation was found between PLMS and PLMW indices in each of the 3 groups (r between=0.89 and 0.97, all P <0.0001).

A greater proportion of patients with narcolepsy and of patients with RBD had a PLMS index >5 compared to controls (P = 0.004) or a PLMS index >10 (P = 0.003). The ratio of PLMS index in REM sleep over PLMS index in NREM sleep was higher for RBD patients than controls (P <0.0004); a similar trend was observed in narcoleptics when compared with controls (P = 0.062). When the subjects with a total PLMS index >5 per hour were considered, all patients with RBD had an index of PLMS >5 in REM sleep, whereas 78.6% of narcoleptics and only 33.3% of controls had a REM sleep PLMS index >5.

REM Sleep Muscle Atonia

The one-way ANOVA showed significant between-group differences in mean percentage of REM sleep muscle atonia (P <0.0000001) (Table 2). Post hoc tests showed that both RBD and narcoleptic patients had a lower percentage of REM sleep muscle atonia than control subjects (P <0.0001 and P = 0.03 respectively), but patients with narcolepsy were also found to have a higher percentage of REM sleep muscle atonia than RBD patients (70.5% ± 21.4% vs 41.4% ± 27.4%, P = 0.0001).

Figure 1a shows the distribution of REM sleep muscle atonia in all 3 groups. To establish the validity of the 20% threshold for REM sleep without atonia in our present population, we calculated the ROC curve using data from RBD patients and controls. At a cut-off of 20%, a sensitivity of 87.5% and a specificity of 81.3% were obtained with an area under curve (AUC) at 0.96. Based on that 20% threshold, 2 subgroups of narcoleptics can be identified: those with normal REM sleep atonia (≥80%: n= 8, 50%) and those with some loss of REM sleep atonia (<80%: n= 8, 50%) (Figure 1a). However, no difference was noted between these 2 subgroups in clinical characteristics such as age at onset, duration of the disease, frequency of cataplexy and sleep paralysis, score on the ESS, or polysomnographic parameters, including the number of SOREMPs and MSLT latency; and PLMW, total PLMS, and REM sleep PLMS indices.

Figure 1.

Distribution of (a) REM sleep muscle atonia, (b) REM sleep phasic EMG activity and (c) REM density in 16 patients with narcolepsy, 16 patients with “idiopathic” RBD, and 16 normal controls. Cutoffs at 80% and 15% were noted for the normal percentage of REM sleep muscle atonia and chin phasic EMG in REM sleep, respectively.

There was no sex effect on the percentage of REM sleep muscle atonia in RBD, narcoleptic, or control subjects. No correlation was found between the percentage of REM sleep muscle atonia and PLMS index in REM sleep in any of the 3 groups.

Phasic EMG and REM Densities in REM Sleep

Between-group differences were seen also for the mean percentage of chin phasic EMG activity in REM sleep (Table 2). Post hoc Tukey HSD analysis showed that patients with narcolepsy and RBD had a higher percentage of chin phasic EMG activity than control subjects (respectively 23.0 ± 11.9 vs 8.7 ± 4.2, P = 0.0006; and 29.3 ± 11.7 vs 8.7 ± 4.2, P= 0.00001). Figure 1b shows the distribution of chin phasic EMG activity in REM sleep in each group. To obtain a threshold for abnormal chin phasic EMG activity in REM sleep in our present population, we calculated the ROC curve using data from RBD patients and controls. The maximal operating point was 15, giving a sensitivity of 87.5% and a specificity of 93.75%, with an AUC at 0.97. Based on that threshold, 11 patients with narcolepsy (68.8%) had an increased phasic EMG activity in REM sleep (Figure 1b).

Between-group differences in REM density are also reported in Table 2. Post hoc Tukey HSD analysis showed that patients with narcolepsy had higher REM density than control subjects (P= 0.037) and RBD patients (P = 0.0019). No correlation was found between percentage of REM sleep atonia and REM density in any of the groups.

There was a negative correlation between percentage of REM sleep atonia and chin phasic EMG activity in REM sleep in narcoleptics (r= −0.49, P <0.05). In addition, we found a positive correlation between the percentage of chin phasic EMG activity in REM sleep and PLMS index in REM sleep in RbD patients (r = 0.55, P=0.028).

INTERPRETATION

This study demonstrated for the first time that patients with narcolepsy, like patients with RBD, present a higher percentage of REM sleep without atonia and an increased density of phasic chin EMG activity during REM sleep than normal controls. Two subgroups of narcoleptic patients with normal or abnormal percentages of REM sleep without atonia) were compared, but no between-group difference was found for any of the clinical or polygraphic variables listed in Tables 1 and 2.

Despite the fact that 50% of our narcoleptic patients showed an increase of tonic REM sleep motor activity, none of these patients reported a clinical history of behaviors associated with dream mentation or had obvious abnormal motor behaviors during REM sleep in the laboratory. In uncontrolled studies, the frequency of RBD in narcolepsy has been estimated to be between 7% and 36%.^3,4,23 The discrepancies between these results may be related to the small number of patients studied, differences in the diagnostic criteria of RBD (clinical vs clinical plus PSG), inclusion of patients treated with anticataplectic medications known to facilitate manifestations of RED,^3,23,24 and recruitment bias in favor of those presenting parasomnia. In the present study, RBD was defined by standard criteria based on patient report and results of PSG recording. All narcoleptic patients had cataplexy, and none was treated with anticataplectic drugs or with any other drugs known to influence sleep or motor activity. However, one limitation of the present study is that video recordings performed in the sleep laboratory are no longer available for further analysis and therefore, simple motor behavior seen in mild forms of RBD may have remained undetected. Overall this result suggests that RBD is rare in narcolepsy, or that only discrete behavioral manifestations may be present. This is congruent with our clinical experience; problematic RBD is rarely seen in untreated narcoleptic patients. On the other hand, half of the narcoleptic patients had REM sleep without atonia. The relation between RBD and REM sleep without atonia remains unclear. A similar dissociation between tonic EMG and behavioral manifestations in REM sleep has been reported in patients with Parkinson disease.²¹ Animal studies suggest that REM sleep without atonia and motor manifestations of RBD are 2 distinct phenomena with different anatomic substrates. Bilateral pontine tegmental lesions in cats release a state of REM sleep without atonia with a minimal increase of motor manifestations.^25,26 Additional rostro-ventral damage into the midbrain is necessary to release tone completely with elaboration of complex motor behaviors in REM sleep.^25,26

Considering phasic motor activity during REM sleep, we found an increase in chin EMG activity in both narcolepsy and RBD. Based on a cut-off of abnormal chin phasic EMG activity in REM sleep at 15%, 68.8% of narcoleptics and 81.3% of RBD patients had increased phasic motor activity in REM sleep. Results of the present study also showed that narcoleptic patients have higher PLMS and PLMW indices than controls; a similar trend was found for RBD patients compared to controls. In addition, the ratio of PLMS index in REM sleep over NREM sleep was higher in RBD patients compared to controls with a similar trend found for narcoleptic patients compared to controls. Previous studies showed increased PLMS activity in human narcoleptics, in the genetically natural canine model of narcolepsy, and in human RBD.^{5,8,9,27–29} These data raised the possibility of some basic mechanism being involved in both narcolepsy and RBD resulting in REM without atonia, increased phasic EMG activity in REM sleep and increase of PLMS in REM sleep. However, motor dyscontrol in narcolepsy is not restricted to REM sleep, but also involves wakefulness with an increase in PLMW index and the presence of cataplexy. We may hypothesize that inhibitory systems of motor regulation are globally damaged in narcolepsy.

There is some evidence (pharmacological, brain imaging, and neuroendocrine) that RBD, PLMS, and PLMW are related to impaired dopaminergic transmission in the CNS.^{11,28,30–36} Dopaminergic abnormalities are critical downstream mediators of the hypocretin deficiency, and dysfunctions in the hypocretin/dopaminergic system are likely to be the most important mechanism involved in the pathophysiology of narcolepsy.^1,37 As different brainstem regions and pathways are involved in REM sleep atonia (involving the subceruleus region) and REM EEG phenomena (involving the preceruleus region), selective brainstem damage may lead to an independent dysfunction with occurrence of dissociated REM state.¹³ The presence of REM sleep with muscle tone and/or RBD may be consequences of direct lesions in the subceruleus region (in typical RBD cases) and in the hypocretinergic system (in mild forms of RBD in narcolepsy). Finally, contrary to what is seen in RBD, narcoleptics have an increase in REM density that suggests that different REM sleep components of phasic motor activity are regulated via independent pathways. Indeed, eye movements of REM sleep are generated by the premotor neurons of the reticular formation whereas phasic EMG activity in REM sleep results from phasic depolarization of alpha motoneurons via both reticular and pyramidal neuron activation.³⁸ One may propose that a global disinhibition of both phasic activities in REM sleep in narcolepsy would be due to a common neurobiological defect, i.e., hypocretin deficiency. In contrast, a different REM sleep motor dyscontrol with dissociation between the levels of REM and chin phasic motor activity was seen in “idiopathic RBD,” which may be related to selective damage within brainstem that avoids the motor control structure for eye movements.

In summary, this study shows that patients with narcolepsy have a high frequency of REM sleep without atonia and of elevated phasic EMG density in REM sleep compared with controls. This study also shows several REM sleep motor dyscontrol similarities between narcolepsy and RBD, suggesting the possibility of a common neurobiological defect of motor inhibition during REM sleep. However, behavioral manifestations in REM sleep seem to be less frequent and severe in narcolepsy than in “idiopathic” RBD. These results also support the idea that a decreased hypocretinergic and/or dopaminergic abnormalities input to brainstem structures may contribute to dissociated sleep/wake states. In order to verify this hypothesis, future studies should measure and correlate CSF hypocretin level to REM sleep without atonia in both patients with RBD and narcoleptic patients.