Sensitivity and Specificity of the Modified Epworth Sleepiness Scale in Children With Craniopharyngioma

Abstract

Study Objectives:

Children with craniopharyngioma are at risk for excessive daytime sleepiness (EDS). Multiple Sleep Latency Testing (MSLT) is the gold standard for objective evaluation of EDS; however, it is time and resource intensive. We compared the reliability, sensitivity, and specificity of the modified Epworth Sleepiness Scale (M-ESS) and MSLT in monitoring EDS in children with craniopharyngioma.

Methods:

Seventy patients (ages 6 to 20 years) with craniopharyngioma completed the M-ESS and were evaluated by polysomnography and MSLT. Evaluations were made after surgery, if performed, and before proton therapy.

Results:

MSLT revealed that 66 participants (81.8%) had EDS, as defined by a mean sleep latency (MSL) < 10 minutes, with only 28.8% reporting EDS on the M-ESS by using a cutoff score of 10. The M-ESS demonstrated adequate internal consistency and specificity (91.7%) but poor sensitivity (33.3%) with the established cutoff score of 10. A cutoff score of 6 improved the sensitivity to 64.8% but decreased the specificity to 66.7%.

Conclusions:

Patients with craniopharyngioma are at high risk for EDS, as documented objectively on the MSLT, but they frequently do not recognize or accurately report their sleepiness. Future sleep studies should investigate whether specific items or alternative self- and parent-reported measures of sleepiness may have greater clinical utility in monitoring sleepiness in this population.

Citation:

Crabtree VM, Klages KL, Sykes A, Wise MS, Lu Z, Indelicato D, Merchant TE, Avent Y, Mandrell BN. Sensitivity and specificity of the modified Epworth Sleepiness Scale in children with craniopharyngioma. J Clin Sleep Med. 2019;15(10):1487–1493.

BRIEF SUMMARY

Current Knowledge/Study Rationale: Because youth with craniopharyngioma have a high prevalence of excessive daytime sleepiness, we validated the Modified Epworth Sleepiness Scale (M-ESS) in 70 pediatric patients with craniopharyngioma as a less resource-intensive tool to monitor sleepiness compared with Multiple Sleep Latency Testing, which is the gold standard but resource intensive. The M-ESS had poor sensitivity with the established cutoff score of 10, but at a cutoff score of 6 it had improved sensitivity but missed 35% of patients with significant objective sleepiness.

Study Impact: Our findings demonstrate poor sensitivity of the M-ESS in children with craniopharyngioma. This may have implications when monitoring sleepiness in children with brain tumors or other CNS lesions.

INTRODUCTION

Craniopharyngioma is a low-grade invasive intracranial tumor that constitutes 1.2% to 4.6% of all childhood central nervous system (CNS) tumors.^1–3 Although various combinations of surgery and radiation therapy (RT) have led to excellent survival rates,^4,5 these patients have significantly higher morbidity than those with other types of brain tumors.^6–9 Craniopharyngioma arises in the midline and is most often located in the suprasellar region of the brain, involving or displacing the optic chiasm, pituitary gland, and/or hypothalamus.¹⁰ Chronic headaches, nausea, vomiting, visual impairment, and hypothalamic–pituitary dysfunction are often present at diagnosis in 52% to 87% of patients.¹⁰ Hypothalamic–pituitary dysfunction is particularly important to consider, as it plays a key role in regulating the sleep–wake cycle.¹¹ Sleep fragmentation and excessive daytime sleepiness (EDS) are commonly reported and observed in youth with craniopharyngioma.^4,12–14

Sleep Disturbances in Youth With Craniopharyngioma

Limited surgical resection followed by RT has resulted in better outcomes than with aggressive surgical resection alone.^5–7 However, RT is associated with considerable short- and long-term effects on sleep, which can result in fatigue, EDS, and insomnia.¹⁵ Mandrell et al conducted a medical chart review on sleep disturbances in survivors of CNS tumors and found that youth with tumors in the suprasellar region of the brain (which includes the hypothalamus) who were also overweight or obese had increased EDS symptomatology, including excessive napping and fatigue.¹⁶ Approximately 65% to 80% of youth with craniopharyngioma experience substantial hypothalamic dysfunction posttreatment, including obesity, fatigue, and sleep disruption.^7,17,18 Sleep disruption may result in maladaptive sleep–wake patterns, with patients often waking early in the morning and napping in the afternoon.¹² Proton therapy is currently the preferred form of delivering RT for patients with craniopharyngioma, as it reduces the total volume of normal-tissue exposure, which can potentially improve the quality of life and functional outcomes of patients.^19,20 Because proton therapy is a relatively newer treatment modality, its effect on the long-term functional status of pediatric patients, including its effect on sleep and fatigue, remains to be determined.

Obesity in Youth With Craniopharyngioma

In addition to sleep disruption, hypothalamic dysfunction in survivors of craniopharyngioma is also a risk factor for the development of obesity.²¹ Obesity and rapid weight gain affect up to 52% of patients with craniopharyngioma, with almost half of these patients reporting an inability to diminish their desire to eat.¹⁴ Hypothalamic obesity, or intractable weight gain due to hypothalamic damage or dysfunction, occurs in 30% to 77% of patients with craniopharyngioma after completion of treatment.²² Unlike obesity, weight gain in response to hypothalamic damage or dysfunction occurs regardless of calorie restriction, pharmacologic treatment, or implementation of healthy lifestyle interventions.^22–25 Risk factors for development of hypothalamic obesity include tumor location (eg, tumors localized to the hypothalamus or thalamus region), tumor histologies prominent in the diencephalon (eg, craniopharyngioma), and direct RT at doses greater than 51 Gy to the hypothalamus.²²

Obesity in patients with craniopharyngioma is associated with higher EDS than that typically reported in otherwise healthy obese patients.^14,26 Müller et al. surveyed a large group of patients with childhood craniopharyngioma for EDS and found high severity of EDS in obese patients with a body mass index (BMI) z-score > 4.¹⁴ More recently, Brimeyer et al found a high correlation between obesity and parent-reported EDS in survivors of pediatric CNS tumors and concluded that obesity is a risk factor for EDS in survivors of CNS tumors.²⁷

Modified Epworth Sleepiness Scale

Because EDS is a frequent symptom in children with craniopharyngioma, it is imperative that the measurement of this construct be psychometrically sound. The Modified Epworth Sleepiness Scale (M-ESS) has been established as a quick and low-cost screening tool to identify EDS in youth by assessing the likelihood of children falling asleep in various everyday situations (from 0 = no chance to 3 = high chance of dozing).²⁸ The M-ESS has been adapted from the Epworth Sleepiness Scale (ESS) and measures the likelihood of falling asleep in eight routine adult situations.²⁸ The modified version uses slight variations of two situations in which EDS may occur that are more relevant to children. The original ESS included two questions evaluating the chance of sleepiness (1) after lunch without alcohol and (2) in the car, while stopped for a few minutes in traffic. These two items have been modified to be (1) after lunch and (2) while doing homework or taking a test, respectively.^29,30 The average score on the M-ESS in healthy children is 5.9 (standard deviation [SD] = 2.2). Previous studies suggest that M-ESS scores > 10 indicate EDS.^29,31 Internal consistency of the M-ESS among pediatric populations has been satisfactory and ranges between 0.70 and 0.75.^28–31 Children with CNS tumors have been found to score higher on the M-ESS than children with other malignancies.³²

Multiple Sleep Latency Testing

Multiple Sleep Latency Testing (MSLT) is the gold standard and most widely used objective characterization of EDS in children.^21,33,34 Unlike the M-ESS, the MSLT can objectively document sleepiness in patients independent of self- or parent-report.^35,36

Although the MSLT can identify an abnormal tendency for sleepiness in individuals, it is relatively time intensive and costly. Self-reported measures of sleepiness, such as the M-ESS, may be clinically useful because they can be completed quickly during a clinic visit, with no equipment, and at no cost. No prior studies have examined the sensitivity and specificity of the M-ESS in youth with craniopharyngioma before proton therapy. This time point to administer the M-ESS was chosen as youth were scheduled to undergo PSG and MSLT as part of the Phase II Trial of Limited Surgery and Proton Therapy for Craniopharyngioma and Observation for Craniopharyngioma after Radical Resection (RT2CR) protocol. This served as a baseline measurement of EDS before proton therapy, with a goal of monitoring EDS symptoms over time. To determine the validity of the M-ESS in monitoring EDS, we administered the measure at the same time point as participants underwent MSLT. We compared the sensitivity and specificity of the M-ESS with that of the gold standard MSLT in children with craniopharyngioma as well as tested the reliability, sensitivity, and specificity of the eight-item version of the M-ESS.²⁹

METHODS

Patients and Sleep Studies

As part of a larger institutional protocol RT2CR designed to treat and monitor patients with craniopharyngioma, 70 children and young adults (ages 6 to 20 years) were assessed by the M-ESS after surgical resection, if performed, and before proton therapy. Patients were excluded if they were previously treated with fractionated RT. The group included patients with a new diagnosis, those with recurrent tumors after prior radical surgery, patients treated with radical surgery alone, and patients who did not undergo surgery and received a diagnosis on the basis of imaging findings alone. Surgery included various combinations of transsphenoidal resection, open craniotomy, and closed stereotactic placement of catheter systems for cyst drainage; patients may have had more than one procedure. Youth were scheduled for overnight PSG, followed by MSLT the next day. Studies were conducted in an AASM-accredited sleep center and scheduled based on the patient’s typical sleep–wake schedule, as determined by a clinical history by the sleep specialist, scored by registered PSG technologists trained in scoring pediatric PSG and MSLT, and reviewed and interpreted by a board-certified sleep specialist with experience in pediatric sleep (MSW). The PSG montage included the electroencephalogram, six channels, including frontal, central, and occipital derivations linked with the contralateral ear; electro-oculogram, two channels; submental electromyogram, one channel; snore channel, one channel; respiratory effort at the chest and abdomen, two channels; air flow at the nose and mouth using nasal pressure transducers and oral thermistors, two channels; oximetry; lower-extremity limb movements, two channels; electrocardiogram, one channel; and simultaneously recorded video tape. Records were scored using sleep–wake scoring and respiratory scoring as recommended by The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications. Because young children have difficulty cooperating with the MSLT protocol,³⁷ patients younger than 6 years were excluded from the current study. If a patient had significantly poor sleep on the PSG, the MSLT was not conducted; if a patient had significantly delayed sleep onset, the MSLT was delayed in a similar fashion. This study was approved by the Institutional Review Board of St. Jude Children’s Research Hospital. Informed consent and assent were obtained for all protocol-based procedures.

Assessment and Measures

Clinical Variables

At baseline, demographic and clinical data were systematically collected from the medical records of children with craniopharyngioma. Table 1 gives demographic and clinical information for patients.

Table 1.

Demographic and clinical characteristics of patients.

Modified ESS

Participants completed the M-ESS (α = .74, current study) to measure daytime sleepiness.²⁹ Youth were asked to rate their propensity to fall asleep in various everyday situations (0 = no chance to 3 = high chance of dozing) for each of the eight items, with a maximum score of 24. Higher scores on the M-ESS indicate higher levels of EDS.²⁹

Multiple Sleep Latency Testing

MSLT is a standardized test to evaluate daytime sleepiness and measure sleep propensity in the absence of alerting factors. MSLTs were performed according to guidelines recommended by the AASM.³⁴ After overnight PSG, participants were evaluated with four or five nap opportunities at 2-hour intervals throughout the day. For each nap opportunity, patients were placed in bed and instructed to lie quietly with eyes closed and to try to fall asleep. As per the MSLT protocol, patients were studied in a quiet, darkened room with no television or other electronic devices on. The recording montage used was consistent with that recommended by the AASM, including central and occipital electroencephalogram derivations linked to the contralateral ear or mastoid region, left and right electro-oculograms, mental/submental electromyogram, and electrocardiogram. A registered PSG technologist observed patients continuously between naps and encouraged them to not fall asleep. If sleep onset occurred during a nap opportunity, patients were allowed to sleep for 15 minutes before being awakened. If no sleep occurred after 20 minutes, the nap opportunity ended. The mean sleep latency (MSL) in minutes was calculated as the arithmetic mean of all nap opportunities; the number of naps and the number of sleep-onset rapid eye movement periods were also recorded. By using EDS cutoff scores for prepubescent versus pubescent children first proposed by Gozal et al, EDS was defined as an MSL of ≤ 15 minutes in prepubescent children with Tanner stage 1 and of ≤ 10 minutes in pubescent youth with Tanner stage 2 or greater.³⁸ Tanner staging was ascertained by reviewing medical records from the institutional database.

Statistical Analyses

Cronbach alpha was calculated for the eight items of M-ESS. Each item was then removed in turn, and Cronbach alpha was calculated for the remaining seven items to check whether any item substantially affected the internal consistency of the M-ESS. The sensitivity and specificity under the conventional cutoff for M-ESS (score > 10) and the area under the receiver operating characteristic (ROC) curve were calculated. Sensitivities and specificities were compared by using three criteria: (1) minimum absolute difference between sensitivity and specificity; (2) distance to perfect classification; and (3) Youden index. Fisher exact test was used to determine the association between sleep disturbance (yes/no) and categories of BMI as a measure of obesity (underweight/normal, overweight, obese). BMI z-score was classified by World Health organization guidelines for participants aged 5 to 19 years, and BMI was classified by Centers for Disease Control and Prevention guidelines for participants older than 19 years.^39,40 Statistical analyses were performed using SAS 9.4 (SAS Institute, Cary, North Carolina, USA).⁴¹

RESULTS

Demographic and Clinical Characteristics

Table 1 gives demographic and clinical features of patients. The average age of participants was 11.6 years (standard deviation [SD] = 3.7 years), 53% were male, and 61% were Tanner stage 1. Of the participants, 63% were white and 21% were black. The BMI z-score of the sample was 1.2 (mean BMI = 23.5; SD = 5.95), with 21.4% in the obese range.

PSG and MSLT Findings

PSG studies showed that 5.7% of participants met the criteria for sleep-disordered breathing (SDB), with a mean apnea-hypopnea index of 1.5 (SD = 2.2). When using previously established cutoff scores,³⁷ 41.4% of youth met criteria for hypersomnia secondary to a medical condition (defined as an MSL of ≤ 15 minutes in prepubescent children and ≤ 10 minutes in pubescent children) and 37.1% met criteria for narcolepsy (defined as an MSL of ≤ 15 minutes and ≥ 2 periods of sleep-onset rapid eye movement in prepubescent children and an MSL of ≤ 10 min and ≥ 2 periods of sleep-onset rapid eye movement in pubescent children; Table 1). Approximately 80.8% of youth defined as overweight and 100% of youth defined as obese experienced sleepiness on MSLT. Furthermore, the proportion of patients in whom sleep disturbances were diagnosed increased with BMI-based weight status; however, this trend was not statistically significant (P = .10). According to MSLT, youth with craniopharyngioma had a multiple sleep latency of 0.4 to 20 minutes (mean ± SD = 7.8 ± 5.6 minutes) with 0 to 4 sleep-onset rapid eye movement periods. Also, 39% of patients had two or more sleep-onset rapid eye movement periods across nap opportunities (Table 1).

Reliability of the M-ESS

The M-ESS had acceptable internal consistency, with a Cronbach alpha of 0.74. Items 1, 4, 5, and 8 had the lowest correlation with the total score (Table 2). With the deletion of each item, the reliability in measuring the underlying construct was similar, with marginal acceptable levels ranging from 0.68 to 0.73, which indicates that no item substantially affected the internal consistency.

Table 2.

Summary of Cronbach alpha with item-wise deletion of variables for the modified Epworth Sleepiness Scale.

Accuracy of the M-ESS

Of 70 patients enrolled, data were available for 66 to assess the accuracy of the M-ESS in identifying patients with EDS. Three patients did not complete the MSLT, and one patient did not answer all questions in the M-ESS. By using the previously established cutoff score of > 10, the sensitivity of the M-ESS was low (18 of 54 [33%]) participants with EDS documented on MSLT reported significant EDS, with a high specificity (11 of 12 [91.7%]) without EDS documented on MSLT reported no significant EDS) in detecting children with EDS (Table 3 and Table 4). Given the poor sensitivity of the modified ESS, ROC analysis was performed to identify an optimal M-ESS cutoff score to identify patients with EDS. Area under the ROC curve was 0.684 with a 95% confidence interval of 0.543–0.825 (Figure 1). Optimal threshold was selected by using the criteria of minimum absolute difference between sensitivity and specificity. Cutoffs selected by the other two criteria were similar but associated with lower sensitivity. ROC analysis revealed the optimum cutoff as a M-ESS score > 6. Using this optimal cutoff score, sensitivity increased to 64.8% (35 of 54) and specificity decreased to 66.7% (8 of 12; Table 3 and Table 4).

Table 3.

Frequency of M-ESS scores in craniopharyngioma patients with or without excessive daytime sleepiness.

Table 4.

Sensitivity, specificity, and positive and negative predictive value of different cutoff scores of the M-ESS in detecting craniopharyngioma patients with excessive daytime sleepiness.

Figure 1. Receiver operating characteristic curve for the modified Epworth Sleepiness Scale (M-ESS).

Points are labeled by cutoff of the M-ESS score.

DISCUSSION

To date, this is the largest cohort of longitudinally evaluated pediatric patients with craniopharyngioma who at presentation had significant EDS documented using the MSLT. Most patients had clinically significant daytime sleepiness, with an average MSL value on MSLT of 7.8 minutes for all patients. Considering the significant morbidity associated with chronic EDS and the time- and resource-intensive evaluations of overnight PSG and MSLT, we determined the reliability, sensitivity, and specificity of the M-ESS in evaluating EDS. Given that children with craniopharyngioma are at significant risk of EDS, the gold-standard MSLT should be used as a diagnostic test. However, a low-cost, brief assessment of sleepiness would be clinically useful, particularly for evaluating response to treatment of EDS. Unfortunately, the M-ESS did not accurately identify EDS in our pediatric patients with craniopharyngioma.

Although literature supports that the M-ESS has adequate internal consistency and excellent specificity for EDS among patients with SDB, we found poor sensitivity for EDS among patients with craniopharyngioma at high risk for hypersomnia or narcolepsy. In a clinical assessment of EDS, it is imperative that instrument psychometrics include high sensitivity to ensure that EDS symptoms are not underestimated.⁴² In our study, 67% of patients assessed by MSLT and in whom EDS was diagnosed would not have met the criteria for EDS when using the M-ESS cut- off score of 10, which indicates its limited utility as a screening or treatment monitoring measure in these high-risk patients.

To improve the sensitivity of M-ESS, we used the recommended cut score from Chan et al,³⁰ who differentiated mild from severe SDB. Sensitivity was improved by using a cutoff score of 6, but 35% of patients assessed by the MSLT and in whom EDS was diagnosed would not have met the criteria for EDS if a lower cutoff score was used. Thus, we cannot recommend a lower cutoff score on the M-ESS that can be used to identify patients with craniopharyngioma. Despite these findings, it is important to highlight that the M-ESS had high specificity and a positive predictive value for EDS; however, this provides limited clinical utility in a sample with such high rates of EDS. A possible explanation for poor sensitivity of the M-ESS in our patients is that children and adolescents with hypersomnia or narcolepsy may have habituated to the sense of sleepiness, typically resulting in self-reported sleepiness being underreported. Healthy adults can quickly habituate to restricted time in bed. A study assessing adults over 14 nights of restricted time in bed demonstrated a decline in vigilance, with no increase in self-reported sleepiness after the first night.⁴³ Youth with craniopharyngioma typically present for treatment after an extended time of experiencing numerous symptoms secondary to hypothalamic dysfunction. It is likely that they habituate to their experience of sleepiness and no longer recognize the effect of EDS on their daytime functioning. Another possible explanation is that youth may have been chastised or disciplined for falling asleep in school, leading them to deny sleepiness when asked. Because of poor recognition of sleepiness, patients’ self-reports might not adequately reflect their objective sleepiness.

Although this study has several strengths, including a relatively large sample of patients with craniopharyngioma, adequate power to detect performance of the M-ESS, and both objective and self-reported evaluation of daytime sleepiness, it has some limitations. First, 80% of our patients had undergone a total or subtotal resection before being evaluated by PSG, MSLT, and M-ESS. The extent to which EDS was present preresection is not known, and we did not evaluate the extent to which EDS may improve over time in the postoperative recovery process. Further, the lack of information about EDS prior to resection also presents a problem in identifying the effect of the tumor or surgery directly on sleepiness. As these youth have not been evaluated for sleepiness prior to the identification of the tumor, it is unclear what the duration of sleepiness is and if the tumor and its resection directly affects EDS. In addition, nearly two-thirds of the sample was prepubertal, which limits the generalizability to some degree for our pubertal patients. However, MSL values across both younger and older patients were quite low, reflecting severe sleepiness across the sample. Furthermore, we used the pediatric version of the scale for all participants to have consistent responses across the cohort. For the few patients aged 18 to 20 years, the adult version of the ESS might have been more sensitive to their daytime sleepiness. Of note, however, we chose to eliminate an item related to use of alcohol in patients younger than 21 years. We also did not evaluate other self-reported measures of EDS, and other EDS screening instruments may have greater clinical utility in our patients. Finally, because response to intervention for EDS was not evaluated, we cannot ascertain the sensitivity of the M-ESS in monitoring treatment response with potentially improved daytime alertness after intervention.

Although the M-ESS does not appear to have adequate sensitivity in pediatric patients with craniopharyngioma in our study, future studies should continue to develop and test assessments by which sleepiness can be evaluated, monitored, and managed in these patients who have significant morbidity. Currently, MSLT continues to be the recommended method for objective evaluation of sleepiness in patients with craniopharyngioma. In the future, specific items from the M-ESS may be identified as being sensitive to EDS in this patient population, and other instruments can be evaluated to monitor sleepiness. As we continue to longitudinally evaluate sleepiness in patients with craniopharyngioma, we hope to establish a better understanding of the sleepiness trajectory and how it intersects with currently available interventions as well as those developed in the future for this high-risk population.

DISCLOSURE STATEMENT

All authors have contributed to the preparation of this manuscript and have approved its final version. The study was supported by Cancer Center Support Grant (CA21765) from the National Cancer Institute and by ALSAC. The authors report no conflicts of interest.

ABBREVIATIONS

AASM

American Academy of Sleep Medicine

BMI

body mass index

CNS

central nervous system

EDS

excessive daytime sleepiness

M-ESS

Modified Epworth Sleepiness Scale

MLST

Multiple Sleep Latency Testing

MSL

mean sleep latency

PSG

polysomnography

SD

standard deviation

SDB

sleep-disordered breathing

ROC

receiver operating characteristic

RT

radiation therapy

REM

rapid eye movement