Is the Epworth Sleepiness Scale Sufficient to Identify the Excessively Sleepy Subtype of OSA?

To the Editor:

Excessive daytime sleepiness (EDS), a prevalent clinical feature in OSA, is associated with adverse consequences.¹ The Epworth Sleepiness Scale (ESS) is the most widely used instrument to characterize subjective EDS in OSA.² Patients with an ESS score of >10 are typically categorized as having EDS.² However, clustering analyses of a broader range of clinical symptoms have identified unique and reproducible symptom-based subtypes of moderate-to-severe OSA that is characterized by disturbed sleep, different degrees of excessive sleepiness, or minimal symptoms.^{3, 4, 5, 6, 7} In the community-based Sleep Heart Health Study, we showed differential risk for cardiovascular disease based on OSA symptom subtype.³ The “excessively sleepy” subtype had higher prevalence of baseline cardiovascular disease and higher risk of incident cardiovascular events compared with other subtypes and individuals without OSA.³ Notably, the increased risk associated with this subtype was not explained simply by the ESS.³ Labarca et al⁷ recently have validated these findings. The authors validated symptom subtypes in patients with moderate-to-severe clinical OSA from Santiago, Chile, and found that the “excessively sleepy” subtype is at a higher risk for cardiovascular death.

These recent findings not only provide important insights into the clinical relevance of the identification of patients with the “excessively sleepy” subtype but also indicate that a comprehensive characterization of a patient’s sleepiness profile beyond the ESS is necessary. Supporting different dimensions of EDS, a recent study found limited associations between ESS and other measures of daytime sleepiness, such as self-reported frequency of not getting enough sleep, feeling unrested, and napping.⁸ The importance of multiple dimensions of sleepiness has also been shown in the general population. Individuals with OSA who report feeling sleepy during the day, with or without ESS >10, had higher prevalence of other disturbed sleep symptoms compared with non-sleepy patients (ESS ≤10 and not feeling sleepy), although those individuals with only ESS >10 did not.⁹ To emphasize these considerations in the context of symptom subtypes of OSA,^{3, 4, 5, 6, 7} we investigated whether the ESS is sufficient to accurately identify the “excessively sleepy” subtype.

Methods

Using data from participants diagnosed with moderate-to-severe OSA (apnea-hypopnea index, ≥15) in two clinical-based cohorts (Icelandic Sleep Apnea Cohort, N = 776; Sleep Apnea Global Interdisciplinary Consortium, N = 1,563), and a community-based cohort (Sleep Heart Health Study, N = 1,177), we applied latent class analyses to 14 common symptoms (13 symptom questions and a categorized version of ESS scores), as described in our previous work.^{3, 4, 5} Institutional review boards in each institution approved the study, and participants signed informed consent. For each cohort, we identified four symptom subtypes previously associated with cardiovascular risk.^3,7 To understand the distribution across subtypes, we compared ESS scores among symptom subtypes using a linear model adjusted for age, sex, BMI, and the apnea-hypopnea index. Next, we assessed the performance of a clinical definition of EDS (ESS >10) and a recent exclusion criteria for a large randomized trial on the effect of CPAP on cardiovascular risk¹⁰ (ESS >15) to predict the “excessively sleepy” subtype. Finally, to answer our primary question of whether multiple domains of sleepiness were needed to distinguish the “excessively sleepy” subtype accurately, we assessed the relative predictive ability of continuous ESS and other individual sleepiness-related symptoms to predict the “excessively sleepy” subtype (vs all others) using a random forest prediction algorithm.

Results

As expected, we observed significantly higher ESS scores in “excessively sleepy” participants in all study cohorts, independent of clinical covariates (Fig 1). Among all participants (N = 3,516), an ESS >10 (Fig 1) had 76.9% balanced accuracy to predict the “excessively sleepy” subtype, with a sensitivity of 96.6%, specificity of only 57.2%, positive predictive value (PPV) of 73.3%, and negative predictive value (NPV) of 93.3%. An ESS >15 (Fig 1) had higher balanced accuracy (84.3%), specificity (84.5%), and PPV (96.3%) but reduced sensitivity (84.1%) and lower NPV (52.2%), when compared with ESS >10. Thus, there is predictive value in the use of ESS thresholds to predict the “excessively sleepy” subtype, but some limitations exist with respect to specificity and NPV.

Figure 1.

Distribution of Epworth Sleepiness Score among OSA symptom subtypes in the Icelandic Sleep Apnea Cohort, the Sleep Apnea Global Interdisciplinary Consortium, and the Sleep Heart Health Study. The numbers in black boxes represent the mean Epworth Sleepiness Score for each subtype. In analyses that were adjusted for age, sex, BMI, and the apnea-hypopnea index, all pairwise comparisons were significant at P < .001, except for the comparison between “disturbed sleep and minimally symptomatic” in the Sleep Apnea Global Interdisciplinary Consortium, where P = .045. The yellow (Epworth Sleepiness Score = 10) and red (Epworth Sleepiness Score = 15) dashed lines represent common thresholds for denoting excessive and severely excessive daytime sleepiness, respectively.

Considering the multidimensional nature of EDS, we next investigated whether predictive performance could be improved with additional sleepiness-related symptoms. A random forest prediction algorithm was applied to understand the relative ability of eight sleepiness-related symptoms to distinguish “excessively sleepy” from all other subtypes (Fig 2). Using five-fold cross-validation, we calculated variable importance scores for each item and then fit models to predict the “excessively sleepy” subtype using an increasing number of symptoms based on order of importance. In the testing sample, the ESS alone showed average balanced accuracy (95% CI over five-fold cross-validation) of 79.5% (range, 77.3% to 81.7%), sensitivity of 90.2% (range, 87.8% to 92.7%), specificity of 68.8% (range, 62.3% to 74.5%), PPV of 88.3% (range, 86.6% to 90.1%), and NPV of 73.2% (range, 68.8% to 77.5%). Although relatively accurate, inclusion of the next most important sleepiness-related symptom (“How often do you feel sleepy during the day?”) significantly improved balanced accuracy (86.2%; range, 83.8% to 88.6%) and increased sensitivity (91.1%; range, 89.0% to 93.2%), specificity (81.3%; range, 76.4% to 86.2%), PPV (92.7%; range, 91.0% to 94.5%) and NPV (77.9%; range, 74.1%-81.7%). The random forest balanced accuracy plateaus after including the top 5 sleepiness-related symptoms (88.2%; range, 86.5% to 89.9%) (Fig 2), with sensitivity (93.0%; range, 90.0% to 95.9%), specificity (83.4%; range, 77.3% to 89.4%), PPV (93.6%; range, 91.7% to 95.6%), and NPV (82.3%; range, 77.0% to 87.5%) all above 80%. All performance metrics, except for NPV, are significantly higher than those with ESS alone (P < .05 across independent cross-validations in testing data). To illustrate the clinical applicability and interpretability of our findings, we report the results of a representative decision tree with the use of the top five sleepiness-related symptoms as predictors in the whole sample. Although the performance of this single decision tree is lower than the overall random forest (eg, 86.0% balanced accuracy; 93.1% sensitivity; 78.8% specificity; 92.0% PPV; and 81.3% NPV), following any branch on the basis of a patient’s reported symptoms provides a diagnostic pathway for assigning them to the “excessively sleepy” subtype (or not).

Figure 2.

A-C, Random forest and decision tree assesses the prediction of the “excessively sleepy” subtype vs all others with the use of sleepiness-related symptom items, in all cohorts. A, Variable importance scores of the full model with the use of all eight sleepiness-related symptoms items. B, Balanced accuracy of sequential models that include the most important to the least important symptom item in order of importance, in both training and testing samples, using a 5-fold cross-validation design. C, A decision tree fit with the top five sleepiness-related questions to illustrate the clinical applicability of the identification of the “excessively sleepy” subtype with the use of these questions. CV = cross-validation; ESS = Epworth Sleepiness Scale; VIMP = random forest variable importance

Discussion

Overall, these data support the idea that we could improve on the use of the ESS alone to define the well-characterized^{4, 5, 6} and clinically relevant^3,7 “excessively sleepy” subtype in individuals with moderate-to-severe OSA. A few additional sleepiness symptom items enhanced predictive performance. Practically, simply asking patients about the frequency of daytime sleepiness provides important additional information beyond the ESS. Predictive performance was optimized when five sleepiness-related symptoms are considered for the identification of “excessively sleepy” patients, as illustrated through an exemplary decision tree (Fig 2). These findings might guide the design of future clinical trials related to EDS and cardiovascular risk or other incident outcomes, moving beyond the ESS alone as a study inclusion criterion. To facilitate these applications, developing efficient and accurate prediction algorithms for the identification of all OSA symptom subtypes with the use of symptom items beyond the ESS and clinical factors (such as age, sex, and BMI) are needed. Future studies that will show the importance of these subtypes for determining outcomes, as has been done for cardiovascular events,^3,7 would further facilitate clinical translation. Ultimately, incorporating the concept of OSA symptom subtypes into clinical practice, with a focus on improved characterization of EDS and identification of patients at greatest cardiovascular risk, will facilitate the application of personalized sleep medicine approaches.