Sleep and Big Data: harnessing data, technology, and analytics for monitoring sleep and improving diagnostics, prediction, and interventions—an era for Sleep-Omics?

Over the last 30 years, there have been impressive advances in characterizing the physiological bases of sleep and circadian rhythms and their variation across populations and health outcomes. The remarkable maturation of research in sleep and circadian biology is exemplified by an exponential increase in sleep-focused peer-reviewed publications, and that a recent Nobel prize was awarded for the elucidation of circadian machinery underscores the importance of sleep and circadian science.

In clinical sleep medicine, there have been improvements in defining sleep disorders—with evidence-based guidelines for screening, treating and managing sleep apnea, insomnia, restless legs syndrome, narcolepsy, and circadian rhythm and other sleep disorders. Accredited sleep centers—delivering guideline-based sleep care—have steadily grown, closing the gap between the number of people suffering from a sleep disorder versus those appropriately treated. The recognition of sleep health as a public health priority is reflected by the inclusion of sleep health goals in the Center for Disease Control’s Healthy People 2020 report.

Nonetheless, there remain fundamental knowledge gaps. Notably, there is a need to better understand the basic functions of sleep, as well as how to modulate sleep as a novel clinical intervention for improving cognition, mood, and longevity. Critical challenges in optimizing treatment of sleep disorders include the substantive heterogeneity of physiological endotypes, symptoms, comorbidities, treatment responses, prognoses, and genetic risk factors among individuals labeled with the same diagnostic code. As clinicians, we sometimes fail to find treatments that adequately resolve the sleepiness experienced by many patients with sleep apnea, or that improve both the nighttime and daytime symptoms that patients with insomnia experience, and may struggle to identify the causes and effective treatment strategies for patients labeled with “idiopathic hypersomnia.” Our diagnostic tools suffer from an over-reliance on manual and subjectively ascertained quantification of individual “events” during sleep records (apneas and arousals) and overly simplistic approaches to summarizing continuous physiological phenomena into discrete categories (e.g. sleep epochs). Traditional diagnostic approaches—developed using a knowledge base over 50 years old—not only limit the richness of sleep data routinely collected over hours if not days, but also contribute to costs and inefficiencies that could be improved with smarter utilization of more automated and sensitive procedures.

Together, these gaps point to a critical need to leverage larger sets of multilevel data more comprehensively and rigorously to better define the complex and dynamic processes underlying sleep and wake states, the dysregulation of which results in sleep-related disorders. A further need is to better define disease subgroups, which ultimately may lead to tailored interventions that meet the goals of precision medicine. Advanced computational and analytical tools open the possibility to interrogate complex physiological and clinical problems at a scale and with precision and power not previously possible, and these tools are especially relevant for addressing the needs of sleep medicine.

“Big data” are now routinely leveraged for far-ranging purposes from consumer marketing and fraud detection, to molecular biology, medical imaging, and cancer diagnostics. These approaches have the potential not just to automate existing manual procedures, but—perhaps more fundamentally—to also identify novel patterns, longitudinal trends, interactions, diagnostic signatures, subgroups, and other facets of sleep health that are presently hidden. To this end, the American Academy of Sleep Medicine recently charged an Artificial intelligence (AI) task force to address the role of AI in sleep medicine. Its two initial papers summarized the opportunities for sleep medicine to benefit from the application of AI—specifically machine learning (ML) tools—which require data sets typically with 1,000s, if not 10,000s, of observations [1, 2]. The task force highlighted the rich but under-utilized information contained within oscillations of sleep, cardiac, and respiratory signals: especially when coupled with demographic, behavioral, genetic and other biological, psychosocial, and lifestyle data, these methods can provide new approaches for diagnosis and patient management, and impact population health. The AASM reports highlighted five areas: (1) improving classification and diagnosis of sleep disorders; (2) predicting disease and treatment prognosis; (3) characterizing disease subtypes; (4) improving sleep scoring via automation; and (5) leveraging nightly PAP downloads for improved adherence support and intervention.

Over the last year, SLEEP aimed to further catalyze interest in the role of novel ML tools and advanced signal processing of large data sets to “inform and transform translational and clinical sleep/circadian research and clinical practice.” Authors were invited to submit rigorous data-driven manuscripts using large data sets that address several topics. In total, 18 papers were published in volumes 43 and 44 of SLEEP. In the current SLEEP Virtual Issue https://academic.oup.com/sleep/pages/big-data-vi, we highlight 10 of those papers, selecting those that provide examples of responses to the following topics:

• New insights derived from big sets of objective sleep data (i.e. PSG, actigraphy) to understand the physiology of healthy sleep and pathophysiology of sleep disorders.

Two papers are highlighted, both of which describe changes in sleep related to age and gender. In Hartmann et al. [3] describe an automated detection algorithm for cyclic alternating pattern (CAP), applied to evaluate sleep micro-architecture in nearly 3,500 older men women from two community cohorts. In addition to demonstrating the feasibility of the signal processing algorithm, the study showed that CAP rates and markers of EEG synchrony varied with age, gender, and sleep quality. Notably, while women had more slow wave sleep than men, they had fewer CAP A1 cycles, suggesting mechanisms for the higher frequency of subjective sleep concerns in women compared to men.

Jonasdottir et al. [4] presented data on over 11 million nights of wearable activity monitoring data (as well as contextual information from a cell phone app) from nearly 70,000 adults from 47 countries. Using a set of highly effective graphs summarizing data for sleep duration and timing (including average and weekday/weekend variability) and wake after sleep onset (WASO) by age and gender, the authors demonstrate characteristic changes in multiple facets of sleep by both age and gender, and notable age and gender interactions. For example, at younger ages (< 44 years), women have more WASO but longer sleep duration than men, with differences narrowing at older ages (> 55 years). Other changes in sleep patterns over the life course were observed, such as more variability in sleep timing and duration and social jetlag among younger adults, and a higher frequency of shorter sleep duration and nighttime awakenings in older adults. The app-related information suggested a role for caregiving in the disrupted sleep of women. While some misclassification of sleep with use of a wearable device is likely, the extremely large dataset, rigorous analytical approach, and comparability of results to estimates from external datasets that used more traditional measurements supports the value of this large-scale approach.

• Improving the accuracy of diagnosis of sleep disorders.

Five articles are highlighted: two report the results of data-driven sleep stagers—one trained on traditional EEG sensors and the other that used ECG and respiratory sensors [5, 6]; one study reported the results of a sleep staging model developed specifically for patients with sleep apnea [7]; one report was of a ML approach for detecting obstructive sleep apnea [8]; and one report provided an example of an automatic feature detector (for K complexes) [9]. In Olesen et al., a deep neural network model was developed using 15,684 polysomnography studies from five cohorts that was evaluated across a range of scenarios [6]. The authors found that the highest levels of accuracy were for models that used data from all cohorts (e.g. accuracy 0.869 ± 0.064; Cohen’s kappa of 0.799 ± 0.098), with poorer performance when training the algorithm on only a single cohort. These results highlighted the need for access to large and diverse data sets for training and testing automated sleep stagers. In the study by Sun et al., a deep neural network trained on only ECG and respiratory effort channels showed good agreement for classifying sleep stage (Cohen’s kappa 0.585), although accuracy was lower when compared to models trained on EEG signals [5]. Notably, no loss of accuracy was observed according to age or presence of sleep apnea, suggesting utility for sleep stage assessments in clinical settings where EEG is not easily collected. The challenge of sleep staging where primary interest is in evaluating sleep disordered breathing was addressed by Korkalainen and colleagues [7]. Using only the photoplethysmogram (PPG) captured from a finger pulse oximeter, the authors showed the ability of a deep learning model to differentiate sleep stages (epoch-by-epoch accuracy of 80.1%). Their paper highlighted the untapped potential of signals routinely collected but not comprehensively mined for improving sleep apnea diagnostic information available with limited channel sleep apnea testing devices. Specifically, oximetry with PPG is widely used in home-based sleep apnea studies where lack of sleep duration data can result in underestimation of the Respiratory Event Index (REI). Including PPG data could improve REI estimation while also providing an estimate of apnea severity in REM sleep—a sleep apnea subphenotype particularly common in women [10] as well as one associated with hypertension risk [11]. Huang further addressed whether sleep apnea could be predicted using a combination of commonly collected demographics, sleep symptoms, anthropometric, and clinical data. Using a support vector machine-based prediction model and fivefold cross-validation in a sleep clinic sample from China, the authors reported that a model built with two (age, waist circumference) to six features predicted an AHI ≥5/h, ≥15/h, and ≥30/h with a sensitivity of 74%, 75%, and 70% and specificity 75%, 69%, and 70%, respectively [8]. Their data-driven approach outperformed other published questionnaire-based screening tools in their sample; however, this approach needs further external validation. Finally, Lechat and colleagues applied a deep neural network and a Gaussian process to identify K complexes, showing that an automated procedure had excellent discrimination for detecting this feature of N2 sleep and arousal, and was able to characterize K complex morphology and density [9]. The authors reported characteristic changes in K complex density and morphology by sleep cycle and age, and suggested that automated detection of EEG features may both provide insight into sleep mechanisms as well as tools for use in sleep scoring quality control.

• Using big data to better understand and predict uptake, response, and adherence to treatments.

Wickwire and colleagues [12] analyzed data from 29,072 older adult Medicare beneficiaries diagnosed with obstructive sleep apnea, identifying 7,111 (24.5%) who initiated CPAP, 3,229 of whom were aged 65 and older. Only 44% of the latter sample maintained possession of CPAP machines by 13 months, with Medicaid eligibility (a proxy for lower SES) the strongest predictor of low adherence. The persistent challenges in optimizing CPAP use among patients with sleep apnea underscores the need for further research to identify levers for improving treatment adherence. There is a critical need for the huge amounts of adherence data that are routinely collected to be linked to other key individual-level data and made more readily available for research both in real time (to promote early intervention) as well as via static downloads.

• Using big data to identify distinct subgroups of patients or trait clusters.

Subphenotyping heterogenous groups of patients according to underlying physiological endotype characteristics has been limited by the technical requirements posed by collecting in-depth respiratory and arousal data which then are analyzed using complex signal processing algorithms. Finnsson and colleague present and validate a method for computing key physiological endotypes—loop gain, arousal threshold, and several other ventilatory parameters, showing that the endo-Phenotyping Using Polysomnography method of Dr Scott Sands et al. can be implemented in a cloud-based environment [13].

• Use of sleep/circadian parameters as predictors of future physical and mental health.

In a longitudinal epidemiological study, Williamson and colleagues applied structural equation modeling to almost 5,000 children ages 5–13 years, measured across multiple waves of data collection [14]. Repeated data on behavioral child sleep problems, internalizing and externalizing symptoms, and health-related quality of life (HRQoL) were collected. Using this rich design, the authors tested for potential bi-directional associations, finding that their data were most consistent with behavioral sleep problems forecasting worse subsequent psychosocial and physical HRQoL. While longitudinal analyses are critical for understanding the natural history of sleep disorders and dissecting causal associations, there is an unfortunate lack of data sets with repeated sleep measurements. Efforts at prospective studies with comprehensive, repeated measurements are needed to realize sleep/circadian parameters as predictors of future outcomes.

Overall, from these papers, we see the promise of ML models for predicting sleep stages, sleep microarchitecture, individual features of the EEG, and sleep apnea, often successful with analysis of only a reduced set of signals or data, enhancing feasibility for larger scale data collection. We also see potential for further mining of administrative, clinical, and epidemiological data sets and use of widely available wearable sensors for sleep monitoring, with opportunities to increase knowledge of the factors contributing to health disparities, to better characterize sleep disorder heterogeneity, to identify the effectiveness of alternative sleep interventions, to discover molecular mechanisms underlying sleep traits, and to predict sleep-related outcomes. In this call for papers, we received few or no responses to several of these topics, however. Specifically, the following areas were under-represented: (1) Using big data to inform and improve treatment and management of sleep and circadian disorders; (2) The use of big data and “omic” technologies (e.g. epigenomics, genomics, metabolomics, microbiomics, transcriptomics, and proteomics) in sleep and circadian sciences; and (3) Implementation of sleep therapies in the real world, including factors related to patients, providers, and the health care system that impact the use of effective sleep therapies; health care utilization; and cost of treated versus untreated sleep disorders. The lack of response may be due to the current challenges in fully integrating sleep data with real world clinical data and genomics. We did see examples of the power for deep learning models to produce accurate predictions, as well as the limitations of the “black box” outputs from these models, which often do not reveal the features driving prediction, limiting interpretability, and precluding identification of biological mechanisms. The deep learning models reported were generally comprised of less than 10,000 observations, while the most robust neural networks usually require massive datasets not available for most sleep data.

This virtual issue of SLEEP also highlights the potential for Sleep-Omics. While the suffix omics was originally adopted to studies of molecular data that define complex structural and functional biological interrelationships, other omics have emerged, such as “spiro-omics” (lung biology) and “food-omics” (food and nutrition). More specifically, omics aims to achieve an improved understanding of a given phenomenon by applying multiple levels of analyses to numerous streams of data, each often of different scale and content. One may argue the polysomnography represents an early example of omics by integrating multiple physiological signals collected over hours and across different sleep stages. An improved understanding of the complex nature and impact of sleep and sleep disorders may be further achieved by a “Sleep-Omics” approach that more comprehensively integrates information on multiple relevant upstream genetic and environmental exposures, behaviors, and physiological inputs (e.g. overnight blood pressure), as well as data on downstream effects of sleep on gene expression, methylation, behaviors, and clinical and physiological outcomes. In addition to polysomnography, data sources for these multiple-level analyses may include data from wearables, medical devices used for both sleep treatments (PAP) as well as for other purposes that incidentally collect relevant data (e.g. implantable cardiac defibrillators), geographic-area data, medical records, patient-reported outcome survey data, imaging markers of sleep/brain function and biomarker and multi-genomic markers, and others.

While the articles in the virtual issue show several promising applications of big data, especially in relationship to automated sleep scoring, no study addressed the full range of potential Sleep-Omics applications (e.g. with genomic/imaging integration) and there were only limited examples of data derived from clinical sources or applied to population health. Often analyzed samples were a magnitude smaller than optimal for ML algorithms (raising concerns regarding over-fitting). Some studies did not report cross-validation in independent samples, as needed to assess generalizability and transportability. A further critical issue is that “big data” in sleep medicine not only requires access to large data sets, but also robust and reproducible analytical tools that can be implemented at scale, and are open, transparent and accessible to the community. In the future, publications on newly proposed methods would ideally often be paired with usable software implementations, which will also have the virtuous side-effect of improving the rigor and reproducibility of the methodological work.

How do we move forward to generate the data repositories needed for a Sleep-Omics framework? Our professional societies should call for the routine collection of standardized key sleep data in clinical settings (data will vary according to primary care vs. sleep clinic/lab but may include brief sleep data intake forms and standardized scales as well as the actual sleep studies) and demand that those data be integrated into the electronic health record in machine readable formats. It is unacceptable that many health care centers routinely bury collected sleep studies within clinical records using formats that are difficult to find and search. Similarly, there is a need for PAP adherence data and home sleep tests that are managed by industry-owned cloud-platforms be made more generally available as research data, finding solutions for logistical and privacy concerns. As patients are increasingly using wearables, those data also should be more readily integrated and accessible in the clinical records, taking advantage of recent tools and approaches developed by the All of US and other programs.

The sleep community also needs to more collaboratively and aggressively support strategic implementation of international research sleep data sharing initiatives to create well-labeled and documented large and diverse sleep data sets accessible to the research community. We codirect the NHLBI-supported National Sleep Research Resource (NSRR; sleepdata.org), which displays many sleep data terms with their corresponding provenance and meta-data, alongside visualization tools and open-source sleep analytic software to complement its growing repository of polysomnography and actigraphy files. The NSRR currently shares over 2TB of sleep data per week with a diverse user community (students, investigators, and industry) using a controlled but user-friendly data approval/access mechanism. In fact, 4 of the 10 articles in this issue used NSRR data. NSRR data could be more powerfully leveraged for Sleep-Omics by further expanding linkages with other covariate data, longitudinal outcomes, and genomic data. To achieve the latter goals, NSRR plans to migrate and interoperate with NHLBI’s emerging cloud-based platform, BioData Catalyst, which will hopefully provide a powerful data ecosystem that combines diverse research data from multiple cohorts. As these data sets are assembled, it will be critical that all efforts are made to ensure diversity of data (training on limited data sets can potentially propagate health disparities), as well as that the labels used to “train” data are accurate and that data are updated regularly to address temporal changes in disease management and clinical care and other longitudinal influences.

If successful, the expanded and more systematic integration of sleep data within clinical records—combined with research on “next-gen” integration of sleep-focused data with imaging, molecular and longitudinal clinical data—will provide the community the tools and data needed to potentially uncover the mysteries of sleep and establish the knowledge-base for improving patient care and population health.

Disclosure Statement

The authors have nothing to disclose.