Strengths, weaknesses, opportunities, and threats of using AI-enabled technology in sleep medicine: a commentary

Abstract

Over the past few years, artificial intelligence (AI) has emerged as a powerful tool used to efficiently automate several tasks across multiple domains. Sleep medicine is perfectly positioned to leverage this tool due to the wealth of physiological signals obtained through sleep studies or sleep tracking devices and abundance of accessible clinical data through electronic medical records. However, caution must be applied when utilizing AI, due to intrinsic challenges associated with novel technology. The Artificial Intelligence in Sleep Medicine Committee of the American Academy of Sleep Medicine reviews advancements in AI within the sleep medicine field. In this article, the Artificial Intelligence in Sleep Medicine committee members provide a commentary on the scope of AI technology in sleep medicine. The commentary identifies 3 pivotal areas in sleep medicine that can benefit from AI technologies: clinical care, lifestyle management, and population health management. This article provides a detailed analysis of the strengths, weaknesses, opportunities, and threats associated with using AI-enabled technologies in each pivotal area. Finally, the article broadly reviews barriers and challenges associated with using AI-enabled technologies and offers possible solutions.

Citation:

Bandyopadhyay A, Oks M, Sun H, et al. Strengths, weaknesses, opportunities, and threats of using AI-enabled technology in sleep medicine: a commentary. J Clin Sleep Med. 2024;20(7):1183–1191.

INTRODUCTION

Artificial intelligence (AI) promises transformations in key health care domains: clinical care, lifestyle management, and population health management. Although AI promises to improve efficiency, increase patient access, and decrease provider burnout, there is considerable skepticism surrounding its unbridled use. In a previously published position statement in 2020, the Artificial Intelligence in Sleep Medicine Committee members recognized that sleep medicine is uniquely positioned to benefit from the advancements of AI-enabled technology.¹ Since then, our committee has spearheaded numerous initiatives aimed at fostering awareness of AI, probing the community’s hesitations, and identifying the needs of members that could be addressed through AI-enabled technology. We implemented a top-down approach and reviewed existing literature to identify areas in sleep medicine with critical gaps that could benefit from AI. We also implemented a bottom-up approach and had extensive discussions with sleep researchers and clinicians interested in implementing AI-enabled technology in their practice, as well as industry representatives who have successfully implemented AI innovations for sleep research and clinical practice. Based on this robust approach over the past 2 years, our committee elucidated and summarized the transformative potential of AI in sleep medicine and research. In this commentary, we execute a meticulous strengths, weaknesses, opportunities, and threats (SWOT) analysis for 3 pivotal areas within sleep medicine: clinical care, lifestyle management, and population health management. The SWOT analysis allows us to provide crucial insights and strategic foresight that helps to identify and prioritize areas where sleep medicine can most effectively leverage AI technologies for impactful advancements.

CLINICAL CARE

AI-enabled technologies are computational methods that mimic human intelligence; these can process and analyze vast amounts of patient data—such as medical histories, genetic profiles, or imaging data—to assist clinicians in diagnosing diseases, predicting patient outcomes, or personalizing treatment plans. The utilization of AI-enabled technology in clinical care is rapidly expanding. AI has been shown to play a role in the assessment, diagnosis, and management of sleep disorders.² AI has facilitated advancements in patient monitoring and management, polysomnography (PSG) sleep scoring, behavioral sleep medicine, and digital therapeutics (see Table 1). Innovative diagnostic markers in PSG have increased our understanding of phenotypes of various sleep disorders.^3–5

Table 1.

Current and future clinical applications of AI in sleep medicine.

Diagnostic Tools Including PSG	Personalized Treatments	Follow-up
Automated sleep stages scoring using EEG6,7 Detection of sleep microstructure events (spindles, K-complex, arousals)8,9 Respiratory and EMG assessments for sleep parameter detection10,11 Hypnodensity graph for narcolepsy or RBD diagnosis5,11 Mining of PSG variables and biometric data to diagnose sleep disorders12 Use of AI to supplement actigraphy data for diagnosis of insomnia13 Use of AI on actigraphy to supplement questionnaire to screen for RBD14 Cardiovascular and neurologic ramifications of OSA predicted by AI15 Brain age index assessing risk for dementia and mortality16,17	AI-driven algorithms to predict failure/success of HGN stimulation18 Identifying different OSA endotypes19,20 Digital behavioral intervention21	PAP adherence22,23 PAP mask fittings24 Prediction of disease phenotypes to tailor treatment recommendations13

AI = artificial intelligence, EEG = electroencephalography, EMG = electromyography, HGN = hypoglossal nerve, OSA = obstructive sleep apnea, PAP = positive airway pressure, PSG = polysomnography, RBD = rapid eye movement sleep behavior disorder.

The implementation of AI-enabled technology needs to be developed and executed in a stepwise fashion. The steps should involve testing, repeated external validations for generalizability in an iterative manner and periodic surveillance for updates as it is implemented in the real world. Implementation of AI-enabled technology is associated with some general SWOT across all applications in clinical sleep medicine.

Strengths

• Efficient analysis of real-world and clinical data

• Identification of complex patterns in sleep data

• Automation of sleep stage scoring and event detection

• Personalized approaches to sleep disorder assessment

• Longitudinal patient monitoring

• Enhanced treatment adherence

• Cost-effectiveness of care

Weaknesses

• Paucity of training datasets

• Impact of biases in training data

• Limitations of gold standard availability and quality

• Lack of standard variables

• Generalizability challenges

• Ineffective use of technology

• Lead time bias

• Integration with clinical workflow

• Lack of financial resources

• Ethical concerns

• Expert interpretation and clinical judgment still needed in complex situations

Opportunities

• Improved patient access to health care

• High-quality clinical research trials

• Enhanced accuracy and objectivity

• Engagement of diverse communities

• Remote monitoring and telemedicine

• Database creation for intervention assessment

• Development of prospective electronic health record

• Predictive analytics for treatment adherence

• Cost-effectiveness demonstration

Threats

• Overreliance on AI technology

• False positives

• HIPAA (Health Insurance Portability and Accountability Act) compliance issues

• Variability in patient acceptance and trust

• Accountability and legal ramifications

• Time and resource lag

• Perceived job insecurity

• Increase in intervention due to data overload

LIFESTYLE

Technology is rapidly advancing how we provide care to patients and improve population health. As indicated above, while some AI technologies are available to health care providers to assess and treat patients or for research, consumer sleep technologies are available for mass use. Consumer sleep technology has a role in improving sleep health through sleep tracking, assessment, and augmentation. These commercially available devices come in different shapes, such as fitness wristband trackers, smartphone apps, smart finger rings, under-the-mattress sensors, and devices that utilize a microphone, camera, thermometer, radar, or Wi-Fi signal to assess sleep. These devices usually have much lower prices than clinical-grade devices, are easy to use, and have user-friendly interfaces. Additionally, some of these devices not only assess sleep but also provide other measurements, such as heart rate, heart rate variability, and blood oxygen level. Understanding these sensors and their use is critical to their potential.^25–27 The quality and quantity of data that these devices provide could be used to give feedback to the customers for lifestyle management and awareness regarding their sleep or identify those at risk for sleep disorders.

There are clear benefits to incorporating AI in consumer sleep technology. For example, sleep trackers can monitor sleep timing and duration over long periods and generate reports to share with one’s provider. AI-enabled technology can give consumers feedback on sleep, which promotes self-efficacy and sleep health. Other devices or software applications may be able to assess risk for sleep disorders such as sleep apnea and alert consumers to discuss the findings with their providers. Therapeutically, such tools can be used to track the treatment of disorders such as insomnia. Mobile applications can be useful to capture user-input data, while more objective data can be collected with consumer actigraphy.²⁸ Seeing such data in real-time will aid regular therapy adjustments, particularly when the next available appointment with a medical provider is months away. For example, actigraphy is used to treat circadian rhythm sleep/wake disorders, but data are downloaded after several days of recording and then interpreted, or the patient must bring data from their wearable. When assessing treatment, adjusting the timing of circadian-based therapies needs to be done more frequently. In the case of non–24-hour rhythm disorder, having actigraphy data available remotely daily can help tailor the timing of treatment with melatonin and light therapy.

Clinicians face many issues in incorporating consumer sleep technologies and AI in practice. Clinicians are often asked questions about consumer devices and need to be able to understand these devices to promote sleep health. Consumers often ask about such technology, either for choosing a device or interpreting data acquired from wearables. It is important for clinicians and consumers to know what the devices are, in what populations and disorders they were trained or validated, how data are measured, and how associated software algorithms create interpretations of such data. One issue in this area is that, often, raw data are not available and predictive algorithms are proprietary; thus, independent validation is sparse and, when published, often limited to frequent sleep pathologies or controls. Setting expectations with patients and what additional data should be collected, if not incorporated in the device or software already, is critical to improving interpretation in the clinical context.²⁷ For example, there should be adequate data collected and viewable. Rather than seeing how many hours a patient slept or the amount of “deep sleep” in 1 night, having 7–14 days with the timings of sleep and wake and the days (eg, weekdays/weekends) or annotation about alcohol or caffeine use significantly improves clinical interpretation. Companies developing devices and AI software also need to consider the clinical context when making interpretations about sleep health to avoid misleading the consumer. Lifestyle modification can include consumer sleep monitoring; sleep optimization, such as mitigating sleep disruption and sleep enhancement; as well as personalized risk identification. The SWOT of AI-enabled technology used in lifestyle modifications to promote good sleep health have been summarized below.

Strengths

• Improved consumer access and scalability

• Value of longitudinal data for monitoring

• Access to digital nonpharmacologic options to improve sleep

• Enhancement of sleep quality and secondary physiology

• Empowerment of patients in identifying sleep disorders

Weaknesses

• Overwhelming amount of data with unclear significance

• Lack of integration infrastructure

• Device-specific proprietary software limiting interpretation

• Sparse clinical guidelines for decision making

• Absence of standard framework for device usage

• Insufficient research and trials supporting current technologies

• Lack of explainable AI for understanding predictions

• Validation gaps in heterogenous populations

Opportunities

• Consumer education on device utility and data interpretation

• Promotion of standardized reports by devices

• Improvement in health care insurance coverage for data monitoring and interpretation

• Clinician education on device usage and integration

• Establishment of standards for digital interventions

• Enhancement of device feedback

• Research promotion linking sleep enhancement to improvement in chronic diseases

Threats

• Cost limitations on access

• Potential misleading of consumers due to data inaccuracies

• Reliability and accuracy concerns of device and software

• Unsupervised use of digital interventions by patients

• Risks of misuse or unsafe practices

• Anxiety and resource utilization from false positives

• Hindrance in seeking clinical evaluation due to presumptive diagnosis or false negatives

• Inadequate patient health literacy for data interpretation

POPULATION HEALTH

As the role of AI expands in the clinical and hospital settings, there is a growing potential for AI applications to impact population health and health policy, particularly as real-world data are being incorporated as part of population health data.^29,30 AI can impact population health in the areas of sleep awareness, surveillance, prevention, and access.

By leveraging natural language processing and reinforcement learning, AI has the potential for risk prediction to impact population health by identifying social determinants of health and outcomes. This can be particularly valuable when using messy and complex data while processing medical records and diagnosis coding, and linking clinical and administrative data to develop comprehensive data prediction research platforms (https://pophealthanalytics.com). By incorporating survey data and person-generated data, AI can have a unique role in health equity analysis.³¹ This has implications for the identification of factors for health outcomes specifically related to designing interventions, predicting outcomes, and allocating resources at the population health level. For example, real-time data related to social determinants that are geo-linked can identify vulnerable populations at risk for sleep disorders and identify areas of health disparities and inequities. This can be achieved using the internet and other social media platforms (similar to the COVID-19 [coronavirus disease 2019] risk identification), in addition to individual-level aggregated data from commercially available sleep-monitoring devices when available. While adopting AI for population health, the concept of algorithmic fairness needs to be considered in machine learning to avoid the exclusion of groups with health disparities while developing AI models.³²

As AI is increasingly used at the population health level, there are emerging issues that need consideration including the accuracy of big data, data privacy, and legal implications and liability. Typically, population health analysis addresses external validity by ensuring the inclusion of the representative population. Using unstructured data without being sure of the inclusion of representative populations can result in a biased performance of the algorithm to impact the external validity of the big-data prediction.^31,33 The area of data privacy is evolving with issues of privacy at the collective and individual level and the need for consistent regulation.³¹ Synthetic data approaches, where patient-identifiable information is replaced with fake data to avoid reidentification, may provide a solution to this issue by using artificially generated data rather than real data at the population and individual level.³⁴

A framework for the application of AI to Sleep Population Health is included in Figure 1 and includes 4 domains—awareness, surveillance, prevention, and access—and linked to the continuum of care for sleep population health from the “patient’s perspective” to include screening, care facilitation, diagnostics, and population health management. AI can enhance the identification of subpopulations (eg, geographic, race/ethnicity, socioeconomic, clinical cohorts, deficiencies at points on the care continuum) specifically in which care gaps and suboptimal outcomes exist. The described domains provide a framework for the health system to develop programs or clinical strategies that can improve health equity and optimize the efficient and effective allocation of resources. The identification of subpopulations amenable to specific programs or strategies further enables the feasibility of personalized precision medicine interventions, with the potential benefit of enhancing both individual and population outcomes. The SWOT of AI-driven technology used in the application of AI to sleep population health have been summarized below.

Figure 1. Four domains for application of AI as a tool to improve sleep population health.

AI = artificial intelligence, EHR = electronic health record, HSAT = home sleep apnea test.

Strengths

• Efficient analysis of diverse sleep data for large-scale studies

• Seamless integration of data from various sources

• Identification of sleep-related health trends and patterns

• Tailored sleep recommendations and interventions based on individual needs

• Detection of subtle sleep disturbances or anomalies for early intervention

• Continuous monitoring and real-time feedback

• Automation of data analysis for faster insights

• Recognition of complex sleep patterns aiding in disparities identification

• Accessibility to sleep education and support via AI-driven assistants

• Acceleration of sleep research through rapid data analysis

• Optimization of resource allocation for cost reduction

• Adaptation and enhancement of AI algorithms over time

Weaknesses

• Lack of standardized utilization and validation across populations

• Absence of standardization in electronic health record data

• Limited diversity in available data for comprehensive analysis

• Potential for algorithmic bias

• Constraints in applying AI across all clinical domains and populations

Opportunities

• Utilization of patient-facing technologies in ambulatory settings

• Identification of sleep disorder risks through electronic health record data analysis

• Triage of patients into specific care pathways using AI

• Real-time monitoring of overlapping chronic conditions

• Enhancement of patient engagement through technology in group settings

Threats

• Validation challenges due to variation in vendor-dependent processes

• Reduced generalizability due to lack of standardized data

• Care redirection based on misallocated resources

• Concerns regarding data privacy

• Risk of misinformation and overreliance on technology replacing face-to-face engagement

BARRIERS AND CHALLENGES TO THE APPLICATION OF AI IN SLEEP MEDICINE

AI-enabled technology may provide opportunities to improve health care by increasing diagnostic consistency and increasing throughput. As with all medical tools, the benefits must be weighed with potential risks associated with using the tool. The main risks posed by AI-enabled technology relate to the proper training, validation, and use of the devices. AI systems are trained on relevant, but finite, data. However, it is well known that AI systems can amplify biases present in the training data or other aspects that impact the training of the AI system.^35,36 These biases may be present despite the best efforts of the developers of the AI systems and in the absence of any discriminatory intent. Additional risks may arise when the deployed environment differs from those during AI training.³⁷ This domain shift problem caused by different distributions between source/reference data and target data is well recognized. Domain shift occurs when training, validation, and test data are drawn from a probability distribution that is different from the distribution of the data on which we use our predictive models. Domain adaptation research is crucial to mitigate this risk.³⁸ Other concerns regarding the clinical adoption of AI medical devices include securing patient privacy in cloud-based systems and integrating the AI system with existing electronic medical systems and clinical workflow. Finally, there are concerns about overreliance on black box algorithms causing clinicians not to be able to establish trust with their patients as well as the possibility of clinicians increasingly needing to validate the outputs of the model and detect incorrect outcomes,³⁹ such as seemingly confident, but inaccurate, responses when large language models answer medical questions. Interpretable AI systems are needed to explain the outputs using terms that the clinicians and/or patients are familiar with. Suggested resources on AI risk-management framework can be found in the Artificial Intelligence Risk Management Framework (AI RMF 1.0).⁴⁰

Some of the risks pertaining to training AI systems can be best mitigated by following good machine-learning practices. An example is the one proposed by the Food and Drug Administration.⁴¹ Most commonly, the dataset is split into 3: a training dataset, a dataset used for hyperparameter tuning and creating a final model, and the test dataset of never-seen data. When examining the AI model, clinicians should determine whether all datasets were clearly defined and reflective of the population they plan to use it on and how robust is the model compared with the gold standard.⁴² Overfitting can occur when the trained model cannot generalize and fits too closely to the training dataset. To avoid overfitting the AI system to the training dataset, it is important to use unseen data for testing the AI system. The need to determine AI system performance across devices, demographics, and clinical facilities is recognized, and several endeavors have been implemented by our colleagues in other medical fields.⁴³

The need for risk mitigation may require multiple stages of regulation by the government, field-specific organizations comprising field experts, institutions, and clinicians. While adding regulatory approval processes may increase trust/safety, it may also slow innovation. Effective strategies have been proposed to ensure that the clinical effectiveness of AI medical devices is preserved after deployment. These include Post Market Surveillance mandated by the Food and Drug Administration and Post Market Clinical Follow Up mandated under the Medical Device Regulations in Europe. A medical algorithm audit has been proposed as a comprehensive method of ensuring clinical effectiveness.⁴⁴ Other important steps toward ensuring the clinical effectiveness of AI medical devices include detailed descriptions of validation datasets and methods.^45–47

One of the key solutions to all the challenges imposed by AI technology is more clinical research on heterogeneous populations comparing AI with traditional approaches. Researchers should conduct independent research to validate the clinical performance of AI medical devices and vendors should be encouraged to publish their validation protocols and results. The guidelines published by the CONSORT-AI and SPIRIT-AI steering groups can help ensure the quality of the reporting and suggest adding standard sections on implementation verification.^32,48,49 Wu et al⁴⁶ collected and reported the methods used to validate Food and Drug Administration–registered AI medical devices and presented their results in an interactive and intuitive manner, which highlighted the need for prospectively evaluating the devices and performing post-market surveillances. Prospective randomized controlled trials to determine noninferiority with the standard of care are important to determine the clinical effectiveness of the algorithm. Some common risks and possible solutions have been described in Table 2. Navigating these challenges and building the AI–human interaction can increase the burden on the clinician and require considerable time and effort. For this reason, the Current Procedural Terminology editorial panel has introduced AI taxonomy for medical services and procedures.⁵⁰ As AI-enabled algorithms find a role in our clinical practice, such endeavor is another way to value the time and effort used to maintain the AI–human interaction and prevent burnout from data overload.

Table 2.

Barriers and possible solutions to incorporating AI in clinical practice.

	Possible Solutions
Challenges of validating AI-enabled algorithms
Variability in reported information about AI-enabled algorithms	Use predefined dataset description forms including detailed information and justification about the training dataset such as: Number of individuals in the dataset Inclusion/exclusion criteria Relevant physiological descriptors such as age, sex, and race Relevant clinical descriptors such as clinical condition, comorbidities, and medications Inclusion criteria of recruited population (ie, information about where and why the data were collected): routine clinical diagnosis of a specific sleep disorder, as part of a research study, a population-based sample, etc Source of testing dataset, specifically whether it comprises the same population as the training dataset or not Performance metrics details utilized to compare performance with standard of care Access to code used for the study and the final weights of the model and its characteristics
Lack of generalizability	Create external heterogenous validation resources that can compare AI algorithm performance with standard of care Need for prospective studies comparing algorithms with standard of care
Overfitting AI-enabled algorithm	Separate validation data from training data Leave-one-institute-out cross-validation Identify common information sources that may be present in validation and training data
Need for reproducibility and repeatability	Test the AI-enabled algorithm on multiple datasets that were built for similar intended use and indication. Of note, certain algorithms, such as those utilizing deep learning, should not be tested on the same training dataset; the training, validation, and test datasets always need to be clearly distinguished.
Researcher-clinician-industry collaboration	Involvement of all key stakeholders throughout the process of building and validating the AI-enabled algorithms through strategic meetings along the timeline
Challenges of implementing AI-enabled technology in practice
Wide variety of AI-enabled products to choose from	Key points for consideration: Will it make an impact on clinical care? Will it make an impact on productivity? Will the returns be sustained or diminish with time? Does it apply to my patients?
Lack of knowledge about AI algorithms	Increase in awareness and education regarding how AI algorithms learn and what are the various biases associated with its use
Lack of knowledge about when to use AI algorithms	Need for prospective studies assessing clinical noninferiority in a similar population and investigating how the AI–human team performs.
Lack of ease in operability	User-friendly interface, improved findability and accessibility
Physiological explanation of the working model	An explanation of why the sleep data show high risk (eg, dementia, where AI says it is because of decreased delta-to-theta power ratio during N3, eg, narcolepsy, where AI says it is because of an SOREMP during nocturnal sleep and presence of mixed REM-wake states)
Difficulty in utilizing AI as a tool complementing human skills	AI algorithm highlights the epochs that were difficult to predict or shows the features used to make predictions for clinician oversight, using natural human languages such as ChatGPT (OpenAI, San Francisco, California, USA)51
Patient privacy and workflow integration concerns	Educate users on how to maintain privacy in the era of cloud-based systems Invest in secure access portals Dedicated IT team providing logistical support
Continuous updates of AI algorithms can create differences in clinical outcome	Aftermarket surveillance by the manufacturing company and the need to revalidate algorithms after significant updates
Patient access to AI algorithm-generated clinical data	Need for shared decision making with clinician taking the lead to integrate data from medical-grade and consumer-grade devices and formulating treatment plans with patient input

AI = artificial intelligence, IT = internet technology, REM = rapid eye movement, SOREMP = sleep-onset REM period.

CONCLUSIONS

AI’s rapid growth in recent years holds tremendous potential in the field of sleep medicine, with broad applications across clinical care, lifestyle management, and population health. In clinical settings, AI serves as a powerful tool for improving patient access by streamlining health care delivery, enabling remote monitoring, and enhancing diagnostic accuracy. It plays a pivotal role in risk stratification, offering clinicians invaluable insights into predicting and preventing sleep disorders. Furthermore, AI-driven prognostic tools empower health care providers to make informed decisions and tailor interventions for patients with sleep-related conditions.

Beyond the clinical realm, AI contributes significantly to promoting good sleep health in individuals’ daily lives. Through wearable devices and smartphone applications, it offers personalized sleep recommendations, encouraging healthier sleep habits and facilitating self-monitoring. Additionally, AI-driven sleep trackers generate valuable data for users, fostering awareness and understanding of their sleep patterns and needs.

On a broader scale, AI’s potential extends to improving population health related to sleep. By analyzing large datasets, it can identify trends and correlations in sleep-related issues, facilitating the development of public health strategies and policies. For instance, AI can aid in the design of interventions targeting shift workers, school start times, and drowsy driving prevention, addressing societal sleep challenges more effectively. However, as we navigate the exciting possibilities of AI in sleep medicine, we must also acknowledge persistent concerns and challenges. These include the need for generalizability of AI models across diverse populations, addressing resource limitations in implementing AI-enabled technology, safeguarding patient privacy in data collection and analysis, establishing clear guidelines for AI’s clinical use, and ensuring that AI enhances rather than replaces human interaction in health care delivery. To harness AI’s full potential, it is crucial to educate our health care community about its capabilities and intricacies while fostering collaboration between researchers, health care professionals, and industry stakeholders. By doing so, we can make AI in sleep medicine more clinically meaningful, continually refine its application, and drive significant advancements in the field, ultimately improving sleep health outcomes for individuals and populations alike.

DISCLOSURE STATEMENT

Azizi Seixas reports being a consultant to Idorsia, Philips, and serving on the Board of Directors for Moshi Kids. Emmanuel Mignot reports being a consultant for Takeda Development Center Americas, Inc, Idorsia, Ambulatory Monitoring, and Avadel Pharmaceuticals, Inc (USA). Shahab Haghayegh reports receiving consulting fees from Achaemenid LLC. Jon Agustsson is an employee of Nox Medical ehf. Sam Rusk is an employee and shareholder of EnsoData, Inc. Steve Van Hout, Matthew Anastasi, and Andrew Sampson are employed by the American Academy of Sleep Medicine. The other authors report no conflicts of interest.

ABBREVIATIONS

AI

artificial intelligence

PSG

polysomnography

SWOT

strengths, weaknesses, opportunities, and threats