Non-contact on-device detection of obstructive sleep apnea from infrared video

You Rim Choi; Dongik Park; Hyun Kyung Lee; Suahn Bae; Sehyun Park; Dong-Kyu Kim; Yong Min Kim; Chan-Soon Park; Hyung-Sin Kim; Hyun-Woo Shin

doi:10.1038/s41467-026-73915-2

. 2026 Jun 4;17:7163. doi: 10.1038/s41467-026-73915-2

Non-contact on-device detection of obstructive sleep apnea from infrared video

You Rim Choi ¹, Dongik Park ¹, Hyun Kyung Lee ², Suahn Bae ¹, Sehyun Park ¹, Dong-Kyu Kim ^3,¹⁰, Yong Min Kim ⁴, Chan-Soon Park ⁵, Hyung-Sin Kim ^1,^✉, Hyun-Woo Shin ^2,^6,^7,^8,^9,^✉

PMCID: PMC13396759 PMID: 42243134

Abstract

Obstructive sleep apnea (OSA) affects nearly one billion people worldwide, yet most cases remain undiagnosed because standard testing requires overnight polysomnography, which is costly and inconvenient. Here we show that SlAction, a contact-free, wall-mounted near-infrared (NIR) video system, can estimate the apnea-hypopnea index (AHI) and detect positional OSA entirely on-device, without attached physiological sensors or cloud-based data transfer. The system was trained and validated on 936 clinical recordings (>5,000 hours) from three hospitals. Motivated by the physiology of respiratory arousals (RA), we identify RA through their associated body movements visible in video as surrogates of breathing disturbances that correlate with apnea severity, enabling real-time analysis with a lightweight deep learning model on low-cost hardware. Across internal and external test sets, SlAction substantially reduces AHI estimation error, improves classification performance for severity and positional OSA, and operates markedly faster than prior video-based approaches. Although evaluated in controlled clinical settings, these findings support the potential of SlAction as a scalable, contact-free, and privacy-preserving approach for OSA screening.

Subject terms: Diagnosis, Medical imaging, Sleep disorders

This study presents a wall-mounted infrared video system that estimates sleep apnea severity and positional apnea directly on-device, without contact sensors or cloud processing. While the present study evaluated performance only in controlled clinical settings, future studies may explore if the system enables scalable, privacy-preserving screening for sleep apnea.

Introduction

Sleep is a fundamental biological process, yet nearly one billion individuals worldwide are estimated to have obstructive sleep apnea (OSA)^1–3, with over 80% of moderate-to-severe cases remaining undiagnosed^4–7. Untreated OSA is associated with hypertension, stroke, and traffic accidents^8,9, resulting in an annual economic burden of approximately US$150 billion in the United States alone¹⁰. Early detection and longitudinal follow-up are therefore recognized public health priorities.

The diagnostic gold standard, overnight polysomnography (PSG), quantifies the apnea-hypopnea index (AHI) using over 15 contact sensors in a sleep laboratory^11,12. However, its high cost ( > US$1,000 per night)¹³, limited capacity (months-long waiting lists), and poor ecological validity (sensor discomfort, first-night effect¹⁴ and single-night sampling^15,16) restrict accessibility. While home sleep apnea tests (HSATs) have emerged, a recent 20-paper meta-analysis found a 39% diagnostic error rate¹⁷. Cutting-edge alternatives, OxiNet¹⁸ (SpO₂-only, requiring sleep staging ground-truth) and DRIVEN¹⁹ (SpO₂ and multichannel thoraco-abdominal belts), still depend on contact sensors and do not support guideline-recommended positional OSA (POSA)²⁰. Thus, an unattended, comfortable and privacy-preserving solution remains unmet.

Recent advances in wearable, nearable, and acoustic technologies have significantly expanded the landscape of non-invasive approaches for OSA screening. Wearable devices, such as wrist-worn inertial sensors²¹, fingertip oximeters²², and patch-based respiratory monitors²³, have demonstrated reliable performance with sensitivity and specificity often exceeding 80% compared to polysomnography (PSG). However, these systems typically require direct physical contact, regular calibration, and user compliance, and they may interfere with natural sleep behavior during prolonged monitoring. Nearable solutions, such as mattress-embedded or under-the-pillow sensors, mitigate some of these issues but are generally restricted to single-user, location-dependent deployment, which limits their scalability in multi-person or dynamic environments.

Acoustic methods^24–26 and radar-based technologies, especially those utilizing Doppler and frequency modulated continuous wave (FMCW) signals^27–29, offer powerful non-contact alternatives for sensing respiratory activity without physical attachment. Radar systems can capture subtle cardiopulmonary micro-motions and may operate through bedding. However, these modalities often face challenges in complex real-world environments. Radar-based systems may be sensitive to subject-sensor distance, null-point effects, and motion artifacts unrelated to respiration³⁰. In addition, both acoustic and radar approaches may provide limited spatial discrimination in multi-person scenarios, such as distinguishing a patient from a bed partner. These limitations motivate complementary non-contact strategies that operate passively, remain robust in practical settings, and provide explicit spatial context.

To address these challenges, we develop SlAction, to our knowledge the first wall-mounted near-infrared (NIR) video-based OSA monitoring system that operates entirely on-device without sleep staging, cloud transfer, or contact sensors (Fig. 1). By leveraging body-shape and motion cues, SlAction enables practical event detection and explicit subject localization, while providing a modular foundation for future multimodal integration and sleep-stage analysis. SlAction is grounded in the hypothesis that involuntary whole-body motions may carry physiologically meaningful signatures of respiratory disturbance. The system estimates AHI and categorizes OSA severity into four clinical levels (Normal, Mild, Moderate, Severe), and determines positional OSA status to inform follow-up discussions, such as positional therapy^31–34 after confirmatory clinical evaluation. SlAction runs in real time on the CPU of a Jetson-Nano-class device (US$120) and streams only summary results via Wi-Fi, enabling same-morning screening/triage reports without recurring consumables.

Fig. 1 — Polysomnography (PSG) is a time-consuming manual process with several limitations, such as the first-night effect and reliance on single-night examinations. Our approach addresses these issues by enabling continuous sleep monitoring using video recordings. *SlAction* estimates the apnea-hypopnea index (AHI) to classify obstructive sleep apnea (OSA) severity and assess positional OSA status. The system operates on-device, discarding video data after processing and retaining only predictive reports for hospital-based tracking, screening/triage and referral for confirmatory PSG, as well as longitudinal follow-up.

For robust development and evaluation, we constructed the Korea Video Sleep Study (KVSS), the largest multi-institutional sleep-video dataset to date (936 patients; > 5,000 hours; three hospitals; mean age ± SD: 51.4 ± 14.4 years; 23.8% female; mean BMI ± SD: 26.3 ± 4.4 kg/m²), 23 × larger than the largest previously published set³⁵, with institutional review board (IRB) approvals at all participating institutions. All recordings are synchronized with full PSG and scored by sleep experts following the American Academy of Sleep Medicine (AASM) 2012 guidelines³⁶, with facial regions masked to ensure participant anonymity. The dataset captures real-world heterogeneity in camera angle, blanket thickness, and body habitus, providing a stringent testbed for algorithmic generalization. The only published video-based model (Akbarian et al.³⁵) reported 86% macro-F1 for OSA classification (AHI ≥ 15) on a limited dataset of 41 subjects without camera-angle variation or blanket occlusion. When re-evaluated on a train/validation/test split of our KVSS dataset, its performance dropped to 54% despite hyperparameter tuning, highlighting the importance of large, heterogeneous cohorts and motivating our SlAction.

Importantly, we identify respiratory arousal (RA), defined by EEG-based cortical arousals associated with apnea and hypopnea (AH) events under AASM criteria, as a reliable and effective video surrogate for AHI estimation and OSA severity assessment. Compared to AHs, which often involve subtle movements and variable durations (10 seconds to several minutes), RAs are typically brief and accompanied by pronounced thoraco-abdominal motion. Consistent with this physiological distinction, our analysis of the KVSS dataset found that RA episodes are, on average, three-fold shorter than AH events and that RA frequency correlates strongly with AHI (r = 0.94), indicating that RA serves as a compact, motion-salient, and temporally precise surrogate for respiratory disturbance in video-based assessment.

During an obstructive event, the upper airway collapses, ventilation decreases, and blood-gas homeostasis is disrupted, leading to increased chemoreflex and mechanoreflex input to the brainstem ventilatory controller³⁷. When this rising respiratory drive fails to reopen the airway, a cortico-thalamic arousal is triggered; sympathetic tone surges and pharyngeal dilator muscles activate, restoring airway patency³⁸. Crucially, while not all obstructive events trigger this cortical arousal^39,40, those that do are associated with a significantly amplified physiological response. The first post-obstruction breath, often termed the recovery breath, is substantially larger when accompanied by a cortical arousal (e.g., ≥2-fold increase in tidal volume) compared to a non-arousal recovery^39,41. This amplified ventilatory and motor response results in a thoraco-abdominal excursion that is markedly larger than baseline breathing, producing a visually salient, high-amplitude chest-motion burst in infrared footage. This strong coupling between the EEG-defined RA label and its visible biomechanical manifestation helps explain why RA frequency tends to track AHI, motivating our use of motion-based RA cues as a practical alternative when direct AH tracking is unreliable in infrared video. Leveraging this insight, we simultaneously detect RA (for AHI) and classify sleep position (for POSA) in a unified model, using short-term video analysis at the 30-second epoch level (a standard PSG scoring unit).

The RA-driven, position-integrated, and epoch-level approach keeps the system lightweight at every stage: the core network is adapted from MoViNet-A2 with only 4.8M parameters⁴², and inference requires only 150 video frames per 30-second epoch. This compact design makes fully on-device operation practical and inherently privacy-preserving. All video is processed locally in real time, even on the CPU of a Jetson-Nano-class board. Raw footage is immediately discarded after inference, and only summary outputs, such as AHI estimates and POSA status are retained.

Evaluation on KVSS demonstrates that SlAction outperforms state-of-the-art video-based and contact-sensor-derived signal-based methods in classifying OSA severity and POSA, underscoring the synergy between video data and RA-based markers. Specifically, compared to the only prior video system³⁵, SlAction is 28.2 × faster, halves the AHI mean absolute error (MAE), and improves macro-F1 by 30 percentage points for severity classification and 23 percentage points for POSA detection. Together, these features make SlAction a practical, scalable, and privacy-preserving solution for accurate OSA monitoring in diverse real-world environments. Although KVSS represents a Korean clinical cohort, its scale and inter-hospital diversity provide a rigorous testbed; independent validation in other populations is warranted.

In this work, we present SlAction, a wall-mounted NIR video-based system for fully on-device estimation of AHI, OSA severity, and POSA status without contact sensors or cloud processing. By leveraging RA as a motion-salient surrogate, SlAction integrates RA detection and sleep position classification within a lightweight epoch-level architecture suitable for real-time embedded deployment. Evaluation on the multi-institutional KVSS dataset demonstrates strong performance in OSA severity and POSA classification while maintaining computational efficiency and privacy-preserving operation. These findings establish RA-driven video analysis as a practical and scalable framework for non-contact sleep apnea monitoring in real-world settings.

Results

KVSS : a diverse sleep video dataset

We constructed the KVSS dataset to facilitate advanced video-based sleep analysis. As the largest known sleep video dataset to date, it comprises 936 patient-level cases collected under realistic clinical and environmental conditions across three hospitals, totaling over 5000 hours of sleep video. As illustrated in Fig. 2a, each case includes anonymized infrared video with facial masking, synchronized comprehensive multimodal PSG signals, and expert annotations following the AASM 2012³⁶ guidelines. To enhance generalizability, videos were captured under diverse recording configurations across hospitals, encompassing variations in room layout and camera placement (Fig. 2b).

Fig. 2 — a Collection of the KVSS dataset during routine polysomnography (PSG) studies. The dataset includes synchronized multimodal PSG signals, anonymized infrared sleep video recordings with masked facial regions, and expert annotations following American Academy of Sleep Medicine (AASM) guidelines. Data were collected from 936 patients across three medical institutions. b Hospital-specific recording settings. Sample frames, total number of patients, camera angles, and shooting distances are presented for each hospital. Images shown in (a) and (b) are schematic visualizations derived from original recordings. The frames were anonymized and converted into illustration-style renderings to remove identifiable features while preserving body position and recording geometry. **c–f** The largest portion of the dataset (Hospital A; n = 475 patients) served as the primary source for model training and evaluation. Demographic and clinical characteristics are shown: sex and age distributions (c), with Male n = 353 and Female n = 122; BMI distribution by sex (d); OSA severity and AHI distribution (e), with Normal n = 39, Mild n = 103, Moderate n = 125, and Severe n = 208; and the proportion of patients with positional OSA across severity levels (f). In (d) and (e), center lines indicate medians; box bounds indicate the 25 and 75th percentiles (interquartile range, IQR); whiskers extend to the most extreme values within 1.5 × IQR; points beyond whiskers are shown as outliers. Each data point represents one independent patient (unit of analysis: individual patient), and no technical replicates were included.

Hospital A contributed 475 recordings, providing the most balanced demographics and AHI distribution, and served as the primary dataset for model training and evaluation. This dataset includes diverse participant profiles, spanning a broad range of age and BMI (Fig. 2c, d). It also reflects clinical trends, with a higher proportion of male participants (Fig. 2c), a majority of patients presenting moderate-to-severe OSA (Fig. 2e), and a notable fraction of mild-to-moderate cases exhibiting positional dependency (Fig. 2f), underscoring the clinical importance of POSA assessment.

Beyond Hospital A, the external cohorts (Hospitals B and C) exhibit distinct clinical and environmental characteristics that strengthen the generalization analysis. Hospital B, a neurology center, included a substantially higher proportion of patients with non-respiratory sleep disorders, such as REM sleep behavior disorder (26%) and periodic limb movements during sleep (PLMS), whereas Hospitals A and C (otolaryngology centers) primarily comprised patients with suspected OSA. Even among the latter, Hospital C’s cohort was more male-dominant (82.4%) and skewed toward severe OSA (66.4%), while Hospital A maintained a more balanced clinical spectrum. Supplementary Figs. 2–3 summarize these inter-hospital differences in PSG-derived surrogate markers, demographic characteristics, and OSA severity distributions.

Moreover, the three hospitals used distinct infrared camera models, room geometries, and mounting positions, resulting in natural variation in illumination, viewing distance, and angle. These differences create recording-domain shifts across clinical sites and enable a pragmatic cross-center evaluation. Accordingly, data from Hospitals B and C served as external test sets for evaluating performance under both clinical and recording-domain shifts.

RA: an alternative visual marker for OSA detection

The AHI is defined as the number of AH events per hour of sleep. In our video-based setting, SlAction detects RAs rather than AHs, and estimates sleep duration based on time spent in bed without explicit sleep staging. Fig. 3a illustrates the relationship between RA frequency and AHI. Instead of relying on the Respiratory Arousal Index (RAI), which requires precise sleep staging (unrealistic for video-based system), we employed a simplified metric: the number of RAs per hour spent in bed (RAs/hr). This measure exhibits a strong correlation (r=0.94) with AHI, consistently across OSA severity levels and sleep positions. This indicates the effectiveness of RA as a proxy for AHI estimation. Furthermore, Fig. 3b shows the distribution of RA and AH durations, indicating that RAs are substantially shorter than AHs (on average by a factor of 2.89, and up to 5.05 times shorter in extreme cases). This significant difference enables short-term, lightweight video analysis when detecting RA instead of AH.

Fig. 3 — Analyses were performed using data from Hospital A (n = 475 patients). a, Linear relationship between RAs per hour spent in bed (RAs/hr) and the Apnea-Hypopnea Index (AHI). Points represent individual patients; colors and marker shapes denote OSA severity and positional OSA status, respectively. Pearson’s r and Spearman’s ρ (95% confidence intervals) are shown. b, Distribution of RA and AH event durations. Histograms with overlaid density curves show that RA durations cluster around 10 s, whereas AH durations exhibit a broader, right-skewed distribution. c, Large Difference Pixels (LDPs) are defined as pixels whose intensity change between successive sampled frames exceeds a predefined motion threshold. The LDP count, i.e., the number of LDPs detected per frame, quantifies the spatial extent of motion. LDP heatmaps visualize characteristic motion patterns across sleep events, where brighter regions indicate a larger extent of detected motion. For each patient and event type, LDP counts were averaged within each event episode and then across all occurrences to obtain a patient-level mean. Violin plots show the distribution of these patient-level means across individuals (Normal n=475, Hypopnea n=474, Apnea Central n=105, Apnea Mixed n=244, Apnea Obstructive n=386, LM n=149, Wake n=475, PLMA n=130, SA n=446, RA n=473). Diamonds indicate medians and vertical bars indicate the interquartile range (25th--75th percentiles). No technical replicates were included. d, Motion characteristics for PLMA, SA, and RA. Representative viewing perspectives (top row; head at the top) are schematic visualizations derived from original recordings. Corresponding heatmaps show motion patterns during the 10 seconds before and after arousal onset. PLMA exhibits prominent limb motion, whereas RA shows more localized thoraco-abdominal motion relative to SA. e, Temporal evolution of normalized LDP intensity aligned to arousal onset. Solid lines indicate the mean across aligned events (RA: n=70,749; SA: n=21,955); shaded areas represent ± 1 standard deviation. The dashed vertical line marks arousal onset. RA demonstrates greater thoraco-abdominal motion intensity than SA before and during arousal, although variability overlaps.

Next, we assess the feasibility of RA detection from sleep video, which presents unique challenges, such as minimal subject movement, low-resolution infrared imaging, and frequent occlusion by blankets. To characterize sleep-related motion, we introduce Large Difference Pixels (LDPs), defined as pixel-level locations exhibiting significant frame-to-frame changes exceeding a predefined threshold. LDP-based analysis captures salient motion patterns while suppressing background noise. Figure 3c presents participant-level mean LDP counts and corresponding heatmaps for each event type. LDP counts reflect the spatial extent of movement, while the heatmaps illustrate characteristic motion intensity patterns obtained by averaging LDP maps across aligned event segments. Arousal events, defined as brief awakenings and classified into periodic limb movement arousal (PLMA), spontaneous arousal (SA), and RA, exhibit higher participant-level mean LDP counts compared to non-arousal respiratory events.

To distinguish these arousal subtypes, whose clinical classification often relies on preceding conditions, we analyzed LDP patterns within a 10-second window before and after each event (Fig. 3d). Specifically, PLMA follows periodic limb movements (PLM) and involves noticeable limb movements both before and during arousal. SA occurs spontaneously without clear triggers, showing moderate chest motion transitioning into uniform body motion. RA events, which follow AH events according to AASM guidelines, consistently exhibit stronger chest motion both before and during arousal, highlighting the potential of their video-based identification.

To further differentiate RA from SA, which are visually similar arousal subtypes in infrared video, we analyzed their temporal motion dynamics at the event level. Figure 3e presents normalized LDP values from 10 s before arousal onset through the arousal episode, with the arousal period rescaled to a common range (0-1) for alignment. Despite considerable overlap in LDP values between RA and SA events, temporal differences in motion distribution are observed in the corresponding heatmaps, with RA events exhibiting more pronounced thoraco-abdominal motion before and during arousal. These findings suggest that simple threshold-based approaches relying solely on LDP magnitude may be insufficient, whereas deep learning models incorporating temporal structure may better differentiate RA from SA, thereby supporting the feasibility of using RA as a practical marker for video-based AHI estimation.

SlAction design and on-device operation

We designed SlAction, a real-time, efficient video-based OSA detection system that leverages RA detection (Fig. 4). Considering the short duration of RAs (Fig. 3b), SlAction processes infrared sleep video in 30-second segments (epochs), the standard unit for sleep staging and event scoring in PSG. Each epoch is pre-processed (cropped and resized) and converted into a multi-frame input. Epochs containing any portion of RAs are labeled as RA to capture brief yet clinically significant arousals. Since over 70% RAs fit within one epoch, epoch-level RA detection significantly mitigates the risk of overcounting prolonged events.

The foundation of SlAction is the Event and Position (EP) detector, a lightweight deep neural network (DNN) based on MoViNet⁴², adapted for 30-second video segments (epochs). Unlike the only video-based prior method by Akbarian et al.³⁵, which executes two separate DNNs every 400 milliseconds on highly overlapping video segments to independently detect long AH events and sleep position, our EP detector analyzes each non-overlapping epoch only once, jointly classifying it into four categories: Supine/Non-RA, Non-Supine/Non-RA, Supine/RA, or Non-Supine/RA. This unified, epoch-level approach significantly streamlines processing, improving speed and memory efficiency.

As shown in Fig. 4b, SlAction runs approximately 48 times faster than the method by Akbarian et al. when evaluated on Jetson Orin Nano with GPU. Notably, SlAction uniquely achieves real-time performance even on the lower-spec Jetson Nano using CPU alone, processing each epoch in approximately 18 s. This rapid processing enables parallel pre-processing of the subsequent epoch for continuous inference. Our non-overlapping segment design permits immediate discarding of each analyzed epoch without impacting the accuracy of subsequent analyses. These results highlight SlAction ’s practicality for privacy-preserving on-device deployment.

By aggregating epoch-level results across the entire night, SlAction produces RA ratios for supine, non-supine, and overall conditions (the proportion of RA epochs to total epochs for each condition). To mitigate minor over-counting caused by RAs spanning multiple epochs, we employed a Softmax Score Aggregation strategy. This method counts isolated RA epochs individually, while grouping consecutive RA predictions by summing their Softmax scores into a single aggregated confidence score. By assigning lower scores to epochs containing fragmented RAs, this method effectively reduces over-counting of prolonged RAs.

Using these refined RA ratios, an AHI estimator separately calculates supine, non-supine and total AHIs via regression analysis to assess both OSA severity and positional dependency. While baseline methods typically use linear regression¹⁹, assuming a linear relationship, our analysis found that the correlation between RAs/hr and AHI (0.94) is slightly lower than that between RAI and AHI (0.99). To account for this deviation and better capture the upward trend at higher AHI values, we applied a third-order polynomial regression, which effectively fits a broader range of AHI values while maintaining simplicity.

Evaluation setup

We compared SlAction with five baselines spanning both video-based approaches and signal-based approaches derived from contact sensors. For video-based baselines, we included Akbarian et al.³⁵, the only prior method addressing both OSA detection and positional assessment. To assess the specific advantage of RA detection, we added SlAction -AH, a variant of SlAction that detects AHs instead. For signal-based baselines, we adopted DRIVEN¹⁹, a state-of-the-art home-use method for OSA severity classification, which detects AHs at the epoch level using thoracic and abdominal movements. To enable POSA detection, we incorporated an additional position classifier based on a gravity switch signal, and implemented two variants to compare RA and AH detection: DRIVEN-RA and DRIVEN-AH. To further evaluate the advantage of video over contact sensor signals for RA detection, we adapted ModernTCN⁴³, a state-of-the-art multivariate time-series model, to follow the same detection structure as SlAction, with physiological signals used in place of video input. For Hospital A (N=475), the data were explicitly divided into 335 training, 70 validation, and 70 test subjects using stratified sampling to preserve sex and OSA severity distributions. All model comparisons and quantitative analyses were performed on this held-out test set (N=70).

Event and position detection

We evaluated the performance of the EP detector against baseline models. As shown in Fig. 5a, SlAction demonstrates strong performance in both event and position classification tasks. For event detection, although signal-based models generally show slightly higher average performance, SlAction consistently ranks among the top-performers, showing the potential of non-contact, video-based sensors. Notably, SlAction achieves higher F1 score, AUROC, and AUPRC than ModernTCN in RA detection, which highlights video’s advantage to capture comprehensive motion patterns of RA beyond thoracoabdominal movement alone. Compared to video-based models, SlAction substantially outperforms the model by Akbarian et al., despite using a lighter architecture and less processing. The gain is likely due to its use of original video frames rather than optical flow, which preserves spatial detail and avoids noise amplification. SlAction also outperforms its variant, SlAction -AH, emphasizing the effectiveness of detecting RA over AH in video analysis. For position classification, video-based models generally surpass signal-based ones, with SlAction attaining the highest scores across all metrics, which shows the superiority of video for capturing position-related cues. Together, these findings confirm that SlAction effectively captures complex motion characteristics through its unified epoch-level design.

Figure 5b–e show classification results of SlAction over the entire sleep period for representative cases across all OSA severity levels. In the Event panel, the ground truth labels for AH and RA events reveal strong temporal alignment, consistent with our earlier finding that RA frequency closely correlates with AHI. SlAction successfully captures the temporal distribution of RA events across OSA severity levels, with predictions closely matching the ground truth. Most misclassifications occur outside the typical overlap between RA and AH, particularly during Wake stages. Further investigation is needed to identify additional causes of error, including possible confusion with AH, SA, or other event types. For position classification, SlAction accurately tracks supine and non-supine positions, even with frequent position changes. These results validate the EP detector’s robustness in estimating sleep position across varying conditions.

AHI estimation

We evaluated SlAction ’s AHI estimation using four metrics: Root Mean Square Error (RMSE), Mean Absolute Error (MAE), R-squared (R²) value, and Spearman correlation (Fig. 6a). SlAction achieves the best performance for all these metrics, demonstrating accurate and consistent AHI predictions. Its high Spearman correlation indicates that SlAction preserves the relative ranking of AHI severity across individuals, an important property for screening/triage to identify and prioritize candidates for confirmatory PSG and clinical evaluation. Notably, SlAction delivers superior AHI estimation compared to DRIVEN-AH despite the latter achieving stronger performance in epoch-level event detection (Fig. 5a). This result underscores the effectiveness of over-counting mitigation through epoch-wise detection of short-term RA and the use of a third-order polynomial regressor for AHI estimation.

The Bland–Altman plot for SlAction in Fig. 6b examines agreement between estimated and ground-truth AHI by plotting the estimation error (Estimated − GT) against the mean AHI. The mean bias and the 95% limits of agreement (LoA; mean bias ± 1.96 SD of the error) summarize the typical subject-level error range and provide an interpretable measure of agreement at the individual level. To facilitate comparison beyond a single method, we additionally provide Bland–Altman plots and a summary of LoA bounds/widths for all evaluated baselines on the same test set in Supplementary Section 3.1. SlAction yields the narrowest LoA width among the compared methods, suggesting the tightest typical error range under the same evaluation setting.

Figure 6c presents the relationship between estimated RA ratios and ground-truth AHI values for test samples, overlaid with the regression curve from the AHI estimator. The curve reveals a clear positive correlation between RA ratio estimates and ground-truth AHI values. Based on this curve, we mapped AHI thresholds to RA ratio thresholds for OSA severity classification. Most test samples fell within the region in which RA ratio-derived severity levels matched the ground-truth AHI categories. Figure 6d provides a more detailed view through a confusion matrix: SlAction correctly classifies most Severe cases (the majority class) and achieves reasonable accuracy for the remaining categories, with no misclassifications spanning more than one severity level. These results demonstrate that the AHI estimator effectively captures the RA-AHI relationship for reliable OSA severity classification.

OSA severity and POSA classification

Expanding on the previous results, Fig. 6e compares SlAction with baseline models across three key clinical tasks: OSA detection, OSA severity classification, and POSA identification. SlAction consistently achieves the highest performance across all tasks and evaluation metrics (accuracy, weighted F1 and macro F1), underscoring its robustness and clinical utility.

In severity classification, SlAction records the highest performance; as a supplementary analysis, it also achieves a relaxed accuracy (R-Acc.) of 100%, indicating that all misclassifications occur only between adjacent severity levels. Adjacent-category errors may still be clinically meaningful; R-Acc. is reported as a descriptor of near-threshold boundary sensitivity. OSA detection, which focuses on identifying patients with moderate-to-severe OSA (AHI ≥ 15), is particularly important for screening and triage. SlAction ’s strong performance in this task supports its use as a practical screening tool for early identification and referral for confirmatory PSG and clinical evaluation.

SlAction ’s best performance in POSA detection further emphasizes its clinical value in enabling pose-targeted treatment strategies. While Akbarian et al.’s video-based model detects position well, its inaccurate AH detection results in the worst overall performance. In contrast, SlAction combines tailored RA detection with reliable position recognition for accurate POSA detection. This synergy highlights the advantage of SlAction ’s unified framework, which maximizes the strengths of video data for holistic, non-contact OSA assessment. For further analysis of position-specific AHI estimation accuracy, including Bland-Altman validation for supine and non-supine positions, see Supplementary Section 3.2.

Evaluation on external datasets

We assessed the generalization capability of SlAction using external datasets from Hospitals B and C. Both datasets exhibit imbalanced OSA severity distributions, with particularly few Normal and Mild cases (Supplementary Fig.3c, g). Environmental differences, including blanket thickness, camera specifications, and variations in shooting angle and distance, further contribute to domain shifts. To address these challenges, we applied two adaptation strategies: fine-tuning the AHI estimator and EP detector using hospital-specific data. For Hospital C, 256 cases were divided into training (76), validation (70), and test (110) sets using stratified sampling to preserve severity and sex distributions. This nonstandard split reflects a small-scale fine-tuning experiment designed to emulate a realistic domain adaptation scenario, where only limited local data are available for model adjustment. For Hospital B, we used the models trained on Hospital A without fine-tuning due to concerns about the reliability of its RA labels (see NOTE below Supplementary Fig. 3); all 205 cases were used exclusively as a test set for AHI estimation and OSA severity classification. Hospital B’s RA/RAI annotations were not used as ground truth for RA-level validation.

Figure 7a presents performance for AHI estimation and severity classification in Hospital B. Despite differences in camera angle and bed placement compared to Hospital A (Fig. 2b), SlAction performs strongly in both tasks, outperforming all baselines. Hospital C presents additional challenges due to its lower-body-focused camera angle that limits visibility of the head and torso, and thicker blankets that further obscure body contours and motions (Fig. 2b). These conditions can complicate both event and position detection from video (not the contact sensor data). Without EP detector fine-tuning (Fig. 7b), SlAction ’s performance is slightly degraded relative to Hospital A, but still better than all baselines in most metrics. After fine-tuning (Fig. 7c), its performance significantly improves across all metrics, with R-Acc. of 100%, higher than all baselines. For POSA detection (Fig. 7d), SlAction delivers the most consistent performance across all hospitals. In Hospital B, it outperforms all baselines. While some signal-based models outperform SlAction in Hospital C, they underperform in Hospitals A and B, indicating a lack of robustness.

Fig. 7 — a,b Comparison of AHI estimation and OSA severity classification performance in Hospital B (n = 205) (a) and Hospital C (n = 110) (b). While signal-based methods showed relatively strong generalization across hospital domains, *SlAction* still achieved superior or competitive performance. c AHI estimation performance evaluated on the test set from Hospital C after fine-tuning the EP detector with a subset of Hospital C data. Fine-tuning significantly improved both AHI estimation and R-Acc. performance. d Bar plot of MF1 for POSA classification across hospitals. *SlAction* shows relatively stable and competitive performance across hospitals. However, in Hospital C, environmental factors, such as camera angle and blanket thickness seem to have constrained performance, particularly in distinguishing POSA cases compared to some signal-based models.

External validation results underscore SlAction ’s robustness (Hospital B) and its ability to adapt to substantial domain shifts through fine-tuning on limited hospital-specific data (Hospital C). Its superior performance compared to the video-based baselines indicates the advantage of detecting RAs with pronounced motions as a more robust and video-friendly strategy than detecting the subtler motions of AH events. Furthermore, SlAction’s consistent outperformance of signal-based models demonstrates the potential of non-contact video sensing as a scalable and versatile alternative for OSA monitoring in various environments.

Subgroup analysis

To assess model robustness and fairness, we evaluated SlAction across demographic and clinical subgroups (see Supplementary Section 4 for full results). Across age and sex groups in the Hospital A test set, performance remained stable, with minimal variation in AHI estimation accuracy and OSA severity classification (Supplementary Figs. 15, 16). The model achieved its highest consistency in middle-aged adults and maintained comparable accuracy between males and females (macro-F1: 61.3 vs. 60.3), confirming SlAction’s age- and sex-independent reliability.

We further analyzed clinical subgroups defined using PSG-report–derived surrogate markers for periodic limb movements during sleep (PLMS; PLMI ≥ 5) and a REM-related surrogate (RSWA reported; suspected-RBD) (Supplementary Fig. 17). PLMS-positive cases showed reduced correlation and OSA-severity macro-F1 in Hospitals A and B, consistent with limb-movement–related motion confounding that can mimic arousal-like cues. Notably, PLMS-positive patients also displayed higher positional OSA prevalence (Supplementary Fig. 17e). In contrast, the RSWA-based surrogate subgroup showed minimal additional performance change. Overall, these results suggest that SlAction shows largely consistent performance across subgroups, while highlighting PLMS as an important source of motion-related confounding that warrants attention in real-world deployments.

Discussion

Expanding sleep medicine from hospital settings to the home represents a critical shift for modern healthcare, particularly as a growing portion of the population suffers from sleep disorders that can underlie numerous physical and mental health problems. To be clinically useful and widely adopted, home-based sleep monitoring must simultaneously achieve user comfort, privacy protection, and reliable screening-level performance that supports timely triage and referral for confirmatory PSG. In this study, we proposed non-contact, video-based sleep monitoring as a practical and scalable solution, founded on a central hypothesis: involuntary, whole-body motions during sleep encode rich physiological information about sleep disorders, which can be decoded by deep learning. To validate this hypothesis, we presented several key contributions: (1) the creation of KVSS, the largest and most diverse multi-institutional sleep video dataset to date; (2) an empirical motion analysis uncovering clinically relevant patterns across sleep-related events; (3) the identification of RA as an effective video-salient marker for respiratory disturbance; (4) the design of SlAction, an on-device, real-time deep learning model for AHI estimation and POSA detection; and (5) an extensive evaluation of SlAction across multiple institutions and deployment on resource-constrained devices, benchmarking its performance against a wide array of state-of-the-art baselines.

The KVSS dataset stands as a significant resource for further research, providing broader scientific value beyond SlAction ’s development and evaluation. Its extensive scale, multimodal nature, and diversity make it particularly valuable for exploring multimodal learning, unveiling correlations between sleep motions and biosignals, characterizing sleep disorders, and understanding variations related to demographics, such as age and BMI. Furthermore, KVSS incorporates a real-world privacy constraint, namely masked facial regions, which requires models to operate without key visual cues from the upper airway and facial regions. SlAction ’s strong performance under these challenging conditions supports the practicality of privacy-preserving, video-based screening and highlights the potential of KVSS to advance ethically aligned AI development in sleep medicine.

The sleep motion analysis using the KVSS dataset (Fig. 3c) revealed distinct, clinically relevant motion signatures related to various sleep events, even in respiratory events typically characterized by subtle motion. For example, Normal breathing exhibited minimal chest movements, consistent with stable respiration. Hypopnea involved slightly stronger chest movement, reflecting increased respiratory effort. Obstructive apnea presented the most pronounced motion, with intense chest and head movements in response to airway obstruction. Central apnea, which arises from a lack of respiratory drive, was generally characterized by minimal motion, occasionally exhibiting abrupt chest movements near the episode’s end due to sudden breath intake. Certain variants, such as Cheyne-Stokes respiration and cluster breathing, showed periodic or crescendo-decrescendo motion patterns reflecting cyclic fluctuations in respiratory effort. Nevertheless, the overall motion magnitude in these central events remained substantially lower than that observed in obstructive apnea or hypopnea, consistent with the absence of sustained respiratory effort. Mixed apnea combined these features, beginning with stillness and transitioning into active respiratory effort. These motion profiles validate the capability of video-based methods to capture physiologically meaningful patterns traditionally monitored via contact sensors, highlighting their potential as either complementary or alternative screening/assessment tools.

A significant finding was the effectiveness of RA as a reliable surrogate marker for AHI estimation, well suited to video-based monitoring. Unlike AH events, which often span prolonged and variable durations, RA events are temporally well defined and typically occur within one or two 30-second epochs. Detecting AH events with sufficient continuity in video requires complex long-term temporal modeling and is susceptible to fragmentation, leading to over-counting and degraded AHI estimation. In contrast, RA represents a compensatory physiological response that restores normal breathing after AH, and is characterized by clear upper-body movements that are more visually salient (Fig. 3e). SlAction leverages these distinctive temporal and visual properties by employing a non-overlapping, epoch-level segmentation strategy combined with Softmax-based aggregation, resulting in robust detection and counting of RAs. This RA-driven framework enables accurate, efficient, and privacy-preserving on-device AHI estimation, reinforcing the practicality of video-based sleep screening and assessment.

Importantly, the advantage of RA over AH detection was modality-specific. In signal-based models, such as DRIVEN that uses thoracic and abdominal belts, AH detection outperformed RA detection. This discrepancy arises because contact sensors excel at capturing localized, detailed motion patterns associated with AH events, whereas video analysis more effectively captures broader, whole-body motion cues of RA events. These findings underscore the importance of selecting event markers that align with the inherent strengths of each sensing modality.

However, this modality-specific advantage also introduces a practical limitation: because SlAction relies primarily on whole-body motion cues characteristic of RA, it cannot fully distinguish other events that exhibit similar biomechanical patterns. As shown in Fig. 5b–e, a moderate proportion of AH events without EEG-defined RA were nonetheless identified as RA by the model. Across the test set, approximately 38% of such non-arousal AH events were detected as RA, particularly frequent in obstructive apneas (71.5%) compared to hypopneas (20.3%), likely reflecting the large, abrupt thoraco-abdominal excursions accompanying airway reopening after complete obstruction. Indeed, analysis of detection errors confirmed that AH events without subsequent RA constituted the largest source of misclassification (34%). This finding indicates that the model often responds to the biomechanical manifestation of respiratory effort and recovery, even when cortical arousal is absent, capturing physiologically meaningful motion rather than spurious noise. Wake periods (13.4%), often involving large body shifts or postural adjustments, also contributed to false positives, while confusion with SA accounted for 11.3% of errors. Some discrepancies may also reflect the inherent variability of the PSG-derived ground truth: RA scoring requires detecting cortical arousals lasting at least three seconds, yet EEG transitions are often subtle or borderline, even for experienced scorers. Thus, some model-detected RAs may correspond to genuine physiological arousals that were not labeled during manual scoring.

An additional design consideration concerns how wakefulness is handled in AHI computation. While wake periods can occasionally contribute to false positives, their broader impact relates to the choice of denominator in AHI calculation. Specifically, we compute AHI using total time in bed (TIB) rather than total sleep time (TST), a pragmatic design that avoids reliance on explicit sleep and wake labeling, which remains challenging in video-only settings. However, prolonged wakefulness may still affect AHI estimation, particularly in patients with comorbid insomnia and sleep apnea (COMISA). To examine this factor, we conducted supplementary analyses (Supplementary Section 3.3) comparing the TIB-based formulation with wake-aware variants. In the present evaluation, explicitly incorporating wake detection increased sensitivity to cross-center domain shifts in the cascaded AHI pipeline, whereas the TIB-based formulation showed more stable performance across centers. In addition, we did not observe a clear association between AHI estimation error and PSG-derived sleep-quality proxies (sleep latency and sleep efficiency) in our cohort. This may be partly attributable to the fact that the current wake classifier achieved an F1 score of 66.0; errors in wake detection (used to estimate TST) can propagate to the final AHI in the wake-aware pipeline. Improved wake modeling (e.g., with richer temporal context⁴⁴ or multimodal cues) may therefore strengthen wake-aware AHI computation in future work.

Beyond wakefulness, a detailed residual analysis was conducted to characterize systematic biases in AHI estimation across the severity spectrum; Supplementary Section 3.4 and Fig. 6b summarize these results. Residuals followed a convex trend, indicating overestimation in the low-AHI range (0–40) and underestimation in high-AHI cases ( > 60), consistent with regression-to-the-mean effects. This bias likely stems from false positives of non-respiratory motion in mild cases and from missed detections in dense, overlapping events among severe cases. Clinically, this yields a conservative tendency to slightly overestimate AHI near diagnostic cut-offs while preserving discrimination for moderate-to-severe OSA, which may be desirable in screening workflows that prioritize identifying candidates for confirmatory PSG. At the same time, in low-AHI individuals, especially in lower-prevalence community or home settings, such overestimation can reduce specificity and increase unnecessary referrals; therefore, low-AHI results should be interpreted with caution. Future work will expand the representation of low-AHI cases and evaluate threshold calibration and robustness in less controlled home environments.

These residual trends also affect OSA severity classification because categorical labels (Normal, Mild, Moderate, Severe) are defined by AHI thresholds: small errors near cut-offs can propagate into adjacent categories, most notably between Normal and Mild where cases cluster around AHI ≈ 5. To reflect this boundary sensitivity, we additionally report relaxed accuracy (R-Acc.) as a supplementary descriptor indicating whether misclassifications are predominantly confined to adjacent categories. We emphasize that R-Acc. does not imply adjacent misclassifications are clinically negligible; rather, it highlights the discretization effect when continuous AHI estimates are mapped to categorical labels. As shown in Fig. 6, methods with higher AHI estimation fidelity (e.g., SlAction and DRIVEN¹⁹) also achieve higher R-Acc., consistent with the role of accurate AHI estimation in severity risk stratification. Nevertheless, broader representation of low-AHI cases remains important to further improve calibration and enhance specificity without compromising sensitivity.

In addition to AHI estimation, understanding the positional dependency of respiratory disturbances offers additional clinical insight, as body position strongly modulates the severity and pattern of obstructive events. In this regard, SlAction also demonstrated reliable detection of positional OSA (POSA), a clinically important subtype. We observed that both video- and signal-based models (SlAction, SlAction -AH, and ModernTCN) benefited from a unified four-class formulation that jointly predicts event and position labels. This unified structure facilitates the learning of shared motion cues: supine RAs exhibited strong, localized chest movements, whereas non-supine RAs displayed more diffuse upper-body movements. By leveraging these shared characteristics, the unified model improved position classification, enabling robust POSA detection. Ablation analyses comparing this unified four-class formulation with a dual-head (multi-task) variant further confirmed its superior cross-site robustness (Supplementary Section 2.2).

Although SlAction showed encouraging cross-institution performance overall, domain differences remained a practical challenge, particularly for sleep position detection. Evaluation on Hospital C revealed decreased positional accuracy under variations, such as camera angles and blanket thickness, which may obscure body-shape and posture cues used for position recognition. However, fine-tuning on Hospital C substantially improved SlAction ’s performance, bringing it closer to the results on Hospital A and demonstrating adaptability to the new environment. In contrast, ModernTCN, a signal-based baseline, still showed reduced positional accuracy on Hospital C even after fine-tuning, suggesting that factors beyond visual variability (e.g., cohort characteristics or class imbalance) may also contribute to the observed drop. In addition, data imbalance in Hospital C, where non-supine cases were underrepresented, may have further weakened discrimination of RA vs. non-RA in the non-supine condition. Given the intermixed nature of domain shifts, exploring self-adaptation strategies using unlabeled video data in target deployment settings is a promising direction for future work and should be validated empirically.

Despite demonstrating adaptability across clinical centers, the present evaluation remains confined to in-hospital recordings. Although these controlled settings provided synchronized PSG references essential for reliable model development, they do not capture the full variability of real-world home environments, where lighting changes, background clutter, and the presence of a bed partner may introduce additional challenges. To move toward practical deployment, SlAction can incorporate design choices intended to improve robustness, including an optional object-detection module for subject localization in multi-person or cluttered scenes (Supplementary Section 2.3) and extensive augmentation strategies to simulate device and illumination variation. Nevertheless, the current results should be interpreted as demonstrating technical feasibility rather than proven efficacy for home monitoring. A large-scale prospective study on real home recordings will be an important next step to validate performance under unconstrained conditions, including broader variability in camera placement and scene context, and to evaluate operating characteristics in lower-prevalence community cohorts (e.g., false-positive burden and predictive values). Furthermore, because recordings with inadequate video quality were excluded in this study, future deployments should also report the operational failure/rejection rate when faced with low-quality or atypical recordings.

While this study focused on AHI estimation and positional OSA assessment, the same framework could serve as a foundation for broader video-based characterization of sleep-disordered breathing. Recent work in sleep medicine has emphasized the need for physiologically richer markers, such as ventilatory burden, event duration, and recovery instability-that better capture disease severity and treatment response^45–47. Infrared video offers a promising avenue for exploring these parameters through interpretable motion dynamics, and extending SlAction ’s capability to include sleep staging may further enhance robustness by providing contextual information on sleep quality and facilitating the identification of distinct sleep disorder endotypes. Together, these extensions could broaden SlAction ’s clinical applicability and pave the way for more personalized and comprehensive sleep medicine solutions. Finally, as the KVSS dataset is limited to Korean clinical cohort, validation across diverse ethnicities and pediatric populations remains an important future step.

Methods

This study was conducted in accordance with relevant ethical regulations and was approved by the Institutional Review Boards of Chungnam National University Hospital (IRB approval no. 2021-04-048), Catholic University of Korea St. Vincent’s Hospital (IRB approval no. VC21RNDI0062), and Hallym University Hospital (IRB approval no. 021-04-016). The requirement for written informed consent was waived by the Institutional Review Boards of the participating institutions in accordance with applicable regulations, including Article 16(3) of the Bioethics and Safety Act of the Republic of Korea. The waiver was granted because this retrospective study involved minimal risk and used fully anonymized data.

KVSS dataset

The KVSS dataset was retrospectively constructed from three hospitals: (A) Chungnam National University Hospital, (B) Catholic University of Korea St. Vincent’s Hospital, and (C) Hallym University Hospital. The final cohort included 936 individuals (713 males and 223 females; mean age ± SD: 51.3 ± 14.6 years) with synchronized polysomnography (PSG), infrared video recordings, and expert annotations. Sex was recorded from electronic medical records. Institution-specific demographic distributions are provided in Fig. 2 and Supplementary Fig. 3.

Participants were retrospectively identified from individuals undergoing clinically indicated overnight PSG examinations for suspected sleep disorders. As a hospital-based cohort, the study population may reflect referral bias and may not represent the general population. No participants received compensation, as this study used pre-existing clinical data collected during routine care. A detailed STARD-compliant overview of data collection, screening, and validation procedures is provided in Supplementary Section 1.1.

Data collection and recording

PSG-video recordings were obtained during routine clinical studies for various sleep-related indications, including suspected OSA, insomnia, periodic limb movement disorder (PLMD), and REM sleep behavior disorder (RBD). Sleep videos were recorded in parallel with PSG to ensure precise temporal alignment. Video acquisition systems varied slightly across institutions, reflecting real-world differences in camera placement and illumination conditions. All recordings were standardized to MP4 format at 640 × 480 resolution and 5 fps. Video frames were subsequently extracted and synchronized with PSG signals using custom Python (v3.10) scripts and open-source libraries including OpenCV (v4.8), NumPy (v1.24), and Pandas (v2.0).

The intended sample size was based on data availability. All eligible PSG examinations with synchronized infrared video collected during the study period were screened according to predefined inclusion and exclusion criteria. As this was a retrospective study, no formal a priori sample-size calculation was performed; the final sample size reflects all available high-quality recordings meeting these criteria. No adverse events related to the video-based index test were expected or observed, as PSG was performed as part of routine clinical care.

PSG acquisition and annotation

PSG recordings were obtained using Embla (Natus Medical, San Carlos, CA, USA) and Nox (Nox Medical, Reykjavik, Iceland) systems, and stored in European Data Format (EDF). All recordings were scored according to the American Academy of Sleep Medicine (AASM 2012) criteria. Annotations encompassed standard AASM-defined events, including sleep stages (Wake, N1, N2, N3, REM), respiratory events (Obstructive Apnea, Hypopnea), movement events (Limb Movement, Periodic Limb Movement), and arousals (Spontaneous, Respiratory Arousal), among others. Scoring was performed by certified sleep technologists and subsequently reviewed by board-certified sleep physicians, following standard clinical workflows at each institution.

Given the retrospective and multi-center nature of the dataset, a full re-scoring of all cases was not feasible. Therefore, all PSG-video pairs underwent cross-checking and quality-control review by at least two sleep experts to verify temporal alignment and resolve apparent inconsistencies. This process preserved the original clinical scorings while addressing apparent discrepancies. The resulting dataset reflects routine inter-center variability under standard clinical practice. We therefore report cross-center results with this variability in mind and present them as a pragmatic evaluation under heterogeneous clinical scoring and recording conditions, rather than as a substitute for controlled re-scoring.

Standardization and de-identification

To comply with the Guidelines for the Use of Healthcare Data, all video recordings underwent a structured de-identification process prior to research use. A custom object-detection-based anonymization pipeline was developed to identify and mask facial features, tattoos, and other identifiable elements across frames. The facial region was manually initialized in the first frame to ensure accurate tracking, after which regions were automatically tracked and irreversibly blurred using a Gaussian filter to prevent visual identification during research processing.

To ensure both privacy compliance and temporal integrity, eight medical professionals independently reviewed each case to verify adequate anonymization and accurate synchronization with the corresponding PSG signals. Only recordings that passed this verification were included in the final dataset. No patient-derived infrared images are included in the manuscript. Figures depicting recording configurations are schematic illustrations and do not contain participant imagery.

Data handling and privacy

The handling of video data differs between the research and deployment phases. During the research phase, videos were de-identified and securely stored for algorithm development and validation. Facial features and other identifiable elements were masked, and the resulting de-identified recordings were retained on encrypted institutional servers solely for research under IRB-approved protocols. These de-identified recordings were used solely for model development and evaluation and are not accessible during deployment. The KVSS dataset is not publicly available due to ethical and privacy restrictions; access is provided only through a controlled access mechanism subject to institutional review and approval.

In contrast, when the trained SlAction system is deployed for real-world use, video streams are processed locally on-device in 30-second segments and immediately converted into numerical outputs (e.g., AHI, severity, positional indices). After inference, each segment is automatically deleted from device memory, and no raw video data are stored or transmitted. This ensures that the deployed system performs continuous, contact-free monitoring without retaining any identifiable information, fully safeguarding patient privacy in real-world operation.

Preliminary analysis

Large difference pixels (LDPs)

To quantify motion intensity in infrared video data, we introduce Large Difference Pixels (LDPs) as a simple yet effective motion descriptor. LDPs correspond to pixels exhibiting substantial frame-to-frame intensity changes exceeding a predefined threshold, thereby representing regions of notable motion. Infrared videos recorded at 5 fps were converted to grayscale, and every third frame (0.6 s interval) was sampled to compute absolute pixel-wise intensity differences between consecutive sampled frames. The motion threshold was empirically determined from static background regions without subject motion. Specifically, it was set to the minimum intensity difference value above which fewer than 1% of background pixels exhibited nonzero frame-to-frame changes. This criterion suppressed background sensor noise while preserving motion signals originating from the subject. Because all recordings analyzed in this study were acquired using identical infrared camera systems at Hospital A, a single fixed threshold value was consistently applied across subjects to ensure comparability.

Although this temporal sampling may not capture very subtle sub-second movements, it effectively represents respiratory and arousal-related motions characterized by multi-second dynamics. Lower thresholds increase sensitivity to minor movements but may introduce background noise, whereas higher thresholds isolate only pronounced motion. In our experiments, the threshold primarily affected the absolute LDP count but did not alter the qualitative distribution or spatial structure of motion in the resulting heatmaps. For event-level analysis, LDP counts were computed for each sampled frame within annotated event segments. The LDP count was defined as the number of LDP pixels detected per sampled frame. For each participant and event type, LDP counts were first averaged within each event episode and then averaged across all occurrences of that event type within the participant to obtain a participant-level mean value used in subsequent statistical analyzes.

SlAction

Object detection

To enhance robustness in real-world environments with variable camera placement or the presence of additional individuals or animals (e.g., a bed partner, caregiver, or pet), an optional object detection module was implemented to localize the primary subject within each frame. A convolutional neural network-based detector, You Only Look Once (YOLOv8; Ultralytics⁴⁸), was trained to detect a single class (person) and generate bounding boxes covering the entire body region. The training dataset was constructed from infrared sleep videos collected at the three participating centers. Still frames were sampled at 10-minute intervals from each participant and manually annotated in YOLO format. Bounding boxes were drawn with sufficient margins to include peripheral limb movements. To improve robustness, an iterative active-learning strategy was employed. An initial model trained on manually labeled data generated pseudo-labels for additional frames, which were manually reviewed and corrected. This prediction-correction-retraining cycle was repeated multiple times. The final training set comprised 3,833 annotated images, with 855 non-overlapping frames reserved for independent evaluation. Performance was assessed using standard object detection metrics, including mean Average Precision (mAP) and mean Intersection over Union (mIoU) (see Supplementary Section 2.3).

Video preprocessing

To prepare input for the EP detector, which is implemented as a deep neural network (DNN), the original MP4-format sleep videos (5 fps) were converted from RGB to grayscale. Each frame was cropped to emphasize the bed region by trimming excess background and resized to 224 × 224 pixels. The cropping region was defined manually once per recording configuration, as camera and bed positions remained fixed throughout acquisition. To ensure consistent subject coverage, every 20th epoch from each recording was visually inspected to confirm that the subject remained fully within the cropped area. Minor variations across recordings were verified to fall within the range of spatial jitter and scaling augmentations applied during training.

Because this controlled setting involved a single subject per frame and a stable camera setup, automated object detection was unnecessary during model development. However, as described in Supplementary Section 2.3, an object detection module can be integrated for real-world deployments under more variable conditions. Following PSG annotation standards, which define a 30-second window as one epoch, videos were segmented accordingly, yielding clips of shape (150, 224, 224). This ensured precise temporal alignment with PSG data and a consistent input size for model training.

RA labeling

Respiratory arousal (RA) events, originally scored as continuous events with defined onset and duration, were mapped onto the 30-second epochs to generate binary RA labels. Because individual RA events may span epoch boundaries or occur near transitions, we adopted an inclusive labeling rule: any epoch containing even a partial RA segment was labeled as RA-positive. This approach preserves event continuity and mitigates truncation effects at epoch boundaries. Epochs without any overlap with RA annotations were labeled as non-RA. This labeling procedure was uniformly applied across all training and evaluation datasets to maintain consistency.

EP detector

The EP detector was formulated as a four-class classification model to jointly predict respiratory events (RA vs. Non-RA) and sleep position (Supine vs. Non-Supine), as illustrated in Fig. 4. Each epoch was assigned one of four labels: Supine/Non-RA, Non-Supine/Non-RA, Supine/RA, or Non-Supine/RA. The Non-Supine category included lateral, prone, and upright positions based on PSG annotations. When a position change occurred within a 30-second window, the pre-change position was used as the epoch label to maintain alignment with PSG scoring. Epochs containing any portion of an RA event were labeled RA to preserve event context. Binary predictions were derived by aggregating class probabilities. The RA score was computed as the sum of Supine/RA and Non-Supine/RA probabilities, while the Supine score was obtained from Supine/RA and Supine/Non-RA probabilities.

MoviNet-A2⁴², a lightweight video recognition architecture (4M parameters), was adopted as the backbone and modified to output four classes. To enhance motion representation while reducing temporal redundancy, three consecutive grayscale frames were grouped into pseudo-RGB channels, transforming the input shape from (1, 150, 224, 224) to (3, 50, 224, 224). This restructuring effectively captured motion-based features via frame differencing, while improving efficiency compared to conventional RGB input.

The model was initialized with Kinetics-600⁴⁹ pre-trained weights and trained for 20 epochs using the AdamW optimizer with an initial learning rate of 0.001 and batch size 64. A progressive unfreezing strategy was applied, and balanced sampling mitigated class imbalance between RA and Non-RA epochs. Data augmentations included random horizontal flipping, rotation ( ± 45^∘), brightness and contrast jittering, random resized cropping, and Mixup⁵⁰ (α = 1) to improve generalization. Further architectural details and ablation analyses are provided in Supplementary Sections 2.1 and 2.2.

AHI estimator

We introduced the RA ratio, defined as the proportion of RA-labeled epochs relative to the total number of recorded epochs:

RA Ratio = \frac{RA epochs}{Total epochs}

Because RA events often span multiple consecutive epochs, naively counting each RA-positive epoch may overestimate their true occurrence. To mitigate this, a Softmax score aggregation strategy was applied. For each epoch, RA confidence was computed by summing the Softmax probabilities of the Supine/RA and Non-Supine/RA classes, consolidating evidence across positional contexts. For consecutive RA-positive epochs, their Softmax scores were aggregated and treated as a single RA event. This strategy reduces the effective contribution of prolonged RA episodes spanning multiple epochs while preserving sensitivity to brief, clinically significant events. AHI was estimated using a third-order polynomial regression model fitted to the RA ratio. While the relationship between RA ratio and AHI was approximately linear (Spearman ρ = 0.94), the polynomial model better captured subtle non-linearities, particularly at higher AHI values, improving prediction accuracy over standard linear regression.

Positional OSA

To identify positional OSA (POSA), position-specific AHI values were estimated by stratifying predicted RA scores according to sleep position. Supine AHI was computed as the ratio of RA-positive epochs among all epochs predicted as Supine, and Non-Supine AHI as the ratio of RA-positive epochs among all epochs predicted as non-supine. A subject was classified as having POSA if Supine AHI was at least twice Non-Supine AHI, following the conventional diagnostic criterion for positional OSA.

Implementation details

All data analysis, model development, and evaluation were conducted in Python (v3.11) on Ubuntu 20.04 LTS using PyTorch (v2.3.0) and Torchvision (v0.18.0) with CUDA (v11.8). Training was implemented with PyTorch Lightning (v2.4.0), and performance metrics were computed using TorchMetrics (v1.4.1). Data processing and statistical analyses were performed using NumPy (v2.0.1), Pandas (v2.2.2), SciPy (v1.14.1), and scikit-learn (v1.5.2). Visualization was conducted using Matplotlib (v3.9.2) and Seaborn (v0.13.2). Subject localization was implemented using YOLOv8 (Ultralytics), with training annotations created using LabelImg and Label Studio Community Edition (open-source).

Baselines

To ensure a fair comparison, all baseline models were reimplemented and evaluated under the same data splits, preprocessing pipeline, and evaluation protocol as SlAction. Unless otherwise specified, baseline models were trained using the same computational environment described above. Hyperparameters were tuned to ensure stable convergence and competitive performance on our dataset while maintaining the original architectural design.

Akbarian et al

This video-based baseline comprises two components: (1) AH event detection for AHI estimation and OSA severity classification, and (2) sleep position classification for positional OSA (POSA) determination.

AH events were detected using a 3D convolutional neural network (3D CNN) trained on optical flow, which captures pixel displacement between consecutive frames. Prior to optical flow extraction, videos were downsampled to 2.5 fps and cropped to 256 × 256 pixels to approximate the upper-body focus used in the original study. Whereas the original setup used 640 × 480 resolution at a 1.5 m camera distance, our recordings were captured from a greater distance (approximately 3–3.5 m), requiring tighter cropping to maintain a comparable region of interest. Optical flow was computed along both x and y directions using the DenseFlow toolkit⁵¹. Each input consisted of a sequence of 23 optical flow frames (approximately 10 s), represented as (23, 2, 256, 256), where 2 denotes the two directional flow components. A segment was labeled as AH only if all 23 frames corresponded to an AH event in the ground truth; otherwise, it was labeled as normal.

The model architecture followed the 3D-CNN structure proposed by Akbarian et al., comprising convolutional layers with batch normalization, dropout, and pooling. During training, data augmentation included random cropping to 224 × 224, horizontal flipping, and normalization to [0, 1]. To address class imbalance, normal segments were sampled every 37 frames (15 s), while AH segments were sampled every 12 frames (5 s). The model was trained using Adam (learning rate 0.001, batch size 128) for 50 epochs. A cosine annealing learning rate schedule was applied, and early stopping was triggered if the validation loss did not improve for 15 consecutive epochs. Binary cross-entropy (BCE) was used as the loss function. During inference, event detection was performed using a sliding window at 2.5 fps (i.e., every 0.4 s).

Sleep position classification was performed on grayscale infrared frames to determine whether the subject was in a supine or non-supine position. The architecture was based on Darknet-19⁵², originally designed for head and body position detection. Each input was a single grayscale frame, and the model output a probability of supine position. Training incorporated data augmentation (random cropping with scale 0.7–1.0, horizontal flipping, brightness/contrast adjustment) and normalization using precomputed mean and variance. To mitigate class imbalance, supine samples were undersampled. The model was trained using SGD (momentum 0.9, learning rate 0.001; batch size 64) for 50 epochs. The learning rate was reduced by a factor of 0.5 if the validation loss did not improve for three consecutive epochs, with a minimum threshold enforced. BCE was used as the loss function, and early stopping was applied after 10 epochs without validation improvement. During inference, position classification was performed frame-by-frame at 2.5 fps.

DRIVEN-RA and DRIVEN-AH

The original DRIVEN framework utilized multiple respiratory-related signals. To ensure a fair comparison with SlAction, we excluded signals not observable in video data (e.g., airflow and oxygen saturation) and retained only thoracic movement, abdominal movement, and position signals, which are visually identifiable. Two variants were implemented: DRIVEN-RA for RA event detection and DRIVEN-AH for AH event detection. Because AH detection is the conventional basis for AHI computation in PSG analysis, DRIVEN-AH serves as a reference signal-based estimator, whereas DRIVEN-RA evaluates whether RA-based estimation is comparably effective in the signal domain. The model comprises a CNN backbone based on EfficientNetV2S and a LightGBM classifier for prediction refinement. The CNN processes 30-second signal segments sampled at 64 Hz. Separate CNNs were trained for each signal type. The CNN predicts both RA/AH event probability and sleep position. To enhance predictive performance, a two-stage architecture was adopted: the final three layers were removed to extract intermediate feature embeddings, which were concatenated and fed into a LightGBM classifier. For RA/AH detection, embeddings from thoracic and abdominal signals (1280 dimensions each) were concatenated to form a 2560-dimensional feature vector. For sleep position classification, a 1280-dimensional embedding from the position signal was used.

The CNN was trained using Adam (learning rate 0.001, batch size 64) for 100 epochs with early stopping (patience 20 epochs) and learning rate decay (factor 0.5 after 5 stagnant epochs, minimum 1 × 10⁻⁶). Categorical cross-entropy was used as the loss function. The LightGBM classifier was configured with a learning rate of 0.05, 31 leaves per tree, feature and bagging fractions of 0.8, and unlimited tree depth (max depth = -1). During inference, PSG signals were processed in 30-second segments, generating probability scores for RA/AH events and sleep position.

ModernTCN

We implemented a signal-based baseline using the same four-class classification formulation as SlAction to compare video-based and signal-based approaches under a unified prediction setting. Instead of video input, thoracic movement, abdominal movement, and position signals were used, as these are also visually observable in sleep videos. To model multivariate time-series signals, we employed ModernTCN⁴³, an extension of the Temporal Convolutional Network (TCN) that incorporates Reparameterized Large Kernel Convolution (RLKConv) to capture long-range temporal dependencies. The architecture was adapted to perform four-class classification using the selected signals as input.

Signals were sampled at 64 Hz and segmented into 30-second windows, yielding inputs of $X \in R^{3 \times 1920}$ , where 3 denotes the signal channels and 1920 corresponds to the number of time points per segment. Inputs were patchified along the temporal dimension to enhance feature representation. The architecture consisted of two downsampling stages, each containing two ModernTCN blocks. Each block comprised a depthwise convolution (DWConv) followed by ConvFFN1 and ConvFFN2. The large and small kernel sizes were set to 23 and 11, respectively, with feature dimensions of 32 and 64 and a downsampling ratio of 2. The final prediction was generated by a fully connected layer producing a four-class probability vector P(Y) = [p₁, p₂, p₃, p₄], corresponding to Supine/RA, Non-Supine/RA, Supine/Non-RA, and Non-Supine/Non-RA. The model was trained using Adam (initial learning rate 0.01) with cross-entropy loss. The patch stride was set to 1 to enable fine-grained temporal modeling.

Evaluation metrics

Model performance was evaluated using standard classification and regression metrics. For multi-class OSA severity classification, we report macro-averaged F1 (MF1) and weighted F1 (WF1) scores. MF1 is defined as the unweighted mean of per-class F1 scores, assigning equal importance to all severity categories (Normal, Mild, Moderate, Severe). WF1 computes the mean F1 score weighted by class prevalence, reflecting overall performance under class imbalance. MF1 therefore emphasizes balanced performance across categories, whereas WF1 captures prevalence-adjusted performance. For AHI estimation, we report root mean squared error (RMSE), mean absolute error (MAE), coefficient of determination (R²), and Spearman correlation coefficient (ρ) between predicted and reference AHI values. For positional OSA (POSA), treated as a binary classification task, we report accuracy, MF1, and WF1 scores.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Supplementary information

Supplementary Information^{(3.4MB, pdf)}

Reporting Summary^{(1.6MB, pdf)}

Transparent Peer Review file^{(3.3MB, pdf)}

Source data

Source Data^{(2.3MB, zip)}

Acknowledgements

This work was supported by the National Research Foundation (NRF) of Korea funded by the Korea government (MSIT) (RS-2023-00212780 to H.-S.K.; RS-2023-00222663 to H.-S.K.; RS-2025-25407378 to Y.R.C.), by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (RS-2025-25463302 to H.-S.K.), by the AI-Bio Research Grant through Seoul National University (to H.-S.K. and H.-W.S.), by the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI) funded by the Ministry of Health and Welfare, Republic of Korea (RS-2024-00439991 to H.-W.S.), by the SNUH Research Fund (grant no. 0320222200 to H.-W.S.), and by the data construction project for artificial intelligence through the National Information Society Agency (NIA) funded by the Ministry of Science and ICT (to H.-W.S.).

Author contributions

Y.R.C. contributed to the research design, algorithm and model architecture development, led the experimental analysis and interpretation of results, and co-led the manuscript writing. D.P. developed the model architecture, implemented the code, and conducted model training. H.K.L. performed practical implementation and coordination of the infrared sleep data acquisition, contributed to dataset organization and quality control, and, together with S.B., reviewed and implemented baseline models and conducted baseline experiments. S.P. co-led the manuscript writing. D.-K.K., Y.M.K., and C.-S.P. participated in and supervised data acquisition at their respective hospitals and provided clinically validated infrared sleep datasets for this study. H.-W.S. conceived the overall study concept, designed and initiated the infrared sleep dataset construction framework across institutions, supervised data collection, and oversaw the entire project. H.-S.K. co-supervised the project, assisted with study design, and provided critical feedback on the manuscript.

Peer review

Peer review information

Nature Communications thanks Matteo Moro and the other, anonymous, reviewers for their contribution to the peer review of this work. [A peer review file is available].

Data availability

The Korea Video Sleep Study (KVSS) dataset that supports the findings of this study cannot be made publicly available because it contains sensitive clinical sleep-video and polysomnography data subject to legal restrictions under the Personal Information Protection Act of the Republic of Korea. De-identified KVSS data have been deposited with the Korean National Information Society Agency (NIA) AI Hub and may be requested through its controlled-access procedures for legitimate, noncommercial academic research purposes, including post-publication replication and verification of the findings reported in this study. Data-access inquiries should be sent to the corresponding author and will be forwarded to the relevant authority for review under these controlled-access procedures. Access requires completion of applicable data-use and security agreements and may require institutional review board approval, where applicable. Administrative inquiries to the corresponding author will be acknowledged within two weeks. Decisions on data-access requests are made by the NIA AI Hub according to its standard review procedures, typically within several weeks of formal application, although timelines may vary depending on the nature of the request and any required approvals. Source data are provided with this paper.

Code availability

The code necessary to reproduce the findings of this study, including model training and evaluation scripts, is publicly available at https://github.com/yourimchoi/slaction.

Competing interests

H.-W.S. is listed as an inventor on a patent application related to respiratory state monitoring and sleep disorder assessment (PCT/KR2021/005672; US application 17/930,569, filed by Seoul National University R&DB Foundation). The application has been examined and is no longer being pursued. H.-W. S. is a founder of OUaR LaB, Inc., serves on its Board of Directors and as Chief Executive Officer, and owns stock in OUaR LaB, Inc., which is subject to certain restrictions under university policy. D.-K.K owns stock in OUaR LaB, Inc. The remaining authors declare no competing interests.

Footnotes

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Hyung-Sin Kim, Email: hyungkim@snu.ac.kr.

Hyun-Woo Shin, Email: charlie@snu.ac.kr.

Supplementary information

The online version contains supplementary material available at 10.1038/s41467-026-73915-2.

References

1.Benjafield, A. V. et al. Estimation of the global prevalence and burden of obstructive sleep apnoea: a literature-based analysis. Lancet Respir. Med.7, 687–698 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Franklin, K. A. & Lindberg, E. Obstructive sleep apnea is a common disorder in the population-a review on the epidemiology of sleep apnea. J. Thorac. Dis.7, 1311 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
3.Senaratna, C. V. et al. Prevalence of obstructive sleep apnea in the general population: a systematic review. Sleep. Med. Rev.34, 70–81 (2017). [DOI] [PubMed] [Google Scholar]
4.McNicholas, W. T. Diagnosis of obstructive sleep apnea in adults. Proc. Am. Thorac. Soc.5, 154–160 (2008). [DOI] [PubMed] [Google Scholar]
5.Reuveni, H. et al. Awareness level of obstructive sleep apnea syndrome during routine unstructured interviews of a standardized patient by primary care physicians. Sleep27, 1518–1524 (2004). [DOI] [PubMed] [Google Scholar]
6.Peppard, P. E. et al. Increased prevalence of sleep-disordered breathing in adults. Am. J. Epidemiol.177, 1006–1014 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Chen, X. et al. Racial/ethnic differences in sleep disturbances: the multi-ethnic study of atherosclerosis (MESA). Sleep38, 877–888 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Gottlieb, D. J. & Punjabi, N. M. Diagnosis and management of obstructive sleep apnea: a review. Jama323, 1389–1400 (2020). [DOI] [PubMed] [Google Scholar]
9.Huttunen, R. et al. Assessment of obstructive sleep apnea-related sleep fragmentation utilizing deep learning-based sleep staging from photoplethysmography. Sleep44, zsab142 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Frost, S. & Sullivan, D. Hidden health crisis costing america billions. Underdiagnosing and undertreating obstructive sleep apnea draining healthcare system. Am. Acad. Sleep Med. Darien19, 1175–1181 (2016).
11.Rundo, J. V. & Downey, R. Chapter 25 - polysomnography. In Levin, K. H. & Chauvel, P. (eds.) Clinical Neurophysiology: Basis and Technical Aspects, vol. 160 of Handbook of Clinical Neurology, 381–392, 10.1016/B978-0-444-64032-1.00025-4 (Elsevier, 2019). [DOI] [PubMed]
12.of the American Academy of Sleep Medicine, A. O. S. A. T. F. Clinical guideline for the evaluation, management and long-term care of obstructive sleep apnea in adults. J. Clin. Sleep. Med.5, 263–276 (2009). [PMC free article] [PubMed] [Google Scholar]
13.Henry, O. et al. A model for sleep apnea management in underserved patient populations. J. Prim. Care Community Health13, 21501319211068969 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
14.Kukwa, W., Migacz, E., Lis, T. & Ishman, S. L. The effect of in-lab polysomnography and home sleep polygraphy on sleep position. Sleep. Breath.25, 251–255 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Roeder, M. et al. Night-to-night variability of respiratory events in obstructive sleep apnoea: a systematic review and meta-analysis. Thorax75, 1095–1102 (2020). [DOI] [PubMed] [Google Scholar]
16.Lechat, B. et al. Multinight prevalence, variability, and diagnostic misclassification of obstructive sleep apnea. Am. J. Respir. Crit. Care Med.205, 563–569 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Massie, F., Van Pee, B. & Bergmann, J. Correlations between home sleep apnea tests and polysomnography outcomes do not fully reflect the diagnostic accuracy of these tests. J. Clin. Sleep. Med.18, 871–876 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Levy, J., Álvarez, D., Del Campo, F. & Behar, J. A. Deep learning for obstructive sleep apnea diagnosis based on single channel oximetry. Nat. Commun.14, 4881 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Retamales, G. et al. Towards automatic home-based sleep apnea estimation using deep learning. npj Digit. Med.7, 144 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Cartwright, R. D. Effect of sleep position on sleep apnea severity. Sleep7, 110–114 (1984). [DOI] [PubMed] [Google Scholar]
21.Hayano, J., Yuda, E., Sakakibara, M. & Yamada, A. Detection of sleep apnea events using wrist-worn inertial sensors: a prospective validation study. Sci. Rep.14, 54817 (2024). [Google Scholar]
22.Holfinger, S., Chung, F., Owens, R. & Strohl, K. P. Validation of fingertip wearable oximetry devices for sleep apnea detection in a clinical cohort. Sleep48, zsaf026 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
23.Gaiduk, M. & Sazonov, E. Wearable and nearable technologies for sleep apnea detection: a systematic review. Sleep. Med. Rev.68, 101738 (2023). [Google Scholar]
24.Narayan, S., Mohanty, S., Subramanian, S. & Raghavan, S. Detection of sleep apnea from smartphone-recorded ambient sound: a validation study. J. Clin. Sleep. Med.14, 1713–1720 (2018). [Google Scholar]
25.Kim, H. J., Lee, J. H., Lee, S. Y. & Park, C. H. Acoustic analysis of breathing sounds for the severity assessment of sleep-disordered breathing. Biomed. Eng. Online17, 44 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Le, T., Nguyen, T. D., Kim, J. & Lee, S. Real-time sleep apnea event detection using acoustic signals in home-like environments. J. Med. Internet Res.25, e44818 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Snigdha, F., Islam, S. M., Boric-Lubecke, O. & Lubecke, V. Obstructive sleep apnea (OSA) events classification by effective radar cross section (ercs) method using microwave doppler radar and machine learning classifier. In 2020 IEEE MTT-S International Microwave Biomedical Conference (IMBioC), 1–3 (IEEE, 2020).
28.Koda, T. et al. Noncontact detection of sleep apnea using radar and expectation-maximization algorithm. IEEE Sens. J.24, 32748–32756 (2024). [Google Scholar]
29.Zhang, H. et al. Event-level identification of sleep apnea using FMCW radar. Bioengineering12, 399 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
30.Islam, M. M. & Lee, K. Limitations of radar-based vital sign monitoring: Distance sensitivity, null points, and orientation constraints. IEEE Sens. J.22, 13072–13088 (2022). [Google Scholar]
31.Randerath, W. et al. European respiratory society guideline on non-cpap therapies for obstructive sleep apnoea. Eur. Respir. Rev.30, 210200 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Lim, S., Lee, H.-K., Kang, Y. J. & Shin, H.-W. Differential effects of sleep position and sleep stage on the severity of obstructive sleep apnea. J. Sleep. Res.18, e14379 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
33.Ahn, H. K., Kang, Y. J., Yoon, W. & Shin, H.-W. Analysing the impact of body position shift on sleep architecture and stage transition: A comprehensive multidimensional study using event-synchronised polysomnography data. J. Sleep. Res.33, e14115 (2024). [DOI] [PubMed] [Google Scholar]
34.On, S.-W. et al. Clinical efficacy of a position-responding mandibular advancement device in patients with obstructive sleep apnea. Clin. Exp. Otorhinolaryngol.17, 302–309 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]
35.Akbarian, S., Ghahjaverestan, N. M., Yadollahi, A. & Taati, B. Noncontact sleep monitoring with infrared video data to estimate sleep apnea severity and distinguish between positional and nonpositional sleep apnea: Model development and experimental validation. J. Med. Internet Res.23, e26524 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Berry, R. B. et al. Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events: deliberations of the sleep apnea definitions task force of the American Academy of Sleep Medicine. J. Clin. sleep. Med.8, 597–619 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Mansukhani, M. P., Kara, T., Caples, S. M. & Somers, V. K. Chemoreflexes, sleep apnea, and sympathetic dysregulation. Curr. Hypertens. Rep.16, 1–12 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Eckert, D. J. & Malhotra, A. Pathophysiology of adult obstructive sleep apnea. Proc. Am. Thorac. Soc.5, 144–153 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Younes, M. Role of arousals in the pathogenesis of obstructive sleep apnea. Am. J. Respir. Crit. Care Med.169, 623–633 (2004). [DOI] [PubMed] [Google Scholar]
40.Osman, A. M., Carter, S. G., Carberry, J. C. & Eckert, D. J. Obstructive sleep apnea: current perspectives. Nat. Sci. Sleep10, 21–34 (2018). [DOI] [PMC free article] [PubMed]
41.Xiao, S.-C. et al. Neural respiratory drive and arousal in patients with obstructive sleep apnea hypopnea. Sleep38, 941–949 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
42.Kondratyuk, D. et al. Movinets: Mobile video networks for efficient video recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16020–16030 (IEEE, 2021).
43.Luo, D. & Wang, X. Moderntcn: a modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations 1–43 (ICLR, 2024).
44.Lee, H. et al. Explainable vision transformer for automatic visual sleep staging on multimodal PSG signals. npj Digit. Med.8, 55 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]
45.Malhotra, A. et al. Metrics of sleep apnea severity: beyond the apnea-hypopnea index. Sleep44, zsab030 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]
46.Parekh, A. et al. Ventilatory burden as a measure of obstructive sleep apnea severity is predictive of cardiovascular and all-cause mortality. Am. J. Respir. Crit. Care Med.208, 1216–1226 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]
47.Butler, M. P. et al. Apnea–hypopnea event duration predicts mortality in men and women in the sleep heart health study. Am. J. Respir. Crit. Care Med.199, 903–912 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Jocher, G., Chaurasia, A. & Qiu, J. Ultralytics yolov8 (2023).
49.Carreira, J., Noland, E., Banki-Horvath, A., Hillier, C. & Zisserman, A. A short note about kinetics-600. arXiv preprint arXiv:1808.01340 (2018).
50.Zhang, H., Cisse, M., Dauphin, Y. N. & Lopez-Paz, D. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations (ICLR, 2018).
51.Wang, S. et al. denseflow. https://github.com/open-mmlab/denseflow (2020).
52.Akbarian, S., Delfi, G., Zhu, K., Yadollahi, A. & Taati, B. Automated non-contact detection of head and body positions during sleep. IEEE Access7, 72826–72834 (2019). [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Information^{(3.4MB, pdf)}

Reporting Summary^{(1.6MB, pdf)}

Transparent Peer Review file^{(3.3MB, pdf)}

Source Data^{(2.3MB, zip)}

Data Availability Statement

The code necessary to reproduce the findings of this study, including model training and evaluation scripts, is publicly available at https://github.com/yourimchoi/slaction.

[CR1] 1.Benjafield, A. V. et al. Estimation of the global prevalence and burden of obstructive sleep apnoea: a literature-based analysis. Lancet Respir. Med.7, 687–698 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR2] 2.Franklin, K. A. & Lindberg, E. Obstructive sleep apnea is a common disorder in the population-a review on the epidemiology of sleep apnea. J. Thorac. Dis.7, 1311 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR3] 3.Senaratna, C. V. et al. Prevalence of obstructive sleep apnea in the general population: a systematic review. Sleep. Med. Rev.34, 70–81 (2017). [DOI] [PubMed] [Google Scholar]

[CR4] 4.McNicholas, W. T. Diagnosis of obstructive sleep apnea in adults. Proc. Am. Thorac. Soc.5, 154–160 (2008). [DOI] [PubMed] [Google Scholar]

[CR5] 5.Reuveni, H. et al. Awareness level of obstructive sleep apnea syndrome during routine unstructured interviews of a standardized patient by primary care physicians. Sleep27, 1518–1524 (2004). [DOI] [PubMed] [Google Scholar]

[CR6] 6.Peppard, P. E. et al. Increased prevalence of sleep-disordered breathing in adults. Am. J. Epidemiol.177, 1006–1014 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR7] 7.Chen, X. et al. Racial/ethnic differences in sleep disturbances: the multi-ethnic study of atherosclerosis (MESA). Sleep38, 877–888 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR8] 8.Gottlieb, D. J. & Punjabi, N. M. Diagnosis and management of obstructive sleep apnea: a review. Jama323, 1389–1400 (2020). [DOI] [PubMed] [Google Scholar]

[CR9] 9.Huttunen, R. et al. Assessment of obstructive sleep apnea-related sleep fragmentation utilizing deep learning-based sleep staging from photoplethysmography. Sleep44, zsab142 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR10] 10.Frost, S. & Sullivan, D. Hidden health crisis costing america billions. Underdiagnosing and undertreating obstructive sleep apnea draining healthcare system. Am. Acad. Sleep Med. Darien19, 1175–1181 (2016).

[CR11] 11.Rundo, J. V. & Downey, R. Chapter 25 - polysomnography. In Levin, K. H. & Chauvel, P. (eds.) Clinical Neurophysiology: Basis and Technical Aspects, vol. 160 of Handbook of Clinical Neurology, 381–392, 10.1016/B978-0-444-64032-1.00025-4 (Elsevier, 2019). [DOI] [PubMed]

[CR12] 12.of the American Academy of Sleep Medicine, A. O. S. A. T. F. Clinical guideline for the evaluation, management and long-term care of obstructive sleep apnea in adults. J. Clin. Sleep. Med.5, 263–276 (2009). [PMC free article] [PubMed] [Google Scholar]

[CR13] 13.Henry, O. et al. A model for sleep apnea management in underserved patient populations. J. Prim. Care Community Health13, 21501319211068969 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR14] 14.Kukwa, W., Migacz, E., Lis, T. & Ishman, S. L. The effect of in-lab polysomnography and home sleep polygraphy on sleep position. Sleep. Breath.25, 251–255 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR15] 15.Roeder, M. et al. Night-to-night variability of respiratory events in obstructive sleep apnoea: a systematic review and meta-analysis. Thorax75, 1095–1102 (2020). [DOI] [PubMed] [Google Scholar]

[CR16] 16.Lechat, B. et al. Multinight prevalence, variability, and diagnostic misclassification of obstructive sleep apnea. Am. J. Respir. Crit. Care Med.205, 563–569 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR17] 17.Massie, F., Van Pee, B. & Bergmann, J. Correlations between home sleep apnea tests and polysomnography outcomes do not fully reflect the diagnostic accuracy of these tests. J. Clin. Sleep. Med.18, 871–876 (2022). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR18] 18.Levy, J., Álvarez, D., Del Campo, F. & Behar, J. A. Deep learning for obstructive sleep apnea diagnosis based on single channel oximetry. Nat. Commun.14, 4881 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR19] 19.Retamales, G. et al. Towards automatic home-based sleep apnea estimation using deep learning. npj Digit. Med.7, 144 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR20] 20.Cartwright, R. D. Effect of sleep position on sleep apnea severity. Sleep7, 110–114 (1984). [DOI] [PubMed] [Google Scholar]

[CR21] 21.Hayano, J., Yuda, E., Sakakibara, M. & Yamada, A. Detection of sleep apnea events using wrist-worn inertial sensors: a prospective validation study. Sci. Rep.14, 54817 (2024). [Google Scholar]

[CR22] 22.Holfinger, S., Chung, F., Owens, R. & Strohl, K. P. Validation of fingertip wearable oximetry devices for sleep apnea detection in a clinical cohort. Sleep48, zsaf026 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR23] 23.Gaiduk, M. & Sazonov, E. Wearable and nearable technologies for sleep apnea detection: a systematic review. Sleep. Med. Rev.68, 101738 (2023). [Google Scholar]

[CR24] 24.Narayan, S., Mohanty, S., Subramanian, S. & Raghavan, S. Detection of sleep apnea from smartphone-recorded ambient sound: a validation study. J. Clin. Sleep. Med.14, 1713–1720 (2018). [Google Scholar]

[CR25] 25.Kim, H. J., Lee, J. H., Lee, S. Y. & Park, C. H. Acoustic analysis of breathing sounds for the severity assessment of sleep-disordered breathing. Biomed. Eng. Online17, 44 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR26] 26.Le, T., Nguyen, T. D., Kim, J. & Lee, S. Real-time sleep apnea event detection using acoustic signals in home-like environments. J. Med. Internet Res.25, e44818 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR27] 27.Snigdha, F., Islam, S. M., Boric-Lubecke, O. & Lubecke, V. Obstructive sleep apnea (OSA) events classification by effective radar cross section (ercs) method using microwave doppler radar and machine learning classifier. In 2020 IEEE MTT-S International Microwave Biomedical Conference (IMBioC), 1–3 (IEEE, 2020).

[CR28] 28.Koda, T. et al. Noncontact detection of sleep apnea using radar and expectation-maximization algorithm. IEEE Sens. J.24, 32748–32756 (2024). [Google Scholar]

[CR29] 29.Zhang, H. et al. Event-level identification of sleep apnea using FMCW radar. Bioengineering12, 399 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR30] 30.Islam, M. M. & Lee, K. Limitations of radar-based vital sign monitoring: Distance sensitivity, null points, and orientation constraints. IEEE Sens. J.22, 13072–13088 (2022). [Google Scholar]

[CR31] 31.Randerath, W. et al. European respiratory society guideline on non-cpap therapies for obstructive sleep apnoea. Eur. Respir. Rev.30, 210200 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR32] 32.Lim, S., Lee, H.-K., Kang, Y. J. & Shin, H.-W. Differential effects of sleep position and sleep stage on the severity of obstructive sleep apnea. J. Sleep. Res.18, e14379 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR33] 33.Ahn, H. K., Kang, Y. J., Yoon, W. & Shin, H.-W. Analysing the impact of body position shift on sleep architecture and stage transition: A comprehensive multidimensional study using event-synchronised polysomnography data. J. Sleep. Res.33, e14115 (2024). [DOI] [PubMed] [Google Scholar]

[CR34] 34.On, S.-W. et al. Clinical efficacy of a position-responding mandibular advancement device in patients with obstructive sleep apnea. Clin. Exp. Otorhinolaryngol.17, 302–309 (2024). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR35] 35.Akbarian, S., Ghahjaverestan, N. M., Yadollahi, A. & Taati, B. Noncontact sleep monitoring with infrared video data to estimate sleep apnea severity and distinguish between positional and nonpositional sleep apnea: Model development and experimental validation. J. Med. Internet Res.23, e26524 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR36] 36.Berry, R. B. et al. Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events: deliberations of the sleep apnea definitions task force of the American Academy of Sleep Medicine. J. Clin. sleep. Med.8, 597–619 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR37] 37.Mansukhani, M. P., Kara, T., Caples, S. M. & Somers, V. K. Chemoreflexes, sleep apnea, and sympathetic dysregulation. Curr. Hypertens. Rep.16, 1–12 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR38] 38.Eckert, D. J. & Malhotra, A. Pathophysiology of adult obstructive sleep apnea. Proc. Am. Thorac. Soc.5, 144–153 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR39] 39.Younes, M. Role of arousals in the pathogenesis of obstructive sleep apnea. Am. J. Respir. Crit. Care Med.169, 623–633 (2004). [DOI] [PubMed] [Google Scholar]

[CR40] 40.Osman, A. M., Carter, S. G., Carberry, J. C. & Eckert, D. J. Obstructive sleep apnea: current perspectives. Nat. Sci. Sleep10, 21–34 (2018). [DOI] [PMC free article] [PubMed]

[CR41] 41.Xiao, S.-C. et al. Neural respiratory drive and arousal in patients with obstructive sleep apnea hypopnea. Sleep38, 941–949 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR42] 42.Kondratyuk, D. et al. Movinets: Mobile video networks for efficient video recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16020–16030 (IEEE, 2021).

[CR43] 43.Luo, D. & Wang, X. Moderntcn: a modern pure convolution structure for general time series analysis. In The Twelfth International Conference on Learning Representations 1–43 (ICLR, 2024).

[CR44] 44.Lee, H. et al. Explainable vision transformer for automatic visual sleep staging on multimodal PSG signals. npj Digit. Med.8, 55 (2025). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR45] 45.Malhotra, A. et al. Metrics of sleep apnea severity: beyond the apnea-hypopnea index. Sleep44, zsab030 (2021). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR46] 46.Parekh, A. et al. Ventilatory burden as a measure of obstructive sleep apnea severity is predictive of cardiovascular and all-cause mortality. Am. J. Respir. Crit. Care Med.208, 1216–1226 (2023). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR47] 47.Butler, M. P. et al. Apnea–hypopnea event duration predicts mortality in men and women in the sleep heart health study. Am. J. Respir. Crit. Care Med.199, 903–912 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]

[CR48] 48.Jocher, G., Chaurasia, A. & Qiu, J. Ultralytics yolov8 (2023).

[CR49] 49.Carreira, J., Noland, E., Banki-Horvath, A., Hillier, C. & Zisserman, A. A short note about kinetics-600. arXiv preprint arXiv:1808.01340 (2018).

[CR50] 50.Zhang, H., Cisse, M., Dauphin, Y. N. & Lopez-Paz, D. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations (ICLR, 2018).

[CR51] 51.Wang, S. et al. denseflow. https://github.com/open-mmlab/denseflow (2020).

[CR52] 52.Akbarian, S., Delfi, G., Zhu, K., Yadollahi, A. & Taati, B. Automated non-contact detection of head and body positions during sleep. IEEE Access7, 72826–72834 (2019). [Google Scholar]

PERMALINK

Non-contact on-device detection of obstructive sleep apnea from infrared video

You Rim Choi

Dongik Park

Hyun Kyung Lee

Suahn Bae

Sehyun Park

Dong-Kyu Kim

Yong Min Kim

Chan-Soon Park

Hyung-Sin Kim

Hyun-Woo Shin

Abstract

Introduction

Fig. 1. Application scenario of SlAction for continuous OSA monitoring.

Results

KVSS : a diverse sleep video dataset

Fig. 2. Korea Video Sleep Study (KVSS): a multi-institutional infrared sleep video dataset.

RA: an alternative visual marker for OSA detection

Fig. 3. Preliminary analysis of motion patterns underlying sleep events and respiratory arousals (RAs).

SlAction design and on-device operation

Fig. 4. Overview of the SlAction framework.

Evaluation setup

Event and position detection

Fig. 5. Performance of event and position detection.

AHI estimation

Fig. 6. Performance of AHI Estimation and Positional OSA Detection.

OSA severity and POSA classification

Evaluation on external datasets

Fig. 7. Performance across different hospitals.

Subgroup analysis

Discussion

Methods

KVSS dataset

Data collection and recording

PSG acquisition and annotation

Standardization and de-identification

Data handling and privacy

Preliminary analysis

Large difference pixels (LDPs)

SlAction

Object detection

Video preprocessing

RA labeling

EP detector

AHI estimator

Positional OSA

Implementation details

Baselines

Akbarian et al

DRIVEN-RA and DRIVEN-AH

ModernTCN

Evaluation metrics

Reporting summary

Supplementary information

Source data

Acknowledgements

Author contributions

Peer review

Peer review information

Data availability

Code availability

Competing interests

Footnotes

Contributor Information

Supplementary information

References

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases