An Evaluation of the NightOwl Home Sleep Apnea Testing System

Abstract

Study Objectives:

The objective of this study was to evaluate the performance of a miniaturized home sleep apnea test, called NightOwl. The system consists of a sensor placed on the fingertip and a cloud-based analytics software. The sensor acquires accelerometer and photoplethysmographic data. The software derives actigraphy from the former, and blood oxygen saturation and peripheral arterial tone, among other features, from the latter.

Methods:

Data of 101 participants who underwent an in-laboratory polysomnography (PSG), while wearing the NightOwl sensor, were collected. In order to establish an external benchmark, all PSG tests were edited by a somnologist of Younes Medical Technologies Ltd. (YMT) after analysis by the Michele Sleep Scoring System (MSSS). The respiratory event index (REI) derived by NightOwl (NightOwl-REI), the apnea-hypopnea index (AHI) derived by Ziekenhuis Oost-Limburg (ZOL-AHI), and the AHI derived by YMT (MSSS-AHI) were compared.

Results:

The NightOwl-REI had a high correlation with the MSSS-AHI (ρ = .87, P < .001), which was close to the correlation between the ZOL-AHI and MSSS-AHI (ρ = .84, P < .001). The NightOwl-REI and ZOL-AHI had a correlation of .77 (P < .001). After categorization of the AHI, the agreement between the NightOwl-REI and the MSSS-AHI was .812 and the agreement between the ZOL-AHI and MSSS-AHI was .743, after double-labeling near-boundary participants.

Conclusions:

The NightOwl-REI achieved a close correlation and REI-categorization with the MSSS-AHI, especially in light of the significant inter-scorer variability of the analysis of the PSG.

Citation:

Massie F, Mendes de Almeida D, Dreesen P, Thijs I, Vranken J, Klerkx S. An evaluation of the NightOwl home sleep apnea testing system. J Clin Sleep Med. 2018;14(10):1791–1796.

INTRODUCTION

In March 2017, the clinical practice guideline for diagnostic testing for adult sleep apnea by the American Academy of Sleep Medicine (AASM) for the first time formulated a strong recommendation that both polysomnography (PSG) and home sleep apnea testing (HSAT) are appropriate diagnostic testing options for uncomplicated adult patients who are at increased risk of moderate to severe sleep apnea.¹

Collop et al.² performed a comprehensive analysis of the evidence for HSAT devices to diagnose obstructive sleep apnea (OSA) in out-of-center settings. The authors concluded that testing devices that analyze changes in peripheral arterial tone (PAT) in combination with actigraphy and blood oxygen saturation (SpO₂) are adequate to diagnose OSA in patient populations with a high pretest probability.

In this paper, we propose a new system for the diagnosis of OSA that measures and analyzes the abovementioned parameters, called NightOwl (Ectosense NV, Leuven, Belgium). The system consists of a small sensor device which is placed on the fingertip, the NightOwl sensor, and a cloud-based analytics platform, the NightOwl software. An illustration of the NightOwl sensor can be found in Figure 1. It is designed to be self-applied and initiated by the patient by attaching the sensor to the fingertip by means of an adhesive patch. The sensor's battery can last for approximately three nights before requiring a recharge. The sensor acquires accelerometer data and reflectance-based photoplethysmography (PPG) signals. The software derives actigraphy from the former, and SpO₂, PAT, and pulse rate, among other features, from the latter. As such, the NightOwl system is able to derive all diagnostic parameters recommended by The AASM Manual for the Scoring of Sleep and Associated Events for home sleep apnea testing utilizing peripheral arterial tonometry.³ In order to assess NightOwl's performance, we compared the respiratory event index (REI), defined as the number of respiratory events per hour of sleep, derived by the NightOwl system, to the apnea-hypopnea index (AHI) obtained from manual analysis of the PSG. We also compared the total sleep time (TST) derived by both systems. This study was performed in a sleep laboratory environment. Replicability of the study results in the home environment is subject to future investigation.

Figure 1. An illustration of the NightOwl sensor.

This figure represents an illustration of the NightOwl sensor placed on the index finger with an adhesive patch.

METHODS

This study was approved by the Committee of Medical Ethics of Ziekenhuis Oost-Limburg (ZOL), Belgium (CME ZOL, reference: 17/034U) and was conducted in accordance with the Declaration of Helsinki. Data of 101 patients who underwent a diagnostic in-hospital PSG in the sleep laboratory of ZOL was successfully acquired. For each participant, a NightOwl sensor was placed on the middle finger. A measurement was considered successful if at least 4 hours of data was acquired, as recommended by the AASM.¹

Analysis of the Polysomnography

All PSG tests were analyzed by somnologists of ZOL using AASM criteria with the recommended 1A rule for the scoring of hypopneas.^3,4 In order to establish an external benchmark to which the NightOwl and ZOL somnologists' analysis could be compared, the PSG data was transferred to Younes Medical Technologies Ltd. (YMT), the corporation behind the Michele Sleep Scoring System (MSSS) developed by Dr. M. Younes. After analysis of each PSG using the MSSS, they were edited by the YMT somnologist. Malhotra et al.⁵ confirmed that the MSSS, complemented with manual editing, standardizes PSG scoring results within and across sleep centers. Using such external benchmark, the impact of inter-scorer variability of current and future reports on NightOwl's performance metrics is reduced.

NightOwl

The NightOwl sensor acquires accelerometry and PPG from which it derives actigraphy, SpO₂, PAT and pulse rate, among other features. Based on the simultaneous analysis of PPG-derived physiological events, such as PAT, respiratory effort correlates, and SpO₂, the NightOwl software derives the REI as well as the TST as main clinical parameters.

Theory Behind PAT Analysis

Changes in caliber of arteries elicited by alterations in the contractile activity of vascular smooth muscle from their basal level are referred to as changes in arterial tone.⁶ The end-state of respiratory events is very often associated with cortical or autonomic arousals, which in turn are associated with sympathetic activation events.⁶ These sympathetic activation events cause vasoconstriction of the digital artery, which constitutes a change in peripheral arterial tone. This will be reflected by a decrease in the amplitude of the PPG signal, as a decreased vascular caliber causes a decreased perfusion of the peripheral tissue.⁷

Data Analysis

Statistical analysis was performed using MATLAB. When computing the intraclass correlation coefficients, the intraclass correlation coefficient type 3,1 (ICC_3,1) variant was chosen as it correctly assumes each patient's recording is assessed by each scorer, the scorers are the only scorers of interest, and reliability is calculated from a ground truth measurement.⁸

The agreement between the AHI severity categories⁹ was expressed by means of a boundary corrected error matrix. Despite the use of scoring guidelines, the manual scoring of the PSG is not perfectly replicable as evidenced by the phenomenon of inter-scorer variability.^5,10 From the inter-scorer variability statistics reported by Malhotra et al.⁵ one can derive that for a manually derived AHI by a single scorer, the 50% confidence interval around that AHI is at least 13.4%. We applied this confidence interval around the MSSS-AHI to identify borderline cases: whenever this interval spanned multiple AHI severity categories, the patient's ground truth AHI was considered a borderline case and that patient received two possible ground truth labels. In cases where two ground truth labels were attributed, an agreement in the error matrix was counted whenever the NightOwl REI category matched either of the two ground truth AHI labels.

PSG and NightOwl data were algorithmically synchronized by matching the instantaneous heart rate traces derived from the ECG trace of the PSG and the PPG trace of the NightOwl. Data epochs that were of too low quality to be interpreted by the sleep technician or the NighOwl algorithm were rejected from the PSG and the NightOwl traces.

RESULTS

A cohort of 101 participants were included, of which 56% were male. Demographic and clinical results show a mean age of 53 years (standard deviation [SD] 13), a mean body mass index of 28.8 kg/m² ± (SD 4.9), and a mean AHI of 26.87 events/h (SD 20.87).

REI Estimation

In what follows, we refer to the AHI determined by the manual analysis of the PSG data by the ZOL somnologist as the ZOLAHI and the YMT somnologist (after analysis by the MSSS) as the MSSS-AHI. The REI derived by NightOwl showed a correlation of .87 (P < .001) and ICC_3,1 of .86 with the MSSSAHI (Figure 2). These results are similar to the correlation of .84 and ICC_3,1 of .84 between the ZOL-AHI and MSSSAHI. The correlation and ICC_3,1 between the NightOwl-REI and ZOL-AHI were .77 and .76 respectively. A Bland-Altman plot illustrating the difference between the MSSS-AHI and respectively the ZOL-AHI and NightOwl-REI as a function of the MSSS-AHI, is shown in Figure 3. The mean difference between the NightOwl-REI and MSSS-AHI was 0.33 events/h and 1.48 events/h for the ZOL-AHI and the MSSS-AHI. The SD of these differences were 20.03 (NightOwl-REI versus MSSS-AHI) and 22.24 (ZOL-AHI versus MSSS-AHI). We can observe that NightOwl sometimes overscores the AHI of patients in the mild (5 to < 15) category. Furthermore, we identify underscoring for AHI above 50. In this study we observed that patients with very severe sleep apnea more often have apneic episodes of long duration accompanied with brisk movements at the event's end stage, resulting in more movement artefacts in the PPG signal of these patients. Low signal quality epochs during which an apneic event manifests itself might sometimes be incorrectly discarded by the NightOwl software. Another contributing factor to the observed standard deviation displayed in the Bland Altman plot can be attributed to the different measurement site of the NightOwl SpO₂ sensor and such sensor of the PSG system, which introduces measurement site related SpO₂ variability, as described by Basanrogulu et al.¹¹ This is a variability that does not exist in the ZOL-AHI to the MSSS-AHI comparison.

Figure 2. Scatter plots of comparison NightOwl-REI, MSSS-AHI and ZOL-AHI.

The dotted line represents the points for which the y-axis values equal the x-axis values of the graph (identity line). ρ = Spearman correlation coefficient, AHI = apnea-hypopnea index, MSSS = Michele Sleep Scoring System, P = P value, REI = respiratory event index, ZOL = Ziekenhuis Oost-Limburg.

Figure 3. Bland-Altman plot showing difference between NightOwl-REI and MSSS-AHI versus MSSS-AHI and ZOL-AHI and MSSS-AHI versus MSSS-AHI.

The dotted line represents the mean difference ± 1 standard deviation. The dashed line represents the mean difference. AHI = apnea-hypopnea index, MSSS = Michele Sleep Scoring System, REI = respiratory event index, ZOL = Ziekenhuis Oost-Limburg.

Table 1 displays the error matrix for the AHI and REI, represented into 4 categories defined as normal (< 5), mild (5 to < 15), moderate (15 to < 30), and severe (≥ 30). The resulting categorization accuracy was .743 for the ZOL-AHI and .812 for the NightOwl-REI when using the MSSS-AHI as benchmark.

Table 1.

Error matrix of ZOL-AHI versus MSSS-AHI and NightOwl-REI versus MSSS-AHI.

When selecting the category boundary AHI as cutoff values for binary classification and applying upper and lower boundary correction, we find sensitivities (Se), specificities (Sp), negative predictive values (NPV), and positive predictive values (PPV) as set out in Table 2. Table 3 reports on the positive and negative likelihood ratios derived from Table 2.

Table 2.

Performance metrics for correct NightOwl-REI classification and ZOL-AHI classification at different MSSS-AHI cutoff values.

Table 3.

Positive and negative likelihood ratios for NightOwl-REI classification and ZOL-AHI classification at different MSSSAHI cut-off values

TST Estimation

The TST derived by NightOwl showed a good correlation of .78 (P < .001) and ICC_3,1 of .78 with the MSSS-AHI (Figure 4), yet significantly lower than the correlation of .87 and ICC_3,1 of .87 between the ZOL-AHI and MSSS-AHI. A Bland-Altman plot illustrating the difference between the MSSS-TST and respectively the ZOL-TST and NightOwl-TST as a function of the MSSS-TST, is shown in Figure 5. The mean difference between the NightOwl-TST and MSSS-TST was −17.9 minutes and −25.7 minutes for the ZOL-TST and the MSSS-TST. The SD of these differences were 51.41 minutes (NightOwl-TST versus MSSS-TST) and 41.82 minutes (ZOL-TST versus MSSS-TST).

Figure 4. Scatter plots of comparison NightOwl-TST and ZOL-TST versus MSSS-TST.

The dotted line represents the points for which the y-axis values equal the x-axis values of the graph (identity line). ρ = Spearman correlation coefficient, MSSS = Michele Sleep Scoring System, P = P value, TST = total sleep time, ZOL = Ziekenhuis Oost-Limburg.

Figure 5. Bland-Altman plot showing difference between NightOwl-TST and MSSS-TST versus MSSS-TST and ZOL-TST and MSSS-TST versus MSSS-TST.

The dotted line represents the mean difference ± 1 standard deviation. The dashed line represents the mean difference. MSSS = Michele Sleep Scoring System, TST = total sleep time, ZOL = Ziekenhuis Oost-Limburg.

DISCUSSION

Our findings show that the NightOwl has a close agreement with the PSG for the estimation of the REI and TST, especially in light of the inter-scorer variability of the manual analysis of the PSG. These results support the feasibility of a highly miniaturized, convenient, yet very accurate HSAT system.

The NightOwl analyses changes in PAT, a mechanism primarily known from the WatchPAT HSAT from Itamar Medical.^12,13 An advantage of the NightOwl system consists of its further miniaturization of the testing equipment as it does not require a dedicated finger probe to obtain a good signal reading, neither does it require a separate wrist-worn unit that contains the processing electronics and the battery.

A known limitation of the analysis of the PAT channel is that changes in the PAT caused by autonomic arousals associated with nonrespiratory events such as periodic limb movements¹⁴ could lead to misclassifications of these changes in the PAT as a respiratory event. However, this limitation of the PAT channel to accurately discriminate between types of autonomic arousal associated events can be reduced by the incorporation of concurrent analysis of other PPG-derived features such as SpO₂.

The main limitation of this research was the single-night in-laboratory setting of the sleep study. Therefore, we could not assess the failure rate and performance of the NightOwl when applied for unattended testing in the home environment. A multi-night assessment of the NightOwl REI and TST in a home environment will be a subject of future investigation. Lastly, event-by-event analysis was not performed during this study and will be included in future work.

DISCLOSURE STATEMENT

Work for this study was performed at Ziekenhuis Oost-Limburg (Hospital East-Limburg). All authors have seen and approved the manuscript. The NightOwl devices used in this study were provided by Ectosense NV. Frederik Massie and Duarte Mendes de Almeida are employees of Ectosense NV. The other authors report no conflicts of interest.

EDITOR'S NOTE

The Emerging Technologies section focuses on new tools and techniques of potential utility in the diagnosis and management of any and all sleep disorders. The technologies may not yet be marketed, and indeed may only exist in prototype form. Some preliminary evidence of efficacy must be available, which can consist of small pilot studies or even data from animal studies, but definitive evidence of efficacy will not be required, and the submissions will be reviewed according to this standard. The intent is to alert readers of Journal of Clinical Sleep Medicine of promising technology that is in early stages of development. With this information, the reader may wish to (1) contact the author(s) in order to offer assistance in more definitive studies of the technology; (2) use the ideas underlying the technology to develop novel approaches of their own (with due respect for any patent issues); and (3) focus on subsequent publications involving the technology in order to determine when and if it is suitable for application to their own clinical practice. The Journal of Clinical Sleep Medicine and the American Academy of Sleep Medicine expressly do not endorse or represent that any of the technology described in the Emerging Technologies section has proven efficacy or effectiveness in the treatment of human disease, nor that any required regulatory approval has been obtained.

ABBREVIATIONS

AASM

American Academy of Sleep Medicine

AHI

apnea-hypopnea index

CME

Committee of Medical Ethics

HSAT

home sleep apnea testing

ICC_3,1

intraclass correlation coefficient type 3,1

MSSS

Michele Sleep Scoring System

NPV

negative predictive value

OSA

obstructive sleep apnea

PAT

peripheral arterial tone

PPG

photoplethysmography

PPV

positive predictive value

PSG

polysomnography

REI

respiratory event index

SD

standard deviation

Se

sensitivity

Sp

specificity

SpO₂

oxygen saturation

TST

total sleep time

YMT

Younes Medical Technologies

ZOL

Ziekenhuis Oost-Limburg