Validation of Watch-PAT-200 Against Polysomnography During Pregnancy

Abstract

Study Objectives:

To determine the relationships between key variables obtained from ambulatory polysomnography (PSG) and the wrist-worn Watch-PAT 200 device in pregnant women.

Methods:

In this prospective cohort study, women in their third trimester of pregnancy underwent full overnight home PSG using the 22-channel MediPalm system and the Watch-PAT 200 device. PSGs were scored by a blinded, experienced technologist using AASM 2007 criteria; the Watch-PAT was scored automatically by the manufacturer's proprietary software.

Results:

A total of 31 pregnant women were studied. Mean age was 30.2 ± 7.1 years; mean gestational age was 33.4 ± 3.0 weeks; mean BMI was 31.9 ± 8.1 kg/m²; 39% of women were nulliparous. Key variables generated by PSG and Watch-PAT correlated well over a wide range, including the apnea-hypopnea index (AHI, r = 0.76, p < 0.001); respiratory disturbance index (RDI, r = 0.68, p < 0.001), mean oxygen saturation (r = 0.94, p < 0.001), and minimum oxygen saturation (r = 0.88, p < 0.001). The area under the curve for AHI ≥ 5 and RDI ≥ 10 were 0.96 and 0.94, respectively. Association between stage 3 sleep on PSG and deep sleep on Watch-PAT was poor. Watch-PAT tended to overscore RDI, particularly as severity increased.

Conclusions:

Among pregnant women, Watch-PAT demonstrates excellent sensitivity and specificity for identification of obstructive sleep apnea, defined as AHI ≥ 5 on full PSG. Watch-PAT may overestimate RDI somewhat, especially at high RDI values.

Citation:

O'Brien LM; Bullough AS; Shelgikar AV; Chames MC; Armitage R; Chervin RD. Validation of Watch-Pat-200 against polysomnography during pregnancy. J Clin Sleep Med 2012;8(3):287-294.

Increasing evidence now shows that sleep disordered breathing (SDB) in pregnancy is associated with adverse outcomes including gestational hypertension^1–3 and possibly gestational diabetes.⁴ During the third trimester of pregnancy, the prevalence of habitual snoring, a characteristic symptom of SDB, is significantly increased compared to that in non-pregnant women and may be as high as 35%.^1,5,6 This suggests that approximately one-third of pregnant women may be at risk for SDB, yet most obstetricians do not routinely screen for SDB. Although SDB during pregnancy may often be considered transient and related to the gestational weight gain, the emerging evidence suggesting that SDB may be associated with adverse pregnancy outcomes highlights a need for simple screening devices for SDB in pregnancy.

However, it is not practical logistically to perform polysomnography (PSG) on all pregnant women who screen at high risk for SDB. Overnight PSG studies are labor-intensive, expensive, not always available, and challenging to perform in a timely manner in pregnant women, particularly in late gestation. A simple, home-based device with a low failure rate would facilitate easy recognition and timely intervention for SDB in this population. The Watch-PAT 200 device (Itamar Medical Cesarea, Israel) is a simple, wrist-worn, level 3 device approved for ambulatory diagnosis of SDB.^7,8 Measurements are made via finger plethysmography, which is uniquely sensitive to changes in autonomic activity.^9,10 Watch-PAT is well validated, including recent validation for sleep staging^11–14 and has been used in clinical practice for many years in the non-pregnant population. Such a device has great potential for the diagnosis of SDB in pregnancy; however, its validity in pregnant women has not been previously studied. We compared key variables obtained from simultaneous ambulatory PSG and Watch-PAT 200 in third trimester pregnant women. The third trimester was chosen because SDB risk is likely to peak at this time and therefore has the greatest potential range of disease severity.

BRIEF SUMMARY

Current Knowledge/Study Rationale: Emerging evidence suggests that sleep-disordered breathing (SDB) is prevalent and consequential in pregnancy; however, simple screening devices for SDB have not been validated in this population. We sought to compare key variables from the Watch-PAT 200 with ambulatory polysomnography in 3rd trimester pregnant women.

Study Impact: The Watch-PAT 200 has excellent sensitivity, specificity, and positive and negative predictive values for the identification of SDB in the 3rd trimester of pregnancy. Its use may facilitate rapid screening of pregnant women in addition to making sleep research more feasible in this population.

METHODS

This study was approved by the Institutional Review Board at the University of Michigan. Women in their last trimester of pregnancy (≥ 28 weeks' gestation) were recruited from obstetric clinics and invited to complete several sleep questionnaires. Those who reported habitual snoring (snoring ≥ 3 nights per week) were classified as at risk for SDB, and those who reported never, rarely, or occasional snoring were classified as non-snorers. An attempt was made to recruit similar numbers of snoring and non-snoring women to undergo an in-home PSG using the MediPalm-20 full PSG system (Braebon, Ontario, Canada) as well as the Watch-PAT 200 device (Itamar, Cesarea, Israel). A research technologist visited the subjects at home in the evening and hooked up the portable PSG and the Watch-PAT. The following morning, the devices were collected and downloaded for analysis.

The Medipalm Ambulatory PSG

All subjects underwent home-based digital PSG, including 6-channel electroencephalogram (EEG), electro-oculogram (EOG), submental electromyogram (EMG), leg electromyogram (LEMG), electrocardiogram (ECG), nasal and oral airflow (thermistor, nasal pressure transducer), chest and abdominal respiratory movement (respiratory inductance plethysmography), oxygen saturation, snoring microphone, and body position sensor. For EEG and EOG, the low and high filter settings were set at 0.3 and 35 Hz, respectively; for EMG the settings were 10 and 100 Hz, respectively. The sampling rate was 250 Hz. All of the sleep studies were scored by a single board-certified sleep technician blinded to study group (habitual snorer vs. non-habitual snorer) according to the recent standard criteria of the American Academy of Sleep Medicine.¹⁵ Apneas were defined as a drop in peak thermal sensor excursion by > 90% of pre-event baseline, where ≥ 90% of duration met amplitude reduction criteria for apnea, with ≥ 10 sec duration. Obstructive apneas were defined as an apnea with continued respiratory effort, SpO₂ desaturation, and/or EEG arousal. Hypopneas were scored if the nasal pressure signal excursion dropped by > 50% of baseline for ≥ 10 sec with either ≥ 3% desaturation or arousal. The apnea-hypopnea index (AHI) comprised the number of apneas and hypopneas per hour of total sleep time; the respiratory disturbance index (RDI) comprised the number of apneas, hypopneas, and respiratory event related arousals (RERAs) per hour of sleep time. Sleep staging and arousals were also scored according to the American Academy of Sleep Medicine (AASM) 2007 rules.¹⁶ All studies were reviewed by a board-certified sleep physician (AVS) who was masked to study group.

Watch-PAT 200

At the same time as the full PSG, all women also wore the Watch-PAT. Using a peripheral arterial tonometry (PAT) finger plethysmograph and a standard SpO₂ probe, the Watch-Pat 200 records the peripheral arterial tonometry (PAT) signal, heart rate, oxyhemoglobin saturation, as well as actigraphy from the inbuilt actigraph. Low and high filter settings for the PAT channel were set at 0.1 and 10 Hz, respectively. Sleep time was estimated using the inbuilt actigraph,¹⁷ and sleep was staged using spectral components of the PAT analysis, the details of which have been previously described.^12,13 Respiratory events were identified by digital vasoconstriction mediated by α-adrenergic receptors that are exquisitely sensitive to the surges in sympathetic activity that accompany respiratory events. Vasoconstriction results in an attenuated PAT signal, and the Watch-PAT proprietary software algorithm analyzed the PAT signal amplitude along with the heart rate and SpO₂ to estimate both the AHI and the RDI. Specifically, respiratory events were scored if one of the following 3 criteria was met: (1) PAT amplitude reduction occurred with acceleration in the pulse rate or increase in wrist activity; (2) PAT amplitude reduction occurred with ≥ 3% oxyhemoglobin desaturation; or (3) ≥ 4% oxyhemoglobin desaturation occurred.¹⁸ Manual editing of the automated scoring is possible but was avoided in this study to allow assessment of algorithm performance rather than that of the algorithm plus human operator.

Statistical Analysis

Data were double entered into a database and checked for outliers and normality of the distribution. Analyses were performed using SPSS version 19 (IBM SPSS Statistics). Comparisons between sleep parameters on each device were made using paired t-tests or Wilcoxon signed rank tests, for normally and non-normally distributed continuous data, respectively. McNemar test was used for dichotomous data. Correlations between parameters were performed using Pearson correlation coefficient for normally distributed data and Spearman rho correlation coefficient for non-normally distributed data (AHI and RDI). Bland-Altman Plots were also constructed. These plots provide a graphical means of comparing 2 measures. The differences between measures are plotted against the average of the 2 methods (i.e., average AHI between the PSG and the Watch-PAT). In addition, receiver operator characteristic (ROC) curves were plotted. These curves are graphical representations of the sensitivity (true positive rate) against false positive rate (1-specificity) for the range of obtained AHI and RDI values. Each point on the ROC curve represents a sensitivity/specificity pair, corresponding to a particular decision threshold. The area under the curve (AUC) is a measure of how well a parameter can distinguish between 2 diagnostic groups (e.g., SDB vs. non-SDB). Finally, analysis of the Cohen κ¹⁹ statistic was performed to test the classification of the severity of respiratory disturbance as measured by AHI and RDI. All tests were 2-tailed and were considered statistically significant if p < 0.05.

RESULTS

A total of 173 third trimester pregnant women were invited to participate in simultaneous PSG and Watch-PAT monitoring, and 37 (21%) participated. Data from 6 women were excluded because of the technical failure of PSG (n = 3), failure of the Watch-PAT to initialize (n = 1), or < 3 h of sleep on PSG (n = 2) and/or Watch-PAT (n = 1). The latter subject who obtained < 3 h sleep on the Watch-PAT was also one of the 2 women who obtained < 3 h sleep on PSG. The second subject removed the PSG sensors without getting ≥ 3 h of recorded sleep time, but continued to wear the Watch-PAT. As comparative data were not available, this subject was excluded from the analysis. Therefore the number of women included in the final analysis is 31. Table 1 shows the demographic information on study participants.

Table 1.

Demographic information for the 31 study participants

Comparison between mean values of sleep variables between the overnight PSG and the Watch-PAT are shown in Table 2. Among all subjects, 26% had obstructive sleep apnea (AHI > 5). Central apneas were rarely identified by PSG; 77% of women had essentially none, and only 1 subject had a central apnea index > 0.5. In the automatic analysis of the Watch-PAT, central and obstructive apneas are not differentiated. The following parameters measured by PSG showed moderate to good overall correlation with their counterparts measured by Watch-PAT: correlation for TST was r = 0.76, p < 0.001; AHI r = 0.73, p < 0.001; RDI r = 0.68, p < 0.001; mean SpO₂ r = 0.94, p < 0.001; and minimum SpO₂ r = 0.88, p < 0.001. There were no moderate or strong correlations between sleep stages defined as stages 1 and 2 sleep (light sleep in the Watch-PAT), stage 3 sleep (deep sleep in the Watch-PAT), and REM sleep (light sleep r = 0.10, p = 0.60; deep sleep r = 0.32, p = 0.08; REM sleep r = 0.29, p = 0.12). Figure 1 shows the correlations between key respiratory parameters by group (habitual snorers vs. non-habitual snorers). Correlations were almost equivalent for habitual snorers and non-habitual snorers. Figure 2 shows the correlations between sleep stages by group. Notably, while correlations for the total sample did not reach statistical significance, when analyzed by group non-habitual snorers showed weak correlations between devices for deep sleep and REM sleep (see Figure 2).

Table 2.

Comparison between PSG and Watch-PAT sleep variables (mean ± SD) for 31 subjects

Figure 1. Correlations between sleep time and respiratory measures from simultaneous Watch-PAT and full polysomnography (PSG) among habitually snoring subjects (closed circles and solid lines) and non-habitually snoring subjects (open circles and dashed lines).

Figure 2. Correlations between sleep stages as assessed by simultaneous Watch-PAT and full polysomnography (PSG) among habitually snoring subjects (closed circles and solid lines) and non-habitually snoring subjects (open circles and dashed lines).

Bland Altman plots are shown in Figure 3 and illustrate the mean difference (solid line) and 95% CI (broken line) for total sleep time, AHI, RDI, and sleep stages.

Figure 3. Bland Altman plots showing the mean difference between PSG and Watch-PAT variables (solid line) and the 95% confidence intervals (broken lines) among habitually snoring subjects (closed circles) and non-habitually snoring subjects (open circles).

ROC curves for AHI and RDI are shown in Figures 4A and 4B. The area under the curve for PSG AHI ≥ 5 was 0.96 (95%CI 0.88-1.00). An optimal Watch-PAT threshold of AHI 6.1 provided a sensitivity of 88% and a specificity of 91%. Notably, a sensitivity of 88% and a specificity of 86% occurred at a Watch-PAT AHI threshold of 4.9, almost exactly at the common clinical threshold for significant SDB on full PSG. The sensitivity, specificity, and positive and negative predictive values for the Watch-PAT to identify PSG AHI ≥ 5 were 0.88, 0.87, 0.70, and 0.95, respectively. For an RDI ≥ 10 on PSG, the area under the curve was 0.94 (95%CI 0.86-1.00). An optimal Watch-PAT threshold of 9.3 provided a sensitivity of 100% and a specificity of 82%. The sensitivity, specificity, and positive and negative predictive values for the Watch-PAT to identify a PSG RDI ≥ 10 were: 1.00, 0.81, 0.50, and 1.00, respectively.

Figure 4. Receiver operator characteristic curves for apnea-hypopnea index (A) and respiratory disturbance index (B).

The κ statistic for categorical agreement of AHI ≥ 5 between PSG and Watch-PAT was 0.71 (p < 0.001). The κ statistic for categories of AHI severity (AHI < 1, 1.0-4.9, 5.0-14.9, 15.0-29.9, ≥ 30) between PSG and Watch-PAT was 0.32 (p = 0.002). For RDI ≥ 10 the κ statistic was 0.41 (p = 0.013). However, for categories of RDI severity (RDI < 1, 1.0-4.9, 5.0-14.9, 15.0-29.9, ≥ 30) the κ statistic was not significant (κ statistic 0.08, p = 0.37; Table 3). The Watch-PAT had a tendency to classify apnea severity as slightly worse than that reflected on the PSG, especially for RDI measurement. For example, there were 18 subjects measured by PSG to have an RDI in the range of 1.0-4.9, but the Watch-PAT classified only 5 in this range, with 12 in the range of 5.0-14.9 and 1 classified in the RDI range 15.0-29.9. Of note, with the exception of the latter subject, most subjects whose RDI was classified too high had RDIs close to the lower boundary of the mild range (generally RDI < 8).

Table 3.

Concordance between PSG and Watch-PAT for severity of AHI and RDI

DISCUSSION

Approaches that allow effective identification of SDB among pregnant women are urgently needed, given that accumulating evidence suggests independent associations between SDB and maternal morbidity, if not adverse fetal outcomes as well.^1–5,20,21 Our findings now demonstrate for the first time that the Watch-PAT 200 ambulatory device has excellent sensitivity and specificity, as well as positive and negative predictive values, for detection of AHI ≥ 5 on full PSG in third trimester pregnant women. This is in agreement with earlier studies using PAT technology in the non-pregnant population.^7–9,22 When using RDI criteria, the Watch-PAT had good sensitivity and specificity but appeared to be more sensitive than PSG to subtle physiological changes. This resulted in a relatively weak positive predictive value but an excellent negative predictive value when using RDI ≥ 10 as the threshold. The Watch-PAT 200 demonstrated good correlation with ambulatory PSG for key respiratory variables such as AHI, RDI, mean SpO₂, and nadir SpO₂, with better correlations found in snoring women. Relationships between sleep stage characteristics by these devices were poor in this cohort.

Previous studies have demonstrated that pregnancy is associated with an increased risk for SDB that may play a role in gestational hypertension and preeclampsia.^1–3,6 The increasing awareness of SDB risk in pregnancy is evidenced by the recent surge in published literature in this field and highlights a currently unmet need. Evaluation for SDB during pregnancy must be prioritized. However, perhaps due to the logistical difficulty and the constrained time frame for PSG in pregnant women, these subjects often are not referred to sleep laboratories. Even the use of full ambulatory PSG at home can be difficult in pregnancy: women are often uncomfortable, particularly in later gestation, and belt placement for respiratory effort can be challenging. Effective diagnostic alternatives that are easy to use with less dependence on skilled technologists could increase both availability of testing and willingness of pregnant women to undergo it.

Considerable effort is expended to develop simple SDB monitoring for use in the home environment. Although many devices correlate well with in-laboratory PSG, the failure rate of portable systems is typically high, necessitating repeat studies. Several previous studies have reported on objective measures of sleep during pregnancy. Such measures have included full in-laboratory PSG^23,24 (including esophageal pressure monitoring²⁵), various portable PSG devices,^26–29 and the Watch-PAT.³⁰ However, none of these studies were designed to validate the devices or methods among pregnant women. Furthermore, the threshold for the presence of disease was not the same across studies, in part due to use of different methodologies to identify SDB; such approaches have included flow limitation,^27,29 autonomic activation,³⁰ limited channel cardiorespiratory monitors,²⁸ and AASM recommended channels.^23,24

The Watch-PAT has been well validated in the non-pregnant population,^{7,8,10,18,31,32} is FDA-approved for home testing, and is reimbursable through most insurance carriers. Our findings validate key respiratory measures generated by the Watch-PAT 200 in third trimester pregnant women. However, the overall agreement between PSG and Watch-PAT in categorization of SDB severity into levels more refined than present vs. absent was only moderate. For AHI, and particularly for RDI, the Watch-PAT in comparison to PSG appeared to overestimate severity at higher ranges. This disagreement could arise for a number of reasons. First, the use of changes in autonomic activity to detect respiratory events and arousals is likely to be more sensitive to subcortical activation that would not be scored on traditional PSG.³³ Second, pregnancy is associated with hemodynamic changes, notably increased blood volume. This may be important in increasing sympathetic activity, particularly in hypertensive pregnancies.^34,35

Furthermore, this study validated the Watch-PAT against portable PSG, which may also differ slightly from in-lab PSG. The discrepancy between Watch-PAT performance in identification of SDB (present vs. absent) and categorization into severity levels may be partially explained by the fact that changes in categorical classification can result from only small changes in AHI or RDI. Given the differences in device performance, additional research seems warranted to compare the clinical predictive value of the Watch-PAT and full PSG in pregnant women. Autonomic activation, with or without cortical activation, may prove to be a better physiological marker of overnight sympathetic activity. Several studies have shown that increased sympathetic activity is associated with higher risk of cardiovascular mortality.³⁶ Therefore, Watch-PAT categorizations of SDB severity could prove more useful than those provided by standard PSG.

Aside from accuracy, an apparent strength of the Watch-PAT approach is that it has a low rejection rate (only once did it fail to initialize in this study). Low rejection rates have been found in other studies with Watch-PAT,⁷ unlike unattended PSG where the rejection rate is typically much higher,^37,38 up to one-third of studies performed.³⁷ In this study, three women had technical failure of PSG, compared to only 1 woman with Watch-PAT. Patients also prefer home-based studies to attended studies.³⁹ Pregnant women often have other young children at home, and in addition do not look forward to the discomfort of a night in a sleep laboratory. To date, only one previous study has applied PAT technology in pregnant women.³⁰ This study recruited 17 women with preeclampsia and 25 matched controls with uncomplicated pregnancies and used both the Watch-PAT 100 (an earlier version of the Watch-PAT 200) and Endo-PAT, a device to measure endothelial function. The authors found that women with preeclampsia had a significantly higher RDI and a lower endothelial function index than controls. Furthermore, blood pressure correlated with RDI and an endothelial function index. These data augment an emerging literature that links SDB and hypertensive diseases of pregnancy.^1,3,20,40

Mean proportions of light sleep, deep sleep, and REM sleep as measured by the two devices were similar, but absolute correlations were poor. Bland Altman plots showed agreement between Watch-PAT and PSG for light sleep and REM sleep, although agreement for deep sleep was not good. The Watch-PAT tended to underscore deep sleep at lower proportions and overscore it at higher proportions. The PAT technology has only recently been used to stage sleep,^12–14 and similar data have not been reported in pregnant women. Pregnant women are known to sleep poorly, but this would not fully explain the discrepancies in sleep staging observed here, as adults with SDB (among whom the algorithms have been validated) also sleep poorly.

There are several limitations to this study. We did not validate Watch-PAT against attended PSG. This was in part due to the logistical difficulty of scheduling third trimester pregnant women to spend a night in the laboratory. Women were recruited whether they reported habitual snoring or not, and this sample is not necessarily representative of pregnant women who may be referred to sleep clinics for evaluation. This aspect of our design, however, is also a strength, in that data are likely representative of third trimester pregnant women more generally. Finally, only women in the third trimester were studied, and our results may not apply to the first and second trimesters. However, validity of the Watch-PAT during the third trimester, when some physiologic changes from baseline may be most extreme, suggests that changes earlier in pregnancy are unlikely to significantly diminish Watch-PAT performance below that previously reported for non-pregnant women and currently observed in the third trimester.

In summary, the Watch-PAT 200 has excellent sensitivity, specificity, and positive and negative predictive values for identification of pregnant women with SDB (defined as AHI ≥ 5). Coupled with a low failure rate and ease of use, particularly for pregnant women, the performance of the Watch-PAT suggests it could be used to facilitate simple and rapid screens of pregnant women, reduce the waiting time for diagnosis, and enable treatment within the short time frame necessary to have the highest chance of positive impact on maternal and fetal outcomes of pregnancy. Our data also suggest that the Watch-PAT could make research on SDB in pregnancy much more feasible, perhaps in larger samples of women for whom research based on full PSG would be inaccessible, undesirable, intolerable, or financially prohibitive.

DISCLOSURE STATEMENT

This was not an industry supported study. MediPalm devices were provided through an educational gift by Braebon Medical Corporation to Dr. Armitage. Dr. O'Brien receives equipment support from Philips Respironics Inc., Dr. Armitage is a Consultant for Eisai Inc., and Dr. Chervin has received educational grants from Philips Respironics Inc., Fisher Paykel Inc., and is named in patents owned by the University of Michigan for signal analysis diagnostic algorithms and hardware relevant to the assessment and treatment of sleep disorders. The other authors have indicated no financial conflicts of interest.