Accuracy of WatchPAT for the Diagnosis of Obstructive Sleep Apnea in Patients with Chronic Obstructive Pulmonary Disease

Abstract

The co-existence of chronic obstructive pulmonary disease (COPD) and obstructive sleep apnea (OSA), termed the overlap syndrome (OVS), is associated with adverse outcomes that may be reversed with treatment. However, diagnosis is limited by the apparent need for in-laboratory polysomnography (PSG). WatchPAT is a portable diagnostic device that is validated for the diagnosis of OSA that might represent an attractive tool for the diagnosis of OVS.

Subjects with established COPD were recruited from a general population. Subjects underwent PSG and simultaneous recording with WatchPAT. Pulmonary function testing and questionnaires were also performed.

A total of 36 subjects were recruited and valid data was obtained on 33 (age 63 ± 7, BMI 28 ± 7, 61% male, FEV₁ 56 ± 20% predicted). There was no significant difference in the apnea–hypopnea index (AHI) between PSG and WatchPAT (19 ± 20 versus 20 ± 15 events/h; mean difference 2(−2, 5) events/h; p = 0.381). The AHI was not significantly different in rapid eye movement (REM) and non-rapid eye movement (NREM) determined by PSG versus REM and NREM determined by WatchPAT. WatchPAT slightly overestimated total and REM sleep time, and sleep efficiency. The sensitivity of WatchPAT at an AHI cut-off of ≥5, ≥15, and ≥30 events/h for corresponding PSG AHI cut-offs was 95.8, 92.3, and 88.9, respectively; specificity was 55, 65.0, and 95.8, respectively.

WatchPAT is able to determine OSA reliably in patients with COPD. The availability of this additional diagnostic modality may lead to improved detection of OVS, which may in turn lead to improved outcomes for a group of COPD patients at high risk of poor outcomes.

Introduction

Chronic obstructive pulmonary disease (COPD) and obstructive sleep apnea (OSA) are both common diseases, each of which affect at least 10% of the adult population over 40 years of age [1–4]. Their coexistence in the same patient is termed overlap syndrome (OVS), a disorder that appears to be increasingly common [5]. The most conservative estimates suggest OVS affects 1% of the US population [1], while studies in patients with COPD suggest a prevalence of up to 65% [6]. Untreated OVS has been associated with poor outcomes, including additional risk of COPD exacerbation and increased mortality [7, 8]. These adverse effects of OSA on those with COPD may be more than additive (i.e. synergistic) [9]. In terms of medical costs, patients with OVS tend to accrue in one study using a Medicaid database, patients with OVS required on average $4,000 more a year in medical expenditures than patients with COPD alone [10]. Treatment of OSA in those with OVS is associated with improvement in mortality [8] and risk of COPD exacerbation [11], while greater average duration of positive airway pressure (PAP) use shows a dose-response relationship toward outcomes [12]. Thus, aggressive diagnosis and treatment of OSA amongst those with COPD appear to be warranted. However, recognition of OVS is hampered by a lack of awareness of this disorder, as well as well-validated clinical screening tools and diagnostic modalities in the COPD population [13].

Polysomnography (PSG) is the current gold standard diagnostic test for OSA, and is the only tool recommended by guidelines to evaluate sleep disordered breathing in patients with COPD [14]. However, PSG is costly and increasingly scarce, as is the expertise needed to score and interpret recordings, all of which can significantly delay the diagnosis and treatment of OSA in COPD patients. Inlaboratory PSG may also be burdensome for patients with chronic medical disorders, increasing the barriers to diagnosis. Of note, a recent ATS research statement focused on limitations of currently used criteria including the flow-based apnea–hypopnea index (AHI) in COPD [15]. For example, a 20 min sustained desaturation from prolonged hypoventilation could be labeled a single hypopnea despite potential for major consequences. Thus, alternative diagnostic methods and approaches are needed. The WatchPAT (Itamar Medical) is a home sleep apnea testing (HSAT) device which has been shown to be accurate for diagnosing sleep-disordered breathing in normal population without significant lung disease [16]. It is based on peripheral arterial tone (PAT), pulse rate, oxygen saturation, actigraphy, snoring recording, and body position. Although manual editing is possible, the post-acquisition review time is minimal. Previous WatchPAT studies excluded patients with COPD. We, therefore, sought to compare WatchPAT to traditional PSG in detecting and quantifying OSA in patients with COPD.

Materials and method

Subjects

Adult patients (≥18 years of age) with known COPD as diagnosed by a pulmonologist (defined as Global Initiative for Chronic Obstructive Lung Disease, GOLD stage 2 or higher and ≥10 pack-years of smoking history) were screened [17] between July 2015 and August 2016. Recruitment was performed outside any clinical care via flyers posted in the community and pulmonary clinics, and from a local community study of COPD. Exclusion criteria for the study were unstable COPD or active cardiovascular disease, defined as recent hospitalization within 3 months; medical conditions that would affect the diagnostic accuracy or application of WatchPAT including history of peripheral vascular disease, peripheral neuropathy, non-sinus cardiac rhythm, permanent pacemaker, finger deformity that precluded adequate sensor application. Informed consent was obtained from all participants after the protocol was approved by the Human Research Protections Program/Institutional Review Board of University of California, San Diego.

Protocol

All subjects completed a comprehensive sleep and COPD-related evaluation that included Epworth Sleepiness Scale (ESS), Pittsburgh Sleep Quality Index (PSQI), pulmonary function tests (spirometry, lung volumes by plethysmography, and diffusion capacity for carbon monoxide, DLCO), COPD symptoms and quality of life questionnaires, and endoPAT (Itamar Medical, Caesarea, Israel) test for cardiovascular risk assessment.

All subjects underwent a standard in-laboratory overnight PSG. Signals recorded included: electrooculography (EOG), electrocardiography (ECG), submental and tibial electromyography (EMG), electroencephalography (EEG), chest and abdominal respiratory movement, nasal and oral airflow (measured by a mask with pneumotach; if the subjects were unable to tolerate the mask, nasal–oral thermistor and nasal pressure were used), oxygen saturation, and snoring intensity. Subjects were encouraged to sleep supine. All of the PSGs were scored by one registered polysomnographic technologist (RPSGT) according to the American Academy of Sleep Medicine guidelines (Chicago criteria) [18]. The scoring was completed without knowledge of the WatchPAT results.

At the same time as the in-lab PSG, all subjects simultaneously wore the WatchPAT 200 (Itamar Medical Ltd., Caesarea, Israel). WatchPAT 200 is a device worn around the wrist with one finger probe and separate snoring sensor. The finger probe records the peripheral arterial tonometry (PAT) signal, heart rate, oxygen saturation with an actigraph built in with the recording device on the wrist. Sleep time was estimated by the actigraphy signal, and sleep stage was determined through PAT analysis, the details of which have been previously described [19]. Respiratory events were identified using a combination of PAT signal attenuation, heart rate changes, and desaturation on pulse oximetry and analyzed by the WatchPAT proprietary software algorithm [20]. Only the automated scoring of WatchPAT studies was used.

Although WatchPAT has no measure of airflow, the internal algorithm classifies respiratory events based on either a 4% oxygen desaturation or a 3% oxygen desaturation with changes in tonometry (indicative of arousal) – which generally parallel the Chicago criteria.

Data analysis

Analyses were conducted using SPSS version 24 (IBM SPSS Statistics). The PSG was considered the gold standard for identifying and quantifying the severity of OSA. Studies were excluded if there was less than 3 h of total sleep time on either test. For rapid eye movement (REM) versus non-rapid eye movement (NREM) analysis we only included studies in which the PSG REM time was ≥30 min. Comparisons between sleep parameters on each device were made using paired t-tests or Wilcoxon signed rank tests, for normally and non-normally distributed continuous data, respectively. Agreement between the parameters was analyzed using the Bland–Altman method. Linear regression was used to evaluate the association between measurements and assess for variable bias across baseline values. Receiver operator characteristic (ROC) curves were constructed to evaluate the diagnostic agreement between WatchPAT results and the PSG at different cut-offs (PSG AHI ≥5/h, ≥15/h, and ≥30/h). We then assessed sensitivity and specificity of WatchPAT values for these PSG cut-offs. Specifically, we evaluated corresponding clinical cut-offs (PAT AHI ≥5/h, ≥15/h, and ≥30/h) as well as “optimal” cut-offs based on the value that provided the maximum value of sensitivity + specificity.

Furthermore, diagnostic agreement was evaluated using an approach similar to that used by White et al. [21]. Small differences in AHI between the two systems that are clinically insignificant (e.g. AHI of 14 vs. 16/h) may be interpreted as diagnostic errors during sensitivity and specificity analysis. Thus, we defined diagnostic agreement if: (1) both AHI from the PSG and WatchPAT were greater than 15 events per hour or (2) PSG AHI was within 10 events per hour of WatchPAT AHI. Overestimate of AHI from WatchPAT was defined if PSG AHI < 15 events/h and WatchPAT AHI was 10 events/h greater than PSG AHI; underestimate of AHI from WatchPAT was defined if WatchPAT AHI < 15 events/h and PSG AHI was 10 events/h greater than WatchPAT AHI (the definition is also shown in Table 1).

Table 1.

Clinical assessment definition of diagnostic agreement between WatchPAT and PSG.

	PSG AHI ≥ 15	PSG AHI < 15
PAT AHI ≥ 15	Agreement	Agreement: PAT AHI minus PSG AHI ≤ 10 Overestimate: PAT AHI minus PSG AHI > 10
PAT AHI < 15	Agreement: PSG AHI minus PAT AHI ≤ 10 Underestimate: PSG AHI minus PAT AHI > 10	Agreement: PAT AHI minus PSG AHI ≤ 10 Overestimate: PAT AHI minus PSG AHI > 10 Underestimate: PSG AHI minus PAT AHI > 10

To explore whether WatchPAT results were affected by underlying pulmonary function, univariable linear regression analysis was performed using pulmonary function measures (FEV₁% predicted, FVC % predicted, TLC % predicted, RV % predicted, RV/TLC % predicted, and DLCO % predicted) with the absolute and percent difference of PAT AHI and PSG AHI. We also evaluated whether OSA severity as determined by WatchPAT or PSG was associated with clinical symptoms, as measured by aforementioned questionnaires.

Results

A total of 36 patients (22 males, 14 females) previously diagnosed with COPD consented to participate in simultaneous PSG and WatchPAT monitoring. Data from three subjects were excluded; one because of WatchPAT technical failure and 2 for <3h of sleep recorded on both PSG and WatchPAT. Three out of 33 subjects (9%) underwent PSG with nasal–oral thermistor and nasal pressure sensors instead of pneumotach as they were intolerant to the mask attached to the pneumotach. Table 2 summarizes the demographic information on the included study participants (n = 33). The prevalence of OSA in this COPD cohort was 72.7% using a PSG AHI cut-off of ≥5 events/h and 39.4% using a PSG AHI cut-off of ≥15 events/h.

Table 2.

Subject characteristics (N = 33).

	Mean ± SD/Median [Interquartile range]
Age (years)	63 ± 7
Male sex (%)	61
BMI (kg/m2)	28.1 ±6.7
Pulmonary function
FEV1/FVC (%)	57 [53, 65]
FEV1 (% predicted)	56 ± 20
TLC (% predicted)	106 ± 21
GOLD stage defined by FEV1
Mild (FEV1, > 80%)	9.1% (3)
Moderate (50% < FEV1 ≤ 80%)	28.5% (16)
Severe (30% < FEV1, ≤ 50%)	27.3% (9)
Very Severe (FEV1 ≤ 30%)	9.1% (3)
COPD related quality of life
mMRC Dyspnea Scale	1 [1, 2]
SGRQ Global Score	40.0 ± 16.9
SF-36 Domains 1–8
Physical function	52.2 ± 21.9
Role physical	48.3 ± 38.3
Role emotional	63.2 ± 43.0
Vitality	46.4 ± 18.5
Emotion well	70.9 ± 20.2
Social function	71.6 ± 21.6
Pain	65.1 ± 20.3
General health	42.2 ± 21.3
Sleep quality
Epworth Sleepiness Scale (ESS)	7.2 ± 4.9
Pittsburgh Sleep Quality Index (PSQI)	9.3 ± 4.9
Poor Sleeper (PSQI > 5)	66.7 %
Polysomnography
Obstructive sleep apnea prevalence	72.7% (AHI ≥ 5/h)
	39.4% (AHI ≥ 15/h)
Mild (AHI 5 to < 15/h)	33.3%
Moderate (AHI 15 to <30/h)	12.1%
Severe (AHI ≥ 30/h)	27.3%
Endothelial function (EndoPAT)
Reactive hyperemia index (RHI)	1.87 ± 0.62
Endothelial dysfunction (RHI ≤ 1.67)	42.4%

As shown in Table 3, there was no statistical difference between PAT AHI and PSG AHI, including during different stages of sleep (REM vs. NREM). Total sleep time was reported as higher by WatchPAT compared to PSG. WatchPAT also estimated more REM sleep time. WatchPAT oximetry measures were slightly higher than those by PSG.

Table 3.

WatchPAT and PSG results comparison (N = 33).

	PSG (Mean ± SD)	WatchPAT (Mean ±SD)	Beta co-efficient (95% CI)	Intra-individual difference [Mean (95% CI)]	p value
Overall AHI (events/h)	19 ± 20	20 ± 15	1 (1, 1)*	2 (−2, 5)	0.381
REM AHI (events/h)	22 ± 21	26 ± 14	1 (1, 2)*	4 (−2, 10)	0.217
NREM AHI (events/h)	17 ± 20	17 ± 16	1 (1, 1)*	0 (−5, 4)	0.915
Total sleep time (min)	283 ± 78	313 ± 68	1 (0, 1)*	29 (7, 52)	0.011
Sleep efficiency (%)	71 ± 17	76 ± 14	1 (0, 1)†	5 (0, 11)	0.081
REM duration (min)	41 ± 29	52 ± 33	1 (0, 1)*	11 (1, 21)	0.033
% REM sleep (%)	13 ± 9	16 ± 8	1 (0, 1)†	2 (−1,5)	0.117
Mean SpO2 (%)	92 ± 2	94 ± 2	1 (1, 1)*	2 (2, 3)	<0.001
SpO2 nadir (%)	82 ± 6	86 ± 6	0 (0, 1)	4 (1,6)	0.004
ODI 3% (events/h)	11 ± 11	10 ± 10	1 (1,1)*	1 (−3, 2)	0.502

^*p < 0.001.

^†p < 0.05.

As shown in Figure 1A, there was a significant association between PAT AHI and PSG AHI (Pearson’s coefficient R = 0.85, p < 0.001). A Bland–Altman plot of PAT AHI and PSG AHI is shown in Figure 1B. The mean intra-individual difference in PAT AHI versus PSG AHI was 2 events/h (95% CI −2 to 5). Limits of agreement (mean difference ± 1.96 SD) was −19 to 22 events/h. At lower levels of AHI, PAT tended to overestimate severity, while at higher levels of AHI, WatchPAT underestimated severity (intercept 7.0; beta −0.28, 95% CI −0.48 to −0.7; p = 0.009). The intra-individual difference in WatchPAT and PSG AHI (both in absolute and percent terms) was not associated with pulmonary function test results (all p > 0.10).

Figure 1.

(A) Comparison between WatchPAT AHI and PSG AHI for each subject. Pearson’s coefficient R = 0.85. Dot-dash line denotes PSG AHI of 15, dotted lines denote PSG AHI of 5 and 30. Circles denote patients with a diagnostic assessment of agreement, squares denote patients with overestimate, and diamonds with underestimate (see Table 1 for definitions). (B) Bland–Altman plot between PSG AHI and WatchPAT AHI. Circles denote patients with a diagnostic assessment of agreement, squares denote patients with overestimate, and diamonds with underestimate (see Table 1 for definitions).

In order to assess the sensitivity and specificity of WatchPAT, we constructed ROC curves using PSG AHI threshold values (5, 15, and 30 events per hour) that correspond with clinical OSA cut-offs. Sensitivity and specificity for the corresponding WatchPAT AHI cut-off, as well as for optimal WatchPAT cut-offs (i.e. maximizing sensitivity + specificity), are shown in Table 4.

Table 4.

ROC and sensitivity/specificity analysis

ROC AUC (95% CI)	PSG AHI ≥ 5 0.919 (0.826, 1.000)	PSG AHI ≥ 15 0.885 (0.745, 1.000)	PSG AHI ≥ 30 0.944 (0.863, 1.000)
Clinical PAT AHI cut-off	≥5	≥15	≥30
Sensitivity (%)a	95.8	92.3	88.9
Specificity (%)a	55.6	65.0	95.8
Optimal PAT AHI cut-off	≥12	≥20	≥28
Sensitivity (%)	83.3	76.9	88.9
Specificity (%)	88.9	90.0	95.8

^aValues are approximated based on nearest cut-off available in ROC table.

When using the White and Westbrook approach to assess the diagnostic agreement between PSG and WatchPAT, concordance was found in 26 of 33 subjects (78.8%). In six cases (18.2%), the AHI was overestimated by WatchPAT; in one case (3.0%), the AHI was underestimated by WatchPAT. The agreement status for each subject is shown in Figure 1(A,B).

Among these COPD subjects, neither PAT AHI nor PSG AHI predicted sleepiness as measured by the Epworth Sleepiness Scale (ESS) (both p > 0.10). WatchPAT or PSG sleep efficiency did not correlate with sleep quality evaluated by Pittsburgh Sleep Quality Index (PSQI) (both p > 0.10).

Discussion

The key findings of this study are that (1) the presence of OSA and the AHI as determined by WatchPAT has good agreement with PSG in patients with COPD, (2) WatchPAT AHI accuracy was not affected by severity of COPD as measured by lung function, and (3) while total and REM sleep times were overestimated by WatchPAT, differences were likely to be of minimal clinical significance.

WatchPAT has been shown to have good performance in generally healthy OSA patients, specifically those without major cardiopulmonary disease. Our work suggests that in those with moderate to severe COPD, there is good agreement with gold standard PSG. We analyzed this agreement in several ways, including Bland–Altman analysis showing minimal overall bias. The findings of our Bland–Altman analysis in COPD patients were very similar to the prior study among OSA patients without significant cardiopulmonary diseases [20]. WatchPAT tends to underestimate disease severity with higher AHI as WatchPAT has more difficulty in detecting each individual hypopnea/apnea event when multiple events occurred over a brief time period. With frequent respiratory events, there is insufficient time for PAT signal to return to baseline to differentiate one event from the next one. In addition, we conducted other analysis designed to assess for potentially clinically significant differences. Specifically, we performed ROC analysis at different PSG AHI cut-offs, in order to show the ability of WatchPAT to diagnose OVS using cut-offs that reflect clinical practice; AHI of 5 is used to maximize sensitivity for OSA, AHI of 15 is used to identify patients with clearly elevated risk of several outcomes, and AHI of 30 to identify those at the highest risk of OSA complications. WatchPAT performed well in detecting these groups. We also performed a diagnostic agreement analysis according to similar methods by White et al., designed to detect significant discordances between WatchPAT and PSG; namely, instances in which the AHI differed by more than 10, when at least one of the modalities indicated an AHI of <15. This analysis helps to identify whether substantial misclassification might take place. We found only a few misclassified individuals by this analysis. Given the convenience and lower cost than PSG, WatchPAT represents a viable option for speeding diagnosis of co-morbid OSA in at-risk populations with COPD.

Second, we did not find an association with worsening lung function and loss of fidelity using Watch PAT vs. PSG. Scoring of respiratory events in patients with chronic lung disease may be challenging for several reasons. Patients with chronic lung disease may have an increased propensity to desaturation, which might both increase the number of hypopneas that meet the desaturation threshold for scoring, but also lead to desaturation events with minimal changes in flow that do not meet the threshold for hypopneas. One might therefore suspect that worsening lung function would be a source of discrepancy between a modality that uses flow (i.e. PSG) versus one that does not (i.e. WatchPAT). However, this was not seen in our study, which included patients with a spectrum of disease severity, including approximately one-third with severe COPD. We note that none of our COPD patients were using supplemental oxygen; further validation in this group would be helpful given the difficulties in performing and scoring PSG in such patients.

Third, sleep staging, particularly REM vs. non REM, appeared generally accurate using built-in automated software. The tendency for WatchPAT to overestimate REM sleep time was found in a previous study by Hedner et al. in OSA patients without significant cardiopulmonary diseases [22]. Actigraphy-based sleep staging has been an area of growing interest in the consumer and sleep populations, but has been minimally evaluated in patients with chronic illnesses. In patients with COPD, complaints potentially referable to poor sleep are common, while objective studies have documented objective issues with inadequate sleep duration, abnormal sleep architecture, and sleep disruption. Further studies examining sleep issues in COPD might benefit from use of WatchPAT to assess simultaneously whether these issues might relate to OSA versus other changes in sleep.

Although the primary aim of this study was not to determine the prevalence of OVS amongst those with COPD, we did recruit an unselected cohort of COPD patients rather than an OSA-referral population. Our reported prevalence of 72.7% using a PSG AHI cut-off of 5/h, and 39.4% using a PSG AHI cut-off of 15/h is notable, and adds to a growing literature suggesting that OSA is common in those with COPD. Nonetheless, we did not have a non-COPD control group, so we are unable to determine whether COPD predisposes to OSA. We did not find an association between OSA severity and sleepiness in this group, and overall, the significance of OVS remains an area of active investigation.

As stated, a recent research statement from the American Thoracic Society has emphasized the need for improved diagnostic techniques and methods for patients with COPD to assess sleep and breathing. Currently used metrics such as the AHI may have limited value in COPD in which prolonged hypopneas can occur and in whom desaturations are common based on baseline hypoxemia. In addition, supplemental oxygen via nasal cannula can limit the utility of nasal pressure to assess airflow. Thus, techniques to assess sleep disordered breathing in COPD are desirable particularly if they are widely available on a large scale and not reliant on airflow measurements. Further efforts will be required whether tonometry-based metrics during sleep can be used to improve assessment of patient-oriented consequences of OVS (i.e. cardiovascular outcomes, symptoms, functional status) compared with PSG.

There are several important limitations to this study. First, the numbers of subjects were relatively modest. Nonetheless, our findings were quite consistent based on observed narrow confidence intervals. Second, we did not study the “sickest” patients with COPD – those with recent exacerbations or those on supplemental oxygen. While the fact that WatchPAT does not rely on flow might make it an attractive option for such patients, additional validation data would be necessary for such use. Third, there were some notable differences between WatchPAT data and PSG; namely, difference in sleep times and oximetry. Sleep time differences were similar to those reported from other actigraphy technology [23], and are of unclear clinical significance, particularly as they did not seem to impact respiratory event indices substantially. Oximetry technology has previously been shown to vary between devices [24]. Small differences of 2–3% are likely with the margin of error between oximetry and gold-standard arterial co-oximetry or direct PaO₂ measurement, and are unlikely to be clinically significant. In addition, our patients are recruited through flyers and from a local community COPD study. This may lead to a selection bias which would have increased the percentage and severity of the detected cases of OSA. Lastly, we used Chicago criteria instead of AASM criteria as we used gold standard pneumotachography for our PSG. Our view is that Chicago criteria are likely the most representative of gold standard airflow, recognizing that other AASM criteria have been proposed based on logistic and financial considerations (e.g. defining hypopnea just based on 4% desaturation likely grossly underestimates respiratory disturbance). Based on our recent publication [25], the various criteria do lead to slight differences in results but that these differences are systematic and predictable based on the criteria which are used. Despite these limitations, we view our findings as important as they may help in the design of large scale clinical trials and in future efforts to predict OSA-related cardiovascular risk in COPD.

Conclusion

These findings support that WatchPAT may be used to detect OSA accurately in patients with COPD. Given the relatively high prevalence of OVS observed in this study and others, growing literature regarding the importance of OVS, and previously limited options for diagnosis, further efforts should be undertaken to optimize the diagnostic approach to OSA amongst patients with COPD.