Don’t hold PAT: watch for and correct oximetry artifact

Abstract

This study evaluated the accuracy of the algorithmic oxygen saturation (SpO₂) nadir detection of WatchPAT (Zoll/Itamar, Caesarea, Israel) compared with visual inspection in a real-world setting. SpO₂ tracings for 209 consecutive adult WatchPAT recordings were reviewed for SpO₂ artifact, with erroneous SpO₂ data removed manually. Error rates for SpO₂ minima were determined across all studies, and relationships between correct and erroneous studies examined. The overall error rate for SpO₂ nadir was 22.5%. Erroneous studies had overall less time spent at SpO₂ ≤ 88%, higher true SpO₂ nadir, lower mean body mass index, and greater artifact time; however, these variables were not associated with the magnitude of discrepancy between manual and algorithmically derived SpO₂ minima. These data demonstrate that SpO₂ nadir determined by WatchPAT algorithms should not be considered universally accurate. Like other home sleep apnea tests, visual inspection and manual correction of the study data are often required to derive accurate clinical results.

Citation:

Plante DT, Rumble ME. Don’t hold PAT: watch for and correct oximetry artifact. J Clin Sleep Med. 2023;19(12):2113–2116.

INTRODUCTION

The ability to reliably measure oxygenation is a prerequisite for successful sleep studies that assess for presence and severity of sleep-disordered breathing. With the rise of portable monitoring for obstructive sleep apnea (OSA), the need to have accurate and reliable pulse oximetry measurements during unattended studies is particularly important, as there is no technician to verify the signal in real time. Even during attended polysomnography (PSG), the veracity of the oximetry signal is an important issue, as it is subject to several sources of artifact such as motion and low perfusion states.¹

While signal processing methods have been developed to automatically remove artifact from pulse oximetry data, visual inspection of the raw data is still required to verify the accuracy of measured hypoxemia during sleep testing.² As the field moves toward increased automation of sleep stage and event scoring,³ evaluation of the real-world performance of automated algorithms becomes a critical aspect of clinical care and research. In the case of hypoxemia associated with OSA, accurately determining the oxygen saturation (SpO₂) nadir is important for risk stratification and prediction of several health-related outcomes, including postoperative complications,⁴ insulin resistance/metabolic syndrome,⁵ and myocardial infarction⁶ in OSA.

The WatchPAT is a Food and Drug Administration–approved device that utilizes peripheral arterial tonometry, pulse oximetry, accelerometry, and other ancillary signals to diagnose OSA. Several studies have compared the accuracy of WatchPAT to PSG.⁷ In the last several years and during the course of the coronavirus disease 2019 (COVID-19) pandemic, WatchPAT has become increasingly utilized in the practice of sleep medicine, and is considered a viable option for home sleep testing in those for whom it is not contraindicated.⁸ WatchPAT uses automated algorithms to remove failed data from the recording to estimate sleep and breathing parameters, including data from the pulse oximeter. However, because of inherent limitations of all algorithmic approaches to artifact detection when compared with visual inspection, this study was conducted to evaluate the real-world accuracy of the WatchPAT in determining minimal SpO₂ values in clinical patients evaluated for OSA.

METHODS

A total of 209 consecutive WatchPAT studies conducted in adults (age ≥ 18 years) between November 11, 2022, and February 15, 2023, at Wisconsin Sleep, the sleep laboratory and clinic of the University of Wisconsin–Madison, that were deemed interpretable for clinical purposes by the interpreting physician were evaluated. Study procedures were reviewed and approved by the University of Wisconsin Health Sciences Institutional Review Board.

WatchPAT studies were performed using WatchPAT 300 devices and analyzed using zzzPAT (Zoll/Itamar, Caesarea, Israel). Each study underwent prospective evaluation and visual examination of the pulse oximetry signal by a board-certified sleep medicine provider who was also a member of the American Academy of Sleep Medicine (AASM) Scoring Manual Committee (D.T.P.). SpO₂ was considered probable artifact if the decline was considered too sudden to be physiological and/or there was probable motion causing artifact. Intervals of artifact were manually removed within the zzzPAT software prior to final study interpretation. Studies were dichotomized by the congruence of the SpO₂ nadir generated automatically vs the value derived after second-order visual inspection. If these values were equivalent, the automated SpO₂ nadir was considered accurate; if different, the automated SpO₂ nadir was considered erroneous.

The SpO₂ nadir automatically generated by the WatchPAT scoring algorithm, SpO₂ nadir after visual inspection, time spent at SpO₂ ≤ 88%, apnea-hypopnea index (AHI; using ≥ 3% oxygen desaturation), total sleep time, technically valid sleep time (upon which respiratory indices were calculated), age, sex, and body mass index (BMI) of each patient were recorded. The overall error rate for SpO₂ nadir was calculated as the proportion of erroneous studies among all evaluated studies. Comparisons of age, sex, BMI, AHI, total time spent at SpO₂ ≤ 88%, and the amount of artifactual time derived by the WatchPAT algorithm (total sleep time minus technically valid sleep time) between erroneous and accurate studies were compared using 2-sample t tests and tests of proportions, where appropriate. Additional relationships between severity measures and degree of SpO₂ nadir measurement error were examined using Pearson’s correlation on an ad hoc basis. Statistics were performed using JMP Pro version 15.0.0 (SAS Institute Inc., Cary, NC, USA).

RESULTS

The overall error rate for SpO₂ nadir was 22.5% (47 out of 209) of studies. The average difference between erroneous and corrected SpO₂ values was 6.6% ± 5.3% (range: 1% to 25%) (Figure 1). This occurred despite a significant relationship between correct and algorithmically derived SpO₂ minima across all participants (r = .88, P < .0001). Among erroneous studies, the amount of additional data removed was 21.8 ± 49.1 seconds per study: 48.9% (23 out of 47) resulted in a change in SpO₂ nadir from a value at or below 88% to above 88%; 34% (16 out of 47) resulted in a change in SpO₂ nadir from below 90% to above 90%.

Figure 1. Erroneous SpO₂ minima detected by WatchPAT.

(A) Individual differences between erroneous and corrected SpO₂ nadirs across all individual participants (n = 47). Participant denoted by blue asterisks is shown in detail in parts B and C as an example. (B) 100-s SpO₂ tracing derived from the native WatchPAT algorithm. Note failure of the algorithm to fully exclude artifactual data, which resulted in a reported SpO₂ nadir of 73%. (C) Manual expansion of the excluded artifactual segment (denoted by the white text box) after visual inspection resulted in a change in SpO₂ nadir from 73% to 87% for the study. SpO₂ = oxygen saturation.

There were no differences in AHI, age, or sex between studies with correct vs erroneous SpO₂ minima (Table 1). BMI was significantly lower and artifact time was significantly higher among erroneous vs correct studies. On average, studies with SpO₂ nadirs correctly determined by the WatchPAT algorithm also had lower true SpO₂ minima and greater time spent at SpO₂ ≤ 88% during the recording than studies with erroneous SpO₂ minima. However, among erroneous studies, there were no significant relationships between the magnitude of SpO₂ difference between erroneous and corrected values and the time spent at SpO₂ ≤ 88% (r = .17, P = .27), true SpO₂ nadir (r = .14, P = .27), BMI (r = .08, P = .57), or artifact time (r = .13, P = .39).

Table 1.

Study and participant characteristics for correct and erroneous studies based on SpO₂ minima.

Measure	Correct (n = 162)	Erroneous (n = 47)	P
AHI, events/h	20.2 ± 19.8	18.5 ± 15.3	.58
SpO2 ≤ 88%, min	9.5 ± 27.9	0.8 ± 1.5	.03
True SpO2 nadir, %	84.6 ± 8.1	88.1 ± 3.1	.004
Age, y	52.2 ± 17.3	55.4 ± 17.4	.27
Sex, % female	53.1	53.2	.99
BMI, kg/m2	33.2 ± 8.6	29.8 ± 7.4	.02
Artifact time, min	9.1 ± 13.6	20.3 ± 29.7	.0003

Artifact time calculated as total sleep time minus valid sleep time from the WatchPAT algorithm. AHI = apnea-hypopnea index, BMI = body mass index, SpO₂ = oxygen saturation.

DISCUSSION

This study demonstrates that the automated SpO₂ artifact detection utilized by WatchPAT results in inaccurate SpO₂ nadir values in approximately 1 out of every 4.5 studies conducted at a single academic sleep center. Inaccurate SpO₂ minima were more likely in those with an overall lower hypoxic burden of OSA and BMI, as well as overall greater artifact time, but were otherwise unrelated to AHI, age, or sex. These results have important implications for the clinical practice of sleep medicine.

The ease of patient use and automated scoring algorithms have likely contributed to the increasing and widespread use of WatchPAT to diagnose OSA. While peripheral arterial tonometry–based technologies are an important advance in the diagnosis and assessment of OSA, it is critical for practicing providers to recognize that, like all sleep technologies, the assumption that “out-of-the-box” use is accurate is likely to result in misdiagnoses and/or inadequate risk stratification. The AASM clinical guidelines on the use of unattended portable monitors to diagnose OSA in adults requires that the raw data be reviewed and edited by qualified sleep technologists and providers.^9,10 In practice, however, it is not clear how often sleep centers and providers edit WatchPAT studies. While methods have been proposed to increase the congruence of WatchPAT sleep staging and respiratory event scoring relative to PSG,¹¹ they do not describe methods for managing pulse oximetry artifact.

The design of this study does not compare the performance of the WatchPAT algorithm with PSG or other objective sleep tests, relying instead on visual removal of SpO₂ artifact. However, it is noteworthy that, in prior validation studies, the WatchPAT both over- and underestimated SpO₂ minima across participants.¹² In fact, several studies performed in selected populations have demonstrated PSG-derived SpO₂ nadirs to be, on average, below that derived from the WatchPAT.^13–15 This is despite the strong overall correlation between SpO₂ minima derived from PSG and WatchPAT reported in validation studies, which is very similar to what was observed in our dataset.^12,13,16

Many of the inaccurate SpO₂ nadirs in this study occurred in conjunction with the WatchPAT algorithm identifying some degree of SpO₂ artifact but failing to remove sufficient samples to lead to an accurate result (see Figure 1 for an example). Future modifications of the algorithm may enhance the overall accuracy of the technology. In the interim, this study demonstrates that careful review of the SpO₂ signal will be required to deliver accurate results and optimal patient care.

DISCLOSURE STATEMENT

Both authors have reviewed and approved the manuscript. Work for this study was performed at the University of Wisconsin-Madison and was not funded. Dr. Plante has received grant support from the National Institute on Aging (NIA), National Institute of Nursing Research (NINR), National Institute of Mental Health (NIMH), American Sleep Medicine Foundation, the Brain and Behavior Research Foundation, Wisconsin Alumni Research Foundation, Alzheimer’s Association, and the University of Illinois at Chicago Occupational and Environmental Health and Safety Education and Research Center/National Institute for Occupational Safety and Health; has served as a consultant for Teva Australia, Harmony Biosciences, Aditum Bio LLC, and Jazz Pharmaceuticals; and served on medical advisory boards for Jazz Pharmaceuticals and Alkermes, all unrelated to the current study. Dr. Rumble has received grant support from Merck, unrelated to the current study.

ABBREVIATIONS

AHI

apnea-hypopnea index

BMI

body mass index

OSA

obstructive sleep apnea

SpO₂

oxygen saturation