Validation of the Apnea Risk Evaluation System (ARES) Device Against Laboratory Polysomnography in Pregnant Women at Risk for Obstructive Sleep Apnea Syndrome

Abstract

Study Objective:

To assess the validity of using the Apnea Risk Evaluation System (ARES) Unicorder for detecting obstructive sleep apnea (OSA) in pregnant women.

Methods:

Sixteen pregnant women, mean age (SD) = 29.8 (5.4) years, average gestational age (SD) = 28.6 (6.3) weeks, mean body mass index (SD) = 44.7 (6.9) kg/m² with signs and symptoms of OSA wore the ARES Unicorder during one night of laboratory polysomnography (PSG). PSG was scored according to AASM 2007 criteria, and PSG AHI and RDI were compared to the ARES 1%, 3%, and 4% AHIs calculated with the ARES propriety software.

Results:

Median PSG AHI and PSG RDI were 3.1 and 10.3 events/h of sleep, respectively. Six women had a PSG AHI ≥ 5 events/h of sleep and 11 had a PSG RDI ≥ 5 events/h of sleep. PSG AHI and RDI were strongly correlated with the ARES AHI measures. When compared with polysomnographic diagnosis of OSA, the ARES 3% algorithm provided the best balance between sensitivity (1.0 for PSG AHI, 0.91 for PSG RDI) and specificity (0.5 for PSG AHI, 0.8 for PSG RDI) for detecting sleep disordered breathing in our sample.

Conclusions:

The ARES Unicorder demonstrated reasonable consistency with PSG for diagnosing OSA in this small, heterogeneous sample of obese pregnant women.

Citation:

Sharkey KM, Waters K, Millman RP, Moore R, Martin SM, Bourjeily G. Validation of the Apnea Risk Evaluation System (ARES) device against laboratory polysomnography in pregnant women at risk for obstructive sleep apnea syndrome. J Clin Sleep Med 2014;10(5):497-502.

Pregnancy results in physiological changes that increase the risk of sleep disordered breathing (SDB). For instance, mechanical changes in gravid women—such as rapid weight gain and increased intra-abdominal pressure—result in decreased functional residual capacity and reduced oxygen reserve¹ and increase the risk of upper airway collapsibility. Other risk factors for SDB in pregnancy include increases in nasal congestion, Mallampati score, and snoring, and— particularly in pregnant patients who develop preeclampsia— decreases in upper airway size.^2–7

SDB in pregnancy has a negative impact on fetal outcomes.^8,9 A retrospective analysis compared 57 pregnant women with sleep apnea to 57 obese controls and 57 normal weight controls and showed higher rates of low birth weight and prematurity in women with OSA compared to obese and normal weight controls.¹⁰ Another study linking nation-wide population-based datasets in Taiwan showed that women with OSA (n = 791) were at an increased risk for preterm birth and small for gestational age compared to women without a diagnosis of OSA (n = 3,955), even after adjusting for multiple confounders.¹¹

SDB also results in adverse outcomes in pregnant women.^12,13 In our recent study of 1,000 postpartum women, symptoms of SDB during pregnancy (measured with the multivariable apnea prediction index¹⁴) were associated with a greater than two-fold increased risk of gestational hypertensive disorders, gestational diabetes, and unplanned Caesarean deliveries.¹³ Furthermore, from a physiologic standpoint, the impact of apneic episodes in pregnancy seems to be more potent than in non-pregnant women. In response to apneas, pregnant patients show a more rapid increase in their carbon dioxide levels and faster rates of oxygen desaturation than non-pregnant controls.¹⁵

BRIEF SUMMARY

Current Knowledge/Study Rationale: This study was performed to assess the performance of the ARES Unicorder, a level 3 device for detecting sleep disordered breathing, in pregnant women.

Study Impact: Our validation of the ARES device in pregnant women allows clinicians and researchers to measure sleep disordered breathing with the ARES in this population with more confidence and adds to the armamentarium of level 3 devices that have been validated in pregnancy.

Given that more than one-third of pregnant women snore¹⁶ and that nearly six million women become pregnant every year, SDB may represent a large diagnostic burden when relying on in-laboratory testing. Home recordings for detection of OSA are more convenient and perhaps more acceptable for pregnant patients who may have other children at home and who may be reluctant to stay overnight in the sleep laboratory. The purpose of this study was to validate a portable recorder, the Apnea Risk Evaluation System (ARES) Unicorder, with traditional in-laboratory polysomnography for detection of SDB in pregnant women. The ARES Unicorder was approved by the Federal Drug Administration (FDA) for evaluation of adult patients with possible sleep apnea in 2004 and has been validated with polysomnography in non-pregnant adults.^17–19 Because of the physiologic changes of pregnancy, we sought to validate the ARES Unicorder with traditional laboratory techniques in pregnant women.

METHODS

Participants

Participants (n = 37) were recruited from pregnant patients ages 18-45 referred for a sleep study to be evaluated for clinical suspicion of obstructive sleep apnea (OSA). Symptoms such as loud snoring, excessive daytime sleepiness, gasping or choking awakenings, and/or witnessed apneas prompted referrals, as did physical exam findings suggestive of OSA, including elevated body mass index, large neck size, Mallampati class 3-4 airway, large tonsils, large/inflamed uvula, and obstructed nasal passages. Participants were required to speak and read English and were excluded if they had a skin condition that would not allow the patient to wear the ARES Unicorder on the forehead or were currently using supplemental oxygen.

Procedures

Participants had concurrent data collection with the ARES Unicorder (Advanced Brain Monitoring, Inc., Carlsbad, CA) and standard in-laboratory polysomnography (PSG) at the Sleep Disorders Center of Lifespan Hospitals. Standard polysomnographic signals were recorded, including EEG (F2-A1, F1-A2, C4-A1, C3-A2, O1-A2, O2-A1); electro-oculogram from surface electrodes taped at the left and right outer canthi; submental electromyogram from electrodes taped over the mentalis/submentalis muscles; bilateral tibial electromyogram from electrodes taped to the anterior tibialis muscles; ECG from surface electrodes taped to the shoulder and side; airflow measured with a nasal pressure transducer; chest and abdominal motion measured with piezoelectric strain sensors for earlier studies and impedance plethysmography for later studies per laboratory protocol; oxyhemoglobin saturation (SpO₂) measured with pulse oximetry; body position; and snoring volume. PSG was acquired with the SomnoStar Pro data acquisition system (Viasys, Inc., Yorba Linda, CA) or Xltek (Natus Medical Incorporated, San Carlos, CA), and data were digitized and stored on a secure server.

Technicians attached the ARES Unicorder in the laboratory on the PSG night. The ARES Unicorder is a wireless physiological recorder worn on the forehead. It measures blood oxygen saturation and pulse rate using an optical sensor imbedded in silicone, head movement and head position using 2 dual-axis accelerometers, airflow and effort of breathing using a standard nasal cannula attached to a pressure transducer, and snoring sounds using a microphone. To acquire simultaneous PSG and ARES recordings, a single nasal cannula was used with a splitter so that airflow signals could be recorded for both methods.¹⁸ The Unicorder dimensions are 4.5 × 7.0 × 5.0 cm, and the silicone pad that sits on the skin measures 2.5 × 3.5 cm. The device was attached using an adjustable elastic strap and foam padding around the forehead sensor. Data were acquired onto a removable 32 MB flash card, which was then downloaded to a desktop computer for data analysis. Technologists were advised that their main priority was to obtain a clinically useful PSG recording. Thus, they did not typically intervene with the ARES Unicorder, other than to remove the device if its use was interfering with PSG acquisition.

Nocturnal PSGs were visually scored in 30-sec epochs according to standard criteria,^20,21 all by the same registered polysomnographic technologist (RM), supervised by sleep medicine physicians (RPM, KMS). Obstructive apneas were defined as ≥ 90% decrease in airflow from baseline lasting ≥ 10 seconds. Hypopneas were scored using AASM 2007 “recommended” criteria, that is, events where the nasal pressure signal dropped to ≥ 30% of baseline for ≥ 10 sec with oxygen desaturation ≥ 4%. Respiratory effort related arousals (RERAs) were scored when there was a sequence of breaths ≥ 10 sec characterized by increasing respiratory effort or flattening of the nasal pressure waveform that led to an arousal from sleep.^20,21 Similarly, arousals were defined according to the “recommended” AASM 2007 criteria, that is, an abrupt change in EEG frequency lasting ≥ 3 sec including alpha, theta, and/or frequencies > 16 Hz during stages N1, N2, N3, or R. Sleep measures generated from the scored data included time in bed, total sleep time, latency to sleep onset, wake after sleep onset, sleep efficiency, arousal index (number of arousals per hour of sleep), N1%, N2%, N3%, and REM %. Respiratory variables included number of apneas, number of hypopneas, apnea+ hypopnea index (number of apneas+ hypopneas per hour of sleep; AHI), respiratory disturbance index (apneas + hypopneas + RERAs; RDI), SpO₂ minimum, and SpO₂ mean. Cardiac variables include baseline waking heart rate and average heart rate during sleep.

Raw ARES Unicorder data were edited by a physician reader (KW) to determine sleep onset and offset times, and to remove obvious artifacts such as device removal, excessive head movements, and dropped SpO₂ signals. Additionally, edited data files were processed using the ARES Insight software (Watermark Medical), which applies automated algorithms to raw data to identify arousals associated with changes in SpO₂ (desaturation/resaturation events), pulse rate, head movement, effort of breathing and airflow, and snoring level that are related to abnormal breathing during sleep. The arousal events are labeled as hypopneas if linked to diminished airflow and 1%, 3%, or 4% percent desaturation, and are labeled as apneas if linked to cessation of airflow or effort. These analyses are combined to calculate an ARES apnea+hypopnea index (AHI). The software also provides data on body position and degree of hypoxemia. For this analysis, we used the ARES AHIs determined with 1%, 3%, and 4% desaturations.

Upon awakening, participants completed an ARES Feedback Questionnaire to provide information about their subjective experience wearing the ARES Unicorder. Patients were asked to score each of the 6 questions on a scale of 1 to 7, with 7 indicating “yes, definitely,” and 1 indicating “not at all.”

The study was approved by the Institutional Review Boards at Rhode Island Hospital and Women and Infants Hospital. Informed consent was obtained at the time of referral for the sleep study. Participants received a $10 gift card after completing the study.

Analysis

Data were analyzed using SPSS (Version 19, IBM SPSS) and Prism (Version 5, GraphPad Software, Inc.). We used Student t-tests to compare sleep measures between participants with and without OSA and Pearson correlations to assess the strength of associations between PSG and ARES measures. Specificity, sensitivity, positive predictive value, negative predictive value, and κ coefficients were computed to determine whether AHIs and RDIs measured with PSG and the ARES AHI measures from the ARES Unicorder provided diagnostic consistency. We also calculated the Lin concordance correlation coefficient (rho_c)^22,23 to allow us to compare our data to the findings of Louis et al.,²⁴ who also tested the ARES Unicorder in pregnant women. Alpha < 0.05 was considered statistically significant.

RESULTS

Patient Demographics

Of the 37 patients who consented to participate, we acquired complete datasets from 16 women. The most common reason for non-completion was cancellation or failure to attend the scheduled sleep study (n = 10). In 7 participants, ARES data collection failed due to equipment malfunction (recorder was not running at the time of download) or technical difficulties during data acquisition (recorder was interfering with PSG and was removed by night technologist so that clinical PSG could proceed). Other reasons for non-completion were miscarriage prior to completing the sleep study (n = 2) and refusal of the ARES device on the night of the PSG (n = 2). Among women who completed the study, average age ± SD was 29.8 ± 5.4 years (range 21 to 41 years), and average gestational age ± SD was 28.6 ± 6.3 weeks (range 17 to 38 weeks). The sample was obese, with a mean ± SD body mass index (BMI) = 44.7+6.9 kg/m² (range = 30.3 to 61.2 kg/m²). The sample was 62.5% white, 18.75% African American, and 18.75% Latina. A summary of polysomnographic sleep data is shown in Table 1.

Table 1.

Polysomnographic sleep and respiratory measures

Sleep Disordered Breathing Measured with Polysomnography

Median AHI was 3.1 events/h of sleep (interquartile range [IQR] = 0.4 to 8.0), and median RDI was 10.3 events/h of sleep (IQR = 3.1 to 17). Six women had an apnea-hypopnea index ≥ 5/h, and 11 had an RDI ≥ 5/h. Distribution of AHIs and RDIs are shown in Figure 1.

Figure 1. Distribution of AHIs and RDIs.

Symbols illustrate individual participants' AHIs and RDIs, and horizontal lines indicate median AHI and RDI for the whole sample.

ARES versus Polysomnography

As shown in Figure 2, we observed strong correlations between ARES AHI and PSG AHI (1% algorithm: r = 0.83, p < 0.001; 3% algorithm: r = 0.80, p < 0.001; 4% algorithm: r = 0.68, p = 0.004) and between ARES AHI and PSG RDI (1% algorithm: r = 0.92, p < 0.001; 3% algorithm: r = 0.90, p < 0.001; 4% algorithm: r = 0.82, p < 0.001). Kappa coefficients, sensitivity, specificity, positive predictive value, and negative predictive value of the ARES measures vs. PSG are shown in Table 2. When we classified OSA using a PSG AHI cutoff ≥ 5 events/h of sleep, the ARES OSA 3% algorithm had the best balance between sensitivity and specificity for detecting SDB. The ARES 3% algorithm identified all 6 women diagnosed with OSA by PSG AHI ≥ 5 events/h, but also resulted in ARES AHI values ≥ 5 in 5 women who did not have PSG AHI values ≥ 5 events/h. The rho_c concordance correlation coefficient for the ARES AHI 3% algorithm was rho_c = 0.74 (95% CI: 0.43-0.89). The ARES 1% algorithm also showed 100% sensitivity but worse specificity than the 3% algorithm, identifying 7 women who did not meet criteria for OSA by PSG. The ARES 4% algorithm showed poorer sensitivity than the 1% and 3% algorithms; use of the 4% algorithm resulted in failure to identify 2 patients who had PSG AHI ≥ 5/h, one with an AHI of 6.2/h and one with AHI of 6.6/h.

Figure 2. Correlations between PSG AHI and PSG RDI and ARES Unicorder AHIs.

Table 2.

Performance of the ARES device compared with PSG for detecting OSA

When RDI ≥ 5/h of sleep was used to define OSA diagnosis on PSG, the ARES 1% algorithm had the best sensitivity, identifying all patients with a PSG RDI ≥ 5. Not unexpectedly, the ARES 1% algorithm also had the worst specificity—2 women with an ARES 1% AHI ≥ 5/h had PSG RDI < 5. Once again, the ARES 3% algorithm had the best balance between sensitivity and specificity, resulting in one false negative and one false positive compared to PSG RDI. The 4% algorithm had the worst sensitivity, failing to detect 5 women with PSG RDI ≥ 5.

Subjective Experience of the ARES Device

A total of 12 patients filled out the questionnaires. Median scores are detailed in Table 3. Participants reported minimal skin irritation and little noise from the ARES Unicorder.

Table 3.

ARES feedback questionnaire results

DISCUSSION

The goal of this study was to validate the ARES Unicorder, a level 3 device for detecting OSA, in pregnant women. These data demonstrate that the ARES has high sensitivity and fair-to-good specificity for diagnosing OSA in pregnancy. In our sample, the ARES 3% AHI algorithm showed the best balance between specificity and sensitivity. Use of the ARES 3% AHI identified 100% of pregnant women who were found to have OSA by standard overnight PSG with AHI ≥ 5 events/h and 91% when RDI ≥ 5 events/h was used to define an OSA diagnosis.

These findings are consistent with previous data validating the ARES device in non-pregnant adults.^17–19 For instance, although our sample size was smaller than the original validation study, correlations between PSG and ARES measures in our sample were high over a wide range of OSA severity (r = 0.68 to 0.92), comparable to r = 0.88 to 0.96 in the report by Westbrook et al.¹⁷ The ARES Unicorder showed excellent sensitivity for detecting OSA, similar to validation studies in non-pregnant adults that have showed sensitivities of 84% to 97%.^17–19

Our data are also consistent with Louis and colleagues' recent analysis of the performance of the ARES device in pregnant women.²⁴ In their study, which used the ARES 3% AHI, the ARES concordance correlation coefficient with PSG was rho_c = 0.70 (95% CI 0.47–0.92), which is very similar to the agreement observed in the present study (rho_c = 0.74 (95% CI 0.43–0.89).

Specificity of the ARES Unicorder was not as strong as sensitivity. We note that ARES manual recommends that the ARES be worn on two consecutive nights, and we speculate that an increase in the number of nights worn could improve specificity. Data indicate that portable monitors for diagnosing OSA outside the laboratory have failure rates as high as 19%,²⁵ and we note that there were some data collection failures in our study, even in the hands of our experienced polysomnographic technologists. In the case of disagreement in diagnosis when two nights of ARES are performed, a third night may be needed to assist in diagnosis or a laboratory study may be indicated. The American Academy of Sleep Medicine recommends in-laboratory follow-up testing when patients with a high pretest probability of OSA have a negative or technically inadequate home study.²⁵ Given the negative outcomes associated with untreated obstructive sleep apnea for pregnant women and their offspring,^9,13,24 a more inclusive criteria for diagnosing OSA may be desirable—though there are no available data to suggest that treatment of this disorder in pregnancy would modify the risk for these adverse outcomes. The recent change (and subsequent reversal) in AASM respiratory scoring rules with respect to the scoring of hypopneas with 3% desaturations or arousals and elimination of routine reporting of respiratory-effort related arousals²⁶—highlights the need for researchers and clinicians using proprietary software to understand what severity of SDB is indicated by the automatic analyses and output. For example, in their recent validation of the Watch-Pat device in pregnant women, O'Brien and colleagues also found that a cutoff close, but not equal to, the gold-standard cutoff of AHI > 5 events per hour would result in better agreement between PSG and the ambulatory monitor.²⁷

A strength of the study is that we used the gold standard for OSA diagnosis, concomitant laboratory polysomnography, to validate the ARES device. On the other hand, application of the device and all-night monitoring by RPSGTs may have improved data acquisition, and does not provide information about how the ARES device would perform in the homes of pregnant women. Self-application of the ARES device by pregnant patients in their homes may not yield equivalent results and further at-home testing is warranted in this population as has been performed in non-pregnant samples.^17,18

Another limitation is that we studied a small, heterogeneous sample of obese pregnant women and that more than 50% of participants enrolled either dropped out of the study before their sleep study or failed to provide adequate data for analysis. Ten women (27% of consented sample) in whom obstructive sleep apnea was suspected never had a sleep study because they cancelled or did not present to the sleep laboratory for their PSG, highlighting the need for at-home alternatives for evaluating for sleep disordered breathing in this population.

In conclusion, we found the ARES device to be a valid measure for detection of OSA in pregnant women. Out-of-center testing is becoming a more common methodology to improve access and cut costs of evaluation for sleep disordered breathing.^28,29 Furthermore, some studies indicate that pregnant women would prefer home testing rather than stay overnight in the sleep laboratory.²⁷ Our data allow clinicians and researchers to measure sleep disordered breathing with the ARES in this population with more confidence and add to the armamentarium of level 3 devices that have been validated in pregnancy.

DISCLOSURE STATEMENT

Two ARES devices were donated to Dr. Bourjeily by Advanced Brain Monitoring, Inc. for the purpose of this study. The company had no role in designing or conducting the study and did not participate in data analysis, manuscript preparation, or decision to publish this work. The authors have indicated no financial conflicts of interest.