Comparison of polysomnographic and portable home monitoring assessments of obstructive sleep apnea in Saskatchewan women

Abstract

OBJECTIVES:

To compare a commercially available, level III in-home diagnostic sleep test (Embletta, Embletta USA) and in-laboratory polysomnography (PSG) in women with suspected obstructive sleep apnea (OSA).

METHODS:

Consecutive women scheduled for routine PSG testing for evaluation of clinically suspected OSA and who met inclusion/exclusion criteria, were invited to participate. An in-home Embletta portable monitor test was performed one week before or after diagnostic PSG.

RESULTS:

Forty-seven of 96 women who met the inclusion/exclusion criteria agreed to participate. The mean (± SD) age of the patients was 52.0±11.0 years, with a mean body mass index of 34.86±9.04 kg/m², and 66% (31 of 47) of patients were at high risk for OSA according to the Berlin score. Paired analysis of the overall population revealed no significant difference in mean apnea/hypopnea index (AHI) between the two diagnostic methods (P=0.475). At an AHI of ≥5, the Embletta test was highly sensitive (90.6%) in determining abnormal versus normal OSA, with a positive predictive value of 82.7%. However, a higher Embletta AHI threshold of ≥10 may be more useful, with a higher level of agreement (kappa coefficient) with PSG testing and a positive predictive value of 92.3%. The in-home study was less useful at distinguishing severe from nonsevere OSA, yielding a sensitivity of 50%.

CONCLUSIONS:

In women believed to be at high-risk for OSA, Embletta in-home sleep testing is useful for the detection of sleep disordered breathing.

There is increasing recognition that obstructive sleep apnea (OSA) is a relatively common disorder in North America (1–4). OSA has been recognized as contributing to the development of significant comorbidities, in addition to being associated with an increased rate of industrial and motor vehicle accidents (5–7).

The gold standard diagnostic tool for OSA is overnight polysomnography (PSG). With the greater level of awareness of sleep apnea among physicians and the general public, larger numbers of referrals for sleep-physician assessment and for PSG are being generated. Timely access to PSG examination is increasingly limited due to both availability and cost. Appropriately used, in-home level III studies have an advantage in terms of lower costs and earlier access (8,9). It is important to establish comparable diagnostic accuracy; however, studies performed to date have focused on OSA in predominantly male populations (9,10). Initially, OSA was primarily considered to be a disease of men, and was under-recognized in women (1,4). More recently, sex differences in OSA symptomatology have been reported, which may contribute to the underdiagnosis of OSA in women (11,12,13). Sex-specific PSG differences have been recognized including a greater predominance of rapid eye movement-associated sleep abnormalities in women (14) and an overall lower mean apnea/hypopnea index (AHI) (15,16). Most research evaluating level III portable monitoring has been performed in men or, at best, a mixed population with small numbers of women. If in-home evaluation is to be used for female patients, it is important to understand the diagnostic value of a level III, in-home sleep test in women specifically, when compared with the PSG standard. In the present prospective cohort study of Saskatchewan women undergoing evaluation for possible OSA, we compared PSG testing with a commercially available in-home portable diagnostic system (Embletta, Embletta USA).

METHODS

Patient recruitment

Consecutive women scheduled for routine PSG testing for evaluation of clinically suspected OSA were invited to participate in the present study. At the Division of Respiratory, Critical Care and Sleep Medicine, University of Saskatchewan (Saskatoon, Saskatchewan), PSG referrals are restricted to physicians experienced in sleep medicine. All study patients were previously assessed and referred for PSG by a sleep physician.

To be included in the present study, patients were required to be 21 to 70 years of age, and be able to provide informed consent. Exclusion criteria were as follows: referring sleep physician’s strong suspicion of another primary sleep disorder (eg, primary insomnia, narcolepsy, restless legs syndrome, a parasomnia or nocturnal seizures), regular shift work in the previous six months, history of lung disease, congestive heart failure, unstable angina, cerebrovascular accident or pregnancy in the previous six months, neuromuscular disease or renal failure.

Sleep history and examination data collection

Sleep history was obtained from both the patient and from the bed-partner (whenever possible). The Pittsburgh Sleep Quality Index (PSQI) (17), the Berlin Questionnaire for assessment of OSA risk (18) and the Epworth Sleepiness Scale were completed (19). The patient’s age, height, weight and body mass index were recorded.

In-home monitoring for OSA data collection

Unattended in-home monitoring for OSA was performed in random sequence one week before or after PSG testing in all patients. An in-home monitoring device (Embletta, model 2601-1 PDS X10Xact Trace [Embletta, USA]) that monitored oxygen saturation, heart rate, rib cage and abdominal movements, nasal airflow pressure, thermal flow, snoring and body position was used. In the afternoon before the test, an experienced technician taught the patient how to apply the device in the home. A registered sleep technologist, blinded to other patient and PSG data, scored the in-home monitoring tests, which were then reviewed and verified by a single blinded sleep physician. The major outcome measures from the in-home monitoring included the AHI (using the American Academy of Sleep Medicine [AASM] alternative criteria for apnea and hypopnea [50% drop in nasal pressure from baseline and a 3% desaturation]) and the oxygen desaturation index (number of events/h when the oxygen saturation decreases by >3%). The patient recorded the approximate time of sleep onset and awakenings in a sleep log. The sleep log data were used to estimate sleep duration. In the event of a technically suboptimal Embletta study, the study was repeated.

Diagnostic PSG

A 15-lead diagnostic PSG was performed in the sleep laboratory in all cases. PSG recordings included three electroencephalogram leads (C4-A1, C3-A2 and O2-A1), an electro-oculogram (two leads), a submental electromyogram, pulse oximetry, an oronasal airflow pressure sensor, chest and rib cage movement using piezoelectric belts, snore (vibration sensor), intercostal diaphragmatic and anterior tibialis electromyography, and an electrocardiogram. Sleep position was recorded by the sleep technician and confirmed by an all-night infrared video camera. Signals were recorded digitally using Sandman software (Mallinckrodt Inc, Canada). Scoring was performed by a registered sleep technician, and subsequently reviewed and verified by a single blinded sleep physician. Sleep staging was analyzed using AASM criteria. The AHI was determined using the AASM alternative criteria (13), and the results were scored blinded. OSA was defined as >5 apneas/h of sleep (of at least 10 s duration) and/or hypopneas/h of sleep (at least a 50% decrease in flow for at least 10 s duration, with either a 3% decrease in oxygen saturation or a significant activation in the electroencephalogram/h of sleep). An AHI of five to 14 events/h was defined as mild OSA, 15 events/h to 29 events/h as moderate OSA and >30 events/h as severe OSA.

The current study was approved by the University of Saskatchewan Biomedical Ethics Board.

Statistical analysis

SPSS version 17.0 (IBM Corporation, USA) was used for data entry and analysis. Frequencies, variable means and SDs were calculated. Continuous variables were evaluated using paired t tests for comparison between diagnostic tools within individual patients. ANOVA was used to compare continuous data between diagnostic categories. Pearson’s correlation coefficients were used to measure the association between diagnostic test results. Fisher’s exact test was used for comparison of categorical data, while kappa coefficients were calculated for level of agreement between diagnostic tests for categorical assignments. Likelihood ratios were calculated for abnormal study results. A ROC curve was constructed to evaluate the Embletta AHI results versus the gold standard PSG.

Using published sample size calculation formulae (20), the study population size permitted detection of a 15% difference in sensitivity for OSA diagnosis between monitoring techniques.

RESULTS

Consecutive female patients referred by sleep physicians for PSG assessment of possible OSA were screened for potential study participation. Of 96 patients who met the inclusion/exclusion criteria, 47 gave informed consent to participate in the present comparative evaluation of PSG and Embletta sleep studies. The mean (± SD) age was 52±11.0 years (range 29 to 68 years), and 68.1% (n=32) of the cohort were postmenopausal. The mean body mass index was 34.86±9.04 kg/m² (range 23.3 kg/m² to 71.5 kg/m²). The mean Epworth Sleepiness Scale score was 9.64±4.44 (range 0 to 18), the mean PSQI was 8.36±3.69 (range 2 to 19) and 66% (31 of 47) of patients were at high risk for OSA according to the Berlin instrument. The mean AHI determined by PSG was 15.09±16.25, and from the Embletta studies was 16.58±14.64. Figure 1 compares the AHI scores between Embletta and PSG testing. Paired analysis of the overall population revealed no significant difference between the AHI means of the two diagnostic methods (P=0.475). Correlation of AHI values from PSG data to those from Embletta data revealed a significant association, with a Pearson’s coefficient of 0.582 (P<0.001). Three in-home Embletta studies were technically suboptimal and were repeated.

Figure 1).

Comparison of apnea/hypopnea index values between polysomnography (PSG) and Embletta (Embletta, USA) assessments

Using the AHI obtained through the PSG study to categorize patients in terms of OSA presence or severity, 15 of 47 had normal studies (AHI ≤5), 24 of 47 had mild to moderate OSA (AHI >5 and <30) and eight of 47 had severe OSA (AHI ≥30). Mean AHI scores for patients with normal, mild to moderately abnormal and severely abnormal studies (using PSG) are compared for both diagnostic methods in Table 1. Patients were supine for a mean of 33.96±22.61% of sleep time for their Embletta study, and 37.95±25.10% of the time for their PSG (P=0.238).

TABLE 1.

Apnea/hypopnea index values based on patient categorization according to polysomnography results

Category (based on polysomnography)	Polysomnography			Embletta*
	Mean ± SD	95% CI	Range	Mean	95% CI	Range
Normal (n=15)	2.5±1.2	1.86–3.18	0.5–4.5	5.7±4.9	2.95–8.34	0.6–15.4
Mild/moderate obstructive sleep apnea (n=24)	13.7±8.3	10.24–17.23	5.2–29.1	18.7±15.1	12.36–25.11	2.8–68.6
Severe obstructive sleep apnea (n=8)	42.7±17.1	28.5–57.0	30.1–80.9	30.6±10.8	21.6–39.6	13.0–51.7

^*Embletta, USA

There was variation in categorization according to AHI between diagnostic techniques. Table 2 describes the modulation of predictive values, sensitivity, specificity, likelihood ratios and kappa coefficient values when different ‘cut-off points’ for the Embletta AHI were used as diagnostic thresholds compared with a standard PSG AHI of 5. With respect to the capacity to distinguish severe (AHI ≥30 on either Embletta or PSG) from nonsevere OSA, Embletta when compared with PSG yielded a positive predictive value (PPV) of 0.444, a negative predictive value of 0.895, a sensitivity of 0.50, a specificity of 0.872 and an accuracy of 80.9%.

TABLE 2.

Comparison of Embletta^* apnea/hypopnea index (AHI) ‘cut-offs’ with the gold standard polysomnography (abnormal >5)

Embletta AHI	Predictive value		Sensitivity	Specificity	Likelihood ratio		Kappa
	Positive	Negative			Positive	Negative
>5	0.8286	0.7500	0.9063	0.6000	2.266	0.156	0.535
>10	0.9231	0.6190	0.7500	0.8667	5.639	0.288	0.557
>15	0.9524	0.5385	0.6250	0.9330	9.328	0.402	0.467
>20	1.0	0.4688	0.4688	1.0	†	–	0.360
>30	1.0	0.3947	0.2813	1.0	†	–	0.200

^*Embletta, USA;

^†Value = 0, with no cases negative by polysomnography at this AHI level

To further compare the ability to determine levels of test result ‘severity’, the diagnostic threshold of the PSG AHI score from 5 to various levels up to 30 were modified and ROC curves were used to evaluate the Embletta study data. This resulted in areas under the ROC curve ranging from 0.879 to 0.831 (P range <0.001 to 0.002). Figure 2 illustrates the ROC curve generated using the Embletta data as the test variable and using the PSG AHI value of 5 as the diagnostic threshold. The area under that ROC curve was calculated to be 0.879 (95% CI 0.782 to 0.976; P<0.001).

Figure 2).

ROC curve evaluating Embletta (Embletta, USA) apnea/hypopnea results versus polysomnography standard

DISCUSSION

In the present study, we found AHI values from home-based Embletta and sleep laboratory-based PSG studies, in our female population, to be comparable. We evaluated the ability of Embletta to distinguish between abnormal and normal studies using the ‘gold standard’ PSG AHI of 5 as the threshold for an abnormal study. We found that with a low Embletta ‘cut-off’ point AHI of 5, the home-based test had a sensitivity of 0.91, a specificity of 0.60, a PPV and negative predictive value of 0.83 and 0.75, respectively, and a positive likelihood ratio of 2.266. With a higher Embletta cut-off point AHI of 15, the sensitivity dropped to 0.63, specificity climbed to 0.93, the PPV and negative predictive values became 0.95 and 0.54, respectively, and the positive likelihood ratio became 9.328. Depending on the value selected as the cut-off point, it is clear that the sensitivity and specificity of the Embletta test will vary. We also considered the capacity for the Embletta test to distinguish severe OSA as manifested by a PSG AHI of ≥30. For severe OSA detected by the Embletta test, we found a diminished sensitivity of 0.50, with a specificity of 0.84.

In a study population of women, these findings support the recent recommendation of the Canadian Sleep Society/Canadian Thoracic Society that the in-home Embletta study may be a valuable tool for the diagnosis of OSA (21). Optimal use of this diagnostic technique will depend somewhat on its application in an appropriate population. In the current study, our data were based on the examination of a group of women who were recognized by sleep physicians to be at high clinical risk of OSA. In populations with a lower prevalence of OSA, different sensitivity and specificity values would be expected. Similarly, depending on the AHI value selected as the threshold for abnormality, sensitivity and specificity would also vary. A recent study (22) also highlighted the potential for interobserver variation in the assessment of Embletta hypopnea counts, which would add another consideration in practical application of in-home testing methods.

The results of the present study were comparable with those of Dingli et al (9) in their 2003 evaluation of Embletta in-home sleep monitoring. In their recent validation study of Embletta in a predominantly male population, Ng et al (10) found in-home testing to have higher predictive values compared with PSG than those found in the present study. This may, in part, be related to methodology in that our Embletta and PSG studies were not performed simultaneously in the sleep laboratory. Instead, the Embletta was ‘field tested’ in the home within one week of the PSG study. Given the nightly variations in sleep patterns and differing study locations, it is not surprising that the degree of agreement was less than that found in simultaneous studies. Another factor that may contribute to variations in results between PSG and Embletta is the percentage of the night spent supine. Although not a statistically significant difference, a greater percentage of the night was spent supine in the PSG studies than in the Embletta studies.

SUMMARY

The in-home level III Embletta diagnostic test is useful for the diagnosis of OSA in women. Although not reproducing the level of accuracy of PSG, it does approximate that standard. As outlined in Table 2, the predictive capacity of Embletta testing varies with the diagnostic threshold applied. In the present study population, it would appear that using an AHI cut-off of 10 provides the highest level of agreement (kappa coefficient) with PSG testing. This suggests that an Embletta cut-off AHI of 10 is reasonable to balance the test sensitivity and specificity, with a PPV of 92%. Overall, we believe that in women with high clinical suspicion and an AHI ≥10, OSA can be diagnosed with reasonable certainty. These individuals can be started on continuous positive airway pressure testing/therapy, along with appropriate follow-up without the need to further confirm the diagnosis with PSG. Use of the Embletta test will, thereby, reduce the need for PSG testing in patients with a positive in-home study. Recognizing the limited access to and the higher cost of PSG, as well as the morbidity and mortality associated with untreated OSA, appropriate use of the Embletta test can improve sleep health service delivery in women. Future studies of patient populations under ‘field conditions’ will enable streamlining of selection of the most appropriate subjects and diagnostic AHI levels for this testing method.