Comparison of Commonly Used Questionnaires to Identify Obstructive Sleep Apnea in a High-Risk Population

Abstract

Study Objectives:

Sleep apnea is associated with adverse health outcomes. Despite being an important comorbidity in obesity, type 2 diabetes, heart failure, and resistant hypertension, it is underdiagnosed in these patient groups. An inexpensive and readily accessible sleep apnea screening tool would help address this problem. We sought to compare three commonly used screening tools.

Methods:

We recruited 812 patients who had not previously been investigated for sleep apnea from our institution's diabetes (n = 512), obesity (n = 129), resistant hypertension (n = 74) and heart failure (n = 43) clinics. Patients completed three frequently used sleep apnea screening questionnaires (STOP-BANG, Berlin, and OSA50). A total of 758 patients had a valid (> 4 hours' duration) level 3 home sleep study. Studies were reported by a sleep physician and were deemed positive if they recorded a respiratory event index (REI) ≥ 15 events/h.

Results:

The 758 patients with valid sleep studies were age 59 ± 11 years and 63% were male. A total of 38% of patients had a positive test. The respective sensitivities and specificities of the screening questionnaires at the recommended screening thresholds (REI ≥ 15 events/h) were STOP-BANG ≥ 3 (95% and 19%), STOP-BANG ≥ 5 (60% and 69%), Berlin (75% and 38%), and OSA50 (88% and 21%). We identified six independent predictors (age, sex, body mass index, neck circumference, snoring ≥ 3 days per week, observed apnea ≥ 3 days per week). However, combining these factors was no better than the STOP-BANG in predicting sleep apnea. All patients with a STOP-BANG < 3 had an REI < 30 events/h.

Conclusions:

There is a high prevalence of undiagnosed symptomatic sleep apnea in high-risk patient groups. The STOP-BANG questionnaire appeared superior, though all questionnaires had significant limitations. Incorporation of STOP-BANG ≥ 3 in this high-risk population might reduce the need for sleep testing in a resource-constrained setting.

Citation:

Kee K, Dixon J, Shaw J, Vulikh E, Schlaich M, Kaye DM, Zimmet P, Naughton MT. Comparison of commonly used questionnaires to identify obstructive sleep apnea in a high-risk population. J Clin Sleep Med. 2018;14(12):2057–2064.

BRIEF SUMMARY

Current Knowledge/Study Rationale: Obstructive sleep apnea has an important comorbidity in type 2 diabetes, obesity, hypertension, and heart failure; yet, uncertainty exists as to the reliability of questionnaires that could predict who should be considered for sleep testing.

Study Impact: We compared the STOP-BANG with the OSA50 and Berlin Questionnaires in a group of patients attending high cardiovascular risk clinics. The STOP-BANG was found to have a superior sensitivity and negative predictive values.

INTRODUCTION

Obstructive sleep apnea (OSA) is a common sleep disorder and increasingly recognized as having a major effect on an individual's health, quality of life, psychosocial well-being, productivity, and safety.^1,2 In the Wisconsin sleep cohort study the prevalence of OSA (apnea-hypopnea index [AHI] ≥ 15 events/h) was 12% in men and 5% in women aged 30 to 60 years, and OSA was associated with obesity and habitual snoring.³ More recently, the HypnoLaus study from Switzerland found a prevalence of OSA (AHI ≥ 15 events/h) of 50% in men and 23% in women age 40 to 85 years.⁴

OSA and associated sleep disturbance appear to be associated with many chronic conditions such as obesity, type 2 diabetes, hypertension, stroke, heart failure, atrial fibrillation, and coronary artery disease. Studies suggest that comorbid OSA increases morbidity and mortality in these conditions.^5–9 OSA also results in excessive daytime sleepiness, reduced function, and impaired workforce performance.¹⁰ OSA contributes to the pathogenesis of chronic cardiometabolic diseases through a broad range of mechanisms and putative pathways.¹¹ Furthermore, obesity, especially central obesity, is considered to be one of the major contributors to cardiometabolic risk in modern society and the dominant risk factor for OSA in adults.¹² Treatment studies have confirmed improvements in sleepiness, quality of life, and hypertension; there has been no conclusive evidence that other cardiovascular endpoints or rate of survival improve.¹³ Although further studies are required, we believe identification of OSA using simple questionnaires in these high-risk groups is still warranted.

Given the high prevalence and additional burden of OSA in those with the common chronic disorders of type 2 diabetes, severe obesity, resistant hypertension, and heart failure, it is important to have simple inexpensive screening diagnostic questionnaires to identify patients for further sleep monitoring to confirm the diagnosis of OSA.

A number of self-administered questionnaires, using readily available patient responses and clinical measures designed to screen for OSA, are available for assessing high-risk patients.¹⁴ More commonly used instruments include the STOP-BANG,^15–17 the Berlin Questionnaire,¹⁸ and the OSA50.¹⁹ However, in practice it is not known how effective they are in screening for clinically significant OSA, especially in patients with high-risk comorbidities.

In patients with the following high-risk conditions, type 2 diabetes, obesity, resistant hypertension and heart failure, we aimed to:

1. Determine the screening characteristics of the Berlin, STOP-BANG, and OSA50 questionnaires.

2. Identify the best-performing individual items in each of the tested questionnaires and determine if these performed better than the three tested questionnaires.

3. Determine if questionnaires have different screening properties in the different disease groups.

4. Determine if disease-specific features improve screening within that disease group (eg, diabetic complications for people with diabetes, ejection fraction for patients with heart failure patients).

METHODS

We prospectively studied 812 consecutive patients from the type 2 diabetes, obesity, resistant hypertension, and heart failure clinics at the Baker IDI Heart and Diabetes Institute and the Alfred Heart Centre.

Inclusion criteria: documented diagnosis of type 2 diabetes, heart failure, hypertension and obesity; age between 18 and 75 years; no previous diagnosis of OSA; no major life-threatening disease other than type 2 diabetes, heart failure, hypertension, and obesity; no sleep study in the previous 2 years; and willing to undergo study assessments and follow the study protocol. Patients with type 1 diabetes were not included. Patients with obesity and type 2 diabetes, resistant hypertension, or cardiac failure were included in their specific disease group, and not within the obese category. Obesity was defined by body mass index (BMI) > 35 kg/m². Heart failure was confirmed by at least two qualified and independent cardiologists, at least one working within a heart transplant service.

Study Procedures

Participants were assessed for general and disease-specific risk factors for OSA. This included demographics; smoking and alcohol consumption; medical history; anthropometric assessment including height, weight, waist, hip, and neck (measured in the midway point of the neck between midcervical spine and midanterior neck using a nonstretchable tape with individuals standing upright and looking straight ahead with shoulders relaxed) circumferences; and blood pressure. Additional general questions assessed if shift work was undertaken: smoking and alcohol consumption, average hours of sleep per day in the last week, quality of sleep, and two commonly used questions to screen for depression. These questions, taken from the Mini-International Neuropsychiatric Interview²⁰ were: during the past 2 weeks have you (1) felt sad, down, or miserable most of the time; and (2) lost interest or pleasure in most of your usual activities? Information was extracted from medical records on haemoglobin (Hb)A1c, retinopathy, and urinary albumin excretion for those with diabetes, and on left ventricular ejection fraction (LVEF) in those with heart failure. All enrolled participants completed the STOP-BANG,²¹ Berlin,¹⁸ and OSA50¹⁹ questionnaires, independently without assistance from the investigators.

All participants, irrespective of their questionnaire findings, underwent a type 3 home sleep study using the Apnea-Link Plus (ResMed Corporation, Poway, California, United States), which has been shown to be highly sensitive and specific compared to polysomnography.^21–23 This device measures nasal pressure, respiratory effort, oximetry, and heart rate. Home sleep studies occurred within 1 month of initial recruitment. Participants were instructed by a trained study coordinator on the appropriate procedures to carry out the home sleep monitoring. After completing the test, the participants returned the device (in person or by postal mail) and the results of the home sleep study were examined by a qualified sleep physician. The sleep physician reported the studies “blind” with no additional clinical information. Studies with less than 4 hours of adequately recorded flow or oximetry signal were repeated. The presence and severity of sleep apnea was determined by the respiratory event index (REI), which is a measure of the frequency of cessation or near-cessation of breathing during sleep. A hypopnea was defined as a > 30% reduction in ventilation associated with a 4% or more drop in SpO₂. Questionnaires were evaluated at REI cutoffs of ≥ 5, ≥ 15, and ≥ 30 events/h. REI ≥ 15 events/h was used as the predetermined threshold of primary interest in our analysis.

Statistical Analysis

Differences between those with and without an REI ≥ 15 events/h were examined using the chi-square test or independent sample t test where appropriate. Sensitivity, specificity, positive predictive value, and negative predictive values were calculated for the questionnaires using home sleep study results as the standard for diagnosis. A Cox and Snell (C&S) R² was calculated as a measure of overall efficacy for each screening test. Components of all screening questionnaires and all additional variables, including condition specific, collected were tested for a significant univariate association with an REI ≥ 15 events/h. Variables associated with an REI ≥ 15 events/h with a value of P > .1 were entered into a binary logistic regression analysis to identify variables independently contributing to variance of REI ≥ 15 events/h. Receiver operator characteristic (ROC) curves were created by plotting sensitivity against 1 − specificity at various REI cutoff points (5, 15, and 30 events/h) and to assess the total area under the curve for each questionnaire. Analysis was performed using SPSS version 22 (IBM Corp., Armonk, New York, United States). Results are given as mean ± standard deviation unless otherwise stated. No formal power calculations were undertaken.

Ethics approval was obtained from our institution's research and ethics committee (Project Number 239/12).

RESULTS

Of the 812 participants enrolled in the study, 758 (93%) recorded a valid REI. Of these, 512 (68%) had type 2 diabetes, 43 (6%) heart failure, 74 (10%) resistant hypertension, and 129 (17%) obesity without one of the other conditions. Of those 54 patients excluded from analysis 25 decided against undergoing sleep testing while an additional 29 had an invalid test and either refused to repeat the test or had a second invalid test. In these high-risk participants who had not been previously assessed for OSA, the prevalence of REI ≥ 15 events/h was 37.7%, with little difference between the different disease categories (Table 1). None of the patients had central sleep apnea. Symptoms of sleep apnea were commonly reported by patients irrespective of their REI (Table 2).

Table 1.

The characteristics of the 758 participants with valid REI measures obtained with home sleep monitoring.

Table 2.

Prevalence of sleep apnea symptoms in patients classified by REI (n = 758).

In the cohort, 52% were never smokers, 8% were ex-smokers, and 40% were current smokers. A total of 12% reported being shift workers. Mean HbA1c was 7.4% in those with diabetes with 33% reporting retinopathy and 31% reporting microalbuminemia. The patients with heart failure had a mean left ventricular ejection fraction of 28%.

Aim 1: Determine the Screening Characteristics of the Following Questionnaires: Berlin, STOP-BANG, OSA50

None of the examined questionnaires performed well at predicting an REI ≥ 15 events/h at their published cutoffs (Table 3). The negative predictive values varied from 71% to 87% for an REI threshold of 15 events/h, with the STOP-BANG > 3 providing the highest value. Analysis of the area under the curve for the ROC of the tests demonstrated poor performance overall (ie, area under the curve [AUC] < 0.7) for detecting sleep apnea irrespective of the definition used (Table 3, Figure 1); however, there was a statistically significant superiority of the STOP-BANG > 3.

Table 3.

Performance characteristics of the three screening questionnaires and new questionnaire for all participants (n = 758).

Figure 1. Receiver operator curves for the 758 high risk participants in predicting an REI ≥ 15 events/h for each of the screening questionnaires.

Stars represent the previously published cut points used in this study. All-Six represents the curve made by the six factors (age ≥ 50 years, male sex, body mass index ≥ 32 kg/m², neck circumference ≥ 40 cm, snoring ≥ 3 days per week, witnessed apnea ≥ 3 days per week). REI = respiratory event index.

Aim 2: Identify the Best Performing Individual Items in Each of the Tested Questionnaires and Combine Them Into a New Questionnaire

Individual components of all the questionnaires, additional general questions, and disease-specific variables were tested for univariate associations with an REI ≥ 15 events/h (Table 4). A feature of the variables associated with an REI ≥ 15 events/h is the dominance of objective variables rather than self-reported responses. There were just two self-reported responses associated with ≥ 15 events/h: snoring and witnessed apneas on most nights. Tiredness, poor sleep quality, waking frequently, symptoms of depression, sleep time, shift work, alcohol consumption, and falling asleep while driving did not have statistically significant associations with recording an REI ≥ 15 events/h.

Table 4.

Univariate associations of potential risk factors with REI ≥ 15 events/h or greater for all participant, type 2 diabetes, and heart failure populations.

Using binary logistic regression, six variables provided independent explanation of variance of an REI ≥ 15 events/h in the whole study group, and these were: increasing age, male sex, higher BMI, greater neck circumference, and snoring and witnessed apneas most nights (Table 5). These six factors together, however, accounted for only 14% (C&S, R² = 0.14) of variance in REI ≥ 15 events/h. ROC curve-derived cutoff values for the best combination of specificity and sensitivity for each of the three continuous variables we chose were: age 50 years or older, BMI ≥ 32 kg/m², and neck circumference ≥ 40 cm, and these were combined with the binary outcome variables to generate an overall risk score of 0 to 6. The six factors are presented in Table 5 showing univariate, adjusted multivariate (including three continuous variables), and adjusted multivariate using cutoff values for continuous variables. The overall variance explained by the continuous compared with binary cutoff values was equivalent C&S, R² = 0.14 for both. Four of the six independent predictors of an REI ≥ 15 events/h were the same as for the analysis looking at an REI ≥ 5 events/h, with neck circumference and observed sleep apnea most days not influencing the overall outcome, and these four factors explained 14.4% (C&S, R² = 0.144) of variance and were very similar to the variance explained in the primary REI ≥ 15 events/h analysis of 14.0% (C&S, R² = 0.14). There were no additional predictive variables. Five of the six variables were independently associated with an REI ≥ 30 events/h, with male sex no longer a predictor, but the influence of the remaining five factors only explained 7.7% (C&S, R² = 0.077) of overall variance. Table 4 shows that the only self-report symptoms to be associated with sleep apnea were the frequency of snoring and of witnessed apnea.

Table 5.

Odds ratios (95% confidence intervals) for the presence of REI ≥ 15 events/h for the six independently associated factors.

Given the poor overall performance of the questionnaires and our six predictors, the six predictors and the STOP-BANG questionnaire were assessed for their usefulness excluding severe (REI ≥ 30 events/h) sleep apnea. Table 6 demonstrates the percentage of patients with severe sleep apnea (REI ≥ 30 events/h) who would be missed depending on what cutoff score was deemed positive. From this table, 14% of patients had a STOP-BANG cutoff value < 3 of whom none had an REI > 30 events/h.

Table 6.

The proportion of participants above each threshold level of REI for the number of predictors based on REI ≥ 30 events/h analysis.

Aim 3: Determine Whether Questionnaires Have Different Screening Properties in the Different Disease Groups

There was significant variation in the number of participants recruited to each disease group. Moreover, there was significant overlap in disease states across groups; for example, a significant proportion of the diabetes and heart failure populations also had obesity or hypertension. This makes comparisons of different groups, classified by the source of their initial recruitment, problematic. Close examination of the four groups individually did not significantly alter results taken from the cohort as a whole, with most variation likely accounted for by loss of power.

Aim 4: Determine Whether Disease-Specific Features Improve Screening Within That Disease Group

Of the diabetes-specific variables collected only in this group the presence of microalbuminuria was independently associated with a risk of an REI ≥ 15 events/h, with an adjusted odds ratio (OR) of 1.6 (95% confidence interval [CI] 1.0–2.5), and this when combined with sex, BMI, neck circumference, and snoring most nights, generated a C&S R² of 0.12. Using conditional stepwise loading of variables, microalbuminuria loaded last and increased the overall R² by 0.008. None of the other diabetes-specific measures contributed.

When LVEF was added to the six general variables for the heart failure subgroup, only neck circumference > 40 cm was predictive of REI ≥ 15 events/h with an individual adjusted OR of 8.4, (CI 2.0–34), with C&S R² of 0.21.

DISCUSSION

This study confirms the previously reported high prevalence of undiagnosed moderate to severe sleep apnea (REI ≥ 15 events/h) in the high cardiovascular risk groups examined, of 35%. Those with sleep apnea were highly symptomatic, with most patients describing snoring and daytime somnolence. However, a large proportion of patients without significant sleep apnea also reported these symptoms, resulting in the poor specificity of the questionnaires. This may be explained by the relatively short self-reported sleep duration of 6.6 hours per night. The questionnaires were highly sensitive, especially for REI ≥ 30 events/h, suggesting they could be used for excluding this high level of sleep apnea. Specifically, in what is to our knowledge the largest comparative study of three widely used screening tools, we found that a STOP-BANG ≥ 3 was the most effective. Moreover, as 14% of patients with STOP-BANG < 3 had an REI < 30 events/h, the STOP-BANG questionnaire may have utility in prioritizing patients in a resource-scarce environment.

Similar to our findings, Pataka et al. reported the STOP-BANG to have a high sensitivity and negative predictive value in addition to being superior to the Berlin Questionnaire.²⁴ Compared with our data, the sensitivity (97% versus 99%) and negative predictive values (97% versus 88%) and ROC (0.67 versus 0.72) were similar to those reported by Pataka et al. However, important differences were that we found a lower positive predictive value (15% versus 53%) possibly because our population was attending a series of high cardiovascular risk clinics in contrast to the study by Pataka et al., in which patients were chosen from a sleep clinic population. Although both the study by Pataka et al. and our studies had large sample sizes (1,853 and 758 patients), important differences were the retrospective versus prospective study design and sleep study type (laboratory polysomnography versus home limited channel), respectively.

We identified six general factors (increasing age, male sex, BMI, increasing neck circumference, snoring ≥ 3 days per week, witnessed apnea ≥ 3 days per week) and one diabetes-specific factor (microalbuminuria) that were independently predictive of REI > 15 events/h. Of note, only two of these factors are subjective in nature. Unfortunately, even within our cohort, combining these factors produced a model that did not have adequate sensitivity and specificity to reliably predict sleep apnea. Our six predictive factors have elements that are consistent with numerous other studies and are very easy to obtain in any clinical setting.^25,26 The literature demonstrates limited validation of the commonly used screening tests with high sensitivity counterbalanced by poor specificity, and a systematic review indicated a pooled sensitivity of 77.0%, and pooled specificity of 53.0%, 50%, and 57% in participants without a history of sleep disorders.^14,27 Indeed, performance characteristics were similar to those obtained in our study.

The absence of a reliable screening questionnaire for sleep apnea in these clinical categories raises the issue of which of these patients should undergo formal physiological testing for sleep apnea, either through in-laboratory or home-based testing. The two main issues regarding this would be a need for evidence of improved outcomes from treating sleep apnea in these high-risk patients and a cost-effective method of identifying those patients most likely to have sleep apnea.

The role of any screening questionnaire would be to reduce the number of physiological tests required by excluding those unlikely to have severe or symptomatic disease. Our data suggest that the STOP-BANG cutoff of ≥ 3 would detect 97% of those patients with an REI ≥ 30 events/h while reducing the number of follow-up tests required by 14% (those patients with a score of 0–2). Alternatively, a cutoff score of ≥ 4 would miss 12% of severe sleep apnea while avoiding further testing in 33% of the population. Thus, cutoff scores could assist those with limited resources.

Limitations

We recognize several limitations to this study. The patients recruited came from a specialist high cardiovascular clinic population, and as such the results may not apply more widely to patients with high-risk conditions in primary care. The population had a mean self-reported sleep duration of 6.6 hours, indicating the likelihood of sleep deprivation. The questionnaires do not cover all symptoms of OSA. Moreover, symptoms do not equate with REI. Additionally, we used the ApneaLink Plus device as opposed to the gold standard in laboratory polysomnography. Although the ApneaLink Plus has been validated in standard OSA populations,²² its use in patients with comorbidi-ties such as heart failure is not as well studied. There were relatively small patient numbers in the heart failure and resistant hypertension groups and this makes extending conclusions from the total cohort to these groups less reliable. However, we do note that there was very little difference between the groups in terms of sleep apnea characteristics or questionnaire performance. We did not determine the Epworth Sleepiness Scale (ESS) score in our patients; this is a commonly used questionnaire to determine sleepiness in patients suspected of having OSA. Studies, including our own,²⁸ have shown the ESS to be a poor predictor of sleep apnea severity and to more accurately predict mood disturbance such as depression. We suggest that, much like the other symptom-based questions used, the ESS questions would have been nonspecific with regard to OSA in this patient group. We also used various REI thresholds of 5, 15, and 30 events/h. Finally, we did not record whether participants had a regular bed partner.

CONCLUSIONS

We have shown that the currently available, and frequently used, questionnaires can identify those patients most likely to have OSA in our high cardiovascular risk population and may assist in this process by excluding those without significant symptomatic sleep apnea and thus reduce the number of tests required.

DISCLOSURE STATEMENT

Work for this study was performed at the Baker Heart and Diabetes Institute, Melbourne, Australia. All authors have read and approved the contents of this manuscript. Funding was provided as an unrestricted grant from the BUPA Health Foundation. The funder had no input into the collection, analysis and interpretation of the data; in the writing of the report; nor in the decision to submit the paper for publication. The authors report no conflicts of interest.

ABBREVIATIONS

AHI

apnea-hypopnea index

BMI

body mass index

C&S

Cox and Snell

LVEF

left ventricular ejection fraction

OSA

obstructive sleep apnea

OSA50

Obesity, Snoring, Apneas, > 50 years

REI

respiratory event index

ROC

receiver operating characteristic

STOP-BANG

Snoring, Tired, Observed apneas, Pressure, BMI, Age, Neck size, Gender