Abstract
Background:
The international classification of sleep disorders (ICSD)-3 was developed to aid in the identification of these disorders. The core criteria A (ICSD-3A) to identify obstructive sleep apnea (OSA), requires the presentence of specific signs and symptoms. This study explores the predictive ability of the ICSD-3A for OSA as compared to objective measures of respiratory event index (REI).
Participants:
A total of 291 participants who completed a home sleep apnea test (HSAT) during the screening evaluation of the Assessing Daily Activity Patterns through occupational Transitions (ADAPT) study, were included.
Methods:
Participants were classified as having mild OSA (REI ≥ 5 and <15), moderate (≥ 15 to < 30), or severe OSA (> 30). Predictive parameters identifying participants as having OSA by the ICSD-3A criteria were assessed using REI classifications as the reference standard, and further compared to a subsample using the STOP-Bang questionnaire.
Results:
The ICSD-3A had a sensitivity of 19.2% for identifying participants as having moderate to severe OSA and specificity of 84.4%. The ICSD-3A had a ROC = 0.53. On the subsample of participants, the STOP-Bang questionnaire’s ROC = 0.61. Results were similar when examining classification of participants with mild compared to no OSA.
Conclusion:
In this population, the ability of the ICSD-3A in detecting moderate to severe OSA as well as mild OSA was low. The ROC for the ICSD-3 did not differ significantly from the STOP-Bang questionnaire’s ROC in this research population.
Keywords: Obstructive sleep apnea, sleep, job loss, international classification of sleep disorders
Introduction
The ICSD-3 diagnostic standards criterion include a section A (ICSD-A) for obstructive sleep apnea (OSA) and requires signs and symptoms related to associated sleepiness, fatigue, insomnia, snoring, subjective nocturnal respiratory disturbance, and observed apnea, or associated medical or psychiatric disorders [1]. Previous research however, suggests that these scoring criteria can affect the prevalence of OSA. Heinzer et al. [2] found that prevalence of OSA using the ICSD-3 criteria was 74% for men aged 40 years and older. Adams, et al.[3] reported prevalence of OSA in 52.2% of men using the ICSD-3A criteria, and that prevalence of one or more comorbidities were present in at least 97% of men previously undiagnosed with OSA, suggesting the ICSD-3A criteria to be too inclusive.
OSA is a common, but often undiagnosed chronic breathing disorder caused by episodes of airflow reduction during sleep [4]. Inadequate pharyngeal and genioglossal muscle function results in repeated upper airway collapse which block the airway leading to partial reduction in breathing (hypopneas) or complete pauses (apneas). Brief arousals from sleep are required to restore breathing. Patients with OSA are at increased risk of daytime sleepiness, hypertension, heart disease, depression, and higher rates of motor vehicle accidents among others [5-9]. The estimated prevalence of moderate to severe OSA, defined by an apnea hypopnea index (AHI) of ≥ 15 events per hour was found to be 17% among males, and 9% among females 50 – 70 years old.[10]. A large percentage of people with OSA may be unaware they have this condition, although they may experience one or several of OSA symptoms. Estimates show that approximately 80% of people with moderate to severe OSA are undiagnosed [10, 11].
Frequently, epidemiological sleep studies require assessment of OSA in potential participants. Polysomnography is the reference standard to diagnose OSA [12]. However, this procedure is costly and labor-intensive and thus, not routinely an option for epidemiological studies looking to exclude subjects with OSA. Several questionnaires have been developed to screen potential participants for OSA [13-17]. However, some of these questionnaires are lengthy or may require additional objective evaluations such as blood pressure or may require complicated scoring computations. The ICSD-3A criteria can be used to screen potential participants for OSA. However, the effect of this criteria on sensitivity and specificity in research populations has not been generally evaluated.
The objective of the present study is to explore the sensitivity and specificity of the ICSD-3A criteria in screening for OSA as compared to objective measures of the respiratory event index (REI) obtained using a home sleep apnea test (HSAT) and to compare these parameters to the STOP-Bang questionnaire [18], a commonly used screening instrument.
Methods
Participants and Procedure
Data analyzed for this study came from an ongoing investigation, the Assessing Daily Activity Patterns through occupational Transitions (ADAPT) project which prospectively examined the relationship between daily activity patterns and weight gain following involuntary job loss among adults ages 25-60 years. Further details of this study have been published elsewhere [19]. In brief, recruitment began in October 2015, primarily through weekly fliers sent by the Arizona Department of Economic Security to individuals applying for unemployment insurance in Tucson, Arizona. A flow diagram has been previously published[20], briefly a total of 446 individuals completed an in-person screening; of these 155 did not meet inclusion criteria, met the exclusion criteria, or withdrew after consent. Eligible criteria included, age between 25 and 60 years, be able to speak, write, and understand English fluently, worked at their previous job for at least 6 months for 30 or more hours per week, and had been laid off or terminated from their place of employment within the last 90 days. Participants were excluded for engaging in any treatments likely to impact weight gain or energy balance-related behaviors, taking medications for sleep, history of sleep disorder diagnosis including OSA, engaging in shift work within the past 30 days, or having a substance abuse or dependence. A comprehensive list of inclusion and exclusion criteria has been previously published [19]. Participants in the current study (N = 291) include those who completed the screening visit for ADAPT study. During the screening visit, participants were administered a screening interview and questionnaire ascertaining demographic information as well as the Duke structured interview for sleep disorders (DSISD) and a HSAT [21]. The ICSD-3A criteria is included as part of the DSISD questionnaire. The demographic questionnaire included questions such as age, gender, ethnic background, and self-reported height and weight. Body mass index (BMI) was computed as kg/m2 using self-reported indices of height and weight. We found no statistically significant differences for age, income, gender, ethnicity, or BMI categories between excluded and included individuals. Therefore, we presume no selection bias was present.
Study data from the screening assessment were collected and managed using the research electronic data capture (REDCap). REDCap is a secure, web-based application designed to support data capture for research studies, providing an intuitive interface for validated data entry, audit trails for tracking data manipulation and export procedures, and procedures for importing data from external sources. The ADAPT project protocol was approved by the University of Arizona’s Institutional Review Board, and informed consent was obtained from all study participants at enrollment.
Measures
Duke Structured Interview for Sleep Disorders (DSISD) tool.
The Duke Structured Interview for Sleep Disorders [22] is a clinical semi-structured interview developed to assess sleep disorder symptoms according to the international classification of sleep disorders (ICSD-3) and the diagnostic and statistical manual of mental disorders 4th edition, text revision (DSM-IV-TR) criteria [23]. For the current study, the ICSD-3A criteria from the DSISD were used. During the interview, participants are asked a series of questions related to possible sleep disturbances. Sections of the questionnaire are skipped if the participant endorses negative answers to screening questions. The sleep apnea syndromes section is asked if participants endorse positive answers to screening questions that meet criteria to advance to this section. In the DSISD Sleep Apnea section, OSA is assessed by seven clinical symptoms consistent with ICSD-3A criteria: a) loud snoring; b) gasping or choking in sleep; c) breathing interruption in sleep; d) holding your breath while sleeping; e) poor, unrefreshing sleep even after an adequate night’s sleep; f) diagnosis of hypertension, a mood disorder, cognitive dysfunction, coronary artery disease, stroke, congestive heart failure, atrial fibrillation, or type 2 diabetes mellitus; and g) sleepiness, fatigue, or insomnia. OSA is considered present by ICSD-3A criteria if participants endorse one or more symptoms from a – g. Participants must also meet the following criterion: sleep disorder is not better explained by another sleep disorder, medical or neurological disorder, psychiatric disorder, medication, or substance use or exposure. An additional question asks participants if these symptoms happen in the last month or in the past (not current). For the purpose of this study, participants who did not meet criteria to advance to the sleep apnea syndromes section or did not endorse positive answers for OSA were considered to screen negative for OSA. Participants were considered to screen positive for OSA if they met ICSD-3A criteria and they reported the symptoms happened within the last month.
The DSISD was administered by research staff trained in sleep disorder diagnosis, who met reliability levels of 75% with a licensed clinician (PH).
ApneaLink Plus.
A home sleep apnea test (HSAT) was performed using the ApneaLink Plus™, a screening device that records respiratory effort, derived heart rate, pulse oxygen saturation and nasal flow to generate an objective REI. Participants were given instructions on the use of the device during the screening visit and wore it at home that night while sleeping and returned it the following day. The Apnealink has a high level of sensitivity (>80%) at all REI levels [24] and a positive predictive value of greater than 80% at REI levels of 10 or more compared to in-lab sleep studies [25]. Raw ApneaLink data was scored by a registered polysomnographic technologist and reviewed by a sleep board-certified sleep medicine physician (SQ) according to the standards published by the American Academy of Sleep Medicine [26]. Categorical REI cutoffs for initial analysis for OSA severity were done at < 5 (no), 5 - < 15 (mild), 15 - < 30 (moderate), and ≥ 30 (severe). Additionally, REI cutoffs for OSA severity at < 15 (none to mild) and ≥ 15 (moderate to severe) were used for sensitivity and specificity analyses.
STOP-Bang questionnaire.
The STOP-Bang questionnaire [14, 18], was added to the ADAPT screening visits protocol later in the study as a supplement screening questionnaire for OSA. The STOP-Bang questionnaire was administered parallel with the DSISD to further enrolling participants (n = 96), and thus participants recruited initially did not have the STOP-Bang questionnaire. The STOP-Bang questionnaire evaluates eight risk factors for OSA as affirmative or negative answers to the following items: snoring (S), tiredness (T), observed apneas (O), blood pressure (P), body mass index > 35 kg/m2 (B) age > 50 years (A), neck circumference ≥ 43 cm for males, or ≥ 41 cm for females (N), and being male gender (G). Neck circumference (cm), and height (cm) were measured with a Gullick II tape measure and a stadiometer, using standardized study protocols [19]. Objective BMI (kg/m2) was also obtained using the InBody270 bioelectrical impedance analyzer, which computes a measure of body composition variables after weight is obtained and entered in the analyzer along with height, age, and sex. For scoring of the STOP-Bang questionnaire, one point was assigned for each affirmative answer, and zero for negative answers. Low risk for OSA was defined as ≤ 2 affirmative answers to the four STOP items, and intermediate risk was defined as 3-4 affirmative answers to the four STOP items [14]. High risk for OSA was defined as any of the following: 1) ≥ 5 affirmative answers to the eight STOP-Bang items, 2) yes to ≥ 2 of the four STOP questions and being male gender, 3) yes to ≥ 2 or more of the four STOP questions and having a BMI > 35kg/m2, 4) yes to ≥ 2 or more of the four STOP questions and having a neck circumference ≥ 43cm in males or ≥ 41cm in females [18]. Participants were initially categorized as having low, intermediate, or high risk for OSA using the STOP-Bang scoring. However, only one participant was categorized as having intermediate risk for OSA, thus scores were dichotomized as low risk and intermediate to high risk for OSA.
Data Analysis
Descriptive characteristics were examined overall and by REI OSA severity category. Demographic variables were described using frequencies or means as appropriate. Chi-square tests were used to compare differences in proportions between categorical variables by OSA categories. Student’s t-test or one-way analysis of variance (ANOVA) were used to compare differences in mean values for continuous variables by OSA categories.
The predictive abilities of the ICSD-3A criteria were compared to objective-OSA derived categories using the following parameters: sensitivity, specificity, percent correctly classified, and likelihood ratio tests for positive and negative results. Logistic regression models were used to calculate the odds ratios of the ICSD-3A criteria to predict objective OSA categories. The Receiver Operating Characteristics (ROC) area under the curve (AUC) was used to evaluate the ICSD-3A’s ability to correctly identify persons classified into mild and moderate to severe OSA by REI [27, 28]. In further analyses, the ICSD-3A predictive ability for OSA was compared to that of the STOP-Bang questionnaire on the subsample of participants who were administered this questionnaire. Finally, the area under ROC curve for the ICSD-3A criteria was compared to the STOP-Bang’s ROC, testing for differences between correlated ROC curves [27].
Because self-reported weight was collected from participants before the inclusion of the STOP-Bang questionnaire, we examined the correlation between self-reported BMI and the objectively measured BMI in this subsample. Statistical analyses were conducted using Stata version 15 software (Stata Corp, College Station, TX). For all analyses, a p-value < 0.05 was considered statistically significant.
Results
Descriptive characteristics of the overall study sample for no, mild, moderate, and severe OSA are reported in Table 1. The overall sample mean age was 41.1 years (SD = 10.3), 43.3% were male, 35.4% were Hispanic, 68.8% were considered overweight or obese, and the mean monthly income was $1,758.51 (SD = $2,068.93). Participants were classified as having no OSA (56%), mild (28%), moderate (10%), and severe OSA (6%). Significant group differences by OSA categories were found for age, gender, ethnic background, BMI category, and the loud snoring symptom (p < 0.05).
Table 1.
Descriptive characteristics by no, mild, moderate, and severe OSA from HSAT REI [n (%)].§
| Total N = 291 |
No OSA N = 163 (56%) |
Mild OSA N = 81 (28%) |
Moderate OSA N = 29 (10%) |
Severe OSA N = 18 (6%) |
p-value* | |
|---|---|---|---|---|---|---|
| Age [mean (SD)] | 41.1 (10.3) | 38.8 (9.8) | 44.2 (10.5) | 42.7 (8.3) | 45.8 (11.0) | <0.001 |
| Monthly income [mean (SD)] | 1,758.51 (2,068.93) | 1,716.78 (1,972.60) | 1,899.78 (2,340.92) | 1,520.00 (1,650.76) | 1,898.40 (2,356.86) | 0.83 |
| Gender | <0.001 | |||||
| Male | 126 (43.3%) | 54 (33.1%) | 40 (49.4%) | 19 (65.5%) | 13 (72.2%) | |
| Female | 165 (56.7%) | 109 (66.9%) | 41 (50.6%) | 10 (34.5%) | 5 (27.8%) | |
| Ethnic background | 0.002 | |||||
| Not Hispanic or Latino | 188 (64.6%) | 113 (69.3%) | 51 (63.0%) | 10 (34.5%) | 14 (77.8%) | |
| Hispanic or Latino | 103 (35.4%) | 50 (30.7%) | 30 (37.0%) | 19 (65.5%) | 4 (22.2%) | |
| Race | ||||||
| AI or AN | 13 (4.7%) | 2 (1.3) | 7 (9.2) | 3 (11.1) | 1 (5.9) | |
| Asian | 1 (0.3%) | 0 | 1 (1.3) | 0 | 0 | |
| B or AA | 24 (8.7%) | 16 (10.3) | 5 (6.6) | 2 (7.4) | 1 (5.9) | |
| NH or OPI | 1 (.3%) | 1 (.6) | 0 | 0 | 0 | |
| White | 197 (71.4%) | 119 (76.3) | 49 (64.5) | 16 (59.3) | 13 (76.5) | |
| Other | 23 (8.4%) | 9 (5.8) | 8 (10.5) | 4 (14.8) | 2 (11.7) | |
| More than one | 17 (6.2%) | 9 (5.7) | 6 (7.9) | 2 (7.4) | 0 | 0.3 |
| BMI Category (from self-report) | ||||||
| Underweight | 5 (1.8%) | 5 (3.1%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) | <0.001 |
| Normal | 84 (29.5%) | 64 (39.8%) | 11 (14.1%) | 5 (17.9%) | 4 (22.2%) | |
| Overweight | 76 (26.7%) | 39 (24.2%) | 27 (34.6%) | 5 (17.9%) | 5 (27.8%) | |
| Obese | 120 (42.1%) | 53 (32.9%) | 40 (51.3%) | 18 (64.3%) | 9 (50.0%) | |
| Loud snoring | ||||||
| No | 224 (77.0%) | 139 (85.3%) | 56 (69.1%) | 16 (55.2%) | 13 (72.2%) | <0.001 |
| Yes | 67 (23.0%) | 24 (14.7%) | 25 (30.9%) | 13 (44.8%) | 5 (27.8%) | |
| Gasping/choking in sleep | ||||||
| No | 275 (94.5%) | 156 (95.7%) | 76 (93.8%) | 25 (86.2%) | 18 (100.0%) | 0.14 |
| Yes | 16 (5.5%) | 7 (4.3%) | 5 (6.2%) | 4 (13.8%) | 0 (0.0%) | |
| Breathing interruptions in sleep | ||||||
| No | 272 (93.5%) | 157 (96.3%) | 74 (91.4%) | 25 (86.2%) | 16 (88.9%) | 0.12 |
| Yes | 19 (6.5%) | 6 (3.7%) | 7 (8.6%) | 4 (13.8%) | 2 (11.1%) | |
| Holding your breath while sleeping | ||||||
| No | 279 (95.9%) | 157 (96.3%) | 77 (95.1%) | 27 (93.1%) | 18 (100.0%) | 0.67 |
| Yes | 12 (4.1%) | 6 (3.7%) | 4 (4.9%) | 2 (6.9%) | 0 (0.0%) | |
| Poor, unrefreshing sleepa | ||||||
| No | 225 (77.3%) | 124 (76.1%) | 65 (80.2%) | 22 (75.9%) | 14 (77.8%) | 0.90 |
| Yes | 66 (22.7%) | 39 (23.9%) | 16 (19.8%) | 7 (24.1%) | 4 (22.2%) | |
| Diagnosis of comorbidityb | ||||||
| No | 273 (93.8%) | 156 (95.7%) | 76 (93.8%) | 24 (82.8%) | 17 (94.4%) | 0.068 |
| Yes | 18 (6.2%) | 7 (4.3%) | 5 (6.2%) | 5 (17.2%) | 1 (5.6%) | |
| Sleepiness, fatigue, or insomnia | ||||||
| No | 215 (73.9%) | 121 (74.2%) | 60 (74.1%) | 20 (69.0%) | 14 (77.8%) | 0.92 |
| Yes | 76 (26.1%) | 42 (25.8%) | 21 (25.9%) | 9 (31.0%) | 4 (22.2%) | |
| No symptoms | ||||||
| No | 273 (93.8%) | 154 (94.5%) | 74 (91.4%) | 28 (96.6%) | 17 (94.4%) | 0.72 |
| Yes | 18 (6.2%) | 9 (5.5%) | 7 (8.6%) | 1 (3.4%) | 1 (5.6%) | |
| ICSD-3A positive screen | ||||||
| No | 244 (83.8%) | 142 (87.1%) | 64 (79.0%) | 24 (82.8%) | 14 (77.8%) | 0.36 |
| Yes | 47 (16.2%) | 21 (12.9%) | 17 (21.0%) | 5 (17.2%) | 4 (22.2%) | |
Mean and standard deviations (SD) when noted.
Even after an adequate night's sleep
Hypertension, a mood disorder, cognitive dysfunction, coronary artery disease, stroke, congestive heart failure, atrial fibrillation, or type 2 diabetes mellitus.
P-values for Chi-square test for categorical variables, and ANOVA for continuous variables (age and monthly income). P-values <.05 are bolded. 6 participants were missing information on BMI. REI (respiratory event index), HSAT (home sleep apnea test). AI (American Indian), AN (Alaska Native), B (Black), AA (African American), NH (Native Hawaiian), OPI (Other Pacific Islander).
Descriptive characteristics of the study sample using combined REI categories of none to mild (84%, n = 244) and moderate to severe OSA (16%, n = 47) are reported in Table 2. Significant group differences were found by age, gender, ethnic background, loud snoring and diagnosis of comorbidity symptoms for the ICSD-3A criteria. Those with moderate to severe OSA were older (Mean age = 43.9, SD = 9.4) compared to those with none to mild OSA (Mean age = 40.6, SD = 10.4). A higher percentage of males (68.1%) had moderate to severe OSA compared to females (31.9%). Loud snoring was reported by 38.3% of those with moderate to severe OSA, and 12.8% reported having a comorbidity (i.e. hypertension, a mood disorder, cognitive dysfunction, coronary artery disease, stroke, congestive heart failure, atrial fibrillation, or type II diabetes mellitus). As assessed by the DSISD, 19.1% of participants met ICSD-3A criteria for OSA. In the subsample of participants who were administered the STOP-Bang questionnaire, statistical differences were seen by REI OSA categories for the following symptoms: snoring, tired, observations of breathing cessation during sleep, neck circumference > 40 cm, and gender (Table 3). A greater proportion of participants classified as having severe OSA reported snoring (60%), observed sleep apnea (40%), having a neck circumference > 40 cm (75%), and being male (100%). However, a greater proportion of those with no OSA reported feeling tired, fatigued, or sleepy during daytime (67%) in comparison to mild or moderate, but not severe OSA. The percent of participants that were classified as positive by the ICSD-3A and as high risk by the STOP-Bang questionnaires was 5.21% (total N=96).
Table 2.
Descriptive characteristics by no to mild and moderate to severe OSA from HSAT REI [n (%)].§
| None to mild OSA N = 244 (84%) |
Moderate to severe OSA N = 47 (16%) |
p-value* | |
|---|---|---|---|
| Age [mean (SD)] | 40.59 (10.35) | 43.87 (9.41) | 0.044 |
| Monthly income [mean (SD)] | 1,779.36 (2,102.52) | 1,649.00 (1,901.55) | 0.70 |
| Gender | <0.001 | ||
| Male | 94 (38.5%) | 32 (68.1%) | |
| Female | 150 (61.5%) | 15 (31.9%) | |
| Ethnic background | 0.034 | ||
| Not Hispanic or Latino | 164 (67.2%) | 24 (51.1%) | |
| Hispanic or Latino | 80 (32.8%) | 23 (48.9%) | |
| BMI Category | 0.076 | ||
| Underweight | 5 (2.1%) | 0 (0.0%) | |
| Normal | 75 (31.4%) | 9 (19.6%) | |
| Overweight | 66 (27.6%) | 10 (21.7%) | |
| Obese | 93 (38.9%) | 27 (58.7%) | |
| Loud snoring | 0.007 | ||
| No | 195 (79.9%) | 29 (61.7%) | |
| Yes | 49 (20.1%) | 18 (38.3%) | |
| Gasping/choking in sleep | 0.32 | ||
| No | 232 (95.1%) | 43 (91.5%) | |
| Yes | 12 (4.9%) | 4 (8.5%) | |
| Breathing interruptions in sleep | 0.059 | ||
| No | 231 (94.7%) | 41 (87.2%) | |
| Yes | 13 (5.3%) | 6 (12.8%) | |
| Holding your breath while sleeping | 0.96 | ||
| No | 234 (95.9%) | 45 (95.7%) | |
| Yes | 10 (4.1%) | 2 (4.3%) | |
| Poor, unrefreshing sleepa | 0.90 | ||
| No | 189 (77.5%) | 36 (76.6%) | |
| Yes | 55 (22.5%) | 11 (23.4%) | |
| Diagnosis of comorbidityb | 0.041 | ||
| No | 232 (95.1%) | 41 (87.2%) | |
| Yes | 12 (4.9%) | 6 (12.8%) | |
| Sleepiness, fatigue, or insomnia | 0.79 | ||
| No | 181 (74.2%) | 34 (72.3%) | |
| Yes | 63 (25.8%) | 13 (27.7%) | |
| No symptoms | 0.55 | ||
| No | 228 (93.4%) | 45 (95.7%) | |
| Yes | 16 (6.6%) | 2 (4.3%) | |
| DSI ICSD-3 positive screen | 0.54 | ||
| No | 206 (84.4%) | 38 (80.9%) | |
| Yes | 38 (15.6%) | 9 (19.1%) |
Mean and standard deviations (SD) when noted.
Even after an adequate night's sleep
Hypertension, a mood disorder, cognitive dysfunction, coronary artery disease, stroke, congestive heart failure, atrial fibrillation, or type II diabetes mellitus.
P-values for Chi-square test for categorical variables, and t-test for continuous variables (age and monthly income). P-values <.05 are bolded. REI (respiratory event index), HSAT (home sleep apnea test).
Table 3.
STOP-Bang item characteristics by OSA categories from HSAT REI [n (%)].§
| Total N=96 |
No OSA N=54 |
Mild OSA N=26 |
Moderate OSA N=11 |
Severe OSA N=5 |
p-value | |
|---|---|---|---|---|---|---|
| 1. Snoring. Do you snore loudly (louder than talking or loud enough to be heard | <0.001 | |||||
| No | 64 (67%) | 44 (81%) | 16 (62%) | 2 (18%) | 2 (40%) | |
| Yes | 32 (33%) | 10 (19%) | 10 (38%) | 9 (82%) | 3 (60%) | |
| 2. Tired. Do you often feel tired, fatigued, or sleepy during daytime? | 0.013 | |||||
| No | 43 (45%) | 18 (33%) | 15 (58%) | 5 (45%) | 5 (100%) | |
| Yes | 53 (55%) | 36 (67%) | 11 (42%) | 6 (55%) | 0 (0%) | |
| 3. Observed. Has anyone observed you stop breathing during your sleep? | 0.034 | |||||
| No | 89 (93%) | 51 (94%) | 25 (96%) | 10 (91%) | 3 (60%) | |
| Yes | 7 (7%) | 3 (6%) | 1 (4%) | 1 (9%) | 2 (40%) | |
| 4. Blood pressure. Do you have or are you being treated for high blood pressure? | 0.89 | |||||
| No | 79 (82%) | 44 (81%) | 21 (81%) | 10 (91%) | 4 (80%) | |
| Yes | 17 (18%) | 10 (19%) | 5 (19%) | 1 (9%) | 1 (20%) | |
| 5. BMI. BMI more than 35 kg/m2? | 0.31 | |||||
| No | 74 (79%) | 46 (85%) | 19 (73%) | 7 (64%) | 2 (67%) | |
| Yes | 20 (21%) | 8 (15%) | 7 (27%) | 4 (36%) | 1 (33%) | |
| 6. Age. Age over 50 yrs. old? | 0.48 | |||||
| No | 75 (78%) | 43 (80%) | 19 (73%) | 10 (91%) | 3 (60%) | |
| Yes | 21 (22%) | 11 (20%) | 7 (27%) | 1 (9%) | 2 (40%) | |
| 7. Neck circumference. Neck circumference greater than 40 cm? | <0.001 | |||||
| No | 69 (73%) | 48 (89%) | 16 (62%) | 4 (36%) | 1 (25%) | |
| Yes | 26 (27%) | 6 (11%) | 10 (38%) | 7 (64%) | 3 (75%) | |
| 8. Gender. Gender male? | 0.003 | |||||
| No | 53 (55%) | 37 (69%) | 13 (50%) | 3 (27%) | 0 (0%) | |
| Yes | 43 (45%) | 17 (31%) | 13 (50%) | 8 (73%) | 5 (100%) | |
| Neck circumference (cm) | 37.34 (4.83) | 35.41 (3.58) | 39.37 (5.64) | 41.09 (4.72) | 39.90 (1.83) | <0.001 |
| Height (cm) | 168.84 (9.61) | 167.62 (9.02) | 168.72 (10.47) | 171.75 (9.91) | 178.20 (6.24) | 0.13 |
| BMI | 30.17 (6.81) | 28.51 (6.58) | 31.68 (5.85) | 34.03 (7.87) | 32.70 (8.88) | 0.035 |
Data are presented as mean (SD) for continuous measures, and n (%) for categorical measures.
P-values for Chi-square test for categorical variables and t-test for continuous variables (neck, height, and BMI). P-values <.05 are bolded. REI (respiratory event index), HSAT (home sleep apnea test).
Predictive parameters comparing the ICSD-3A criteria to the STOP-Bang with objective measures of REI for mild OSA are reported in Table 4. The ICSD-3A criteria had a sensitivity of 20.9% to identify participants with mild OSA, a specificity of 87.1%, and a ROC AUC of 0.54 (95% CI: 0.49, 0.59). The odds of being classified as having mild OSA were 1.80, however, this finding was not significant (95% CI: 0.89, 3.63, p = .103). The STOP-Bang questionnaire had a sensitivity of 43.7% and a specificity of 77.5%, and the ROC AUC was 0.61 (95% CI: 0.47, 0.74). There were no significant differences comparing the predictive abilities of the ICSD-3A criteria to the STOP-Bang questionnaire in identifying mild OSA in this subsample (χ2(1) = 1.66, p = .197).
Table 4.
Predictive parameters for the ICSD-3A for mild OSA and STOP-Bang subsample analyses.
| ICSD-3A total sample (N = 244) |
STOP-Bang sample (N = 80) |
|||
|---|---|---|---|---|
| Value | 95% CI | Value | 95% CI | |
| Sensitivity % | 20.9 | - | 23.1 | - |
| Specificity % | 87.1 | - | 77.8 | - |
| Correctly classified % | 65.1 | - | 60.0 | - |
| Positive likelihood ratio | 1.63 | - | 1.04 | - |
| Negative likelihood ratio | 0.91 | - | 0.99 | - |
| Odds Ratio | 1.80 | 0.89-3.63 ns | 1.05 | 0.34-3.20 ns |
| ROC AUCs | 0.54 | 0.49-0.59 | 0.50 | 0.41-0.60 |
Note.
= non-significant.
Predictive parameters comparing the ICSD-3 criteria to the STOP-Bang with objective measures of REI for moderate to severe OSA are reported in Table 5, and Figure 1 and 2. The ICSD-3A criteria had a sensitivity of 19.1% to identify participants with moderate to severe OSA, a specificity of 84.43%, and an ROC AUC of 0.52 (95% CI: 0.46, 0.58). The odds of being classified as having moderate to severe OSA by the ICSD-3 criteria were 1.28, however, this finding was not significant (95% CI: 0.57, 2.87, p = .543).
Table 5.
Predictive parameters for the ICSD-3A for moderate to severe OSA and STOP-Bang subsample analyses.
| ICSD-3A total sample (N = 291) |
STOP-Bang sample (N = 96) |
|||
|---|---|---|---|---|
| Value | 95% CI | Value | 95% CI | |
| Sensitivity % | 19.2 | - | 43.7 | - |
| Specificity % | 84.4 | - | 77.5 | - |
| Correctly classified % | 73.9 | - | 71.8 | - |
| Positive likelihood ratio | 1.23 | - | 1.94 | - |
| Negative likelihood ratio | 0.96 | - | 0.73 | - |
| Odds Ratio | 1.28 | 0.57 - 2.87ns | 2.68 | 0.88 - 8.20ns |
| ROC AUCs | 0.52 | 0.46 - 0.58 | 0.61 | 0.47 - 0.74 |
Note.
= non-significant.
Figure 1.
ROC AUCs of the predictive ability of ICSD-3A to identify obectively measured moderate to severe OSA.
Figure 2.
ROC AUCs of the predictive ability of the STOP-Bang to identify objectively measured moderate to severe OSA.
The STOP-Bang questionnaire had a sensitivity of 43.7% and a specificity of 77.5%, and the ROC AUC was 0.61 (95% CI: 0.47, 0.74). The odds of being classified as having moderate to severe OSA by the STOP-Bang questionnaire were 2.68; however, this only approached statistical significance (95% CI: 0.88, 8.20, p = .084). There was no significant difference comparing the predictive ability of the ICSD-3A criteria to the STOP-Bang questionnaire in identifying moderate to severe OSA in this subsample (χ2(1) = 0.54, p = .464). Further, in this same sample, objectively measured BMI was found to be highly correlated with self-report BMI (r = 0.96, p < .001). We further compared the demographic characteristics between the participants who only completed the DSISD (n=195) and those who completed the DSISD and the STOP-Bang questionnaire (n=96). We found no statistically significant differences for age, income, gender, ethnicity, or BMI category.
Discussion
Results from this study found that the ICSD-3A criteria had low sensitivity but high specificity to identify participants with either mild or moderate to severe OSA. Therefore, in this community population of recently unemployed individuals, the ICSD-3A criteria have high rates of false negatives. When compared to the STOP-Bang questionnaire’s predictive ability on a subsample of participants, the STOP-Bang questionnaire sensitivity was higher than that of the ICSD-3A criteria, while the specificity was lower. Both, the ICSD-3A criteria and the STOP-Bang questionnaire had relatively high specificity to classify participants as having moderate to severe OSA. Because a high specificity will yield low false positive rates, researchers could confidently qualify the minority of participants without OSA as eligible for clinical studies. Therefore, the ICSD-3A criteria perhaps could be better utilized as a screening tool for research studies aiming to exclude participants with moderate to severe OSA due to its high specificity, while the STOP-Bang questionnaire may be more useful in screening for OSA. The sensitivity of the STOP-Bang for this population was low, at 43.7% for moderate to severe OSA as compared to other studies. We have previously reported STOP-Bang sensitivity of 87.0% in a population where 87.1% were age ≥ 50 years and 29.4% reported witnessed stop breathing [29], compared to the current study, where only 22.0% of participants were ≥ 50 years and 7% had reported witnessed stop breathing. Other studies have reported differences in population characteristics as well [30]. These differences in population composition may account for the different sensitivity and specificity findings.
The use of assessment tools is generally contingent upon the population being assessed, and the likelihood that the assessed population has the condition of interest. The prevalence of OSA for this population was previously unknown. Although the STOP-Bang questionnaire had higher sensitivity than the ICSD-3A criteria, either of these methods would benefit from the additional use of the HSAT to exclude OSA in research-study populations not selected for OSA attributes.
Because obesity has been associated with increased risk of OSA, supplementary evaluations adding obesity (BMI ≥ 30) as a measure in the ICSD-3A criteria were performed. Adding obesity to the ICSD-3A criteria did not yield significantly different results to those not including obesity. Overall, these results support the necessity of objective testing to identify moderate to severe cases of OSA. Furthermore, they suggest that symptom identification is of limited utility in the prediction of moderate to severe OSA.
This study included individuals who were recently involuntarily unemployed; thus, results may not be generalizable to individuals, who have not recently experienced a life stressor. Poor unrefreshing sleep (20%) and sleepiness, fatigue or insomnia (24%) were two of the symptoms that were reported most highly in the sample, which could be attributed to mental health reactions associated with the stressful life event of job loss. Given that the ICSD-3A criteria only require one or more symptoms to screen positive for a sleep related breathing disorder, these symptoms could have accounted for its poor sensitivity. Further research is necessary on the prevalence of obstructive sleep apnea symptoms in underserved populations who are experiencing stress and insomnia. The STOP-Bang questionnaire is a simple tool, that can be completed easily by the participant, although this tool requires anthropometric measurements and the observation of sleep apnea by a third person.
Results from this study should be considered with a few limitations. First, OSA severity may have been underestimated. The diagnosis of OSA was based on the ApneaLink, a type III home sleep testing device that does not record sleep. Assessment by this device can result in an underestimation of the actual AHI because the denominator in the REI is inferred from the recording time; this can result in a computed REI that is lower than the actual AHI. Also, arousals terminating hypopnea are not detected. Thus, some hypopnea events may be missed if not accompanied by at least a 3% oxygen desaturation. Second, a larger number of participants could have minimized variability and resulted in greater precision of our estimates. The population of this study was majority female, and in the mild OSA group only 32.9% were obese. Furthermore, the sample being studied had previously received a phone screening for exclusion criteria which included having a history of sleep disorder diagnosis. Thus, people with an OSA diagnosis were excluded on phone screen. We acknowledge that the low sensitivity of the ICSD may in part be related to this cohort having characteristics that would be less prone to have OSA. The STOP-Bang questionnaire was added later to the protocol. Thus, only a subset of participants had this data available. This small sample size could be a limiting factor for our data analysis results. Further evaluations that include larger sample sizes are encouraged. Nonetheless, this study is one of the first to compare the ICSD-3A criteria to objective testing and the STOP-Bang questionnaire.
In summary, application of the ICSD-3 criteria for OSA in a community population yields low sensitivity and high specificity. These data support the necessity of objective testing for ICSD-3 OSA diagnosis. In this study ICSD-3A OSA symptoms did not predict positive cases of mild or moderate to severe OSA, therefore, further work is necessary to explore optimal sensitivity for various OSA symptoms required as part of ICSD Criterion.
Acknowledgements
The authors would like to thank the staff and participants of the Assessing Daily Activity Patterns Through Occupational Transitions Study (ADAPT). The authors would like to gratefully acknowledge the assistance of the Arizona Department of Economic Security in study recruitment, and the support of the University of Arizona Collaboratory for Metabolic Disease Prevention and Treatment. Darlynn M. Rojo-Wissar is supported by the National Institute of Mental Health’s Psychiatric Epidemiology Training Program (5T32MH014592-39; PI: Zandi, Peter).
Funding (information that explains whether and by whom the research was supported)
This work is supported by the US National Institute of Health, National Heart, Lung, and Blood Institute (NHLBI,1R01HL117995-01A1).
Footnotes
Conflicts of interest/Competing interests
SQ is a consultant for Jazz Pharmaceuticals, Whispersom and Best Doctors, and serves as the chair of the Scoring Manual Committee and is a member of the Hypopnea Taskforce for the American Academy of Sleep Medicine. The authors have no conflict of interest
Ethics approval and Consent to participate
The ADAPT project protocol was approved by the University of Arizona’s Institutional Review Board, and informed consent was obtained from all study participants at enrollment.
References
- 1.Medicine AAoS (2014) International classification of sleep disorders. 3rd. Darien, Ill, USA: American Academy of Sleep Medicine. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Heinzer R, Vat S et al. (2015) Prevalence of sleep-disordered breathing in the general population: the HypnoLaus study. The Lancet Respiratory Medicine 3:310–318. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Adams R, Appleton S et al. (2016) Are the ICSD-3 criteria for sleep apnoea syndrome too inclusive? The Lancet Respiratory Medicine 4:e19–e20. [DOI] [PubMed] [Google Scholar]
- 4.White DP (1995) Sleep-related breathing disorder. 2. Pathophysiology of obstructive sleep apnoea. Thorax 50:797. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Li X, Sotres-Alvarez D et al. , Associations of Sleep Disordered Breathing and Insomnia with Incident Hypertension and Diabetes: A Prospective Analysis of Data from the Hispanic Community Health Study/Study of Latinos, in C99. SRN: CARDIOMETABOLIC CONSEQUENCES OF SLEEP DISORDERED BREATHING AND THE ROLE OF CPAP THERAPY2019, American Thoracic Society. p. A5605–A5605. [Google Scholar]
- 6.Yk Peker, Hedner J et al. (2002) Increased incidence of cardiovascular disease in middle-aged men with obstructive sleep apnea: a 7-year follow-up. American journal of respiratory and critical care medicine 166:159–165. [DOI] [PubMed] [Google Scholar]
- 7.Peppard PE, Szklo-Coxe M et al. (2006) Longitudinal association of sleep-related breathing disorder and depression. Archives of internal medicine 166:1709–1715. [DOI] [PubMed] [Google Scholar]
- 8.Redline S, Yenokyan G et al. (2010) Obstructive sleep apnea–hypopnea and incident stroke: the sleep heart health study. American journal of respiratory and critical care medicine 182:269–277. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Yaggi HK, Concato J et al. (2005) Obstructive sleep apnea as a risk factor for stroke and death. New England Journal of Medicine 353:2034–2041. [DOI] [PubMed] [Google Scholar]
- 10.Peppard PE, Young T et al. (2013) Increased Prevalence of Sleep-Disordered Breathing in Adults. American Journal of Epidemiology 177:1006–1014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Young T, Evans L et al. (1997) Estimation of the clinically diagnosed proportion of sleep apnea syndrome in middle-aged men and women. Sleep 20:705–706. [DOI] [PubMed] [Google Scholar]
- 12.Robertson B, Marshall B, and Carno M-A, Polysomnography for the sleep technologist: instrumentation, monitoring, and related procedures 2013: Elsevier Health Sciences. [Google Scholar]
- 13.Chai-Coetzer CL, Antic NA et al. (2011) A simplified model of screening questionnaire and home monitoring for obstructive sleep apnoea in primary care. Thorax 66:213–219. [DOI] [PubMed] [Google Scholar]
- 14.Chung F, Yegneswaran B et al. (2008) Stop questionnairea tool to screen patients for obstructive sleep apnea. Anesthesiology: The Journal of the American Society of Anesthesiologists 108:812–821. [DOI] [PubMed] [Google Scholar]
- 15.Netzer NC, Stoohs RA et al. (1999) Using the Berlin Questionnaire to identify patients at risk for the sleep apnea syndrome. Annals of internal medicine 131:485–491. [DOI] [PubMed] [Google Scholar]
- 16.Pallesen S, Bjorvatn B et al. (2008) A new scale for measuring insomnia: the Bergen Insomnia Scale. Perceptual and motor skills 107:691–706. [DOI] [PubMed] [Google Scholar]
- 17.Weaver TE, Maislin G et al. (2003) Self-efficacy in sleep apnea: instrument development and patient perceptions of obstructive sleep apnea risk, treatment benefit, and volition to use continuous positive airway pressure. Sleep 26:727–732. [DOI] [PubMed] [Google Scholar]
- 18.Chung F, Abdullah HR, and Liao P (2016) STOP-Bang questionnaire: a practical approach to screen for obstructive sleep apnea. Chest 149:631–638. [DOI] [PubMed] [Google Scholar]
- 19.Haynes PL, Silva GE et al. (2017) Longitudinal assessment of daily activity patterns on weight change after involuntary job loss: the ADAPT study protocol. BMC public health 17:793. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Haynes PL, Wolf RL et al. (2020) Unemployed Individuals Reporting Hindrance Work Stress at Previous Job Have Increased Likelihood of Insomnia Disorder. International Journal of Behavioral Medicine:1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Edinger JD, Wyatt JK et al. (2011) Testing the reliability and validity of DSM-IV-TR and ICSD-2 insomnia diagnoses: results of a multitrait-multimethod analysis. Archives of general psychiatry 68:992–1002. [DOI] [PubMed] [Google Scholar]
- 22.Edinger J, Wyatt J et al. (2009) Reliability and validity of insomnia diagnoses derived from the Duke Structured Interview for Sleep Disorders. Sleep 32:A265. [Google Scholar]
- 23.Buysse DJ, Reynolds CF III et al. (1994) Clinical diagnoses in 216 insomnia patients using the International Classification of Sleep Disorders (ICSD), DSM-IV and ICD-10 categories: a report from the APA/NIMH DSM-IV Field Trial. Sleep 17:630–637. [DOI] [PubMed] [Google Scholar]
- 24.Erman MK, Stewart D et al. (2007) Validation of the ApneaLink™ for the screening of sleep apnea: a novel and simple single-channel recording device. Journal of clinical sleep medicine 3:387–392. [PMC free article] [PubMed] [Google Scholar]
- 25.Wang Y, Teschler T et al. (2003) Validation of microMESAM as screening device for sleep disordered breathing. Pneumologie (Stuttgart, Germany) 57:734–740. [DOI] [PubMed] [Google Scholar]
- 26.Berry RB, Brooks R et al. (2012) The AASM manual for the scoring of sleep and associated events. Rules, Terminology and Technical Specifications, Darien, Illinois, American Academy of Sleep Medicine; 176. [Google Scholar]
- 27.DeLong ER, DeLong DM, and Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845. [PubMed] [Google Scholar]
- 28.Rice ME and Harris GT (2005) Comparing effect sizes in follow-up studies: ROC Area, Cohen's d, and r. Law and human behavior 29:615–620. [DOI] [PubMed] [Google Scholar]
- 29.Silva GE, Vana KD et al. (2011) Identification of patients with sleep disordered breathing: comparing the four-variable screening tool, STOP, STOP-Bang, and Epworth Sleepiness Scales. Journal of Clinical Sleep Medicine. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Chiu H-Y, Chen P-Y et al. (2017) Diagnostic accuracy of the Berlin questionnaire, STOP-BANG, STOP, and Epworth sleepiness scale in detecting obstructive sleep apnea: a bivariate meta-analysis. Sleep medicine reviews 36:57–70. [DOI] [PubMed] [Google Scholar]