Abstract
Objectives
Polysomnogram is the gold standard for the diagnosis of sleep-disordered breathing (SDB); a sensitive and specific alternative strategy would be ideal, due to its low availability, and screening patients at high risk of OSA is very important. This study aimed to determine the operating characteristics of screening tests in patients with and without cardiovascular disease (CVD).
Material and Methods
Epworth sleepiness scale (ESS), Berlin, STOP-bang and Pittsburgh sleep quality index (PSQI) were applied in adults with and without cardiovascular disease in three Colombian cities, as well as anthropometric measurements and a polysomnogram. Operating characteristics were calculated for each test and the best cut-off values in patients with and without CVD were obtained.
Results
964 patients (median age: 58), 662 with and 302 without CVD were included. The prevalence for SDB (AHI =5) were 43.4 % (OSA), 16.2% (central apnea), and 12.4 % (other). In patients without CVD, the highest sensitivity for OSA and central apnea was for PSQI (80-85%). The highest specificity was for STOP-bang (68%) and Berlin (78.6%). In CVD the best sensitivity was for PSQI (81.9%) followed by Berlin (71.9%) and the best specificity for STOP-bang (82.1%). No isolated questionnaire showed good diagnostic performance (AUC=0.6) and the cut-off values had no variations except for ESS.
Conclusion
Screening tests showed low operating characteristics for the diagnosis to SDB, but better performance in patients with CVD. They are not recommended as the only diagnostic test, but they can be useful to guide the initial diagnostic process.
Keywords: Sleep Apnea, Respiratory Disorders, Screening, Scales, Questionnaire, Altitude
INTRODUCTION
The frequency of sleep disorders in the world is high; it have been reported as 56% in the US population, 31% in Western Europe, and 23% in Japan1 and Latin America is not the exception: studies in different countries have estimated the prevalence of sleep disorders and specifically for obstructive sleep apnea (OSA) with values around 19% in Colombia2 and 30% of the studied population in Sao Paulo, Brazil3; in the PLATINO study the estimated prevalence of OSA were in Santiago of Chile 8.8% and 5.5%; Mexico City 4.4% and 2.4%; Montevideo 3.7% and 0.5%; and Caracas 1.5% and 2.4% for men and women, respectively4.
OSA and central sleep apnea (CSA) are clearly associate with cardiovascular disease (CVD) and patients have increased risk of developing CVD and having worse outcomes; OSA is associated with an increased incidence of high blood pressure (HBP), type 2 diabetes mellitus, atrial fibrillation, heart failure, coronary heart disease, stroke, and death5 and specifically in patients with severe OSA, an increase in fatal and non-fatal cardiovascular events has been reported (adjusted HR of 2.8 and 3.1, respectively)6, 7. On the other hand, the prevalence of OSA and CSA is higher in population with CVD8; it is ideal to have access to accessible and highly sensitive tests in these patients.
The polysomnogram (PSG type 1) is the gold standard for the diagnosis of sleep-disordered breathing (SDB) and is defined as an apnea-hypopnea index (AHI) greater than 5 per hour with symptoms or greater than 15 without symptoms. The American Academy of Sleep Medicine (AASM) recommends a polysomnogram study in high-risk patients with coronary heart disease, cerebrovascular disease, arrhythmia or heart failure who have symptoms suggesting a sleep disorder may be present9, 10.
The availability of sleep laboratories, as well as clinical diagnostic suspicion is low. As such, sleep disorders are frequently underdiagnosed. Likewise, there are logistical difficulties for the exam such as the waiting time for the appointment, transportation, and the time it takes to perform the exam.
Although the studies show a poor diagnostic performance of the screening questionnaires, they are frequently used in Colombia due to the limited availability of the gold test and there are some characteristics such as altitude that suggest that their performance may be different. While clinical signs or questionnaires offer the advantage of being convenient, quick, and inexpensive, the discriminatory power for the diagnosis of SDB by themselves has been shown to be low. However, most of the published studies on this topic come from sleep laboratories in populations that had a high prevalence of OSA and are therefore more likely to be symptomatic. That could change the performance of the test in this population; this could be the case in CVD patients. Some studies have shown that, for example, there are gender differences in the reporting of classic OSA symptoms and in the performance of sleep questionnaires in adults11, 12, 13. In Colombia, there is a high frequency of sleep disorders2 and CVD is considered the main cause of mortality14; but their relationship is not known.
There are few studies that include the general population; the objective of this study is to describe the operational characteristics of four screening tests to diagnose sleep disorders, compared to the polysomnogram as the gold standard in patients with and without CVD.
MATERIAL AND METHODS
Study design
This is a study of operative characteristics of four sleep disorder screening scales; a retrospective analysis was carried out. It included 964 adults over the age of 18; 662 with associated CVD attending the Heart Institute, from three cities in Colombia (Bogotá (2,630 m.a.s.l), Santa Marta (2 m.a.s.l) and Bucaramanga, (959 m.a.s.l)), and the rest without CVD. Patients with mental illnesses that limited the filling of questionnaires were excluded. The protocol was approved by the research ethics committee of the Pontificia Universidad Javeriana in Bogota.
Measuring instruments
Each participant answered a 40-item questionnaire before the polysomnogram was carried out. This questionnaire included demographic data, contact information and the screening tests: Epworth sleepiness scale (ESS), Pittsburgh sleep quality index (PSQI), Berlin and STOP-bang questionnaire.
Overnight PSG were performed using the standard PSG, (an Alice 5 equipment; Philips Respironics, 1010 E.E.U.U) was used; electroencephalogram, electrooculograms, chin electromyogram, nasal pressure detected by airflow pressure transducer, respiratory effort, electrocardiography, pulse oximetry and position were recorded. All the sleep scoring and respiratory events were analyzed using software and analyzed manually by a sleep specialist (who did not know the results of the questionnaires applied to the participants), in accordance with the standards established by the American Academy of Sleep Medicine (AASM)15.
CVD was defined as the presence of structural heart disease, or heart disease diagnosed in follow-up medical visits, including: heart failure of ischemic or valvular origin, coronary heart disease, and arrhythmia; supported with studies such as electrocardiogram, echocardiogram, myocardial perfusion images, Holter electrocardiogram or cardiac catheterization. HBP was considered if a patient was previously diagnosed with hypertension or if he was treated with antihypertensive drugs and was considered as a different group. Other medical conditions including diabetes, hyperlipidemia, chronic obstructive pulmonary disease (COPD), thyroid disease, neurologic disease, and psychological drug administering history were also recorded.
Considering that an additional measures was taken for the ESS in the sleep laboratory before the polysomnogram, the correlation and concordance index between the two measures was calculated.
For the ESS a score =11 was considered as abnormal16; for the Berlin questionnaire, high probability for sleep apnea was defined as a score =2 out of the 3 categories17; for the PSQI, a cut off of 5 was used to categorize “good sleepers” (<5) and “bad sleepers” (>5)18; these scales were validated in Colombia. The STOP-bang questionnaire for sleep apnea was developed to assess the likelihood of OSA in the surgical field and has been validated in the general population; a score =3 indicated intermediate or high risk for OSA19.
Regarding the polysomnogram-based definition, a diagnosis of OSA is considered when there is an AHI =5 per hour with a majority of events obstructive with associated symptoms or greater than 15 without symptoms. For the diagnosis of CSA an AHI =5 per hour and it is required that more than 50% of the events be classified as of central origin. According to the AHI, the severity of the disease is classified as follows: mild (AHI>5 and <15); moderate (>15 and <30) and severe (>30/hour)20.
Statistical analysis
All data were analyzed using STATA (14.0) (StataCorp; College Station, TX, USA), normality of variables was tested by the Shapiro-Wilk W test. An unpaired, two-tail t-test and a chi-squared or Fisher’s exact test analysis were used for comparison between the groups. Operating characteristics (sensitivity (S), specificity (E) and predictive values) for the four questionnaires were calculated according to the severity of each of the disorders, data obtained from the PSG report and the proposed cut-off values for each of the questionnaires. ROC (receiver operating characteristic) curves were constructed for the diagnosis of SDB in patients with CVD and the curves were compared using the equality test of 2 or more ROC areas, confidence intervals were estimated at 95%. Likewise, it was sought if there were better S and E cut-off points for the diagnosis of OSA and CSA for each of the tests in the group of patients with CVD, by means of non-parametric analysis (De Long) and the Liu’s method21.
RESULTS
General characteristics
Table 1 shows the demographic and clinical characteristics of the patients. 964 patients from urban areas were included, 66% were from the city of Bogotá, 25% from Bucaramanga and 9% from Santa Marta. 662 adults had CVD (arrhythmia, heart disease of any type or both) and 302 came from the general population. Unlike patients without CVD, the majority of patients with CVD were men (65.2%) with a median age of 63 years. The distribution of body mass index (BMI) was similar in both groups, the majority in the overweight range. The frequency of BMI over 30 was 10.7% for men and 9.7% for women. Neck and abdominal perimeter median values were greater in the patients with CVD and was 98 cm (DS11.9) in the general population.
Table 1.
Sample description according to the presence of cardiovascular disease.
| With CVD n (631) | Without CVD n (326) | p-value | |
|---|---|---|---|
| Age, median years (IQR) | 63 (55-71) | 47(34-57) | <0.001 |
| Male sex, n (%) | 416(65.2) | 140(42.9) | <0.001 |
| BMI, kg/m2, n (%) | |||
| <18 | 1 (0.1) | 2(0.6) | 0.078 |
| 18-24.9 | 216(33.9) | 123(37.7) | |
| 25-29.9 | 297(46.6) | 127(39.0) | |
| >30 | 124(19.4) | 74(22.7) | |
| Neck perimeter, cm, n (%) | 346(54.3) | 120(36.8) | <0.001 |
| =>40 cm | |||
| Abdominal perimeter, cm, n (%) >90 cm | 525(82.3) | 234(71.8) | <0.001 |
| *OSA, n (%) | |||
| Mild | 135(49.3) | 77(53.5) | 0.597 |
| Moderate | 83(30.3) | 37(25.7) | |
| Severe | 56(20.4) | 30(20.8) | |
| *CSA, n (%) | |||
| Mild | 38(31.1) | 9(26.5) | 0.869 |
| Moderate | 33(27.1) | 10(29.4) | |
| Severe | 51(41,8) | 15(44.1) | |
| Epworth | 293(45.9) | 187(57.3) | 0.001 |
| =>11 | |||
| Berlin | 442(69.3) | 99(30.4) | <0.001 |
| =>2 categories | |||
| Pittsburgh | |||
| <5: Normal | 135(21.1) | 65 (20.0) | 0.721 |
| 5-7: Medical attention | 206(32.3) | 112(34.3) | |
| 8-14: Attention and treatment | 246(38.6) | 118(36.2) | |
| 14-21: Severe sleep disorder | 51(8.0) | 31(9.5) | |
| STOP-bang | |||
| =>3 High risk | 540(84.6) | 176(54) | <0.001 |
| Comorbidities, n (%) | |||
| HBP | 495(51.3) | 53(5.5) | <0.001 |
| Diabetes | 140(14.5) | 12(1.24) | <0.001 |
| Depression | 111(11.5%) | 25(2.6) | <0.001 |
| Anxiety | 103(10,7) | 33(3.4) | 0.011 |
| Hypothyroidism | 110(11.4) | 39(4.05) | 0.032 |
| COPD | 40(4.15) | 2(0.21) | <0.001 |
| GERD | 68(7.05) | 41(4.25) | 0.374 |
Notes: IQR = Interquartile range; BMI = Body mass index; OSA = Obstructive sleep apnea; CSA = Central sleep apnea; HBP= High blood pressure; COPD = Chronic obstructive pulmonary disease; GERD = Gastro esophageal reflux disease; CVD = Cardiovascular disease; *Results by PSG.
Comorbidities
Regarding comorbidities, 49% had heart disease of any etiology; 3% had some type of arrhythmia (either brady or tachyarrhythmia). 423 patients had coronary heart disease, the majority were men (68%) and 76% had some SDB (41% with OSA the mild majority and 19% with predominantly severe CSA).
The prevalence of HBP in the total group was 56.8%; other comorbidities were diabetes (16%), hypothyroidism (15%), depression and anxiety (14%), and chronic obstructive pulmonary disease (COPD) (4.3%), all of them more frequent in the CVD group (statistically significant) except for gastro esophageal reflux disease (GERD) (11.3%, p=0.374). Table 1 shows the difference in frequencies according to the presence of CVD.
SDB
The prevalence of SDB defined as AHI=5 was 72% in the general population. Most patients presented AHI between 5 and 15 per hour, that is, mild, with a similar distribution in patients with and without CVD (Table 1). As for each disorder, the frequency OSA and CSA was 43.4% and 16.2%, respectively.
The percentage of cases of OSA (24.9% vs. 8.6%; p>0.01) and CSA (56.7% vs. 22.7%; p<0.01) classified as severe was higher in Bogotá than other cities. The Cheyne Stokes pattern was present in 54 patients (5.6% of the sample); 15 of them had OSA and 39 CSA.
Of the patients with HBP, 45% had SDB; 15% of them with an AHI greater than 30; the frequency of hypothyroidism and diabetes was also higher in these patients.
Operating characteristics of the tests
For the ESS, the correlation and concordance index between the two performed measures (surveys and sleep laboratory) was 0.74 (95% CI: 0.71-0.76). Table 2 shows the operating characteristics of the different questionnaires for the diagnosis of SDB. For the patients without CVD, it was found that for the diagnosis of OSA and CSA, the highest sensitivity (S) was found the PSQI, with values between 80 and 85%. The highest specificity (E) for the diagnosis of OSA was for the Berlin questionnaire (78.6%).
Table 2.
Operating characteristics for sleep-disordered breathing according to the presence of cardiovascular disease.
| OSA | CSA | |||
|---|---|---|---|---|
| CI 95 % | Without CVD | With CVD | Without CVD | With CVD |
| Epworth | ||||
| Prevalence % | 44 (39-49.7) | 43 (39-46.9) | 10 (7.3-14.3) | 19(16-22.4) |
| Sensitivity % | 58.3(49.8-66.5) | 47.4(41.4-53.5) | 47.1 (29.8-64.9) | 47.5(38.4-56.8) |
| Specificity % | 43.4(36.1-50.9) | 55.2(49.9-60.4) | 41.4 (35.7-47.3) | 54.5 (50-58.8) |
| PPV % | 44.9(37.7-52.3) | 44.4(38.6-50.3) | 8.56(4.97-13.5) | 19.8 (14.4-24.8) |
| NPV % | 56.8 (48.2-65.2) | 58.3(52.9-63.5) | 87.1 (80.3-92.1) | 81.4(76.9-85.4) |
| ROC area | 0.50(0.45-0.56) | 0.51(0.47-0.55) | 0.44 (0.35-0.53) | 0.51 (0.46-0.55) |
| Berlin | ||||
| Prevalence % | 44 (39-49.7) | 43(39-46.9) | 10(7.3-14.3) | 19(16-22.4) |
| Sensitivity % | 41.7(33.5-50.2) | 71.9(66.2-77.1) | 41.2(24.6-59.3) | 69.7 (60.7-77.7) |
| Specificity % | 78.6(71.9-84.3 | 32.7(27.9-37.8) | 70.9(65.3-76) | 30.8 (26.9-35) |
| PPV % | 60.6(50.3-70.3) | 44.6(39.9-49.3) | 14.1(7.95-22.6) | 19.2(15.7-23.2) |
| NPV % | 63(56.4-69.3) | 60.7 (53.5-67.6) | 91.2(86.7-94.5) | 81.1(74.9-86.3) |
| ROC area % | 0.60 (0.55-0.65) | 0.52(0.48-0.55) | 0.56(0.47-0.64) | 0.50 (0.45-0.54) |
| STOP-bang | ||||
| Prevalence % | 44(39-49.7) | 43 (39-46.9) | 10(7.3-14.3) | 19 (16-22.4) |
| Sensitivity % | 29.2(21.9-37.3) | 12(8.44-16.5) | 32.4 (17.4-50.5) | 16.4(10.3-24.2) |
| Specificity % | 40.7(33.5- 48.2) | 82.1(77.8- 85.9) | 52.4 (46.5- 58.2) | 84.9(81.5-87.9) |
| PPV % | 28(21-35.9) | 33.7(24.4-43.9) | 7.33(3.72-12.7) | 20.4 (12.9-29.7) |
| NPV % | 42(34.7-49.7) | 55.4(51.1-59.6) | 86.9(81-91.5) | 81.1 (77.6-84.3) |
| ROC area | 0.34 (0.29-0.40) | 0.47(0.44-0.49) | 0.42 (0.33-0.50) | 0.50(0.45-0.54) |
| Pittsburgh | ||||
| Prevalence % | 43 (39-46.9) | 44(39-49.7) | 19(16-22.4) | 10 (7.3-14.3) |
| Sensitivity % | 80.3 (75.1-84.8) | 81.9(74.7-87.9) | 81.1(73.1-87.7) | 85.3(68.9-95) |
| Specificity % | 22.3(18.1-26.9) | 21.4(15.7-28.1) | 21.7 (18.2-25.5) | 20.5 (16.1-25.6) |
| PPV % | 43.7 (39.4-48.2) | 45.2(39.1-51.5) | 19.7 (16.3-23.4) | 11.1 (7.57-15.6) |
| NPV % | 60 (51.2-68.3) | 60(47.1-72) | 83(75.5-88.9) | 92.3 (83-97.5) |
| ROC area % | 0.51 (0.48-0.54) | 0.51(0.47-0.56) | 0.51(0.47-0.55) | 0.52 (0.46-0.59) |
Notes: OSA = Obstructive sleep apnea; CSA = Central sleep apnea; CVD = Cardiovascular disease; CI = Confidence interval; PPV = Positive predictive value; NPV = Negative predictive value; ROC = Receiver operating characteristic.
In patients with CVD, the best S for the diagnosis of OSA and CSA was for the PSQI (81.9%) followed by the Berlin questionnaire (71.9%) and the best E for the STOP-bang questionnaire (82.1%). Specifically, in patients with coronary heart disease, E was better too for the STOP-bang questionnaire (86%) (Supplementary Table 1).
When calculating the operating characteristics for the SDB according to the AHI (Table 3), both for moderate to severe OSA and CSA, the highest S was maintained for the PSQI (79.2%), followed by Berlin. For diagnosis of moderate to severe OSA, STOP-bang showed E 90.8%, followed by ESS (50%).
Table 3.
Operating characteristics for OSA and CSA according to severity (AHI)*.
| OSA | ||||||
|---|---|---|---|---|---|---|
| CI 95% | Prevalence % | Sensitivity % | Specificity % | PPV % | PNV % | ROC Area |
| Epworth | ||||||
| Mild* | 21 (19-24) | 52.9 (45.9-59.9) | 51.1 (47.4-54.7) | 22.7 (19- 26.7) | 80(76.1-83.4 | 0.52(0.48-0.55) |
| Moderate* | 51 (46-55.6) | 49.5(42.6-56.5) | 47.1(40.1-54.1) | 49.1 (42.2-56) | 47.5(40.5-54.6) | 0.48 (0.43-0.53) |
| Severe* | 8.9 (7.2-10.9) | 53.5 (42.4-64.3) | 50.6(47.2-53.9) | 9.58(7.1-12.6) | 91.7(88.9- 94) | 0.52 (0.46- 0.57) |
| Berlin | ||||||
| Mild | 21 (19-24) | 68.9 (62.1-75.2) | 47.4 (43.8-51) | 26.2(22.6-30.2) | 84.9 (81.1-8.1) | 0.58(0.54-0.61) |
| Moderate | 51 (46-55.6) | 54.2(47.3-61.1) | 31.1 (24.8-37.9) | 44.7(38.6-51.1) | 39.8 (32.1-47.8) | 0.42 (0.38-0.47) |
| Severe | 8.9(7.2-10.9) | 73.3 (62.6-82.2) | 45.6 (42.2-48.9) | 8.07 (5.72-11) | 94.6 (92-96.5) | 0.59 (0.54-0.64) |
| STOP-bang | ||||||
| Mild | 21(19-24) | 9.22 (5.64-14) | 69.8 (66.4-73) | 7.66 (4.68-11.7) | 73.9 (70.5-77.1) | 0.39(0.36-0.42) |
| Moderate | 51 (46-55.6) | 26.4 (20.6-32.9) | 90.8 (86-94.4) | 74.7 (63.3-84) | 54.5 (49.1-59.9) | 0.58(0.55-0.62) |
| Severe | 8.9 (7.2-10.9) | 6.98 (2.6-14.6) | 72.4 (69.4-75.4) | 2.42(0.89-5.19) | 88.8 (86.3-91) | 0.39 (0.36-0.42) |
| Pittsburgh | ||||||
| Mild | 21 (19-24) | 82.5 (76.6-87.4) | 21.6(18.8-24.7) | 22.3(19.3-25.4) | 82(76-87.1) | 0.52 (0.49-0.55) |
| Moderate | 51 (46-55.6) | 79.2 (73.2-84.5) | 17.5 (12.6-23.4) | 49.7(44.2-55.2) | 45 (33.8-56.6) | 0.48(0.44-0.52) |
| Severe | 8.9(7.2-10.9) | 79.1(69-87.1) | 20.7 (18.1-23.6) | 8.9(6.98-11.1) | 91 (86.1-94.6) | 0.49(0.45-0.54) |
| CSA | ||||||
| Epworth | ||||||
| Mild | 11(9.4-13.5) | 48.6 (38.9-58.4) | 50.1 (46.7-53.5) | 11(8.38-14.2) | 88.4(85.2-91.1) | 0.49(0.44-0.54) |
| Moderate | 30 (23-38) | 44.7(30.2-59.9) | 51.4 (41.6-61.1) | 28.4 (18.5-40.1) | 68.3 (57.1-78.1) | 0.48 (0.39-0.56) |
| Severe | 0.5 (0.44-0.57) | 51.5(38.9-64) | 50.3 (47-53.7) | 7.08 (4.95-9-76) | 93.4 (90.8-95.4) | 0.50 (0.44-0.57) |
| Berlin | ||||||
| Mild | 11(9.4-13.5) | 62.4 (52.6-71.5) | 44.7 (41.3-48.1) | 12.6 (9.89-15.7) | 90.3 (87-93) | 0.53 (0.48-0.58) |
| Moderate | 30(23-38) | 66 (50.7-79.1) | 37.6(28.5-47.4) | 31.3 (22.4-41.4) | 71.9 (58.5-83) | 0.51 (0.43-0.6 |
| Severe | 0.5 (0.44-0.57) | 62.1 (49.3-73.8) | 44.3 (41-47.6) | 7.58 (5.49-10.1) | 94.1 (91.4-96.1) | 0.53 (0.47-0.59) |
| STOP-bang | ||||||
| Mild | 11 (9.4-13.5) | 15.6(9.36-23.8) | 73 (69.9-75.9) | 6.85 (4-10.7) | 87.2 (84.5-89.5) | 0.44(0.40-0.48) |
| Moderate | 30 (23-38) | 29.8 (17.3-44.9) | 84.4 (76.2-90.6) | 45.2(27.3-64) | 73.6 (65-81.1) | 0.57 (0.49-0.64) |
| Severe | 0.50 (0.44-0.57) | 18.2(9.76-29.6) | 73.7 (70.7-76.6) | 4.84 (2.52-8.3) | 92.5 (90.3-94.3) | 0.46 (0.41-0.50) |
| Pittsburgh | ||||||
| Mild | 11 (9.4-13.5) | 82.6 (74.1-89.2) | 21.2 (18.5-24.1) | 11.8(9.58-14.3) | 90.5(85.6-94.2) | 0.51 (0.48-0.55) |
| Moderate | 30 (23-38) | 80.9 (66.7-90.9) | 17.4 (10.8-25.9) | 29.7 (21.9-38.4) | 67.9(47.6-84.1) | 0.49 (0.42-0.55 |
| Severe | 0.50 (0.44-0.57) | 81.8 (70.4-90.2) | 20.9 (18.3-23.7) | 7.07(5.3-9.1) | 94 (89.8-96.9) | 0.51 (0.46-0.56) |
Notes: OSA = Obstructive sleep apnea; CSA = Central sleep apnea; AHI = Hypopnea apnea index; CI = Confidence interval; PPV = Positive predictive value; NPV = Negative predictive value; ROC = Receiver operating characteristic; *Severity of SDB (OSA and CSA according to AHI): >5 and <15 mild, >15 and <30 moderate, >30/hour severe.
Figure 1 shows a comparison of the receiver operating characteristic (ROC) curves with the respective area under the curve (AUC) for the diagnosis of OSA and CSA in patients with CVD; none of the scales showed a discrimination ability significantly better than the others. (OSA chi2=0.5595, CSA chi2=0.2585).
Figure 1.
Operating characteristics curves (AUC-ROC) for the diagnosis of OSA and CSA in patients with cardiovascular disease. Notes: OSA = Obstructive sleep apnea; CSA = Central sleep apnea; ROC: Receiver operating characteristic.
Table 4 shows the different cut-off values evaluated for each test in the population with CVD. For the ESS the best cut-off point value was found to be greater than 9, with usual value>11, both for screening OSA (S: 60%, E: 45%) and CSA (S: 52, E: 51%) For the Berlin questionnaire, the cut-off value was similar to the usual >2; for OSA (S: 72%, E: 33%) and CSA (S: 70%, E: 31%). For the STOP-bang questionnaire, the best cut-off value was a score =3; OSA (S: 88, E: 18) and CSA (S: 64.8, E: 38.6). For the PSQI, a similar cut-off value was found >5, for OSA (S: 80%, E: 22%) and CSA (S: 83.6%, E: 15.2%).
Table 4.
Areas under the curve (AUC) for each test in the diagnosis of OSA and CSA.
| OSA | CSA | |||||
|---|---|---|---|---|---|---|
| Test | AUC | CI 95% | Cut-off value | AUC | CI 95% | Cut-off value |
| Epworth | 0.54 | 0.496 0.586 | >9 | 0.51 | 0.454 0.568 | >9 |
| Berlin | 0.55 | 0.509 0.590 | >2 | 0.47 | 0.427 0.527 | >2 |
| Pittsburgh | 0.51 | 0.465 0.555 | >5 | 0.53 | 0.480 0.590 | >5 |
| STOP-bang | 0.55 | 0.507 0.595 | >3 | 0.52 | 0.465 0.576 | >3 |
Notes: OSA = Obstructive hypopnea sleep apnea syndrome; CSA = Central sleep apnea; AUC: Areas under the curve; CI = Confidence interval.
DISCUSSION
The operating characteristics for the SDB the highest S was for the PSQI; for moderate to severe OSA it was (79.2%), followed by Berlin (54.2%), ESS (49.5%), and STOP-bang (26.4%). For diagnosis of moderate to severe OSA, STOP-bang showed E 90.8%, followed by ESS (47.1%), Berlin (31.1%), and PSQI (20%). Our results show that, when evaluating the operating characteristics of the questionnaires similar to that reported in the literature, no questionnaire shows good diagnostic performance when used by itself; all had AUC of 0.6 or less. This is consistent with previous results where, for example, for ESS the results have been poor in their ability to screen OSA (AUC 0.56, accuracy of 51-59% for the cutoff AHI=5) and have often shown better specificities than sensitivities. Although many of the studies in this regard have shown to be of poor quality or in highly selected populations, the above implies that, although the availability of the gold standard is limited in many regions, for now it is not possible to recommend the questionnaires or algorithms of prediction as the only diagnostic method20, 22.
The frequency of OSA in the population with and without CVD in this study was about 43% and for CSA of 19 and 10%, respectively. Our results are in agreement with what is described in the literature; in patients with heart failure for example, the reported prevalence of OSA varies widely23 according with the cut-off point of AHI employed and the association with CVD (For AHI>5 a prevalence of 55 an 87% is described in the low and high risk groups, respectively)20: in general, it is higher than those reported for the general population. According to a cross-sectional analysis of sleep heart health study, the presence of OSA with an AHI=11 confers a relative increase of 2.38 times in the probability of having heart failure independent of other factors24.
In our results, the high frequency of CSA found in a population with and without CVD is striking, in contrast to what is reported in literature. These changes can be explained both by altitude (66% of the population was from Bogota) and by the presence of heart failure; in the CVD group where the majority were men and with more comorbidities. Several studies done at high altitudes indicate that an increase in central apneas occurs even among healthy individuals and in the same way, AHI is higher for individuals with OSA, and obstructive events convert to predominately central events at altitude25; likewise, a study carried out in the city of Bogotá in patients with decompensated heart failure showed that all the patients had OSA, most were severe, with the presence of central sleep apneas26. On the other hand, the frequency of SDB was higher in the group of men, which is similar to previous reports.
In this study, for the detection of SDB, in general, the highest S (about 81%) was found for PSQI. It was validated in Colombia and an S of 89% and E of 86% have been described27. These results are expected given the usefulness of this test to assess overall sleep quality. The Berlin questionnaire showed moderate S for the diagnosis of OSA and CSA (61% and 63.5%) with poor E and better performance in the group of patients with CVD (S: 72%-70%). Polanía-Dussan et al. (2013)17 found a S of 87%, E of 70%, AUC of 0.78 in their study validating Berlin test for Colombia, which contrasts with our results (AUC around 0.5). The above is partly explained by the population included in their study, most patients attending sleep laboratories were from the city of Bogotá. Other studies have not shown good performance especially for AHI=5 as the cut-off value (S of 76% and E of 45%) with similar findings to ours, although with high methodological variability20. It is important to note that some studies have shown that, for example, classic OSA symptoms, such as drowsiness, snoring, and apnea, are reported more frequently in men, while fatigue, initial insomnia, depression, and headaches are more common in the women12, 13, 28; in the same way, comorbidities can produce changes in symptoms that are similar to the symptoms of OSA and then affect the performance of diagnostic test, which in turn can produce false positive results28.
In contrast to previously reported studies, the STOP-bang questionnaire showed a better E and low sensitivity for the diagnosis of OSA and CSA (82.1% and 84.9) in the group of patients with CVD, followed by ESS (55%). On the contrary, it showed poor performance in the group of patients without CVD. Previously, good performance was described in mainly perioperative patients with an S of 84%, 93% and 100% for AHI > of 5, 15 and 30, respectively, and generally lower E (47% for moderate OSA and 37% for severe OSA)19; the different results to those reported in the literature are probably due to the fact that the STOP-bang was developed in the surgical population and our study included patients with CVD and also the general population without CVD; many of them from high altitude.
In 2017, a meta-analysis was published describing the diagnostic performance for the Berlin, STOP-bang, stop and ESS questionnaires for detection of OSA according to the AHI; they concluded that STOP-bang is a more accurate tool with better diagnostic S and OR to detect OSA and could be used for early diagnosis in clinical settings. However, it should be borne in mind that age, gender difference, BMI and the presence of comorbid conditions in the participants are factors that affect the accuracy of screening tools and should be considered when applying theses questionnaires, as it is recognized that there is a high probability of bias due to heterogeneity in the studied populations29. A systematic review carried out in Canada (2010)30 also describes that there is inconsistency in the accuracy of the tests due to the heterogeneity of the designs (population, type of questionnaire, and validity) and highlight the usefulness of stop and STOP-bang for screening of OSA in surgical population due to its better methodological quality and ease of use.
The performance of screening questionnaires for identifying OSA in populations with increased cardiovascular risk is not yet fully established; knowing the importance of SDB in cardiovascular outcomes in our study, we specifically evaluated the performance of tests in patients with CVD and we assessed for the existence of a possible better cut-off score for each questionnaire; only for the ESS had a slightly lower cut-off value as compared to the previously established value (greater than 9) for the diagnosis of OSA and CSA. A study published in 2013, evaluating the performance of ESS for the diagnosis of OSA (AHI>5), strikingly found high E (82.7%, 95% CI: 77.3-87.3) for ESS and moderate S (61.6%, 95% CI: 59.3-63.9) with cut-off values >9 for men and 6 for women31. Our results imply that in patients with CVD being at high risk, whenever using this scale as part of the initial assessment, a lower cut-off value increasing S should be considered. However, there is no doubt that these patients should be evaluated through the golden standard.
Specifically, within the group of patients with coronary heart disease in our study, a high frequency of SDB (76%) was found, mostly in men, with AHI between 5 and 15 and >30. For the diagnosis of OSA and CSA, the best S in this group was shown by the PSQI and the best E was the STOP-bang questionnaire, for CSA with slightly higher values. Some authors have explored the use of questionnaires such as Berlin32, 33 and ESS34, 35 as a screening tool for OSA when this diagnostic suspicion is found alongside coronary risk. However, these studies have not taken the gold standard into account.
As limitations, it is a retrospective analysis study with the possibility of bias; additionally, the data did not allow discriminating by subgroups because the sample size was not calculated for this purpose. However, our study has strengths: the number of patients included is considerable; in addition, our results provide knowledge regarding the performance of screening tests in a sample of population from three cities being representative of different altitudes, including the general population and those with CVD.
Although the prevalence of SDB and specifically OSA is high and its association with CVD is clear, the questionnaires used for screening have poor operating characteristics. Despite the insufficient availability of sleep laboratories in many regions, according to the results of this study and previous evidence, it is not possible to recommend questionnaires or clinical prediction rules as a single or independent diagnostic test to replace the polysomnogram, since they neither rule out or confirm the diagnosis. However, the use of screening tests that, although imperfect, could improve decision-making processes regarding an initial diagnostic strategy could be justified. Likewise, taking into account the influence of altitude on sleep physiology, it is necessary to understand the performance of different diagnostic tests in populations with different altitudes. High-risk patients with CVD, however, must prioritarily be assessed by the gold standard. In general, more studies are required to evaluate the performance of new proposals (combination of tests, series, parallel studies) or prediction models in populations that are not highly selected, also seeking to reduce the likelihood of inherent biases.
ACKNOWLEDGEMENTS
We would like to offer our gratitude to Dr. Liliana Margarita Otero Mendoza, who allowed access to the database employed to conduct this study, and whose help was invaluable to the completion of this work.
Appendix.
Supplementary Table 1.
Operating characteristics of screening tests in patients with coronary heart disease.
| OSA | ||||
|---|---|---|---|---|
| Prevalence 44% (40-49.3) CI 95% | Epworth | Berlin | STOP-bang | Pittsburgh |
| Sensitivity% | 48.4 (41.1-55.8) | 73.4 (66.5-79.6) | 11.2 (7.05-16.6) | 80.3 (73.9-85.7) |
| Specificity % | 53.6(47-60.1) | 32.8(26.8-39.2) | 83.4 (78-87.9) | 21.3 (16.2-27.1) |
| PPV % | 45.5 (38.5-52.7) | 46.6 (40.8-52.5) | 35 (23.1-48.4) | 44.9 (39.5-50.4) |
| NPV% | 56.5 (49.7-63.1) | 60.6 (51.6-69.2) | 54 (48.7-59.2) | 57.5 (46.4-68) |
| ROC area | 0.51 (0.46-0.55) | 0.53 (0.48-0.57) | 0.47 (0.44-0.50) | 0.50 (0.46-0.54) |
| Prevalence 19% ( 16-23.2) CSA | ||||
| Sensitivity% | 49.4 (38.1-60.7) | 70.4 (59.2-80) | 14.8(7.9-24.4) | 80.2 (69.9-88.3) |
| Specificity % | 53.2 (47.8-58.6) | 30.1(25.3-35.3) | 86(81.8-89.5) | 20.8 (16.6-25.5) |
| PPV% | 20 (14.7-26.2) | 19.3 (14.9-24.2) | 20 (10.8-32.3) | 19.3 (15.3-24) |
| NPV% | 81.6 (75.9-86.5) | 81.1 (73.2-87.5) | 81(76.6-84.9) | 81.6 (71.9-89.1) |
| ROC area | 0.51 (0.45-0.57) | 0.50 (0.44-0.55) | 0.50 (0.46-0.54) | 0.50 (0.45-0.55) |
Notes: OSA = Obstructive hypopnea sleep apnea syndrome; CSA = Central sleep apnea; CI = Confidence interval; PPV = Positive predictive value; NPV = Negative predictive value; ROC = Receiver operating characteristic.
REFERENCES
- 1.Léger D, Poursain B, Neubauer D, Uchiyama M. An international survey of sleeping problems in the general population. Curr Med Res Opin. 2008 Jan;24(1):307–317. doi: 10.1185/030079907x253771. [DOI] [PubMed] [Google Scholar]
- 2.Ruiz AJ, Sepúlveda MAR, Marínez PH, Muñoz MC, Mendoza LO, Centanaro OPP, et al. Prevalence of sleep complaints in Colombia at different altitudes. Sleep Sci. 2016 Apr/Jun;9(2):100–105. doi: 10.1016/j.slsci.2016.05.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Tufik S, Santos-Silva R, Taddei JA, Bittencourt LRA. Obstructive sleep apnea syndrome in the Sao Paulo epidemiologic sleep study. Sleep Med. 2010 May;11(5):441–446. doi: 10.1016/j.sleep.2009.10.005. [DOI] [PubMed] [Google Scholar]
- 4.Menezes AM, Pérez-Padilla R, Jardim JR, Muiño A, López M V, Valdivia G, et al. Chronic obstructive pulmonary disease in five Latin American cities (the PLATINO study): a prevalence study. Lancet. 2005 Nov;366(9500):1875–1881. doi: 10.1016/S0140-6736(05)67632-5. DOI: http://doi.org/cx9fqb. [DOI] [PubMed] [Google Scholar]
- 5.Tietjens JR, Claman D, Kezirian EJ, Marco T, Mirzayan A, Sadroonri B, et al. Obstructive sleep apnea in cardiovascular disease: a review of the literature and proposed multidisciplinary clinical management strategy. J Am Heart Assoc. 2019 Jan;8(1):e010440. doi: 10.1161/JAHA.118.010440. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Marin JM, Carrizo SJ, Vicente E, Agusti A. Long-term cardiovascular outcomes in men with obstructive sleep apnoea-hypopnoea with or without treatment with continuous positive airway pressure: an observational study. Lancet. 2005 Mar;365(9464):1046–1053. doi: 10.1016/S0140-6736(05)71141-7. [DOI] [PubMed] [Google Scholar]
- 7.Maeder MT, Schoch OD, Rickli H. A clinical approach to obstructive sleep apnea as a risk factor for cardiovascular disease. Vasc Health Risk Manag. 2016 Mar;12:85–103. doi: 10.2147/VHRM.S74703. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Johnson KG, Johnson DC. Frequency of sleep apnea in stroke and TIA patients: a meta-analysis. J Clin Sleep Med. 2010 Apr;6(2):131–137. [PMC free article] [PubMed] [Google Scholar]
- 9.Bianchi MT, Eiseman NA, Cash S, Mietus J, Peng CK, Thomas RJ. Probabilistic sleep architecture models in patients with and without sleep apnea. J Sleep Res. 2012 Jun;21(3):330–341. doi: 10.1111/j.1365-2869.2011.00937.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Clinica Las Condes (CLC) Patologías del sueño. Rev Med Clin Condes. 2013 May;24(3):341–493. [Google Scholar]
- 11.Ulasli SS, Gunay E, Koyuncu T, Akar O, Halici B, Ulu S, et al. Predictive value of Berlin questionnaire and Epworth sleepiness scale for obstructive sleep apnea in a sleep clinic population. Clin Respir J. 2014 Jul;8(3):292–296. doi: 10.1111/crj.12070. [DOI] [PubMed] [Google Scholar]
- 12.Pataka A, Kotoulas S, Kalamaras G, Schiza S, Sapalidis K, Giannakidis D, et al. Gender differences in obstructive sleep apnea: the value of sleep questionnaires with a separate analysis of cardiovascular patients. J Clin Med. 2020;9(1):130–130. doi: 10.3390/jcm9010130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.US Preventive Services Task Force Screening for obstructive sleep apnea in adults: US preventive services task force recommendation statement. JAMA. 2017 Jan;317(4):407–414. doi: 10.1001/jama.2016.20325. [DOI] [PubMed] [Google Scholar]
- 14.Ministerio de Salud y Protección Social (CO) Dirección de Epidemiología y Demografía. Grupo de Gestión de Conocimiento. Guía metodológica observatorio de salud cardiovascular, diabetes y enfermedad crónica renal OCADER - Colombia [Internet] Bogotá: Ministerio de Salud y Protección Social; 2013. Available from: https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VS/ED/GCFI/OERCAV_00122.10.2013v1.pdf. [Google Scholar]
- 15.American Academy of Sleep Medicine (AASM) Sleep-related breathing disorders in adults: recommendations for syndrome definition and measurement techniques in clinical research. The report of an American Academy of Sleep Medicine Task Force. Sleep. 1999 Aug;22(5):667–689. [PubMed] [Google Scholar]
- 16.Chica-Urzola HL, Escobar-Cordoba F, Eslava-Schmalbach J. Validación de la escala de somnolencia de Epworth. Rev Salud Pública. 2007;9(4):558–567. doi: 10.1590/s0124-00642007000400008. [DOI] [PubMed] [Google Scholar]
- 17.Polanía-Dussan IG, Escobar-Cordoba F, Eslava-Schmalbach J, Netzer NC. Validación colombiana del cuestionario de Berlín. Rev Fac Med. 2013;61(3):231–238. [Google Scholar]
- 18.Escobar-Córdoba F, Eslava-Schmabach J. Colombian validation of the Pittsburgh sleep quality index. Rev Neurol. 2005 Feb;40(3):150–155. [PubMed] [Google Scholar]
- 19.Chung F, Abdullah HR, Liao P. STOP-bang questionnaire a practical approach to screen for obstructive sleep apnea. Chest. 2016 Mar;149(3):631–638. doi: 10.1378/chest.15-0903. [DOI] [PubMed] [Google Scholar]
- 20.Kapur VK, Auckley DH, Chowdhuri S, Kuhlmann DC, Mehra R, Ramar K, et al. Clinical practice guideline for diagnostic testing for adult obstructive sleep apnea: an American Academy of Sleep Medicine Clinical Practice Guideline. J Clin Sleep Med. 2017 Mar;13(3):479–504. doi: 10.5664/jcsm.6506. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Liu X. Classification accuracy and cut point selection. Stat Med. 2012 Oct;31(23):2676–2686. doi: 10.1002/sim.4509. [DOI] [PubMed] [Google Scholar]
- 22.Kushida CA, Littner MR, Morgenthaler T, Alessi CA, Bailey D, Coleman J, et al. Practice parameters for the indications for polysomnography and related procedures: an update for 2005. Sleep. 2005 Apr;28(4):499–521. doi: 10.1093/sleep/28.4.499. [DOI] [PubMed] [Google Scholar]
- 23.Lyons OD, Bradley TD. Heart failure and sleep apnea. Can J Cardiol. 2015 Jul;31(7):898–908. doi: 10.1016/j.cjca.2015.04.017. [DOI] [PubMed] [Google Scholar]
- 24.Gottlieb DJ, Yenokyan G, Newman AB, O’Connor GT, Punjabi NM, Quan SF, et al. Prospective study of obstructive sleep apnea and incident coronary heart disease and heart failure: the sleep heart health study. Circulation. 2010 Jul;122(4):352–360. doi: 10.1161/CIRCULATIONAHA.109.901801. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Pagel JF, Kwiatkowski C, Parnes B. The effects of altitude associated central apnea on the diagnosis and treatment of obstructive sleep apnea: comparative data from three different altitude locations in the mountain west. J Clin Sleep Med. 2011 Dec;7(6):610–615. doi: 10.5664/jcsm.1462. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Vargas-Ramirez L, Gonzalez-Garcia M, Franco-Reyes C, Bazurto-Zapata MA. Severe sleep apnea, Cheyne-Stokes respiration and desaturation in patients with decompensated heart failure at high altitude. Sleep Sci. 2018 May/Jun;11(3):146–151. doi: 10.5935/1984-0063.20180028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Buysse DJ, Reynolds CF, Monk TH, Berman SR, Kupfer DJ. The Pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989 May;28(2):193–213. doi: 10.1016/0165-1781(89)90047-4. [DOI] [PubMed] [Google Scholar]
- 28.Peñafiel S, Rossel GS, Meza JC, Nesvadba DF, Tapia AP, Mardones CA, et al. Gender differences in clinical features and performance of sleep questionnaires in adults with obstructive sleep apnea syndrome. Rev Med Chile. 2019 Oct;147(10):1291–1302. doi: 10.4067/s0034-98872019001001291. [DOI] [PubMed] [Google Scholar]
- 29.Chiu HY, Chen PY, Chuang LP, Chen NH, Tu YK, Hsieh YJ, et al. Diagnostic accuracy of the Berlin questionnaire, STOP-BANG, STOP, and Epworth sleepiness scale in detecting obstructive sleep apnea: a bivariate meta-analysis. Sleep Med Rev. 2017 Dec;36:57–70. doi: 10.1016/j.smrv.2016.10.004. [DOI] [PubMed] [Google Scholar]
- 30.Abrishami A, Khajehdehi A, Chung F. A systematic review of screening questionnaires for obstructive sleep apnea. Can J Anaesth. 2010 May;57(5):423–438. doi: 10.1007/s12630-010-9280-x. [DOI] [PubMed] [Google Scholar]
- 31.Pataka A, Daskalopoulou E, Kalamaras G, Passa KF, Arqyropoulou P. Evaluation of five different questionnaires for assessing sleep apnea syndrome in a sleep clinic. Sleep Med. 2014 Jul;15(7):776–781. doi: 10.1016/j.sleep.2014.03.012. [DOI] [PubMed] [Google Scholar]
- 32.Jesus EV, Dias-Filho EB, Mota BM, Souza L, Marques-Santos C, Rocha J, et al. Sospecha de apnea obstructiva del sueño definida por el cuestionario de Berlín predice eventos en pacientes con síndrome coronario agudo. Arq Bras Cardiol. 2010 Sep;95(3):313–320. doi: 10.1590/s0066-782x2010005000103. [DOI] [PubMed] [Google Scholar]
- 33.Massierer D, Martinez D, Fuchs SC, Pellin P P, Garcia MS, Zacharias AL, et al. Obstructive sleep apnea, detected by the Berlin Questionnaire: an associated risk factor for coronary artery disease. Cad Saúde Publica. 2012 Aug;28(8):1530–1538. doi: 10.1590/s0102-311x2012000800011. [DOI] [PubMed] [Google Scholar]
- 34.Peker Y, Hedner J, Kraiczi H, Löth S. Respiratoy disturbance index: an independent predictor of mortality in coronary artery disease. Am J Respir Crit Care Med. 2000 Jul;162(1):81–86. doi: 10.1164/ajrccm.162.1.9905035. [DOI] [PubMed] [Google Scholar]
- 35.Cheung YY, Tai BC, Loo G, Khoo SM, Cheong KY, Barbe F, et al. Screening for obstructive sleep apnea in the assessment of coronary risk. Am J Cardiol. 2017 Apr;119(7):996–1002. doi: 10.1016/j.amjcard.2016.11.058. [DOI] [PubMed] [Google Scholar]