Abstract
Study Objectives:
Drug-induced sleep endoscopy (DISE) has been suggested to be a valuable method for determining the obstruction patterns causing sleep-disordered breathing. However, since DISE is not performed throughout the duration of sleep but for less than 1 hour, the reproducibility and reliability of DISE are questionable. Therefore, we aimed to determine the test-retest reliability of DISE using midazolam.
Methods:
Thirty-four patients diagnosed with obstructive sleep apnea were prospectively included in this study. The patients underwent 2 separate DISE examinations that were performed at different days using the same drug and technique. For a more accurate comparison, the depth of sleep and examination time were identically controlled. VOTE classification was used to classify the obstruction findings, and the findings of upper airway obstruction were compared between the 2 tests.
Results:
There were 30 men and four women; the mean age was 45.4 ± 13.1 years. The mean apnea-hypopnea index was 38.3 ± 22.6, and the lowest oxygen saturation was 77.5% ± 12.4%. The lateral wall of the oropharynx, tongue base, and epiglottis showed very good agreement, and the velum showed good agreement between the first and second DISE examinations.
Conclusions:
The reliability of DISE is debatable because it observes only a small portion of the total sleep, but this study suggests that DISE is a reliable test because the findings of 2 separate DISE examinations on different days showed a high concordance rate.
Citation:
Kim J-S, Heo, SJ. Test-retest reliability of drug-induced sleep endoscopy using midazolam. J Clin Sleep Med. 2020;16(5):675–678.
Keywords: obstructive sleep apnea, endoscopy, reproducibility of results, midazolam
BRIEF SUMMARY
Current Knowledge/Study Rationale: Drug-induced sleep endoscopy (DISE) is a valuable method for determining the obstruction patterns of OSA. The diagnostic value of DISE can be improved if numerous reliability studies are conducted. However, to date, only 1 study has investigated the test-retest reliability of DISE and there has been no reliability study using midazolam. This study was conducted to evaluate the test-retest reliability of DISE by using midazolam at the same sleep depth and for the same test time.
Study Impact: This study suggests that DISE may be a reliable test because the findings of 2 separate DISE examinations on different days showed a high concordance. Identical settings for sleep depth and examination time in the first and second DISE examinations may have been responsible for the higher agreement noted in this study.
INTRODUCTION
Positive airway pressure (PAP) is recognized as the treatment of choice for obstructive sleep apnea (OSA).1 Surgery has been considered as an alternative treatment option for patients who do not tolerate PAP treatment. Precise identification of obstruction patterns is one of most important factors to increase the success rate of surgery.2 Several examination methods are available for determining upper airway obstruction sites, such as cephalometry, computed tomography, magnetic resonance imaging, tonsil and tongue grading, and Müller’s maneuver. However, although obstruction of the upper airway causing OSA occurs during sleep, these methods were performed during wakefulness.
Drug-induced sleep endoscopy (DISE) is known to be the best method for determining obstruction sites of the upper airway, because it is performed during sleep when the airway collapse causing OSA occurs.1,3 However, since DISE is not performed throughout the period of sleep but for less than 1 hour, the reproducibility and reliability of DISE assessments are questionable. To date, only 1 study has investigated the test-retest reliability of DISE.4 In that study, DISE showed good test-retest reliability (moderate to substantial according to the site). However, the study did not use objective methods for assessing sleep depth, such as bispectral index (BIS). Evaluations with identical sleep depth are important to compare DISE findings because the obstruction patterns during DISE in deep sleep show a greater degree of obstruction than those in light sleep.5,6 In addition, although the examination time affects the obstruction findings,7 the duration of examination varied widely across previous studies. For a more accurate comparison, the depth of sleep and examination time should be identically controlled.
The diagnostic value of DISE can be improved if numerous reliability studies are conducted by several institutions with properly designed settings. DISE is usually carried out with midazolam as well as propofol,3,8 but there has been no reliability study using midazolam. Therefore, we tried to evaluate the test-retest reliability of DISE by using midazolam at the same sleep depth and for the same test time.
METHODS
Participants
Between January 2014 and June 2016, patients diagnosed as having OSA (apnea-hypopnea index > 5 events/h) by type I polysomnography were prospectively included in this study. Exclusion criteria were as follows: (1) age < 18 years; (2) inadequate general condition to perform DISE such as pregnancy or allergy to midazolam; (3) a previous history of sleep surgery; (4) selection of PAP as a treatment approach; (5) lack of consent to participate in this study. This study was conducted after approval of the Institutional Review Board of our hospital, and written informed consent for participation in the study was obtained from all participants.
Drug-induced sleep endoscopy
All patients underwent 2 separate DISE examinations on different days at the same place (outpatient clinic). The first DISE examination was performed to identify the obstruction sites of the upper airway and plan the treatment method. The second DISE was performed the day before admission for surgery to validate the obstruction pattern observed in the first DISE examination. Both DISE examinations were performed by the same doctor using the same protocol. Our DISE protocol was previously described.7 The nasal mucosa was topically anesthetized and shrunk using a cotton pledget soaked with 4% lidocaine and 0.1% epinephrine. After sufficient anesthesia of the nasal mucosa, the patient was placed in the supine position. Peripheral oxygen saturation was monitored using a pulse oximeter, and the depth of sleep was monitored using BIS evaluations. Midazolam was injected intravenously at a dose of 0.05 mg/kg to induce sleep. If sufficient sedation (BIS range of 65–75) was not achieved, an additional midazolam dose of up to 9 mg was administered under careful monitoring. A flexible nasopharyngoscope was inserted through the nasal cavity at the onset of sleep-disordered breathing. DISE was performed for 20 minutes, and video images were recorded.
The recorded video images were reviewed by 1 surgeon who was not aware of the information of patients, results of sleep study, and surgical plan. VOTE classification was used to assess the findings of obstructions. Patterns of upper airway obstruction between the first and second DISE examinations were compared to determine test-retest reliability. We also compared sites displaying any degree of obstruction (obstruction grade 1 or 2) and no obstruction (obstruction grade 0) between the first and second DISE examinations.
Statistical analysis
Data were statistically analyzed using MedCalc software (version 12.7.4). The paired t test was used to compare the dose of administered midazolam between the first and second DISE examinations. To evaluate the agreement between the upper airway obstruction findings in the 2 DISE examinations Cohen’s weighted kappa was used according to the following interpretation: Cohen’s weighted kappa < 0.20, poor agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, good agreement; and > 0.81, very good agreement.
RESULTS
A total of 34 patients was included in this study. The numbers of men and women were 30 and 4, respectively; their mean age was 45.4 ± 13.1 years, and mean body mass index was 27.0 ± 2.5 kg/m2 (Table 1). Mean apnea-hypopnea index was 38.3 ± 22.6 events/h; mean nadir of peripheral oxygen saturation was 75.5% ± 12.4%; and the mean duration of peripheral oxygen saturation < 90% was 28.7 ± 49.1 minutes. Mean Epworth Sleepiness Scale score was 8.9 ± 4.6; mean Friedman tonsil and tongue grade were 1.3 ± 0.6 and 2.6 ± 0.7, respectively; and the mean Friedman stage was 2.6 ± 0.7. Midazolam was additionally administered in 15 (44.1%) and 12 (35.3%) patients in the first and second DISE, respectively (P = .621). The midazolam dose was not significantly different between the first and second DISE examinations (4.3 ± 1.3 mg vs 4.1 ± 1.5 mg, P = .692). The mean difference of time interval between the first and second DISE was 31.2 ± 8.4 days.
Table 1.
Baseline characteristics of patients.
| Variables | Number or Mean ± SD | Range |
|---|---|---|
| Sex: Male/Female | 30/4 | |
| Age, y | 44.3 ± 12.1 | 18–60 |
| BMI, kg/m2 | 26.3 ± 3.0 | 21.7–31.0 |
| AHI, events/h | 33.4 ± 19.9 | 7.2–81.9 |
| Nadir of SpO2, % | 75.5 ± 12.4 | 54.1–92.0 |
| Duration of SpO2 < 90%, min | 28.7 ± 49.1 | 0–226.4 |
| ESS score | 8.9 ± 4.6 | 3–19 |
| Friedman tonsil grade | 1.3 ± 0.6 | 1–3 |
| Friedman tongue position | 2.6 ± 0.7 | 1–3 |
| Friedman stage | 2.6 ± 0.7 | 1–3 |
Continuous data are presented as a number or as the mean ± standard deviation (SD) and range; n = 34 patients. AHI = apnea-hypopnea index, BMI = body mass index, ESS = Epworth Sleepiness Scale, SpO2 = peripheral oxygen saturation.
In a comparison of the configuration of the upper airway obstruction between the first and second DISE examinations, the velum showed 97.1% agreement and a Cohen’s weighted kappa value of 0.94, whereas the epiglottis showed 100% agreement and a Cohen’s weighted kappa value of 1.00 (Table 2). In a similar comparison of the grade of upper airway obstruction between the 2 DISE examinations, the velum showed 85.3% agreement and a Cohen’s weighted kappa value of 0.54, the lateral wall of the oropharynx showed 97.0% agreement and a Cohen’s weighted kappa value of 0.96, the tongue base showed 91.2% agreement and a Cohen’s weighted kappa value of 0.88, and the epiglottis showed 100% agreement and a Cohen’s weighted kappa value of 1.00 (Table 3).
Table 2.
Comparison of the configuration of upper airway obstruction between the first and second DISE examinations.
| Obstruction Site | 1st DISE | 2nd DISE | Agreement, % | Cohen’s Weighted Kappa | 95% CI | ||||
|---|---|---|---|---|---|---|---|---|---|
| AP | Lat | Circ | AP | Lat | Circ | ||||
| Velum | 11 | 1 | 22 | 10 | 1 | 23 | 97.1 | 0.94 | 0.82-1.06 |
| Epiglottis | 6 | 3 | NA | 6 | 3 | NA | 100 | 1.00 | 1.00-1.00 |
AP = anteroposterior, CI= confidence interval, Circ = circular, DISE = drug-induced sleep endoscopy, Lat = lateral, NA = nonavailable.
Table 3.
Comparison of the grade of upper airway obstruction between the first and second DISE examinations.
| Obstruction Site | Obstruction Grade of 1st DISE | Obstruction Grade of 2nd DISE | Agreement, % | Cohen’s Weighted Kappa | 95% CI | ||||
|---|---|---|---|---|---|---|---|---|---|
| 0 | 1 | 2 | 0 | 1 | 2 | ||||
| Velum | 0 | 7 | 27 | 1 | 4 | 29 | 85.3 | 0.54 | 0.17–0.91 |
| Lateral wall of oropharynx | 10 | 14 | 10 | 10 | 15 | 9 | 97.0 | 0.96 | 0.89–1.03 |
| Tongue base | 9 | 17 | 8 | 8 | 18 | 8 | 91.2 | 0.88 | 0.75–1.01 |
| Epiglottis | 25 | 7 | 2 | 25 | 7 | 2 | 100 | 1.00 | 1.00–1.00 |
CI = confidence interval, DISE = drug-induced sleep endoscopy.
In a comparison of sites displaying any degree of obstruction in the upper airway between the first and second DISE examinations, the velum showed 96.7% agreement, both lateral wall of the oropharynx and epiglottis showed 100% agreement and Cohen’s weighted kappa of 1.00, and the tongue base showed 97.1% agreement and a Cohen’s weighted kappa value of 0.91 (Table 4).
Table 4.
Comparison of the sites displaying any degree of obstruction between the first and second DISE examinations.
| Obstruction Site | Obstruction Grade of 1st DISE | Obstruction Grade of 2nd DISE | Agreement, % | Cohen’s Weighted Kappa | 95% CI | ||
|---|---|---|---|---|---|---|---|
| 0 | 1 or 2 | 0 | 1 or 2 | ||||
| Velum | 0 | 34 | 1 | 33 | 96.7 | NA | NA |
| Lateral wall of oropharynx | 10 | 24 | 10 | 24 | 100 | 1.00 | 1.00–1.00 |
| Tongue base | 9 | 25 | 8 | 26 | 97.1 | 0.91 | 0.72–1.09 |
| Epiglottis | 25 | 9 | 25 | 9 | 100 | 1.00 | 1.00–1.00 |
CI = confidence interval, DISE = drug-induced sleep endoscopy, NA = nonavailable.
DISCUSSION
This study showed very good agreement in the test-retest reliability of DISE. Although direct comparison between the previous study4 and this study is difficult due to their different classifications and statistical methods, the degree of agreement was slightly higher in this study. The higher agreement may be attributable to the controlled depth of sleep and equal duration of both DISE examinations. The previous study used the Modified Ramsay score and Observer’s Assessment of Alertness/Sedation score to evaluate the sleep depth. These methods are self-report assessment tools and use stimuli such as light glabellar tap, loud sound, and shaking to assess the depth of sleep. These stimuli may disturb the chance of maintaining a constant depth of sleep during DISE. Unlike these methods, BIS indicates sleep depth numerically using electroencephalography and does not disrupt sleeping. Since this study used BIS to control sleep depth, better comparison might be achieved.
Another possible factor contributing to the higher agreement in this study compared is the duration of DISE. Since obstructive patterns change depending on the duration of the DISE procedure, the DISE examination time is important.7 Longer DISE examinations can reveal more obstruction sites, which can be missed when the examination time is too short. This study used the same duration of DISE to reduce confounding factors of reliability.
In addition, the slight difference in the agreement between this study and the previous study4 may be influenced by the use of different sedating agents (midazolam vs propofol). Although previous studies showed a good correlation in the comparison of obstruction pattern between midazolam and propofol in DISE,9,10 the agreement of the tongue base and velum was low in the study of Viana et al9 and in the Carrasco-Llatas et al10, respectively. Thus, although midazolam had a higher concordance rate with propofol than other drugs in the study of Viana et al9, the difference of sedative drugs could influence on the different agreement in the findings of DISE.
Information regarding the obstruction sites presented by DISE is useful to create a treatment plan for OSA. In the study by Thaler et al,2 multilevel surgery using DISE showed good surgical outcomes. This favorable outcome might have resulted from the fact that DISE properly determined the obstruction pattern of upper airway. However, in the study by Golbin et al,11 patients who underwent uvulopalatopharyngoplasty with or without tonsillectomy without DISE did not show a statistically significant difference in outcomes compared with those who underwent DISE with other procedures, including transoral robotic surgery. Thus, DISE did not contribute to the success rate in that study. Although both studies used DISE to prepare a surgical plan and used the same surgical procedures, the results were quite different. These contradictory results may be attributable to inappropriate surgical procedures, but they may also be due to the reliability problem of DISE. Therefore, many studies on the reliability of DISE are needed.
In the study by Kezirian et al,12 the interrater reliability of DISE was moderate to substantial. Carrasco-Llatas et al,13 also concluded that DISE is a reliable technique even when assessing interobserver agreement between an expert observer and an observer in training. However, Vroegop et al14 insisted that experience in performing DISE is necessary to obtain reliable observations because both interobserver and intraobserver agreement were higher in experienced versus nonexperienced surgeons. In contrast, the study by Altintas et al15 showed poor to good interobserver consistency of DISE and concluded that more studies are required. Although there are several studies investigating the interobserver reliability of DISE, a test-retest study of DISE was reported in only 1 paper. Therefore, we assume that this study will be helpful to determine the reliability of DISE.
This study was limited by the small sample size. Although DISE has proven to be a safe technique,1 many patients were not willing to attend this study due to the repetitive use of sedative drugs. Patients agreed to receive sedative drugs for determination of the treatment method, but they were reluctant to receive additional sedative drugs for research. The previous study was performed over 5 years, but included only 32 patients.4 A multicenter study is likely to be needed because there is a limit to the number of participants that can be included in a single-center test-retest study of DISE.
CONCLUSIONS
The reliability of DISE is debatable because it only observes a small portion of the total sleep, but this study suggests that DISE may be a reliable test because the findings of two separate DISE examinations on different days showed a high concordance rate. Similar to DISE with propofol in the previous study, DISE with midazolam also showed good test-retest results. Identical settings for sleep depth and examination time in the first and second DISE examinations may have been responsible for the higher agreement noted in this study. To better understand the reliability of DISE, more patients need to be included in a study that is well controlled and performed in an identical setting.
DISCLOSURE STATEMENT
All authors have seen and approved the manuscript. The authors report no conflicts of interest.
ABBREVIATIONS
- BIS
bispectral index
- DISE
drug-induced sleep endoscopy
- OSA
obstructive sleep apnea
- PAP
positive airway pressure
REFERENCES
- 1.De Vito A, Carrasco Llatas M, Vanni A, et al. European position paper on drug-induced sedation endoscopy (DISE). Sleep Breath. 2014;18(3):453–465. 10.1007/s11325-014-0989-6 [DOI] [PubMed] [Google Scholar]
- 2.Thaler ER, Rassekh CH, Lee JM, Weinstein GS, O’Malley BW Jr. Outcomes for multilevel surgery for sleep apnea: Obstructive sleep apnea, transoral robotic surgery, and uvulopalatopharyngoplasty. Laryngoscope. 2016;126(1):266–269. 10.1002/lary.25353 [DOI] [PubMed] [Google Scholar]
- 3.De Vito A, Carrasco Llatas M, Ravesloot MJ, et al. European position paper on drug-induced sleep endoscopy: 2017 Update. Clin Otolaryngol. 2018;43(6):1541–1552. 10.1111/coa.13213 [DOI] [PubMed] [Google Scholar]
- 4.Rodriguez-Bruno K, Goldberg AN, McCulloch CE, Kezirian EJ. Test-retest reliability of drug-induced sleep endoscopy. Otolaryngol Head Neck Surg. 2009;140(5):646–651. 10.1016/j.otohns.2009.01.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Hong SD, Dhong HJ, Kim HY, et al. Change of obstruction level during drug-induced sleep endoscopy according to sedation depth in obstructive sleep apnea. Laryngoscope. 2013;123(11):2896–2899. 10.1002/lary.24045 [DOI] [PubMed] [Google Scholar]
- 6.Kellner P, Herzog B, Plossl S, et al. Depth-dependent changes of obstruction patterns under increasing sedation during drug-induced sedation endoscopy: results of a German monocentric clinical trial. Sleep Breath. 2016;20(3):1035–1043. 10.1007/s11325-016-1348-6 [DOI] [PubMed] [Google Scholar]
- 7.Heo SJ, Park CM, Kim JS. Time-dependent changes in the obstruction pattern during drug-induced sleep endoscopy. Am J Otolaryngol. 2014;35(1):42–47. 10.1016/j.amjoto.2013.08.017 [DOI] [PubMed] [Google Scholar]
- 8.Vanderveken OM. Drug-induced sleep endoscopy (DISE) as a guide towards upper airway behavior and treatment outcome: the quest for a vigorous standardization of DISE. Sleep Breath. 2018;22(4):897–899. 10.1007/s11325-018-1743-2 [DOI] [PubMed] [Google Scholar]
- 9.Viana A, Zhao C, Rosa T, et al. The effect of sedating agents on drug-induced sleep endoscopy findings. Laryngoscope. 2019;129(2):506–513. 10.1002/lary.27298 [DOI] [PubMed] [Google Scholar]
- 10.Carrasco Llatas M, Agostini Porras G, Cuesta Gonzalez MT, et al. Drug-induced sleep endoscopy: a two drug comparison and simultaneous polysomnography. Eur Arch Otorhinolaryngol. 2014;271(1):181–187. 10.1007/s00405-013-2548-3 [DOI] [PubMed] [Google Scholar]
- 11.Golbin D, Musgrave B, Succar E, Yaremchuk K. Clinical analysis of drug-induced sleep endoscopy for the OSA patient. Laryngoscope. 2016;126(1):249–253. 10.1002/lary.25516 [DOI] [PubMed] [Google Scholar]
- 12.Kezirian EJ, White DP, Malhotra A, Ma W, McCulloch CE, Goldberg AN. Interrater reliability of drug-induced sleep endoscopy. Arch Otolaryngol Head Neck Surg. 2010;136(4):393–397. 10.1001/archoto.2010.26 [DOI] [PubMed] [Google Scholar]
- 13.Carrasco-Llatas M, Zerpa-Zerpa V, Dalmau-Galofre J. Reliability of drug-induced sedation endoscopy: interobserver agreement. Sleep Breath. 2017;21(1):173–179. 10.1007/s11325-016-1426-9 [DOI] [PubMed] [Google Scholar]
- 14.Vroegop AV, Vanderveken OM, Wouters K, et al. Observer variation in drug-induced sleep endoscopy: experienced versus nonexperienced ear, nose, and throat surgeons. Sleep. 2013;36(6):947–953. 10.5665/sleep.2732 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Altintaş A, Yegin Y, Celik M, Kaya KH, Koc AK, Kayhan FT. Interobserver consistency of drug-induced sleep Endoscopy in diagnosing obstructive sleep apnea using a VOTE classification system. J Craniofac Surg. 2018;29(2):e140–e143. 10.1097/SCS.0000000000003876 [DOI] [PubMed] [Google Scholar]