Predicting obstructive sleep apnea severity from craniofacial images using ensemble machine learning models

Abstract

Obstructive sleep apnea (OSA) is a prevalent disease affecting 10 to 15% of Americans and nearly one billion people worldwide. It leads to multiple symptoms including daytime sleepiness; snoring, choking, or gasping during sleep; fatigue; headaches; non-restorative sleep; and insomnia due to frequent arousals. Although polysomnography (PSG) is the gold standard for OSA diagnosis, it is expensive, not universally available, and time-consuming, so many patients go undiagnosed due to lack of access to the test. Given the incomplete access and high cost of PSG, many studies are seeking alternative diagnosis approaches based on different data modalities. Here, we propose a machine learning model to predict OSA severity from 2D frontal view craniofacial images. In a cross-validation study of 280 patients, our method achieves an average AUC of 0.780. In comparison, the craniofacial analysis model proposed by a recent study only achieves 0.638 AUC on our dataset. The proposed model also outperforms the widely used STOP-BANG OSA screening questionnaire, which achieves an AUC of 0.52 on our dataset. Our findings indicate that deep learning has the potential to significantly reduce the cost of OSA diagnosis.

Description of Purpose

Obstructive sleep apnea (OSA) is a sleep disorder that occurs when the upper airway gets obstructed, preventing normal breathing [1]. The disease is characterized by episodes of decreased breathing (hypopnea) or cessation of breathing for 10 seconds or more (apnea) during sleep. It has been recognized to be associated with multiple severe complications including drowsy driving and motor vehicle crashes; neuropsychiatric dysfunction; cardiovascular disorders; metabolic syndromes; and liver disease. Overall, OSA is a highly prevalent disease that affects 10 to 15% of Americans and nearly one billion people worldwide [2].

OSA is diagnosed by polysomnography (PSG), a formal in-lab study that monitors patients’ respiration, oxygen saturation, heart rate and electrocardiogram, and electroencephalography (EEG) during sleep. Although PSG is the gold standard, it is expensive (approximately $2100), not universally available, and time-consuming, so about 80% of OSA patients cannot be diagnosed due to lack of access to the test. To make the OSA diagnosis more accessible, many studies have sought alternative diagnosis approaches based on different data modalities, such as computed tomography, magnetic resonance imaging, and craniofacial scan [3–7]. However, radiology imaging data is cumbersome and expensive to acquire. As one of the most easy-to-access modalities, frontal view craniofacial images have significant interaction with the OSA pathogenesis and thus have been studied as a predictor for OSA diagnosis [3, 7, 8]. Here, we propose to develop a machine learning-based model for 2D frontal view craniofacial images to predict OSA severity.

Methods

Dataset

Our dataset contains data from 280 patients who seen our Atrium Health Wake Forest Baptist (AHWFB) sleep medicine clinics. Patients were included in our dataset if they are over 18 years old and provided consent to take part in our study. Craniofacial images from patients’ frontal view were taken by cameras and relevant clinical variables were also collected. The ground truth of OSA was determined by patients’ PSG results in the form of apnea-hypopnea index (AHI). The data were labeled positive (mild/moderate/severe OSA) if their AHI ≥ 5 or negative (none/minimal OSA) if their AHI < 5.

DeepXGBoost

We propose a deep-learning based craniofacial image analysis model, DeepXGBoost, for OSA prediction. It is composed of two modules – an image feature extractor and XGBoost classifier.

For the image feature extractor, we propose a spectral decoupled neural network (SDNet) to learn features related to OSA from 2D frontal-view craniofacial images. The backbone of our SDNet is ResNet50 initialized with ImageNet pretrained weights [9]. To improve the feature learning ability of our model on small datasets, we propose to add a spectral decoupling constraint on top of the regular binary cross-entropy (BCE) loss function [10]. We then train the SDNet on craniofacial images. After the training of SDNet, we use it as a feature extractor to extract feature vectors from images. Then, we train an XGBoost classifier [11] based on the features vectors to predict OSA severity.

Integration of craniofacial images and clinical data

We also use patients’ clinical data as a secondary modality to predict OSA severity. We use a separate XGBoost model to predict OSA from clinical variables. We select variables according to their correlation with AHI using Pearson correlation coefficients. After training, the clinical model outputs are aggregated with DeepXGBoost outputs by average pooling.

Experimental Design

We perform five-fold cross-validation of each model on our dataset to classify patient’s OSA severity. We compare our method with a commonly used OSA screening questionnaire (i.e. STOP-BANG) [12] and a 2D craniofacial model for OSA prediction recently published [8]. We report AUC to evaluate the performance of the models.

Results

Table 1 reports the performance of our DeepXGBoost (craniofacial), ensemble model of DeepXGBoost (craniofacial) and clinical variable model, and the comparison methods. Overall, our craniofacial image-based DeepXGBoost outperform the comparison methods. After combining our DeepXGBoost with clinical variable model’s outputs, the performance is further improved to 0.780 AUC.

Table 1.

Results of OSA severity prediction of different methods

Modalities	Methods	AUC (± standard variation)
Questionnaire	STOP-BANG	$0.529\pm 0.061$
Craniofacial	EfficienNet-OSA (He et al. 2022) [8]	$0.638\pm 0.115$
Craniofacial	DeepXGBoost	$0.733\pm 0.045$
Clinical variables	XGBoost	$0.749\pm 0.037$
Craniofacial + clinical variables	DeepXGBoost (craniofacial) + XGBoost (clinical variables)	0.780 ± 0.066

New or breakthrough work to be presented

We are among the first to develop deep learning based methods to predict OSA severity from 2D craniofacial images. Compared to the widely studied 3D craniofacial scan based methods [7, 13], our proposed modality is much easier to acquire and thus much more practical. To the best of our knowledge, there is only one existing study predict OSA from 2D craniofacial images using deep learning [8]. However, this study is a simple application of EfficientNet model [14] and only achieves moderate results on our dataset (see Table 1 for details). On the other hand, we innovatively utilize spectral decoupling [10] to tackle craniofacial analysis and achieve reasonable prediction performance.

Conclusions

The main purpose of this study was to develop a machine learning based model to predict OSA severity from 2D craniofacial images. To achieve our purpose, we innovatively developed a deep-learning based model: DeepXGBoost which shows the potential of deep learning in identifying OSA patients. We have shown that our DeepXGBoost outperforms the comparison craniofacial-base OSA prediction model and a commonly used OSA screening questionnaire. After combining our DeepXGBoost with clinical model’s output, our method achieves an average AUC of 0.780. With this prediction performance and accessible data modality, our method has the potential to serve as a convenient pre-assessment tool for OSA diagnosis.