Moderate to severe OSA screening based on support vector machine of the Chinese population faciocervical measurements dataset: a cross-sectional study

Abstract

Objectives

Obstructive sleep apnoea (OSA) has received much attention as a risk factor for perioperative complications and 68.5% of OSA patients remain undiagnosed before surgery. Faciocervical characteristics may screen OSA for Asians due to smaller upper airways compared with Caucasians. Thus, our study aimed to explore a machine-learning model to screen moderate to severe OSA based on faciocervical and anthropometric measurements.

Design

A cross-sectional study.

Setting

Data were collected from the Shanghai Jiao Tong University School of Medicine affiliated Ruijin Hospital between February 2019 and August 2020.

Participants

A total of 481 Chinese participants were included in the study.

Primary and secondary outcome

(1) Identification of moderate to severe OSA with apnoea–hypopnoea index 15 events/hour and (2) Verification of the machine-learning model.

Results

Sex-Age-Body mass index (BMI)-maximum Interincisal distance-ratio of Height to thyrosternum distance-neck Circumference-waist Circumference (SABIHC2) model was set up. The SABIHC2 model could screen moderate to severe OSA with an area under the curve (AUC)=0.832, the sensitivity of 0.916 and specificity of 0.749, and performed better than the STOP-BANG (snoring, tiredness, observed apnea, high blood pressure, BMI, age, neck circumference, and male gender) questionnaire, which showed AUC=0.631, the sensitivity of 0.487 and specificity of 0.772. Especially for asymptomatic patients (Epworth Sleepiness Scale <10), the SABIHC2 model demonstrated better predictive ability compared with the STOP-BANG questionnaire, with AUC (0.824 vs 0.530), sensitivity (0.892 vs 0.348) and specificity (0.755 vs 0.809).

Conclusion

The SABIHC2 machine-learning model provides a simple and accurate assessment of moderate to severe OSA in the Chinese population, especially for those without significant daytime sleepiness.

Strengths and limitations of this study.

• This is the first study to assess the screening value of faciocervical measurements in predicting moderate to severe obstructive sleep apnoea (OSA).

• Support vector machine, a well-known machine-learning method, was used to set up the model to screen moderate to severe OSA in the Chinese population.

• Instead of most questionnaires based on symptoms and comorbidities, the Sex-Age-Body mass index-maximum Interincisal distance-ratio of Height to thyrosternum distance-neck Circumference-waist Circumference model which is entirely based on measurements, avoids the confounding effect of symptoms and comorbidities.

• The cross-sectional design of the study makes it difficult to establish the causal relationship between variables.

• The study is a single-centre study with a relatively small sample size, and further confirmation with large-sample, multicentre, prospective clinical trial is needed in the future.

Introduction

Obstructive sleep apnoea (OSA) is a common breathing sleep disorder that affects about 936 million adults globally,¹ of which about 80% are estimated to be undiagnosed.² In the Chinese population, approximately 175 million adults have mild to severe OSA, of whom about 65 million adults have moderate to severe OSA.¹ In addition to cardiovascular injury and metabolic syndrome, OSA has recently been regarded as a risk factor for perioperative complications, including hypoxaemia, pneumonia, pulmonary embolism, unplanned transfer to the intensive care unit and even death, especially for those who received abdominal or vascular surgery.³ The rates of postoperative cardiovascular events show a rise in moderate to severe OSA (25.1%) compared with no or mild OSA (16.8%).⁴ Thirty-eight per cent of surgical patients had moderate to severe OSA, of which 68.5% were not diagnosed before surgery.⁵ Guidelines have been recommended to develop a local protocol for screening possible OSA patients before elective surgery.^{6 7} The most common screening scales, including STOP-BANG (snoring, tiredness, observed apnea, high blood pressure, BMI, age, neck circumference and male gender), Berlin questionnaire and OSA50, are mainly based on symptoms and comorbidities,⁸ which might lead to missed diagnoses for those without significant daytime sleepiness.

Several factors contribute to the pathogenesis of OSA, including obesity, faciocervical anatomy and alteration in pharyngeal muscle function, etc.⁹ Asians have relatively smaller upper airways compared with Caucasians.¹⁰ Only in Asians, smaller upper airways are predictors of upper airway collapsibility, and an anatomic imbalance between tongue and mandible volume influenced upper airway collapsibility among Caucasians.¹¹ The above evidence prompts faciocervical characteristics that may predict OSA for Asians, such as thyromental distance (TMD), thyrosternum distance (TSD), maximum interincisal distance (MID) and Mallampati test score, which have been widely used in predicting difficult intubation.¹² However, no studies assessed whether faciocervical characteristics are suitable for predicting OSA. Thus, it is necessary to evaluate the relationship between faciocervical characteristics and OSA.

During the past two decades, machine-learning models have provided simple but effective approaches for improving diagnostic accuracy.¹³ Support vector machine (SVM) is a well-known classification technique and has achieved great success in bioinformatics applications. Thus, the study aimed to build a new model via SVM and monitor its effectiveness in screening OSA.

Methods

Study design

Participants with suspected OSA (snoring, witnessed apnoea or excessive daytime sleepiness, etc) were enrolled in the study from Ruijin Hospital, Shanghai Jiao Tong University School of Medicine between February 2019 and August 2020. All participants underwent anthropometric measurements and subsequent overnight polysomnography (PSG). Participants were grouped according to their Apnoea-Hypopnoea Index (AHI) gain into the following: (1) no or mild OSA (subjects without moderate to severe OSA: AHI<15 events/hour), (2) moderate to severe OSA (moderate to severe OSA: AHI≥15 events/hour). Exclusion criteria: (1) patients showing complications with severe respiratory diseases, such as severe chronic obstructive pulmonary disease, interstitial lung disease, or acute asthma; (2) patients showing complications with serious cardiovascular diseases such as acute myocardial infarction, acute heart failure or chronic congestive heart failure (grades III and IV); (3) patients with mental illnesses who could not cooperate with the examination; (4) patients who receiving non-invasive positive pressure ventilation therapy and (5) patients who might have other sleep disorders under clinical evaluation.

Written informed consent was obtained from all participants. Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Clinical characteristics and anthropometric measurements

Age, sex, height, weight, body mass index (BMI), neck circumference (NC), waist circumference (WC), Epworth Sleepiness Scale (ESS) and STOP-BANG questionnaire were recorded. The STOP-BANG questionnaire is a scoring model consisting of eight questions and its scores are based on yes/no answers (score: 1/0). The eight questions included snoring, tiredness, observed apnoea, high blood pressure, BMI, age, NC and gender. Faciocervical measurements including Mallampati score, MID, TMD and TSD were measured. The Mallampati score was evaluated when participants were asked to sit upright and open their mouths as wide as possible. In grade I, the entire uvula, faucial pillar and soft palate are visible. In grade II, part of the uvula and palate are visible. In grade III, the soft palate is visible, but the uvula is obscured.¹⁴ MID was recorded by asking the participant to sit upright and open the mouth as wide as possible.¹⁵ The TMD was measured as the straight distance between the thyroid notch and the lower border of the mental prominence, while the head was fully extended, and the mouth closed.¹⁶ TSD was measured as the distance between the thyroid notch and the upper border of the sternum. The distance was rounded to the nearest 0.5 cm. The ratios of height to TMD (H/TMD) and height to TSD (H/TSD) were calculated.¹⁶

Polysomnography

All participants underwent overnight PSG. No coffee, tea, caffeine-containing products or sedative hypnotics were taken before sleep. The PSG monitoring included electroencephalography, electrooculography, chin electromyography, electrocardiography, measurements of thoracal and abdominal movements, and airflow pressure and thermistor (Alice 5, Philips Respironics, USA), with the addition of oxygen saturation (Nonin, Herrsching, Germany). Sleep recordings were scored according to the American Academy of Sleep Medicine (AASM) 2007 criteria.¹⁷ The diagnosis of moderate to severe OSA was defined by the presence of an obstructive AHI≥15 events/hour according to AASM guidelines.

Statistical analysis

Continuous variables with a normal distribution are presented as mean±SD, while values without a normal distribution are presented as median (25–75th percentiles). Categorical variables are presented as numbers and percentages. Independent samples t-test and Mann-Whitney U test were used to determine differential risk factors between the two groups. The χ² test and Fisher’s exact test were used to compare the categorical data, as appropriate. Logistic regression analyses were performed to calculate the OR. Receiver operating characteristic (ROC) analysis was used to determine the area under the curve (AUC) with a 95% CI, sensitivity and specificity. All tests were two sided and used a significance level of 0.05. All analyses were performed using the SPSS software (V.24.0; SPSS).

Model construction via SVM

SVM is a representative machine-learning algorithm for classification, which could be viewed as a nonlinear regression model. We used the SVM to capture the potential hyperplane that maximises the margin between moderate or severe OSA and no or mild OSA. The SVM has the following format:

$$C={\Sigma }_{i=1}^n{\alpha }_{i}k\left(x_i,x\right)+b$$

Where c is the output of the model based on the new data x, which could be regarded as the classifier (0 or 1), x_i (i=1,…, n) is the training dataset, α_i and b are parameters determined by the algorithm.¹⁸ We used the Gaussian radial basis function as the kernel function, with γ of 0.23 and the box constraint set to 20. Based on the results of the statistical analysis, sex, age, NC, WC, BMI, MID and H/TSD were set as independent variables (p<0.05). We performed a significant principal component taking into account the strong collinearity among parameters, such as BMI and WC. Five principal components were selected according to the accumulative variance contribution of more than 90% and scree plot.¹⁹

The performance of the SVM methods was evaluated by the 10-fold cross-validation. The participants were stratified sampling into 10 subsamples: one formed the test dataset for verifying the effectiveness of the model, and the others formed the training dataset to predict moderate to severe OSA for the model. The SVM model, namely SABIHC, that is, sex, age, BMI, MID, H/TSD, NC and WC (2 Cs) was estimated by the training dataset and then applied to predict OSA in the testing dataset. The flow chart including screening, randomisation and algorithm is presented in figure 1.

Figure 1.

Flow chart showing screening, randomisation and algorithm. A total of 481 participants were enrolled in our study after excluding 31 patients. Age, sex, neck circumference (NC), waist circumference (WC), body mass index (BMI) and faciocervical measurements (maximum interincisal distance (MID), height to thyrosternum distance (H/TSD)) were potential factors related to OSA due to the significant difference between the two groups (p<0.05). We chose the following parameters to set up the SABIHC2 model based on training dataset, whose name refers to sex, age, BMI, MID, H/TSD, NC and WC (2 Cs). Then, the model was verified on testing dataset. H/TMD, ratio of height to thyromegaly distance; OSA, obstructive sleep apnoea; SABIHC2, Sex-Age-Body mass index-maximum Interincisal distance-ratio of Height to thyrosternum distance-neck Circumference-waist Circumference; SVM, support vector machine.

The screening performance was assessed using sensitivity, specificity, AUC, negative likelihood ratio (−LR), positive likelihood ratio (+LR) and accuracy. Visual optimal hyperplanes for the SVM are shown in figure 2. Finally, we compared the difference in predictive power between the STOP-BANG questionnaire and the SVM model. Analyses were performed using the Scikit-learn package (V.0.20.3) based on Python (V.3.5.1).

Figure 2.

Optimal hyperplane for support vector machine (SVM). The green area means the SVM predicting Apnoea–Hypopnoea Index (AHI) <15 events/hour, the purple means the SVM predicts the AHI ≥15 events/hour. Green dots represent no or mild obstructive sleep apnoea (OSA) (AHI <15 events/hour), blue dots represent moderate to severe OSA (AHI ≥15 events/hour). The accuracy of SABIHC2 model based on all patients was 0.857, of which 0.916 for moderate to severe OSA (blue dots in purple area), and 0.748 for no or mild OSA (green dots in green area). The boundary was obtained from the SVM classifier and the figure was created using python. SABIHC2, Sex-Age-Body mass index-maximum Interincisal distance-ratio of Height to thyrosternum distance-neck Circumference-waist Circumference.

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Results

A total of 512 participants were recruited for the study at first, among which 31 patients were excluded due to short total sleep time (<200 min, 13 cases) or missing data (18 cases). Finally, 481 participants (325 males and 156 females; aged between 14 and 77 years) were enrolled in our study (figure 1). The participants were divided into no or mild OSA (AHI<15 events/hour, n=171, mean AHI 7.01±4.47 events/hour) and moderate to severe OSA (AHI≥15 events/hour, n=310, mean AHI 42.4±20.7 events/hour). Increased BMI, NC, WC, older age and a higher percentage of males were found in moderate to severe OSA (p<0.001). Meanwhile, they had a higher Microarousal index (32.5±13.2 vs 16.4±8.3 events/hour) and lower lowest pulse oxygen saturation (72.1±13.3 vs 87.13%±6.3%) than no or mild OSA (p<0.001) (table 1).

Table 1.

Subject demographics, crude and adjusted associations between morphometric variables and moderate to severe OSA

	All patients (n=481)	AHI<15 (n=171)	AHI≥15 (n=310)	P value*	Crude OR (95% CI)†	P value‡	Adjusted OR for BMI (95% CI)†	P value‡
General characteristics
Male (%)	325 (67.6%)	93 (54.39%)	232 (74.84%)	<0.001	0.401 (0.270 to 0.595)	<0.001	0.463 (0.308 to 0.697)	<0.001
Age	47.32±12.94	44.05±13.80	49.11±12.08	<0.001	1.032 (1.016 to 1.048)	<0.001	1.037 (1.021 to 1.054)	<0.001
NC	39.05±4.17	37.63±3.74	39.83±4.20	<0.001	1.146 (1.090 to 1.205)	<0.001	1.121 (0.054 to 1.193)	<0.001
WC	94.97±11.32	90.58±11.93	97.39±10.20	<0.001	1.109 (1.041 to 1.081)	<0.001	1.074 (1.041 to 1.107)	<0.001
BMI	26.26±4.16	25.18±4.16	26.85±4.04	<0.001	1.109 (1.056 to 1.166)	<0.001	N/A	N/A
Faciocervical measurements
MID	4.88±0.88	4.76±0.79	4.94±0.92	0.028	1.275 (1.025 to 1.585)	0.029	1.298 (1.037 to 1.626)	0.023
Mallampati test=1	195 (40.5%)	77 (45.02%)	118 (38.06%)	0.12	N/A	N/A	N/A	N/A
Mallampati test=2	142 (29.5%)	54 (31.58%)	88 (28.39%)		0.589 (0.371 to 0.938)§	0.026§	0.679 (0.421 to 1.095)§	0.113§
Mallampati test=3	144 (29.9%)	40 (23.40%)	104 (33.55%)		0.627 (0.381 to 1.031)§	0.066§	0.713 (0.428 to 1.186)§	0.192§
H/TMD	18.98±2.75	18.69±2.30	19.14±2.95	0.085	1.064 (0.991 to 1.142)	0.086	1.071 (0.998 to 1.150)	0.057
H/TSD	18.92±2.97	17.49±2.06	19.70±3.10	<0.001	1.448 (1.311 to 1.599)	<0.001	1.458 (1.417 to 1.614)	<0.001

Data are presented as mean± SD or n (%).

*t-test or χ² test as appropriate.

†ORs are depicted for moderate to severe OSA relative to no or mild OSA.

‡Logistic regression.

§ORs of Mallampati test are depicted for Mallampati test=2 (or 3) relative to Mallampati test=1.

AHI, Apnoea–Hypopnoea Index; BMI, body mass index; H/TMD, ratio of height to thyromegaly distance; H/TSD, ratio of height to thyrosternum distance; MID, maximum interincisal distance; N/A, not applicable; NC, neck circumference; OSA, obstructive sleep apnoea; WC, waist circumference.

Associations between faciocervical characteristics and OSA

Greater MID (4.94 vs 4.76, p=0.028) and H/TSD (19.70 vs 17.49, p<0.001) were found in the moderate to severe group compared with no or mild OSA. MID (OR=1.275, adjusted OR=1.298) and H/TSD (OR=1.448, adjusted OR=1.458) are associated with moderate to severe OSA, even after controlling for BMI. No significant intergroup differences were found in Mallampati test scores and H/TMD (p>0.05) (table 1).

Setup and predictive accuracy of the Sex-Age-Body mass index-maximum Interincisal distance-ratio of Height to thyrosternum distance-neck Circumference-waist Circumference model

Age, sex, NC, WC, BMI and faciocervical measurements (MID, H/TSD) were chosen as covariates because a significant difference was found between the two groups. We set up the Sex-Age-Body mass index-maximum Interincisal distance-ratio of Height to thyrosternum distance-neck Circumference-waist Circumference (SABIHC2) model, whose name refers to sex, age, BMI, MID, H/TSD, NC and WC (2 Cs). The AUC was 0.832 (95% CI 0.790 to 0.875), with a sensitivity of 0.916 and specificity of 0.749. Corresponding LR+and LR− were 3.649 and 0.112, respectively (table 2, figure 3A). The accuracy of the SABIHC2 model was 0.857, of which 0.916 for the moderate to severe group (blue dots in the purple area; figure 2), and 0.748 for no or mild OSA (green dots in the green area; figure 2).

Table 2.

Performance of SABIHC2 model

	Sensitivity	Specificity	AUC	95% CI	+LR	−LR
SABIHC2	0.916	0.749	0.832	0.790 to 0.875	3.649	0.112
STOP-BANG	0.487	0.772	0.631	0.581 to 0.682	2.136	0.665

AUC, area under the curve; +LR, positive likelihood ratio; −LR, negative likelihood ratio; SABIHC2, Sex-Age-Body mass index-maximum Interincisal distance-ratio of Height to thyrosternum distance-neck Circumference-waist Circumference; STOP-BANG, snoring, tiredness, observed apnea, high blood pressure, BMI, age, neck circumference, and male gender.

Figure 3.

Receiver operating characteristic (ROC) curve of SABIHC2 model and Stop-Bang. (A) The ROC curve of SABIHC2 model and STOP-BANG. The AUC of the SABIHC2 model was 0.832 (95% CI 0.790 to 0.875), with a sensitivity of 0.916 and specificity of 0.749. The AUC of STOP-BANG questionnaire was 0.631 (95% CI 0.581 to 0.682), with a sensitivity of 0.487 and specificity of 0.772. (B) The ROC curve of SABIHC2 model and STOP-BANG based on asymptomatic patients (Epworth Sleepiness Scale, ESS<10) and sleepiness patients (ESS≥10). In asymptomatic patients, SABIHC2 model (orange line) remarkably demonstrated better predictive ability than Epworth Sleepiness Scale questionnaire (soft blue line), with AUC (0.824 vs 0.530), sensitivity (0.892 vs 0.348) and specificity (0.755 vs 0.809). Similarly, SABIHC2 model (red line) had higher predictive power than STOP-BANG questionnaire (dark blue line) in sleepiness patients (ESS≥10), with sensitivity (0.941 vs 0.632), and specificity (0.740 vs 0.727) and AUC (0.841 vs 0.720). AUC, area under the curve; ESS, Epworth Sleepiness Scale; SABIHC2, Sex-Age-Body mass index-maximum Interincisal distance-ratio of Height to thyrosternum distance-neck Circumference-waist Circumference; SEN, sensitivity; SPE, specificity; STOP-BANG, snoring, tiredness, observed apnea, high blood pressure, body mass index, age, neck circumference and male gender.

Discriminative ability of the SABIHC2 model and STOP-BANG questionnaire

To compare the predictive ability between the SABIHC2 model and the STOP-BANG questionnaire, we calculated the ROC curve of STOP-BANG for moderate to severe OSA. The AUC was 0.631 (95% CI 0.581 to 0.682), with a sensitivity of 0.487 and specificity of 0.772. (table 2, figure 3A).

As the STOP-BANG questions are based on the symptoms and comorbidities and may result in missed diagnoses, we further compared whether there are differences in the predictive capacity of the SABIHC2 model and STOP-BANG questionnaire both for the symptomatic (ESS ≥10) and asymptomatic patients (ESS <10). For asymptomatic patients, the SABIHC2 model demonstrated better predictive ability than the STOP-BANG questionnaire, with AUC (0.824 vs 0.530), sensitivity (0.892 vs 0.348) and specificity (0.755 vs 0.809). Similarly, the SABIHC2 model had higher predictive power than the STOP-BANG questionnaire in patients experiencing sleepiness (ESS ≥10), with AUC (0.841 vs 0.720), sensitivity (0.941 vs 0.632) and specificity (0.740 vs 0.727) (figure 3B).

Discussion

In the current study, we measured a broad range of anthropometric variables and assessed the association with moderate to severe OSA. Sex, age, BMI, MID, H/TSD, NC and WC were significant determining factors. We also developed the SABIHC2 predictive model, which provides a simple and accurate assessment of moderate to severe OSA.

Faciocervical measurements are essential factors related to OSA. The majority of previous studies focused on the soft-tissue of the upper airway. A significant correlation between the oropharyngeal soft tissue (Mallampati test and tonsillar enlargement, etc) and AHI was reported.^{20 21} However, the Mallampati test was not a significant parameter in our study. It is possible that the association between soft tissue and OSA is weak in Asian populations.¹¹ No previous studies explored the relationship between moderate to severe OSA and faciocervical measurements, such as MID, H/TMD and H/TSD, which have been frequently used to evaluate difficult intubation. Herein, we found that MID and H/TSD were strongly associated with moderate to severe OSA, and H/TSD showed a significant correlation with moderate to severe OSA, even controlling for BMI. The result may suggest that H/TSD is a potential factor for moderate to severe OSA in Chinese subjects.

As indicated previously, the overall scarcity and labour and financially onerous nature of PSG has prompted exploration of other suitable screening approaches, ranging from questionnaires to simplified multichannel recording. The STOP-BANG is one of the most widely used questionnaires. Some limitations of the STOP-BANG questionnaire should be considered. First, it is based on simplified categories (0 or 1) and this might reduce its screening accuracy.²² Second, the evaluation is based on symptoms and comorbidities, which may lead to the omission of diagnosis for snorers without significant daytime sleepiness or hypertension.²³ Thus, we developed a new screening tool based on faciocervical measurements by machine learning.

Considering non-linear relationships and data interaction effects,²⁴ the machine-learning algorithm (SVM) was used to find the complex linkages between variables. Distinct from previous machine-learning models which were based on age, sex, NC, WC and BMI,^{25 26} we included the structural faciocervical measurements to construct the SABIHC2 model. This showed much better performance than the STOP-BANG in screening for OSA.

The results confirmed that the SABIHC2 model performed better than the STOP-BANG questionnaire in predicting moderate to severe OSA, especially for asymptomatic patients (ESS <10), which prompted that it may be particularly useful for this cohort of patients. As most models work, we are planning to develop a software or application in the future, to allow healthcare worker-friendly installation and application.

To our knowledge, this is the first study examining the predictive power of faciocervical measurements, which may become a new index to predict OSA besides sex, age and BMI. However, several limitations should be mentioned. First, the study cohort was based on the Chinese population. The results obtained are hard to generalise for other populations. Second, the cross-sectional design of the study makes it difficult to establish the causal relationship between variables. Third, the sample size was relatively small, and it was a single-centre study, which may affect the validation of the machine-learning algorithm model.

In conclusion, we confirmed that faciocervical measurements are associated with moderate to severe OSA in the Chinese population. The machine-learning model called the SABIHC2 model was set up based on faciocervical and anthropometric measurements. The model is more effective than the STOP-BANG questionnaire in predicting moderate to severe OSA, especially for those without significant daytime symptoms.

Data availability statement

Data are available in a public, open access repository. Extra data can be accessed via the Dryad data repository at http://datadryad.org/ with the doi: 10.5061/dryad.qnk98sfhg

Ethics statements

Patient consent for publication

Not required.

Ethics approval

The study was approved by the ethics committee of the Shanghai Jiao Tong University School of Medicine at Ruijin Hospital, with the following reference number: 2018-107.