Objective Pharyngeal Phenotyping in Obstructive Sleep Apnea With High-Resolution Manometry

Abstract

Objective.

Drug-induced sleep endoscopy (DISE) is a commonly used diagnostic tool for surgical procedural selection in obstructive sleep apnea (OSA), but it is expensive, subjective, and requires sedation. Here we present an initial investigation of high-resolution pharyngeal manometry (HRM) for upper airway phenotyping in OSA, developing a software system that reliably predicts pharyngeal sites of collapse based solely on manometric recordings.

Study Design.

Prospective cross-sectional study.

Setting.

An academic sleep medicine and surgery practice.

Methods.

Forty participants underwent simultaneous HRM and DISE. A machine learning algorithm was constructed to estimate pharyngeal level-specific severity of collapse, as determined by an expert DISE reviewer. The primary outcome metrics for each level were model accuracy and F1-score, which balances model precision against recall.

Results.

During model training, the average F1-score across all categories was 0.86, with an average weighted accuracy of 0.91. Using a holdout test set of 9 participants, a K-nearest neighbor model trained on 31 participants attained an average F1-score of 0.96 and an average accuracy of 0.97. The F1-score for prediction of complete concentric palatal collapse was 0.86.

Conclusion.

Our findings suggest that HRM may enable objective and dynamic mapping of the pharynx, opening new pathways toward reliable and reproducible assessment of this complex anatomy in sleep.

Obstructive sleep apnea (OSA) is a common sleep disorder characterized by repetitive upper airway collapse that has been independently associated with hypertension, cardiovascular disease risk, and decreased work productivity.^1–5 Positive airway pressure (PAP) is an efficacious first-line treatment for OSA, but effectiveness is compromised by noncompliance, with only 39% to 50% of patients maintaining adequate usage.^6–8

Surgical procedural selection for CPAP-intolerant patients with OSA can be a challenging task for even experienced surgeons. Drug-induced sleep endoscopy (DISE) has recently found favor in the United States for the surgical assessment of dynamically collapsing upper airway anatomy.⁹ The Food and Drug Administration requires DISE examination to determine eligibility for hypoglossal nerve stimulation therapy, as patients with complete circumferential collapse of the soft palate (CCC) are considered less likely to respond to the treatment.¹⁰ Despite its growing popularity, DISE has limitations. Evaluation techniques have not been standardized, with multiple anesthetic agents and pharyngeal classification systems having been described.^9,11–16 DISE is expensive, requires sedation, only approximates natural sleep, and inter- and intra-rater assessments of collapse sites and patterns are variable.^17–22 Other dynamic imaging modalities have been investigated but have not found widespread favor due to various intrinsic limitations such as cost, lack of dynamic assessment, and difficulty approximating natural sleep conditions.

Manometry assesses pressure changes within the lumen of a hollow organ. Over the past decade high-resolution trans-nasal esophageal manometry, with multiple solid-state pressure sensors at closely spaced intervals, has become the gold standard for assessment of esophageal motility disorders.²³ Conventional pharyngeal manometry for OSA was investigated as early as the 1970s.²⁴ In 1992, Woodson and Wooten reported that it enabled comparison of pressure changes at multiple pharyngeal sites simultaneously with minimal disruption of sleep.^25,26 Nevertheless, this technique for dynamic pharyngeal assessment in sleep has received little further attention in the literature.

Here we present an initial investigation of high-resolution pharyngeal manometry (HRM) in OSA. We hypothesized that a machine learning algorithm could use HRM measurements collected during DISE to reliably predict a surgeon’s DISE findings without any input of endoscopy data. Our findings suggest that HRM and machine learning algorithms may enable objective and dynamic mapping of the pharynx without a high labor cost for physician interpretation of HRM data, opening new pathways towards reliable and reproducible assessment of this complex anatomy in sleep.

Methods

Study Design

This study was approved by the Vanderbilt University Medical Center Institutional Review Board (IRB#: 170755; ClinicalTrials.gov ID: NCT03198416). Participants were recruited as a convenience sample from a group of patients with OSA scheduled to undergo DISE as part of routine clinical management. Demographic data were collected for each participant.

Experimental Procedures

Manometry Catheter Placement

A custom-designed pharyngeal HRM catheter was placed transnasally prior to sedation and connected to a recording system (Solar GI HRM, Laborie/Medical Measurement Systems BV). The 6-French catheter was comprised of 21 solid-state sensors spaced at 8mm intervals, except for a 6 cm gap between the first and second most distal sensors to aid anchoring it in the esophagus (Supplemental Figure S1, available online). Participants reported a catheter discomfort score on a 10-point visual analog scale (0 = no pain, 10 = intolerable pain) after placement.

DISE

Participants were sedated using standard methods.²⁷ Intravenous propofol was manually titrated throughout to maintain a bispectral index score between 50 and 70.²⁸ Simultaneous HRM and DISE recordings were collected for approximately 10 minutes.

Experimental Protocol

Data Preprocessing

A custom software interface was created for visualizing and scoring HRM data as heatmaps that were synchronized with endoscopy video (Figures 1 and 2; Supplemental Video S1, available online). Within each heatmap, time was represented on the x-axis, with the anatomic position of each sensor represented on the y-axis. Instantaneous pressure values were coded by color, with atmospheric pressure at the warm (red) end of the spectrum transitioning to cooler (blue) colors at the most negative recorded values. Positive pressure values of the UES were represented by violet colors.

Figure 1.

An example image from the combined high-resolution manometry and endoscopy data. X-axis: time; y-axis: sensor position; color: pressure level (scale, left). Pressures ranged from positive (violet) to atmospheric (red) to ≤−40 mm Hg (blue).

Figure 2.

Single-breath high-resolution manometry heatmaps illustrating the four sites of collapse using the VOTE classification.¹⁴ See Figure 1 legend for heatmap construction description. E, epiglottis; O, oropharynx; T, tongue base; V, velum.

A single expert scorer (D.K.) observed all combined data recordings (endoscopy with manometry heatmaps). Pharyngeal collapse patterns for each flow-limited breath and the overall exam were documented with the VOTE (velum, oropharynx, tongue base, epiglottis) classification system.¹⁴

Data Analysis

Our team developed a machine learning model that estimated the degree of collapse at each pharyngeal level with the VOTE classification using 92 features based on descriptions of frequent visual HRM patterns observed by the expert scorer across multiple participants (Supplemental Table S1, available online), including largest negative connected component (LNCC) features. LNCCs represented the dominant shape of the negative pressure envelope within each inspiration across multiple sensors (Figure 3).

Figure 3.

An example of a high-resolution manometry heatmap (left panel) conversion to the largest negative connected component (LNCC; right panel). A LNCC (bright area) represents the dominant shape of the negative pressure envelope within an inspiration. LNCC, largest negative connected component.

Multiple supervised machine learning models were deployed to predict each participant’s level-specific VOTE score, including K-nearest neighbor (KNN), support vector machine, logistic regression, random forest, and Naive Bayes. Each level-specific classification was treated as an independent binary prediction.

Evaluation proceeded in two phases. First, 31 patients were used to train and test classifiers using 10-fold cross validation (90% train, 10% test). Second, using the features selected during the first phase, a single model was trained with 31 patients and tested on a holdout of 9 patients. For each phase, the accuracy and F1-score were calculated for each level-specific classification. Accuracy represents a model’s ability to correctly classify a label but, if the ratio of classes is highly skewed, accuracy results may be misleading. For example, if one label occurs in the data set 90% of the time, then a very simple algorithm could always pick the majority class, thus easily attaining 90% accuracy, even if the minority class is always incorrectly predicted. In contrast, the F1-score was included as an additional outcome metric, which is calculated as the harmonic mean of precision (the ability to correctly identify true positive cases of a collapse pattern) and recall (the ability to find all true positive cases of a collapse pattern in a data set), giving equal weight to both recall and precision. Intuitively, precision and recall are competing metrics: attaining perfect recall can reduce precision and vice versa. In this study, F1-scores were averaged across all level-specific classifications with an unweighted average and weighted average by level classification frequency due to class imbalances among the scored patterns.

Student’s t-test and Fisher’s Exact tests were used for evaluation of differences in demographic and polysomnographic variables using a p value of less than .05 for significance. Additional methodological details are provided in a separate supplement.

Results

Sixty participants were consented for the study. Three participants withdrew consent before initiation of study procedures, and two participants declined to complete HRM catheter placement due to procedural discomfort. All five participants were excluded from further analysis. Fifty-five participants completed all study procedures, reporting a mean catheter discomfort score of 1.5 ± 1.3 (mean ± SD) on a 10-point visual analog scale. Fifteen participants were excluded from scoring procedures due to incomplete endoscopy video capture in the earliest recruited participants. The HRM manufacturer’s system was not designed for extended video capture and it discontinued recording of synchronized endoscopy video after approximately 1 minute despite continuing to display it. The loss of video data was not identified until an interim data review and was corrected in subsequent procedures by recording high-resolution color endoscopy from a separate video tower and then synchronizing it to the initial clip of HRM endoscopy video, leaving 40 participants for analysis (Table 1). Included participants were primarily elderly white males with severe OSA. There were no significant demographic differences between included and excluded participants (p > .05). The distribution of overall VOTE collapse patterns for included participant is depicted in Figure 4.

Table 1.

Mean Demographic Variable Values for Participants Included in Final Analyses

Variable	Mean (SD)
Age, y	60.3 (10.3)
Gender (M:F)	31:9
White, %	100
BMI, kg/m2	31.2 (10.7)
AHI, events/h	38.9 (23.7)
O2 nadir, %	79.8 (7.7)

Standard deviations are given in parentheses. There were no significant differences between included and excluded participants.

Abbreviations: AHI, apnea-hypopnea index; BMI, body mass index.

Figure 4.

The distribution of airway collapse patterns observed across the 40 included participants using the VOTE classification system.¹⁴

Machine Learning/Data Mining Results

Individual breaths were found to have low predictive power for estimating either breath- or patient-level VOTE scores (data not shown). When estimating patient-level VOTE scores, the best results were achieved by a KNN model. For the training phase, the model predicted the degree of collapse for a given pharyngeal structure using only HRM data, with receiver operating characteristic area-under-the-curve values ranging from 0.89 to 0.98 and F1-scores ranging from 0.71 to 0.98 (Table 2; Supplemental Figure S2, available online). The F1-score for prediction of CCC was 0.89. The average weighted and unweighted F1-score across all categories was 0.86, with an average weighted and unweighted accuracy of 0.91 and 0.90, respectively. On the holdout set, the machine learning model attained an average F1-score of 0.96 and an average accuracy of 0.97 (see Supplemental Tables S2–S11, available online for task-specific confusion matrices). Weighting (ie, varying the importance of a class by its frequency when calculating a statistic) did not affect the results. The F1-score for prediction of CCC was 0.86.

Table 2.

Area Under the Curve and F1-Scores From the Best Model Predicting VOTE Scores Based Only on High-Resolution Manometry Data

VOTE Score	Area under the curve (training)	F1-score (training)	F1-score (holdout)
V = 1	0.98	0.96	0.99
V = 2	0.95	0.95	1.00
O = 1	0.99	0.98	1.00
O = 2	0.87	0.71	0.89
T = 1	0.89	0.81	0.92
T = 2	0.86	0.81	0.93
E = 1	0.98	0.86	-
E = 2	0.89	0.78	1.00

E = 1 did not occur within the holdout set (n = 9).

Abbreviations: E, epiglottis; O, oropharynx; T, tongue base; V, velum.

A variety of machine learning features exhibited predictive value depending on the level and degree of collapse (Figures 5–8). In general, measurements based on frequency data from a few sensors provided the greatest predictive value for the different classifiers, although the relevant sensor or sensors varied depending on the classifier of interest (see Supplemental Tables S12–S20, available online). At least one LNCC feature was in the top 5 most important features for each classifier except for partial tongue base collapse.

Figure 5.

An example of high-resolution manometry data from partial anterior-posterior velopharyngeal collapse. Sensors (white lines) record pressure (mm Hg; color scale) at 100 Hz (time; x-axis) at 8 mm intervals within the pharynx (y-axis).

Figure 8.

An example of high-resolution manometry data from complete epiglottic collapse. Sensors (white lines) record pressure (mm Hg; color scale) at 100 Hz (time; x-axis) at 8 mm intervals within the pharynx (y-axis).

Discussion

This study demonstrated that automated HRM processing by a machine learning algorithm can accurately summarize the sites and degree pharyngeal collapse using the VOTE classification, a system widely understood by sleep surgeons. The algorithm achieved excellent classification ability during training with area-under-the-curve values ranging from 0.89 to 0.98. When tested against a previously unseen holdout set, the HRM algorithm achieved F1-scores for pharyngeal structure and degree of collapse ranging from 0.89 to 1.0, indicating excellent to near-perfect agreement with a human DISE scorer. These findings suggest that HRM may be a useful tool for assessing the complex and dynamic collapse patterns of the pharynx in sedated or natural sleep, potentially opening new pathways toward objective assessment of the upper airway before and after sleep surgery interventions.

Individual sensor frequency information was important to several of the KNN classifiers, suggesting that this type of data has high predictive value for ascertaining the structure and degree of collapse within the pharyngeal column. Nevertheless, relevant sensors ranged across the entire pharyngeal length, implying that the high sensor density was important for classifier accuracy. Notably, important sensors were not always co-located to the site of collapse. Further work is necessary to ascertain why sensor data from disparate airway levels yielded important classifier information, but the unique pressure signatures observed across different collapse patterns suggest that there is a complex interplay between dynamically collapsing pharyngeal structures.

There are several key differences between this study and prior evaluations of upper airway manometry,^{24–26,29–31} primarily in its ability to map complex pressure patterns to the VOTE classification, a system readily understood by sleep surgeons. From a technical perspective, our HRM system utilized closely spaced sensors that enabled high-resolution mapping of the entire pharynx, even if the relative catheter position shifted. This close sensor spacing enabled our HRM system to achieve excellent discrimination and agreement with the DISE reviewer even during classification of oro- and hypopharyngeal collapse patterns. In contrast, prior manometry studies did not differentiate between the lateral pharyngeal walls, tongue base, and epiglottis during inspiratory collapse, although these structures can substantially affect pharyngeal and hypoglossal nerve stimulator surgical outcomes.^32,33 Lastly, our system was constructed from solid-state sensors on a 6-French catheter, which is smaller than manometry catheters used previously in natural sleep for surgical planning.^25,26 We suspect HRM studies during natural sleep may be possible, as other natural sleep studies did not demonstrate significant changes in upper airway collapsibility or polysomnographic variables with manometry catheters of similar diameter.^34,35 A prior report of esophageal manometry during polysomnography noted that placement of a 6-French catheter was tolerated by all patients in which it was attempted, with no clinically significant effects on sleep architecture.³⁶

Multiple machine learning models were evaluated for accuracy and F1-scores including KNN, support vector machines, logistic regression, random forest, and Naive Bayes. Empirical training and validation results demonstrated the KNN classifier performed best in our experimental setup, likely due to the limited number of subjects in this study. The optimal decision boundaries for HRM predictions are likely not linear, and variations among patients even within a single class can make predictions within small data sets difficult. More data-intensive models struggle to construct accurate decision boundaries with smaller data sets, but the KNN classifier likely performed well as it functions by clustering similar patients. As more data are collected, we expect more advanced and data-intensive classifiers to perform better.

It would be optimal for our HRM algorithm to report the distribution of collapse patterns observed in a single patient. We attempted this task during exploratory analyses by VOTE scoring each inspiration but discovered that single breath predictions were inaccurate, possibly due to variability in the site and magnitude of flow limitation between endoscopically similar breaths (data not shown; see Supplemental Video S1, available online). Further investigation revealed that while all scored breaths displayed visual evidence of reduced airway caliber, only 10% of them generated significant negative pressure gradients indicative of flow-limited inspiration.

A discrepancy between endoscopic and manometric data suggests that either the manometry catheter was insensitive to negative pressure changes or that the human reviewer was overly sensitive to airway collapse, scoring many breaths with positive collapse findings despite the absence of flow-limited inspiration. Per the manufacturer’s documentation (Unisensor AG), the catheter’s sensors are accurate to within ±2% of the true value from −50 to 300mm Hg, confirming that the hardware was sufficiently sensitive and accurate for this application. Accurate manometry hardware implies that our expert scorer was overly sensitive to airway collapse, although he has participated in multiple large reviews of DISE where consensus agreement among expert scorers was on par with agreement rates reported elsewhere in the literature.^32,33

Dynamic changes in pharyngeal cross-sectional area may not always be indicative of the site and degree of inspiratory flow limitation. Isono et al previously reported that inspiratory airflow is independently related to cross-sectional area as well as driving pressure across the velopharynx concluding that, at a given cross-sectional area, airflow limitation may be present at one inspiratory driving pressure and absent at another. Prior work also supports the theory that the structure observed to collapse may not represent the primary site of flow limitation,³⁷ and may not be visualized during DISE if the endoscope is distal to the site of flow limitation. HRM, however, generates a global view of the upper airway, and generation of negative pressure gradients only occurs when site-specific flow limitation exists. Further work is needed to determine whether disagreements between HRM and DISE findings affect global assessments of upper airway collapse patterns and surgical decision-making.

Aspects of our data suggest that HRM analysis may have several advantages over DISE. Foremost are the objective nature of the collected data and the aforementioned global evaluation of the pharynx. The subjective nature of DISE introduces variability in inter- and intra-rater agreement when visually grading the level of anatomic collapse.^38–41 Agreement between scorers declines further when assessing finer degrees of detail and there is a known learning curve for accurate interpretation.^38,40,41 Simultaneous data capture from throughout the pharynx creates the potential for interesting new analyses. For instance, some recorded breaths suggest that proximal sites of flow obstruction and decreasing pressure early in the inspiratory phase may trigger collapse of susceptible downstream structures (see Supplemental Figure S3, available online) raising questions regarding the optimal surgical intervention. Substantial further work is required to assess the utility of pre- and post-management HRM and whether a scoring system’s accuracy remains high across multiple expert scorers.

Several limitations of this work should be acknowledged. First, HRM data were compared to DISE, a subjective exam with known inter- and intra-rater variability.^{18–20,40,41} Despite its limitations, DISE has nevertheless become the de facto clinical standard for dynamic anatomic assessment of the upper airway with large case series evaluating findings and associations with surgical outcomes,^32,33,42,43 and was intentionally selected for comparison as other dynamic assessment studies including natural sleep endoscopy, cine-MRI, and 4D-CT remain largely confined to the research realm. Second, this study only explored the agreement between DISE and HRM on a per-structure basis as the task of scoring and evaluating VOTE score agreement across all examined structures is, mathematically, orders of magnitude more difficult and requires a much larger sample size. Nevertheless, a per-structure analysis approach is consistent with the existing literature evaluating inter-rater agreement in DISE.^{32,33,38,40,41} Furthermore, our analysis for presence of CCC suggests that HRM systems have the potential to identify meaningful patterns for the most detailed levels of analysis where DISE inter-rater agreement breaks down substantially,³⁸ although further evaluation of more subtle patterns of velopharyngeal and epiglottic collapse is required. Third, agreement between HRM and DISE was assessed for only one expert reviewer in this preliminary study. Further studies are required to assess inter- and intrarater agreement between a machine learning algorithm and multiple expert DISE reviewers, as well as examination reproducibility. Fourth, substantial class imbalances existed in our data, with a high proportion of patients exhibiting palatal and tongue base collapse. Nevertheless, the distribution of collapse patterns is similar to other large case series in the literature,⁴² and our system performed well even against structures that collapsed less frequently, such as the epiglottis and oropharyngeal lateral walls. Lastly, further studies are needed to better elucidate the discrepancies observed between the majority of HRM and DISE breaths, where observed dynamic airway collapse did not translate to negative pharyngeal pressure gradients. Regardless, exclusion of this significant percentage of breaths did not appear to substantially impact the algorithm’s patient-level agreement with the expert scorer for the degree of structural collapse.

Conclusion

This initial exploratory pilot study suggests that HRM findings associate well with DISE collapse patterns of the upper airway that are familiar to surgeons. Further DISE and natural sleep HRM studies are required in more diverse populations to validate catheter tolerability, pharyngeal collapse pattern accuracy, and value in surgical decision-making. It may ultimately have utility as a diagnostic tool for objective, phenotyping of complex, dynamic collapse patterns of the upper airway during DISE and possibly even natural sleep.

Figure 6.

An example of high-resolution manometry data from partial oropharyngeal collapse. Sensors (white lines) record pressure (mm Hg; color scale) at 100 Hz (time; x-axis) at 8 mm intervals within the pharynx (y-axis).

Figure 7.

An example of high-resolution manometry data from complete tongue base collapse. Sensors (white lines) record pressure (mm Hg; color scale) at 100 Hz (time; x-axis) at 8 mm intervals within the pharynx (y-axis).