Predicting synkinesis caused by Bell's palsy or Ramsay Hunt syndrome using machine learning‐based logistic regression

Abstract

Objective

To investigate whether machine learning (ML)‐based algorithms, namely logistic regression (LR), random forest (RF), k‐nearest neighbor (k‐NN), and gradient‐boosting decision tree (GBDT), utilizing early post‐onset parameters can predict facial synkinesis resulting from Bell's palsy or Ramsay Hunt syndrome more accurately than the conventional statistics‐based LR.

Methods

This retrospective study included 362 patients who presented to a facial palsy outpatient clinic. Median follow‐up of synkinesis‐positive and ‐negative patients was 388 (range, 177–1922) and 198 (range, 190–3021) days, respectively. Electrophysiological examinations were performed, and the rate of synkinesis in Bell's palsy and Ramsay Hunt syndrome was evaluated. Sensitivity and specificity were assessed using statistics‐based LR; and electroneurography (ENoG) value, the difference in the nerve excitability test (NET), and scores of the subjective Yanagihara scaling system were evaluated using early post‐onset parameters with ML‐based LR, RF, k‐NN, and GBDT.

Results

Synkinesis rate in Bell's palsy and Ramsay Hunt syndrome was 20.2% (53/262) and 40.0% (40/100), respectively. Sensitivity and specificity obtained with statistics‐based LR were 0.796 and 0.806, respectively, and the area under the receiver operating characteristic curve (AUC) was 0.87. AUCs measured using ML‐based LR of “ENoG,” “difference in NET,” “Yanagihara,” and all three components (“all”) were 0.910, 0.834, 0.711, and 0.901, respectively.

Conclusion

ML‐based LR model shows potential in predicting facial synkinesis probability resulting from Bell's palsy or Ramsay Hunt syndrome and has comparable reliability to the conventional statistics‐based LR.

Level of Evidence

3.

This study compared the accuracy of machine learning (ML) algorithms (logistic regression [LR], random forest, k‐nearest neighbor, and gradient‐boosting decision tree) to predict facial synkinesis resulting from Bell's palsy or Ramsay Hunt syndrome using early post‐onset parameters against conventional statistics‐based LR. Results showed that ML‐based LR has potential and comparable reliability to predict synkinesis probability. AUCs ranged from 0.711 to 0.910 for the different algorithms and parameters.

1. INTRODUCTION

Facial synkinesis is one of the most devastating sequelae of facial paralysis and is manifested by a variety of abnormal involuntary facial movements concomitant with voluntary movements of another facial muscle group, ¹ , ² such as closing the eyes while smiling or moving the mouth while blinking. It leads to uncomfortable facial movements ³ while eating and/or drinking and can significantly reduce the quality of life. ⁴ , ⁵ The mechanism of facial synkinesis is believed to involve aberrant axonal branching and misdirection during the process of nerve regeneration, leading to inappropriate neural connections and overactivation of facial muscles. However, facial synkinesis comprises a multifactorial pathophysiological principle, and its mechanism is still unclear. ¹

Predicting synkinesis early after the onset of the facial nerve paralysis can benefit patients with a wider range of treatment options. Indeed, early rehabilitation intervention in facial nerve patients has been reported to reduce complications and sequelae such as synkinesis and dry eye. ⁶ Moreover, synkinesis may arise from dysfunction in both the central and peripheral nervous systems, and the effectiveness of drug therapy such as botulinum toxin injections in addition to rehabilitation has also been demonstrated. ⁷

The electroneurography (ENoG) value of <40% and the ratio of the interpalpebral distance of <85% ⁸ , ⁹ , ¹⁰ have been reported to be correlated with the occurrence of synkinesis. Based on our previous studies, ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ we hypothesized that the characteristics of patients with early‐onset facial paralysis measured with values of “ENoG,” “the difference in the nerve excitability test (NET),” and “Yanagihara score” could be used to exquisitely predict facial synkinesis using machine learning (ML). Previous studies demonstrated the effectiveness of ML models in predicting clinical outcomes in numerous medical conditions, as well as in identifying prognostic factors. ¹⁸ , ¹⁹ , ²⁰ However, in many cases, it is still under debate which is the most reliable algorithm for prediction. ²¹ Classical statistical models, such as logistic regression (statistics‐based LR), are widely used to predict the risk of a disease. In contrast, some studies have reported the effectiveness of the gradient boosting decision tree (GBDT) model. ¹⁹ , ²² In otolaryngology, optimized algorithms, such as the support vector machine, ²³ , ²⁴ k‐nearest neighbor (k‐NN), ²⁵ , ²⁶ decision tree, ²⁶ and random forest (RF), reportedly predict postoperative performances. ²⁷ LR is a method that predicts outcomes by analyzing the probability of a specific event based on binary data. SVM maps data to a high‐dimensional space to identify a separating boundary for prediction. The predictions of k‐NN are based on the classes of the closest neighboring data points. Decision trees progress through questions based on the data features to perform classification and prediction, whereas RF combines multiple decision trees as an ensemble learning method and provides predictions from diverse perspectives. Gradient boosting decision trees improve models by minimizing errors and combining weak learners to create a powerful model (Table 1). Therefore, we aimed to optimize LR (ML‐based LR), RF, k‐NN, and GBDT models to predict facial synkinesis caused by Bell's palsy or Ramsay Hunt syndrome.

TABLE 1.

Characteristics of logistic regression and algorithms of machine learning.

Algorithm	Principle	Advantages	Disadvantages
Logistic regression (LR)	Linear regression with log‐odds transformation	Simple implementation	Limited modeling capabilities
Gradient boosting decision tree (GBDT)	Ensemble of decision trees trained sequentially	Powerful for nonlinear relationships	Prone to overfitting
Support vector machines (SVM)	Maximum margin separation in high‐dimensional space	Applicable to nonlinear classification	Requires extension for multi‐class classification
k‐nearest neighbors (k‐NN)	Classification based on nearest data points	Easy to implement	Increased computational complexity for large datasets
Decision tree	Hierarchical decision‐making based on data features	Intuitive interpretation and high readability	Prone to overfitting
Random forest	Ensemble of decision trees with random feature selection	High prediction accuracy and reduced risk of overfitting	Difficult model interpretation and understanding

2. MATERIALS AND METHODS

This retrospective study was conducted in compliance with the tenets of the Declaration of Helsinki. The study was approved by the regional ethical standards committee of the University of Tokyo Hospital (approval number: 2478). Informed consent was obtained from all subjects prior to participation in the study. We selected 362 patients diagnosed with Bell's palsy or Ramsay Hunt syndrome who had been referred to the Department of Otolaryngology at the University of Tokyo Hospital between April 2012 and June 2022. All patients received standardized treatments as outlined in previous reports. ²⁸ , ²⁹ In an effort to mitigate synkinesis, all patients with ENoG values below the threshold of 20% were diligently referred to the Department of Rehabilitation, which operates independently from the Department of Otolaryngology. The inclusion criterion was patients who have been followed up for more than 6 months after the onset of facial palsy. The follow‐up period was evaluated using the median for the following reasons: it is robust against outliers; it can effectively handle skewed distributions; and it can appropriately handle ordinal or nonnumerical data. Median follow‐up of synkinesis‐positive and ‐negative patients were 388 (range, 177–1922; average, 541) and 198 (range, 190–3021; average, 207) days. To estimate our findings, we included patients who underwent ENoG and NET between 8 and 28 days following onset of facial palsy.

For subjective examination, we adopted the Yanagihara grading system, which is widely used to assess facial movement and is influenced by the status of facial movement and synkinesis. ¹⁶ , ³⁰

Electrophysiological examinations, including ENoG, NET, and blink reflex (BR), were performed according to our previous report. ¹³ Briefly, electrophysiological recordings were performed using a Neuropack electromyography system (Nihon Kohden, Tokyo, Japan) in patients while they were awake. For ENoG, supramaximal stimulation was provided for 0.2 ms at a rate of 1 Hz through bipolar surface electrodes. To obtain maximal compound action potential amplitude, the anode was placed immediately outside the stylomastoid foramen, and the cathode in front of the earlobe. Surface disc electrodes were placed on the nasolabial fold with an interpolar distance of 20 mm. A ground electrode was placed on the lower jaw. The peak‐to‐peak ENoG response amplitudes on the affected side were compared to those on the unaffected side. The ENoG value was defined as the percentage of the response amplitude on the currently affected side to that on the opposite side. For NET, the examination on the marginal mandibular region were performed. A stimulating surface electrode with an anode was placed on the mandibular notch. After identifying contraction in the depressor labii inferioris muscle, stimulation intensity was reduced to an intensity at which muscle twitching disappeared. This threshold value (mA) for visible muscle contraction was defined as the NET threshold. The difference in NET was obtained by subtracting the NET threshold of the unaffected side from that of the affected side of the face. For BR, the responses of the orbicularis oculi and orbicularis oris were recorded under stimulation with 18 mA for 0.2 ms via the supraorbital margin. According to previous studies, ¹¹ , ¹⁶ , ³¹ , ³² facial synkinesis was defined as abnormal reflex waves in the orbicularis oris muscles with an approximately similar latency as the BR of orbicularis oculi muscles (synkinetic potential).

We extracted sex, age, diagnosis, duration of synkinesis following facial palsy, ENoG value, difference in NET, and Yanagihara score as multivariable components in a basic statistical analysis. We performed an LR statistical analysis using the statistical software “EZR” (https://www.jichi.ac.jp/saitama-sct/SaitamaHP.files/statmed.html), and the cut‐off was calculated using the Youden's index. ³³ The Youden's index, a single metric that balances sensitivity and specificity, was used to provide a more comprehensive evaluation when comparing the results of the logistic regression analysis with those obtained using machine learning. For different classifiers, we used the LR, RF, k‐NN, and GBDT to classify ENoG values for the categorization of facial synkinesis. The dataset from 362 patients was divided into a training set (66.6%) used to develop the classifier models and a validation set (33.3%) used to assess the prediction accuracy of each model. We optimized the hyperparameters using grid search optimization on the test data to maximize the detection efficacy (F‐score). To evaluate the reliability of algorithms, we depicted a receiver operating characteristic curve; the area under the curve (AUC), accuracy, precision, recall, and F‐score were calculated using Python 3.6.3 (Python Software Foundation) based on a machine learning framework, that is, the scikit‐learn toolbox (0. 24.4). We performed the Mann–Whitney U test to compare the two groups stratified according to synkinetic potential, and a p‐value <.05 was considered statistically significant.

3. RESULTS

We have identified 362 patients with Bell's palsy or Ramsay Hunt syndrome admitted during the study period. The odds ratio (OR) suggested that the prevalence of synkinesis was influenced by diagnosis, but not by sex or affected side (Table 2). Ninety‐three patients developed synkinesis, and facial movement was restored in 269 patients without synkinesis (Figure 1). ENoG, Difference in NET, and Yanagihara values were significantly different between the patients with and without synkinesis (Figure 1).

TABLE 2.

Odds ratio of diagnosis, sex, and affected side.

	Odds ratio (95% CI)	Risk ratio
Diagnosis (n)
Bell (262)	0.38 (0.23–3.30)	0.51
Hunt (100)	2.635 (1.59–4.34)	2.00
Sex (n)
Male (187)	1.42 (0.88–3.22)	1.30
Felmale (175)	0.71 (0.44–1.34)	0.77
Side (n)
Right (187)	1.00 (0.62–1.60)	1.00
Left (175)	1.00 (0.63–1.61)	1.00

FIGURE 1.

Basic statistical analysis of “ENoG,” “the difference in NET,” “Yanagihara,” and “age” in patients with and without synkinesis using the Mann–Whitney U test. *p < .05.

Subsequently, we investigated the relationships between individual components. “Difference in NET” and “age” were not correlated with “ENoG” in the synkinesis‐negative group, while there was a positive correlation between “Yanagihara” and “ENoG” (Figure 2). In contrast, “Difference in NET,” “Yanagihara,” and “age” were not correlated with “ENoG” in the synkinesis‐positive group.

FIGURE 2.

Scatter plots of the relationship between “ENoG” and “the difference in NET,” “Ynagihara,” and “age” in patients with and without synkinesis. Red lines on the graphs indicate simple linear regression. The intercept and regression coefficient are denoted in the top right section of diagrams.

We performed a statistics‐based LR analysis using the ENoG value and onset of facial synkinesis as the explanatory variable and objective variable, respectively (Figure 3). A cut‐off value of 14.785 (sensitivity, 0.80; specificity, 0.81) was determined for the Youden's index, with a corresponding AUC of 0.87 (95% confidence interval: 0.825–0.91).

FIGURE 3.

Receiver operating characteristic curve denoting the sensitivity and specificity of the Youden's index.

To evaluate ML algorithms, we plotted receiver operating characteristic curves. The AUC of LR (ML‐based LR), RF, k‐NN, and GBDT was 0.9095, 0.8041, 0.8011, and 0.8423, respectively (Figure 4A). Therefore, we adopted the ML‐based LR algorithm to predict facial synkinesis. We used the individual components “ENoG,” “Difference in NET,” and “Yanagihara” and all three components (“all”) as the explanatory variables (Figure 4B). The AUCs of “ENoG” (0.910) and “all” (0.901) were proximate values, whereas those of “Difference in NET” (0.834) and “Yanagihara” (0.711) were lower values. The accuracies of “ENoG” (0.844), “Difference in NET” (0.844), and “all” (0.853) were higher than that of “Yanagihara” (0.688). Precision and recall display trade‐off relationships. The precision of “ENoG,” “Difference in NET,” “Yanagihara,” and “All” was 0.625, 0.800, 0.358, and 0.733, respectively. The F‐score is an evaluation index for binary classification tasks that represents the harmonic mean value between precision and recall. The ML model becomes the most efficient and balanced when its associated F‐score becomes the closest to 1.0. “ENoG” provided the highest F‐score of 0.638 (Table 3).

FIGURE 4.

Predicting facial synkinesis using a machine learning (ML)‐based model fitted with parameters extracted from clinical data. FPR, false positive ratio; and TPR, true positive ratio. A, Receiver operating characteristic (ROC) curves of the logistic regression, random forest, k‐nearest neighbors, and gradient boost decision tree, along with the corresponding areas under the curve. B, ROC curves based on ML with logistic regression.

TABLE 3.

Area under the receiver operating characteristic curve (AUC), accuracy, precision, recall, and F‐score based on machine learning‐based logistic regression algorithm.

	AUC	Accuracy	Precision	Recall	F score
ENoG	0.910	0.844	0.625	0.652	0.638
Difference in NET	0.834	0.844	0.800	0.347	0.485
Yanagihara	0.711	0.688	0.358	0.609	0.452
All	0.901	0.853	0.733	0.478	0.579

4. DISCUSSION

We have investigated the reliability of ML‐based algorithms, including LR, RF, k‐NN, and GBDT in predicting synkinesis caused by Bell's palsy or Ramsay Hunt syndrome. ML‐based LR was likely to be the best algorithm, and the reliability of “ENoG” and “All” were almost identical. ML‐based LR model was more reliable than that without ML (statistics‐based LR). Indeed, that the AUC of the ML‐based LR model (0.9095, Figure 4) was higher than that of the model without ML (0.87, Figure 3).

To our knowledge, there are no reports on ML‐based predictions for synkinesis, and our study it the first one to demonstrate that ML‐based LR is the best algorithm for predicting synkinesis. Technological advancements in recent years have led to GBDT's predictive accuracy surpassing that of LR. ¹⁹ , ³⁴ , ³⁵ , ³⁶ Unexpectedly, our findings based on the present dataset suggest that the predictive accuracy of the ML‐based LR algorithm exceeds that of GBDT (Figure 4B). The underlying cause appears to be the sample size; however, it is plausible that further optimization of hyperparameters could lead to an enhancement of GBDT in predictive performance. ¹⁹ Therefore, traditional methods, rather than novel methods, such as GBDT, may be more suitable as prediction algorithms for clinical data in small‐scale studies.

The prediction performances of “ENoG” and “all” appeared to be equivalent (Figure 4A). The AUC (“ENoG,” 0.910; “all,” 0.901) and accuracy (“ENoG,” 0.844; “all,” 0.853) were proximate values, suggesting that “ENoG” has a greater impact as a predictive factor compared with “NET” and “Yanagihara.” Specifically, “Difference in NET” and “Yanagihara” displayed bimodal and random distribution, respectively (Figures 1 and 2). We considered the “ENoG” value as a reliable component and that “ENoG” conducted between 8 and 28 days from onset of facial palsy can predict synkinesis.

Consequently, we assessed “ENoG,” “NET,” and “Yanagihara” as explanatory variables to construct a synkinesis prediction model. The LR model was effective in predicting pathological co‐movements following facial palsy. The AUCs of the model equipped with “ENoG,” “NET,” and “Yanagihara” scores as the explanatory variables and the AUC of the model with “ENoG” alone were almost similar, indicating that the initial value of “ENoG” alone may also partially predict pathological co‐movement.

We observed the bipolar distribution of “the difference in NET” in patients with synkinesis, which indicated that “the difference in NET” appeared to be an inadequate predictor of synkinesis. However, compared with the distribution of “ENoG” and “Yanagihara,” that of “the difference in NET” displayed lower variation in patients without synkinesis, suggesting that a lower “difference in NET” may be a highly specific indication of the absence of synkinesis. The OR and risk ratios revealed a correlation between synkinesis and Ramsay Hunt syndrome (Table 2). We suggest that rehabilitation should be used to prevent synkinesis when a patient is diagnosed with Ramsay Hunt syndrome.

This study has limitations. First, owing to the retrospective nature of the study, we failed to eliminate the possibility of bias. Second, the sample size was small, and the ML algorithm required optimization. For sample size analysis, Seto et al. suggested that the reliability of a light gradient boosting machine was higher than that of LR following an increase in the sample size by an order of >10. ⁴ , ¹⁹ Third, increasing the number of explanatory variables may have improved the prediction reliability of the models included. Furthermore, we were unable to determine whether “ENoG,” “NET,” and “Yanagihara” were the optimal explanatory variables. Fourth, our results were based on the theory that synkinesis is correlated with the degree of nerve injury. Researchers should elucidate these mechanisms to prevent synkinesis.

The predictive accuracy of machine learning models is influenced by the sample size. However, it is possible to improve predictive accuracy by enhancing data quality, selecting relevant features, and optimizing model parameter settings in cases where the population is small. Further studies should aim to overcome these current challenges.

5. CONCLUSION

The reliability of the ML‐based LR model using “ENoG” for predicting synkinesis owing to facial palsy was equivalent to that of the classifier model comprising “ENoG,” “Difference in NET,” and “Yanagihara.” This ML‐based model enabled the prediction of facial synkinesis caused by Bell's palsy or Ramsay Hunt syndrome.

AUTHOR CONTRIBUTIONS

Conceptualization: Megumi Kishimoto‐Urata, Shinji Urata, and Kenji Kondo. Initial draft writing: Megumi Kishimoto‐Urata, and Shinji Urata. Review and editing: Shinji Urata, Hironobu Nishijima, Shintaro Baba, Yoko Fujimaki, and Kenji Kondo. Supervision: Tatsuya Yamasoba. Project administration: Megumi Kishimoto‐Urata, Shinji Urata, Kenji Kondo, and Tatsuya Yamasoba.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

Kishimoto‐Urata M, Urata S, Nishijima H, et al. Predicting synkinesis caused by Bell's palsy or Ramsay Hunt syndrome using machine learning‐based logistic regression. Laryngoscope Investigative Otolaryngology. 2023;8(5):1189‐1195. doi: 10.1002/lio2.1145

DATA AVAILABILITY STATEMENT

Data sharing is not applicable to this article as no new data were created or analyzed in this study.