Development of a Predictive Model for Persistent Dizziness Following Vestibular Schwannoma Surgery

Abstract

Objective:

In an era of vestibular schwannoma (VS) surgery where functional preservation is increasingly emphasized, persistent postoperative dizziness is a relatively understudied functional outcome. The primary objective was to develop a predictive model to identify patients at risk for developing persistent postoperative dizziness after VS resection.

Methods:

Retrospective review of patients who underwent VS surgery at our institution with a minimum of 12 months of postoperative follow-up. Demographic, tumor-specific, preoperative, and immediate postoperative features were collected as predictors. The primary outcome was self-reported dizziness at 3-, 6-, and 12-month follow-up. Binary and multiclass machine learning classification models were developed using these features.

Results:

1,137 cases were used for modeling. The median age was 67 years, and 54% were female. Median tumor size was 2 cm, and the most common approach was suboccipital (85%). Overall, 63% of patients did not report postoperative dizziness at any timepoint; 11% at 3-month follow-up; 9% at 6-months; and 17% at 12-months. Both binary and multiclass models achieved high performance with AUCs of 0.89 and 0.86 respectively. Features important to model predictions were preoperative headache, need for physical therapy on discharge, vitamin D deficiency, and systemic comorbidities.

Conclusion:

We demonstrate the feasibility of a machine learning approach to predict persistent dizziness following vestibular schwannoma surgery with high accuracy. These models could be used to provide quantitative estimates of risk, helping counsel patients on what to expect after surgery and manage patients proactively in the postoperative setting.

Level of Evidence:

Level IV

Lay Summary:

Surgery for vestibular schwannoma can result in long-term dizziness. Here, we developed models to predict individual patients’ risk for developing long-term dizziness. These models could be used to help counsel patients and to manage patients proactively.

Introduction

Active management of vestibular schwannoma (VS) has evolved towards conservative strategies, with many tumors now being observed with serial imaging studies in lieu of surgery or stereotactic radiation therapy. When surgery is pursued, there is a trend towards emphasizing preservation of function over gross total resection.¹

Persistent postoperative vestibular dysfunction is a potentially disabling functional outcome of VS surgery,^2–7 however this is less well studied than other functional outcomes such as facial nerve function and hearing outcomes. Vestibular dysfunction is inevitable following VS surgery given ablation of the ipsilateral vestibular apparatus, and while this is generally expected to resolve with contralateral compensation, this is not always the case for some patients.

In this study, we sought to assess postoperative vestibular symptoms following VS surgery at long-term follow-up, and develop a predictive model to identify patients at risk of developing persistent postoperative vestibular symptoms. Improved prediction of functional outcomes following surgery could help counsel patients on what to expect after surgery and manage patients proactively in the postoperative setting.

Methods

This study was approved by the Mass General Brigham Institutional Review Board (#2021P001712). The Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines were adhered to.

Data Source and Features

A retrospective chart review was performed including patients who had undergone surgery for VS at our institution from 1994 to 2017. Only patients with at least 12 months of postoperative follow-up were included. Patients with NF2 were excluded from this study. Given the retrospective nature of this study, no formal sample size calculation was performed a priori, and all available data was collected and utilized to maximize model performance and generalizability. The commonly accepted minimum of 10 events per variable for model development studies is far exceeded in our study given the very large study cohort.⁸

Several predictive features were collected for each patient. Demographic features included age at time of surgery and gender. Tumor-specific features included tumor size and surgical approach. Tumor size was measured as the largest extrameatal diameter.⁹ Preoperative features included years of follow-up (from diagnosis to surgery), preoperative headache, preoperative dizziness, hydrocephalus, and preoperative medications used to treat vestibular problems. Postoperative features included CSF leak, House-Brackmann score, completeness of resection, infection, need for physical therapy on discharge, and postoperative medications used to treat vestibular problems. Vestibular diagnoses were recorded including BPPV, Meniere’s Disease, and vestibular neuritis. Systemic comorbidities were recorded including hypertension, diabetes, obesity, migraine, neuropathy, vitamin D deficiency, cancer history, thyroid disorders, neurological disorders, cardiac disease, psychiatric disease, ophthalmic disease. A summative variable indicating the presence of any systemic comorbidity was also included.

Outcome Variable

Subjective patient report of dizziness at the 3-, 6-, and 12-month postoperative visits was recorded as a binary “yes or no” variable. While all patients were asked about vestibular symptoms at postoperative visits, this was not systematically recorded; rather this was inferred from whether it was reported in the clinic notes.

Data Preprocessing

The dataset was split to create training (85% of the data) and testing sets (15%) for modeling. Several preprocessing steps were taken to optimize model performance. Numerical data were normalized and transformed and highly correlated features were removed. Lastly, cases with any missing data were excluded.

Machine Learning Model Development and Interpretation

Two modeling approaches were designed according to the prediction task. The first was a binary classification outcome as either i) no reported postoperative vestibular symptoms versus ii) any reported postoperative vestibular symptoms at the 3-, 6-, or 12-month timepoints. In the second approach, a four-way multiclass classifier was developed to represent the outcome variable as either i) no postoperative vestibular symptoms, ii) vestibular symptoms at 3 months, iii) at 6 months, or iv) at 12 months.

For each approach, 16 candidate classification models were trained and evaluated including: random forest, light gradient boosting machine, ADA boost, extra trees, CatBoost, gradient boosting, logistic regression, extreme gradient boosting, ridge, linear discriminant analysis, SVM with linear kernel, decision tree, K-nearest neighbors, naïve Bayes, quadratic discriminant analysis, and a naïve dummy model. The models were trained using 5-fold cross-validation, and parameters were tuned to maximize area under the curve (AUC). Final models were developed on the entire training set using the optimal set of parameters, and then used to generate predictions for samples on the separate holdout testing set.

Model performance was assessed by the AUC. Model interpretability was assessed using feature importance analysis depending on the model type. Data processing and modeling were conducted in a python environment using Jupyter Notebooks (Project Jupyter) and associated machine learning libraries (pycaret).

Results

Study Cohort Characteristics

Initial chart review produced 1,480 patients meeting inclusion criteria. After excluding cases with missing data, 1,137 cases remained for modeling. The median age was 67 years, and 54% were female. The median tumor size was 2 cm. The most common surgical approach was suboccipital (85%), followed by translabyrinthine (10%) and middle fossa (5%). The remainder of predictive features are summarized in Table 1. Regarding the outcome variable, 717 (63%) patients had no postoperative vestibular symptoms at any timepoint. 126 (11%) reported vestibular symptoms at the 3-month timepoint, 100 (9%) at the 6-month timepoint, and 194 (17%) at the 12-month timepoint.

Table 1.

Study Cohort Characteristics

Demographic features
Age, median (IQR)	49 (18)
Sex
Female	608 (54%)
Male	529 (46%)
Tumor-specific features
Tumor size, median (IQR)	2 (1.6)
Surgical approach
Suboccipital	959 (84%)
Translabyrinthine	110 (10%)
Middle fossa	68 (6%)
Preoperative features
Years of follow-up, median (IQR)	7 (12)
Preoperative headache	372 (33%)
Preoperative dizziness	465 (41%)
Hydrocephalus	74 (7%)
Pre-op vestibular medications	16 (1.4%)
Postoperative features
CSF leak	112 (10%)
HB grade
I	913 (80%)
II	64 (6%)
III	57 (5%)
IV	57 (5%)
V	41 (4%)
VI	5 (0%)
Resection
Gross total	941 (83%)
Subtotal	172 (15%)
Near total	24 (2%)
Infection	39 (3%)
Physical therapy on discharge	109 (9.6%)
Post-op vestibular medications	195 (17.2%)
Vestibular comorbidity	7 (1%)
Systemic comorbidities
Hypertension	185 (16%)
Diabetes	55 (5%)
Ophthalmic disease	104 (9.1%)
Cardiac disease	83 (7%)
Cancer history	116 (10%)
Thyroid disorder	70 (6.2%)
Psychiatric disease	109 (9.6%)
Neurologic disorder	122 (11%)
Neuropathy	53 (5%)
Obesity	40 (4%)
Vitamin D deficiency	92 (8%)
Migraine	99 (9%)
Any systemic comorbidity	474 (42%)
Postoperative dizziness outcome
None	717 (63%)
3-month	126 (11%)
6-month	100 (9%)
12-month	194 (17%)

Binary Classifier Development.

After development of 16 candidate models, the best performing models were logistic regression (AUC 0.90), linear discriminant analysis (AUC 0.90), and gradient boosting (AUC 0.89). Given that all 3 models had similar performance, the simplest model, logistic regression, was chosen for further analysis. When tested on the holdout set, the logistic regression model produced an AUC of 0.89. Feature importance analysis showed that preoperative headache, need for physical therapy on discharge, and vitamin D deficiency were the top three most important features in the model’s predictions. The top ten most important features are depicted in Figure 1, with bars indicating their relative importance. Further inspection of the model revealed odds ratios of 2.19 (p < 0.01) for preoperative headache, 2.11 (p < 0.01) for physical therapy, and 1.88 (p < 0.01) for vitamin D deficiency. Odds ratios from the logistic regression model for the top ten most important features are provided in Table 2.

Figure 1.

This figure shows the feature importance scores for the top ten most important features to the binary classification logistic regression model. For logistic regression, feature importance can be measured by the variable coefficients, which represent the log odds for a given variable.

Table 2.

Logistic Regression for Binary Classifier

	OR	p-value
Pre-op headache	2.19	<0.01
PT	2.11	<0.01
Vit D deficiency	1.88	<0.01
Ophthalmic disease	1.68	<0.01
Neuropathy	1.53	<0.01
Pre-op dizziness	1.46	<0.01
Post-op vestibular meds	1.44	<0.01
Neurologic disorder	1.37	<0.01
Vestibular comorbidity	1.33	<0.01
Tumor size	1.31	<0.01

Four-way Classifier Development.

The best candidate models were linear discriminant analysis (AUC 0.84), gradient boosting (AUC 0.83), and logistic regression (AUC 0.83). The linear discriminant analysis (LDA) model was chosen for further analysis. This produced an AUC of 0.86 on the holdout testing set. On feature importance analysis, the top three most important features were preoperative headache, need for physical therapy on discharge, and the presence of any systemic comorbidity. On further inspection of the LDA model, the variable coefficients provided further insight into the magnitude of their effects on the model’s predictions: 0.89 for preoperative headache, 0.56 for physical therapy, and 0.51 for systemic comorbidities. Table 3 lists the coefficients for the top ten most important features in the LDA multi-class model, and Figure 2 shows these coefficients to illustrate their relative importance.

Table 3.

Linear Discriminant Analysis for Multi-class Classifier

	|Coefficient|
Pre-op headache	0.89
PT	0.56
Systemic comorbidity	0.52
Post-op vestibular meds	0.46
Ophthalmic disease	0.45
Vitamin D deficiency	0.45
Pre-op dizziness	0.43
Migraine	0.40
Neurologic disorder	0.38
Hypertension	0.33

Figure 2.

This figure shows the feature importance scores for the top ten most important features to the multi-class classification linear discriminant analysis (LDA) model. For LDA, feature importance can be measured by the variable coefficients in the LDA equation.

Discussion

We successfully developed predictive models for persistent vestibular symptoms following vestibular schwannoma surgery using a cohort of 1,000+ cases. The models demonstrated excellent performance, with the binary classification logistic regression model producing an AUC of 0.89, and the four-way classifier LDA model producing an AUC of 0.86. It is known that logistic regression is very powerful for binary classification, however for multiple classes linear discriminant analysis usually performs better, and this was indeed the case in our modeling.¹⁰ Interpretations of model predictions provided insights into factors associated with persistent postoperative vestibular symptoms following vestibular schwannoma surgery. Such factors included preoperative headache, need for physical therapy on discharge, vitamin D deficiency, and systemic comorbidities.

This is the largest study of its kind that attempts to predict the risk of postoperative vestibular symptoms following VS surgery, and the first to do so with a machine learning approach. Compared to approaches taken in prior studies, a machine learning approach has the potential for superior prediction accuracy and would offer individualized estimates of risk based on unique patient factors. The literature is widely variable when it comes to identifying factors predictive of postoperative dizziness. The review by Saman et al. on balance dysfunction following VS surgery provides an excellent and comprehensive summary of the relevant studies.¹¹ One commonly identified risk factor is preoperative dizziness, with prior studies having linked the presence, duration, and quality of preoperative dizziness to elevated risk of postoperative dizziness.^12–14 In our study, preoperative dizziness was an important feature in both the binary and multi-class models, with an odds ratio of 1.46 (p < 0.01) in the binary logistic regression model. Preoperative headache was the most important feature in our models, with an odds ratio of 2.19 (p < 0.01) in the binary classification model, and this is also consistent with prior studies. A study by Carlson et al. on long-term dizziness handicap in patients with VS found that headache was significantly associated with dizziness, though this study included tumors undergoing observation and radiotherapy in addition to surgery.² Other features such as need for physical therapy on discharge and certain comorbidities such as ophthalmic disease and neuropathy are intuitively linked to heightened risk of postoperative vestibular dysfunction – need for physical therapy on discharge would be indicative of more severe vestibular dysfunction in the postoperative period, and ophthalmic and neuropathic disease would likely complicate central compensation by limiting the compensatory mechanisms of vision and proprioception. Lastly, the impact of psychological factors on persistent postoperative dizziness should not be overlooked. While psychiatric disease was not identified as a top predictor in our models, there was a high incidence of psychiatric disease in our cohort (9.6%). Prior studies have shown a significant incidence of psychological problems such as anxiety and depression following VS surgery,¹⁵ with one identifying depression as a risk factor for poor postural recovery.¹⁶ Taken together, these findings reinforce that such features are important to take note of as risk factors, and they would contribute to individualized risk estimates for postoperative vestibular dysfunction, helping to guide patient counseling and postoperative management.

Persistent postoperative vestibular symptoms can impair quality of life, underscoring the potential value of being able to predict this outcome. A recent study on health-related quality of life (HRQoL) after VS surgery showed that SF-36 scores were strongly associated with dizziness handicap inventory (DHI) scores and self-reported dizziness.⁴ Several other studies as well as a systematic review on quality of life in VS patients managed with observation, radiosurgery, or microsurgery, have likewise shown that dizziness may in fact be the strongest predictor of long-term quality of life reduction.^5,6 More specifically, one study evaluating the impact of VS-associated symptoms found that balance disturbance had significant psychological and functional impact.¹⁷ Given the clear importance of dizziness in quality of life and as a functional outcome following surgery, being able to predict this outcome could be very useful in guiding management.

Predictive models could be used to guide management decisions for VS by providing clinicians and patients with quantitative estimates of risk. For example, predictive models have been developed to predict VS growth,^18,19 VS recurrence following surgery,^20,21 and VS treatment response following radiation treatment.^22,23 In our study, the binary classification model could provide a probabilistic estimate of having vestibular dysfunction at the 3-, 6-, or 12-month marks; and the 4-way classification model would provide individual probabilities for each of these time points. Predictions would be individualized to a patient based on their unique combination of factors. Patients could be counseled accordingly when discussing surgery and managed accordingly postoperatively, for example with more extensive vestibular therapy.

This study has several limitations: 1) Vestibular dysfunction was measured as a subjective, binary outcome. This fails to account for the variable nature of dizziness, for example its severity and quality. Future studies may consider collecting more rigorous measures of vestibular symptoms such as DHI for predictive modeling. 2) The outcome of dizziness was not systematically collected; rather this was inferred from whether dizziness was reported in the clinic notes. While all patients were asked about vestibular symptoms at postoperative visits, the lack of systematic collection introduces information bias into our data. This was a result of the retrospective nature of our study and our desire to maximize sample size for the purposes of modeling. 3) Some demographic variables were not collected and thus not included in the model such as race, ethnicity, and socioeconomic status – these factors could affect model predictions and should be considered in future studies. 4) The surgeon was not collected as a variable; differences in surgical technique among surgeons may affect dizziness as an outcome, and this should be considered in practice. However, this would not be feasible to include in the model as it could not be generalized outside of the setting in which it was developed. 5) This model uses preoperative and immediate postoperative features as predictors; this optimizes performance for predicting long-term vestibular symptoms, however at the expense of being able to be used in the preoperative setting alone, for example to counsel patients who are still deciding whether to undergo surgery. Future adaptations of this work could modify the model to include only preoperative features. 6) This is a model development study and not a model validation study – external validation on cohorts from other institutions and patient populations is necessary before such models can be considered generalizable.

Conclusion

In summary, we have demonstrated the feasibility of a machine learning approach to predict persistent vestibular symptoms following vestibular schwannoma surgery with high accuracy. Risk factors that emerged included preoperative headache, need for physical therapy on discharge, vitamin D deficiency, and systemic comorbidities. Vestibular symptoms significantly impact quality of life in vestibular schwannoma patients, underscoring the value of being able to predict this to guide patient counseling and management. Future work could focus on modeling with better measures of dizziness such as DHI in order to make predictions more meaningful.

Glossary of Machine Learning Terminology

Binary classification model

A model that predicts a binary outcome, or an outcome with only two possible options (i.e., no dizziness vs. any dizziness)

Multiclass classification model

A model that predicts an outcome with multiple possible options (i.e., no dizziness or dizziness at 3-, 6-, and 12-month timepoints)

Types of classification models

16 types of classification models were used in this study. An overview of all of these is beyond the scope of this manuscript, however the interested reader is referred here ²⁴ for further descriptions

Area under the curve (AUC)

A common measure of classification model performance that represents the area under the receiver operating characteristic (ROC) curve. The ROC curve plots the true positive rate vs. false positive rate at different probability thresholds predicted by the model. The AUC represents model performance across all these thresholds, providing an overall summary of model performance on a scale from 0 to 1

5-fold cross-validation

This is a model training technique in which the training set is divided into 5 subsets, and a model is trained on four subsets and tested on the fifth. This is repeated with each combination of subsets to estimate average model performance for a certain set of parameters