Machine Learning Approaches to Define Candidates for Ambulatory Single Level Laminectomy Surgery

Abstract

Study Design:

retrospective cohort study.

Objectives:

To test and compare 2 machine learning algorithms to define characteristics associated with candidates for ambulatory same day laminectomy surgery.

Methods:

The American College of Surgeons National Surgical Quality Improvement Program database was queried for patients who underwent single level laminectomy in 2017 and 2018. The main outcome was ambulatory same day discharge. Study variables of interest included demographic information, comorbidities, preoperative laboratory values, and intra-operative information. Two machine learning predictive modeling algorithms, artificial neural network (ANN) and random forest, were trained to predict same day discharge. The quality of models was evaluated with area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) measures.

Results:

Among 35,644 patients, 13,230 (37.1%) were discharged on the day of surgery. Both ANN and RF demonstrated a satisfactory model quality in terms of AUC (0.77 and 0.77), accuracy (0.69 and 0.70), sensitivity (0.83 and 0.58), specificity (0.55 and 0.80), PPV (0.77 and 0.69), and NPV (0.64 and 0.70). Both models highlighted several important predictive variables, including age, duration of operation, body mass index and preoperative laboratory values including, hematocrit, platelets, white blood cells, and alkaline phosphatase.

Conclusion:

Machine learning approaches provide a promising tool to identify candidates for ambulatory laminectomy surgery. Both machine learning algorithms highlighted the as yet unrecognized importance of preoperative laboratory testing on patient pathway design.

Introduction

Demand for lumbar spine surgery has grown rapidly in recent years^1,2 with an increased push toward ambulatory surgery. This has accelerated after the institution of the Affordable Care Act which emphasizes cost-effective practice.^3-5 Ambulatory lumbar decompression surgery, including hemi-laminectomy and laminectomy, has been associated with positive outcomes and a good safety profile, and is supported by financial incentives.^6-9

Currently, around 50% of all laminectomy patients are outpatient admissions, with half of those being discharged on the day of surgery. ¹⁰ Therefore, it is estimated that over one quarter of all laminectomy patients constitute ambulatory same day discharges. While more hospitals are adopting outpatient laminectomy practice, it remains widely debated how to effectively achieve the goal of ambulatory surgery. Patient selection has been a focus of previous research, and many risk factors have been identified. ¹⁰ However, published models lack actual or potential clinical applicability.

We therefore speculate that there are unrecognized factors essential for clinical decision making that escape the standard methodological approaches. It is also possible that previously identified factors and successful same day discharge are not associated in a linear fashion, and thus may not be effectively established with traditional regression-based analysis strategies. Recently, machine learning algorithms have been adopted into medicine, including orthopedics, with reported success.^11,12 Machine learning approaches usually require limited or no intervention, and could manage complex non-linear relationships. The aim of this study was therefore to test and compare 2 machine learning algorithms in identifying factors associated with ambulatory same day discharge in single level laminectomy patients. These data may be used to refine definitions of optimal candidates for ambulatory surgery.

Methods

This retrospective cohort study was approved by our institutional review board (IRB# 2017-0716). Informed consent was exempted from requirement by IRB since the data does not include patient identification information. We acquired 2017 and 2018 data from the American College of Surgeons National Surgical Quality Improvement Project (NSQIP). Patients were identified via Current Procedural Terminology (CPT) codes for hemi-laminectomy and laminectomy (CPT 63,030 and 63,047, n = 40,465). The study followed The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines. ¹³ We excluded patients with an American Society of Anesthesiologists (ASA) physical status classification score of 4 and 5 (n = 810), those with concurrent surgical procedures (n = 626), emergency admissions (n = 3,206), or missing length of stay (LOS) (n = 179). The final cohort included 35,644 patients.

Study Variables

Patients were grouped into ambulatory same-day discharge or LOS ≥ 1 day. All available preoperative and intraoperative variables (n = 40 variables) in the NSQIP database were deemed eligible for the modeling process. Variables included demographic information, comorbidities and disease burden, preoperative laboratory values and surgery-specific variables. Demographic information included age, sex, race and ethnicity, and body mass index (BMI). Comorbidities and disease burden variables included ASA physical status classification, patient functional status (independent, partially dependent, totally dependent), and the presence of any of the following before surgery: diabetes, smoking, ventilator dependence, severe chronic obstructive pulmonary disease (COPD), ascites, congestive heart failure within 30 days prior to surgery, hypertension requiring medication, preoperative renal failure, current need for dialysis, history of disseminated cancer, current open wound/wound infection, steroid use, history of bleeding disorders, >10% loss of body weight in the 6 months preceding surgery, sepsis within 48 hours prior to surgery, red blood cell transfusions within 72 hours prior to surgery, and preoperative dyspnea. Preoperative laboratory values included serum sodium, blood urea nitrogen (BUN), serum creatinine, serum albumin, total bilirubin, serum glutamic oxaloacetic transaminase (SGOT), alkaline phosphatase, while blood cell (WBC) count, hematocrit, platelet count, partial thromboplastin time (PTT), and International Normalized Ratio (INR) values. Surgery-specific variables included primary diagnosis (spinal stenosis, disc disorder, other), year of surgery, type of anesthesia (spinal/epidural or general), and duration of surgery.

Statistical Analysis

All data processing was performed using SAS version 9.4 (SAS institute, Cary, NC). All statistical analyses were performed using Python version 3.6 (Python Software Foundation). Missing data patterns were evaluated and imputed to the median of the non-missing values of the respective variables in the modeling exercise, a commonly utilized approach in machine learning projects. ¹⁴ Continuous variables were compared using a 2-sample t-test or Wilcoxon rank-sum test. Categorical variables were analyzed using chi-square or Fisher’s exact tests. Statistical significance was defined as p < 0.05.

Two machine learning algorithms, artificial neural networks (ANNs) and random forest (RF) models, were applied for data analysis. ANN is a machine learning computing algorithm, which learns to perform tasks without requiring task-specific pre-defined rules. This type of machine learning can understand and represent more complex inner relationships, whether linear or non-linear, among variables in a large dataset. ¹⁵ Random forest is a regression-based classification algorithm with the aggregation of a large number of decision trees trained on randomly sampled subsets of complex dataset.^16,17 Random forest applies randomness when building each individual tree, and thus prediction by grouping could be more accurate than any individual tree. Both models were trained and tested using cross-validation with 80% data as a training set and 20% as a validation set.

Model Development

ANNs and RF models were trained in an 80% random subset of our data to predict ambulatory same day discharge after single level laminectomy versus LOS ≥1. All variables were normalized prior to training to balance feature differences in ranges and scales. A class weight was inserted into all models to balance the frequency differences between 2 study cohorts to avoid favoring the dominant. ¹⁸ ANN classification models were trained using the Keras open-source Python package with a batch size of 100 and of 50 iterations. The RF classification model was applied using the Python sklearn package with a grid search algorithm to pick the best combination of tuning parameters. The model process engaged the number of trees, the number of features at every split, and maximum depth using a 10-fold cross-validation.

Once models were optimized, the quality was evaluated using the remaining 20% validation data. Area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were reported.

Permutation importance was calculated to select top weighted features in the ANN model, i.e. the most important predictive variables. Permutation importance was calculated by looking at how much the accuracy score decreases by shuffling values for a feature in test dataset. ¹⁹ A larger importance score means the model is more dependent on the feature. For the RF model, feature importance (i.e. variable importance) is calculated as the decrease in node impurity weighted by the probability of reaching that node. The more an attribute is used to make key decisions, the higher its relative importance would be.

Results

Among 35,644 single level laminectomy surgeries 13,230 (37.1%) involved a discharge on the same day of surgery. Same day discharge patients were younger (52.6 +/- 15.7 versus 61.0 +/- 14.6 years of age), more often had an ASA score of I/II (69.0% versus 51.0%), and had shorter durations of surgery (78.2 +/- 39.2 versus 126.1 +/- 77.0 mins) (Table 1, all p < 0.001). Same day discharge patients were more likely with diagnosis of spinal stenosis (51.4%) than disc disorder (30.8%, Table 1, p < 0.001). Same day discharge patients were less likely to have diabetes (13.0% vs 20.2%), or hypertension (39.2% vs 55.5%, Table 1). There were also differences in several laboratory values, including BUN, albumin, alkaline phosphatase, hematocrit, platelet, and INR (Table 1).

Table 1.

Patient Characteristics Grouped by Discharge Days.

Characteristics	Ambulatory same day discharge	Discharge> = 1 day	Standardized difference	P values
N (%)	13230 (37.1)	22414 (62.9)
Age, mean (sd)	52.6 (15.7)	61.0 (14.6)	-0.545	<.001
BMI, mean (sd)	30 (6.2)	31 (6.4)	-0.157	<.001
Female, n (%)	5430 (41.0)	10004 (44.6)	-0.073	<.001
White, n (%)	10582 (80.0)	17452 (77.9)	0.049	<.001
Hispanic, n (%)	685 (5.2)	1414 (6.3)	0.058	<.001
ASA, n (%)			0.408	<.001
Level I	1369 (10.3)	913 (4.1)
Level II	7772 (58.7)	10519 (46.9)
Level III	4088 (30.9)	10982 (49.0)
Year of surgery, n (%)			0.202	<.001
2017	6538 (49.4)	11702 (52.2)
2018	6692 (50.6)	10712 (47.8)
Function Status			0.202	<.001
Independent	13186 (99.7)	22077 (98.5)
Partially dependent	43 (0.3)	322 (1.4)
Totally dependent	1 (0)	15 (0.1)
Length of operation (min)	78.2 (39.2)	126.1 (77.0)	-0.784	<.001
Diagnosis			0.602	<.001
Spinal stenosis	11524 (51.4)	3553 (26.9)
Disc disorder	6896 (30.8)	7823 (59.1)
Other	3994 (17.8)	1854 (14.0)
Anesthesia Type			0.142	<.001
General anesthesia	13076 (98.8)	22316 (99.6)
Spinal or Epidural	97 (0.7)	68 (0.3)
Other	57 (0.4)	30 (0.1)
Diabetes	1722 (13.0)	4532 (20.2)	-0.194	<.001
Smoking	2705 (20.4)	3869 (17.3)	0.082	<.001
Ventilation dependent	0 (0)	1 (0)	0.012	0.371
Severe COPD	289 (2.2)	881 (3.9)	-0.102	<.001
Ascites	2 (0)	1 (0)	0.017	0.138
Congestive heart failure (CHF) in 30 days before surgery	6 (0)	58 (0.3)	-0.053	<.001
Hypertension requiring medication	5186 (39.2)	12449 (55.5)	-0.332	<.001
Renal failure	1 (0)	4 (0)	-0.009	0.657
Currently on dialysis	10 (0.1)	31 (0.1)	-0.019	0.106
Disseminated cancer	11 (0.1)	56 (0.2)	-0.041	<.001
Open wound/wound infection	3 (0)	16 (0.1)	-0.023	0.059
Steroid use for chronic condition	421 (3.2)	926 (4.1)	-0.051	<.001
>10% loss body weight in last 6 months	21 (0.2)	38 (0.2)	-0.003	0.893
Bleeding disorders	98 (0.7)	331 (1.5)	-0.070	<.001
Pre-op Transfusions (RBC within 72 Hours Prior to Surgery Start Time)	1 (0)	10 (0)	-0.023	0.064
Dyspnea	335 (2.5)	958 (4.3)	-0.096	<.001
Sepsis within 48 Hours Prior to Surgery	21 (0.2)	60 (0.3)	-0.024	0.038
Serum sodium	139.9 (11.4)	139.6 (2.7)	0.027	0.002
BUN	16.1 (5.2)	17.1 (6.4)	-0.163	<.001
Serum Creatinine	0.9 (0.4)	0.9 (0.4)	-0.048	0.047
Albumin	4.2 (0.2)	4.2 (0.3)	0.146	<.001
Bilirubin	0.5 (0.2)	0.5 (0.3)	-0.014	0.020
SGOT	22.6 (10.1)	23 (10.4)	-0.046	0.020
Alkaline phosphatase	70.4 (14.7)	72.4 (19.5)	-0.113	<.001
WBC	7.4 (2.2)	7.4 (2.2)	0.008	0.363
Hematocrit	42.3 (3.7)	41.6 (4.1)	0.171	<.001
Platelet	247.9 (58.7)	243.8 (64.5)	0.066	<.001
PTT	28.8 (2.6)	28.9 (2.9)	-0.036	0.374
INR	1 (0.1)	1 (0.2)	-0.070	<.001

BMI: Body mass index; ASA: American Society of Anesthesiologists classification; COPD: Chronic obstructive pulmonary disease; BUN: Blood urea nitrogen; SGOT: Aspartate aminotransferase test; WBC: White blood cell; PTT: Partial thromboplastin time; INR: International normalized ratio.

Both ANN and RF modeling approaches resulted in satisfactory model performance with an AUC of 0.77 and 0.77, respectively. Additional testing, resulted in accuracies of 0.69 and 0.70, PPVs of 0.77 and 0.69, NPVs of 0.64 and 0.70, sensitivity values of 0.83 and 0.58, and specificity values of 0.55 and 0.80 respectively (Table 2).

Table 2.

Summary of Model Validation Results.

	ANN (ambulatory vs day> = 1)	Random forest (ambulatory vs day> = 1)
AUC	0.77	0.77
Accuracy	0.69	0.70
PPV	0.77	0.69
NPV	0.64	0.70
Sensitivity	0.83	0.58
Specificity	0.55	0.80

ANN: Artificial neural network; AUC: Area under the curve; PPV: Positive predictive value; NPV: Negative predictive value

The top 3 most important predictive variables were the same in both the ANN and RF model approaches: duration of surgery, age, and primary diagnosis (Table 3). For the remaining 17 of the top 20 identified features, 11 were shared between ANN and RF models, including ASA classifications and Hispanic race. Interestingly, 7 of the 11 features were preoperatively measured laboratory values, most of which were not previously referenced for ambulatory pathway determination (Table 3).

Table 3.

Ranking of the Top 20 Important Features From The Models (ANN & Random Forest).

ANN ambulatory vs Day> = 1		Random forest ambulatory vs Day> = 1
Feature	Importance	Feature	Importance
Duration of operation	0.0018	Duration of operation	0.363
Age	0.0017	Diagnosis	0.106
Diagnosis	0.0016	Age	0.100
ASA	0.0016	BMI	0.047
Diabetes	0.0015	HCT	0.045
Hispanic	0.0013	Platelet	0.040
Female	0.0013	ASA	0.040
HCT	0.0013	WBC	0.034
Steroid use for chronic condition	0.0013	Creatinine	0.033
INR	0.0011	BUN	0.027
Year	0.0010	PTT	0.023
PTT	0.0009	Hypertension requiring medication	0.021
Function status	0.0009	Sodium	0.017
Hypertension requiring medication	0.0008	alkaline phosphatase	0.016
Platelet	0.0007	Albumin	0.016
Albumin	0.0007	INR	0.013
Severe COPD	0.0018	SGOT	0.012
alkaline phosphatase	0.0017	Year	0.009
Creatinine	0.0016	Bilirubin	0.008
White	0.0016	Hispanic	0.007

ANN: Artificial neural network; ASA: American Society of Anesthesiologists classification; BMI: Body mass index; HCT: Hematocrit; WBC: White blood cell; INR: International normalized ratio; BUN: Blood urea nitrogen; PTT: Partial thromboplastin time; COPD: Chronic obstructive pulmonary disease; SGOT: Aspartate aminotransferase test.

Discussion

With the growing emphasis on value-based care and financial incentives, ambulatory surgical practice is rapidly expanding across all healthcare subspecialties and institutions. Many studies have supported the safety and feasibility of outpatient spine surgery.^{6- 9} However, there is minimal evidence to guide the practices of ambulatory lumbar laminectomy. Therefore, identifying ambulatory surgery candidates effectively and potential risk factors for failure might prove to be cost-effective without compromising healthcare quality. We applied machine learning strategies to predict ambulatory discharge after lumbar laminectomy with the expectation of introducing less potentially biased assumptions as usually observed with traditional regression analysis. Our analysis indicated good model quality, including AUC, accuracy, predictive values, and specificity. We believe this model could continue self-improving with more and more patient data available, and it is our expectation that this model has the potential for future clinical application.

In NSQIP data from 35,644 patients we found that ambulatory same day discharge laminectomy occurred in 37.1% of all patients. Our study results support previous reports that younger, healthier patients, and shorter surgery are more likely to be associated with home discharge on the day of surgery. Age has been described as a predictor of LOS in various lumbar spine surgeries, including lumber decompression,^10,20 discectomy, ²¹ and lumbar fusion. ²² Longer operative time has been associated with higher complexity and more intraoperative complications, ²³ and has been reported as a risk factor for failure of ambulatory discharge after lumbar spine surgery. ¹⁰ The same study also reported that primary diagnosis was related to odds of ambulatory discharge, and patients with a disc disorder were more likely to be discharged home on the day of surgery. ¹⁰

Both ANN and EF highlighted the importance of laboratory values in identifying ambulatory surgical patients. Elevated baseline hemoglobin level was reported as a predictor for early postoperative discharge after lumbar fusion. ²⁴ However, other laboratory values, such as platelets, INR, PTT, albumin, alkaline phosphatase, creatinine, and WBC, have not been previously reported. Although a basic laboratory exam is not routinely performed across all ambulatory patients, the fact that machine learning algorithms highlighted these features should promote further investigation. Ambulatory patients may be reasonably healthy without clinical indication for laboratory testing. However, laboratory results might provide insights of how well patient may tolerate a surgical insult and reveal pseudo-healthy conditions before clinical symptoms are seen. As routine basic laboratory exam is usually low-cost, it may be reasonable to consider including laboratory exam as a component of pre-operative evaluation.

Recently, machine learning approaches have been applied in several clinical contexts including in orthopedics, for example in predicting readmission after hip fracture surgery, with some success.^15,25 Advantages of machine learning algorithms over conventional regression analysis include that they do not require pre-determined rules and are therefore able to handle more complicated relationships between variables, either linear or non-linear^15,25 Machine learning is therefore believed to be less biased. In addition, machine learning is an evolving process with continuous building, learning, and improving behind the scenes, and it becomes more powerful with accumulation of more patient data.

To the best of our knowledge, this is the first study applying machine learning algorithms to predict ambulatory patients after laminectomy. Both ANN and RF have shown good model quality. In addition, these machine learning algorithms provided more details on the quality, including good accuracy, good predictive values, and great specific, which cannot be estimated with the traditional regression approaches.

Our study has limitations. First, it represents retrospective analysis of prospectively collected data. Not all hospitals practice ambulatory laminectomy, not all hospitals discharge laminectomy patients according to the same rules, and therefore patient cohort assignment for analysis might be biased. Second, the analysis is limited only to those data elements that were available in the NSQIP database. Third, machine learning is a black box process. Although we enlisted several features for discussion, how and why these features affect the ambulatory discharge process could not be determined by the machine learning algorithms used. Our study should be interpreted as hypothesis generating, and future studies based on our findings especially about the laboratory values are indicated.

In conclusion, machine learning classification is promising in identifying candidates for ambulatory lumbar laminectomy surgery. The models applied in this study possess good quality, accuracy, specificity, and predictive values. More importantly, the models could serve clinically and continuously improve in parallel with more patient data.