Machine learning application for prediction of surgical site infection after posterior cervical surgery

Abstract

Surgical site infection (SSI) is one of the most common complications of posterior cervical surgery. It is difficult to diagnose in the early stage and may lead to severe consequences such as wound dehiscence and central nervous system infection. This retrospective study included patients who underwent posterior cervical surgery at The Second Affiliated Hospital and Yuying Childrens Hospital of Wenzhou Medical University from September 2018 to June 2022. We employed several machine learning methods, such as the gradient boosting (GB), random forests (RF), artificial neural network (ANN) and other popular machine learning models. To minimize the variability introduced by random splitting, the results underwent 10‐fold cross‐validation repeated 10 times. Five measurements were averaged across 10 repetitions with 10‐fold cross‐validation, the RF model achieved the highest AUROC (0.9916), specificity (0.9890) and precision (0.9759). The GB model achieved the highest sensitivity (0.9535) and the KNN achieved the highest sensitivity (0.9958). The application of machine learning techniques facilitated the development of a precise model for predicting SSI after posterior cervical surgery. This dynamic model can be served as a valuable tool for clinicians and patients to assess SSI risk and prevent it in clinical practice.

1. INTRODUCTION

Surgical site infection (SSI) is a known and serious postoperative complication following cervical spine surgery. ¹ Including spinal surgery, SSI is associated with increased patient morbidity and mortality, healthcare expenses due to readmission and reoperation, and poor prognosis. ² , ³ Identifying risk factors for SSI will help develop interventions to reduce the incidence of these infections. Several risk factors for SSI have been reported, and include increased BMI, advanced age, smoking, diabetes, malnutrition, longer operative times, increased intraoperative bleeding, hypoproteinaemia, subcutaneous fat thickness and a history of prior SSI. ⁴ , ⁵ , ⁶ , ⁷ , ⁸ , ⁹ Nevertheless, the results of these previous studies were not always consistent. Moreover, previous studies have shown that the rate of SSI after posterior cervical spine surgery is much higher than anterior surgery, with rates up to 18%. ⁴ , ¹⁰

Artificial intelligence has been widely used in medical research and the predictive validity of machine learning is widely recognized. The machine learning approaches facilitated the successfully development of outcome prediction tools in healthcare that incorporate perioperative parameters and clinical variables. ¹¹ , ¹² Most of the current machine learning‐based predictive models compare the effectiveness of different algorithms to identify the most suitable one. ¹³

Given the significantly higher incidence of infection reported in the literature after posterior cervical spine surgery, an objective and ideal prediction model would help clinicians identify potential susceptible patients. In this work, we aimed to develop and internally validate various machine learning models to predict SSI after posterior cervical surgery. The employed models encompassed the gradient boosting classifier (GB), random forests classifier (RF), artificial neural network classifier (ANN), logistic regression classifier (LR), decision tree (DT), support vector machine classifier (SVM), K‐nearest neighbours classifier (KNN) and Naive Bayesian (NB).

2. METHODS

2.1. Study design

This retrospective cohort study included patients who underwent posterior cervical surgery (The Second Affiliated Hospital and Yuying Childrens Hospital of Wenzhou Medical University, Wenzhou 325 000, China) from September 2018 to June 2022. The International Classification of Diseases, Tenth revision (ICD‐10) diagnosis codes M4790 (cervical spondylosis), S1410 (cervical spinal cord injury), M4880 (ossification of the posterior longitudinal ligament of the cervical spine), G9590 (intraspinal occupying lesion), M4800 (cervical spinal stenosis) and S1310 (atlantoaxial dislocation) were utilized to access and review patient data from our electronic medical records. The flowchart of this machine learning study design is shown in Figure 1.

FIGURE 1.

Workflow picture. ANN, artificial neural network; AUROC, area under the receiver operating characteristic; DT, decision tree; GB, the gradient boosting classifier; KNN, K‐nearest neighbours classifier; LR, logistic regression; NB, Naive Bayesian; RF, random forests; SVM, support vector machine classifier.

2.2. Inclusion and exclusion data

Diagnosis of incisional SSI adhered to CDC's National Healthcare Surveillance Network (NHSN) standardized criteria. ¹⁴ , ¹⁵ Superficial SSI diagnosis requires at least one of the following: (1) Purulent drainage from the superficial incision, (2) Organisms isolated from an aseptically obtained culture from the superficial incision (3) Pain, localized swelling, erythema or heat, and superficial incision is deliberately opened. (4) Diagnosis by the surgeon. Deep SSI mainly involves deep soft tissue, and its diagnosis requires at least one of the following: (1) Purulent drainage from the deep incision, (2) A deep incision spontaneously dehisces or is intentionally opened when a patient shows signs like fever (>38°C) or localized pain, unless the site is culture‐negative. (3) Abscesses or other evidence of infections detected by radiological or histopathological examination. (4) Diagnosis by the surgeon. In addition, exclusion criteria were as follows: (1) Primary spinal infection such as spinal tuberculosis; (2) Previous history of spinal surgery; (3) Simultaneously combining other surgeries such as combined anterior cervical surgery or thoracic spine surgery; (4) Minimally invasive posterior cervical surgery; (5) Incomplete patient clinical data.

2.3. Data collection

This study was approved by the Ethics Committee at the Second Affiliated Hospital, Wenzhou Medical University. All methods were performed in accordance with the Declaration of Helsinki and relevant guidelines. The written informed consent was obtained from every volunteer involved. Demographic and clinical data were extracted from our electronic medical records. To predict SSI after posterior cervical surgery, we collected all available clinical features, including preoperative clinical characteristics, laboratory parameters and operative records. Preoperative MRI and/or CT was used to determine the ratio of the subcutaneous fat thickness to the distance between the lamina and the skin surface, with all measurements being performed at the level of the C2‐7 vertebra with use of axial and sagittal views (Figure 2). ⁴

FIGURE 2.

Sagittal T2‐weighted cervical magnetic resonance image demonstrating the measurement of the morphometric parameters.

Finally, 151 of the 553 patients who underwent posterior cervical surgery met the exclusion criteria. Eight‐two of these patients did not have complete clinical data, 35 had primary spinal disease, 25 had multiple site combined surgery and nine had minimally invasive posterior cervical surgery. Therefore, this study cohort consisted of the remaining 402 patients, with 44 individuals developing a SSI.

2.4. Data preprocessing

To reduce sensitivity to variable scales during the training process, all variables were standardized to a range of 0–1. The low incidence of SSI after posterior cervical surgery resulted in significant data imbalance among the study groups, posing a challenge for predictive modelling. ¹⁶ , ¹⁷ To improve the model performance, the sample numbers between groups should be balanced and the data should be standardized. The dataset was balanced using the Tomek links (SMOTE‐Tomek). ¹⁸

2.5. Statistical analysis

To evaluate the machine‐learning performance, we performed a 10‐fold cross‐validation to avoid overfitting. Eight distinct machine learning algorithms were employed for data analysis: the gradient boosting classifier (GB), random forests classifier (RF), artificial neural network classifier (ANN), logistic regression classifier (LR), decision tree (DT), support vector machine classifier (SVM), KNN and Naive Bayesian (NB). First, patients were randomly divided into 10 groups. Then, one group was chosen, and its associated nodules were utilized to form the test dataset, while the remaining nine groups were treated as the training dataset. Subsequently, the machine learning models were trained in the training dataset and then used to predict SSI in the test dataset. This same process was repeated until all groups were predicted. To minimize the variability introduced by random splitting, the 10‐fold cross‐validation was repeated 10 times. Model performance was assessed by comparing predictions with the true SSI status. The accuracy, area under the receiver operating characteristic (AUROC), sensitivity, specificity and precision were then calculated in different models. These metrics collectively offer a comprehensive evaluation of the predictive models' diagnostic capacity. The data were analysed using R4.3.1 software (R Foundation for Statistical Computing, Vienna, Austria). ¹⁹

3. RESULTS

3.1. Data characteristics

A total of 402 patients meeting the inclusion criteria were collected: age, gender, diabetes, trauma, spinal cord injury, BMI, subcutaneous fat thickness, ratio of fat thickness to lamina‐skin distance, operation time, number of vertebral bodies spanned, intraoperative bleeding volume, glucose, albumin, Hb, PLT, WBC, CRP, ESR, postoperative drainage volume, postoperative drainage time. The distribution characteristics between the SSI group and the non‐SSI group are presented in Table 1. In addition, the heat map illustrated the correlation of patient's clinical features (Figure 3).

TABLE 1.

The distribution of each variable that meets the screening condition.

Patient characteristics	Non‐SSI	SSI	p‐Value
	(N = 358)	(N = 44)
Age	62.4 ± 11.7	65.0 ± 9.7	0.150
Gender
Female	98 (27.4%)	11 (25.0%)	0.739
Male	260 (72.6%)	33 (75.0%)
Diabetes
No	299 (83.5%)	35 (79.6%)	0.508
Yes	59 (16.5%)	9 (20.5%)
Trauma
Yes	114 (31.8%)	24 (54.5%)	0.003
No	244 (68.2%)	20 (45.5%)
SCI
Yes	211 (58.9%)	15 (34.1%)	0.002
No	147 (41.1%)	29 (65.9%)
BMI	23.0 ± 2.7	27.1 ± 5.1	<0.001
SFT	17.1 ± 5.9	10.2 ± 4.5	<0.001
Ratio of fat thickness to lamina‐skin distance	0.21 ± 0.08	0.28 ± 0.07	<0.001
OP time	127.3 ± 36.6	153.6 ± 62.8	<0.001
Number of vertebral bodies spanned	4.4 ± 0.9	4.8 ± 1.1	0.015
Intraoperative bleeding volume	131.5 ± 101.2	176.8 ± 121.1	0.006
Glucose	6.2 ± 2.2	7.2 ± 3.0	0.006
Albumin	42.3 ± 4.8	39.3 ± 3.7	<0.001
Hb	137.4 ± 17.3	129.0 ± 12.9	0.002
PLT	230.3 ± 68.6	220.2 ± 62.8	0.350
WBC	7.9 ± 2.6	9.0 ± 3.0	0.009
CRP	13.0 ± 19.3	20.7 ± 35.8	0.028
ESR	19.4 ± 19.1	25.6 ± 27.4	0.056
Postoperative drainage volume	216.5 ± 117.4	349.0 ± 278.8	<0.001
Postoperative drainage time	2.3 ± 0.7	2.5 ± 1.0	0.090

Note: p‐value < 0.05 are shown in bold.

Abbreviations: BMI, body mass index; CRP, C‐reactive protein; ESR, erythrocyte sedimentation rate; Hb, haemoglobin; OP‐time, operation time; PLT, platelet; SCI, spinal cord injury; SFT, subcutaneous fat thickness; WBC, white blood cell.

FIGURE 3.

Heat map of the correlation of patient's clinical features. BMI, body mass index; CRP, C‐reactive protein; ESR, erythrocyte sedimentation rate; Hb, haemoglobin; OP‐time, Operation time; PLT, platelet; SFT, subcutaneous fat thickness; WBC, white blood cell.

3.2. Overall model performance

Table 2 summarizes the accuracy, AUROC, sensitivity, specificity, positive predictive and negative predictive of eight machine learning models, calculated under 10‐fold cross‐validation. All measurements were averaged across 10 repetitions. Among all models, the RF exhibited the highest AUROC (0.9916), specificity (0.9890) and precision (0.9759), which indicated a robust overall capacity to differentiate SSI and non‐SSI group. It is more capable of predicting the non‐SSI following posterior cervical surgery. Besides, the high precision of the RF meant that among its predicted SSI group, a significant proportion of experienced SSI occurrences. The GB model outperforms others in terms of accuracy (0.9535). The high sensitivity of the KNN shows its strongness in predicting the occurrence of SSI (0.9958). To sum up, all machine learning models demonstrated varied performance when predicting the occurrence of SSI following posterior cervical surgery. Given that no single model outperforms others across all five measurements, the choice of models can be tailored based on specific requirements for diagnosing SSI.

TABLE 2.

The model prediction performance measured by five measurements.

Model	Accuracy	AUROC	Sensitivity	Specificity	Precision
GB	0.9535	0.9891	0.9270	0.9670	0.9306
RF	0.9702	0.9916	0.9331	0.9890	0.9759
ANN	0.9367	0.9668	0.9571	0.9239	0.8744
LR	0.9086	0.9625	0.8559	0.9349	0.8761
DT	0.8772	0.9028	0.8279	0.9031	0.8108
SVM	0.9590	0.9902	0.9432	0.9657	0.9370
KNN	0.9479	0.9898	0.9958	0.9219	0.8634
BN	0.8922	0.9611	0.8702	0.9034	0.8197

Note: Each measurement was calculated under 10‐fold cross‐validation and then averaged across 10 repetitions. The highest values among the eight models are underscored.

3.3. Model uncertainty measurement

The five measurements were point estimations of the model performance (Table 3). To gain a deeper understanding of the uncertainty of the model predictions, a bootstrap analysis was employed to construct empirical distributions for the eight model performance measurements (Figure 4). ²⁰ During each step of the 10‐fold cross‐validation, we conducted resampling with replacement on the initial training dataset, creating a bootstrap training dataset. Subsequently, machine learning models were trained on this bootstrap training dataset and its prediction performance was evaluated on the test dataset. This resampling process following the previous steps was repeated 1000 times to derive the empirical distributions of prediction accuracy, AUROC, sensitivity, specificity, positive predictive and negative predictive.

TABLE 3.

The summary statistics of model performance calculated by bootstrap.

Model	Measurement	Accuracy	AUROC	Sensitivity	Specificity	Precision
GB	95% CI	(0.9061, 0.9319)	(0.9355, 0.9688)	(0.6904, 0.7647)	(0.9572, 0.9797)	(0.7997, 0.9061)
	Mean	0.9190	0.9521	0.7275	0.9684	0.8529
RF	95% CI	(0.9184, 0.9456)	(0.9491, 0.9820)	(0.6470, 0.7628)	(0.9845, 0.9975)	(0.9091, 0.9879)
	Mean	0.9320	0.9656	0.7049	0.9910	0.9485
ANN	95% CI	(0.8963, 0.9276)	(0.8896, 0.9348)	(0.6993, 0.8188)	(0.9356, 0.9695)	(0.7417, 0.8706)
	Mean	0.9120	0.9122	0.7591	0.9526	0.8062
LR	95% CI	(0.8928, 0.9194)	(0.9346, 0.9624)	(0.6472, 0.7614)	(0.9422, 0.9719)	(0.7454, 0.8666)
	Mean	0.9061	0.9485	0.7043	0.9570	0.8060
DT	95% CI	(0.8331, 0.8809)	(0.7363, 0.8257)	(0.5290, 0.6167)	(0.9047, 0.9563)	(0.5853, 0.7864)
	Mean	0.8570	0.7810	0.5728	0.9305	0.6858
SVM	95% CI	(0.9115, 0.9338)	(0.9458, 0.9773)	(0.6111, 0.7671)	(0.9719, 0.9945)	(0.8682, 0.9690)
	Mean	0.9227	0.9615	0.6891	0.9832	0.9186
KNN	95% CI	(0.9030, 0.9541)	(0.9317, 0.9541)	(0.7114, 0.8457)	(0.9461, 0.9719)	(0.7856, 0.8732)
	Mean	0.9205	0.9429	0.7785	0.9590	0.8294
BN	95% CI	(0.9031, 0.9315)	(0.9281, 0.9596)	(0.6614, 0.7881)	(0.9530, 0.9846)	(0.7996, 0.9210)
	Mean	0.9173	0.9438	0.7248	0.9688	0.8603

Note: The empirical 95% confidence intervals and means of five measurements were calculated for each model. The highest mean values among the eight models are underscored.

FIGURE 4.

The empirical distributions of model performance constructed by bootstrap. Five performance measurements (Accuracy, AUROC, Sensitivity, Specificity and Precision) of eight models were calculated on bootstrap samples. ANN, artificial neural network; AUROC, area under the receiver operating characteristic; DT, decision tree; GB, the gradient boosting classifier; KNN, K‐nearest neighbours classifier; LR, logistic regression; NB, Naive Bayesian; RF, random forests; SVM, support vector machine classifier.

A comparison between the five empirical distributions and their summary statistics is shown in Figure 5 and Table 3. The performance ranking of eight machine learning models showed a similar asymptotical pattern compared to their point estimations. Notably, the random forest demonstrated the highest average prediction accuracy, followed by the GB and SVM model. In terms of AUROC, the RF, GB and SVM model outperformed other models. Also, the RF maintained high performance in specificity and precision. The KNN showed advantages in terms of sensitivity, while relatively less satisfactorily across all five measurements.

FIGURE 5.

Receiver operating characteristic analysis of eight models' ability to predict SSI. ANN, artificial neural network; DT, decision tree; GB, the gradient boosting classifier; KNN, K‐nearest neighbours classifier; LR, logistic regression; NB, Naive Bayesian; RF, random forests; SVM, support vector machine classifier.

3.4. Variable importance analysis

An effective method for assessing each predictor's impact on the outcome is utilizing Shapley (SHAP) values to rank predictors based on their contributions to the model. ²¹ , ²² SHAP values were used to assess the impact of each variable on the model prediction. Each variable's average absolute SHAP value on the SHAP multi‐class summary plot indicated its respective average impact on model output (Figure 6). The albumin emerged as the most impactful variable, followed by subcutaneous fat thickness and BMI. In the SHAP one‐versus‐rest summary plot (Figure 7), SHAP values were calculated and indicated the predictability of the variables across various outcomes. The variables were ranked based on their distributions on predictions within each group. Every dot was colour‐coded and represented a prediction, and the colour of the dot represented the value of the variables (red: positive association, blue: negative association).

FIGURE 6.

Top 10 patient clinical feature importance. Variables are sorted from high to low based on their relative importance. BMI, body mass index; CRP, C‐reactive protein; Hb, haemoglobin; PLT, platelet; SFT, subcutaneous fat thickness; WBC, white blood cell.

FIGURE 7.

Shapley additive explanations (SHAP) plot. BMI, body mass index; CRP, C‐reactive protein; Hb, haemoglobin; PLT, platelet; SFT, subcutaneous fat thickness; WBC, white blood cell.

4. DISCUSSION

In recent years, with the development of spinal surgery technology, the number of spinal operations has increased. Postoperative SSI problems have also been concerned. Postoperative SSI can lead to prolonged hospitalization, mortality, poor outcomes and heavy burden to hospitals and society. ²³ , ²⁴ Furthermore, the incidence of SSIs is not determined by a single factor since SSIs are caused by various factors acting synergistically. Thus, a multivariate method is required for a comprehensive assessment of the SSI risk. This study used clinical data to predict SSI after posterior cervical surgery using ML algorithm. The establishment of eight models based on multiple variables and were evaluated for accuracy, AUROC, sensitivity, specificity, positive predictive and negative predictive through 10‐fold cross‐validation. Similar results were observed in both point estimation and uncertainty measurement, as shown in Tables 2 and 3. While no single model outperforms others among all measurements, RF and GB, displayed better overall diagnostic accuracy (measured by accuracy and AUROC) and were good at predicting non‐SSI after posterior cervical surgery (measured by specificity). Moreover, the GB and RF models were both tree‐based methods that aggregated results from individual trees. The GB model added each tree in sequence and allowed self‐correction from errors at each step to enhance the model, while the RF model simultaneously constructed all trees. ²⁵ , ²⁶ Consequently, the GB models often require more training time in theory, but they can provide better performance. Many studies have indicated that both GB and RF exhibited excellent performance in predictive modelling of health outcomes. ²⁷ , ²⁸ , ²⁹ , ³⁰ The KNN showed the capacity to predict SSI (measured by sensitivity). Consequently, KNN is more suitable for early SSI screening, but at the cost of false positive prediction. Notably, random forest demonstrated the highest precision both in point estimation and uncertainty measurement, making the most accurate predictions among predicted SSI cases. Thus, the KNN can be ensembled with RF or GB to enhance the model's predictive ability.

Although there is currently no academic harmonization of indicators for evaluating nutritional status, serum albumin levels are often used to identify malnutrition. ³¹ Consistent with our findings, Klein et al. found that preoperative nutritional status was a significant predictor of postoperative complications in 114 patients undergoing elective lumbar spinal fusion. ³² A retrospective cohort study of 136 spinal fusion procedures manifested that Patients with hypoalbuminaemia were more than four times more likely to develop postoperative complications than those with non‐hypoalbuminaemia. ³³ Preoperative nutritional supplementation has been shown to reduce the incidence of complications. ³⁴ , ³⁵ , ³⁶ As a result, the Serum albumin levels became the most influential characteristic in our model. We believed that nutritional supplementation might be offered to patients with hypoalbuminaemia prior to posterior cervical surgery.

BMI and subcutaneous fat thickness (SFT) have gained extensive attention in preoperative counselling and optimization. SFT is not routinely measured but BMI is an easy‐to‐perform method. Thus, BMI is often used as a surrogate measure due to its correlation with obesity and SFT. ³⁷ Previous studies have shown that BMI‐defined obesity is a statistically significant factor in SSI. ³⁸ , ³⁹ However, (BMI) is a measure of weight‐for‐height, it cannot accurately reflect localized fat distribution in the surgical area. Recent studies have shown that local SFT in spine surgery is actually a better predictor of SSI than BMI. ⁴ , ⁴⁰ Our study suggests that local SFT represented a more important and accurate marker for the development of SSI than BMI. The ratio of fat thickness to lamina‐to‐surface distance was also shown to be a significant risk.

Studies have shown that tissue growth and recovery at the incision site tissue rely on blood perfusion. ⁴¹ , ⁴² , ⁴³ We found that Hb and postoperative drainage volume were correlated with SSI after posterior cervical surgery. As for traumatic indications and neurological status, traumatic injury and spine cord injury were also found to correlate with SSI occurrence. ¹⁶ , ⁴⁴ Patients with traumatic injuries, especially those with spinal cord injuries, usually experience higher energy shocks, increasing soft tissue damage and secondary tissue hypoxia. ⁴⁵ , ⁴⁶ Moreover, blood indicators such as CRP, WBC, PLT and glucose were also significant prediction factors for SSI occurrence.

As of writing, this is the first study that aims to develop an objective predictive tool for SSI after posterior cervical surgery based on machine learning algorithms. The constructed prediction model exhibits good predictive performance, further simplifying the assessment and intervention for clinicians on SSI. When high‐risk SSI cases are identified, appropriate intervention measures can be implemented before surgery, such as elevating albumin, stabilizing blood glucose, preoperative incision skin preparation and prophylactic antibiotic use. Identifying risk factors for SSI holds significant value in conducting a comprehensive risk–benefit analysis of prophylaxis following posterior cervical surgery. The goal is to reduce morbidity, mortality, and length of stay for surgical patients and to achieve cost savings for healthcare institutions through adequate management of risk factors.

The limitations of our study should also be taken into account. First, the nature of retrospective studies can introduce selection and subjective bias. Second, our clinical data were collected from a single‐centre investigation with a single ethnic population. As a result, our model requires external validation. Finally, there are no other potential risk factors were included in our study. Additional data are necessary, which may improve the predictive validity of our models.

5. CONCLUSIONS

In conclusion, we employed machine learning to develop a dynamic model with strong predictive capabilities for SSIs, which can serve as a valuable tool for clinicians and patients to assess SSI risk and take appropriate measures to prevent it in clinical practice. We believe this dynamic prediction model can complement SSI judgement, support rational and reasonable treatment that ultimately benefits patients.

FUNDING INFORMATION

This work was supported by grants from the Wenzhou Science and Technology Planning Project (ZY2021005) and the National Natural Science Foundation of China (82202757).

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

ETHICS STATEMENT

This study was approved by the Ethics Committee at the Second Affiliated Hospital, Wenzhou Medical University. All methods were performed in accordance with the relevant guidelines and regulations. Informed consent forms are available from the corresponding author.

Lu K, Tu Y, Su S, et al. Machine learning application for prediction of surgical site infection after posterior cervical surgery. Int Wound J. 2024;21(4):e14607. doi: 10.1111/iwj.14607

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.