Convolutional neural networks using preoperative CT to predict short-term recurrence after incisional hernia repair

Abstract

Convolutional neural networks (CNNs) are deep neural networks that use convolutional layers to extract spatial features from data. The application of CNNs in assessment of postoperative outcomes of repair of incisional hernias demands further investigation. The aim of this study was to compare the ability of various CNNs to predict recurrence after incisional hernia repair. This was a single-center, retrospective study of patients who had undergone preoperative computed tomography (CT) scans prior to repair of incisional hernias. We trained three CNNs (ResNet-50, ResNet-152, and VGG16) to predict hernia recurrence after surgery. Images were batched into training versus test sets (training: test 4:1). The performance of the CNNs was assessed using conventional receiver operating characteristics curves and the areas under the curves. Gradient-weighted class activation maps were used to identify which regions were actively used in predicting hernia recurrence. The study cohort comprised 528 patients and 44,380 CT images. Among 528 patients (mean age 61.8 years, 62.9% female), 73 (13.8%) experienced recurrence at a mean follow-up of 17.3 months. The area under the curve was 0.840 (0.809–0.871) for ResNet-50; 0.743 (0.706–0.780) for ResNet-152; and 0.693 (0.654–0.732) for VGG16. ResNet-50 predicted incisional hernia recurrence better than ResNet-152 or VGG16 (P < 0.001, respectively). Image-based CNNs using routine, preoperative CT images were successful in predicting incisional hernia recurrence after repair. However, the clinical efficacies of the three CNNs varied. These differences require further clinical investigation.

Introduction

Hernia recurrence is a crucial outcome following surgical intervention¹, serving as a reliable measure of the efficacy of the procedure. The ability to accurately predict the risk of incisional hernia recurrence would have significant clinical value, enabling surgeons to make informed decisions with their patients concerning the optimal timing of and necessity for surgical intervention. Thus, development of reliable models for predicting hernia recurrence remains a critical area of research in the field of hernia surgery. The limitations of traditionally used models for predicting and analyzing risk factors for dichotomous medical outcomes, such as logistic regression (LR), have been increasingly recognized^2,3. LR assumes a linear relationship between combinations of independent variables and the dependent variable, as well as a normal distribution of the independent variables. These assumptions restrict its flexibility in capturing complex relationships, which is particularly relevant when modeling complex systems, such as surgical outcomes, where accurate prediction is essential. It is therefore necessary to explore alternative methods that can better capture complex relationships and improve the accuracy of predictions.

Convolutional neural networks (CNNs), a type of nonlinear model, are more flexible than LR models, enabling assessment of more complex relationships^4,5. CNNs consist of processing units that communicate through an interconnected network in a way that resembles the way humans make decisions. Training CNNs typically involves using a back-propagation algorithm to adjust the mathematical equations encoded in the CNNs. This is accomplished by feeding the CNNs a training set containing samples of input and output data, and comparing the CNNs’ output with human-provided output. If the CNN’s prediction is incorrect, the algorithm learns through a feedback process to adjust the equations and produce more accurate predictions. Once a CNN has been trained, it can predict the output for new input data. CNNs are reportedly useful in both the diagnosis and prediction of outcomes of various medical conditions^6–8. The use of CNNs to predict medical outcomes has the potential to assist clinicians in making informed decisions, improving clinical outcomes, and reducing treatment costs. However, to our knowledge, CNNs have not yet been utilized to predict recurrence after incisional hernia repair. Therefore, this study aimed to evaluate and compare the performance of three CNN architectures ResNet-50, ResNet-152, and VGG16 in predicting incisional hernia recurrence from preoperative CT scans.

Methods

Study design

In this study, after obtaining approval from the Institutional Review Board of Beijing Chaoyang Hospital, we retrospectively examined data of patients who had undergone incisional hernia repair between 2016 and 2019 in this tertiary care institution. Eligibility for inclusion required fulfillment of several criteria: (1) patients with a history of previous abdominal surgery and a confirmed diagnosis of incisional hernia via surgical examination; (2) complete medical records were obtained, including demographic information, preoperative abdominal computed tomography (CT) scans of the whole hernia defect (Fig. 1), and surgical records; (3) complete postoperative follow-ups were achieved. Exclusion criteria precluded: (1) patients with recurrent incisional hernia; (2) patients deceased during the follow-up period. The minimum follow-up duration to classify an incisional hernia recurrence was 6 months. Despite the retrospective design, post hoc evaluations verified adequate statistical strength of sample size. Using preoperative CT images, three independent CNNs were trained to predict hernia recurrence. Axial cuts were deidentified and rendered into representative 5-mm slices, images containing only the hernia being included for analysis. Multiple CT images per patient were aggregated for analysis to avoid slice-level bias. All images were deidentified, batched, and stored in a secure folder for training and validation of the CNN models.

Fig. 1.

Representative axial computed tomography images of a incisional hernia patient. (A)–(D) are 4 representative computed tomography cuts in order from cranial to caudal.

CNNs: incisional hernia recurrence

A binary outcome was identified for each CNN and associated with the image set for each patient. Recurrence of incisional hernia after surgery was identified through both clinical examination and CT scan⁹. A palpable mass was identified in the area of the previous surgery by clinical examination and confirmed by CT scanning.

Model architecture, training, and validation

Dataset preparation and splitting

The study utilized preoperative axial computed tomography (CT) scans from 528 patients who underwent incisional hernia repair. A total of 44,380 CT images were processed for the analysis. The dataset was split into training and test sets using 4:1 ratio. This resulted in training set: 35,505 images, test set: 8875 images. The models were completely blinded to the test set images during the entire training phase; This set was reserved solely for final internal validation.

Image preprocessing

Axial CT slices were de-identified to protect patient privacy. Only images containing the hernia defect were included for analysis. All images were standardized to a resolution of 224 × 224 pixels. Images were processed with three color channels (RGB format), likely by replicating the single-channel grayscale CT data across all three channels to match the input requirements of the pre-trained CNN architectures.

Model architectures

Three distinct Convolutional Neural Networks ResNet-50, ResNet-152 and VGG16 were selected for training and comparison. The models were initialized with weights pre-trained on the ImageNet dataset.

Training protocol

The models were trained for a fixed number of 100 epochs. The training process involved feeding batches of the preprocessed training images into the networks. The networks learned to map the input CT images to a binary outcome: recurrence or non-recurrence after incisional hernia repair. For each individual CT slice analyzed, the trained model generated a continuous output value between 0 and 1, representing the predicted probability of recurrence (values closer to 1 indicating a higher likelihood of recurrence). To obtain a final prediction per patient, the predictions from the multiple CT slices associated with that specific patient were aggregated.

Validation and evaluation

After the 100-epoch training period, the models underwent internal validation using the held-out test set (8,875 images). No recalibration of the models was performed during this validation phase. Model performance was primarily evaluated using Receiver Operating Characteristic (ROC) curves and the corresponding Area Under the Curve (AUC) metrics calculated on the test set predictions. The ROC curves were used to assess how well the model distinguished between positive and negative samples. The AUC is a single scalar value that summarizes the performance of the ROC curve. It represents the probability that the model ranks a randomly selected positive sample higher than a randomly selected negative sample. The AUC ranges from 0 to 1, where a higher value indicates better discriminative ability of the model. By utilizing ROC curves and AUCs, we obtained a more comprehensive measure of the models’ performance, enabling us to make comparisons across different models. Classification accuracy and loss metrics were also tracked throughout the training process.

Statistical analysis

To determine the prediction model parameters for the CNNs, the predictions for each image analyzed by the CNNs were compared against the actual outcomes. Predictions were classified as binary outcomes, with a particular probability of 50% or more in favor of an outcome considered to predict the designated outcome. The predictions for each patient image were then pooled for an overall patient prediction. ROC values for the CNNs were generated using Python 3.7. The threshold for statistical significance was defined as a P < 0.05.

Results

Patient characteristics

The study cohort comprised 528 patients who had undergone incisional hernia repair and 44,380 images. Hernias had recurred in 73 patients (13.8%), the mean follow-up time being 17.3 months. The patients’ mean age was 61.8 (SD 13.5) years; 196 (37.1%) were male and 332 (62.9%) were female. Open surgery was performed on 244 patients (46.2%) and laparoscopic surgery on 284 (53.8%). Table 1 shows more comprehensive patient characteristics.

Table 1.

Patient demographics and clinical characteristics.

Variable	Categories	All patients (n = 528)
		n	%
Age	17–59	194	36.7
	60–79	300	56.9
	> 80	34	6.4
Sex	Male	196	37.1
	Female	332	62.9
Open versus laparoscopic	Open	244	46.2
	Laparoscopic	284	53.8
BMI	< 25.0	206	39.0
	> 25.0	322	61.0
Smoking status	Current smoker	102	19.3
	Nonsmoker	426	80.7
Preoperative conditions	Hypertension	207	39.2
	Diabetes	60	11.4
	Cardiovascular disease	69	13.1
	History of malignancy	129	24.4
Hernia location	Central	198	37.5
	Lateral	330	62.5
Mesh type	Synthetic	505	95.6
	Biologic	23	4.4
Intraoperative complication	No	506	95.8
	Yes	22	4.2

CNN models’ performance

There were 28,984 nonrecurrence images and 6521 recurrence images in the training set and 7245 nonrecurrence images and 1630 recurrence images in the test set. Figure 2 presents the accuracy and loss graphs in the training process for ResNet-50, ResNet-152, and VGG16. The model performances for the three CNNs are shown in Fig. 3. ResNet-50 had an AUC of 0.840 (0.809–0.871), ResNet-152 an AUC of 0.743 (0.706–0.780), and VGG16 an AUC of 0.693 (0.654–0.732). ResNet-50 performed better than ResNet-152 and VGG16 in predicting recurrence of incisional hernias (P < 0.001 respectively). The outcomes achieved by the three CNNs are compared in Table 2.

Fig. 2.

Accuracy and loss curve obtained by three CNNs.

Fig. 3.

Receiver operating characteristic curves. ResNet-50 performed with an ROC of 0.840 (0.809–0.871), ResNet-152 performed with an ROC of 0.743 (0.706–0.780), and VGG16 performed with an ROC of 0.693 (0.654–0.732). ResNet-50 performed significantly better than ResNet-152 and VGG16 in prediction of incisional hernia recurrence (P < 0.001 respectively).

Table 2.

CNN outcomes for predicting incisional hernia recurrence.

CNNs	AUC (95% CI)	Sensitivity (%, 95% CI)	Specificity (%, 95% CI)	P value
ResNet50	0.840 (0.809–0.871)	85.2 (77.1–91.4)	68.1 (58.6–76.6)	Ref
ResNet152	0.743 (0.706–0.780)	86.5 (78.6–92.3)	65.3 (55.7–74.0)	< 0.001
VGG16	0.693 (0.654–0.732)	67.3 (57.4–76.2)	62.1 (52.4–71.1)	< 0.001

Gradient-weighted class activation map (Grad-CAM)

Deep learning architectures have the unique ability to automatically extract features during the classification process. However, identification of specific features being extracted from the dataset remains challenging. Nevertheless, during classification with deep learning architectures use of certain tools can help in locating regions that are actively used in predicting the relevant class. One such tool is Grad-CAM, which enables a classification-trained machine learning model to localize class-specific image regions by visualizing the pixels of interest and color-coding them according to their importance to the output^10,11. Darker shades denote the most distinctive features of the relevant class, whereas lighter colors denote less distinctive regions. In this study, we conducted Grad-CAM analysis for patients with and without recurrence. Figure 4 provides examples of Grad-CAM images obtained for each group.

Fig. 4.

Grad-CAM images of recurrence and non-recurrence cases. (A) Original computed tomography images. (B) Grad-CAM images.

Discussion

This study presents a proof-of-concept investigation in the field of prediction of recurrence after incisional hernia repair. To our knowledge, it is the first study to investigate the use of CNNs for predicting recurrence of incisional hernias using routinely available, objective, preoperative CT images. These findings serve as a proof of concept for future research aimed at improving models for objective risk stratification based on preoperative imaging. The ability to predict surgical outcomes using widely available imaging, rather than relying solely on the surgeon’s judgment, may lead to improved preoperative planning, surgical decision-making, and ultimately, improved patient outcomes.

CNNs have several advantages over traditional logistic regression. For instance, CNNs implicitly detects any complex nonlinear relationships between independent and dependent variables and identifies all possible interactions between predictor variables. However, CNNs have some drawbacks. They are considered a “black box” in terms of analyzing them, in that they do not yield interpretable odds ratios that would determine the direction and magnitude of the effect of each variable on the outcome^12,13. Additionally, CNNs may overfit the data, which limits their generalizability to external data¹⁴. It is also challenging to optimize the parameters of CNNs models, such as the training time and number of layers, to ensure optimal performance in predicting the outcomes¹⁵.

Risk factors for recurrence after incisional hernia repair have been examined in several previous studies. Henriksen et al.¹⁶ compared outcomes of incisional laparoscopic versus open hernia repair and found that patients were significantly less likely to require repair of recurrence after laparoscopic than after open repair of 2–6 cm-wide defects. Large abdominal wall defects have traditionally been managed by performing Rives-Stoppa retromuscular repair to achieve adequate coverage. Bueno-Lledo et al.¹⁷ found that using immunosuppressive drugs, development of a postoperative hematoma, and requiring surgery for such a hematoma were associated with increased risk of hernia recurrence after Rives-Stoppa retromuscular repair. In a study of an international prospective cohort by the French Society of Surgery, Romain et al.¹⁸ found that history of hernia, lateral hernia, concomitant digestive tract surgery, development of surgical site complications, and not using mesh were strong independent risk factors for recurrence. A previous review and meta-analysis of published reports on open hernia repair found that recurrence rates varied between different locations of mesh. The pooled recurrence rates were 16.5% (0–36%) for onlay, 30.2% (0–80%) for inlay, 7% (0–48%) for sublay, and 14.7% (0–56%) for underlay¹⁹. These findings suggest that risk factors for incisional hernia recurrence are complex and should be reduced preoperatively where possible to minimize postoperative recurrences.

Integrating CNN-based recurrence predictions into preoperative planning may also guide mesh selection, surgical strategy, and patient counseling. For high-risk patients, surgeons could choose larger and higher-strength synthetic mesh with surgical technique which they are most proficient to minimize recurrence and improve long-term clinical outcomes. Identifying patients who should undergo incisional hernia repair in a community hospital or specialized hernia referral centers has become a critical focus of evolving and specialized hernia management²⁰. CNNs could be a valuable tool in this field. If accurate, they could efficiently and objectively identify patients with high surgical complexity or at high risk of complications, contributing to creation of an objective incisional hernia repair tiering system that ensures patients receive optimal care.

Our study demonstrated ResNet-50 performed significantly better than ResNet-152 and VGG16, attributed to its residual connections that mitigate gradient vanishing and enhance feature propagation, unlike VGG’s sequential convolutional stacks²¹. While deeper architectures like ResNet could extract more complex features, our limited dataset made ResNet-50’s balanced parameter efficiency and inference capability optimal. Therefore, model selection should consider both architectural advantages and dataset constraints.

There are several limitations to consider when evaluating our research. First and most importantly, incisional hernia recurrence is a multifactorial outcome determined by a complex interplay of patient biology (comorbidities, tissue quality), hernia morphology (size, location, loss of domain), and crucially, surgical techniques (mesh type, placement, fixation). Our CNN models were intentionally designed to analyze only preoperative CT images and did not incorporate any of these pivotal clinical or surgical variables. This represents a significant simplification. Therefore, our models predict incisional hernia recurrence risk based solely on the preoperative anatomical substrate. Second, the generalizability of our results may be compromised by the fact that this was a single-institution study. The relatively small dataset (528 patients, 44,380 CT images) posed an intrinsic risk of overfitting due to insufficient training samples, particularly for CNN architectures. This study also lacked external validation, the performance observed in our single-center cohort may not fully translate to other institutions with different patient demographics, CT imaging protocols, or surgical practices. Furthermore, there are inherent limitations within the means of deep learning by artificial intelligence, including the inability to interpret the algorithm’s assessment of image features, which is a fundamental characteristic of all learning-based algorithms.

Despite these limitations, we consider this study an important first step in evaluating the potential of CNNs to predict recurrence after incisional hernia surgery. Although CNNs cannot yet substitute for physicians in the decision-making process, they may serve as a valuable adjunctive tool for predicting surgical outcomes. Future researches should bridge the gap between imaging biomarkers and clinical reality. The logical and necessary next step is to develop multimodal predictive models that integrate features extracted by CNNs from preoperative images with structured clinical and operative data (surgical approach details, mesh characteristics, patient comorbidities). If validated, models derived from this approach could help stratify patients, potentially guiding decisions on surgical approach, mesh selection, and the appropriateness of referral to specialized hernia centers. It is also necessary to conduct researches with multi-center, large-sample-size data to enhance model performance.

Conclusion

The present study provided evidence that CNNs developed using only preoperative imaging inputs may predict hernia recurrence in patients who have undergone incisional hernia repair. This study’s findings also provide a foundation for future research on image-based CNNs algorithms and their relationship with surgical outcomes. Additionally, they indicate that it is crucial to select appropriate CNN models for predicting surgical outcomes. Future multicenter studies with larger datasets are needed to validate the generalizability of CNN-based recurrence prediction and to support its integration into clinical practice.

Author contributions

YC L and MG W conceived of and designed the study and the statistical analysis. XW X wrote the article and followed the patients up. YC L participated in revision and review of the article. B Z accomplished model training and testing. MG W and YC L contributed equally to this article.

Funding

This study was supported by Beijing Municipal Administration of Hospitals Incubating Program (PG2023007) and the Beijing Natural Science Foundation (No. 7222071).

Data availability

The data presented in this study will be available upon request to the corresponding author.

Code availability

To facilitate reproducibility and transparency, the complete training and validation code will be made publicly available as supplementary material or through an online repository in accordance with the journal’s data and code availability policies.

Declarations

Competing interests

The authors declare no competing interests.

Ethical approval

This retrospective study was approved by Beijing Chaoyang Hospital Institutional Review Board (IRB). The informed consent was waived by Beijing Chaoyang Hospital Institutional Review Board due to the retrospective nature of the study.

Human and animal rights

This study did not involve animals. All procedures involving human participants were performed in accordance with the ethical standards of the Institutional Research Committee and with the 1964 Helsinki declaration and its later amendments or, comparable ethical standards.