Automated CT image prescription of the gallbladder using deep learning: Development, evaluation, and health promotion

Abstract

Aim

Most previous research on AI‐based image diagnosis of acute cholecystitis (AC) has utilized ultrasound images. While these studies have shown promising outcomes, the results were based on still images captured by physicians, introducing inevitable selection bias. This study aims to develop a fully automated system for precise gallbladder detection among various abdominal structures, aiding clinicians in the rapid assessment of AC requiring cholecystectomy.

Methods

The dataset comprised images from 250 AC patients and 270 control participants. The VGG‐16 architecture was employed for gallbladder recognition. Post‐processing techniques such as the flood fill algorithm and centroid calculation were integrated into the model. U‐Net was utilized for segmentation and features extraction. All models were combined to develop a fully automated AC detection system.

Results

The gallbladder identification accuracy among various abdominal organs was 95.3%, with the model effectively filtering out CT images lacking a gallbladder. In diagnosing AC, the model was tested on 120 cases, achieving an accuracy of 92.5%, sensitivity of 90.4%, and specificity of 94.1%. After integrating all components, the ensemble model achieved an overall accuracy of 86.7%. The automated process required 0.029 seconds of computation time per CT slice and 3.59 seconds per complete CT set.

Conclusions

The proposed system achieves promising performance in the automatic detection and diagnosis of gallbladder conditions in patients requiring cholecystectomy, with robust accuracy and computational efficiency. With further clinical validation, this computer‐assisted system could serve as an auxiliary tool in identifying patients requiring emergency surgery.

The proposed system achieves state‐of‐the‐art performance in the automatic detection and diagnosis of gallbladder conditions in patients requiring cholecystectomy, with robust accuracy and computational efficiency, such as (A) a patient with acute cholecystitis and gallstones and (B) a healthy participant.

INTRODUCTION

Cholecystitis, or inflammation of the gallbladder, is most commonly caused by bile duct obstruction from gallstones but can also result from biliary sludge or gallbladder polyps. Acute cholecystitis (AC), occurring in 95% of cases, often necessitates cholecystectomy. ¹ , ² , ³ Ultrasound is frequently the first‐line imaging test due to its advantages such as relatively low cost, real‐time capability, and portability. ⁴ , ⁵ Although the timing is still being debated, an early cholecystectomy within 24 h of hospital admission is preferred based on a systematic review to prevent devastating complication. ⁶ This argues for an immediate diagnosis of AC, which requires physicians' skill in interpreting ultrasound images. ⁷ Different from cholelithiasis, ultrasound may not be the best imaging tool for AC. ⁸ Previous studies had pointed out poor sensitivities of ultrasound, and in all of the misdiagnosed cases, the contributing factor was a failure to identify gallbladder wall thickening. ⁹ , ¹⁰ , ¹¹ The diagnosis of AC is usually achieved by evaluating imaging features, and the results may vary depending on the ultrasound image reader. Many cases are still reported as equivocal on preoperative imaging. Therefore, an objective analytical technique for the discrimination of GB diseases is required.

Medical image analysis, including ultrasound and CT, revolves around three core tasks: classification, detection, and segmentation. Artificial intelligence (AI) has significantly improved diagnostic accuracy in various gastrointestinal conditions by addressing operator dependency and subtle differences. CT has become more popular for AI analysis in evaluating gallbladder diseases. ¹² , ¹³ It has been reported that distention, mural striation, and decreased gallbladder wall enhancement can significantly distinguish complicated from noncomplicated AC. ¹⁴ According to the literature, deep learning models have achieved a 90.8% accuracy rate in segmenting gallstones from CT images, while ultrasound‐based models demonstrate an average precision of 79.81% for gallstone segmentation. ¹⁵ However, challenges remain, including the labor‐intensive manual delineation of gallbladder size and inherent selection bias in test images, which often require manual pre‐selection before applying deep learning. Many previous studies used selected representative ultrasound or CT images for deep learning, but these approaches were not fully automatic AI image analyses, as they still required human intervention to select the images. This operator dependency could lead to exaggerated or missed findings. To address this, AI systems should aim for fully automated detection of gallbladder target areas. Developing an algorithm to automatically select approximately 20 relevant images from a set of 120 abdominal CT scans is crucial for enabling autonomous AI diagnosis of gallbladder diseases.

Improved preoperative diagnoses and timely surgical interventions allow for earlier treatment of complicated cholecystitis. In this study, our primary objective is to demonstrate the effectiveness of a deep learning‐based CT‐AI algorithm for the precise automatic classification and detection of the gallbladder among various abdominal structures. By enhancing image feature recognition, we aim to develop automated systems capable of processing large volumes of CT data, quickly and accurately identifying areas of concern to improve the clinical utility of medical imaging in AC diagnosis. Our secondary goal is to apply deep learning for the semantic segmentation of AC in CT images. We propose using a convolutional neural network (CNN) to differentiate between severe AC cases requiring surgery and healthy conditions in CT scans, with the diagnostic performance of the CNN being compared to interpretations by experienced physicians. Ultimately, we aim to establish a fully automated system that assists and educates clinicians in rapidly assessing severe AC cases requiring surgery, improving surgical accuracy, facilitating clinical decision‐making, and promoting health.

MATERIALS AND METHODS

Clinical metrics and CT annotation

The study involved patients who presented to the ER and underwent abdominal CT scans due to a diagnosis of AC, followed by laparoscopic cholecystectomy within 24 h, with pathology confirming the diagnosis. From January 2015 to January 2020, a total of consecutive 250 patients, aged between 20 and 80 years, were collected as the AC group. Additionally, 250 patients with acute appendicitis, who had a clearly visible and healthy gallbladder on their CT scans, and 20 patients without AC or appendicitis were included as a control group. The CT images, consisting of 1‐mm contiguous sections, were obtained using a Philips Brilliance 256‐slice CT scanner (Best, The Netherlands). Furthermore, we recruited an additional 40 cases (20 with AC and 20 healthy controls) from another institution, a branch of our hospital located in a different city, using a different type of CT machine (Philips Brilliance 64‐slice CT scanner) as a source of external validation. All images were independently reviewed for accuracy by three board‐certified experts (two general surgeons and one radiologist). Any cases flagged as suspicious by any of these three experts were excluded from the study. Approximately 28% of these CT images were annotated for CNN training, with each gallbladder manually outlined and labeled by the same board‐certified radiologist. The remaining CT images were reserved for validation and testing. A flowchart of the patient selection process is shown in Figure S1.

Detection of the gallbladder region

Our first goal is to perform initial filtering of these CT images and train a model to automatically locate the gallbladder among the various abdominal organs. Each case provides around 90–110 CT images, and we collected a total of 27,437 CT images from the AC group and 29,614 images from the control group. Initially, we manually selected 4324 high‐quality CT images from the AC group (n = 40) and 4387 from the control group (n = 40). These images were then used to train a neural network to accurately identify CT images that contain the gallbladder. The remaining images were set aside for validation.

We utilized the VGG‐16 model architecture for gallbladder image recognition. When the model predicts potential gallbladder structures in the mask, we apply the Flood Fill algorithm to fill in different sections of the mask and obtain the coordinates of each pixel within these sections. We then developed a model capable of segmenting the gallbladder region, incorporating additional bilateral filtering for noise reduction during image preprocessing. However, some erroneous segmentations still appeared in non‐gallbladder areas. To address this, we designed a post‐processing workflow that integrates the continuity of CT images with the model‐predicted masks to filter out non‐gallbladder regions. In this process, we calculate the center point of each contour. To determine which masks represent the gallbladder, we leverage the fact that gallbladder images should have continuity across slices. Groups with more centroids should correspond to the gallbladder segmentation region. Finally, we trace back to the corresponding masks from the centroids, resulting in a complete set of gallbladder segmentation masks. After iterating this process, this training method demonstrated substantial improvements over state‐of‐the‐art methods.

Semantic segmentation and feature extraction

We further processed the masks obtained from the VGG‐16 model by converting the images to grayscale and applying normalization. We also applied dilation to the mask regions in order to expand the surrounding area of the gallbladder, which enhances the model's ability to recognize features indicative of acute cholecystitis. To denoise the images, we used a bilateral filter, which preserves the edges of different organs while reducing noise. Subsequently, we employed the U‐Net architecture as the basis for our segmentation model. We used the model to learn features associated with AC compared to healthy control groups, including gallbladder wall thickening, gallstones, and pericholecystic fluid. For training, we used the VGG‐16 model. However, even within the same AC case, the images might include both inflamed and normal gallbladder images, leading to issues in defining the training dataset. Images lacking clear signs of AC were removed from the dataset to enhance the model's ability to learn relevant features.

Finally, we integrated all the aforementioned methods and models sequentially to develop an automated AC recognition system. In this system, the images first undergo grayscale conversion, normalization, and resizing to 256 × 256 pixels before entering the gallbladder recognition model, which identifies images containing the gallbladder. Once the images containing the gallbladder are identified, we process them using a bilateral filter, resize them to 512 × 512 pixels, and input them into the gallbladder segmentation model to extract the gallbladder contour masks. We extract a 256 × 256 pixel area centered on the mask's centroid and feed this into the AC recognition model. The model then outputs the probability that each image contains features of AC. If any image in a series is found to contain such features, the entire case is classified as positive for AC. The overall image processing method we used can be summarized in Figure 1. Details on those processing are provided in Appendix S1.

FIGURE 1.

System architecture of the proposed study.

Statistical analysis

In order to analyze the predicted result in more detail, the confusion matrix was employed to analyze the predicted and actual results for gallbladder identification, segmentation, and AC diagnosis. The confusion matrix provided a detailed breakdown of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) for each group. The accuracy was defined as: (TP + TN)/(TP + TN + FP + FN), while the recall was defined as TP/(TP + FN) and the specificity was defined as TN/(TN + FP).

Ethical considerations

The research protocol (NO: B202305161) was reviewed and approved by the Institutional Review Board.

RESULTS

Gallbladder organ image recognition

In addition to the cases used for training (40 cases each from the AC and control groups), the remaining patients (210 cases from the AC group and 230 from the control group) were utilized for validation in gallbladder identification. Our initial recognition results yielded average performance, with an accuracy of 90.9% and a recall of 90.2%. The worst‐case scenario demonstrated an accuracy of 87.9% and a recall of only 44.4%, while the best case achieved an accuracy of 96.4% and a recall of 100%. Upon further analysis, it was found that in cases with poorer outcomes, the gallbladder was embedded within the liver parenchyma, whereas in better‐performing cases, the gallbladder was adjacent to the liver. Examination of our dataset revealed that most CT images featured the gallbladder next to the liver. However, in a subset of patients, the gallbladder's location relative to the liver is atypical, and these uncommon anatomical variations are likely responsible for the misclassification errors by the VGG16 model. Given the 10% misidentification rate of the gallbladder in the initial phase, this could negatively impact the performance of subsequent segmentation models and complicate image filtering tasks. To address this, we incorporated a bilateral filter for image preprocessing and utilized centroid positions and centroid continuity as features to refine image filtering (Figure 2). Following these adjustments, we observed an increase in accuracy to 95.3% and a recall of 91.2%, significantly improving over the VGG16‐only model, effectively filtering out CT images without a gallbladder.

FIGURE 2.

Initial gallbladder recognition results from CT Images among various abdominal Organs. The initial segmentation results yielded moderate performance (A), with an accuracy of 90.9% and a recall of 90.2% (B). The U‐Net model failed to fully exclude non‐gallbladder regions, prompting the integration of centroid continuity (C) to enhance post‐processing and image refinement (D). These modifications resulted in an increased accuracy of 95.3%.

Gallbladder image segmentation

Approximately 28% of the CT images (70 from each of the AC and control groups) were annotated for CNN training. The remaining images (180 cases in the AC group and 200 in the control group) were reserved for validation and feature learning. Initial observations indicated that while the U‐Net model did not completely remove non‐gallbladder regions, the predicted masks still included non‐gallbladder objects with shapes similar to the gallbladder, indicating the need for further model refinement. We preprocessed the images using a bilateral filter to reduce noise while preserving the gallbladder edges. Subsequently, by leveraging centroid continuity to determine the gallbladder's location, we removed non‐gallbladder segments. A comparison of our new segmentation results with the ground truth using Intersection over Union (IoU) values showed that the U‐Net alone achieved an IoU of 66.95%, while the U‐Net with preprocessing filter improved to 72.71%. Across three commonly used segmentation metrics, the U‐Net with filtering consistently outperformed the standalone U‐Net model (Table 1).

TABLE 1.

Diagnostic performance of individual and ensemble models.

Model	Dice coefficient (%)	Precision (%)	IoU (%)
U‐Net	79.87	69.84	66.95
U‐Net with bilateral filter	83.99	79.28	72.71

Note: The U‐Net with post‐filtering consistently outperformed the standalone U‐Net model across all metrics.

Abbreviation: IoU, Intersection over Union.

Feature learning

We utilized VGG16 for transfer learning, and the training dataset included 708 images of gallbladders with AC (n = 45) and 866 images of normal gallbladders (n = 55) (Figure 3). During testing, the model achieved an accuracy of 83.9% and a recall of 77.1%. This indicates that the model could capture relevant image features but exhibited suboptimal accuracy. The likely cause was the labeling approach during the creation of the training dataset. Initially, we labeled all 15 images in each acute cholecystitis case as indicative of the condition, even though not all images showed acute cholecystitis features, leading to inconsistencies and undermining the test results' reliability. After revising the dataset by removing images lacking AC characteristics, we retrained our model on 198 AC cases and 202 control cases, which improved the model's ability to learn AC‐specific features. Following these changes, the model was tested on 120 cases, achieving an accuracy of 92.5% and a sensitivity of 0.904, with a calculated specificity of 0.941. A receiver operating characteristic (ROC) curve analysis was conducted to evaluate model performance and determine the optimal threshold. By setting the threshold between 0 and 1, with intervals of 0.05, and analyzing the resulting confusion matrices, the area under the curve (AUC) was found to be 0.97 (Figure 4). The true positive rate was approximately 0.9, and the false positive rate was below 0.1. These results indicate that the model is highly effective for clinical decision support, offering clinicians reliable indications of suspected acute cholecystitis and assisting in further medical image evaluations.

FIGURE 3.

Representative CT Images utilized in the analysis. The images include a healthy participant (A) and cases of acute cholecystitis characterized by gallbladder wall thickening (B), gallstones (C), and pericholecystic fluid accumulation (D).

FIGURE 4.

(A) Receiver Operating Characteristic (ROC) Curves for the CNN performance. The area under the ROC curve (AUC) was 0.97. (B) The ensemble model demonstrated strong diagnostic performance for acute cholecystitis, achieving an accuracy of 92.5%.

Automated system for AC recognition

We integrated the previously mentioned models (gallbladder recognition, segmentation, and feature extraction for AC) to develop a fully automated recognition system (Figure 5). In a test set of 120 cases, the system achieved an overall accuracy of 86.7%, with an average processing time of 3.59 s per case and a maximum time of under 6 s (Table 2). The system's overall accuracy reached 86.7%, with a sensitivity of 86.5% and specificity of 86.8%. Compared to the accuracy of 92.5% achieved in the previous step, there was a decrease of approximately 5%, attributed to differences between the manually labeled training data and the automatically segmented regions. Using an additional 40 cases from another institution (a branch of our hospital located in a different city) for external validation, the true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) were 17, 18, 2, and 3, respectively, resulting in an accuracy of 87.5%, with a sensitivity of 85% and specificity of 90%. Ultimately, our ensemble model, which integrates two CNN architectures, demonstrates strong capabilities in distinguishing the gallbladder from other abdominal structures and facilitating accurate recognition of acute cholecystitis features.

FIGURE 5.

Experimental outcomes from the automated detection system for (A) a patient with acute cholecystitis and gallstones, and (B) a healthy participant.

TABLE 2.

Diagnostic performances of the ensemble model.

Case numbers	Average processing time	Accuracy (%)	Sensitivity (%)	Specificity (%)
120	3.59 s (2.77 s~5.73 s)	86.7	86.5	86.8

Note: The TP, TN, FP, and FN are 45, 59, 9, and 7, respectively.

DISCUSSION

To date, there have been few studies that have utilized fully automated AI to analyze abdominal CT images for diagnosing gallbladder disease. While many studies have applied AI to image diagnosis in cases of acute abdominal disease, the majority have focused on analyzing ultrasound images. However, these studies require human intervention to perform the ultrasound and manually select test images. ¹⁶ , ¹⁷ , ¹⁸ Operator variability remains a significant concern, as different physicians may produce varying images. Consequently, ultrasound‐based studies have limitations, including the use of intentionally selected still images, which introduce selection bias. In contrast, CT imaging offers a more standardized protocol for acquiring test images, and the results are generally not dependent on the operator. Therefore, for achieving fully automatic AI detection of specific diseases, CT imaging may be a more suitable choice than ultrasound.

Detection and classification are fundamental for achieving fully automated AI diagnosis in clinical practice. The novelty of our work lies in the development of an algorithm capable of processing large volumes of CT data to achieve precise, fully automatic classification and detection of the gallbladder among various abdominal structures. Current mainstream object detection algorithms can be divided into one‐stage and two‐stage detection algorithms. One‐stage detection algorithms directly generate the category probabilities and positional coordinates of objects, ¹² while two‐stage detection algorithms first generate candidate regions (region proposals) and then refine the classification of these regions, as seen in models like R‐CNN. ¹⁹ Our study results align with previous literature, indicating that two‐stage detection algorithms excel in accuracy, while one‐stage detection algorithms offer better time efficiency. In our research on gallbladder detection (the first stage), the model successfully identified the gallbladder region, though some erroneous segmentations occurred in non‐gallbladder areas. To address this, we processed the gallbladder mask by taking the intersection of contours from every two consecutive images, with overlapping regions suggesting a lower likelihood of noise. We then calculated the centroid of each contour and used it as a seed point for the Flood Fill algorithm. A modified region‐growing algorithm was applied to control the seed region's growth rate, with tailored growing and termination conditions. By calculating the centroid positions of the segmented masks, these centroids were printed in sequence based on image order. We traced these centroids both forward and backward from the middle image, grouping the points and eliminating erroneous segmentations. As a result, our automated gallbladder detection method achieved performance comparable to radiologists' inter‐reader reproducibility. This method has the potential to enable automated, efficient, and reproducible segmentation (the second stage) for abdominal CT imaging. Additionally, our proposed method's gallbladder misclassification rate (4.7%) is comparable to a recent study using a modified YOLOv3 network trained with 5986 abdominal CT images (~3%). ¹² Our method may be more robust than previous methods in terms of testing performance.

AC is characterized by specific CT scan findings, including irregular thickening and poor contrast enhancement of the gallbladder wall, increased density of surrounding fatty tissue, gas within the gallbladder lumen or wall, membranous structures inside the lumen, and perigallbladder abscesses. Chang et al. identified key indicators for distinguishing AC from healthy individuals, highlighting a short‐axis diameter exceeding 4.0 cm, mural striation, and reduced gallbladder wall enhancement. ¹⁴ In the present study, we collected over 27,437 CT images from 250 AC patients and employed a deep learning model based on the VGG16 architecture for AC recognition, which automatically marked AC locations with 95.3% accuracy, outperforming state‐of‐the‐art methods. Additionally, our model reduced the misdiagnosis rate of negative samples in object detection. While the sensitivity (90.4%) and specificity (94.1%) of our model are slightly below those of surgeon‐performed ultrasound (with pooled sensitivities of 96% and specificities of 99%), ²⁰ our approach offers a significant advantage: it is fully automated, eliminating the need for manual intervention, which is commonly required in ultrasound imaging. Our algorithm segments a CT slice in approximately 0.029 s, and the entire process of segmenting a typical CT scan (about 90–120 slices) and detecting AC takes an average of 3.59 s, demonstrating its efficiency over previous methods. Furthermore, our model improves the Dice coefficient to 83.99%, offering high diagnostic accuracy, reduced time requirements, and lower diagnostic costs, laying the foundation for automatic detection of gallstone types through feature extraction. Importantly, the images used in our study were not pre‐cropped to include only the gallbladder, yet we still achieved satisfactory performance in AC detection. The performance of our model in gallstone detection was notably higher than that reported in a previous study using a different dataset, which achieved an accuracy of 86.7%. ¹⁵ By improving diagnostic accuracy, our model can serve as an auxiliary tool in identifying AC features, potentially shortening patients' lengths of stay in emergency departments and allowing for earlier cholecystectomy.

In summary, our proposed new method offers three key advantages: (1) It automatically sets parameters, reducing the workload for physicians. (2) It assists and educates physicians in accurately assessing the relative position between the gallbladder and other abdominal structures. (3) It provides contour information for the gallbladder and gallstones, enabling feature extraction and the diagnosis of cholecystitis. However, our method also has several limitations. First, only patients who had undergone cholecystectomy were included in the study, which may have introduced selection bias. We excluded patients with suspected or chronic cholecystitis, who do not require surgery, even though their CT image features could offer valuable insights for clinical experts. Studying these features requires a substantial number of cases, particularly those involving chronic cholecystitis, which are currently limited in clinical practice due to the infrequency of CT imaging in such patients. Second, as a retrospective study, this research does not fully reflect a real clinical setting. Future prospective studies, including patients who have not undergone cholecystectomy, are needed to further validate our ensemble model. Nevertheless, the presence of variability in the image data is beneficial for creating a more generalizable CNN. Beyond this preliminary study, we aim to develop a clinically applicable CNN by utilizing DICOM data, incorporating clinical information such as WBC counts and CRP levels (Large Multimodal Models, LMM)—which showed significant differences in this study—and using images from various CT systems. Despite its limitations, this study combines the realities of clinical practice with rigorous scientific analysis. Future work should focus on developing customized software for user‐specific applications, which can be integrated into real‐time evaluations.

CONCLUSION

The proposed approach achieves promising performance in the automatic detection, segmentation, and diagnosis of gallbladder conditions in patients requiring cholecystectomy. While the current method does not alter the established indications for selecting AC patients for cholecystectomy—largely due to the inherent “black‐box” nature of the CNN decision‐making process—our ensemble model successfully integrates two CNN architectures to achieve precise differentiation of the gallbladder from other abdominal structures. This facilitates accurate feature recognition of gallbladder conditions in such patients. Looking ahead, computer‐assisted systems have the potential to serve as valuable auxiliary tools in identifying patients requiring emergency surgical interventions.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

ETHICS STATEMENT

Approval of the research protocol: The research protocol (NO: B202305161) has been reviewed and approved by the Institutional Review Board of Tri‐Service General Hospital.

Informed consent: Consent to publish has been obtained from all participants.

Registry and registration no. of the study /trial: B202305161 of this research was reviewed and approved by the Institutional Review Board of Tri‐Service General Hospital.

Supporting information

Appendix S1

Yang C‐Y, Kao H‐L, Chen YC, Kuo C‐F, Liu CH, Liu S‐C. Automated CT image prescription of the gallbladder using deep learning: Development, evaluation, and health promotion. Acute Med Surg. 2025;12:e70049. 10.1002/ams2.70049

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.