Development of an Artificial Intelligence Platform for Surgical Tool Detection and Phase Recognition in Laparoscopic Distal Gastrectomy

Abstract

Purpose

Surgical tool detection and phase recognition are the foundational areas for objective surgical assessment and future integration with intelligent operative systems. We aimed to develop and validate an artificial intelligence (AI) model for automated surgical tool detection and phase recognition in laparoscopic distal gastrectomy for gastric cancer.

Materials and Methods

In this retrospective, single-center study, we included 93 annotated laparoscopic distal gastrectomy videos from Seoul National University Hospital. A You Only Look Once version 8 (YOLOv8) model was trained to detect 14 surgical instruments across 60,102 annotated images using a refined tip-focused annotation strategy. The surgical workflow was divided into 12 predefined phases. Tool presence inferred from the YOLOv8 output was structured and used as input for a bidirectional long short-term memory (bi-LSTM) network for temporal phase recognition. Performance was evaluated using the mean average precision (mAP), precision, recall, F1 score, and accuracy.

Results

The YOLOv8 model achieved a 94% mAP at an intersection over union threshold of 0.50 (mAP50), with a precision >95% for recognizing frequently used tools. The bi-LSTM model achieved a precision of 82%, recall of 84%, F1 score of 83%, and accuracy of 85%. Phase prediction was the highest for clear anatomical transitions and lowest for visually ambiguous phases.

Conclusions

This AI-based framework demonstrated reliable performance in surgical tool detection and phase recognition. Using the YOLO-inferred tool as input to the bi-LSTM phase model improved robustness and scalability. This approach offers a foundation for real-time surgical guidance, quality benchmarking, and structured resident training in gastric cancer surgery.

INTRODUCTION

Artificial intelligence (AI) has gained attention as a promising tool for analyzing surgical videos and workflows, with potential applications in objective intraoperative assessment and surgical quality improvement [1,2]. In gastric cancer surgery, contemporary clinical guidelines emphasize standardized evidence-based surgical management to improve oncological outcomes and ensure consistent quality across institutions [3]. However, intraoperative performance assessment remains largely subjective and dependent on retrospective reviews, particularly for complex procedures such as laparoscopic and robotic gastrectomy. This limitation underscores the need for automated and reproducible methods to analyze surgical workflows. In this context, AI-based surgical phase recognition has emerged as a key technology for supporting standardized surgical evaluation, intraoperative decision making, and future integration with intelligent operative platforms [4,5].

In laparoscopic and robotic gastrectomies, phase recognition is a fundamental component that must be established for objective surgical assessment and future integration with semi-automated or robot-assisted operative systems. Recent studies have demonstrated the potential of AI models to identify surgical phases and assess performance metrics [6,7,8,9]. Moreover, the ability of AI to detect complex intraoperative factors, such as anatomical variability and surgeon proficiency, has proven valuable for surgical quality control [10,11,12]. Convolutional neural networks (CNNs) and vision transformers have shown strong performance in phase prediction [13,14,15,16,17,18,19]. Nonetheless, CNNs primarily extract spatial features from individual frames and often lack the capability to model temporal relationships between procedural steps, resulting in instability during phase transitions [19]. Transformer-based architectures address this limitation by capturing long-range temporal dependencies through self-attention mechanisms. However, they typically require large annotated datasets, extensive computational resources, and complex training pipelines, which may limit their scalability and real-time applicability in clinical settings [20,21]. Furthermore, many prior studies relied heavily on manually curated ground-truth labels, which may not adequately reflect real-world variability such as transient tool occlusion, motion blur, or inter-institutional heterogeneity [22,23].

To overcome these limitations, we developed a novel AI-based system combining You Only Look Once version 8 (YOLOv8) for surgical tool detection and bidirectional long short-term memory (bi-LSTM) networks for temporal phase recognition [24,25]. Unlike conventional CNN- or transformer-based approaches that rely solely on raw image features, our model first encodes the surgical context through explicit tool recognition and then leverages sequential modeling to capture the temporal relationships between procedural steps. YOLO, introduced in 2016 by Redmon et al. [26], reframed object detection as a single-step regression task, enabling real-time localization and classification in a unified network. Over the years, the YOLO architecture has evolved significantly, culminating in YOLOv8, which introduces an anchor-free detection pipeline, a decoupled head structure, and enhanced spatial learning capabilities [27]. These advancements enable robust and efficient tool recognition even under occlusion, motion blur, and instrument variability, making YOLOv8 particularly well suited for surgical video analysis. Long short-term memory (LSTM) is a type of recurrent neural network (RNN) that uses a loop structure to process sequential data and retain long-term information using memory cells [28]. This overcomes the vanishing gradient problem of traditional RNNs by capturing dependencies spanning different time steps. The bi-LSTM combines the power of LSTM with bidirectional processing, allowing the model to capture sequential data in both forward and backward directions [29,30]. Consequently, it leverages the temporal context to accurately identify transitions between procedural steps, even in cases of visual ambiguity or overlapping tool use.

We aimed to develop an AI model to automatically recognize surgical tools and phases in laparoscopic distal gastrectomy. By establishing a system capable of using frame-level visual features and tool-encoded temporal data, we aimed to demonstrate that AI-assisted surgical analysis can provide a solid foundation for further research on intraoperative quality assurance. We hypothesized that this integrated system can improve the accuracy, consistency, and generalizability of surgical phase recognition, ultimately contributing to future real-time quality assurance, surgeon training, and operative benchmarking in gastric cancer surgery.

MATERIALS AND METHODS

We conducted a retrospective single-center study involving 93 annotated videos of laparoscopic distal gastrectomies performed by 2 experienced gastrointestinal surgeons at Seoul National University Hospital, a high-volume center. These videos were recorded from August 2021 to September 2022, and included cases of laparoscopic distal gastrectomy performed for histologically proven, primary, and resectable (cT1-4a, N0-3b, and M0) gastric adenocarcinoma. We included adult patients who underwent radical gastrectomy with standard lymphadenectomy, while those with insufficient video quality or incomplete surgical procedures were excluded. This study was conducted in accordance with the principles of the Declaration of Helsinki and was approved by the Institutional Review Board (IRB) of Seoul National University Hospital (IRB No. 2602-096-1717), which waived the requirement for informed consent owing to the use of retrospective anonymized data.

Dataset preparation

A total of 93 videos of laparoscopic distal gastrectomies were collected and processed. All videos were initially downsampled to 1 frame per second (FPS), yielding up to 3,600 frames per hour of surgical footage. Given the impracticality of annotating all frames, we implemented a semi-structured rule-based sampling strategy with manual oversight to select 500–600 representative frames per video. Frame selection was guided by information from the surgical phase and approximate tool usage patterns within each phase. Using these criteria, frames were sampled across the phases and tool categories to ensure a balanced representation while minimizing redundancy. In cases in which the tools were underrepresented, the frame rate was increased to 2 FPS or higher to capture additional instances. The sampling strategy was as follows:

• • Ensure coverage of all predefined surgical phases

• • Capture diverse appearances of surgical tools

• • Avoid temporally adjacent, visually similar frames

• • Enrich underrepresented tool classes

The sampling strategy was phase-agnostic rather than phase-stratified; frames were selected to maximize visual diversity rather than to enforce a fixed number of samples per phase. Consequently, the final frame distribution reflects both the phase duration and intra-phase visual complexity. The quantitative distribution of annotated frames per surgical phase, along with the mean (standard deviation) phase duration, is summarized in Table 1.

Table 1. Distribution of annotated frames and phase durations for each surgical phase across the training, validation, and test datasets.

Phase	All videos (n=93)	Training set (n=75)	Validation set (n=9)	Test set (n=9)	Duration (minutes), mean ± SD
PC00 (Interval)	6,206	4,712	875	619	-
PC01	5,761	4,634	593	534	12.7±9.1
PC02	3,276	2,341	449	486	20.4±12.2
PC03	5,426	4,159	301	966	41.4±19.9
PC04	858	672	64	122	4.1±2.7
PC05	907	528	138	241	8.5±3.6
PC06	2,102	1,547	249	306	1.9±0.8
PC07	5,147	3,873	687	587	20.2±9.0
PC08	704	540	100	64	10.2±3.9
PC09	1,183	967	139	77	4.5±2.7
PC10	4,462	3,532	605	325	1.8±3.5
PC11	8,763	6,534	1,230	999	16.1±11.0
PC12	15,307	12,417	1,649	1,241	17.6±8.5
Total	60,102	46,456	7,079	6,567	-

PC = phase count; SD = standard deviation.

For the phase estimation task, we followed the Cholec80 format using a binary scheme in which tool presence was marked as “1” and absence as “0” [22]. Initially, tool-presence data were generated by manually recording the tool entry and exit times within each video. However, an internal review of the sample videos revealed an annotation discrepancy rate of approximately 15%. This estimate was derived from a secondary audit of randomly selected sample videos in which timeline-based manual tool annotations were compared against frame-level verification performed at 1 FPS. This discrepancy was primarily due to transient tool occlusion, partial visibility, and omission of inactive background instruments, such as gauze. To address this limitation, we used a trained YOLOv8 model to generate tool-presence data for phase recognition. This approach introduced realistic detection noise into the input data, allowing the bi-LSTM model to learn the temporal context while remaining robust against detection errors under real-world surgical conditions.

Tool detection dataset

Bounding box annotations were performed using Annovie (MTEG, Seoul, Korea), a specialized medical image labeling software platform [31]. An initial YOLOv8 model trained on a partial dataset consisting of 20 videos was used to generate automatic annotations for the remaining images, which were manually corrected by human annotators.

Traditional annotation rules, such as marking an object when more than half is visible, were insufficient because of the complexity of surgical tools, which vary in appearance based on the camera position (e.g., zoom level), occlusion, structural similarity of tools, and image blurring from blood, motion, or fog. To improve robustness in complex surgical environments, we adopted a refined annotation strategy focusing on the tool tip and a small part of the shaft instead of the entire instrument (Fig. 1). This approach was based on the observation that tool tips contain the most discriminative visual features, whereas shafts are often visually similar across instruments. The annotation protocol was defined as follows:

Fig. 1. Conventional versus refined annotation strategies for surgical tool detection. (A) Conventional annotation of the entire surgical instrument. (B) Refined annotation focusing on the tool tip and proximal shaft to improve discriminative feature learning and detection robustness.

• • Bounding boxes were centered on the tool tip and adjacent shaft.

• • Partially visible tools were annotated if the tip was identifiable.

• • For elongated or deformable objects (e.g., gauze and drains), the bounding box encompassed the most visually distinct region.

• • Overlapping tools were annotated independently.

Using this annotation strategy, the model was able to detect tools even when only a small portion of the tip was visible and learned the distinguishing features of both tool components. Overall, 60,102 images were annotated using 14 distinct surgical tool classes.

Phase estimation dataset

Phases were manually annotated across all videos by 2 experienced gastrointestinal surgeons based on a predefined phase classification system using the SurgStory platform of MTEG [31]. These phases were designed to reflect clinically meaningful procedural steps and anatomical transitions during laparoscopic distal gastrectomy, including the key oncological dissection and reconstruction steps. The phase classification framework was developed based on the standard operative workflow of distal gastrectomy, alignment with established lymphadenectomy principles, and consensus among experienced gastrointestinal surgeons at our institution [32].

Each phase corresponded to a distinct procedural milestone, yielding 12 phases: PC01 (Preparation), PC02 (Left greater curvature dissection), PC03 (Right greater curvature dissection), PC04 (Lesser curvature preparation), PC05 (Right lesser curvature dissection), PC06 (Duodenal transection), PC07 (Suprapancreatic dissection), PC08 (Right paracardial dissection), PC09 (Proximal stomach transection), PC10 (Stomach specimen removal), PC11 (Anastomosis), and PC12 (Closure) (Table 2). To improve annotation consistency, phase transitions were determined based on the dominant surgical action, while temporal continuity was maintained to avoid excessive fragmentation of the phases. Ambiguous transitions were resolved through a consensus review by the 2 annotators based on the primary operative objective at the corresponding time point.

Table 2. Definition of phases for laparoscopic distal gastrectomy.

Phase	Step	Description
PC01	Preparation	Trocar insertion, peritoneal exploration, may involve marking tumor location or proximal/distal resection lines
PC02	Left greater curvature dissection	Left side omentectomy, LN#4sb & #4d dissection, LGEA ligation, stomach greater curvature clearing
PC03	Right greater curvature dissection	Right side omentectomy, LN#6 dissection, RGEA ligation
PC04	Lesser curvature preparation	Lesser omentum division, may involve liver retraction
PC05	Right lesser curvature dissection	LN#5 dissection, RGA ligation, may involve LN#12a dissection
PC06	Duodenal transection	Distal resection using stapler
PC07	Suprapancreatic dissection	LN#8 & LN#7 & LN#9 dissection, LGA ligation, may involve LN#11p dissection
PC08	Right paracardial dissection	LN#1 & #3 dissection
PC09	Proximal stomach transection	Proximal resection using stapler(s)
PC10	Stomach specimen removal	Specimen removal, may involve usage of laparoscopic specimen retrieval bag
PC11	Anastomosis	Reconstruction using stapler and/or handsewn sutures
PC12	Closure	Gauze retrieval, saline irrigation, drain insertion, hemostasis, anti-adhesive material application

PC = phase count; LN = lymph node; LGEA = left gastroepiploic artery; RGEA = right gastroepiploic artery; RGA = right gastric artery; LGA = left gastric artery.

Image preprocessing

All images were resized to a standardized resolution of 640×640 pixels and normalized according to the default YOLOv8 preprocessing pipeline. To improve the generalizability of the model, we applied standard YOLOv8 data augmentation techniques during training. This included spatial augmentations, such as horizontal flipping, scaling, and translation, along with color-space adjustments for brightness, contrast, and saturation. We also used advanced composition techniques, specifically Mosaic and MixUp augmentation, which combine multiple images into a single training sample. These preprocessing steps were designed to simulate variability in surgical environments, including different lighting conditions, camera angles, and visual occlusions.

AI model development

The AI workflow was divided into 2 stages: tool detection and phase recognition.

Tool detection

YOLOv8 was used to detect the surgical tools present in each video frame, identifying the tool, its location, and its detection probability. YOLO models process entire images in a single forward pass, thereby enabling high-speed inference. The YOLOv8 architecture mainly consists of 3 blocks: Backbone, Neck, and Head (Fig. 2A) [27]. The backbone, also called the feature extractor, is responsible for extracting meaningful features from the input. The neck functions as a bridge between the backbone and head, performs feature fusion operations, and integrates contextual information. The head generates network outputs such as bounding boxes and confidence scores for object detection.

Fig. 2. Architecture of the proposed AI-based surgical workflow recognition system. (A) YOLOv8-based surgical tool detection network. (B) The bi-LSTM network for temporal phase recognition. (C) Overall workflow of the proposed model, illustrating surgical video input, tool detection using YOLOv8, structured tool feature extraction, and temporal phase prediction using the bi-LSTM model.

AI = artificial intelligence; YOLOv8 = You Only Look Once version 8; bi-LSTM = bidirectional long short-term memory; Conv = convolution; Concat = concatenation; C2F = cross-stage feature fusion module; SPPF = spatial pyramid pooling–fast layer; LSTM = long short-term memory.

Phase recognition

The bi-LSTM neural network was used for phase recognition. The forward and backward processing architectures of the bi-LSTM are shown in Fig. 2B [33]. The input to the bi-LSTM consisted of tool-presence data from YOLOv8 formatted in Cholec80. The model generated phase predictions for each second of the surgical video. The bi-LSTM model, which is a time-series model, allowed the prediction of the specific phase based on the temporal sequence of the detected tools (Fig. 2C). This integration enabled the model to accurately predict phases even when the same tool appeared in different phases, owing to the temporal context.

Two-stage model pipeline

Of the 93 videos, 75 were used for training, 9 for validation, and 9 for testing. This split was applied consistently across the tool detection and phase-recognition tasks. To ensure a proper separation between training and evaluation, the YOLOv8 tool detection model was trained exclusively on the training dataset. The tool-presence features for the validation and test datasets were generated using a trained detector without further fine-tuning. The bi-LSTM phase-recognition model was subsequently trained using only the training and validation data. No information from the test dataset was used during the model development.

Performance evaluation and statistical analysis

The model performance was evaluated using widely adopted standard metrics: mean average precision (mAP), intersection over union (IoU), accuracy, precision, recall, and F1 score [34,35]. For object detection, the average precision (AP) reflects the area under the precision-recall curve for a single class, summarizing both the precision and recall performance. mAP extends this to multiple classes by averaging AP values across multiple tool types to provide a holistic view of performance. The IoU computes the overlap between a predicted bounding box and a ground-truth bounding box, calculating the accuracy of object localization. Specifically, we report mAP50, which refers to the mAP at an IoU threshold of 0.50, commonly used to reflect correct but possibly loose detections. mAP50–90 indicates the average of the mAP values calculated at varying IoU thresholds ranging from 0.50 to 0.90, offering a more stringent and comprehensive evaluation of the model’s performance across different levels of detection difficulty.

For phase recognition, the accuracy is measured by quantifying the proportion of all correctly predicted labels, including true positives (TP) and true negatives (TN). Precision indicates the proportion of TP among all positive predictions and assesses the model’s capability to avoid false positives (FP). Recall refers to the fraction of TP among all actual positives, including TP and false negatives (FN), and thus measures a model’s ability to detect all instances of a class. The F1 score is the harmonic mean of the precision and recall, providing a balanced assessment of the model’s performance while considering FP and FN. These are formally defined as:

Accuracy = (TP + TN)/(TP + FP + TN + FN)

Precision = TP/(TP + FP)

Recall = TP/(TP + FN)

F1 score = 2 × Precision × Recall/(Precision + Recall)

Together, these metrics offer a comprehensive evaluation of spatial and temporal model performances across surgical tool detection and phase classification.

Region of interest (ROI)-stratified performance analysis

To evaluate the potential influence of the object scale on detection performance, we performed a stratified analysis based on the relative area of each annotated bounding box within the image. The object instances were grouped into 5% area-ratio intervals according to the proportion of the bounding box area relative to the total image area, and the AP50 performance was evaluated across each size category.

RESULTS

Our newly developed AI model demonstrated robust performance in the detection of all surgical tools, with an mAP50 of 94%. Representative examples of automated tool detection using laparoscopic gastrectomy videos are shown in Fig. 3. The model demonstrated precision exceeding 95% for detecting tools frequently used in dissection and vessel sealing, such as the LigaSure™ (Medtronic, Minneapolis, MN, USA) energy device, polymer clip applier, and gauze. In contrast, less commonly used instruments such as the dissector showed lower detection rates, likely owing to the limited annotated instances available for training. In addition, tools with heterogeneous appearances across different brands or designs exhibited moderately reduced precision. The full-performance metrics for surgical tool detection are listed in Table 3.

Fig. 3. Representative examples of manual annotations (left) and automated predictions (right) of surgical tool detection in laparoscopic distal gastrectomy. (A) Complete agreement between annotation and model prediction. (B) False-negative prediction for dissector detection. (C) False-negative prediction for grasper detection. (D) Missed manual annotation for background gauze despite successful model detection.

Table 3. Performance metrics for surgical tool detection.

Surgical tool	AP50	AP50–90
Drain	0.96	0.74
Gauze	0.96	0.77
Grasper	0.95	0.72
LigaSure™	0.98	0.86
Linear stapler	0.98	0.87
Needle holder	0.99	0.82
Obturator	0.95	0.63
Polymer clip applier	0.98	0.89
Specimen retrieval bag	0.98	0.80
Suction irrigator	0.93	0.74
Trocar	0.95	0.69
Dissector I	0.84	0.69
Dissector II	0.81	0.58
Dissector III	0.83	0.62
Average	0.94	0.75

AP50 = average precision at an intersection over union threshold of 0.50; AP50–90 = average precision averaged across intersection over union thresholds from 0.50 to 0.90.

Our model also successfully predicted the surgical phases with an overall precision of 82%, recall of 84%, F1 score of 83%, and accuracy of 85%. The highest recall rate was observed for proximal stomach transection (PC09, 99%), whereas the lowest was observed for lesser curvature preparation (PC04, 53%). The low recall rate was most likely due to the visual similarities in the surrounding anatomical structures and tool overlap. It is also important to note that not all surgical videos included the full sequence of phases from PC01 to PC12, which may have influenced the performance of the phase-wise model. Notably, the phase duration alone did not determine the number of annotated frames. Certain short phases remained relatively well-represented because they contained visually distinctive instruments or procedural events that were preferentially sampled to enhance tool diversity. The full-performance metrics for surgical phase recognition are provided in Table 4 and Fig. 4.

Table 4. Performance metrics for surgical phase recognition.

Phase	Precision	Recall	F1 score
PC00 (Interval)	0.89	0.87	0.88
PC01	0.99	0.98	0.99
PC02	0.92	0.80	0.86
PC03	0.84	0.88	0.86
PC04	0.55	0.53	0.54
PC05	0.81	0.69	0.75
PC06	0.66	0.77	0.71
PC07	0.87	0.83	0.85
PC08	0.73	0.84	0.78
PC09	0.81	0.99	0.89
PC10	0.90	0.89	0.89
PC11	0.93	0.92	0.93
PC12	0.76	0.90	0.83
Average	0.82	0.84	0.83

PC = phase count.

Fig. 4. Confusion matrix of the bi-LSTM model for surgical phase recognition across 12 predefined operative phases.

bi-LSTM = bidirectional long short-term memory; PC = phase count.

Fig. 5 illustrates an example of surgical phase recognition in a representative full-length laparoscopic distal gastrectomy video, with an overall precision of 0.88. Fig. 5A compares the ground-truth annotations with the model-predicted phases across the operative timeline. Fig. 5B highlights the regions of agreement and disagreement between the ground-truth and model predictions, demonstrating the model’s ability to accurately identify most phase transitions, while revealing occasional discrepancies during visually ambiguous or transitional periods.

Fig. 5. Example of surgical phase recognition in a representative full-length laparoscopic distal gastrectomy video. (A) Comparison between ground-truth annotations and model-predicted phases across the operative timeline. (B) Visualization of regions of agreement and disagreement between ground-truth and model predictions.

PC = phase count.

In the ROI-stratified performance analysis, AP50 performance remained consistently high across a broad range of object sizes, including smaller ROI categories (Fig. 6). Stable detection accuracy was observed across most of the 5% area–ratio intervals, suggesting that the overall model performance was not disproportionately driven by larger annotated objects such as gauze. The isolated reduction in AP50 observed in the 80%–85% ROI category likely reflects the extremely limited sample size of this subgroup (n=2). These findings indicate that the discrepancy between AP50 and AP50–90 more likely reflects the challenges of precise localization in laparoscopic environments rather than the systematic failure to detect smaller surgical instruments.

Fig. 6. ROI-stratified AP50 performance according to relative object size. Detection performance was evaluated across object size categories based on the ratio between bounding box area and total image area. Bars represent AP50 values within each 5% area-ratio interval, with corresponding sample sizes shown for each category. The dashed red line indicates the global AP50 score of the overall model. Stable performance across smaller ROI categories suggests that overall detection accuracy was not disproportionately influenced by larger annotated objects.

ROI = region of interest; AP50 = average precision at an intersection over union threshold of 0.50.

DISCUSSION

The integration of AI into surgical workflows has demonstrated potential for improving intraoperative decision-making and standardizing surgical quality. In this study, we developed a robust AI pipeline that combines YOLOv8 for surgical tool detection and bi-LSTM for phase recognition, specifically tailored to laparoscopic distal gastrectomy for gastric cancer. Our model achieved strong performance in both tool detection (mAP50 94%) and phase prediction (accuracy 85%), supporting the feasibility of AI-driven intraoperative assistance for gastric cancer surgery.

Previous studies have reported variable results regarding tool recognition and phase prediction owing to differences in surgical environments, toolsets, and annotation strategies [36]. Reports from cholecystectomy datasets have demonstrated mAP50 values ranging from 50% to 80%, underscoring the difficulty of achieving consistent tool detection across institutions [22,37,38,39]. Moreover, the tool recognition performance varies by surgical domain, with better predictions in pancreatic and colon surgeries than in gastric and thyroid surgeries [40]. In line with these observations, our results demonstrated tool-specific performance variability [41,42]. Tools such as energy devices and gauze, which are frequently used and are visually distinct, showed near-perfect detection, whereas instruments such as dissectors, which are used less frequently, had lower precision. This observation suggests that class imbalance and tool visibility remain the key challenges in video-based AI applications.

To overcome these issues, our model adopted a customized annotation strategy focusing on tool tips and proximal shafts rather than entire instruments, leading to enhanced detection precision even under partial occlusion or motion blur. This strategy aligns with emerging best practices in surgical computer vision, as emphasized by the Society of American Gastrointestinal and Endoscopic Surgeons consensus framework, which highlights annotation methodology as a critical determinant of model performance and generalizability [43]. Nevertheless, the relatively lower detection performance observed with certain instruments, such as the dissector subtypes, likely reflects their high visual similarity and limited discriminative features at the shaft level. In the current framework, feature learning was performed implicitly using the YOLOv8 architecture through pretrained representations without explicit feature-level engineering. Future approaches may benefit from incorporating fine-grained feature representations such as tip morphology or articulation patterns to improve discrimination. Specifically, we are considering a part-based annotation strategy in which the instrument shaft and functional tip are annotated separately. Prior studies have suggested that part-aware instrument modeling may improve localization and classification performance by enabling the model to distinguish shared structural components from discriminative end-effector features [44,45].

Our findings are further supported by recent studies that have validated the superiority of the YOLO architecture for surgical tool detection. Pan et al. [46] developed a dual-branched-head YOLO model that reached an mAP of 96.8% by separating the classification and localization tasks. Additionally, Goldbraikh et al. [47] showed that YOLO-based multitask networks can simultaneously perform tool localization and tool–hand interaction detection. In addition to achieving a comparable detection performance, our framework integrates YOLO-derived tool features into a sequential phase-recognition model, providing a structured representation of the surgical context. This approach improves scalability and reduces reliance on extensive manual annotations, which are inherently variable and labor-intensive, often requiring more than 12 minutes to annotate 1 minute of surgical video [48,49].

The temporal context plays a pivotal role in surgical phase recognition. Although transformer-based models can achieve high phase-recognition accuracy and cross-institutional generalizability, their computational complexity and data requirements may limit their widespread clinical implementation [5,50,51,52]. By contrast, our approach using bi-LSTM leverages temporal dependencies in tool usage patterns while maintaining computational efficiency, offering a practical balance between performance and scalability. Our overall phase prediction F1 score of 83% and accuracy of 85% are comparable to the results from previous studies, which showed an F1 score of 87.4% in cholecystectomies [53] and an accuracy of 87% in robotic gastrectomies [7].

As expected, higher recall was achieved in phases involving clear anatomical events such as proximal stomach transection (PC09), whereas subtler preparatory steps, such as lesser curvature dissection (PC04), demonstrated lower recall. More broadly, the reduced performance observed in phases PC04–PC06 reflects the inherent complexity of these steps in laparoscopic distal gastrectomy, particularly D2 lymphadenectomy. These phases involve subtle transitions between anatomically adjacent regions with overlapping tool usage patterns, making them difficult to distinguish based solely on the visual appearance or tool presence. This limitation may arise from the insufficient visual discriminability of anatomical landmarks and the absence of an explicit anatomical context in the current model. Although increasing the volume and diversity of training data may partially improve performance, further advances will likely require the integration of additional information, such as anatomical structure recognition, vessel segmentation, or instrument trajectory analysis. The incorporation of multimodal inputs, including kinematic or spatiotemporal features, may enhance the ability of the model to distinguish these clinically critical phases.

Annotation quality is an important factor. Although manual annotations are commonly regarded as the reference standard, they are inherently subject to variability, particularly when defining phase boundaries during transitional or ambiguous operative segments. In this study, we estimated that phase-labeling discrepancies may reach approximately 15% based on an internal review of annotated videos. Such label noise may influence performance metrics, particularly in phases with subtle transitions (e.g., PC04–PC06), and should be considered when interpreting model accuracy. Future studies incorporating multiple annotators and formal interobserver agreement analyses will be essential to further improve annotation reliability.

Although IoU-based metrics, such as AP50, may theoretically be influenced by the object size, our ROI-stratified analysis demonstrated a stable detection performance across a broad range of spatial scales, including smaller instruments. These findings suggest that the observed discrepancy between AP50 and AP50–90 more likely reflects the difficulty of precise localization in laparoscopic environments rather than a systematic size-related detection bias.

One of the core strengths of this study is the shift from manual ground-truth labels to AI-generated tool features as inputs for phase classification. Although human annotations are the gold standard, they often fail to fully capture inconsistencies in intraoperative visibility, including transient tool occlusion, blood contamination, and camera motion. By training the bi-LSTM model on YOLOv8-derived tool predictions, we better modeled a real-world setting where surgical assistance systems must interpret imperfect data. To the best of our knowledge, this study is among the first to demonstrate surgical phase recognition driven primarily by AI-generated tool features rather than ground-truth labels. This design requires the bi-LSTM model to achieve robust phase recognition despite the potentially imperfect tool recognition from YOLOv8.

Importantly, the clinical relevance of this study extends beyond the technical performance metrics. Surgical tool recognition and phase distinction are the foundational steps for identifying outcome-relevant intraoperative factors and enabling quantitative surgical analysis. By automatically recognizing the surgical context, our model provides a structured framework that facilitates automated safety checks, objective benchmarking, and structured assessments of procedural proficiency. These capabilities have important implications not only for surgeon training and credentialing but also for the scaling of surgical quality across institutions. Although real-time intraoperative deployment was not evaluated in this study, the use of YOLOv8, an architecture specifically optimized for rapid, single-stage object detection, offers potential advantages in terms of computational efficiency compared with multistage detection pipelines. This suggests the feasibility of future practical intraoperative implementation following further optimization and prospective validation. Prior studies have demonstrated that surgical instrument usage can affect patient outcomes [54], whereas phase-aware analytics can distinguish surgeon expertise and correlate this with operative efficiency and outcomes [51]. Integrating phase recognition with outcome-linked metrics, such as estimated blood loss, operative time, margin status, lymph node yield, or complication rates, may ultimately allow AI systems to actively support quality enhancement in gastric cancer surgery. Our model lays the groundwork for future AI platforms that can continuously learn from large-scale surgical video data, deliver real-time performance feedback, and enhance the patient safety and outcomes of gastric cancer surgery.

This study has some limitations. First, our model was trained using a single-center dataset, which potentially limits its generalizability across institutions with varying instruments, camera settings, and surgical techniques. Accordingly, this study should be interpreted as a single-center proof of concept, and future multicenter validation is necessary to confirm its broader applicability. Second, although phase annotation was independently performed by 2 experienced gastrointestinal surgeons, interobserver agreement was not formally assessed. Owing to the potential subjectivity in defining phase transitions, particularly in visually ambiguous or transitional segments, future studies incorporating additional annotators and interobserver agreement analyses will be important to further validate annotation reliability. Third, the frame selection strategy was phase-agnostic, meaning that frame counts per phase reflected both phase duration and visual complexity rather than a uniform sampling density. As shown in Table 1, substantial variations existed in both the phase duration and annotated frame counts. Although phase duration alone did not determine sample availability, this distributional imbalance may have influenced phase-level performance. Future studies should explore phase-stratified sampling and data augmentation strategies to mitigate potential sampling biases. Fourth, our current implementation processes video data offline, and future efforts should focus on adapting this model for real-time intraoperative deployment. Despite these limitations, our study established a scalable and clinically meaningful framework for AI-assisted surgical analysis. Rather than representing a final solution, this study should be viewed as a foundational step toward comprehensive AI-driven operative analytics. With continued development and validation, such systems have the potential to provide real-time feedback, enhance surgical training, and improve patient outcomes in gastric cancer surgery.

Beyond its technical performance, this system represents a foundational step toward real-time surgical intelligence. Automated recognition of the surgical context can support intraoperative decision making, reduce human error, and enhance surgical quality assurance. With further development, these models can serve as powerful platforms for surgical training, automated quality control, and outcome-linked benchmarking. Future work should focus on expanding datasets across institutions and surgeons to improve generalizability, as well as integrating multimodal intraoperative signals (e.g., instrument kinematics and patient vital signs) to further enhance predictive robustness. Ultimately, this study highlights the potential of AI not only to recognize what happens during surgery but also to elevate how surgery may ultimately be assessed, guided, and optimized in real time.