Deep learning-based real-time intraoperative detection of thoracic duct

Abstract

Background

The thoracic duct (TD) is fragile in esophageal cancer surgery, and its injury may cause deadly chylothorax. Its identification is critical to prevent chylothorax and ensure patient safety. Deep learning-based image processing software may assist surgeons. This study aimed to develop and validate a deep learning-based system for real-time intraoperative detection of the thoracic duct.

Methods

From 30 thoracoscopic esophagectomy videos (prone position), 2,500 images (1,400 TD annotations; 1,100 background images) were extracted. The dataset was divided into training/validation (2,000/500 images) and 40 test images from 5 independent videos. The YOLOv5-seg model’s performance was evaluated via dice coefficient & intersection over union and was statistically compared with resident and attending surgeons.

Results

The artificial intelligence (AI) model’s segmentation performance was superior to that of resident surgeons but inferior to that of attending surgeons. The AI system achieved a 92.5% accuracy, with a mean dice coefficient of 0.677 and an intersection over union (IoU) of 0.569. The AI system demonstrated significantly lower Dice performance than attending surgeons (0.677 vs. 0.797, P=0.01), but significantly higher performance than resident surgeons (0.677 vs. 0.577, P=0.02). However, the AI significantly outperformed the resident group (accuracy: 85.0%; dice: 0.577 vs. AI, P=0.02; IoU: 0.461 vs. AI, P=0.009), confirming its relative performance advantage over less-experienced surgeons.

Conclusions

This study demonstrates the technical feasibility of a deep learning–based real-time TD segmentation system. The model achieved segmentation performance superior to that of resident surgeons but inferior to that of attending surgeons and suggests its potential as a technical foundation for future AI-guided intraoperative applications.

Highlight box.

Key findings

• A deep learning-based model achieved real-time thoracic duct segmentation with a Dice coefficient of 0.677, outperforming resident surgeons while remaining inferior to attending surgeons.

What is known and what is new?

• Thoracic duct identification during thoracoscopic esophagectomy is technically challenging due to its small size, anatomical variability, and low contrast, and is critical for preventing postoperative chylothorax.

• This study presents a real-time artificial intelligence (AI)-based segmentation system for intraoperative thoracic duct recognition using thoracoscopic video and demonstrates its performance relative to surgeons with different experience levels.

What is the implication, and what should change now?

• The model may serve as a decision-support tool to assist less experienced surgeons and improve intraoperative identification of the thoracic duct, with potential for future integration into real-time surgical navigation and AI-assisted operative systems.

Introduction

The thoracic duct (TD), due to its 1-2 mm diameter, proximity to the esophagus and aorta, and the narrow surgical field, is notoriously vulnerable (3–12%) to injury during thoracoscopic surgery (1,2). Chylothorax resulting from TD injury can cause patients to lose a large quantity of lipids postoperatively, affecting postoperative prognosis and patient safety (3). Therefore, successful identification of the TD is essential to a successful esophageal surgery.

The incidence of chylothorax following esophageal cancer surgery typically ranges from 1% to 9% (4,5). Previous studies have attempted to enhance visualization of the TD by having patients ingest high-fat substances before surgery or by using fluorescent dyes to label the TD (6-8). Most of these methods, such as the use of imaging or contrast agents, are generally more complex in operation, and their effectiveness also varies from person to person. We instructed patients to take 30 mL of olive oil the night before esophageal surgery to distend the TD, making it easier to recognize during the operation. However, the final effect varies among patients, and in some patients, the TD remains inconspicuous (Figure 1). Deep learning, particularly semantic segmentation using convolutional neural networks (CNNs), has been successfully applied to medical image processing (9,10). This technique enables precise, pixel-level classification of tissues and demonstrates strong robustness, even when analyzing challenging thoracoscopic images that are noisy, incomplete, or of poor quality (11-13).

Figure 1.

Intraoperative visualization variability of the thoracic duct (shown as green circles and arrowheads) during thoracoscopic esophagectomy.(A,B) The thoracic duct demonstrates clear anatomical structure with optimal exposure; (C,D) the thoracic duct maintains structural clarity but exhibits adhesion to surrounding tissues; (E,F) the thoracic duct appears diminutive in caliber with suboptimal visualization.

To reduce the risk of TD injury, particularly for less experienced surgeons, we developed a real-time deep learning-based navigation system intended for intraoperative TD recognition in thoracoscopic surgery. In this study, we focused on the technical validation of this system by assessing its segmentation performance and feasibility in comparison with resident and attending surgeons using a retrospective dataset. We present this article in accordance with the TRIPOD reporting checklist (available at https://jtd.amegroups.com/article/view/10.21037/jtd-2025-1-2437/rc).

Methods

Study design

This study was designed as a single-institution retrospective technical validation study at the Zhongshan Hospital, Fudan University, Shanghai, China. Thoracoscopic videos used for model development and evaluation were retrospectively collected from patients who had undergone esophagectomy during the study period. All video reviews and image annotation were performed retrospectively after surgery. A thoracoscopic video dataset of patients undergoing esophagectomy from March 2024 to January 2025 was retrospectively collected.

Institutional approval

This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the local ethics committee of Zhongshan Hospital, Fudan University, Shanghai, China (No. B2025-537), and written informed consent was obtained from all participants at the time of hospitalization. The study protocol is publicly accessible on the websites of each participating institution.

Datasets

This study utilized thoracoscopic esophagectomy videos encompassing mediastinal lymph node dissection and mobilization of the middle/upper esophageal segments (Table 1). On the evening before esophageal surgery, patients were instructed to consume 30 mL of olive oil to enhance TD visualization, facilitating its intraoperative identification. A total of 30 surgical videos (14 TD-positive, 16 TD-negative) were utilized to construct the dataset.

Table 1. Training and evaluation dataset patient characteristics.

Variables	Values (n=30)
Age, years	66.5 [51–82]
Gender
Male	21
Female	9
BMI, kg/m2	23.4 [19.4–28.7]
Location
Upper	13
Middle	17

Data are presented as number or median [range]. BMI, body mass index.

All videos were recorded at 30 frames per second (fps). Among these, 14 videos confirmed the presence of the TD. From these 14 videos, video clips containing TD structures (5–8 clips) and those without (5–8 clips) were first delineated, from which static images were extracted. All TD annotations were performed by a single chief thoracic surgeon with extensive experience in thoracoscopic esophagectomy, who identified this structure based on its anatomical location, morphological characteristics, and intraoperative visual features. To ensure annotation consistency, all labeled images were reviewed iteratively during dataset construction, and ambiguous cases were discussed within the research team before final inclusion.

Subsequently, we further delineated the anatomical boundaries of the TD using Labelme (https://github.com/wkentaro/labelme), resulting in 1,400 annotated images. To establish background references devoid of the target structure, 50 non-duct images were extracted from each of the original 14 videos. Additionally, 25 images per video were acquired from 16 TD-negative videos to augment background diversity. This process culminated in a comprehensive dataset comprising 2,500 images (1,400 annotated images + 1,100 background images).

The dataset was partitioned into training and validation sets at a 4:1 ratio, with TD-containing images and background images being randomly allocated for model training, selection, and tuning. This resulted in 2,000 images in the training set and 500 images in the validation set (Figure 2). For independent evaluation, 5 independent surgical videos (separate from the original 30 videos) were selected to build the test set, including 2 additional internal videos and 3 videos from other medical centers. A total of 40 static frames (8 per video) were extracted for testing. For the external test set, static frames were purposively selected to include both TD-positive and -negative images across different surgical phases. This strategy was adopted to ensure that the test set contained representative and challenging scenarios. These external videos differed in surgical camera systems, image color balance, and preoperative fat-loading practices, producing a domain shift relative to the internal training data.

Figure 2.

Datasets construction process. Workflow includes video selection, frame extraction, manual annotation, background sampling from duct-negative videos, dataset partitioning, and creation of an independent test set. TD, thoracic duct.

Model

In this study, we used the YOLOv5-seg image segmentation model. It was forked and optimized by z1069614715 from Ultralytics’ YOLOv5 for instance segmentation tasks (available at https://github.com/z1069614715/yolov5-seg) and is based on the YOLOv5 CNN developed and open-sourced by Ultralytics LLC (https://github.com/ultralytics/yolov5). Compared to the original YOLOv5 versions, YOLOv5-seg integrates segmentation capabilities alongside object detection, enabling simultaneous object detection and instance segmentation. The YOLOv5 architecture integrates adaptive anchor box calculation and adaptive image scaling, enabling standardized processing of thoracoscopic video frames with different resolutions and providing corresponding feedback. The output layer of the model generates two segmentation classes: background and TD. Data augmentation techniques, such as Mosaic augmentation, were applied to optimize the accuracy (Figure 3).

Figure 3.

Model architecture and training pipeline for thoracic duct segmentation. The YOLOv5s-seg segmentation network integrates detection and instance segmentation, adaptive anchor computation, and Mosaic-based data augmentation for enhanced robustness.

Model training was trained on an NVIDIA GeForce RTX 3090 (24GB GDDR6X VRAM) graphics processing unit (GPU).

Evaluation and validation

The external test set contained both TD-positive and TD-negative images; we defined accuracy as the proportion of images in which the annotator [artificial intelligence (AI) or surgeon] correctly classified the presence or absence of the TD. A correct prediction required both (I) identifying a TD when present [intersection over union (IoU) >0], and (II) making no segmentation when the image contained no TD. Thus, accuracy accounted for three types of errors: (I) missed detections in TD-positive images; (II) false identifications in TD-negative images; and (III) incorrect localization.

The Dice coefficient was employed to evaluate the performance of TD identification. The Dice coefficient provides a sensitive quantification of subtle discrepancies between segmentation outputs and ground-truth annotations, thereby enabling robust performance evaluation of computational models and facilitating accurate clinical decision-making by healthcare professionals (14). The Dice coefficient is defined as:

$$Dicecoefficient=\frac{2\left|A\cap B\right|}{\left|A\right|+\left|B\right|}$$ [1]

The IoU, particularly for object detection and semantic segmentation, provides an intuitive measure of spatial overlap between prediction outputs and ground-truth annotations. Therefore, it was selected as a secondary evaluation metric in this study. The IoU is defined as:

$$IoU=\frac{\left|A \cap B\right|}{\left|A\cup B\right|}$$ [2]

A and B represent the segmentation regions annotated by the AI model or manually annotated by the surgeon. (The dice coefficient and IoU range from 0 to 1, with 1 indicating perfect segmentation and 0 indicating no overlap.)

Surgeons were divided into two groups: 3 resident surgeons and 3 attending surgeons. Attending surgeons had experience in over 200 thoracoscopic esophagectomy procedures, while resident surgeons had fewer than 50 cases. The Dice coefficients and IoU for both the AI model and the surgeons were calculated on the test set. All participating surgeons independently evaluated the same set of 40 static test images. Surgeons were blinded to the ground-truth annotations and AI-generated segmentation results and were not provided with any additional clinical information. Image assessment was performed offline using static frames without time constraints.

Statistical analysis

We compared the Dice coefficients of the model separately with those of the surgeons in the attending group and the resident group. First, to determine whether the variances of the data from different groups were homogeneous, we performed an F-test for homogeneity of variance. Based on the results of the F-test, if the variances were homogeneous, we used a two-sample t-test; otherwise, we applied Welch’s two-sample t-test to compare the performance of the model with each group of surgeons. A P-value less than 0.05 was considered statistically significant, indicating a significant difference between the compared groups. Statistical analyses were performed using RStudio (https://posit.co/products/open-source/rstudio) and Prism (https://www.graphpad.com).

Results

Performance validation and latency evaluation

Videos from 30 patients (Table 1) were used for training and validation. Based on the evaluation results, YOLOv5s-seg was selected as the inference model. The YOLOv5s-seg model demonstrates a Dice coefficient of 0.8278 and an IoU of 0.7292; meanwhile, its graphic prediction accuracy closely approximates that of resident surgeons.

It also has the lowest number of missed detections and the fastest response time at 17.7 ms per frame. This near-real-time performance enabled seamless analysis of 2K-resolution thoracoscopic videos at 60 fps, with the computational latency (17.7 ms) approximating the inter-frame interval (16.7 ms) in standard video streams (Table 2).

Table 2. Comparative performance of 3 models in computational efficiency and response speed (500 images).

Model	Dice	IoU	Miss	False	Accuracy (%)	Response time (ms)
yolov5s-seg	0.8278	0.7292	8	20	94.4	17.7
yolov5m-seg	0.8174	0.7220	18	20	92.4	22.5
yolov5l-seg	0.8253	0.7292	14	18	93.6	33.8

IoU, intersection over union.

Test performance

Performance comparisons demonstrated hierarchical results: attendings (97.5% accuracy, Dice =0.797, IoU =0.692) > AI (92.5%, 0.677, 0.569) > residents (85.0%, 0.577, 0.461) (Table 3, Figure 4A,4B). The AI system demonstrated statistically significant performance advantages over the resident group, though marginally inferior to attending-level benchmarks. All statistical comparisons reached significance at P<0.05, and quantitative comparisons revealed: Dice coefficient: AI vs. attendings (P=0.01), AI vs. residents (P=0.02) (Figure 5A).

Table 3. Dice and IoU coefficients of surgeons.

Group	Surgeon	Dice	IoU
Attendings	A1	0.800	0.693
	A2	0.795	0.691
	A3	0.796	0.692
	Mean ± SD	0.797±0.002	0.692±0.001
Residents	R1	0.582	0.466
	R2	0.577	0.459
	R3	0.575	0.458
	Mean ± SD	0.577±0.004	0.461±0.004

IoU, intersection over union; SD, standard deviation.

Figure 4.

Performance comparison of AI and surgeons. (A) Accuracy, missed detections, and false detections across attendings, AI, and residents. (B) Mean Dice coefficients and IoU values showing a three-tier hierarchy: attendings > AI > residents. AI, artificial intelligence; IoU, intersection over union.

Figure 5.

Statistical comparison of segmentation performance. (A) Distribution of Dice coefficients among attendings, AI, and residents. (B) Distribution of IoU across groups. AI, artificial intelligence; IoU, intersection over union.

The IoU, a widely adopted evaluation metric in computer vision tasks, particularly for object detection and semantic segmentation, provides an intuitive measure of spatial overlap between prediction outputs and ground-truth annotations. Therefore, it was selected as a secondary evaluation metric in this study.

IoU: AI vs. attendings (P=0.004), AI vs. residents (P=0.009) (Figure 5B).

This three-tiered performance hierarchy (attendings > AI > residents) suggests the potential utility of the AI system as a technical decision-support tool bridging the expertise gap between specialists and general surgeons.

Subgroup qualitative analysis

Well-distended TD (lipid-filled): when the TD appeared as a thickened milky-white structure with clear margins (Figure 6A), all groups—including the AI—achieved high recognition accuracy regardless of exposure angle.

Figure 6.

Representative cases in thoracic duct identification. (A) Easily identifiable, lipid-distended thoracic duct. (B) Ambiguous lymphatic structures cause misidentification, particularly among residents. (C) Poorly distended duct with low contrast, challenging for all groups, including AI. AI, artificial intelligence; IoU, intersection over union.

Ambiguous lymphatic structures: in cases where adjacent small lymphatic channels mimicked TD morphology (Figure 6B), residents frequently misidentified non-duct structures. The AI and attending surgeons demonstrated more robust discrimination under these conditions.

Poorly distended or low-contrast TD: in images with minimal lipid absorption, where the duct appeared thin, flattened, or color-blended with surrounding tissue (Figure 6C), performance decreased for all groups. These challenging scenarios represented a principal source of mis-segmentation in both human and AI annotators.

Application

A representative video demonstrating AI-based TD recognition is provided as Video 1. In this video, the real-time segmentation output is overlaid directly onto the thoracoscopic display, allowing the TD to be highlighted continuously during dissection. For comparison, a synchronized split-screen view is presented, showing the original unprocessed surgical footage alongside the AI-annotated stream. This visualization clearly illustrates how the model enhances intraoperative recognition by providing stable, high-contrast boundary delineation even in frames with suboptimal lighting, motion artifacts, or low tissue contrast.

Video 1.

Download video file ^{(8.1MB, mp4)}

Real-time artificial intelligence-assisted thoracic duct segmentation during thoracoscopic esophagectomy, with segmentation results overlaid on the surgical video.

Discussion

Thoracoscopic surgery has emerged as the mainstream surgical approach, with robotic-assisted surgery increasingly adopted in clinical practice (9-11). Fully autonomous AI-powered surgical robots are currently under active investigation (12). Machines’ ability to extract and analyze information from images and videos is pivotal, whether providing intraoperative decision support to surgeons to reduce postoperative complications or enabling precise intraoperative navigation for robotic-assisted procedures (13).

Our deep learning model achieved a TD segmentation Dice score of 0.677 in thoracoscopic esophagectomy, outperforming resident surgeons but slightly lagging behind attendings. This gap reflects the TD’s small size (<2 mm) and low tissue contrast, which makes it more challenging to identify than larger structures like the prostate or gallbladder (15-17). Despite these challenges, the AI model’s precision could significantly reduce the risk of injury for less experienced surgeons.

Performance decline from the validation set (Dice 0.828) to the external test set (Dice 0.677) was expected because the validation images were drawn directly from the same 30 procedures used for training, whereas the test set included 2 additional internal videos and 3 external-center videos. The external procedures differed in camera type, lighting, tissue color balance, and preoperative lipid-loading protocols, creating a domain shift. Importantly, not only the AI model but also attending and resident surgeons showed reduced segmentation performance under these external conditions, confirming that the challenge originated from image domain variability rather than model overfitting. Despite this, the AI system remained closer to attending-level performance and substantially outperformed residents, supporting its utility as an intraoperative navigation aid for less-experienced surgeons.

Accurate TD identification is crucial yet challenging, particularly in robotic esophagectomy (18). Frame-level semantic segmentation represents a necessary first step toward future 3D reconstruction, advanced visualization, and adaptive learning frameworks, by providing real-time anatomical delineation that may support subsequent exploration of dynamic intraoperative scenarios (19).

The selection of YOLOv5-seg over newer iterations (v6/v7/v8) or alternatives like Mask R-CNN was driven by three aligned considerations: temporal relevance, real-time supremacy, and clinical deployability. During our 2024 study period, YOLOv5 represented the real-time detection state-of-the-art, with its segmentation variant offering medical imaging stability (20). Crucially, YOLOv5s-seg achieved 17.7 ms latency—uniquely meeting the 16.7 ms frame-processing requirement for 60 fps thoracoscopy (21). This performance advantage proved essential for intraoperative navigation, where newer YOLO versions risk computational overhead despite accuracy gains, and two-stage architectures like Mask R-CNN (>50 ms latency) fundamentally lack real-time capability (22).

There are several limitations in this study that should be acknowledged.

First, TD annotations were generated retrospectively and performed by a single expert thoracic surgeon. Although expert-based annotation is commonly adopted in early-stage surgical AI studies, the lack of formal multi-expert annotation and inter-observer agreement analysis may limit the generalizability of the ground-truth definition. Future studies will incorporate multi-expert consensus labeling and quantitative agreement assessment to strengthen annotation robustness.

Second, the training dataset was constructed from retrospectively labeled surgical videos, and the model demonstrated less stable performance in non-dissected regions than in dissected areas, suggesting limited robustness during early dissection phases. Future work will emphasize learning from non-dissected anatomy and early surgical stages while refining algorithms to reduce overfitting.

Third, although the system demonstrated technical feasibility and real-time performance, it has not yet been integrated into prospective clinical workflows, and its impact on intraoperative decision-making or postoperative outcomes remains untested.

In addition, the model occasionally misidentified other anatomical structures as the TD. Although false positives are generally less clinically critical than missed detections, future iterations will aim to further reduce misidentification by incorporating training strategies targeting common anatomical mimics.Finally, although the inclusion of an external test set improved robustness evaluation, the external images were purposefully selected to cover different surgical stages and levels of anatomical difficulty rather than being randomly sampled, which may introduce selection bias. This design was intended to assess model robustness under heterogeneous and challenging conditions. Future studies using larger multi-center datasets with standardized and randomized sampling strategies are required for more comprehensive validation.In conclusion, the findings of this study demonstrate that deep learning models achieve promising accuracy in TD image segmentation, highlighting their potential for intraoperative anatomical guidance in thoracic surgery.

Conclusions

This study demonstrates the technical feasibility of a deep learning-based real-time semantic segmentation system for TD identification during thoracoscopic esophagectomy. In a retrospective evaluation, the proposed system achieved segmentation performance superior to that of resident surgeons while remaining inferior to attending-level performance. These findings suggest that such systems may serve as a promising technical foundation for future intraoperative decision-support applications. Future prospective, multi-center studies are required to further validate their clinical utility and impact on surgical outcomes.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the local ethics committee of Zhongshan Hospital, Fudan University, Shanghai, China (No. B2025-537), and written informed consent was obtained from all participants at the time of hospitalization.

Data Sharing Statement

Available at https://jtd.amegroups.com/article/view/10.21037/jtd-2025-1-2437/dss

jtd-18-04-330-dss.pdf ^{(71.4KB, pdf)}

DOI: 10.21037/jtd-2025-1-2437