Detection of blood stains using computer vision-based algorithms and their association with postoperative outcomes in thoracoscopic lobectomies

Abstract

OBJECTIVES

Our goal was to develop high throughput computer vision (CV) algorithms to detect blood stains in thoracoscopic surgery and to determine how the detected blood stains are associated with postoperative outcomes.

METHODS

Blood pixels in surgical videos were identified by CV algorithms trained with thousands of blood and non-blood pixels randomly selected and manually labelled. The proportion of blood pixels (PBP) was computed for key video frames to summarize the blood stain information during surgery. Statistical regression analyses were utilized to investigate the potential association between PBP and postoperative outcomes, including drainage volume, prolonged tube indwelling duration (≥5 days) and bleeding volume.

RESULTS

A total of 275 patients undergoing thoracoscopic lobectomy were enrolled. The sum of PBP after flushing (P < 0.022), age (P = 0.005), immediate postoperative air leakage (P < 0.001), surgical duration (P = 0.001) and intraoperative bleeding volume (P = 0.033) were significantly associated with drainage volume in multivariable linear regression analysis. After adjustment using binary logistic regression analysis, the sum of the PBP after flushing [P = 0.017, odds ratio 1.003, 95% confidence interval (CI) 1.000–1.005] and immediate postoperative air leakage (P < 0.001, odds ratio 4.616, 95% CI 1.964–10.847) were independent predictors of prolonged tube indwelling duration. In the multivariable linear regression analysis, surgical duration (P < 0.001) and the sum of the PBP of the surgery (P = 0.005) were significantly correlated with intraoperative bleeding volume.

CONCLUSIONS

This is the first study on the correlation between CV and postoperative outcomes in thoracoscopic surgery. CV algorithms can effectively detect from surgical videos information that has good prediction power for postoperative outcomes.

Surgery has progressively shifted towards the minimally invasive surgery paradigm, such as laparoscopic surgery and video-assisted thoracoscopic surgery.

INTRODUCTION

Surgery has progressively shifted towards the minimally invasive surgery paradigm, such as laparoscopic surgery and video-assisted thoracoscopic surgery. Compared to thoracotomy, video-assisted thoracoscopic surgery (VATS) has become the standard of care for early and locally advanced non-small-cell lung cancer due to decreased acute postoperative outcomes, less impairment in pulmonary function, lower postoperative morbidity, shorter hospital stays, comparable long-term oncological results and improved postoperative quality of life [1, 2]. During VATS, digital cameras visualize and record the surgical procedures in surgical videos. These surgical videos contain rich information about patient anatomy at the surgical site, the surgical procedure, the motions of instruments during the operation and so on. It is intuitive that VATS with fewer blood stains in surgical sites may translate to reduced postoperative drainage and shorter hospital stays. Although an experienced surgeon can effortlessly tell if a surgical site is “clean”, it is a challenge to quantify blood stains in VATS videos.

Artificial intelligence (AI) has developed quickly in the last decade. Various powerful computer vision (CV) algorithms have been proposed to give computers human-like ability to recognize images and videos [3]. Some of them have been successfully applied to process medical images in radiology and pathology [4–8]. Recently, many efforts have been made to process surgical videos by CV algorithms, to extract meaningful procedural or anatomical information [9], to identify different surgical phases [10–13], to recognize various surgical instruments [14, 15] and to achieve more efficient surgical technique evaluations and teaching [16–19]. To the best of our knowledge, however, most of these efforts focus on analysing laparoscopic procedures [10–13]. Thus far, no research efforts have been exerted on analysing videos of thoracoscopic operations via CV algorithms.

We focus on CV analysis of blood stains during thoracoscopic operation, because massive bleeding events are one of the major intraoperative accidents, and blood effusions are associated with postoperative complications such as prolonged hospital stays [20, 21]. There are already some efforts in the literature using AI to identify blood stains. Hassan et al. developed a texture-feature-descriptor-based algorithm that operated on the normalized grey level co-occurrence matrix of the magnitude spectrum of the images to detect blood stains [22]. With the advent of pixel-based blood stain detection methods, Pan et al. extracted features of blood stains and proposed an intelligent blood stain detection method based on a probabilistic neural network [23]. Usman et al. proposed a pixel-based approach to detect blood stains in wireless capsule endoscopy videos by using a support vector classifier [24]. Novozamsky et al. defined a new colour space based on pixel-wise methods such that the separability of blood pixels and the intestinal wall should be maximized in a capsule endoscopy video [25]. Interestingly, for a state-of-the-art study, there is a knowledge gap between CV analysis and postoperative prognosis.

Given the facts just mentioned, we conducted a retrospective study to bridge this gap by developing high-throughput CV algorithms that can conveniently process a large number of thoracoscopic surgery videos. We hypothesized that CV algorithms could effectively extract potential signals from surgical videos of thoracoscopic surgery and that these computer-detected visual features are associated with postoperative outcomes.

MATERIAL AND METHODS

Patient selection and postoperative outcomes

A total of 333 patients undergoing thoracoscopic lobectomies at Peking University People's Hospital between 1 January 2020, and 30 December 2020 were enrolled for this retrospective study. Among the enrolled patients, 58 were excluded due to incomplete baseline medical records (5 patients) and incomplete surgical videos (46 patients) or because they had postoperative chylothorax (7 patients), resulting in 275 selected patients (Fig. 1).

Figure 1:

Illustrated flow charts of patient enrollment.

The prolonged duration of an indwelling tube was defined as a tube-carrying time ≥5 days. Postoperative drainage volume, prolonged indwelling of the tube and intraoperative bleeding volume as postoperative outcomes were extracted from the medical records and recorded.

Ethical statement

The study was approved by the institutional ethical committee (2021PHB033-001). Consent was obtained from all patients.

Surgical videos

For each selected patient, the corresponding surgical video recorded by the thoracoscopy system (KARL STORZ-ENDOSCOPE TC 200, KARL STORZ SE & Co. KG, Tuttlingen, Germany) was extracted for analysis. All selected videos recorded the entire process from setting up the working channel to closing the incisions and were de-identified by assigning a new ID that was not linked to a patient's name or hospital ID. The frame width of all selected videos was 1920 pixels, and the frame height was 1080 pixels. Each video was filmed at approximately 20 frames per second. For subsequent analysis and processing, we cut each video into a series of key frames at a resolution of 1 frame per second.

Identifying blood pixels from surgical videos

Given a surgical video with red, green and blue (RGB) encoding, let $\left({\mathit{Pixel}}_R,{\mathit{Pixel}}_G,{\mathit{Pixel}}_B\right)$ be the intensity vector of the red, green and blue channels of a particular pixel, and let $\left({\mathit{Frame}}_R,{\mathit{Frame}}_G,{\mathit{Frame}}_B\right)$ be the counterpart of the frame containing the pixel. A classic way to identify blood pixels from the video is the method based on dynamic thresholding proposed by Garcia-Martinez et al. [26], which classifies a particular pixel with

$${\mathit{Pixel}}_{B/R}=\frac{{\mathit{Pixel}}_B}{{\mathit{Pixel}}_R}\mathrm{ }\mathrm{and}\mathrm{ }{\mathit{Pixel}}_{G/R}=\frac{{\mathit{Pixel}}_G}{{\mathit{Pixel}}_R},$$

and claims that the pixel is a blood pixel if the following 2 conditions are satisfied:

${\mathit{Pixel}}_{B/R}<{\tau }_{B/R}=a\cdot {\mathit{Frame}}_{B/R}+b\mathrm{ }\mathrm{and}\mathrm{ }{\mathit{Pixel}}_{G/R}<{\tau }_{G/R}=a\cdot {\mathit{Frame}}_{G/R}+b,$ where $a$ and $b$ are parameters that can be specified based on training data, ${\mathit{Frame}}_{B/R}=\frac{{\mathit{Frame}}_B}{{\mathit{Frame}}_R}\mathrm{ }\mathit{and}\mathrm{ }{\mathit{Frame}}_{G/R}=\frac{{\mathit{Frame}}_G}{{\mathit{Frame}}_R}$ being the colour ratio baselines of the corresponding frame. Please note that the involved thresholds, i.e. ${\tau }_{B/R}$ and ${\tau }_{G/R},$ are dynamic in nature, because ${\mathit{Frame}}_{B/R}$ and ${\mathit{Frame}}_{G/R}$ vary across different frames. Equipped with the optimized model parameters obtained based on a training dataset ${\mathcal{D}}_t$ composed of 10,000 blood pixels and 10,000 non-blood pixels randomly selected from 60 frames belonging to different patients, the Garcia-Martinez method performs fairly well in general (specificity = 98.9%, sensitivity = 98.8% and accuracy = 98.85%) according to an independent validation dataset ${\mathcal{D}}_v$ composed of 1,000 blood pixels and 1,000 non-blood pixels [26]. However, we also observed that the Garcia-Martinez method tends to miss many blood pixels once the blood stains are located in the shadow of surgical instruments, possibly due to the presence of a shadow that may greatly reduce the intensity of the red channel and make the thresholds unstable.

We further improved the foregoing method by claiming a blood pixel based on both dynamic and static thresholds as follows:

$${\mathit{Pixel}}_{\mathit{Gray}}<{\tau }_Y,\mathrm{ }\mathit{Pixe}{l}_{\mathit{Red}}>{\tau }_R\mathrm{ }\mathrm{and}\mathrm{ }{\mathit{Pixel}}_{\mathit{Gray}/R}<{\tau }_{Y/R}=\alpha \cdot {\mathit{Frame}}_{\mathit{Gray}/R}+\beta ,$$

where ${\mathit{Pixel}}_{\mathit{Gray}}$ and ${\mathit{Pixel}}_{\mathit{Red}}$ stand for the grey scale and red channel intensity of the pixel,

$${\mathit{Pixel}}_{\mathit{Gray}/R}=\frac{\mathit{Pixe}{l}_{\mathit{Gray}}}{\mathit{Pixe}{l}_{\mathit{Red}}},\mathrm{ }\mathit{Fram}{e}_{\mathit{Gray}/R}=\frac{\mathit{Fram}{e}_{\mathit{Gray}}}{\mathit{Fram}{e}_{\mathit{Red}}},$$

${\tau }_Y$ and ${\tau }_R$ are 2 static thresholds, and ${\tau }_{Y/R}$ is 1 dynamic threshold with $\alpha$ and $\beta$ being control parameters. We found that settings ${\tau }_Y=80,\mathrm{ }{\tau }_R=14,\mathrm{ }\alpha =0.38\mathrm{ }\mathrm{and}\mathrm{ }\beta =0.24$ optimize the performance of the foregoing rules on the training dataset ${\mathcal{D}}_t$ and result in improved performance (specificity = 98.9%, sensitivity = 99.2% and accuracy = 99.05%) on the validation dataset ${\mathcal{D}}_v$ (Fig. 2). A graphical illustration of the improved method is shown in Fig. 3.

Figure 2:

Blood pixel recognizer and determination of its parameters. RBG: red, blue, green.

Figure 3:

Graphical illustration of the bleeding feature extraction method. PBP: proportion of blood pixels; SPBP: sum of PBP of the entire operation (SumPBP); SPBPF, sum of PBP from flushing to the end of the operation (SumPBP-Flushing).

Measuring bleeding intensity at multiple levels

For each keyframe of a surgical video, we summarized its bleeding intensity by the proportion of blood pixels (PBP). Similarly, we summarized the overall bleeding intensity of the operation by the sum of the PBP (SumPBP), which sums the overall frame-level PBPs in the video and thus considers both the average bleeding intensity and the duration of the operation. Apparently, a larger PBP or SumPBP indicates more serious bleeding in a frame or an operation. Moreover, because bleeding at the final stage of the operation may have a direct impact on postoperative outcomes, we also defined the sum of PBP from flushing to the end of the operation (referred to as SumPBP-Flushing) as an alternative measurement to reflect an additional perspective of the bleeding situation during the procedure. In the following discussion, we try to associate these CV-driven measurements with postoperative outcomes via statistical analysis.

Statistical analyses

Numerical variables are presented as median values (interquartile ranges). Categorical data are expressed as numbers and percentages. Univariable and multivariable linear regression analyses were used to explore the correlation between preoperative variables and drainage volume and bleeding volume. In view of the model assumption of multivariable linear regression analysis, postoperative drainage volume was transformed to log equivalents. Additionally, we applied a Box-Cox transformation to intraoperative bleeding volume to make the model obey the normality assumption of linear regression analysis. Therefore, we transformed the bleeding volume to ${-{10}^4\times \left(\mathrm{bleeding}\mathrm{ }\mathrm{volume}\right)}^{-1/5}$. For convenience of display, we use ${-{10}^4\times \left(\mathrm{bleeding}\mathrm{ }\mathrm{volume}\right)}^{-1/5}$ as the dependent variable.

The tube indwelling duration data are not normally distributed even after log conversion. For prolonged tube indwelling duration, independent samples t-tests were used to assess differences in numerical variables between 2 groups; categorical variables were compared in 2 groups using the χ² test. If the frequency of an observation was less than 5 in the contingency table, Fisher’s exact test was used instead. To determine the independent predictor of prolonged tube indwelling duration, the variables with P < 0.2 in univariable analysis were selected for binary logistic regression analysis. A bilateral P value < 0.05 was considered statistically significant. All statistical analyses were performed using SPSS Software v.26.0 (IBM Corporation, Armonk, NY, USA).

RESULTS

Patient characteristics

In total, 275 patients and their surgical videos met the inclusion criteria. Among included patients, 134 (48.70%) were male. The patients included in the study were predominantly older, with a median age at surgery of 62 [interquartile range (IQR) 53–67]. No patients had to be returned to the operating room. The median operation duration was 109.70 min (IQR 90.25–137.96). The median postoperative drainage volume was 410 ml (IQR 275–750). Eighteen (6.5%) patients had a prolonged tube indwelling duration, and all of their drainage was light-red bloody. The median intraoperative bleeding volume was 30 ml (IQR 20–50). A total of 128 (46.5%) patients had immediate postoperative air leakage. The median SumPBP and median SumPBP-Flushing were 1016.24 (IQR 704.98–1787.41) and 73.92 (IQR 39.51–141.61), respectively (Table 1).

Table 1:

Patient characteristics

Characteristics	No. (%)/median (IQR)
Age at surgery, years	62 (53.00–67.00)
Gender
Male	134(48.70)
Female	141(51.30)
BMI (kg/m2)	24.11(21.78-25.95)
Smoker	88(32.00)
Hypertension	107(38.9)
Diabetes mellitus	38(13.8)
Coronary artery disease	21(7.6)
Cerebrovascular disease	18(6.5)
Laterality
Right upper lobe	95(34.5)
Right middle lobe	28(10.2)
Right lower lobe	55(20.0)
Left upper lobe	49(17.8)
Left lower lobe	48(17.5)
Clinical stage
IA1	19(6.9)
IA2	96(34.9)
IA3	70(25.5)
IB	41(14.9)
IIA	14(5.1)
IIB	9(3.3)
IIIA	19(6.9)
IIIB	3(1.1)
Clinical T stage
T1a	18(6.5)
T1b	96(34.9)
T1c	77(28.0)
T2a	54(19.6)
T2b	16(5.8)
T3	11(0.0)
T4	1(0.4)
Clinical N stage
N0	245(89.1)
N1	8(2.9)
N2	18(6.5)
Bleeding volume	30(20-50)
Immediate postoperative air-leakage	128(46.5)
Surgery duration, min	109.70(90.25-137.96)
SumPBP	1016.24 (704.98-1787.41)
SumPBP-Flushing	73.92(39.51-141.61)
Drainage volume (ml)	410(275-750)
Prolonged tube indwelling duration	38(13.8)
Histological subtype
Adenocarcinoma	224(81.5)
Squamous	23(8.4)
Large cell, mixed, other	28(10.2)

N = 275.

BMI: body mass index; IQR: interquartile range; PBP: proportion of blood pixels; SumPBP: sum of PBP; SumPBP-Flushing: sum of PBP from flushing to the end of the operation.

Association between proportion of blood pixels and postoperative drainage volume

We used univariable linear regression analysis to analyse the correlation between preoperative variables and drainage volume. From Table 2, we can see that age at surgery (P< 0.001), sex (P< 0.001), immediate postoperative air leakage (P< 0.001), duration of surgery (P< 0.001), SumPBP (P< 0.001), SumPBP-Flushing (P< 0.001), clinical N stage (P = 0.040) and bleeding volume (P< 0.001) were positively correlated with postoperative drainage volume. In the multivariable linear regression analysis, age at surgery (P = 0.005), immediate postoperative air leakage (P < 0.001), surgical duration (P = 0.001), bleeding volume (0.033) and SumPBP-Flushing (P < 0.022) were significant predictors of drainage volume with $R^2=0.228$.

Table 2:

Univariable analysis with proportion of blood pixels and drainage volume; multivariable linear regression model with proportion of blood pixels and log (drainage volume) (model R² = 0.228)

Characteristics	Univariable analysis			Multivariable linear regression analysis
	Regression Coefficient(β)	95% CI	P-value	Regression Coefficient(β)	95% CI	P-value
Age at surgery	10.786	5.803-15.769	< 0.001	0.005	0.001-0.008	0.005
Male	−228.153	−329.657-126.650	< 0.001
Immediate postoperative air leakage	250.608	150.735-350.482	< 0.001	0.130	0.061 ∼ 0.199	< 0.001
Surgery duration	4.099	2.977-5.220	< 0.001	0.001	0.001 ∼ 0.002	0.001
SumPBP	0.162	0.108-0.215	< 0.001
SumPBP-Flushing	1.196	0.843-1.548	< 0.001	< 0.001	< 0.001 ∼ 0.001	0.022
Clinical N stage	106.706	5.160-208.252	0.040
Bleeding volume	1.563	0.917-2.208	< 0.001	< 0.001	< 0.001 ∼ 0.001	0.033

PBP, proportion of blood pixels; SumPBP: sum of PBP; SumPBP-Flushing: sum of PBP from flushing to the end of the operation.

Association between proportion of blood pixels and prolonged tube indwelling duration

The patients with prolonged tube indwelling duration were significantly older (P = 0.047), and the proportion of male patients and immediate postoperative air leakage were significantly higher than those of patients with tube-carrying times less than 5 days (P = 0.023, P < 0.001, respectively). In addition, the difference between the 2 groups also achieved statistical significance in surgery duration, Sum-PBP and SumPBP-Flushing (P = 0.001, P = 0.006 and P < 0.001, respectively). After being adjusted using binary logistic regression analysis, only immediate postoperative air leakage and SumPBP-Flushing remained in the model [P < 0.001, odds ratio (OR) 4.616, 95% confidence interval (CI) 1.964–10.847, P = 0.017, OR 1.003, 95% CI 1.000–1.005) (Table 3).

Table 3:

Univariable analysis and binary logistic regression analysis between characteristics and prolonged tube indwelling duration

Characteristics	Univariable analysis			Binary logistic regression analysis
	Prolonged tube indwelling duration (n = 38)	Non-prolonged tube indwelling duration (n = 237)	P-value	OR	95% CI	P-value
Age at surgery	62.97 ± 11.36	59.43 ± 9.97	0.047a
Male	25(65.8)	109(46.0)	0.023b
BMI	23.38 ± 3.33	24.22 ± 3.30	0.143a
Immediate postoperative air leakage	29(78.4)	99(41.8)	< 0.001b	4.616	1.964-10.847	< 0.001
Surgery duration	139.43 ± 60.10	115.11 ± 38.74	0.001a
SumPBP	1716.10 ± 1154.18	1272.14 ± 872.60	0.006a
SumPBP-Flushing	192.85 ± 263.00	102.76 ± 101.11	< 0.001a	1.003	1.000-1.005	0.017
Histological subtype			0.056c
Adenocarcinoma	31(81.6)	193(81.4)
Squamous	6(15.8)	17(7.2)
Large cell, mixed, other	1(3.6)	27(11.4)
Bleeding volume	73.42 ± 85.57	49.96 ± 76.80	0.086a

^aIndependent samples t-test.

^bχ² test.

^cFisher’s exact test.

BMI: body mass index; CI: confidence interval; OR: odds ratio; PBP: proportion of blood pixels; SumPBP: sum of PBP; SumPBP-Flushing: sum of PBP from flushing to the end of the operation.

Association between proportion of blood pixels and intraoperative bleeding volume

In univariable linear regression analysis, age at surgery (P = 0.013), sex (P < 0.001), immediate postoperative air leakage (P = 0.008), surgery duration (P < 0.001), SumPBP (P < 0.001) and SumPBP-Flushing (P = 0.001) were positively correlated with intraoperative bleeding volume (Table 4). Next, we used multivariate linear regression analysis to study the factors related to the bleeding volume. The results showed that surgical duration (P < 0.001) and SumPBP (P = 0.005) were significant predictors of bleeding volume with $R^2=0.273$ (Table 4).

Table 4:

Univariable linear analysis with proportion of blood pixels and bleeding volume and multivariable linear regression model with proportion of blood pixels and $10000\times \mathrm{ }{\left(\mathrm{bleeding}\mathrm{ }\mathrm{volume}\right)}^{-1/5}$ (model r² = 0.273)

Characteristics	Univariable analysis			Multivariable linear regression analysis
	Regression coefficient(β)	95% CI	P value	Regression coefficient(β)	95% CI	P value
Age at surgery	10.952	2.345-19.560	0.013
Male	360.72	188.224-533.213	<0.001
Immediate postoperative air leakage	237.23	61.350-413.109	0.008
Surgery duration	0.140	0.110-0.170	<0.001	0.097	0.053-0.141	<0.001
SumPBP	0.364	0.280-0.448	<0.001	0.171	0.053-0.289	0.005
SumPBP-Flushing	1.029	0.404-1.653	0.001

CI: confidence interval; PBP: proportion of blood pixels; SumPBP: sum of PBP; SumPBP-Flushing: sum of PBP from flushing to the end of the operation.

DISCUSSION

We found that surgical duration and SumPBP-Flushing were independent predictors of postoperative drainage volume and prolonged tube indwelling duration. Surgical duration and SumPBP were significantly correlated with bleeding volume. The results suggested that high throughput CV algorithms and computer-detected visual features could be utilized to predict postoperative outcomes of thoracoscopic surgery.

The drainage volume is one of the decisive factors in deciding whether to remove the drainage tube. In this study, we extracted 2 variables, SumPBP and SumPBP-Flushing; the last variable was the only variable that had a significant correlation with drainage volume. Before flushing with water, surgeons dealt with bleeding points carefully and thoroughly, which may be the reason for excluding SumPBP after multivariable linear regression analysis. Next, we can build a sample set of surgical videos with high-level PBP, from which surgeons would learn how to avoid bleeding and perfect their techniques.

We chose to use simple pixel-based methods instead of more sophisticated pixel-based methods [25, 27], feature-based methods [22] or neural network-based methods [28, 29] for blood stain recognition, mainly in consideration of the computational costs. A typical surgical video in our study exceeds 2 h and thus is composed of more than 2 × 60×60 = 7,200 images, each of which contains 1920 × 1080 pixels, resulting in more than 150 billion pixels per video. Because sophisticated methods typically involve expensive non-linear operations, it takes a long time (i.e. a couple of weeks) for them to process such a huge number of pixels, which is clearly unfeasible in practice. To the best of our knowledge, the pixel-based method based on dynamic thresholding [26], which involves just straightforward linear operations, is the only available method that is computationally feasible for this study. By enhancing the dynamic thresholding approach in reference [26] with extra static thresholds on additional colour channels, we came up with an improved approach in this study that enjoys both high accuracy and computational feasibility. The new method also reduced the influence caused by shadows.

The vulnerability of all CV-based techniques is intrinsic to the nature of the quality of the surgical video. One of the most important factors is the stability of the camera. Unlike laparoscopic surgery, perspective changes occur more frequently in thoracoscopic surgery. Except for robotic surgery, a camera shake is inevitable, even for a qualified assistant. A sudden movement of the camera would require a false positive prediction to identify reddish tissue as blood stains or a false negative prediction when the target is out of view. We speculate that this is why the current research on CV in clinical medicine is focused mainly on laparoscopic surgery. Additionally, the PBP value may be influenced by the intensity of the light. If the light is too weak, the picture quality decreases, but if the light is too strong, the white reflections of blood stains increase the difficulty of identifying blood. This vulnerability is intrinsic to the nature of surgical videos recorded by the camera.

In addition to the quality of the videos, the algorithm can only detect blood stains that occur within the view of the camera. Another difference from laparoscopic surgery is that the lens is much closer to the specific target in thoracoscopic surgery, which means that the field of view is smaller. Obviously, the smaller the field of view is, the less information there will be. We can use the size of devices and the ratio of devices to the blood stain diameters to measure the blood stain area more accurately and to reduce the impact of the camera position on the PBP value. The algorithm will be extremely complicated and is still in the exploratory stage. These challenges just mentioned highlight the fact that the clinical application of surgical vision is still in the early stages of development.

To the best of our knowledge, no research efforts have yet been exerted on analysing videos of thoracoscopic operations via CV algorithms. Our purpose was to bridge this gap and analyse the correlation between CV and postoperative outcomes in thoracoscopic surgery. Due to the correlation between the PBP value and drainage volume, prolonged tube indwelling duration and bleeding volume, especially the correlation between PBP and bleeding volume, the algorithm might have great potential for clinical applications. The PBP value has the potential to be used as one of the reference indices for postoperative fluid management, especially for older patients and patients with poor heart, lung or kidney function. In addition, the algorithm has the potential to help us better predict postoperative recovery times and improve bed utilization. The PBP value also has the potential to help us adjust postoperative anticoagulation strategies. If the PBP value of the patient is too high, we might have more confidence not to give anticoagulant drugs or to reduce the dose of anticoagulant drugs. The PBP value may help us better choose the suction power, diameter and number of drainage tubes. The algorithm itself cannot assess surgical skills. However, this goal might be reached in the near future by combining this study’s algorithm with our ongoing surgical skills assessment research.

It was beyond the scope of this study to assess the predictive power of other outcomes, and we will include other variables to enhance the ability of this algorithm. In the future, we might be able to predict many detailed conditions of the patient immediately after the operation. To be bolder, AI algorithms might be able to guide surgeons during surgery in real time in the future just like an expert looking over their shoulder.

CONCLUSIONS

In our study, we showed that SumPBP-Flushing was a significant predictor of drainage volume and prolonged tube indwelling duration (P < 0.022; P = 0.017, OR 1.003, 95% CI 1.000–1.005, respectively) and that SumPBP was significantly correlated with intraoperative bleeding volume (P = 0.005). We successfully proved our hypothesis that CV algorithms can effectively extract potential signals from videos of thoracoscopic operations and that these signals have good predictive power for postoperative outcomes.

Funding

This work was supported by the National Key Research and Development Program [2020AAA0109600]; Beijing Natural Science Foundation [Z190021]; National Natural Science Foundation of China [61877001, 11771242]; Strategic Research and Consulting Project of Chinese Academy of Engineering [2021-XZ-11]; Peking University Baidu Fund [2020BD013] and Advanced Institute of Information Technology, Peking University, Zhejiang Province [2020-Z-17].

Conflict of interest: The authors have no conflicts of interest to disclose.

Data Availability Statement

The data underlying this article will be shared on reasonable request to the corresponding author.

Author contributions

Jian Zhou, Jun Wang: Conceptualization; Data curation; Funding acquisition; Investigation; Project administration; Supervision; Validation; Writing-review & editing. Ke Deng: Conceptualization; Data curation; Investigation; Project administration; Supervision; Validation; Writing-review & editing. Hao Xu, Tingxuan Han: Formal analysis; Investigation; Methodology; Resources; Software; Validation; Writing-original draft. Guanchao Jiang, Fan Yang: Investigation; Project administration; Supervision; Validation. Haifeng Wang, Shanggui Liu, Guanghao Hou, Lina Sun: Investigation; Resources.

Glossary

Abbreviations

AI

Artificial intelligence

CV

Computer vision

IQR

Interquartile range

PBP

Proportion of blood pixels

SumPBP

Sum of PBP

SumPBP- Flushing

Sum of PBP from flushing to the end of the surgery

VATS

Video-assisted thoracoscopic surgery