Combined indocyanine green and quantitative perfusion assessment with hyperspectral imaging during colorectal resections

Abstract

Anastomotic insufficiencies still represent one of the most severe complications in colorectal surgery. Since tissue perfusion highly affects anastomotic healing, its objective assessment is an unmet clinical need. Indocyanine green-based fluorescence angiography (ICG-FA) and hyperspectral imaging (HSI) have received great interest in recent years but surgeons have to decide between both techniques. For the first time, two data processing pipelines capable of reconstructing an ICG-FA correlating signal from hyperspectral data were developed. Results were technically evaluated and compared to ground truth data obtained during colorectal resections. In 87% of 46 data sets, the reconstructed images resembled the ground truth data. The combined applicability of ICG-FA and HSI within one imaging system might provide supportive and complementary information about tissue vascularization, shorten surgery time, and reduce perioperative mortality.

1. Introduction

During colorectal surgery, anastomotic insufficiencies represent a frequent complication, occurring in 2.7% to 19.2% of cases [1–3]. They can lead to longer hospital stays, higher treatment costs, a more complex disease course, and a higher mortality rate [4–6]. Their development is influenced by various factors such as obesity, intraoperative difficulties, male sex associated with a narrow pelvis and more challenging rectal resections, preoperative total protein concentration in serum, anticoagulant treatment, and bowel perfusion after resection [4,7–9]. Bowel perfusion is highly affected by the position of the resection margin. This is determined subjectively according to the clinical appearance of the bowel and depends on the surgeon’s experience. Objectification might help to prevent the anastomosis from being located in a poorly perfused bowel area. Therefore, there has been considerable interest in recent years in supportive imaging modalities for perfusion measurement. The focus of the current paper is on indocyanine green (ICG)-based fluorescence angiography (FA) and hyperspectral imaging (HSI).

ICG is a fluorescent agent that binds to plasma proteins. The absorption maximum is near the isosbestic point of oxygenated and deoxygenated hemoglobin (≈ 800 nm), depending on the solvent and concentration [10]. It emits light at higher wavelengths in the near-infrared (NIR) spectral range. The maximum fluorescence signal is present at ≈ 840 nm [11]. For imaging, ICG is dissolved in isotonic saline, administered intravenously, and excited at its absorption maximum, for example, with a laser. An infrared camera detects the fluorescence intensity in real-time, which correlates with tissue perfusion [12].

In conventional systems, the fluorescence intensity is usually displayed as a grayscale video or as an overlay to a red, green, blue (RGB) video in shades of green or blue. But pure intensity values are hardly comparable between patients, e.g., due to their dependence on laser excitation intensity, camera distance, and angle to the examined object. Approaches using time-dependent features from the fluorescence-time curve instead of raw intensities seem to fulfill the need for quantitative and objective perfusion assessment quite well [12].

The latter is already possible with medical HSI. An object is illuminated with a broadband light source. Reflectances are measured with an image sensor in a variety of bands of the electromagnetic spectrum, mainly in the visual and NIR range from 400 to 1000 nm. Depending on the tissue composition, light interacts diversely with its components, including absorption, scattering, or reflection processes. Thus, the measured spectral information is like the fingerprint of the tissue. It allows the calculation of physiological parameters, e.g., the perfusion status, for which the absorption characteristics of oxygenated and deoxygenated hemoglobin are used. HSI does not require an exogenous fluorophore that alters the optical properties of the object. It is therefore repeatable.

ICG-FA has been used for intraoperative assessment of vascularization of colorectal bowel segments [13–15]. Across the studies, the number of anastomotic insufficiencies was lower in the groups with intraoperative imaging compared with the control groups. Larger cohorts are needed to test a significant benefit of imaging. Compared with angiography and despite the advantage of being a non-invasive procedure, HSI has been much less studied in colorectal resections. However, it has been able to differentiate between well- and poorly perfused colorectal segments before resection [16]. In recent years, only three comparative studies between HSI and ICG-FA have appeared to assess perfusion [17–19]. The authors demonstrated that both imaging modalities can support perfusion assessment and concluded that a combination of both modalities could provide complementary information.

The combined application requires two camera systems and additional operating time since it is mandatory to exchange them to image the same region of interest. Reconstruction of the ICG signal from hyperspectral reflectance data would enable the combination with only one system and would save the resources mentioned above. For the first time, hyperspectral data collected during colorectal resections were retrospectively analyzed to determine spectral features that correlate with information obtained from ICG-FA. Two signal processing pipelines were developed to utilize these spectral features and allow visualization of the reconstructed signal. The results were compared to the original ICG-FA data and physiological parameters provided by the HSI system.

2. Methods

2.1. Patients and surgical procedure

The present study was performed at the University Hospital of Leipzig, Germany. Between January 2020 and January 2021, 128 patients underwent colorectal resections. Due to the SARS-CoV2 pandemic, only cancer patients and patients with exacerbated inflammatory bowel disease underwent surgery. After excluding patients with non-left colorectal resections, recurrent disease, multivisceral resections, a known intolerance to ICG, patients under 18 years of age, pregnant patients, or patients in whom intraoperative ICG-FA or HSI was not available, 62 patients remained for the clinical trial. The study was registered at clinicaltrials.gov (NCT04226781) on 13/01/2020 and approved by the local ethics committee of the Faculty of Medicine of the Leipzig University (No. 026/18-ek) on 31/01/2018. All patients provided written informed consent. All methods were performed in accordance with the declaration of Helsinki.

Postoperatively, 16 patients had to be excluded because the boundary zone was outside the imaged area (n = 12) or because the data collection deviated from the scheme shown in Fig. 1 (n = 4). In the end, the cohort consisted of 46 patients (females: n = 15; 32.6%; males: n = 31; 67.4%) with carcinoma (n = 35; 76.1%), complex diverticulitis (n = 10; 21.7%), or inflammatory bowel disease (n = 1, 2.2%). The median age of patients was 63 years and ranged from 25 to 88 years. Comorbidities included cardiovascular disease (n = 30; 65.3%), metabolic disease (n = 22; 47.8%), respiratory disease (n = 9; 19.6%), and alcohol or nicotine consumption (n = 8; 17.4%). Ten patients (23.3%) were not taking any medications. Seventeen patients (37%) had received preoperative neoadjuvant therapy.

Fig. 1.

Schematic representation of the data acquisition process during colorectal surgery. Measurement of the pre-cube was performed with the TIVITA Tissue (Diaspective Vision, Am Salzhaff, Germany) prior to administration of ICG. Real-time recording of the fluorescence signal of ICG after its injection was performed using the EleVision IR Platform (Medtronic, Minneapolis, USA). The post-cube containing spectral information of ICG was acquired afterward.

Operations were performed laparoscopically (with and without robotic assistance) (n = 42; 91.3%) or openly because of prior abdominal surgery or large tumor size (n = 4; 8.7%). A small salvage incision was made for ante corpore image acquisition during laparoscopic surgery. After cutting the marginal artery and before the actual resection, imaging was performed to verify tissue perfusion. For ICG-FA, 35 patients (76.1%) received 2.5 mg ICG and 11 patients (23.9%) received 5 mg ICG absolute to investigate the dose dependence of the results. Since the maximum daily dose recommended by the manufacturer is 5 mg/kg, patients were not exposed to any major risk in this case.

2.2. Data acquisition and annotation

The hyperspectral camera TIVITA Tissue (Diaspective Vision, Am Salzhaff, Germany) and the EleVision IR Platform (Medtronic, Minneapolis, USA) were used for data acquisition. The TIVITA Tissue is a push broom scanning system that measures spectral reflectance data R(x,y,λ), called hypercube, of 640 × 480 pixels (x,y) from 500 to 1000 nm (λ) with a spectral resolution of 5 nm. The basic techniques of hyperspectral imaging and a detailed description of the TIVITA Tissue is described elsewhere [20–22]. It provides an RGB image and five false-color parameter images: the oxygen saturation of tissue (StO₂), the perfusion of deeper blood vessels (NIR Perfusion Index [NIR PI]), the water and lipid content in tissue (Tissue Water Index [TWI] and Tissue Lipid Index [TLI], respectively), and hemoglobin levels in organs (Organ Hemoglobin Index [OHI]) [23–25]. The EleVision IR Platform served for fluorescence imaging, exciting the ICG molecules at 805 nm and providing video data with a temporal resolution of 22 ms and an image size of 960 × 720 pixels [26]. The bit depth of all acquired images was 8 bit.

The images were taken as shown in Fig. 1. The surgeon established a transection line immediately prior to bowel resection. At this time, both imaging systems were used to assist in decision making. First, the HSI camera was placed at a distance of 50 cm from the colon. The acquired and so-called pre-cube R_pre(x,y,λ) included spectral reflectance data. After switching the camera systems, but simultaneously with the administration of ICG, continuous measurement of the fluorescence intensity began in auto exposure mode. Again, the distance between the camera and the object was 50 cm. Video duration differed between patients because of discrepancies in the clinical evaluations of the videos made by different surgeons. After the renewed exchange of the systems and before the surgical procedure was continued, the post-cube R_post(x,y,λ) was acquired analogously to the pre-cube. This time the spectral data was affected by the ICG.

To identify imaged areas that may provide relevant information for ICG signal reconstruction from hyperspectral data, an annotation process was conducted after data acquisition using TIVITA Suite (Diaspective Vision GmbH, Am Salzhaff, Germany) and ImageJ (Wayne Rasband and contributors, National Institutes of Health, USA). In StO₂ parameter images as well as in the last frame of the ICG video, five circular markers were placed on both sides of a tweezer (see Fig. 2). The tweezer indicated the boundary between high- and low-perfused areas and was positioned by the surgeon based on visual inspection of the tissue before data acquisition started. The spectral information of pixels that were within the edges of the markers thus represented high- and low-perfused tissue according to clinical judgment. Spectral information from tissue influenced and uninfluenced by ICG was also obtained by annotating images from pre- and post-cubes. Marker positions corresponded anatomically in all images.

Fig. 2.

A) RGB image of the colon examined, acquired with the EleVision IR Platform. A ruler and tweezer were used to mark the boundary between high- and low-perfused areas after visual assessment by the surgeon. The colored boxes correspond to the image sections shown in B through D. B) StO₂ parameter image generated from the pre-cube. Five markers were placed on both sides of the tweezer. Marker i is located in the low-perfused area and marker ii in the high-perfused area. C) Anatomically corresponding markers in the last frame of the ICG video. Mean fluorescence intensity is indicated for markers i and ii (min. intensity = 0, max. intensity = 255). D) Again, anatomically corresponding markers to B and C in the StO₂ image calculated from the post-cube.

2.3. ICG reconstruction methods

To reconstruct the ICG signal from hyperspectral data, the influence of ICG on the data must be considered. First, a spectral analysis of mean marker spectra was performed. The mean marker spectra are the spectral reflectances in the 500 to 1000 nm range that were spatially averaged over the pixels within the marker edges. They were calculated for each marker and for both pre- and post-cubes. Additionally, these mean marker spectra were assigned the mean ICG fluorescence intensity from the last frame of the ICG video, ranging from zero to 255. An example of the mean spectral reflectances and assigned ICG values for markers i and ii as shown in Fig. 2 can be found in Fig. 3(A).

Fig. 3.

A) Spatially averaged reflectance spectra for markers i and ii (see Fig. 2) from pre- (ICG = 0) and post-cubes (ICG≠0). The reflectance data of both cubes of marker i, which is located in the low-perfusion area that was almost unaffected by ICG show no significant differences. In contrast, the reflectance of marker ii is lower in the spectral range around 800 nm when ICG is present. However, the differences between high- and low-perfused regions (marker i vs. marker ii) are much larger than between ICG presence and absence (marker ii: ICG = 0 vs. ICG = 168). B) First derivation of the differential reflectance spectrum of markers i and ii. For tissue unaffected by ICG (marker i), the signal runs around the zero line. If ICG is present (marker ii), a local minimum and maximum form around 780 and 820 nm, respectively. C) Second derivation of the reflectance data of the two markers from the pre- and post-cubes. In presence of ICG (darker green line), the mean is much higher in the spectral range from 790 to 810 nm. In contrast to A, the difference between ICG presence and absence is much higher than between high- and low-perfused areas.

2.3.1. Spectral analysis

Considering the light absorption and emission characteristics of ICG, the spectral data from 760 to 840 nm were the focus of spectral analysis. In addition to the pure reflectance data R(x,y,λ) of pre- and post-cubes, the first and second derivatives and the differential reflectance spectrum R_diff [Eq. (1)] of pre- and post-cubes were also investigated.

$$R_{\mathit{\text{diff}}}(x,y,\lambda )=R_{post}(x,y,\lambda )-R_{pre}(x,y,\lambda )$$ (1)

The differential spectrum was used because if the data acquisition of both cubes was done under the same conditions and the only difference was that the data in the post-cube was affected by the ICG, the difference of both cubes would correlate with the ICG signal.

From differences in this spectral data originating from high- and low-perfused tissue and from tissue influenced and uninfluenced by ICG, twelve spectral features were extracted (see supplemental Table S1). These represent minimum, maximum, or mean values of spectral data or their differences in several spectral ranges.

Spectral features based on the differential reflectance spectrum of both cubes formed the trivial approach for ICG signal extraction. However, the use of both cubes requires a registration process as described below and a higher measurement effort in clinical applications. Therefore, an attractive alternative for ICG signal reconstruction are spectral features calculated exclusively from the post-cube.

After a quantitative comparison of all features as described in the supplementary material, one spectral feature per approach emerged as the most promising candidate. These are (a) the difference of maximum and minimum in several spectral ranges of the first derivative of the differential reflectance spectrum of the pre- and post-cubes (Fig. 3(B)) and (b) the mean value in the spectral range from 790 to 810 nm of the second derivative of the reflectance spectrum in the post-cube (Fig. 3(C)). An overview of the two spectral features is depicted in Table 1. After selecting these two spectral features for ICG signal reconstruction, the described minimum, maximum, and mean values could be calculated using the full hyperspectral data from pre- and post-cubes (the raw data) instead of the mean marker spectra that resulted from data annotations.

Table 1. Calculation of spectral features from reflectance data R with $\dot{\mathbf{R}}=dR/d\lambda$ and $\ddot{\mathbf{R}}={\mathbf{d}}^2\mathbf{R}/d{\lambda }^2$ .

	Reflectance Data R	Value	Spectral Range in nm
(a)	${\dot{R}}_{diff}(x,y,\lambda )$	$max(R_{\lambda 1})-min(R_{\lambda 2})$	$\begin{array}{l}\{{\lambda }_1\in \mathbb{Z}|810\le {\lambda }_1\le 830\}\\ \{{\lambda }_2\in \mathbb{Z}|770\le {\lambda }_2\le 790\}\end{array}$
(b)	${\ddot{R}}_{post}(x,y,\lambda )$	$\text{mean}(R)$	$\{\lambda \in \mathbb{Z}|790\le \lambda \le 810\}$

2.3.2. Difference method

For annotated data, the spectral information originated from corresponding anatomical regions. For raw data, R_diff could not be derived directly from Eq. (1). This was due to repositioning of the camera described in Sec. 2.2. Between the two measurements with the TIVITA Tissue that yielded in R_pre and R_post, the system was replaced with the EleVision IR Platform for fluorescence imaging. As a result, imaged areas may not have been identical (see Fig. 2(A)), and a registration process was required. Twenty-six manually determined corresponding landmarks in RGB images generated from R_pre and R_post were used to calculate the homography for a perspective transformation.

The differential reflectance spectrum [Eq. (2)] and its first derivative were calculated from the registered post-cube R_reg and pre-cube R_pre for each pixel. This was followed by spectral smoothing by Gaussian filtering (σ=2) and determination of spectral feature (a) according to Table 1. Normalization to the 80^th percentile, spatial smoothing with a median kernel of size 5 × 5 pixels, and value rescaling to [0,1] completed the first processing pipeline for ICG signal reconstruction from hyperspectral data. Ongoing, this method is referred to as the difference method.

$$R_{\mathit{\text{diff}}}(x,y,\lambda )=R_{reg}(x,y,\lambda )-R_{pre}(x,y,\lambda )$$ (2)

2.3.3. Post-cube method

An alternative data processing resulted in the post-cube method. Here, only Gaussian filtered (σ=4.4) spectral data of the post-cube served for the calculation of spectral feature (b) (see Table 1), which made registration unnecessary. Normalization to the 90^th percentile, spatial smoothing, and rescaling of the values followed. The smoothing hyperparameters varied between the two methods because different signal with different noise components served as input.

2.3.4. Visualization

The results of both methods were imaged with three color maps (see Fig. 4). A simple gray scale from zero to 255 allowed direct comparison with the original ICG data. The reconstructed ICG signal was also overlaid with the RGB image in shades of green as provided by the EleVision IR Platform. The color coding in the range from zero to 100 matched the visualization technique of the TIVITA Tissue.

Fig. 4.

RGB (A), ICG (B), and StO₂ (C) image compared with reconstructed ICG signals (D-I) for one selected patient. A and C were generated from the post-cube (TIVITA Tissue). B represents the last frame of the ICG video (EleVision IR Platform). D-F show the results of the difference method, G-I those of the post-cube method. The reconstructed images were visualized with a green-shaded overlay of the RGB image (D, G), an 8-bit gray scale (E, H), and color-coding analogous to the TIVITA Tissue system (F, I). In A, the position of the fixed point P_F is shown as the tip of the tweezer. In B and C, the central and distal points (P_C and P_D, respectively) were additionally visualized. Both were used to determine the width of the boundary zone w_CD according to C.

2.4. Evaluation of ICG reconstruction methods

The initial evaluation of the two methods was qualitative. Clinicians visually compared the reconstructed ICG images and the last frame of the ICG video (the ground truth). The last frame was chosen because it was not possible to reconstruct an image that correlated with a previous measurement. Although the clinical relevance of the last frame may be less than that of the frame with the highest intensity, it is the best possible solution for reconstruction and therefore suitable for the evaluation of the reconstruction methods. For each method, a success rate was calculated by dividing the records in which the reconstructed ICG image matched the ground truth by the total number of records (n = 46).

In addition, the boundary zones between high- and low-perfusion tissue were determined and quantitatively compared between the ground truth data and the images resulting from the ICG reconstruction methods for those data sets where the ICG signal was successfully reconstructed.

Because the transition between (a) well-perfused and (b) ischemic areas was smooth, the boundary could not be assumed to be a sharp edge. Therefore, the maximum pixel value v_max in (a) and the minimum pixel value v_min in (b) were determined automatically for all images using custom LabView-based software (National Instruments, Austin, USA). Their mean value $\overline{v}=0.5(v_{max}+v_{min})$ was used to define the most central point ${\mathbf{P}}_C=(x_C,y_C,\overline{v})$ and the most distal point ${\mathbf{P}}_D=(x_D,y_D,\overline{v})$ of the boundary zone and the corresponding boundary lines perpendicular to the ruler in the imaged area (see Fig. 4(C)). The Euclidean distance between both boundary lines was defined as the width w_CD of the boundary zone. Furthermore, the Euclidean distances from P_C and P_D to an intraoperatively set fixed point P_F, i.e., the tip of the tweezer (see Fig. 4(A)-(C)), were determined as d_FC and d_FD, respectively. Distances became negative when the fixed point was more distal than P_C or P_D to indicate possible distal shifts of the boundary zone over time.

Because the time-dependent fluorescence signal of ICG is normally used to assess perfusion when it reaches its maximum value, comparison of the reconstructed ICG images with the last frame of the ICG video (ICG_end) is of little clinical relevance if there is a significant time delay between ICG_end and the frame with the highest ICG signal intensity (ICG_max). To interpret all results in that context, the boundary zones in ICG_end and ICG_max were compared using the same method. Furthermore, the time between these two frames was recorded to investigate the time dependence of the intensity of the contrast agent.

2.5. Influence of ICG on parameter images

To allow the combined use of ICG-FA and HSI regardless of the order, the influence of ICG on the physiological parameter images generated from reflectance spectra must be negligible. Otherwise, correction of the spectral data is essential. The TIVITA Tissue system provides the physiological parameters described in Sec. 2.2. The parameter images from the pre-cube were unaffected by ICG and formed the ground truth. Spectral data acquired after ICG administration (the post-cube) might lead to faulty physiological parameters.

To investigate the influence of ICG on physiological parameters, the parameter images of both cubes were registered after manual determination of 26 landmark pairs using perspective transformation. Pairwise difference images were calculated. For each difference image, the absolute mean value over all pixels was determined. Absolute means above 25.0% occurred only as a result of the registration process prior to difference image calculation and were not caused by ICG. This was verified by visual inspections of the difference images. Therefore, a value scaling of all absolute mean values to 25.0% was performed.

In addition to these technical evaluations, the boundary zones in StO₂ parameter images of pre- and post-cubes were examined analogously to measurements described in the previous section. The time between the two recordings was measured to check whether the boundary zones change in a clinically relevant way over time.

2.6. Statistics

Statistical analyses were performed in Excel 2013 (Microsoft Corporation, Redmond, USA) and IBM SPSS Statistics Standard v24 (IBM Corporation, Chicago, USA) to test for significant differences between the distances d_FC and d_FD and the widths of the boundary zones w_CD in ICG, reconstructed ICG, and StO₂ parameter images. Descriptive statistics results are presented as median values (minimum, maximum), absolute frequencies (relative frequencies in %), and mean (standard deviation [SD]).

Samples were first tested for normal distribution using the Kolmogorov-Smirnov test with a significance level of p = 0.05. A t-test and Wilcoxon signed-ranks test were then performed for dependent normally distributed samples and non-normally distributed samples, respectively. The null hypothesis for the t-test stated that the means between the two compared measurements were equal. For the Wilcoxon signed-ranks test, the median value was taken instead of the mean. For both tests, the significance level was set at p = 0.05.

Furthermore, the position changes of P_C and P_D in the StO₂ parameter images before and after ICG administration and the contrast deterioration between ICG_max and ICG_end were considered as a function of time. For the latter, the maximum and minimum intensity values in both images were determined. The correlation coefficient was calculated using Stearman's rank correlation (p = 0.05) because the samples were not normally distributed.

3. Results

3.1. Clinical results

The median hospital stay was 11.5 days (9-71 days). According to the Clavien-Dindo-classification [27], ten postoperative grade III-IV complications (21.7%) were noted. Six patients (13.0%) had anastomotic leakage and five other complications, such as right ureteral leakage, mechanical passage obstruction, left leg hypaesthesia, abdominal wound burst, and SARS-CoV2 infection.

3.2. ICG signal extraction from hyperspectral data

After visual inspection, a signal correlating with the ground truth (the last frame of the ICG video) could be reconstructed in 40 of 46 data sets using at least one of the two methods described in Sec. 2.3. Regardless of the ICG dose administered, the difference method performed better than the post-cube method in all 46 data sets (80.4% vs. 39.1%). The same could be observed in cases where 2.5 mg ICG was administered (n = 35, difference method: 85.7%, post-cube method: 25.7%). Here, the success rate of the difference method was even higher than in all data sets. In contrast, when considering data sets where 5 mg ICG was administered (n = 11), the ICG signal could be reconstructed by the post-cube method in 81.2%. The success rate of the difference method remained lower at 63.6%.

The reconstructed images from both methods through all visualization techniques, ground truth, and RGB and StO₂ parameter images from the post-cube are illustrated in Fig. 4 for one patient. Here, both methods were able to reconstruct the signal. However, the post-cube method (G-I) performed better than the difference method (D-F) compared with the ground truth data shown in B. Clinicians considered visualizations analogous to those of ICG-FA systems to be most intuitive (D, E, G, H). When HSI parameters were displayed (F, I), color coding provided high-contrasts. By augmenting the RGB images with the reconstructed signals in shades of green, organ structures and vascularization information could be imaged at a glance (D, G).

The results of the quantitative evaluations of the ICG reconstruction methods according to Sec. 2.4 can be seen in Fig. 5. The distances d_FC and d_FD and the width of the boundary zones w_CD were higher in the reconstructed ICG images when comparing the difference method with the ground truth data. The central point was on average 1.5 mm (SD: ± 10.4 mm) more distal, i.e., more towards the poorly perfused intestinal section (Fig. 5(A)). The distal point was also more distal (4.2 mm ± 10.1 mm) compared with the fixed point (Fig. 5(C)). The mean change in the width of the boundary zones was 1.8 mm (SD: ± 6.0 mm), as illustrated in Fig. 5(E). The results of the dependent samples t-test were p = 0.397 and p = 0.015 for the central and distal points, respectively. A p-value of 0.084 resulted from the Wilcoxon signed-ranks test for width.

Fig. 5.

Bland-Altman plots for comparison of reconstructed ICG images with ground truth data in terms of distances from the fixed to the central point (A-B), distances from the fixed to the distal point (C-D), and widths of the boundary zones (E-F). A, C, and E include the results of the difference method, B, D, and F those of the post-cube method. Abbreviations: SD – standard deviation, d_FC,ICG – distance fixed point to central point in ground truth ICG image, d_FC,Diff – distance fixed point to central point in image reconstructed by difference method, d_FC,Post – distance fixed point to distal point in image reconstructed by post-cube method. The abbreviations referring to the distances from fixed to distal point and the widths of the boundary zones between central and distal points are analogous to these.

When viewing the images resulting from the post-cube method, the central and distal points were more central than in the ground truth data. The mean deviation of the central point was 4.1 mm (SD: ± 7.0 mm) and that of the distal point was 2.1 mm (SD: ± 9.2 mm). The width of the boundary zones decreased by 2.0 mm on average (SD: ± 6.1 mm). These results can be seen in Fig. 5(B), (D), (F). For the central and distal points, the t-test yielded p-values of 0.026 and 0.345, respectively. The Wilcoxon signed-ranks test used for the width yielded p = 0.196.

Comparing ICG_end to ICG_max according to Sec. 2.4, the distances between the central point and the fixed point d_FC and between the distal point and the fixed point d_FD differed by -6.9 mm and -8.4 mm on average, respectively. The standard deviations were 7.9 mm and 4.7 mm, respectively. The width of the boundary zone w_CD changed by -2.1 mm on average (SD: ± 4.7 mm). These deviations were significant, as indicated by the Wilcoxon signed-ranks test for d_FC and w_CD and the t-test for d_FD. The resulting p-values were p < 0.001 for d_FC, p < 0.0005 for d_FD, and p = 0.022 for w_CD.

In addition, the boundary zone diffused into the poorly perfused intestinal segment in 93.5% of the cases during video. The mean video length was 173.3 s (SD: ± 109.9 s). The median time between i) ICG_max and ICG_end was 78 s (4 s, 305 s), ii) ICG_end and the post-cube was 82 s (20 s, 261 s), and iii) ICG_max and the post-cube was 188 s (25 s, 463 s). Spearman's rank correlation yielded a correlation coefficient of -0.072 (p = 0.635) when examining the effect of time on maximum ICG intensity values in ICG_max and ICG_end, and a correlation coefficient of -0.607 (p < 0.001) for the minimum ICG intensity values.

3.3. Influence of ICG on parameter images

The influence of ICG on the HSI parameter images could be assessed by the differences between these images calculated using the pre- and post-cubes. Their scaled absolute mean showed only small deviations. The means and standard deviations of the scaled mean absolute differences were 6.5% ± 2.0% for the oxygen saturation of tissue (StO₂), 5.1% ± 1.9% for the perfusion index in the near-infrared spectral range (NIR PI), 8.6% ± 1.9% for the organ hemoglobin index (OHI), 6.2% ± 1.4% for the tissue water index (TWI), and 1.8% ± 1.4% for the tissue lipid index (TLI).

The effect of ICG on HSI data was assumed to be directly proportional to the volume of fluorescent agent injected and also to be high immediately after injection and to decrease over time. However, as shown in supplemental Figure S1, the assumption proved to be correct only in the case of StO₂ images. For NIR PI, OHI, and TWI, no dependence on the ICG bolus was observed. Contrary to the original assumption, the deviations increased slightly the larger the time difference was. TLI parameter images did not show relevant deviations.

Clinical evaluations of StO₂ parameter images showed that the central point of the boundary zone was on average 1.1 mm (SD: 4.4 mm) more distal in images from the post-cube than in those from the pre-cube. The distal point also shifted distally after ICG administration (mean: 0.6 mm; SD: 4.1 mm). The width of the boundary zones was 10.7 mm ± 7.6 mm in images from the pre-cube and 10.2 mm ± 6.6 mm in images from the post-cube. The difference in width was 0.5 mm on average (SD: 4.9 mm). The dependent samples t-test revealed no significant differences between the central points (p = 0.107) and the distal points (p = 0.353). The Wilcoxon signed-ranks test also showed no significant difference regarding the width of the boundary zones (p = 0.490) before and after ICG administration.

Comparing the displacements of the central point, distal point, and width with the time elapsed between the two HSI measurements, which was 410.5 s ± 183.33 s, the Spearman's rank test yielded correlation coefficients of 0.059 (p = 0.699), 0.160 (p = 0.288), and 0.159 (p = 0.290), respectively.

4. Discussion

4.1. Intraoperative perfusion assessment

The intraoperative utility of ICG-FA and HSI has been described in several studies [13–16]. In the present study, both techniques could be well used to objectively support the determination of the resection margin in colorectal resections. Their combination provided complementary information on tissue vascularization.

4.2. ICG signal extraction from hyperspectral data

Both ICG reconstruction techniques presented in this work were able to extract information from hyperspectral data and provide parameter images that matched the ICG-FA data. Thus, a combination of both imaging modalities is possible with only one HSI system, saving surgical time and intraoperative technical effort. Furthermore, all information can be generated at a glance to facilitate the intraoperative decision for resection.

After quantitative comparisons, neither the widths of the boundary zones nor the positions of the set distal points in the case of the post-cube method and the positions of the set central points in the case of the difference method showed significant differences in the successfully reconstructed images compared to the ground truth. Nevertheless, the difference and post-cube methods failed in some cases. The reasons are multifactorial and include the given ICG dose per kilogram body mass, ICG signal intensity, time difference between acquisition steps, other differences during ICG administration, specular artifacts, and, in the case of the difference method, the influence of the registration process.

Although this feasibility study is based on data from specific imaging systems, the reconstruction methods can be integrated into alternative systems that provide spectral reflectance data between 770 and 830 nm with appropriate spectral resolution.

4.2.1. ICG dose and signal intensity

One challenge in attempting to extract an ICG signal from spectral data acquired with hyperspectral camera systems is the lack of excitation of the molecules by a laser emitting in the absorption range of ICG. In the case of the TIVITA Tissue, only a halogen lamp with much lower intensity was available. Therefore, the spectral differences between volumes with and without ICG were extremely small (see Fig. 3). A low ICG dose intrinsically resulted in low ICG signal intensities, and the signal extraction seemed impossible.

At the beginning of this clinical trial, patients were administered 2.5 mg ICG regardless of their body mass. The reasons were the high sensitivity of the ICG-FA system, uncertainties in the preoperative determination of body mass, and the lower risk of an intraoperatively incorrectly administered dose. Additionally, there are no medical guidelines or standardizations describing an appropriate ICG dose. Exemplarily, 0.25 mg ICG per kg body mass was used in [28] whereas in [29] the bolus delivery varied from 3.75 to 7.5 mg ICG. A dose of 5 mg ICG resulted in an increase in the success rate of the post-cube method, while the difference method performed well at lower doses. This dose is lower than the maximum daily dose recommended by the manufacturer and therefore does not pose a risk to patients. Unfortunately, the difference method has a higher procedural cost due to the need for two hypercubes and a registration process, making the post-cube method more attractive. A threshold for ICG dose to guarantee the success of ICG reconstruction methods could not be established, so further investigations need to be performed.

4.2.2. Time difference and diffusion of ICG

Another possible reason for the failure of ICG reconstruction methods is the time delay between ICG administration and post-cube uptake. ICG has a half-life of 2-4 min [12] and its fluorescence intensity decreases quite rapidly. The same difficulties occur at low intensities. However, no significant deterioration of maximum ICG intensity values was observed over time, so further studies need to be performed.

In contrast, minimum ICG intensity values decreased significantly with increasing video duration. Because these values were measured in the poorly perfused colorectal area, whereas the maximum values were in the well-perfused areas, it can be assumed that the lack of significant decrease in maximum values is mainly due to a larger and longer-lasting supply of ICG molecules.

Additionally, the diffusion of ICG into the surgeon-identified ischemic areas of the colorectum over time should be mentioned [30]. Clinically, this means that the boundary zone shifts to a less-perfused portion of the intestine over time and that the surgeon should determine the resection line when the fluorescence signal reaches its maximum. Various studies have shown that this can be expected approximately 30-40s after administration [19,31,32]. Due to the technical intraoperative conditions, the post-cube was recorded a few minutes after administration. Again, the ICG signal intensity was reduced. This influencing factor could be minimized in future studies by recording the hyperspectral data at the time of the expected ICG maximum.

4.2.3. Differences in the administration of ICG

During ICG administration, there were differences in injection methods between patients that may have influenced the inflow velocity and thus the ICG concentration and signal intensity in the studied area. A relevant factor is the site at which the ICG was applied, as both peripheral and central venous access were used. It is known that the rate at which drugs are absorbed differs here. Furthermore, there were differences in the speed of application by the anesthesiologist and also in whether or not the ICG was rinsed off after application. As a result, there may have been a difference in fluorophore concentration between patients. Clinical parameters between patients may also differ intraoperatively, thus negatively affecting the ICG concentration in the colorectal area.

4.2.4. Specular artifacts

Glossy artifacts locally distorted spectral data and partially prevented the correct reconstruction of an ICG signal. Although these areas could be masked out, the resulting images were difficult to interpret when these artifacts strongly affected the imaged areas, as only a few pixels remained. Alternative data processing considering the affected spectral ranges could increase the robustness of reconstruction methods in the future.

4.2.5. Registration error

With regard to the success of the difference method, another factor must be considered. Several artifacts occurred during data acquisition. Interchanging camera systems resulted in shifts in the imaged area or wave-shaped artifacts due to camera vibrations when hyperspectral imaging began immediately after the camera positioning. In some cases, organ manipulations also occurred. All of this required registration of pre- and post-cubes. Although the homography was calculated based on manually set landmarks and data sets with tremendous artifacts were excluded from the study population, registration errors occurred. Higher registration errors resulted in lower success rates for the difference method because it is based on calculating the difference of cubes.

4.3. Influence of ICG on parameter images

The results of technical and clinical comparisons of parameter images generated from hyperspectral data before and after ICG administration demonstrated the combined applicability of ICG-FA and HSI regardless of the order using only one hyperspectral camera system. To investigate the influence of ICG, difference images were calculated from the parameter images of the pre- and post-cubes. Their absolute means were less than 10.0% on average. These results demonstrated the small influence of ICG on the HSI parameters and clinical acceptability, especially since no significant differences were found with respect to the width and location of the boundary zone. Also, the time between the two HSI measurements showed no significant impact on the location of the boundary zone. However, future investigations need to be conducted with higher ICG doses depending on body mass and additional organs.

4.4. Clinical outcome

Six anastomotic insufficiencies (13.0%) were noted. This rate is consistent with typical frequencies ranging from 2.7% and 19.2% reported in the current literature [1–3]. In addition to the factors that can be influenced to prevent anastomotic insufficiency, such as resection in the well-perfused bowel region, many factors cannot be controlled. First, male gender is a commonly described risk factor [7,8]. The cohort in this study included a majority of 31 male patients (67.4%). Increased BMI ≥ 30 kg/m² has also been described as a risk factor for anastomotic insufficiency [7]. In this study, 16 patients (34.8%) with a BMI ≥ 30 kg/m² underwent surgery. An elevated ASA score ≥ 3 has been described as another risk factor [9]. Here, 11 patients (23.9%) were included.

Due to the SARS-CoV2 pandemic situation, only crucial operations were performed, so more patients with worse prerequisites were included in the cohort. These poorer prerequisites may have been another reason for the relatively high incidence of anastomotic insufficiencies, as shown elsewhere [2,8].

Overall, studies with larger patient cohorts are needed to reveal the determining factor for the failure of reconstruction methods and to evaluate the potential but also the relevant clinical benefit of innovative supportive imaging.

5. Conclusion

The feasibility of reconstructing signals from hyperspectral data that correlate with the results of indocyanine green-based fluorescence angiography was demonstrated in this study. Perfusion assessments during colorectal surgery based on both imaging modalities may now be possible using only one HSI system. Larger patient cohorts are needed for further evidence and improvement of the reconstruction methods developed. Care should be taken to minimize confounding factors that may adversely affect data processing, for example, by applying mass-based ICG concentrations. The minimum concentration that guarantees the success of ICG signal reconstruction has not been determined.

Excessive doses may prevent the simultaneous applicability of standard HSI because ICG affects spectral data. The concentrations used in this study did not significantly affect the physiological parameters provided by the HSI system. The boundary zone between well- and poorly-perfused intestinal areas can also be assessed by HSI after ICG application. The clinical benefit of objective perfusion assessment by imaging in reducing anastomotic insufficiencies should be further evaluated in larger studies before imaging can be integrated into the surgical procedure in a standardized manner. In addition, a three-arm study (A: subjective assessment by the surgeon, B: assessment by ICG-FA, C: assessment by HSI including fluorophore application) is recommended to determine the most promising, reliable, and safe method.

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Supplemental document

See Supplement 1 ^{(1MB, pdf)} for supporting content.