Algorithm for mapping cutaneous tissue oxygen concentration using hyperspectral imaging

Abstract

The measurement of tissue oxygenation plays an important role in the diagnosis and therapeutic assessment of a large variety of diseases. Many different methods have been developed and are currently applied in clinical practice for the measurement of tissue oxygenation. Unfortunately, each of these methods has its own limitations. In this paper we proposed the use of hyperspectral imaging as new method for the assessment of the tissue oxygenation level. To extract this information from hyperspectral images a new algorithm for mapping cutaneous tissue oxygen concentration was developed. This algorithm takes into account and solves some problems related to setting and calculation of some parameters derived from hyperspectral images. The algorithm was tested with good results on synthetic images and then validated on the fingers of a hand with different blood irrigation states. The results obtained have proved the ability of hyperspectral imaging together with the developed algorithm to map the oxy- and deoxyhemoglobin distribution on the analyzed fingers. These are only preliminary results and other studies should be done before this approach to be used in the clinical setting for the diagnosis and monitoring of various diseases.

1. Introduction

The level of blood oxygenation in certain areas of the body depends on multiple factors like: pulmonary hematosis, systemic blood circulation, local blood circulation. Therefore, measuring blood oxygenation can bring information regarding all the aforementioned physiological functions. Levels of oxy- and deoxyhemoglobin can characterize with good accuracy blood oxygenation and can be used as monitoring parameters for pulmonary and cardiovascular functions (mainly in acute conditions like shock syndrome), but also in evaluation of peripheral vascular disease (with prognostic value in diabetic foot ulcers) or in assessing the viability of various tissues (like vascularized flaps in plastic surgery- mainly perforator or free-style free flaps or burned skin) and even in evaluation of venous circulation.

Various methods for directly or indirectly determination of the tissue oxygenation have been developed and are currently used in both clinical practice and experimental studies such as: blood gas analysis [1], polarographic electrode technique [2], transcutaneous oxygen measurement [3], pulse oximetry [4], near-infrared spectroscopy [5], magnetic resonance imaging [6], electron paramagnetic resonance [7], and positron emission tomography [8]. Each of these methods has however its own limitations, in terms of accuracy, invasiveness, time-consuming, expensiveness, etc. More recently, hyperspectral imaging technique (HSI) has proven its value to assess tissue oxygenation [9–11] providing in a non-invasive way maps of distribution of both hemoglobin species. This information can be only obtained by applying some specific methods for the processing and analyzing of the large volumes of data contained in a hyperspectral image. Different methods to assess tissue oxygenation in hyperspectral images have been proposed and tested so far: principal component analysis (PCA) and fuzzy C-means clustering [12], multivariate analysis and least-squares regression [13], the Gaussian superposition [14] and Monte Carlo simulation [15]. Principal component analysis (PCA) and fuzzy C-means clustering assigns each data point a degree of membership to a component or cluster. Monte Carlo simulation requires a large amount of calculation. Gaussian superposition algorithm deconvolves the relative contributions of chromophores by approximating them with superposition of Gaussians.

In this study, an algorithm based on the modified Beer-Lambert law [16] and Levenberg-Marquardt nonlinear least-squares analysis method [17] was proposed for mapping cutaneous tissue oxygen concentration from hyperspectral images. This algorithm considers and responds to the following problems: (a) how to set the scaling factor; (b) which is the methodology for the calculation of scaling factor; (c) how to calculate the start values of parameters for minimization; (d) which is the best method for validation of the algorithm taking into account a number as large as possible of situations; (e) what is the way in which hyperspectral images are turned into maps of oxy- and deoxyhemoglobin distribution on the fingers of a hand with different blood irrigation states.

2. Materials and methods

We determined the apparent absorption (expressed in arbitrary units) of the tissue using the modified Beer-Lambert law:

$$A\left(x,y,\lambda \right)=-{\log }_{10}\left[\frac{I_t\left(x,y,\lambda \right)-B\left(x,y,\lambda \right)}{I_c\left(x,y,\lambda \right)-B\left(x,y,\lambda \right)}\right]$$ (1)

where I_t (x,y,λ) is the hyperspectral image of the fully illuminated analyzed tissue, B (x,y,λ) is the hyperspectral image taken with the lens completely closed and I_c (x,y,λ) is the hyperspectral image of a fully illuminated reference tile, x and y are pixel coordinates and λ is the band coordinate (wavelength). All these three hyperspectral images were acquired as 12-bit digital data hypercubes using a hyperspectral camera.

The apparent absorption in cutaneous tissues depends on oxyhemoglobin (C_oxy(x,y)) and deoxyhemoglobin (C_deoxy(x,y)) concentrations and also on a factor G(x,y), independent of wavelength, that takes into account light scattering by the tissue outside of the acceptance angle of the camera lens. All these three parameters should be determined separately for each point (x,y) of the hyperspectral image. The apparent absorption can be described as in

$$A\left(x,y,\lambda \right)=G\left(x,y\right)+s\cdot \left[C_{oxy}\left(x,y\right)\cdot {\epsilon }_{oxy}\left(\lambda \right)+C_{deoxy}\left(x,y\right)\cdot {\epsilon }_{deoxy}\left(\lambda \right)\right]$$ (2)

In Eq. (2) ε_oxy (λ) denotes oxyhemoglobin extinction coefficient at a given wavelength λ, ε_deoxy (λ) denotes deoxyhemoglobin extinction coefficient (both expressed in cm⁻¹/Mol/L) and s is a scaling coefficient, unique for the whole image.

The distributions of oxyhemoglobin and deoxyhemoglobin concentrations are obtained, along with the factor G by nonlinear least squares minimization method using Levenberg–Marquardt algorithm. G(x,y), C_oxy(x,y) and C_deoxy(x,y) are the parameters to be obtained from minimization of the residuals res. All these parameters were marked as r₁ ( = G), r₂ ( = C_oxy) and r₃ ( = C_deoxy) and calculated independently in each point (x,y) of the image.

$$res={{\displaystyle \sum _i\left\{A-r_1-s\cdot \left[r_2\cdot {\epsilon }_{oxy}\left({\lambda }_i\right)+r_3\cdot {\epsilon }_{deoxy}\left({\lambda }_i\right)\right]\right\}}}^2$$ (3)

The Minimization process requires to specify a starting point (a set of initial values for the parameters). This approach raises two important problems to be solved: setting the value of s and finding an appropriate set of initial values for parameters.

The approach that we are proposing consists of the following steps. First step consist of determining s from the condition that the contribution at apparent absorption when C_oxy and C_deoxy equals 1:

$$s\cdot \max \left[{\epsilon }_{oxy}\left(\lambda \right)+{\epsilon }_{deoxy}\left(\lambda \right)\right]=1$$ (4)

This procedure is independent of the analyzed image, only depending on the selected spectral range, as can be seen in Eq. (5).

$$s=\frac{1}{\max \left[{\epsilon }_{oxy}\left(\lambda \right)+{\epsilon }_{deoxy}\left(\lambda \right)\right]}$$ (5)

This value can be improved by a calibration that involves a subsequent measurement of the oxyhemoglobin concentration in a selected point, using a different device and method, and then recalculating of s. Indeed, if Eq. (2) is rewritten as a function of oxyhemoglobin saturation (sat (x,y)) and of total effective hemoglobin concentration (hemo (x,y)), where

$$\begin{array}{l}hemo\left(x,y\right)=C_{oxy}\left(x,y\right)+C_{deoxy}\left(x,y\right)\\ sat\left(x,y\right)=\frac{C_{oxy}\left(x,y\right)}{hemo\left(x,y\right)}\end{array}$$ (6)

apparent absorption equation becomes

$$A\left(x,y,\lambda \right)=G\left(x,y\right)+s\cdot hemo\left(x,y\right)\cdot \left\{sat\left(x,y\right)\cdot {\epsilon }_{oxy}\left(\lambda \right)+\left[1-sat\left(x,y\right)\right]\cdot {\epsilon }_{deoxy}\left(\lambda \right)\right\}$$ (7)

It can be easily noticed that saturation depends only on the ratio of C_oxy and C_deoxy, therefore it is not influenced by scaling coefficient s. Consequently the product s·hemo(x,y) remains unchanged when modifying s. Denoting by C_oxy (x₀,y₀) and hemo (x₀,y₀) concentrations in a point (x₀,y₀) for a value s of the scaling coefficient and C_oxy’ (x0,y0) and hemo’ (x₀,y₀) concentrations in the same point (x₀,y₀) for a value s’ of the scaling coefficient and taking into account that

$$s\cdot hemo\left(x_0,y_0\right)=s^{\prime }\cdot hem{o}^{\prime }\left(x_0,y_0\right)$$ (8)

the value of s’ can be deduced as

$$s^{\prime }=s\cdot \frac{hemo\left(x_0,y_0\right)}{hem{o}^{\prime }\left(x_0,y_0\right)}=s\cdot \frac{C_{oxy}\left(x_0,y_0\right)}{C_{oxy}^{\prime }\left(x_0,y_0\right)}$$ (9)

Hence distributions of C_oxy and C_deoxy are recalculated in every point of the image:

$$\begin{array}{l}{C}_{oxy}^{\prime }\left(x,y\right)=C_{oxy}\left(x,y\right)\cdot \frac{C_{oxy}^{\prime }\left(x_0,y_0\right)}{C_{oxy}\left(x_0,y_0\right)}\\ C_{deoxy}^{\prime }\left(x,y\right)=C_{deoxy}\left(x,y\right)\cdot \frac{C_{deoxy}^{\prime }\left(x_0,y_0\right)}{C_{deoxy}\left(x_0,y_0\right)}\end{array}$$ (10)

The second step involves setting appropriate starting values for parameters in minimization.Minimization was performed separately for each pixel of the image so a possible approach may be setting starting values for each pixel apart. However algorithm tests have shown that it is convergent for a wide range of starting values. Therefore the starting values were set uniquely for the whole image. Starting values of the parameters are r₀₁ ( = G₀), r₀₂ ( = C_oxy0) and r₀₃ ( = C_deoxy0).

Because the concentration distributions of oxyhemoglobin and deoxyhemoglobin usually vary between 0 and 1 values as r₀₂ = 0.5 and r₀₃ = 0.5 were considered as a reasonable choice.

To set a starting value (r₀₁) for G we take into account that it represents contributions of other factors not depending on wavelength and it was assumed that the maximum of the apparent absorption A corresponds to a point in which C_oxy = 1 and C_deoxy = 1. In this case r₀₁ = max (A) −1.

A hyperspectral camera model PS-FW-11-V8E-OEM (SPECIM, Oulu, Finland), having a resolution of 1392 (horizontal) x 1040 (vertical) pixels (of 6.45 µm x 6.45 µm) was used to acquire the hyperspectral images. The slit of the spectrograph is 30 µm x 8.98 mm, providing a spectral resolution of 2 nm. A binning of 4 x 4 was the solution to get a small image acquisition time and a reasonable dimension of the hypercube, so the frame rate was 41 Hz, horizontal resolution was 348 samples and the vertical resolution was 258 spectral bands (in the range 335 – 836 nm). This hyperspectral camera uses a Schneider Optics (NY, USA) lens, model Xenoplan 1.4/17, with a focal length of 17 mm and maximum relative aperture of 1:1.4. Because the hyperspectral camera forms the image of a line (spectrograph slit), it is necessary to scan the sample by moving the sample with a motorized linear positioning stage model M-505.2 S2 (Physik Instrumente, Karlsruhe, Germany), driven by a stepper controller model C-663.10 (Physik Instrumente, Karlsruhe, Germany). This way 207 lines were acquired during 5 s. Thus the acquired hypercube had the dimensions 348 samples x 258 bands x 207 lines. The color depth was 12 bits.

The spectral range of interest was selected as being between 500 and 660 nm, so we used only the 83 bands of the hypercube ranging in the spectral range of interest. The analyzed intensity distribution was stored into the matrix I(x,λ,y) having dimensions (348 x 83 x 207). The experimental layout is presented in Fig. 1.

Fig. 1.

Experimental setup: 1. hyperspectral camera, 2. hyperspectral camera lens, 3. motorized linear positioning stage, 4. stepper controller.

An Avantes (Apeldoorn, Netherlands) reference tile model WS-2 was used to get I_c hypercube. Before using it, the values of I_c from a pixel of the central area of the reference tile are extended to all pixels of the image (the reference tile is uniform but is smaller than the image).

3. Mapping algorithm

The mapping algorithm starts by acquiring the 3 hypercubes or (for evaluation) by creating the test image. Next step is to calculate the apparent absorption 3D-matrix from the hypercubes data. The third step of the algorithm sets the starting values of the minimization parameters. In the fourth step minimization is performed, resulting the distributions of the 3 parameters. In the last (fifth) step other two distributions (total effective hemoglobin concentration and saturation) are derived, all results are output as data and graphs (see Fig. 2).

Fig. 2.

Cutaneous tissue oxygen concentration mapping algorithm.

3.1 Test image

The test image was designed as a square 101 x 101 pixels hypercube with a vertical variation of the oxyhemoglobin concentration from 0 to 1 in steps of 0.01 (101 horizontal strips, Fig. 3(a)), i.e. C_oxy (x,y) = y/100 and a horizontal variation of the deoxyhemoglobin concentration from 0 to 1 in steps of 0.01 (101 vertical strips, Fig. 3(b)), i.e. C_deoxy (x,y) = x/100.

Fig. 3.

Target parameter distributions in test image: (a) oxyhemoglobin concentration, (b) deoxyhemoglobin concentration and (c) factor G.

For the factor G a distribution of circles centered in the middle of the image, at coordinates (51,51), was chosen. It is calculated as

$$G\left(x,y\right)=\frac{\sqrt{{\left(x-51\right)}^2+{\left(y-51\right)}^2}\cdot G_{\max }}{\sqrt{5000}}$$ (11)

G_max was set to 0.2. Its distribution is presented in Fig. 3(c).

3.2 Calculations

Apparent absorption is calculated using (1), starting values for parameters are, as above mentioned, r₀₁ = max (A) −1, r₀₂ = 0.5 and r₀₃ = 0.5.

The three parameters are calculated by minimizing the residuals in Eq. (3) using nonlinear least squares minimization method using Levenberg–Marquardt algorithm [17].

Total effective hemoglobin concentration distribution and oxyhemoglobin saturation distribution are calculated as in Eq. (6).

4. Results

4.1 Evaluation using the test image

The evaluation of the algorithm using the test image consisted in comparing the distributions of G_t, C_oxyt and C_deoxyt set at the test image design time (so called ‘targets’) to the distributions G, C_oxy and C_deoxy (‘results’) obtained from minimization.

As we can see in Fig. 4, the graphs of ‘results’ (‘Validation test’) look identical to ‘target’ graphs (Fig. 3). To highlight the difference between results and targets following statistics were calculated:

Fig. 4.

Calculated parameter distributions in validated test image: (a) oxyhemoglobin concentration, (b) deoxyhemoglobin concentration and (c) factor G.

• •The average of the ‘result’ distribution, called ‘Reference’.
• •The ratio of maximum of the absolute difference between ‘result’ and ‘target’ distributions and ‘Reference’, called ‘Maximum’.
• •The ratio of minimum of the absolute difference between ‘result’ and ‘target’ distributions and ‘Reference’, called ‘Minimum’.
• •The ratio of the average of the difference between ‘result’ and ‘target’ distributions and ‘Reference’, called ‘Average’.
• •The ratio of standard deviation of the difference between ‘result’ and ‘target’ distributions and ‘Reference’, called ‘Standard deviation’.

The results are presented in Table 1.

Table 1. Statistics of results.

Distribution	Reference	Maximum	Minimum	Average	Standard deviation
Coxy (x,y)	0.5	4.9338·10−12	0	−7.0417·10−14	2.7139·10−13
Cdeoxy (x,y)	0.5	5.3780·10−12	0	7.6774·10−14	2.9582·10−13
Hemo (x,y)	1	2.2204·10−13	0	3.2068·10−15	1.2199·10−14
Sat (x,y)	0.5	3.1616·10−11	0	−2.0873·10−13	5.5315·10−13
G (x,y)	0.1093	1.6027·10−12	0	−2.2916·10−14	8.8146·10−14

It is remarkable that an excellent match between ‘results’ and ‘targets’ was recorded. The relative differences are in the range 10⁻¹¹ - 10⁻¹⁵ and they are all resulted from the computational errors.

This test also demonstrated that the setting of the starting values was correct, the minimization retrieving correctly the ‘target’ distributions even in points were they differ much from the starting values.

Furthermore, this test also proved that the setting of the starting values was correct, the minimization retrieving correctly the ‘target’ distributions even in points were they differ much from the starting values.

Processing time for an image using a system equipped with an i5-4440 processor at 3.1 GHz and 8 GB RAM was about 4 minutes. This depends on the processor’s speed, the available RAM and the number of processor cores (parallel processing option can be used) and, obviously, on hypercube dimension (better spatial and spectral resolution increases hypercube dimension and consequently the processing time). An insufficient RAM may cause a severe slow of the processing because hard disk available space is used to compensate the insufficient memory, therefore a balance should be done between accuracy and available resources.

4.2 Experimental data

Experimental data were collected and processed in order to verify the validity of the algorithm. The experiment consisted in acquisition of a hyperspectral image of a right hand of a healthy subject: at the base of the middle finger a tourniquet was applied in order to cut off the venous return and, for the ring finger, the tourniquet induced complete ischemia after the blood from the finger was removed with a milking maneuver. The other two fingers in the image (index and the little finger) served as reference.

Following the same procedure as in the case of the test image apparent absorption A(x,λ,y) was calculated for the whole image and the distributions of concentrations of oxyhemoglobin, of deoxyhemoglobin, of oxyhemoglobin saturation and of total effective hemoglobin as well as the distribution of factor G were deduced. All these results are presented in Fig. 5.

Fig. 5.

Calculated parameter distributions in acquired image of fingers: (a) apparent absorption at the wavelength 540 nm, (b) oxyhemoglobin concentration, (c) deoxyhemoglobin concentration, (d) factor G, (e) total effective hemoglobin concentration and (f) oxyhemoglobin saturation in the analyzed image.

Figure 5(a) presents an apparent absorption map at the wavelength 540 nm just for illustration. The absorption varies between 0 and 1.2521 for this wavelength and between 0 and 1.2894 for the whole wavelength range that was analyzed (500 – 660 nm). It is important to note that a mask was applied over the measured absorption data. The mask has ones inside the area of interest AOI, subject’s hand) and zeros outside this area (the background). Minimum of the absorption inside AOI is 0.5154, while outside AOI it is 0.

The distribution of oxyhemoglobin concentration (Fig. 5(b)) varies between 0 and 0.2599. The two reference fingers (index and the little finger) presented the highest values of oxyhemoglobin concentration, while in the middle finger (with no venous circulation) the concentration dropped drastically and in the ring finger (with ischemia and blood emptied) the concentration decreased down to very low levels and even to zero.

The distribution of deoxyhemoglobin concentration (Fig. 5(c)) varies between 0 and 0.6581, values that are usual. For the reference fingers the variation lies between 0.3 and 0.5. At the middle finger an increase of deoxyhemoglobin concentration may be noticed, due blood stasis and higher oxygen extraction by the tissues. The ring finger, almost blood emptied, presents a decrease of the deoxyhemoglobin concentration compared to references fingers.

Figure 5(d) presents the distribution of the factor G, which has a maximum value of 1.2254, the background (outside AOI) being obviously 0. The distribution in fingers is quite uniform, except for the ring finger (that was blood emptied), which presents lower values in the central area, and also the contour (extremities) of all fingers, where the factor G is greater than 1. This phenomenon is due to the finger ‘profile’ (at the edge the thickness of the finger sharply decrease) and it is related to the different length travelled by the light through the finger tissue.

The variation of the total effective hemoglobin concentration (Fig. 5(e)) lies between 0 and 0.7672. In this case again it may be seen a lower level of total effective hemoglobin concentration in the middle finger compared to the references and also a sharp decrease in the ring finger.

The same situation, but more strongly noticeable, occurs in the distribution of saturation (Fig. 5(f)), which varies between 0 and 0.3840.

5. Conclusion

As a conclusion the study attempted and succeeded to solve was that of determining how to set the scaling coefficient. It is shown that the scaling coefficient cannot be considered a minimization parameter (as oxyhemoglobin and deoxyhemoglobin concentrations or the factor G) but it is a constant for the entire image, and providing a methodology for its calculation. More than that this study provided a validation method for this algorithm taking into account a number as large as possible of situations. Therefore the algorithm is accompanied by a validation method based on a test image (hypercube) that allows testing a very large number (10,201) of different combinations of minimization parameters.

The validation method of the hyperspectral image analysis algorithm by using an appropriate test image proved useful and effective. The test image includes a large range of usual values of the analyzed parameters (oxyhemoglobin concentration, deoxyhemoglobin concentration and the factor G) with very fine variations, allowing the simulation of a very large number of possible situations. Because the analysis is performed independently in each pixel of the hyperspectral image it was possible to simulate a different situation (combination of parameters) in each pixel. By setting the dimension of the test image to 101 x 101 pixels it results 10,201 different situations. A vertical variation of oxyhemoglobin concentration was superposed over a horizontal variation of deoxyhemoglobin concentration and a radial variation of the factor G. The results indicate a very good accuracy, the error (of the order of 10⁻¹¹) is due to the related computing processes. Analysis of an experimental hyperspectral image produced pertinent hemoglobin data. In conclusion the algorithm is accurate enough for skin oxygenation assessment.