Hyperspectral imaging of human skin aided by artificial neural networks

Abstract

We developed a compact, hand-held hyperspectral imaging system for 2D neural network-based visualization of skin chromophores and blood oxygenation. State-of-the-art micro-optic multichannel matrix sensor combined with the tunable Fabry-Perot micro interferometer enables a portable diagnostic device sensitive to the changes of the oxygen saturation as well as the variations of blood volume fraction of human skin. Generalized object-oriented Monte Carlo model is used extensively for the training of an artificial neural network utilized for the hyperspectral image processing. In addition, the results are verified and validated via actual experiments with tissue phantoms and human skin in vivo. The proposed approach enables a tool combining both the speed of an artificial neural network processing and the accuracy and flexibility of advanced Monte Carlo modeling. Finally, the results of the feasibility studies and the experimental tests on biotissue phantoms and healthy volunteers are presented.

1. Introduction

Biomedical application of hyperspectral imaging [1] is nowadays an intensively developing area of the research supported by the recent technical progress in the instrument development and significant cost reduction. Potential applications of the considered technique are of high social impact and include monitoring and diagnostics of diabetic ulcer formation [2,3], rheumatic complications [4,5], wound healing control [6,7], diagnosis of melanoma [8,9] and other malignancies [10], support of surgical and treatment procedures [11,12] and others. Moreover, the development of portable and low-cost imaging systems [13–15] demonstrates strong potential for implementation of hyperspectral imaging technology in our everyday life and the day-to-day practice of clinicians.

Spectral images of skin and other biotissues contain information about the spatial distributions and concentrations of biological chromophores [16,17] such as oxy- and deoxyhemoglobin, melanin, bilirubin, etc. The methods of in vivo assessment and mapping of main skin chromophores and estimation of blood oxygen saturation have a long history [18]. Conventional approach usually utilizes 2–4 wavelengths in the visible and near-infrared ranges or RGB color sensors [19,20]. Multispectral measurements typically involve 4 to ∼10 wavelengths providing more consistent data for the analysis, while hyperspectral imaging devices are capable of measuring at up to hundreds of spectral bands.

Basic physiological skin parameters such as blood volume fraction, blood oxygenation and melanin content are usually retrieved by fitting the measured intensity spectrum data with an analytical or numerical model of skin diffuse reflectance. Unfortunately, analytic expressions for biotissue reflectance are known only for a limited amount of simplified geometries and boundary conditions such as, for instance, semi-infinite or two-layer medium. In particular, Yudovsky et al. [21] proposed an analytical expression for the diffuse reflectance of semi-infinite homogeneous and two-layer absorbing and anisotropically scattering media based on the modified two-flux Kubelka-Munk approximation.

The diffusion model is also a frequently used approximation for description of radiative transport in biotissues. Corresponding analytical solution for two-layer media was reported by Kienle et al. in [22] and later generalized for $N$-layer medium [23]. Application of the diffusion model for processing the hyperspectral skin images was also demonstrated in [24]. However, the diffusion model is known to have certain significant limitations [25] in an accurate description of radiative transport in low scattering or highly anisotropic media as well as at short source-detector distances and near the surface regions where the diffusion scattering regime is not yet established.

The Monte Carlo (MC) method is considered as a ’gold standard’ for modeling light transport in a turbid tissue-like scattering medium [26] enabling accounting realistic biotissue geometry and complex boundary conditions. Practical realization of this computational approach is often limited by the required computer facilities. Since recently, the progress in computer technologies and parallel computing, including GPU-accelerated counting enable one to perform enormously extensive computational modeling in a reasonable time [27,28]. The subsequent use of machine learning technologies such as Artificial Neural Networks (ANN) naturally allows utilizing the versatility and flexibility of the MC approach for real-time hyperspectral data processing. Indeed, all the time-consuming calculations are performed in advance and the generated numerical solutions could be used as a training dataset for the supervised machine learning.

The aim of the present study is to implement in practice an ANN-based approach to hyperspectral imaging for quantitative diagnostics and characterization of human skin, including 2D mapping of skin chromophores (hemoglobin, melanin, etc.), mapping of blood oxygenation kinetics and evaluation of skin perfusion. To the best of our knowledge, the current work is the first one that applies MC-trained ANN for processing the hyperspectral data of skin functional tests and verifies the proposed approach through the phantom measurements.

2. Methods and materials

2.1 Hyperspectral imaging system

A compact hand-held imaging system was constructed on the basis of the hyperspectral snapshot camera (Senop Optronics, Finland) utilizing a micro Fabry-Perot tunable filter providing spectral resolution of 6–10 nm within the total range of 500–900 nm. The schematic layout of the constructed device is presented in Fig. 1. A broadband illumination unit utilizing 50W halogen lamp is based on the fiber-optic ring illuminator (Edmund Optics, USA) providing uniform distribution of light intensity in the camera focal plane with the average irradiance of 4.3±0.5 mW/cm² in the camera field of view on the skin surface. The use of the ring illuminator made it possible to combine the axes of illumination and detection. The illumination ring and the camera are equipped with the rotatable broadband linear polarizers fixed at the crossed position for the reduction of specular reflection from the measured object. The 3D model of the unit was implemented using CAD software and printed out with a 3D material printer. The considered measurement approach allows capturing the reflected signal of the entire field of view at a certain waveband. The scanning is performed in spectral domain. Generally, the constructed device is capable of recording spatially and spectrally resolved reflectance used for further ANN analysis. Thus, the diffuse reflectance of skin/biotissue phantom was recorded with the spectral step of 5 nm from the area of 8×8 cm² at 1010×1010 pixels resolution of CMOS sensor. Normalized spatially-resolved tissue reflectance was calculated as a pointwise ratio of dark-noise-corrected light intensity reflected from the skin/phantom and that reflected from the diffuse reflectance standard. A hypercube from the reflectance standard was recorded every time before the measurement of an object. Gray Spectralon (50% reflection, Labsphere, Inc., USA) was used as a calibration standard. Use of gray standard allows one to avoid the oversaturation of CMOS sensor matrix possible for white reflectance standard (100% reflection) at the fixed integration time.

Fig. 1.

Schematic design of the developed prototype of hyperspectral imaging system: 1 – lens hood (spacer), 2 – fiber ring illuminator delivering the broadband radiation from 50W halogen lamp, 3 – ring-shaped polarizer of illumination unit, 4 – camera polarizer; 5 – built-in diffuse reflectance standard (grey Spectralon – 50% reflection).

2.2 Biotissue phantom

A solid biotissue phantom with predefined optical properties equal to those of human bloodless dermis was used to validate the abilities of the proposed approach to measure blood oxygen saturation in the embedded vessels. The layout and the dimensions of the developed phantom are shown in Fig. 2(a,b). The phantom contains two tilted plain hollow channels (0.25×1 mm² cross-section) located at the different angles inside the phantom. The embedding depth linearly increases from 0.3 to 2 mm for superficial channel and from 0.3 to 4 mm for the deep one. The detailed description of the manufacturing and characterization of biotissue phantoms is given in [29–31]. In short, polyvinyl chloride (PVC)-based matrix was used as a transparent host for ZnO nanoparticles introducing scattering. The proper amount of added scattering particles was estimated on the basis of Mie theory taking into account their size distribution. The average diameter of the particles was $0.34\mu$m. To control the absorption coefficient of the phantom, a black plastic color composed of CI Pigment Black 7 was added. Two glass capillaries were installed at different angles as described above inside the phantom mold prior to the phantom solidification. After the solidification, the capillaries were gently removed thus forming the hollow channels. The channels were further connected via the micropipette tips and plastic tubing to the syringe pump.

Fig. 2.

3D schematics of the biotissue phantom with the embedded blood vessels (a); photograph (top view) of the manufactured phantom with the channels filled with fully oxygenated blood (b); three-layer tissue phantom model used for calculations of diffuse reflectance spectra (c); measured optical properties of the biotissue phantom (d,e).

A suspension of washed human red blood cells (RBC) (45% v/v) obtained from a healthy donor was pumped through the channels with the rate of 10 ml/h to avoid the sedimentation process. The used RBC suspension was prepared according to the following procedure. First, a whole blood (hematocrit 45%) from a healthy donor was drawn by venipuncture and put into an ethylenediaminetetraacetic acid (EDTA)-covered tubes to prevent coagulation. Written consent was obtained from the volunteer. The studies were conducted in accordance with the obtained ethical permission from the Finnish Red Cross. Secondly, to prevent uncontrolled aggregation the RBC were washed in phosphate buffered saline (PBS, pH 7.4). For that purpose the whole blood was centrifuged at 3000 g for 10 min, and plasma supernatant was replaced with PBS. This procedure was repeated three times to remove residual plasma containing the proteins inducing the uncontrolled aggregation. Finally, washed RBC mass was suspended in PBS to the initial value of hematocrit. All the measurements were conducted at room temperature of ${20}^{\circ }$ C.

Optical properties of the fabricated phantom including absorption and scattering coefficients as well as the anisotropy factor in the visible and near-infrared spectral region were derived on the basis of the inverse adding-doubling method from the diffuse reflectance, diffuse transmittance and collimated transmittance measured with the spectrophotometer (Gooch & Housego, USA) equipped with the integrating spheres [30,32]. Additionally, the refractive index of the phantom was measured using the multiwavelength Abbe refractometer (Atago, Japan). The results of the characterization of the phantom optical properties are presented in Fig. 2(d, e). Figure 2(d) also shows the comparison of the retrieved absorption and reduced scattering coefficients of the created phantom with those measured from the human ex vivo dermis. The data for human dermis are taken from [33]. The measurements for the reduced scattering (see Fig. 2(d)) are in a good agreement for the whole spectral range 500–1100 nm. For the absorption coefficient, good agreement is observed for the spectral range ∼650–1100 nm. The measured scattering anisotropy factor (Fig. 2(e)) is on the level of 0.8 for the considered range that is also typical for the biotissues including skin [34,35]. Thus, the developed phantom can be considered as a relevant model of human bloodless dermis/skin for red and infrared spectral range.

Before the measurements, the prepared RBC suspension was carefully stirred on the air to increase its oxygen saturation up to 100%. To confirm the oxygen level, 0.5 mL of the suspension was placed into a sealed 1-mm-thick plain glass cuvette and the optical properties of the suspension for the spectral range of 600–1000 nm were estimated on the basis of the spectrophotometric measurements and the inverse adding-doubling procedure similarly as described above. Good agreement of the obtained absorption spectrum with the absorption of oxygenated blood (hematocrit 45 %) [36] confirms the predetermined oxygen saturation level.

2.3 Human skin model

A seven-layer model of skin (Fig. 3) was used to calculate the diffuse reflectance spectra. The utilized model accounts for the in-depth distribution of blood in the dermis that is missing in basic two-layer models considering dermal blood distribution as uniform. The account of the dermal blood distribution is essential due to the dependence of the light penetration depth on its wavelength (see e.g. [37]). Thus, nonuniform blood distribution in the dermis will have an effect on the shape of the reflectance spectrum. The basics of the considered model have been previously described in [38,39] and a brief summary is given below. The main parameters of the model are presented in Table 1. The absorption coefficient of each layer takes into account the blood volume fraction $C_{blood}$ in various vascular beds, oxygen saturation $S$, water content $C_{H_{2}O}$ and melanin fraction $C_{mel}$ [38]:

$${\mu }_a^{strat.corneum}(\lambda )=(1-C_{H_{2}O}){\mu }_a^{baseline}(\lambda )+C_{H_{2}O}{\mu }_a^{water}(\lambda ),$$ (1)

$${\mu }_a^{epidermis}(\lambda )=(1-C_{H_{2}O})(C_{mel}{\mu }_a^{mel}(\lambda )+(1-C_{mel}){\mu }_a^{baseline}(\lambda ))+C_{H_{2}O}{\mu }_a^{water}(\lambda ),$$ (2)

$$\begin{array}{c}{\mu }_a^{layer}(\lambda )=(1-C_{H_{2}O})(C_{blood}(S{\mu }_a^{oxy}(\lambda )+(1-S){\mu }_a^{deoxy}(\lambda ))+\\ +(1-C_{blood}){\mu }_a^{baseline}(\lambda ))+C_{H_{2}O}{\mu }_a^{water}(\lambda ).\end{array}$$ (3)

Here, ${\mu }_a^{mel}$ is the absorption coefficient of melanin, ${\mu }_a^{baseline}$ is the absorption coefficient of water-free tissues, ${\mu }_a^{water}$ is the absorption coefficient of water, ${\mu }_a^{oxy}$ and ${\mu }_a^{deoxy}$ are the absorption coefficients of oxygenated and deoxygenated blood, respectively, for the hematocrit $Ht$ = 45%. The scattering coefficients of the layers are approximated based on the combination of Mie and Rayleigh scattering [35,38]:

$${\mu }_s^{\prime epidermis}(\lambda )=1.08\cdot {10}^7\cdot {\lambda }^{-2.364}+13.571\cdot {\lambda }^{-0.267},$$ (4)

$${\mu }_s^{\prime dermis}(\lambda )=1.19\cdot {10}^7\cdot {\lambda }^{-2.427}+7.15\cdot {\lambda }^{-0.258}.$$ (5)

Fig. 3.

Skin tissue layers taken into account in the used seven-layer model

Table 1. Parameters used for assessment of the absorption coefficients of the skin layers.

Layer	Thickness, $\mu$m	Blood volume	Saturation	Water	Melanin
		fraction, $C_{blood}$, %	S, %	$C_{water}$, %	$C_{mel}$, %
$Stratum$ $corneum$	20$K_{epi}$	0	0	5	0
Living epidermis	50$K_{epi}$	0	0	20	$K_{mel}$
Papillary dermis	100	4$K_b$	$S$	50	0
Upper blood net dermis	80	20$K_b$	$S$	60	0
Reticular dermis	1620	4$K_b$	$S$	70	0
Deep blood net dermis	200	10$K_b$	$S$	70	0
Subcutaneous fat	5900	7$K_b$	$S$	15	0

Therefore, 25 nonzero parameters are required by the considered model (see Table 1) to describe the absorption of the skin. To make the model more practical, some groups of the parameters were reduced by the introduction of the coefficients that links the parameters inside the groups. The main idea in this simplification was to consider a minimal set of parameters influencing the reflectance spectra shape and admitted to change independently. This approach assumes that the parameters of each group can vary coherently. Thus, as seen from Table 1, parameter $K_b$ links blood volume fractions of dermal and sub-dermal layers. The average value of blood oxygen saturation $S$ same for all layers was taken into account. The parameter $K_{epi}$ describes the thickness of epidermis and Stragum corneum. Finally, four parameters, $K_b$, $S$, $K_{mel}$, $K_{epi}$ were considered as variables to model diffuse reflectance of the skin. The rest of parameters were considered as fixed and equal to their typical values.

2.4 Monte Carlo simulations of diffuse reflectance spectra

For the training of ANN and estimation of blood oxygen saturation, the diffuse reflectance spectra of biotissue phantom were simulated by MC technique using three-layer tissue model (see Fig. 2(c)) accounting the blood layer embedding depth $d$ and its oxygen saturation $S$ as the variable parameters. The thickness of the blood layer was selected equal to the channel thickness of the biotissue phantom. GPU accelerated Monte Carlo model of photon migration in scattering tissue-like media [26] was used for routine simulation of the spectra in the training dataset for all possible combinations of the considered parameters and cross-validated for biotissue phantoms with [40]. Totally 2,091 spectra were simulated accounting $d$ = [0–4] mm with a step of 0.1 mm and $S$ = [50–100]% with a step of 1% for the spectral range 500–900 nm with a step of 5 nm. Measured optical properties of the biotissue phantom presented above were utilized for the simulations. Optical properties of blood (hematocrit 45%) for different oxygen levels were adopted from [36].

A similar approach was used to simulate diffuse reflectance spectra of the skin. The differences in the spatial distribution of blood, melanin, blood oxygen saturation, hematocrit, the water content of the skin, as well as the numerical aperture of the detector were taken into account. All possible combinations of four variable parameters were obtained for the simulated skin reflectance spectra accounting blood volume coefficient $K_b$ = [0–2] with a step of 0.1, blood oxygen saturation $S$ = [30–100]% with a step of 1%, melanin concentration $K_{mel}$ = [0–0.1] with a step of 0.01 and epidermis thickness coefficient $K_{epi}$ = [1–5] with a step of 1. The complete training data set contained 82,005 spectra in the range of 500–900 nm with a step of 5 nm.

2.5 Neural networks

Matlab R2018b Deep Learning Toolbox was chosen to build and train an interconnected set of neural networks for the retrieval of skin parameters $K_b$, $S$, $K_{mel}$ and $K_{epi}$ as well as biotissue phantom parameters $d$ and $S$. The toolbox allows using the multilayer perceptron (MLP) network for curve fitting and regression. The MLP network has an input layer, one hidden layer and a linear output. It can be regarded as a nonlinear parameterized mapping of the input feature vector to the output space of data fitting results. In the considered problem statement, the inverse solution for tissue parameters characterization can be classified as a construction of a multiple regression model. The favorable number of the neurons in the hidden layer was evaluated in a series of tests. In the tests, the number of neurons was sequentially increased from 1 to $N$, where $N$ is the number of data points in the considered spectrum range used for the estimation of skin/phantom parameters. The number of hidden neurons providing the best fitting performance was selected for further use.

A general flowchart of data processing is shown in Fig. 4. All the mentioned steps can be divided into two major groups. The first group consisting of numerical tissue model adjustment, simulation of the training dataset and training the ANN is the most time consuming but performed prior to the measurements. The second group is the application of trained ANN to the results of the hyperspectral measurements and obtaining the required parameters. This group does not demand significant computational resources and can be performed in line with the experimental data acquisition. At first, the epidermal thickness and melanin content are estimated (step 4 in Fig. 4) using the corresponding trained ANN. The best fit for these parameters was obtained in the spectral range of 680–725 nm and 725–770 nm, respectively. Then, for the estimated epidermal thickness and melanin content, the blood volume fraction and blood oxygenation were retrieved for the spectral range of 535–705 nm and 675–825 nm, respectively (step 5 in Fig. 4). Preprocessing of the measurement data aiming at improving the signal-to-noise ratio consists of the spatial averaging of the input hyperspectral cube in a 5×5 pixel window. All the input measurement data vectors for the considered spectral ranges as well as the training dataset were normalized prior to the network training procedure. The normalization procedure was implemented in the data processing algorithm and dedicated to the mitigation of possible height variations of the skin surface in the camera field of view. The vector normalization (by $L^2$-norm) for both the training dataset and measured data was used to obtain the corresponding unit vectors.

Fig. 4.

Flowchart of hyperspectral data processing. Steps 1–3 are performed prior to the measurements. Steps 4 and 5 – in line with the data acquisition. Note: step 4 could be omitted for repeated measurements of the same area of interest.

3. Results and discussion

3.1 Phantom studies

The simulated diffuse reflectance spectra for the biotissue phantom at two values of blood layer embedding depth $d$ and different blood oxygen level $S$ are presented in Fig. 5(a, b). Strong blood absorption in the spectral region $\lambda$ < 600 nm is the cause of low reflection in this range. It is clearly seen that the increase of the embedding depth also increases the reflection in the range of 500–600 nm by making the absorbing layer deeper thus fewer photons are capable of reaching it and get absorbed. For the considered spectral range, similar changes are also typical for the skin reflectance spectra in the case of decrease of blood volume fraction (see Fig. 5(c)) or increase of the epidermis thickness (see Fig. 5(f)). The reflection in the range of 600–800 nm is less depth-sensitive and depends mostly on the blood oxygen level. With the decrease of the blood oxygen level, the reflection in the range of 600–800 nm drops and the characteristic deoxyhemoglobin absorption peak at 760 nm also becomes visible as a pit on the phantom reflection spectra. Similar changes in spectra caused by the variation in blood oxygen saturation are observed for the skin (see Fig. 5(d)).

Fig. 5.

Simulated spectra of diffuse reflectance from biotissue phantom (a, b) at different embedding depths $d$ of blood layer and oxygen saturation $S$. Simulated reflectance spectra for human skin at different values of blood volume coefficient $K_b$ – (c), blood oxygen saturation $S$ – (d), melanin concentration $K_{mel}$ – (e) and epidermis thickness coefficient $K_{epi}$ – (f).

The simulated reflectance spectra were used for training of the ANN for the recovery of the oxygen level in the blood channels inside the biotissue phantom. Trained ANN was applied for processing the hyperspectral measurements of phantom diffuse reflection. The spectral region of 550–800 nm was selected for the blood oxygen estimation as the most prominent changes of reflectance are observed in this region as can be seen from Fig. 5(a, b). Figure 6(a) shows the measured diffuse reflectance along the superficial channel. The distance along the channel linearly corresponds to the channel embedding depth. Thus, the increase in the channel embedding depth causes the significant increase of the reflection in the range 500–600 nm similarly as it was observed in the simulations. Figure 6(b) shows the comparison of the measured reflectance spectra (dots) with the simulated ones (lines) selected as a best fit by the trained ANN for three values of the capillary embedding depth. In all cases, the recovered blood oxygen saturation differs less than 4% from the actual value. The map of the recovered blood oxygen saturation for the whole biotissue phantom is presented in Fig. 6(c). The corresponding values for the superficial and deep channels are shown in Fig. 6(d). It is seen that the correct value of blood oxygenation, in this case, can be recovered up to the depth of about 2 mm that corresponds approximately to the middle of the channel 2. Further increase of the embedding depth causes the immediate signal loss.

Fig. 6.

Spectral dependence of the diffuse reflectance measured along the deepening superficial channel (a); measured reflectance versus best fit selected by the trained ANN (b); 2D map of blood oxygen saturation in the biotissue phantom (c); recovered values of blood oxygen saturation in the deepening superficial and deep channels of the biotissue phantom (d).

The results described above were obtained using two-parameter fit accounting the dependence of the simulated spectra on the channel embedding depth $d$ and blood oxygen saturation $S$. The phantom studies also allow to investigate the effect of the chosen tissue model on the blood oxygen level recovering accuracy. Fig. 6(d) (the green and the black lines) shows the recovered oxygen level in the deepening superficial channel of the phantom using the biotissue model with the fixed embedding depth of the blood layer ($d$ = 0.5 and 1 mm, correspondingly). Thus, it is seen that the application of the model with a fixed embedding depth of the blood layer may lead to significant (up to 40%) underestimation of the recovered blood oxygen saturation. In this case, the best fit performed by the neural network is obtained for a reflectance spectrum corresponding to lower oxygen saturation that is causing the underestimation in the recovered oxygen level. The correct value is obtained only for the embedding depth corresponding to that in the biotissue model.

3.2 In vivo skin imaging

The proposed imaging approach was applied to human skin in vivo. Typical dependencies of the simulated skin reflectance spectra on the variable parameters ($K_b$, $S$, $K_{mel}$, $K_{epi}$) obtained for seven-layer skin model are presented in Fig. 5(c–f). The increase of blood volume fraction in the skin (shown in Fig. 5(c)) causes the decrease of skin reflectance within the entire spectral range of 500–900 nm. However, the most significant changes are observed in the region of strong blood absorption (500–600 nm) that is also observed in the experiments with variable mechanical pressure on the skin [41]. The most prominent changes in the skin reflectance caused by the variation in blood oxygen saturation are observed in the range of 600–800 nm (see Fig. 5(d)). The influence of the melanin content in the epidermis is shown in Fig. 5(e). It is seen that an increase in melanin content decreases the reflectance in the entire considered spectral range. However, the change in the shape of spectra is significantly different compared to the increase of blood volume fraction. Increase in the epidermal thickness (see Fig. 5(f)) acts differently for the spectral region 500–600 nm and 600–900 nm. Below 600 nm the increase in the epidermal thickness causes the increase in skin reflectance, however, for the longer wavelengths, similar increase causes the decrease in the reflectance. This effect can be explained by the significantly lower (up to two orders of magnitude) absorption of oxyhemoglobin in the red-IR spectral region relative to the absorption in the green region. Thus, the oxygenated blood turns into a moderate absorber and the decrease in the reflectance in the red-IR region is caused by the increased optical density of the epidermis.

The developed hyperspectral imaging system was used to perform trial measurements and the arterial occlusion tests with healthy volunteers. Several examples of the obtained hyperspectral images of the dorsal and palmar side of a human hand are shown in Visualization 1 ^{(4.9MB, avi)} . According to the University of Oulu Ethics Committee regulations, informed consent was obtained from all tested subjects. Caucasian and Indian male skin corresponding to the type II and V of Fitzpatrick scale (Fig. 7(a,b)) was considered. During the occlusion test, the middle or ring finger of the right hand was occluded with a rubber band. The pressure applied was above both systolic/diastolic pressures that ensured suppression of blood circulation within the finger and, hence, its oxygenation. Hyperspectral images of the intact skin before the occlusion, after 3 minutes of continuous occlusion and 1 minute after the release of the occluding ring were obtained. Finally, 2D maps of blood volume fraction and blood oxygen saturation were reconstructed. Spatial variations in the thickness of the epidermis and melanin content of the skin were also taken into account. Figure 7(c,d) shows a comparison of the measured skin reflectance spectra with those selected as the best fit by the trained ANN. A good general agreement of measured and simulated spectra is demonstrated. The measurements were performed for Caucasian skin at a distal phalanx of a ring finger before (Fig. 7(c)) and during the occlusion (Fig. 7(d)) resulting in a simultaneous decrease of blood volume fraction and oxygenation.

Fig. 7.

Photographs of the Caucasian (a) and Indian (b) skin in vivo used for the measurements. Measured skin diffuse reflectance versus best fit selected by the trained ANN for Caucasian skin at different values of blood volume fraction and oxygenation before (c) and during (d) the occlusion. The measurements were performed at a distal phalanx of a ring finger. Red and green colors indicate the parts of the spectrum used for estimation of skin blood volume fraction and oxygenation, correspondingly.

The reconstructed values of epidermal thickness and melanin content for the considered skin types are presented in Fig. 8(a,i) and Fig. 8(b,j), correspondingly. The average value of epidermal thickness in both cases is within 140–180 $\mu$m that is typical for palmar skin. The uneven distribution of the epidermal thickness across the measured area can be explained by the intra-individual variability related to uneven loads to the different parts of the palm during the lifetime. The average recovered value of melanin content is 1.3% for Caucasian skin and 8% for Indian skin. Fig. 8(c–e) and (k–m) present the reconstructed maps of blood volume fraction during the occlusion test for both considered skin types. It is seen that in the case shown in Fig. 8(c) the blood content for the distal phalanges is up to 3 times higher than for the middle or proximal ones. This increase could also be noticed from the photograph presented in Fig. 7(a). For the second case (see Fig. 8(k)), the corresponding increase is about 2 times. During the occlusion (see Fig. 8(d,l)), in both cases, the significant decrease in blood content is observed. The release of the occlusion ring causes the appearance of the reactive hyperemia resulting in the increase of blood volume fraction (see Fig. 8(e,m)) that can be reliably detected by the developed system in case of low and high melanin content of the skin. Figure 8(f–h) and (n–p) present the obtained maps of the blood oxygenation before, during and after the occlusion for both of skin types. The average blood oxygen level is of 81–84% for not occluded tissues. The obtained parameter represents tissue oxygenation and corresponds to the average between the oxygen saturation of the arterial (95–99%) and venous (∼75%) blood [42]. Three-minute occlusion lowers the blood oxygen content down to the level of 60–65% (see Fig. 8(g,o)). After the release of the occlusion ring, the oxygenation rapidly recovers back to its initial values (see Fig. 8(h,p)).

Fig. 8.

Reconstructed values of the epidermal thickness (a, i) and melanin content (b, j) for the Caucasian and Indian skin, correspondingly. Retrieved maps of blood volume fraction before (c, k), during (d,l) and 1 min after (e,m) the ring/middle finger occlusion for the considered skin types. Corresponding maps of skin blood oxygenation before (f,n), during (g,o) and 1 min after (h, p) the occlusion.

Thus, the developed hyperspectral system has shown the ability to sense the alterations of blood volume fraction and blood oxygen level in human skin in vivo and biotissue phantoms. The melanin content and epidermal thickness of skin have been taken into account to obtain correct values of blood fraction and oxygenation for different skin types. The implementation of the ANN-based processing allows for fast recovery of the considered skin parameters. The mean processing time of one hyperspectral measurement including retrieval of 2D maps of the above-mentioned skin parameters on an Intel Core i7 computer with 4 CPU cores and 16 Gb RAM requires about 3s. The potential of further decrease in the processing time using GPU units is also available.

An alternative approach for the assessment of blood content and oxygen saturation in biotissues is spatial frequency-domain imaging (SFDI) [43]. In this approach, the tissue is illuminated by a sinusoidal light pattern of different spatial frequencies and wavelengths. The benefit is the control over the penetration depth of the light in biotissues by accounting for corresponding spatial frequencies. However, this approach also utilizes a simplified tissue model (one- or two-layer) [44] that limits its application in the hyperspectral domain.

3.3 System limitations and future directions

Although the performance of the proposed data processing algorithm allows for near real-time processing of the hyperspectral images, the time of the hypercube acquisition (∼30s depending on the exposure time) and the duration of the data transfer process (∼10s) via USB interface create a bottleneck in the pipeline of the measurement procedure thus imposing the main limitation on the system performance. Nevertheless, it should be noted that in this paper, we focus mostly on the data processing rather than data acquisition. The spatial resolution of the system is estimated to be of 0.5 mm. The limited field of view applies certain restrictions on the large-scale skin imaging as it would require performing consecutive measurements with subsequent image stitching.

The future work will focus on the development intended to overcome the described limitations. In particular, critical assessment of data acquisition procedure and optimization of the illumination intensity will create a possibility to improve the system performance towards the real-time applications.

4. Conclusion

A compact, hand-held hyperspectral imaging system utilizing ANN-based processing for the reconstruction of 2D maps of blood volume fraction and blood oxygen saturation in the skin was developed. For the correct quantitative evaluation of the parameters mentioned above, the melanin fraction distribution and epidermal thickness of the skin were taken into account. The developed approach is capable of close to the real-time hyperspectral image processing based on the advanced seven-layer skin model.

Phantom studies have shown that the correct estimation of the thickness of the superficial bloodless layer plays an important role in the accurate estimation of blood oxygenation. Using the proposed method, the blood oxygen saturation in the 250-$\mu$m-thick capillary can be correctly assessed up to the embedding depth of 2 mm in the biotissue phantom mimicking human skin.

The system was also used to perform trial measurements including occlusion tests with healthy volunteers. As a result, 2D spatial distribution of blood volume fraction, blood oxygen saturation, as well as melanin content and epidermal thickness of the skin, were obtained.

The presented approach has a strong potential for clinical applications, e.g. for monitoring and diagnosis of chronic skin ulcer and other relevant skin diseases along with cosmetological defects.

Funding

Academy of Finland10.13039/501100002341 (290596, 314369, 318281); Ministry of Science and Higher Education of Russian Federation (0035-2019-0014).

Disclosures

The authors declare no conflicts of interests relevant to this article.