Skip to main content
PMC home page
Scientific Reports logoLink to Scientific Reports
. 2014 May 12;4:4924. doi: 10.1038/srep04924

Polarization-Sensitive Hyperspectral Imaging in vivo: A Multimode Dermoscope for Skin Analysis

Fartash Vasefi 1, Nicholas MacKinnon 1, Rolf B Saager 2, Anthony J Durkin 2, Robert Chave 1, Erik H Lindsley 1, Daniel L Farkas 1,3,a
PMCID: PMC4017245  PMID: 24815987

Abstract

Attempts to understand the changes in the structure and physiology of human skin abnormalities by non-invasive optical imaging are aided by spectroscopic methods that quantify, at the molecular level, variations in tissue oxygenation and melanin distribution. However, current commercial and research systems to map hemoglobin and melanin do not correlate well with pathology for pigmented lesions or darker skin. We developed a multimode dermoscope that combines polarization and hyperspectral imaging with an efficient analytical model to map the distribution of specific skin bio-molecules. This corrects for the melanin-hemoglobin misestimation common to other systems, without resorting to complex and computationally intensive tissue optical models. For this system's proof of concept, human skin measurements on melanocytic nevus, vitiligo, and venous occlusion conditions were performed in volunteers. The resulting molecular distribution maps matched physiological and anatomical expectations, confirming a technologic approach that can be applied to next generation dermoscopes and having biological plausibility that is likely to appeal to dermatologists.

The use of light to diagnose disease is as old as human civilization, but over the past thirty years this field has exploded as the medical industry seeks to harness the power of new sensors, new light sources and digital data processing in the service of better health care1. When light interacts with tissue, it is usually altered in some way before being remitted and detected by either the human eye, an image sensor or a point probe. Photons can be scattered by structural proteins such as collagen or membranes, have their polarization altered following multiple scattering events, or be absorbed by molecules such as hemoglobin or melanin. Light can have its wavelength distribution shifted by interaction at the atomic and molecular levels, producing fluorescence or Raman signals. Interpreting these changes can provide diagnostically useful information about the underlying structure of the tissue, provided that there is a plausible biological rationale for the change.

Billions of dollars have been spent to bring devices to the market, but many of those that showed promise in smaller studies failed – in either performance, adoption rate, or both – when applied to larger populations2,3,4. This is especially true of systems trying to better diagnose complex diseases like cancer.

Some commercial products and many research devices for skin analysis attempt to define tissue characteristics based on spectral measurements followed by feature extraction algorithms and statistical analysis5,6. These statistical classifiers are used to decide whether a tissue has a particular pathology, but there is little information that can be directly related to the tissue biology providing a model that does not distinguish between correlation and causation. This makes it difficult to evaluate the algorithm for the biological plausibility that usually engenders clinical confidence in a medical device6.

In skin studies, which constitute our focus here, it has been noted that in one commercially applied technology (SIAscopy), the limited multi-wavelength measurements are inadequate for the light-tissue model being applied7, because the results do not adequately correlate with pathology8.

A simple test of biological plausibility, where measured results are compared to known published, physiologically reasonable values, might lead to better algorithms and more accurately reflect the underlying biology. Instead, we see instances of results that are contrary to physiological expectations, such as local variation in oxygen saturation under perfectly normal pigmented nevi9, data showing that people of different races have different regional oxygen saturation10, or that collagen fluorescence is different under pigmented and non-pigmented regions11.

Instead of statistical classifiers, which tend to be only indirectly linked to physiological features, we believe that technological development should be focused on elucidating physiologically important structures and processes both faster and more accurately, so that clinicians may detect, quantify and manage treatment of skin problems including melanoma or basal cell carcinoma, chronic wounds like diabetic or pressure ulcers resulting from a compromised dermis, burn wounds, as well as fungal or bacterial infections.

There are a variety of algorithms that have been used to quantify skin chromophores that employ tissue light-transport models. Various forward models can be employed ranging from Beer-Lambert12 and Kubelka-Munk9, to the approximation of the Radiative Transfer Equation (RTE)10. The governing equation for light transfer through tissue can be solved using Monte Carlo13,14,15, finite element16, or discrete ordinate17 methods. These approaches vary in terms of computational speed. Real time algorithms usually are associated with relatively simple models such as ratiometric analysis18,19. Real time computation (30 ms to 1000 ms) is ideal for extracting high resolution skin chromophore two-dimensional maps from three- dimensional spectral image stacks with millions of voxels. These rapid quantification algorithms range from ratiometric calculations of skin reflectance maps at various wavelengths to Beer-Lambert20 or two-flux Kubelka-Munk models (up to few minutes) for homogenous turbid media21,22. Alternatively, models of light propagation can accommodate heterogeneity by incorporating two or more layers. This typically increases complexity by enabling prediction of layer thicknesses as well as chromophore concentrations for each specific layer23,24. The complex geometry of skin requires computationally intensive non-linear regression (e.g. Levenberg-Marquardt25) to fit the measured spectral signature with the estimated spectral signature derived from the related forward model.

In the past, we applied optical imaging to the research and clinical challenges involved in understanding, detecting and treating skin cancer including melanoma, using spectral imaging systems ranging from the microscopic to the macroscopic26,27,28,29,30,31.

It has become evident that, even with complex algorithms, misestimation of chromophore concentrations has been reported. High skin melanin content usually leads to over-estimation of deoxy-hemoglobin and total hemoglobin and consequent under-estimation of hemoglobin oxygenation. Recent studies by Kapsokalyvas et al.19 and Kuzmina et al.32 have shown unusual estimation of hemoglobin contrast affected by melanin hyperpigmentation. The problem persists in complex models where dark-skinned subjects always seem to have much lower oxygenation compared to Caucasian subjects, as presented by Yudovsky et al.10 and Vyas et al.9. Terstappen et al.8 showed a poor correlation between the SIAscans and histopathological findings in pigmented skin lesions, and attributed this error to misrepresentation of melanin and blood content due to high concentrations of melanin disturbing the quantification algorithm determining blood and collagen distributions. This issue is particularly critical for assessment of suspicious lesions for skin cancer (melanoma and non-melanoma) where high melanin content masks accurate determination of hyper vascularization and metabolism, which are both classic indicators of cancer33.

Some researchers have tried to minimize the effect of melanin on the misestimation of other chromophores. Kapsokalyvas et al.19 used two color polarization images to extract image contrast related to superficial melanin and employed it to correct the blood map. Another approach34,35 used two orthogonal polarization measurements of skin lesions and computed an image based on degree of linear polarization. They predicted that the degree of polarization image would eliminate the effect of superficial melanin which they suggested acts like a neutral density filter, attenuating both the superficial and deeply penetrating light equally. However, they showed in other work that this method was only partially effective in a benign pigmented nevus with a high melanin concentration35.

Our study here describes a method which combines two depth-sensitive techniques: polarization and hyperspectral imaging, to produce a new multimode dermoscope that accurately determines the spatial distribution of melanin and hemoglobin oxygenation in a skin lesion. We have developed a method that accurately separates the contribution of superficial melanin in order to quantify the deep melanin relative concentration so that oxy- and deoxy-hemoglobin distribution can be accurately assessed. This provides biologically plausible measurements that can be used to determine the lesion anatomy and physiology. The superficial melanin is primarily found in melanosomes migrating to the skin superficial layer as a part of normal epidermal replacement36. The deep melanin is primarily associated with the melanocytes found on the basal layer that separates the epidermis and dermis layers.

A linearly polarized, multi-wavelength light source is used to illuminate the skin while both parallel and perpendicular polarization images of the remitted light are recorded simultaneously by two cameras. We chose skin with a melanocytic nevus (high melanin) and skin with vitiligo (low melanin) as well as skin under the influence of venous occlusion (changing hemoglobin) in order to demonstrate the effectiveness of this method for accurately distinguishing and quantifying hemoglobin and melanin distributions.

Results

Hyperspectral and polarization data from SkinSpect

As shown in Figure 1, color images of skin with a melanocytic nevus and with vitiligo, in both parallel and cross polarization modes, have been acquired. The cross polarization images show how the superficial and specular reflectance from the air/tissue interface is reduced and how more subsurface details (such as lesion boundary, micro-vascular patterns) become visible compared to the parallel polarization images34. We present the cross-polarized optical density spectrum (OD⊥), see Figure 1c, as the negative logarithm of calibrated reflectance image stacks Z⊥(x,y,λ) next to the polarized attenuation spectrum APOL (Figure 1d).

Figure 1. SkinSpect skin images and spectra of melanocytic nevus and vitiligo.

Figure 1

Open in a new tab

(a) parallel-polarized color image of skin with nevus; (b) cross-polarized color image of skin with nevus; (c) Cross polarized optical density spectra, OD⊥, of nevus core (red), nevus halo (blue), and normal skin (green), (d) Polarized attenuation spectra, Apol, of nevus core, nevus halo, and normal skin; (e) parallel-polarized color image of skin with vitiligo; (f) cross-polarized color image of skin with vitiligo (g) Cross polarized optical density spectra, OD⊥, of vitiligo (red) and normal skin (blue and green), (h) Polarized attenuation spectra, Apol, of vitiligo and normal skin.

The optical density spectra (OD⊥) and the polarized attenuation spectra (APOL), as described in eq. (5), from three regions of interest including the melanocytic nevus core (central region), halo (boundary region), and surrounding normal skin (Figure 1c–d). The optical density spectrum (OD⊥) of the melanocytic nevus core (red square) shows the highest overall spectrum optical density (red line) due to its high melanin concentration. The relatively strong melanin contribution in the melanocytic nevus core results in a high polarized attenuation (APOL). As expected, the opposite attenuation trend in the skin exhibiting vitiligo is demonstrated in Figure 1g and Figure 1h. Both the OD⊥ and APOL spectra show the absence of melanin in the area with vitiligo. Consequently, oxy- and deoxy hemoglobin attenuation are the primary contributors to the skin absorption feature. By comparing the APOL and OD⊥ spectra, it is clear that the slopes of these lines between 615 nm and 670 nm are correlated with the expected melanin concentration. For example, as shown in Figure 1, the slope of the attenuation spectrum in the melanocytic nevus core area (red lines) is steeper compared to the surrounding normal skin (green lines).

In Figure 2, we present color images from two sides of a subject's finger (dorsal and volar) captured under parallel and cross-polarization modes during application of venous occlusion. Figure 2a and Figure 2b shows color images of subject's finger (dorsal-side) The occlusion condition were induces by a plastic cuff on the imaged finger. A series of 300 × 150 pixels areas from the same field of view at the dorsal side of the finger were cropped and concatenated to form a photographic strip chart (time sequence) shown, before putting on the cuff, during occlusion, and after removal of the cuff. The same experiment was repeated with the same subject's hand while probing the volar-side of the finger. Figure 2e and Figure 2f shows color images captured by parallel and cross polarization cameras, respectively. The volar side of the finger usually has less melanin concentration compared to the dorsal side of the finger. The darker color images during occlusion are caused by higher hemoglobin absorption due to more blood pooling in superficial blood vessels. Image contrast in cross polarization mode is enhanced due to the rejection of specular and superficial reflectance and preferentially selecting deeper penetrating light.

Figure 2. SkinSpect skin images (time sequence) and spectra of same anatomical region of interest during venous occlusion.

Figure 2

Open in a new tab

(a) Color parallel-polarized image sequence and (b) Color cross-polarized image sequence of dorsal-side finger; (c) Cross polarized optical density spectra, OD⊥, and (d) Polarized Attenuation spectra, Apol, of dorsal-side finger; (e) Color parallel-polarized image and (f) Color cross-polarized image of volar-side finger; (g) Cross polarized optical density spectra, OD⊥, and (h) Polarized Attenuation spectra, Apol, of volar-side finger. (Red line: pre-occlusion, green line: during occlusion, blue line: post-occlusion).

By choosing dorsal and volar sides of the subject's finger; we can show the effect of hemoglobin variations (both oHb and Hb) in two skin locations with different amounts of melanin. The volar side of the finger usually has a lower melanin concentration9. The color images clearly show that there is more attenuation due to blood accumulation in superficial blood vessels during the occlusion. The image contrast is also enhanced by viewing through crossed polarizers which rejects the specular and superficial reflectance, which contribute little information regarding the subsurface skin composition.

Figure 2 also show the optical density (OD⊥) and polarized attenuation (APOL) spectra from three representative images before, during, and after occlusion for the dorsal and volar sides of the finger. Both OD⊥, and APOL spectra show higher attenuation during the occlusion period due to increased blood volume. During occlusion, the shape of attenuation spectra in the 500 nm–600 nm range more closely match the single absorption peak of deoxy-hemoglobin absorption spectrum as compared to the two absorption peaks of oxy hemoglobin. This change in absorption trend is a result of progressive deoxygenation of the trapped blood due to the occlusion. The optical density spectra (OD⊥) and polarized attenuation spectra (APOL) (see Figure 2c–d, and Figure 2g–h) show increase in magnitude in the 500 nm–600 nm range. In addition, there is a change in the absorption peak shape (related to hemoglobin). These changes are similar for both the dorsal and volar sides of the finger during the occlusion period. In the graphs of the spectra (Figure 2), each solid line represents the mean of the corresponding pixel area (10 × 10 pixels) shown in the related color images. The error bar represents the standard deviation of the attenuation at each wavelength for the pixels in the designated areas. The boxes in color images appear to be from different locations but are from the same anatomical location. Position change is due to slight movement of the finger during data acquisition.

Image analysis for skin compositional mapping

Figure 3 shows the derived chromophore maps of the skin with a melanocytic nevus (a–d) as well as skin with vitiligo (e–h). The skin melanin maps (see Figure 3b and Figure 3f) were calculated from the optical density spectra (OD⊥) in cross-polarization mode. For relative melanin estimation, we used a three-chromophore model including melanin, oxy-hemoglobin and deoxy-hemoglobin employing curve-fitting algorithms with the extinction coefficients of the chromophores as primary vectors. Figure 3c shows how high melanin concentration is conducive to misestimation of the hemoglobin concentrations. We applied our deep melanin estimation method described above to correct this hemoglobin misestimation. Figure 3d shows how this approach corrects the hemoglobin over-estimation in the nevus. The melanin corrected polarized attenuation spectrum (APOL-Mel_corrected) was employed for hemoglobin estimation using a two-chromophore (oHb and Hb) model and curve-fitting algorithms with the extinction coefficients of oHb and Hb as primary vectors in the 500 nm–577 nm spectral range.

Figure 3. Calculated relative skin chromophore distribution of melanocytic nevus and vitiligo showing reduction of melanin-hemoglobin misestimation.

Figure 3

Open in a new tab

(a) Color cross-polarized image of nevus; (b) relative melanin (Mel); oxy-hemoglobin (oHb), deoxy-hemoglobin (Hb), total hemoglobin (tHb), and Oxygen Saturation Percentage (OSP) distribution for skin with nevus (c) before melanin correction, (d) after melanin correction; (e) Color cross-polarized image of vitiligo; (f) relative melanin (Mel), oxy-hemoglobin (oHb), deoxy-hemoglobin (Hb), total hemoglobin (tHb), and Oxygen Saturation Percentage (OSP) map of skin with vitiligo (g) before melanin correction, (h) after melanin correction (i) relative melanin, total hemoglobin concentration profile at A-A′ in melanocytic nevus (j) relative melanin, total hemoglobin concentration profile at B-B′ in vitiligo.

Total hemoglobin was calculated by the summation of oxy-hemoglobin and deoxy-hemoglobin. The oxygenation saturation parameter (OSP) was calculated as a ratio of oxy-hemoglobin by the total hemoglobin as a percentage. By comparing Figure 3c and Figure 3d, it is clear that without the melanin correction step, the skin area with a strong melanin contribution leads to the hemoglobin overestimation (Figure 3c) while melanin correction causes the biologically implausible melanin-related hemoglobin artifact to be nearly eliminated.

Chromophore maps of skin with vitiligo were derived to evaluate the efficiency of the algorithm in skin tissue lacking melanin. The relative melanin distribution map for areas with vitiligo (see Figure 3f) matched our expectations for melanin. Without correction of melanin-hemoglobin effect, the estimated oxy- and deoxy-hemoglobin shows high correlation with melanin in vitiligo, the same effect shown in highly pigmented nevus (see Figure 3g). By applying the same melanin correction method to the APOL spectrum, we confirmed the same correction effect in the areas with vitiligo, providing a more biologically plausible hemoglobin distribution (see Figure 3h). By comparing the melanin correction effect on hemoglobin distribution, the melanocytic nevus is more strongly affected due to the greater melanin difference to the surrounding normal skin then when the correction is observed for the vitiligo condition. The line profiles of Figure 3i and Figure 3j show the relative distribution of melanin and total hemoglobin before and after melanin correction. The line profiles of hemoglobin before correction show that the more melanin present, the more significant is the effect on the accuracy of the apparent hemoglobin distribution.

To illustrate the efficiency of our SkinSpect skin chromophore estimation algorithm for skin with blood flow variations and/or ischemia, Figure 4 compares the cross-polarized color images and skin chromophore map sets for the dorsal-side and volar-side of a human finger during venous occlusion. We used the melanin corrected polarized attenuation spectra from a region of interest (100 × 150 pixels) and fitted to a two chromophore skin model (oHb and Hb) in the 500 nm–577 nm range. The deep melanin estimation was the same method that was presented for nevus and vitiligo described in a previous section of this manuscript. During venous occlusion, the oxygen saturation decreases, while the blood volume and deoxy hemoglobin concentrations increase15. We are aware that the cuff pressure can change the venous occlusion into venous and arterial occlusion state which results in different oxy and deoxy characteristics15. The deep melanin does not change during, before and after occlusion, as expected. The algorithm presented here and applied to pigmented lesions is effective for relative oxygenation saturation and total blood concentration estimation since results that were obtained for dorsal and volar sides of a finger (with different melanin contents) agree with the physiological values for oxygenation percentage (OSP) of around 65% during the occlusion and about 80% during perfusion as shown by other researchers37,38.

Figure 4. Relative molar absorptivity of Oxy-hemoglobin (oHb), deoxy-hemoglobin (Hb), total hemoglobin (tHb), melanin and oxygen saturation (OSP) m aps with corresponding color cross-polarized image of (a) dorsal finger and (b) volar finger during finger cuff occlusion.

Figure 4

Open in a new tab

Discussion

We have demonstrated the feasibility and usefulness of polarization-sensitive hyperspectral imaging as a noninvasive technique for determining anatomical and functional characteristics of skin with melanin or hemoglobin variations. The obtainment of two oppositely linear polarized hyperspectral datacubes, and an analysis algorithm that we developed have eliminated the effect of attenuation due to superficial melanin and scattering and provided a polarized attenuation function for more accurate skin chromophore quantification than current multi-wavelength imaging techniques that result in unlikely correlations between melanin and hemoglobin in their chromophore maps8,26 or in implausible oxygen saturation for skin with high melanin content9. The method that has been presented here yields biologically plausible chromophore maps when analyzing highly pigmented regions of skin. The algorithm we recommend here includes first estimating deep melanin concentration using the polarized attenuation function (APOL) slope between 600 nm and 700 nm, and then subtracting the estimated melanin absorption between 500 nm and 600 nm to provide a melanin-corrected APOL function. The resulting function can be decomposed to quantify hemoglobin and its oxygenation state relatively independent of errors introduced by melanin-hemoglobin cross-talk. The consistent results we get from vitiligo and highly pigmented skin and skin measurement with venous occlusion show that our algorithms are effectively disentangling the measurement cross-talk between hemoglobin and melanin. We expect that improved measurement accuracy and analysis algorithms can be attained by the planned expansion of our previous homogenous phantom analysis (PDMS based with multiple absorbers mimicking melanin, and oxy and deoxy hemoglobin)40 to multi-layer tissue phantoms with predetermined thicknesses, and absorber concentration and distribution.

Access to the broad range of hyperspectral data in the visible and near-infrared wavelength range allows the algorithm to employ various wavelength sub-ranges for chromophore concentration estimation to the effect that melanin-hemoglobin misestimation is further minimized. Additional optical measurements from the near-infrared range (including all our 31 wavebands) can be used in both visible and near-infrared spectral decomposition algorithms to aid in better quantitation of hemoglobin and other tissue constituents (such as fat and water). The current hemoglobin quantification method (in the visible range) can be correlated and cross-validated with additional near-infrared-based spectral decomposition described in prior work by others39.

Another possible improvement includes considering different pathlengths for the areas with higher hemoglobin and deep melanin concentrations (essentially, a shorter pathlength due to the added absorption and scattering). In our current approach we slightly underestimate the actual melanin and hemoglobin concentrations but are still able to show differences in the spatial distribution of these chromophores. Toward more absolute quantification, we will modify our methods by considering the pathlength change and validate this by multi-layer tissue-mimicking phantom analysis.

Methods

Imaging equipment

We developed a research prototype dermoscope (SkinSpect) with customized optics, hardware and software that enables the multimode imaging-based measurement of skin lesions. This hyperspectral imaging technology has been previously used in two clinical studies. The technology was determined to be non-significant risk (NSR) and has received IRB approvals from the University of Pittsburg and Cedars-Sinai Medical Center. Informed consent was obtained from all volunteers. A preliminary report on the multimode dermoscope concept was presented recently elsewhere22.

As shown in Figure 5a, the system consists of a console and a handpiece probe. A computer (UNIX operating system) controls the specimen illumination and data acquisition, image processing, archiving and data transmission. Additionally, a touch-screen computer (Windows OS) is employed to create a user-friendly, touch-responsive interface and manage patient record data and visual interfaces (Figure 5b). The spectrally programmable OneLight Spectra illumination system has a Xenon arc light source and microelectromechanically-based wavelength selection ability over the range from 468 nm to 857 nm. The handpiece contains two cameras; a central chassis; a beam splitter; and fiber guides that direct the light from the console illumination source to a fixture that positions this assembly at the correct depth to illuminate the tissue surface. Our device provides diffuse illumination to skin in a geometry that limits the amount of specular reflection to the detector. A ring-shaped linear polarizer is placed in front of the fiber optics to allow only linearly polarized light to illuminate the tissue surface. The two cameras each have a polarization filter installed and oriented orthogonally to one another. This configuration captures images of the skin that maintain the linear polarization present in reflectance from both surface and deeper layers of tissue), and cross polarization images. A synchronized image acquisition by the two spatially registered cameras generates two images of an 11 mm × 16 mm area of skin in both parallel and cross polarizations. Image stacks are acquired by hyperspectral imaging of the target area enabled by the sequential illumination with 33 wavelength bands from visible (468 nm) to near infrared range (857 nm), with a wavelength step interval of ~13 nm. Moreover, digital color images can be generated by programming the light source for broadband illumination to mimic typical Bayer filters that are used in conventional color cameras. These color images are provided for display or for comparison with standard dermoscopes. Additional system details are described in22. Figure 5c shows a sample of the SkinSpect representation after sequential image capture. The minimum spatially resolvable line-width detected by the P and X cameras was approximately 110 μm, measured by imaging a USAF 1951 resolution test target.

Figure 5.

Figure 5

Open in a new tab

(a) SkinSpect research prototype system diagram and the (b) SkinSpect console; (c) SkinSpect data output includes reflection images over the range from 467 nm to 857 nm, in parallel and cross polarization modes.

Data acquisition

To analyze the effect of melanin on hemoglobin oxygenation quantification, two volunteer subjects were selected, one with a melanocytic nevus and the other with skin exhibiting vitiligo, both on the subjects' arms. We also analyzed the effect of hemoglobin oxygenation variation on melanin quantification by venous occlusion. The volunteers were seated in a comfortable position during data acquisition in order to minimize artifacts due to subject movement.

For the occlusion measurements, we initiated three measurements of the subject's finger before initiating occlusion (by a plastic cuff on subject's finger). Five post-occlusion measurements were taken, then another five measurements after cuff removal (during reperfusion). All data were taken at 30 second intervals. Two sets of measurements, one from the volar surface of the finger and the other from the dorsal surface of the finger were acquired. This allowed us to compare the effect of melanin change on tissue oxygenation estimation as the volar side of the finger had less melanin9.

During the imaging study, we also collected parallel and cross polarization hyperspectral images from a Spectralon reflection target (Labsphere, North Sutton, NH) under the same acquisition conditions we used when acquiring the skin images. Spectralon has a reflectance greater than 98% over the spectral region of interest. These measurements were used to correct for spatial and spectral variations in instrument response and to convert the measured reflectance data from tissue to relative attenuation data as described below. Background images containing the CCD dark current image and low level stray light, which are superimposed on the measurement data, were captured and subtracted from the reference target images as well as the skin images.

Hyperspectral data analysis

A calibration step is required to adjust the spatial and spectral responses of the instrument, to correct for detector response, light source characteristics, and the instrument transfer functions. The imaging software determines camera exposure times for individual wavebands to optimize the cameras' dynamic range independent of illumination intensity variations. The calibration datacubes in both parallel (Z∥) and perpendicular (Z⊥) polarization states are computed using the following equations:

graphic file with name srep04924-m1.jpg

where Inline graphic and Inline graphic are the reflectance measurements of skin hyperspectral images by parallel and cross polarized cameras. Inline graphic and Inline graphic are the reflectance measurements of Spectralon hyperspectral images by the same parallel and cross-polarized cameras. Because Spectralon and skin scatter light differently, this portion of the calibration process may introduce a small error, requiring a “calibration factor” (fx,y,λ), as discussed in27. This calibration factor may be ignored as it cancels out, as shown in the following equations.

Both Z∥ and Z⊥ are affected by the superficial melanin absorption (Tmel_x,y,λ) acting as an absorption filter on the skin surface (described by S. L. Jacques et. al. in34). In order to remove the effect of superficial melanin attenuation on the spectrum of deeper skin chromophores, we introduce the polarization attenuation function, APOL:

graphic file with name srep04924-m2.jpg
graphic file with name srep04924-m3.jpg

where ZSuperficial is the reflectance of the skin superficial layer obtained by subtraction of the cross polarization image cube from the parallel polarization image cube34,41,42. Equation (2) shows how the calibration factor (fx,y,λ) and scattering function (Gx,y,λ) at each pixel (x, y)and wavelength (λ) can be corrected by the division of ZSuperficial by Z∥. RSuperficial is the backscatter light mainly from the pigmented epidermis. R∥ includes superficially and deeply penetrating reflected light affected by both superficial and deep melanin as well as oxy- and deoxy- hemoglobin. Conventionally optical density function has a minus sign in the logarithmic function, OD⊥ = −log(Z⊥(x,y,λ)), however in the Apol logarithmic function, because we have Z∥ in the denominator, the minus sign is not required. Both Z∥ and ZSuperficial include surface glare. By introducing the APOL function, by division of Z∥ and ZSuperficial, the surface glare signal which may affect absorber quantification will be substantially canceled out.

The natural logarithm of HSuperficial and H∥ can be linearly correlated with chromophore concentration using the Beer-Lambert equation as shown in eq. (4) and eq. (5)

graphic file with name srep04924-m4.jpg
graphic file with name srep04924-m5.jpg

where Cm−s, Cm−d, CHb, and CoHb are the relative concentration of superficial and deep melanin, deoxy- and oxy hemoglobin, respectively; εm, εHb, εoHb are the absorption coefficients for melanin, deoxy-hemoglobin, and oxy-hemoglobin, respectively; Lm−s, Lm−d, LHb, and LoHb are the optical pathlength of superficial and deep melanin, deoxy- and oxy hemoglobin, respectively. By substituting eq. (4) and eq. (5) into the eq. (3), the polarization attenuation datacube will be derived as below:

graphic file with name srep04924-m6.jpg

APOL isolates the absorption of deep melanin, oxy- and deoxy hemoglobin thereby simplifying the quantification of these components. The term “deep” refers to light penetration into the reticular dermis to a depth of approximately 300 μm or more34. In order to simplify the regression analysis to a linear regression problem and avoid adding nonlinear complexity, we assumed the pathlengths for the deep layer (dermis) to be equal for both deep melanin and hemoglobin (Lm−d ≈ LHb ≈ LoHb). We understand this approximation limits us to extracting only relative concentration differences in our spatial maps but we believe this provides diagnostic utility.

Hemoglobin quantification method

Oxy-hemoglobin has two absorption coefficient maxima at 542 nm and 574 nm wavelengths and deoxy-hemoglobin exhibits a single absorption coefficient maximum at 545 nm. Melanin has a steadily linearly decreasing absorption trend in the spectral range from 600–700 nm and the slope of this curve increases proportional to the melanin content of an individual's skin43. Light absorption by melanin and hemoglobin are similar in magnitude at wavelengths between 500–580 nm and hemoglobin or melanin concentration changes can be confused with one another during linear regression analysis.

Instead, both oxy- and deoxy-hemoglobin absorption drops by one to two orders of magnitude at wavelengths longer than 600 nm, while the melanin absorption is still strong. The slope of the APOL function from 615 nm to 670 nm can be correlated with the concentration of deep melanin and is less affected by the influence of hemoglobin absorption. Therefore the deep melanin spatial distribution, Meld(x,y), can be estimated as:

graphic file with name srep04924-m7.jpg

Now, the APOL function can be corrected for the deep melanin absorption determined between 615 nm and 670 nm. The corrected spectrum APOL-Mel_corrected can be analyzed to determine the oxy-and deoxy-hemoglobin concentrations using the linear least-square regression analysis in the 500 nm–577 nm wavelength range (7 wavebands). This range encompasses the local absorption spectrum maxima of both oxy- and deoxy-hemoglobin. The resulting two-dimensional hemoglobin maps enable visualization of the superficial capillary network, as well as venous and arterial plexi, which are independent of melanin variations.

Image registration

The images we collect are captured from two different cameras and sequentially over time. We must ensure that the camera images creating Z∥ and Z⊥ are aligned. We also need to ensure that within the entire image capture system the handpiece, operator and patient movements do not result in motion artifacts. Finally we need to know how well the image spot sizes (determined by the system optical resolution and light diffusion in tissue) correlate between parallel and cross polarized images at all wavelengths.

In Figure 6a, we show the skin mole boundary determined from the grayscale image at 500 nm compared to the image at 700 nm. The images show a boundary difference (likely due to wavelength dependent differences in melanin absorption and scattering), however the images are closely registered showing virtually no motion artifact during hyperspectral scans. The optical resolution of the system is 7 pixels (~300 μm) in the cross polarization mode, as shown by imaging a USAF 1951 resolution target. The image registration shift between the boundaries of USAF 1951 target at 500 nm and 700 nm was 2 pixels as shown in as Figure 6b. We think this is due to chromatic aberration in the optical system from the beam splitter. If we assume the spot size is roughly equivalent to the optical resolution in cross polarized mode, the registration shift is less than the spot size resolution and image re-registration due to motion artifact during hyperspectral imaging and chromatic aberration is likely unnecessary. We also analyzed motional artifact effect by capturing three successive measurement sequences. The measurement sequence requires 4 seconds and we waited 10 seconds between the scans. As shown in as Figure 6c, skin mole boundaries in images from three successive scans at the same wavelength (500 nm) and polarization state show at most 5 pixels translation and the image translation is slightly less than the optical resolution. Figure 6d shows the image registration between two polarization images by overlapping the boundary image of USAF 1951 target from Z∥ at 500 nm on the grayscale reflectance image of Z⊥. The image misregistration between Z∥ and Z⊥ is less than the limiting optical resolution of the system.

Figure 6.

Figure 6

Open in a new tab

(a) Skin mole boundaries determined from the grayscale image at 500 nm (white) compared to the image at 700 nm (green); (b) resolution target boundaries determined from the grayscale image at 500 nm (white) compared to the image at 700 nm (green); (c) skin mole boundaries determined from the successive scans (typical, not best result); (d) image registration between parallel and cross polarization images.

Author Contributions

F.V., N.M. and D.L.F. conceived the algorithm idea. R.C., E.H.L. and D.L.F. designed and built the imaging system. F.V. and N.M. acquired and analyzed the data. R.B.S. and A.J.D. analyzed the data validation. F.V., N.M. and D.L.F. wrote the main manuscript text and prepared figures. All authors reviewed the manuscript.

Acknowledgments

We would like to thank our colleagues and collaborators, and particularly Drs. Greg Bearman and Eugene Gussakovsky (Spectral Molecular Imaging, Inc.), Milind Rajadhyaksha, Ashfaq Marghoob and Allan Halpern (Memorial Sloan Kettering Cancer Center), and Laurent Pilon (UCLA) for very helpful suggestions and encouragement pertaining to this work. Drs. Durkin and Saager acknowledge partial funding support by NIH P41EB015890 from the National Institute of Biomedical Imaging and Bioengineering, and Dr. Farkas acknowledges the US Department of Health and Human Services for support under the Qualifying Therapeutic Discovery Program of the Patent Protection and Affordable Care Act of 2010.

Footnotes

R.B.S. and A.J.D. declare no competing interests. F.V. is a Research Scientist for SMI. N.M. and R.C. are Consultants for SMI. D.L.F. is CEO and Chairman of SMI. E.H.L. was the President of SMI in the past. SMI is a for-profit corporate entity that intends to use the multimode dermoscope and approach described in the manuscript for early diagnosis of melanoma, and commercialize the device, after securing regulatory approval for it.

References

  1. Kollias N. & Stamatas G. Optical non-invasive approaches to diagnosis of skin diseases. J Invest Derm Symp P 7, 64–75 (2002). [DOI] [PubMed] [Google Scholar]
  2. Elbaum M. et al. Automatic differentiation of melanoma from melanocytic nevi with multispectral digital dermoscopy: a feasibility study. J Am Acad Dermatol 44, 207–18 (2001). [DOI] [PubMed] [Google Scholar]
  3. Monheit G. et al. The performance of MelaFind: a prospective multicenter study. Arch Dermatol 147, 188–94 (2011). [DOI] [PubMed] [Google Scholar]
  4. Kupetsky E. A. & Ferris L. K. The diagnostic evaluation of MelaFind multi-spectral objective computer vision system. Expert Opinion on Medical Diagnostics 0, 1–7 (2013). [DOI] [PubMed] [Google Scholar]
  5. Gutkowicz-Krusin D. et al. Precision of automatic measurements of pigmented skin lesion parameters with a MelaFind multispectral digital dermoscope. Melanoma Res 10, 563–70 (2000). [DOI] [PubMed] [Google Scholar]
  6. Bergstrom K. G. MelaFind was approved by FDA: where does it fit in dermatology? J Drug Dermatol 11, 420–422 (2012). [PubMed] [Google Scholar]
  7. Moncrieff M., Cotton S., Claridge E. & Hall P. Spectrophotometric intracutaneous analysis: a new technique for imaging pigmented skin lesions. Br J Dermatol 146, 448–57 (2002). [DOI] [PubMed] [Google Scholar]
  8. Terstappen K., Suurküla M., Hallberg H., Ericson M. B. & Wennberg A. M. Poor correlation between spectrophotometric intracutaneous analysis and histopathology in melanoma and nonmelanoma lesions. J Biomed Opt 18, 061223 (2013). [DOI] [PubMed] [Google Scholar]
  9. Vyas S., Banerjee A. & Burlina P. Estimating physiological skin parameters from hyperspectral signatures. J Biomed Opt 18, 057008 (2013). [DOI] [PubMed] [Google Scholar]
  10. Yudovsky D. & Pilon L. Retrieving skin properties from in vivo spectral reflectance measurements. J Biophotonics 4, 305–314 (2011). [DOI] [PubMed] [Google Scholar]
  11. Na R., Stender I. M., Henriksen M. & Wulf H. C. Autofluorescence of human skin is age-related after correction for skin pigmentation and redness. J. Invest Derm 116, 536–540 (2001). [DOI] [PubMed] [Google Scholar]
  12. Martinez L. A non-invasive spectral reflectance method for mapping blood oxygen saturation in wounds. Proc. of the 31st Applied Imagery Pattern Recognition Workshop Washington, DC, USA, IEEE, 112–116 (10.1109/AIPR.2002.1182263) (2002). [Google Scholar]
  13. Zeng H., MacAulay C. E., Palcic B. & McLean D. I. “Monte Carlo modeling of tissue autofluorescence measurement and imaging.”. Advances in Laser and Light Spectroscopy to Diagnose Cancer and Other Diseases, SPIE OE/LASE'94 Los Angeles, CA, USA 94–104 (10.1117/12.175985) (1994). [Google Scholar]
  14. Wang L., Jacques S. L. & Zheng L. MCML—Monte Carlo modeling of light transport in multi-layered tissues. Comput Meth Prog Bio 47, 131–146 (1995). [DOI] [PubMed] [Google Scholar]
  15. Tsumura N., Kawabuchi M., Haneishi H. & Miyake Y. Mapping pigmentation in human skin from a multi-channel visible spectrum image by inverse optical scattering technique. J. Imaging Sci. Technol. 45, 444–450 (2001). [Google Scholar]
  16. Katika K. M. & Pilon L. Steady-state directional diffuse reflectance and fluorescence of human skin. Appl Optics 45, 4174–4183 (2006). [DOI] [PubMed] [Google Scholar]
  17. Guo Z. & Kim K. “Ultrafast-Laser-Radiation Transfer in Heterogeneous Tissues with the Discrete-Ordinates Method.”. Appl Optics 42, 2897–2905 (2003). [DOI] [PubMed] [Google Scholar]
  18. Kapsokalyvas D. et al. Spectral morphological analysis of skin lesions with a polarization multispectral dermoscope. Opt Express 21, 4826–4840 (2013). [DOI] [PubMed] [Google Scholar]
  19. Diebele I. et al. Clinical evaluation of melanomas and common nevi by spectral imaging. Biomed Opt Express 3, 467–472 (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Attas M. et al. Visualization of cutaneous hemoglobin oxygenation and skin hydration using near-infrared spectroscopic imaging. Skin Res Technol 7, 238–245 (2001). [DOI] [PubMed] [Google Scholar]
  21. Anderson R. R. & Parrish J. A. The optics of human skin. J Invest Derm 77, 13–19 (1981). [DOI] [PubMed] [Google Scholar]
  22. MacKinnon N. B. et al. In vivo skin chromophore mapping using a multimode imaging dermoscope (SkinSpec™). Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues XI, San Francisco, CA, USA, Proc. SPIE 8587, 85870U (10.1117/12.2005587) (2013). [Google Scholar]
  23. Saager R. B., Truong A., Cuccia D. J. & Durkin A. J. Method for depth-resolved quantitation of optical properties in layered media using spatially modulated quantitative spectroscopy. J Biomed Opt 16, 077002 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Yudovsky D. & Durkin A. J. Spatial frequency domain spectroscopy of two layer media. J Biomed Opt 16, 107005 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Zonios G., Bykowski J. & Kollias N. Skin melanin, hemoglobin, and light scattering properties can be quantitatively assessed in vivo using diffuse reflectance spectroscopy. J Invest Dermatol 117, 1452–1457 (2001). [DOI] [PubMed] [Google Scholar]
  26. Kirkwood J. M. et al. Systemic interferon-treatment leads to Stat3 inactivation in melanoma precursor lesions. Mol Med 5, 11–20 (1999). [PMC free article] [PubMed] [Google Scholar]
  27. Jacques S. L. & McAuliffe D. J. The melanosome: threshold temperature for explosive vaporization and internal absorption coefficient during pulsed laser irradiation. Photochem. Photobiol 53, 769–775 (1991). [DOI] [PubMed] [Google Scholar]
  28. Yang P. et al. Macroscopic spectral imaging and gene expression analysis of the early stages of melanoma. Mol Med 5, 785–794 (1999). [PMC free article] [PubMed] [Google Scholar]
  29. Farkas D. L. & Becker D. Applications of spectral imaging: detection and analysis of human melanoma and its precursors. Pigm Cell Res 14, 2–8 (2001). [DOI] [PubMed] [Google Scholar]
  30. Valesky M., Spang A. J., Fisher G. W., Farkas D. L. & Becker D. Non-invasive, dynamic fluorescence imaging of human melanomas reveals that targeted inhibition of bFGF and FGFR-1 blocks tumor growth by inducing melanoma cell apoptosis. Mol Med 8, 103–112 (2002). [PMC free article] [PubMed] [Google Scholar]
  31. Pfaff-Smith A. et al. Fluorescence imaging analysis of upstream regulators and downstream targets of STAT3 in melanoma precursor lesions obtained from patients before and after systemic low-dose interferon-α treatment. Mol Imaging 2, 65–73 (2003). [DOI] [PubMed] [Google Scholar]
  32. Kuzmina I. et al. Towards non-contact skin melanoma selection by multi-spectral imaging analysis. J Biomed Opt 16, 060502 (2011). [DOI] [PubMed] [Google Scholar]
  33. Troyanova P., Borisova E., Stoyanova V. & Avramov L. Laser-induced autofluorescence spectroscopy of benign and dysplastic nevi and malignant melanoma. International Conference on Lasers, Applications, and Technologies 2005: Laser Technologies for Environmental Monitoring and Ecological Applications, and Laser Technologies for Medicine St. Petersburg, Russia, Proc. SPIE 6284, 62840K (10.1117/12.714174) (2005). [Google Scholar]
  34. Jacques S. L., Ramella-Roman J. C. & Lee K. Imaging skin pathology with polarized light. J Biomed Opt 7, 329–340 (2002). [DOI] [PubMed] [Google Scholar]
  35. Jacques S. L., Ramella-Roman J. C. & Lee K. Imaging superficial tissues with polarized light. Laser Surg Med 26, 119–129 (2000). [DOI] [PubMed] [Google Scholar]
  36. Lin J. Y. & Fisher D. E. Melanocyte biology and skin pigmentation. Nature 445, 843–850 (2007). [DOI] [PubMed] [Google Scholar]
  37. Zuzak K. J., Schaeberle M. D., Lewis E. N. & Levin I. W. Visible reflectance hyperspectral imaging: characterization of a noninvasive, in vivo system for determining tissue perfusion. Anal Chem 74, 2021–2028 (2002). [DOI] [PubMed] [Google Scholar]
  38. Matthijs D., Hondebrink E., van Leeuwen T. & Steenbergen W. Time domain algorithm for accelerated determination of the first order moment of photo current fluctuations in high speed laser Doppler perfusion imaging. Med Bio Eng Comp 47, 1103–1109 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Meglinski I. V. & Matcher S. J. Quantitative assessment of skin layers absorption and skin reflectance spectra simulation in the visible and near-infrared spectral regions. Physiological measurement 23, 741 (2002). [DOI] [PubMed] [Google Scholar]
  40. Vasefi F. et al. Quantifying the optical properties and chromophore concentrations of turbid media using polarization sensitive hyperspectral imaging: optical phantom studies. Imaging, Manipulation, and Analysis of Biomolecules, Cells, and Tissues XI, San Francisco, CA, USA. Proc. SPIE 8587, 85870Z (10.1117/12.2005371) (2013). [Google Scholar]
  41. Morgan S. P. & Stockford I. M. Surface-reflection elimination in polarization imaging of superficial tissue. Opt. Lett. 28, 114–116 (2003). [DOI] [PubMed] [Google Scholar]
  42. Arimoto H. Multispectral Polarization Imaging for Observing Blood Oxygen Saturation in Skin Tissue. Appl Spectrosc 60, 459–464 (2006). [DOI] [PubMed] [Google Scholar]
  43. Kollias N. & Baqer A. On the assessment of melanin in human skin in vivo. Photochem Photobiol 43, 49–54 (1986). [DOI] [PubMed] [Google Scholar]

Articles from Scientific Reports are provided here courtesy of Nature Publishing Group

RESOURCES