Comparison of pump-probe and hyperspectral imaging in unstained histology sections of pigmented lesions

Abstract

Microscopic variations in melanin composition can be mapped through linear and nonlinear optical responses. Though instrumentation to measure linear attenuation is simple and inexpensive, the nonlinear response provides more degrees of freedom with which to spectroscopically resolve pigments. The objective of this study is to assess differences in imaging melanin contrast by comparing hyperspectral (linear) versus pump-probe (nonlinear) microscopy of unstained histology sections of pigmented lesions. The images and analysis we have presented here show that pump-probe uncovers a greater variation in pigment composition, compared with hyperspectral microscopy, and that the two methods yield complimentary biochemical information.

1. Introduction

Accurate detection of early-stage melanoma depends on finding biomarkers that indicate the capacity of a primary lesion to metastasize. The optical properties of melanin are particularly interesting as potential biomarkers [1]. The composition of melanin can vary, depending on the interplay between a melanocyte’s genes, gene expression, and surrounding tissue [2]. Variations in melanin composition – for example, oxidation, eumelanin/pheomelanin ratio, iron content, and aggregation – can lead to large differences in the nonlinear optical response, as measured by pump-probe spectroscopy [3,4], but only subtle differences in the linear absorption spectrum [5]. The sensitivity of (nonlinear) pump-probe microscopy to the chemical composition of melanin has led to its use for melanoma imaging [6], noninvasive in vivo histopathology [7,8] and the ability to predict metastatic behavior from biopsies of the primary tumor [9]. Linear absorption properties of melanin have also been used for melanoma diagnosis in vivo, through reflectance spectra of skin [1,5], which is inherently a combined measurement of melanin, hemoglobin, and scattering. However, no comparison has been made between nonlinear and linear optical microscopy of melanin pigment in unstained biopsy sections.

It can be argued that nonlinear spectroscopy provides better imaging contrast for melanin pigment, owing to more degrees of freedom (pump wavelength, probe wavelength, and pump-probe time delay), but this has not yet been experimentally demonstrated. Therefore, the purpose of this study is to compare imaging contrast in hyperspectral versus pump-probe microscopy. To that end, we examined with both techniques $n=15$ unstained histology sections of melanomas and pigmented lesions, as a source for natural variations in melanin content. The goal of this study was not to distinguish between different types of lesions (e.g. melanoma versus dysplastic nevus, as has already been done for pump-probe microscopy [6]); instead, we seek to quantify intra-sample pigment composition variability, as revealed by each imaging technique. Treating each image as a source of thousands of independent spectroscopic measurements of melanin pigment, we used principle component analysis to quantify pigment composition variability yielded by each technique, finding pump-probe microscopy to be more sensitive to variations in melanin composition than hyperspectral microscopy.

2. Materials and methods

2.1 Specimens

We acquired images from $n=10$ cutaneous and $n=5$ conjunctival unstained biopsy sections (see Table 1). Histology sections were cut from paraffin-embedded blocks, 5 µm thick for cutaneous and 10 µm for conjunctival specimens, de-paraffinized and coverslipped with an embedding medium (Surgipath Acrytol, Leica Biosystems, Buffalo Grove, IL, USA). This study was approved by the Duke University Institutional Review Board; specimens were pulled from archived pathology blocks, under an informed consent waiver.

Table 1. List of specimens.

#	Origin	Diagnosis	Figure Reference
1	Conjunctival	Compound cystic nevus	S1(a),(e)
2	Conjunctival	Combined nevus	S2
3	Conjunctival	Melanoma	S1(b)
4	Conjunctival	Benign nevus / primary acquired melanosis	S1(c),(d)
5	Conjunctival	Benign nevus	S1(f)
6	Cutaneous	Invasive melanoma	S1(o)
7	Cutaneous	Normal pigmentation	2(d)
8	Cutaneous	Dysplastic from satellite lesion	S1(i)
9	Cutaneous	Satellite melanoma	S1(j)
10	Cutaneous	Melanoma in situ	S1(k)
11	Cutaneous	Invasive melanoma	2(c), S1(1)
12	Cutaneous	Dysplastic from invasive melanoma	S1(g),S1(h)
13	Cutaneous	Mostly normal with dome dysplasia diagnosed as invasive malignant melanoma	2(b)
14	Cutaneous	Invasive melanoma	S1(m),2(a)
15	Cutaneous	Invasive melanoma	3, S1(n)

2.2 Hyperspectral microscopy and pre-processing

Hyperspectral images were acquired with a CytoViva Hyperspectral Microscopy System (CytoViva, Inc., Auburn, AL, USA), equipped with a halogen lamp source (Fiber-Lite DC-950, Dolan-Jenner Industries, Boxborough, MA, USA), an imaging spectrograph (V10E, SPECIM, Oulu, Finland), and a CCD camera (pco.pixelfly, PCO, Kelheim, Germany), in brightfield transmissive illumination mode. The microscope was set up to disperse the transmitted spectrum of a line focus on specimen (x-axis) along the vertical axis of the CCD imaging sensor. The specimen was translated along the y-axis by a motorized stage to build up a matrix of transmitted spectra as a function of location and wavelength: $S_{x,y,\lambda }$, with $\lambda$ ranging from 460—900 nm. For a clear illustration of the concept, we refer the readers to Ref [10].

Transmittance for each pixel was defined as $T=S/I$, where $I$ is the illumination (lamp) spectrum. Ideally, because the images were recorded one row at a time, by translating the sample, image pixels in the same column share the same illumination spectrum, $I_{x,\lambda }$, i.e., the illumination does not change as the sample is translated across the y-axis. Under this assumption, transmittance for each pixel is $T_{x,y,\lambda }=S_{x,y,\lambda }/I_{x,\lambda }$, thus compensating for illumination and detection inhomogeneity across the line focus. In practice, we notice temporal fluctuations of the overall illumination in long time scales (e.g., 5 mins). This can be compensated by recalibrating the source illumination spectrum frequently. Alternately, we assume that the illumination spectrum for each image can be modeled as, $I=a{I}_{x,\lambda }$, where $a$ serves as a normalization factor calculated from the average top 94 – 97% brightest pixels in each image. To provide data for this normalization, we made sure the field of view contained some blank regions that exhibited neither absorption or scattering in tissue. We omit the brightest pixels (98—100%) because these can contain values in which the CCD is saturated. Linearized absorbance for each pixel and wavelength was then calculated as $A_{j,k,\lambda }=-\log T_{j,k,\lambda }$.

2.3 Pump-probe microscopy

Corresponding pump probe images were acquired with equipment previously described [11]. We chose the following acquisition parameters: 0.3 mW each for 730 nm pump and 817 nm probe. Pulses were ~250 fs in duration. The inter-pulse delay was scanned from −1 to + 4 ps.

2.4 Analysis methods

We quantify the variation in absorbance spectra and pump-probe responses on a pixel-by-pixel level with a geometrical representation of principal component analysis (PCA) [9]. We describe this process in detail for the hyperspectral images below. PCA identifies a set of orthogonal basis spectra, or principal components (PCs), that can be combined to reconstruct the spectrum at any given pixel. The proportion of PCs found in each pixel is then used for further analysis. For the hyperspectral images, PCs for the entire data set were calculated from the top 20% most absorptive pixels of each image (without this restriction, PCA produces a basis that provides large separation between melanin and non-pigmented areas, but little separation between different types of melanin). In all, ~1.7 million spectra contributed to the PC calculation. Figure 1(a) shows the top three PCs, along with the fraction of data variance explained by the top five PCs in the inset. The first component contains the largest percent of the data variance (95.1%). The other two components account for small spectral shifts (PC1 has 4% of the variance, and PC 3 has 0.3%). On pump-probe images, the process is similar, except that pixels from each image that contributed to the PCA basis were restricted to those having a signal intensity 2 standard deviations above the mean, after removing non-melanin pixels (i.e. surgical ink and hemoglobin) with phasor analysis; for details, we refer the reader to [9,11].

Fig. 1.

(a) Principal components from hyperspectral images and their variances (inset). (b) Two-dimensional histogram of angles, along with proposed assignments for regions of $\left(\theta ,\varphi \right)$ space.

Because the PCs are orthonormal, the projections of the spectra onto the top three PCs spans a three-dimensional space, depicted in the inset of Fig. 1(b). By describing this space in spherical coordinates, the azimuth angle $\theta$ and elevation angle $\varphi$ contain all the spectral information of interest. A two-dimensional cumulative histogram of these angles for each pixel is shown in Fig. 1(b), where pixels with similar absorption spectra are expected to cluster together. This mapping of each pixel onto $(\theta , \varphi )$ is then used to produce the false color visualizations shown in the results.

3. Results and discussion

3.1 Individual Images

Representative hyperspectral and pump-probe images are shown in Fig. 2. A detailed comparison of false color images and $(\theta , \varphi )$ PCA projection histograms between linear andnonlinear measurements is shown in Fig. 3, for a selected case of interest. The remainder of the images are included in Fig. 4.

Fig. 2.

Representative false-color images rendered from angular projection PC analysis of both hyperspectral (top) and pump-probe images (bottom, with confocal reflectance overlaid in grayscale): (a, b) invasive cutaneous melanoma, (c, d) normal. Scale bars 100 µm. Dashed boxes: hyperspectral image field of view.

Fig. 3.

Invasive cutaneous melanoma. (a, c) False-color images. Scale bars 100 µm Dashed box: hyperspectral image field of view. (b, d) pigment histograms and representative hyperspectral and pump-probe response curves. In the histograms, markers * and ** indicate points in principal component space at the extrema of the distributions. Corresponding spectra and pump-probe responses are labeled by * and **.

Fig. 4.

False-color images rendered from angular projection PC analysis of hyperspectral (top) and pump-probe images (bottom). For legend, see Table 1. Scale bars 100 µm.

These false color images were generated from PCA, color coding each pixel based on the angles $\theta$ of its projection onto the first two PCs—see Refs [9,11]. As established in [9], pump-probe image azimuth angles (${\theta }_{P-P}$) lie in the range $1<{\theta }_{P-P}<2.7$ for melanin and around ${\theta }_{P-P}\approx 2.8$ for surgical ink. Small angles (colored magenta through yellow in Figs. 2-5) indicate a response dominated by excited state absorption, while larger angles (yellow through green) indicate an increasing contribution from ground state bleaching (cyan indicates surgical ink). The azimuthal angles for hyperspectral imaging (${\theta }_{HS}$) lie in a narrow distribution around zero. The negative ${\theta }_{HS}<0$ component appears to be predominant in the stratum corneum and in the dermis, in regions devoid of a pump-probe response from melanin. With this information, we infer that for the most part, ${\theta }_{HS}<0$ corresponds to melanin absorption, while ${\theta }_{HS}<0$ corresponds to components other than optical absorption in melanin, such as absorption in surgical ink and scattering, which is a major factor in attenuation measurements in biological tissue. The regions of negative ${\theta }_{HS}$(stratum corneum and dermis) suggest keratin and collagen as primary sources of scattering.

Fig. 5.

Conjuctival combined nevus. (a, c) False-color images. Scale bars 100 µm. (b, d) pigment histograms and representative hyperspectral and pump-probe response curves. In the histograms, markers * and ** indicate points in principal component space at the extrema of the distributions. Corresponding spectra and pump-probe responses are labeled by * and **.

We confirm that ${\theta }_{HS}<0$ in the hyperspectral images corresponds to something other than melanin absorption by comparing curves (*) and (**) in Fig. 3(b). These curves are average spectra that map to the extrema of the distribution for the image shown in Fig. 3(a). Because these curves are mapped onto the principal component basis that was derived from the entire set of images, they are representative of the entire data set. From this, it can be confirmed that hyperspectral pixels with ${\theta }_{HS}<0$ (*) have non-monotonic spectra (i.e. they exhibit a peak), which is uncharacteristic of melanin [12]. (Only one specimen did not follow this trend, a conjunctival combined nevus shown in Fig. 5; this specimen showed an uncharacteristically high melanin density.)

Figure 3(c), 3(d) show the same invasive melanoma, imaged with pump-probe microscopy. Again, the curves (*) and (**) are spectra that map to the extrema of the distribution of pump-probe responses. By comparison with Fig. 3(b), it can be seen clearly that pump-probe reveals a wider spread of the pigment distribution, with significantly greater variation in the shape of the nonlinear response than is revealed by the (linear) hyperspectral measurement.

3.2 Mathematical morphology

Judging from the false-color images in Figs. 2-5, and the principal component projection histograms in Fig. 3(b), 3(d) and Fig. 5(b), 5(d), it can be asserted qualitatively that the two imaging modalities provide similar morphological information, but very different information on pigment composition. To quantify this observation, we have distilled each image into four parameters (P1-P4) that capture morphology and pigment composition variability. The idea here is to generate a set of parameters from each image that allows for a side-by-side comparison of the spatial distribution of pigment expression as revealed by both modalities, in a way that is indifferent to variations in the field of view between the two techniques. This avoids the need for co-registration and discarding information from non-overlapping regions. What follows is a description of these parameters, expected outcomes, and then results.

The two parameters that capture image morphology are calculated with mathematical morphology methods previously described [9,13]. First, we generate images based on the radius of the principal component projection magnitude, R (see Fig. 1(b) inset), for both pump-probe and hyperspectral images. This yields images related to the overall melanin concentration as determined by each method. Next, we calculate parameter P1 as the entropy derived from the 2-dimensional autocorrelation of R, and P2 as the entropy derived from R. As expected, it can be seen from the images in Fig. 2 that hyperspectral and pump-probe provide similar morphology information – the contours of the dermis, basal layer, epidermis, and stratum corneum look the same. Given these similarities, despite differences in field of view, we expect a positive correlation between hyperspectral and pump-probe morphology parameters, e.g. $r_1=\text{corr}\left(P{1}_{\text{HS}},P{1}_{P-P}\right)>0.5$.

The two parameters that capture pigment composition variability are calculated from the principal component projections that were illustrated in Fig. 1(b). Specifically, the parameter P3 is the mean azimuth angle across the field of view, and P4 is the standard deviation of azimuth angle $\theta$.P3 is expected to reflect the inter-sample pigment variability, while P4 is expected to reflect the intra-sample variability. Given that transient spectroscopy is sensitive to many more properties of melanin composition compared to linear absorption [3], we expect these parameters to be loosely correlated at most, e.g. $\left|r_4\right|=\left|\text{corr}\left(P{4}_{\text{HS}},P{4}_{P-P}\right)\right|<0.5$.

The results of calculating P1, P2, P3, and P4 from each of the images are shown in Fig. 6, where the horizontal coordinate of each point is calculated from the pump-probe image, and the vertical coordinate is calculated from the corresponding hyperspectral image. We excluded all non-melanin pixels (${\theta }_{HS}<0$ and, ${\theta }_{P-P}>2.8$) from this analysis, and for consistency with this rule, excluded the conjunctival combined nevus as unrepresentative (Fig. 5). As shown, the morphology parameters are correlated ($r=0.79 \text{and} 0.68$ for P1 and P2, respectively), and the pigment composition parameters are not correlated ($r=-0.35 \text{and} r=-0.17$ for P3 and P4, respectively); both results are consistent with what was expected. The correlation between hyperspectral and pump-probe measurements of morphology parameters P1 and P2 confirms that, even though the images are not perfectly co-registered, they are sufficiently overlapped for quantitative comparison. In other words, the structural features as determined by each method yield similar results even though the images do not overlap on a pixel-by-pixel basis. (This is indeed the advantage of using of mathematical morphology—i.e., images with similar textures yield similar parameters.) Note that correlation in the morphology parameters validates the use of non-coregistered images for comparing pigment composition between the pump-probe and hyperspectral. Thus, we conclude that neither average pigment composition nor microscopic variation in pigment are correlated between hyperspectral and pump-probe.

Fig. 6.

Parameters from hyperspectral vs. pump-probe images. The linear correlation coefficient, r, is given for each scatter plot. (a,b) structural/morphological parameters; (c,d) pigment composition and heterogeneity parameters. See text for full description.

4. Conclusions

The images and analysis we have presented here show that pump-probe uncovers a greater variation in pigment composition, compared with hyperspectral microscopy, and that the two methods yield complimentary biochemical information. This is supported by the wider spread in ${\theta }_{p-p}$ in the pump-probe response distributions (compare Fig. 3(b) vs Fig. 3(d) and Fig. 5(b) vs Fig. 5(d)), and by the lack of correlation between pigment composition measurements between the two techniques (Fig. 6(c), 6(d)), respectively. This suggests that the response at a 730 nm pump / 817 nm probe combination is more sensitive to subtle differences in pigment composition when compared with the linear absorption spectrum. This is consistent with previous findings where chemical changes, e.g. differences in iron content, produced only subtle changes in the linear absorption spectrum [14], but resulted in dramatic differences in the pump-probe response [3]. Considering that pump-probe has access to additional spectroscopic degrees of freedom with which to separate pigments (pump wavelength, probe wavelength, and relative timing of pump and probe pulses), this result should not be surprising. Nevertheless, we found interesting and unique (i.e., uncorrelated to pump-probe microscopy) contrast in the hyperspectral images.

Drawing conclusions about differences in melanin composition between benign and malignant lesions, however, was not the purpose of this work; other studies have addressed that question [6,9], showing the potential diagnostic and prognostic value of pump-probe microscopy. It remains to be seen whether hyperspectral microscopy of unstained pigmented lesions can differentiate disease states, e.g. benign versus malignant. Even though the contrast offered by hyperspectral imaging is much weaker than pump-probe, it is much simpler and has a lower cost; therefore, further investigations may be warranted to determine its clinical significance.

Disclosures

The authors declare that there are no conflicts of interest related to this article.

Funding

NIH R01-CA166555 (WSW), the Burroughs Welcome Fund 1012639 (FER), NIH F32CA183204 (FER), NIH F32CA168497 (JWW), NSF CHE-1309017 (MCF), NSF CHE-1610975 (MCF), the Duke Center for In-Vivo Microscopy NIH P41EB015897, and Duke University.