Label-Free Hyperspectral Imaging and Quantification Methods for Surgical Margin Assessment of Tissue Specimens of Cancer Patients

Abstract

Hyperspectral imaging (HSI) is a relatively new modality in medicine and can have many potential applications. In this study, we developed label-free hyperspectral imaging for tumor margin assessment. HSI data, hypercube (x,y,λ), consists of a series of images of the same field of view that are acquired at different wavelengths. Every pixel in the hypercube has an optical spectrum. We collected surgical tissue specimens from 16 human subjects who underwent head and neck (H&N) cancer surgery. We acquired both HSI, autofluorescence images, and fluorescence images with 2-NBDG and proflavine from the specimens. Digitized histologic slides were examined by an H&N pathologist. We developed image preprocessing and classification methods for HSI data and differentiate cancer from benign tissue. The hyperspectral imaging and classification method was able to distinguish between cancer and normal tissue from oral cavity with an average accuracy of 90±8%, sensitivity of 89±9%, and specificity of 91±6%. This study suggests that label-free hyperspectral imaging has great potential for surgical margin assessment in tissue specimens of H&N cancer patients. Further development of the imaging technology and quantification methods is warranted for its application in image-guided surgery.

I. Introduction

Surgery plays a major role in treating cancer and cures approximately 45% of all cancer patients [1, 2]. To cure a cancer patient by surgery, the surgeon needs remove the entire tumor during surgery. Unfortunately, up to 39% of patients who undergo surgery leave the operating room with positive or close margins [1, 3, 4]. It has been reported that a complete resection is a major predictor of patient survival for solid cancers [5]. Various methods have been developed for tumor margin assessment. Visual appearance and palpation are often used by a surgeon to differentiate between malignant and benign tissue [6]. However, this assessment is subjective and inaccurate. Intra-operative frozen tissue is commonly used to assess positive margin at initial surgery [7]. Small samples from the surgical bed are selected to evaluate presence or absence of residual cancer [8]. However, intraoperative frozen section diagnosis may suffer from errors that occur during sampling and histological interpretation. In addition, histological processing can take time [9], which is labor intensive and prolongs surgery time. Fluorescence imaging guided cancer resection has been shown to improve the number of complete resections [10-16]. However, fluorescence-based approaches require the injection of a fluorescence imaging agent during surgery and the agent needs to be approved by the U.S. Food and Drug Administration. There are clinical needs to develop label-free imaging technology and quantification methods to aid decision making during image-guided surgery.

Hyperspectral imaging (HSI), also called imaging spectrometer [17], was developed for remote sensing and has been explored for various applications by NASA [18]. With the advantages of acquiring two dimensional images across a wide range of electromagnetic spectra, HSI has been applied to numerous areas including vegetation and water resource control [19] [20], food quality and safety control [21] [22], forensic medicine [23][24], and biomedical area [25][26].

Hyperspectral imaging, as a label-free imaging technology, does not require a contrast agent and offers great potential for objective assessment of cancer margins. Light delivered to the biological tissue undergoes scattering from inhomogeneity of biological structures and absorption primarily in hemoglobin, melanin and water as it propagates through tissue [27] [28]. The tissue absorption, fluorescence and scattering characteristics change with the progression of diseases [29]. The reflected, fluorescent and transmitted lights from tissue, where are captured by HSI, carry diagnostic information about tissue pathology [30] [31] [29, 32]. Recent advancements of hyperspectral cameras, image analysis methods and computational power make many exciting applications possible, for example, cancer detection and image-guided surgery [33].

In this study, we developed hyperspectral image preprocessing and quantification methods to distinguish tumor from benign tissue. We evaluated the hyperspectral imaging and classification method in surgical tissue specimens from head and neck cancer patients who underwent surgery.

II. Tissue Collection and Imaging Experiments

A. Tissue Collection and Histological Processing.

Human subjects who underwent head and neck cancer surgery at Emory University Hospitals Midtown were recruited into the study. The study protocol was approved by the Institutional Review Board of Emory University. During surgery, fresh surgical specimens were collected and sent to pathology for histological assessment. Three tissue samples, i.e., i) clinically visible tumor, ii) surrounding benign tissue, and iii) tumor with adjacent benign tissue, were collected from the main specimen of each consented subject.

Each tissue specimen was transported to the research laboratory for hyperspectral and fluorescence imaging experiments, as described in the next section. After imaging, the resected specimen were fixed in formalin overnight and then sent to pathology for the standard histologic processing with hematoxylin and eosin (H&E) staining. The pathologic slides were digitally scanned, and an experienced pathologist outlined the tumor border on the digitized images for validation

B. Imaging System and Image Acquisition

We used a Maestro (PerkinElmer Inc., Waltham, MA) imaging camera to acquire the hyperspectral dataset. This wavelength-scanning system consists of a Xenon light source, a solid-state liquid crystal filter and a 12-bit high-resolution charge-coupled device (CCD). Details about this system has been described in our previous papers [34] [35]. This system is capable of obtaining reflectance images over the range of 450 - 950 nm, as well as fluorescence images under different excitation light sources [36].

Each tissue specimen was scanned using the following steps: 1) Acquire white and dark reference hypercube before tissue imaging. White reference image cubes were acquired by placing a standard white reference board in the field of view. The dark reference cubes were acquired by keeping the camera shutter closed. 2) Acquire reflectance hyperspectral images of the specimen from 450-900 nm with 5 nm intervals. 3) Acquire fluorescence imaging with 2-NBDG (Cayman Chemical, Ann Arbor, Michigan) by incubating tissue, as described in [37]. 4) Acquire florescence imaging with proflavine (Sigma Aldrich, St. Louis, Missouri) similarly as described in Step 3) above.

III. Quantitative Image Analysis Methods

Fig. 2 shows the flowchart of the hyperspectral image processing, feature extraction, image classification, and cross validation steps, as described in details below.

Fig. 2.

Fresh surgical specimens of tumor, normal tissue, and tumor with adjacent normal tissue from a tongue cancer patient. After the hyperspectral image acquisition, the average spectral curves of the three types of tissue are shown on the right, indicating the difference in their spectra.

A. Data Normalization

The normalization is to remove the spectral non-uniformity of the illumination device and the influence of dark current. The raw data is normalized using the following equation:

$$I_{\mathit{reflect}}(\lambda )=\frac{I_{\mathit{raw}}(\lambda )-I_{\mathit{dark}}(\lambda )}{I_{\mathit{white}}(\lambda )-I_{\mathit{dark}}(\lambda )}$$

Where I_refelct(λ) is the calculated normalized reflectance value at the wavelength λ. I_raw(λ) is the intensity value of the sample pixel. I_white(λ) and I_dark(λ) are the corresponding pixel intensities from the white and dark reference images at the wavelength λ.

B. Glare Detection and Removal

Glare was usually from the specular reflection of the moist tissue surface and the region does not contain useful diagnostic information of the tissue. As we reported in [38], the glare detection method includes the three steps: i) Estimate the first-order derivatives of spectral curves with a forward difference method; ii) Calculate the standard deviation (SD) of each derivative curve and generate an SD image for each hypercube. Glare pixels show higher SD than normal pixels. iii) Compute the intensity histogram of each SD image, fit the histogram with a loglogistic distribution, and then experimentally identify a threshold that separates glare and non-glare pixels.

C. Hyperspectral Image Classification

The classification step is to differentiate tumor pixels from normal tissue. A 2-class (tumor vs. normal tissue) classification was performed for each pixel on the HSI image. To reduce computational time without reducing accuracy, spectral curves were averaged in non-overlapping blocks of 5× 5 to yield a spectral signature per block. Blocks containing glare pixels were excluded from classification process. We extract features from the hyperspectral data. The extracted spectral features included i) first-order derivatives of each spectral curve, which reflect the variations of spectral information across the wavelength range; ii) second-order derivatives of each spectral curve, which reflect the concavity of the spectral curve; iii) mean, stand deviation (std), and total reflectance at each pixel, which summarize the statistical characteristics of the spectral fingerprint; and iv) Fourier coefficients (FCs), which were initially found to be effective for target detection in the remote sensing field. For HSI classification, we implemented ensemble linear discriminant analysis (LDA) and support vector machine (SVM) using MATLAB (MathWorks, Natick, MA). After classification, each block on the image was assigned a label as cancerous or normal.

D. Pathological Validation

Pathologic images of the same surgical specimen were used to validate the cancer detection of hyperspectral image classification. On the digitized H&E-stained pathological images, the tumor margin was outlined by an experienced H&N pathologist. To assess the performance of the classification, we chose the regions of interest (ROIs), where the tumor or normal tissue were histopathologically confirmed by the pathologist, for analysis and validation.

The accuracy, sensitivity and specificity of the classifiers for each patient were calculated based on the number of correctly classified tumor and normal pixels/blocks of all the specimens belonging to the patient. The accuracy, sensitivity, and specificity are defined in the following equations (TN: true negative, TP: true positive, FP: false positive, FN: false negative):

$$\begin{array}{c}\hfill \text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{FP}+\text{FN}+\text{TN}};\hfill \\ \hfill \text{Sensitivity}=\frac{\text{TP}}{\text{TP}+\text{FN}};\text{Specificity}=\frac{\text{TN}}{\text{TN}+\text{FP}}\hfill \end{array}$$

IV. Results

We collected three types of tissue specimens from each of 16 human subjects, which include: i) Clinically visible tumor tissue without necrosis, ii) benign tissue, and iii) Tumor with adjacent benign tissue at the tumor-benign interface. Fig. 2 shows the three tissue specimens and their corresponding spectral curves. The three types of tissue demonstrate different spectral curves

For each subject, we used the images of the tumor and benign tissue to train the classification algorithms and then used the tumor with adjacent benign tissue to test the performance of the classification methods. In another word, the classification method built the training model with the spectral features extracted from the tumor and benign tissue and was then evaluated on the tumor-benign interface tissue of the same patient. Using the reflectance spectra from hyperspectral imaging, the hyperspectral imaging method was able to distinguish between cancer and normal tissue of the oral cavity with an average accuracy of 90±8%, sensitivity of 89±9%, and specificity of 91±6%.

Fig. 3 shows the photographs of the tumor and normal tissue as well as the tumor with adjacent normal tissue for a tongue cancer patient. The corresponding histological slides, hyperspectral images, autofluorescence images, fluorescence images with the vital dyes 2-NBDG and Proflavine are also shown for the comparison. In our preliminary results of these 16 human subjects, hyperspectral imaging outperformed autofluorescence imaging, 2-NBDG and proflavine fluorescence imaging for cancer detection on the surgical specimens. The tumor margin as assessed by the classification method was close to that of the histological image outlined by the pathologist.

Fig. 3.

Surgical specimens from a tongue cancer patient. Left: Photograph and histological images of the tumor, normal tissue, and the tumor with adjacent normal tissue. Right: Hyperspectral reflectance composite image, autofluorescence image, the fluorescence images with the vital dyes 2-NBDG and Proflavine of the same tissue specimens. Although a single HSI composite image may visually look similar as the other three mages, the quantitative analysis and classification of the hundreds of bands from HSI data are able to provide a tumor map at the pixel level.

V. Conclusion

Hyperspectral imaging is a label-free imaging technology that does not require the injection of an imaging contrast agent. It offers potential for noninvasive detection of cancer and for rapid assessment of surgical tissue specimens. Our quantitative image analysis methods are able to distinguish cancer from benign tissue in fresh surgical specimens of head and neck cancer patients.

In this study, we used the tumor and normal tissue from the same patient to train the classification and then to classify the tumor tissue with adjacent normal tissue. This approach provides reliably results and high accuracy for differentiating tumor from normal tissue. This approach is reasonable during surgery because this technology is to help the surgeon to differentiate the tumor margin while the clinician normally knows the tumor core but is not certain about the boundary of the tumor. We are also testing another approach that uses different patients’ data to train the classification and then test the method on the new patient. This requires a large database and we are collecting tissue from more than 120 patients. The combination of the two approaches may be able to provide a useful tool for the surgeon to achieve complete resection and thus improve survival and outcome.

Fig. 1.

The flowchart of the hyperspectral image processing, feature extraction, classification and validation.