In Vivo Cancer Detection in Animal Model Using Hyperspectral Image Classification with Wavelet Feature Extraction

Abstract

Hyperspectral imaging (HSI) is a promising optical imaging technique for cancer detection. However, quantitative methods need to be developed in order to utilize the rich spectral information and subtle spectral variation in such images. In this study, we explore the feasibility of using wavelet-based features from in vivo hyperspectral images for head and neck cancer detection. Hyperspectral reflectance data were collected from 12 mice bearing head and neck cancer. Catenation of 5-level wavelet decomposition outputs of hyperspectral images was used as a feature for tumor discrimination. A support vector machine (SVM) was utilized as the classifier. Seven types of mother wavelets were tested to select the one with the best performance. Classifications with raw reflectance spectra, 1-level wavelet decomposition output, and 2-level wavelet decomposition output, as well as the proposed feature were carried out for comparison. Our results show that the proposed wavelet-based feature yields better classification accuracy, and that using different type and order of mother wavelet achieves different classification results. The wavelet-based classification method provides a new approach for HSI detection of head and neck cancer in the animal model.

1. INTRODUCTION

Hyperspectral imaging (HSI) has been investigated for cancer detection on both in vivo^1–7 and ex vivo tissue^8–13. There are abundant information in hyperspectral data, but the traditional statistic-based classification methods are not very efficient with raw hyperspectral data^{14, 15}. For classification tasks using spectra data from hyperspectral images, feature extraction is necessary. Feature extraction methods based on the Fourier transformation (FT) and tensor decomposition were previously explored^{1, 16}. Fourier analysis, being an effective feature extraction method, decomposes a signal into cosine waves of different frequencies. Similar to some extent, wavelet transformation (WT) decomposes signal into shifted and scaled version of a mother wavelet. However, because of the variety of mother wavelets, WT can serve as a more flexible feature extraction method, and has the potential to dig deeper information than other methods. Wavelet transformation has been employed in many hyperspectral imaging applications such as denoising^{17, 18}, image compression¹⁹, anatomy detection²⁰, image fusion²¹, dimensional reduction²², and feature extraction^{14, 23–25}. It decomposes data and extracts approximate and detailed information, which enhances the discrimination of subtle differences among spectra.

This work is to investigate the feasibility of discrete wavelet transformation (DWT) as a feature extraction method for hyperspectral images of head and neck cancer detection. We explore three types of wavelet-based features, i.e., the approximation output of 1-level DWT, the approximation output of 2-level DWT, and the catenation of all the outputs of a 5-level DWT. A tumor-bearing animal model is utilized in this study, and support vector machine (SVM) is used for classifications with raw data and wavelet-based features.

2. METHODS

2.1. Animal Model

The dataset used in this experiment includes 12 HSI data from 12 mice with head and neck tumors. We used the HNSCC cell line M4E to initiate the tumor. The M4E cells were transfected of pLVTHM vector so that they contained green fluorescence protein (GFP), which was later used to generate the reference images for tumor detection. The experimental details about the animal experiments were reported in our previous papers^{3, 26–31}.

2.2. Hyperspectral imaging system

The camera used to capture hyperspectral images was a spectral-scanning CRI system (Perkin Elmer Inc., Waltham, Massachusetts), containing a fiber-optic lighting system (Cermax-type, 300-Watt, Xenon light source), a liquid crystal tunable filter (LCTF), and a 16-bit charge-coupled device (CCD)¹⁶. The images were acquired within the wavelength range of 450 nm to 950 nm with 2-nm increment. The original dimension of hyperspectral images is 1392×1040×251.

2.3. Pre-processing and spectra extraction

The pre-processing of hyperspectral images included calibration, background removal, curvature correction, noise and glare removal, and GFP bands removal.

Each hyperspectral image was calibrated using the image of a standard white reference board and the dark reference image, as described in Equation (1) below.

$$\mathit{\text{Reflectance}}(\lambda )=\frac{I_{raw}(\lambda )-I_{\mathit{\text{dark}}}(\lambda )}{I_{\mathit{\text{white}}}(\lambda )-I_{\mathit{\text{dark}}}(\lambda )}$$ (1)

where Reflectance(λ) is the normalized reflectance for wavelength λ, I_raw(λ) is the intensity value in raw hyperspectral image, I_white (λ) and I_dark (λ) are the intensity values in the white and dark reference images, respectively.

The standard white reference board is a disk, which only covers part of the area in the whole image. Therefore, the synthesized RGB image of the white reference board was used to generate a mask. For each hyperspectral image, only the area within the mask was calibrated, while the values of other pixels were all set to zero, as Figure 1(a) shows. The calibrated area includes both the mouse tissue and background, which is a metal board. Metal has very different spectrum with tissue, which would cause mismatch in the registration process. Therefore, a mask was generated using the strong reflective characteristic of metal in red-infrared range, and the mouse tissue was extracted from the background. Then, intensity-based rigid registration was carried out among all bands1, in order to eliminate the influence of mice motion caused by breathe and heartbeat. Considering the intensity variation among all 251 bands, we calculated average images of every 10 bands instead of an average image of all bands, and each band was registered to the corresponding average image. The shape of mouse body and the protrusion of the tumor make the amplitude of spectra differ a lot. Spectra of the same point captured at different distances can have the same shape with different amplitudes. Therefore, curvature correction was carried out on each hyperspectral image. For each individual spectrum, a constant was calculated as its total reflectance of all wavelengths, and the spectrum was divided by this constant to remove the intensity variation³². Because the original spectra contain noise, a 3-order median filter was applied to each hyperspectral image to remove the noise. There were a few glare pixels in hyperspectral images, mainly at the top of tumors. The spectra of these pixels have deformed shapes and higher amplitudes. A threshold was set for the sum of each spectrum, and the pixels with glares were removed. Then, the bands of 508 nm and 510 nm were removed from each hyperspectral image to avoid the influence of GFP, and the bands after 900 nm were removed because of the noise.

Figure 1.

Illustration of preprocessing results. (a) Synthesized RGB image from hyperspectral data where only the area within the mask was calibrated and other pixels were set to zero. (b) Synthesized RGB image after background removal and curvature correction, overlapped with tumor mask generated from GFP fluorescence image.

In this study, GFP fluorescence images were used as ground truth for tumor detection¹⁵. Here we generated mask of tumor using fluorescence images to differentiate cancerous tissue from normal tissue, as Figure 1(b). After the aforementioned pre-processing, average spectra of each 10×10 pixels were extracted from cancerous tissue and normal tissue, respectively. We chose to use average spectra instead of spectrum of each single pixel to reduce the impact of animal motion among all bands caused by breath and heartbeat. The extracted average spectra were used as raw spectra.

2.4. Wavelet transformation

A one-dimensional discrete wavelet transform can be considered as a combination of a high-pass filter (HPF) and a low-pass filter (LPF). The input signal is filtered, in other words decomposed, by translations and dilations of the mother wavelet. The output of each filter has a dimension lower than that of the input. The low-pass filter extracts the approximate information, while the high-pass filter extracts the details. In this work, a 5-level DWT was applied to all the raw spectra as Figure 2 shows, and all the detail outputs (cDi, i=1, 2, 3, 4, 5) and the final approximation output (cA5) were concatenated to generate the wavelet feature W.

Figure 2.

Implementation of multi-level discrete wavelet transform and the composition of proposed feature W.

In this work, wavelets from seven wavelet families (Haar, Daubechies, Symlets, Coiflets, Biorthogonal, Discrete Meyer, and Fejer-Korovkin) were used as mother wavelets. Each wavelet family has specific characteristics of symmetry, support length, and orthogonality. A wavelet family can have multiple wavelets except the Haar and Discrete Meyer wavelet. Wavelets in the same family differ in many ways. For example, wavelets in the Daubechies family generally have the same shape but different tap numbers, while those in the Biorthogonal family vary in both shape and tap number. To select the best mother wavelet for our application, we tested 16 wavelets in total, including 1 Haar wavelet (“haar”), 1 Discrete Meyer wavelet (“dmey”), 6 Daubechies wavelets (“db2”, “db4”, “db8”, “db16”, “db24”, “db32”), 3 Coiflets wavelets (“coif1”, “coif4”, “coif5”), 1 Symlet wavelets (“sym4”), 2 Biorthogonal wavelets (“bior3.3”, “bior4.4”), and 2 FejerKorovkin wavelets (“fk4”, “fk22”).

DWT has the intrinsic characteristic of preserving the shape of input signal. The approximation output of low-pass filter still has the peak and valley of the input with a reduced dimension. Therefore, except for the wavelet-based feature W, we also investigated the discriminative ability using only the approximation output of a 1-level wavelet transformation (cA1 in Figure 2) and a 2-level wavelet transformation (cA2 in Figure 2). We compare the classification results of using (1) raw data, (2) 1-level DWT approximation output, (3) 2-level DWT approximation output, and (4) wavelet-based feature W that is the catenation of all outputs of a 5-level DWT. The two approximations used for classification were generated using Haar wavelet, because it is very commonly used and has low dimension. We used 16 different wavelets from 7 wavelet families to generate the wavelet-based feature W, and W generated with each mother wavelet was tested separately, in order to find the optimal mother wavelet for this application.

2.5. Support vector machine (SVM) classification

In this study, we applied SVM with Gaussian radial basis kernel using MATLAB® (MathWorks, Natick, Massachusetts). To compare the discriminative ability, we used different features of the 12 mice, including the raw spectra, the approximation output of 1-level DWT, the approximation output of 2-level DWT, and the wavelet features W generated using different mother wavelets, to train and test the SVM classifier. We performed leave-one-out cross validation. A grid search was carried out to select the optimal values for parameters C and g. After determine the optimal values, an SVM was trained with 11 mice data, and tested on the testing mouse data.

2.6. Evaluation metrics

In this study, we use accuracy, specificity and sensitivity to evaluate the performance of classification, as shown in Equation (2). Accuracy is defined as the ratio of the number of correctly labeled pixels to the total number of pixels in the testing image. Specificity and sensitivity are calculated from true positive (TP), true negative (TN), false positive (FP), and false negative (FN), where positive corresponds to cancerous tissue and negative to normal tissue. Specificity is the ratio of TN to the sum of TN and FP, while sensitivity is the ratio of TP to the sum of TP and FN.

$$\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{FP}+\text{TN}+\text{FN}};\text{ Sensitivity}=\frac{\text{TP}}{\text{TP}+\text{FN}};\text{ Specificity}=\frac{\text{TN}}{\text{TN}+\text{FP}}$$ (2)

3. RESULTS AND DISCUSSION

3.1. Classification with raw data and wavelet approximate data

In the first stage, classification results of using (i) raw data, (ii) approximation output of 1-level Haar DWT, and (iii) approximation output of 2-level Haar DWT were compared. Classification using raw data got an average accuracy of 76.5%, as well as 64.5% average specificity and 85.4% average sensitivity. The average accuracy and average specificity of SVM classification using 1-level DWT approximation output were 74% and 62.9%, which were lower but comparable to those of raw data, while the 87% sensitivity was slightly higher than that of raw data. Because DWT separates approximation and detail, the dimension of approximation output is only half of raw spectral data’s dimension. Therefore, calculation time was decreased by at least 1/2 by using the 1-level DWT approximation. For the 2-level DWT approximation, the classification performance was not satisfying, because many details were lost. The average accuracy, specificity and sensitivity were 41.1%, 48%, and 27.1%, respectively. However, it proves the importance of detail information in spectral classification tasks.

3.2. Classification with wavelet features W

In the second stage, the performance of wavelet feature W generated using different mother wavelets was analyzed, in order to prove the feasibility of the proposed feature W, and to select an optimal mother wavelet for our application. We firstly used wavelets that are from different families but have similar tap numbers, thereby to compare the performance of different wavelet families. Twelve wavelets from six wavelet families were tested. The 12 wavelets were divided into two groups according to their tap numbers. Average accuracy, specificity and sensitivity were obtained from the testing results of 5 mice, as shown in Table 1. The Coiflets and the Daubechies wavelets performed better in both the small tap number group (tap number = 6 to 10) and the large tap number group (tap number = 22 to 64).

Table 1.

The average classification performance of 5 mice using wavelets from different families with similar tap number.

Mother wavelet	Tap number	Average Accuracy (%)	Average Specificity (%)	Average Sensitivity (%)
coif 1	6	90.2	86.6	95.4
db4	8	89.7	86.3	94.7
sym4	8	89.1	85.3	94.9
fk8	8	89.2	87.1	94.3
bior 3.3	8	88.8	86.8	95.8
bior 4.4	10	88.6	86.2	94.4
fk22	22	90.9	87.7	95.2
coif4	24	91.1	89.3	95.7
coif5	30	91.0	88.9	95.8
db16	32	90.7	89.2	94.9
db32	64	91.3	90.5	95.9
dmey	63	89.2	86.0	96.7

Secondly, we tested wavelets within the Daubechies wavelet family with different tap numbers. The reason we chose the Daubechies family is that (i) it shows better performance than others in Table 1, and (ii) it has wavelets with a wide tap number range from 2 (“db1”) to 90 (“db45”). The classification results of using feature W generated with 7 different Daubechies mother wavelets are shown in Figure 4. Apparently, the wavelets with larger tap numbers (db16, db24, and db32) generally worked better than those with smaller tap numbers (db1, db2, db4, db8).

Figure 4.

Classification performance of 5 mice using wavelets in Daubechies family with different tap numbers.

Finally, we chose Daubechies 32 as mother wavelet and tested the wavelet-based feature W on 12 mice. The classification results of all 12 mice are shown in Table 2. The classification using feature W achieved an average accuracy of 87.5%. The average specificity and average sensitivity were 90% and 83.1%, respectively. Additionally, we compared the wavelet-based method to our previous work using the same mouse dataset and a tensor-based method¹⁶, and the proposed wavelet-based feature got better accuracy than the previous tensor-based feature on 7 mice. The wavelet-based feature W shows favorable results for Mouse #892, #893, #894, #895, #897, and #885. For Mouse #891, where random scattering caused some tissue artifacts and the spectra of cancerous tissue and normal tissue overlapped, the wavelet method improved the classification results than raw data and tensor-based feature. However, our wavelet-based method had limitation for spectral data with large peak noise, which was caused by mice motion and vessels. Therefore, the classification results of Mouse #889 and Mouse #890 were not satisfactory.

Table 2.

Classification accuracy, specificity, and sensitivity of 12 mice using the proposed wavelet-based feature W.

Mouse ID	Accuracy (%)	Specificity (%)	Sensitivity (%)
895	96.4	95.7	99.2
893	96.3	93.4	99.3
894	94.8	96.5	90.0
885	94.2	92.0	96.0
892	93.3	96.7	85.4
897	93.3	91.5	96.9
888	91.4	87.7	94.2
896	87.4	99.2	59.2
891	85.0	75.8	97.7
898	82.4	88.7	80.3
890	68.0	90.0	38.1
889	67.8	72.8	61.4
Average	87.5	90	83.1

Furthermore, we compared the classification results using raw data, 1-level DWT approximation, 2-level DWT approximation, and the proposed wavelet-based feature W (based on “db32” mother wavelet), as shown in Figure 5. Our proposed feature W obviously improved the accuracy and specificity than raw data. The 1-level DWT approximation had a comparable performance with raw data but decreased calculation time because of its lower dimension. Figure 6 shows some of the segmentations of well-classified mice.

Figure 5.

The performance of all 12 mice using raw data, 1-level Haar DWT approximation output, 2-level Haar DWT approximation output, and the proposed feature W based on “db32” mother wavelet.

Figure 6.

Examples of tumor detection results using the wavelet feature W. (a) Classification result of Mouse #893. (b) Classification result of Mouse #894. (c) Classification result of Mouse #892. (d) Classification result of Mouse #897.

4. CONCLUSION

In this work, we investigated the feasibility of discrete wavelet transformation as a feature extraction method for in vivo head and neck cancer detection using hyperspectral imaging. We tested three kinds of features, i.e. the approximation output of 1-level DWT, the approximation output of 2-level DWT, and the catenation of all detail outputs of a 5-level DWT and the final approximation output. The three kinds of features showed different discriminative abilities for our application. The 1-level DWT approximation yielded comparable classification performance with raw spectra data, while decreased the calculation time to 1/2 of raw spectral data. The 2-level DWT approximation had apparently worse discriminative ability, since it lost many details in the raw data. The wavelet feature W generated using all detail outputs and the final approximation output showed the best discriminative performance, because it extracted details in different levels and kept the approximate shape of raw data as well.

Among the seven wavelet families, we found that the Coiflets and the Daubechies worked better than others. The accuracy of our SVM classification was higher when using wavelet features generated with larger-tap mother wavelet. Among the 16 wavelets we have tested, the Coiflet 4, 5 and the high-order Daubechies were more efficient than other wavelets. With the proposed wavelet feature, we were able to distinguish between tumor and normal tissue with an average accuracy of 87.5%, an average specificity of 90%, and an average sensitivity of 83.1%. Furthermore, our method has shown better discriminative ability for overlapped spectra.

Figure 3.

Illustration of wavelet transformation outputs of tumor and normal tissue. (a) Typical raw spectra. (b) Output of 1-level Haar wavelet low-pass filter. (c) Output of 2-level Haar wavelet low-pass filter. (d) Wavelet feature W using Haar wavelet as mother wavelet. (e) Wavelet feature W using Daubechies 4 as mother wavelet. (f) Wavelet feature W using Daubechies 32 as mother wavelet.