Multiscale Fuzzy C-Means Image Classification for Multiple Weighted MR Images for the Assessment of Photodynamic Therapy in Mice

Abstract

We are investigating in vivo small animal imaging and analysis methods for the assessment of photodynamic therapy (PDT), an emerging therapeutic modality for cancer treatment. Multiple weighted MR images were acquired from tumor-bearing mice pre- and post-PDT and 24-hour after PDT. We developed an automatic image classification method to differentiate live, necrotic and intermediate tissues within the treated tumor on the MR images. We used a multiscale diffusion filter to process the MR images before classification. A multiscale fuzzy C-means (FCM) classification method was applied along the scales. The object function of the standard FCM was modified to allow multiscale classification processing where the result from a coarse scale is used to supervise the classification in the next scale. The multiscale fuzzy C-means (MFCM) method takes noise levels and partial volume effects into the classification processing. The method was validated by simulated MR images with various noise levels. For simulated data, the classification method achieved 96.0 ± 1.1% overlap ratio. For real mouse MR images, the classification results of the treated tumors were validated by histologic images. The overlap ratios were 85.6 ± 5.1%, 82.4 ± 7.8% and 80.5 ± 10.2% for the live, necrotic, and intermediate tissues, respectively. The MR imaging and the MFCM classification methods may provide a useful tool for the assessment of the tumor response to photodynamic therapy in vivo.

1. INTRODUCTION

Photodynamic therapy (PDT) is a novel and promising modality in the treatment of cancer [1, 2]. The therapy uses a tumor-localized drug called a photosensitizer excited by irradiation with a laser light of a particular wavelength, which generates reactive singlet oxygen that efficiently kills cells and ablates tumors. Both the photosensitizer and the light are inert by themselves, therefore, systemic toxicities in PDT are minimized. PDT is minimally invasive as only the small laser fiber is mounted externally to deliver the light to tumors.

Imaging techniques provide a powerful tool for assessment of PDT efficacy. We are investigating in vivo small animal imaging techniques to study the tumor response to PDT using Pc 4, a photosensitizer developed at our institution [3]. High-resolution magnetic resonance imaging (MRI) can show anatomical and morphological changes of lesions [4]. In this study, we used an MR imaging system to acquire multiple weighted MR images from tumor-bearing mice before, after and 24 hours after PDT. The tumor response to the treatment is defined by the degree of tumor necrosis or apoptosis. To quantitatively evaluate PDT, we developed an automatic image classification method to differentiate live, necrotic and intermediate tissues within the treated tumor on the MR images.

MR images are affected by multiple factors such as noise, intensity inhomogeneity and partial volume effects. Partial volume effects occur where pixels contain a mixture of multiple tissue types, which make the assignment of a single class to these boundary regions more difficult. Gaussian mixture model based classification estimate the mixture of each pixel by modeling the image histogram [5, 6]. These methods assume the intensity of single tissue type is a Gaussian distribution. Considering partial volume effects and image smoothing from post processing, the intensity distribution may deviate from a Gaussian model. Fuzzy c-means algorithm (FCM) [7, 8] employs fuzzy partitioning to allow one voxel to belong to tissue types with different membership grades between 0 and 1. Different modified FCM has been proposed to compensate for intensity inhomogeneity and spatial information [8]. However, FCM is sensitive to the initial guess and noise with regard to both speed and stability [8-11]. In this report, we apply an anisotropic diffusion filter to smooth noise while preserving edge boundaries, therefore the result of a k-means classification on the coarsest level is sufficient enough for the initial guess of FCM method. A multiscale fuzzy C-means (FCM) classification method was applied along the scales of the anisotropic diffusion filter. The method gets accurate classification step by step and fast convergence at fine scales. The object function of the standard FCM was modified to allow multiscale classification where the result from a coarse scale is used to supervise the classification in the next scale. In the next section, we describe the modified fuzzy c-means classification. Results on synthetic data and actual MRI are reported in the following sections.

2. METHOD

2.1 Multiscale Space from Anisotropic Diffusion Filtering

Multiscale space represents images by using a series of images at varying spatial resolution in which an image contains less local information as the scale increases. Anisotropic diffusion filter, introduced by Perona and Malik[12], is a partial differential diffusion equation model, the image achieves more smooth while preserving inter-region edges as discrete time step increasing. Our multiscale description of images is generated by anisotropic diffusion filter with the time step as scale.

The anisotropic diffusion equation is described as

$$\frac{\partial I(x,t)}{\partial t}=div(g\left(\Vert \nabla I(x,t)\Vert \right)\nabla I(x,t))$$ (1)

I(x,t) is the intensity of MR volumes at time step or scale t; ∇ and div are spatial gradient and divergence operator. g(x, t) is the diffusion coefficient and chosen as a function of the magnitude of the gradient of intensity images

$$g\left(\Vert \nabla I(x,t)\Vert \right)=e^{-{(\mid \nabla I(x,t)\mid ∕k)}^2}$$ (2)

The constant k is chosen to be the gradient magnitudes produced by noise, and can be fixed manually or estimated using noise estimator described by Canny[13]. By applying an anisotropic diffusion filter to the original MR images, we generated series of images and form the scale spaces. The scale level of original images is 0. When the scale increases, the images become blurred and contain more general information. Fig.1 illustrates the scale space constructed from anisotropic diffusion filtering.

Fig.1.

Illustration of scale-space constructed by anisotropic diffusion filtering. Scale space is composed by the stack of the original image filtered at different time step, t=0 is the original image, and the bigger the scale level, the less local information appears. The brain image is from BrainWeb project and with 5% noise and 20% field inhomogeneities.

2.2 Multiscale Fuzzy C-means (MFCM)

The classification was performed from the coarsest scale to the original images. The classification result at a coarser level t+1 was used to initialize the classification at a higher scale level t. The finial classification is the result at the scale level 0. During the classification processing at the level t+1, the pixels with the highest membership above a threshold are identified and assigned to the corresponding class. These pixels are labeled as training data for the next level t.

The objective function of the FCM at level t is

$$J=\sum _{k=1}^c\sum _{i=1}^N{u}_{ik}^p{\Vert x_i-v_k\Vert }^2+\frac{\alpha }{N_R}\sum _{k=1}^c\sum _{i=1}^N{u}_{ik}^p\left(\sum _{x_r\in N_i}{\Vert x_r-v_k\Vert }^2\right)+\beta \sum _{k=1}^c\sum _{i=1}^N{(u_{ik}-u_{ik}^{\prime })}^p{\Vert x_i-v_k\Vert }^2$$ (3)

Where u_ik stands for the membership of the pixel i belonging to the class k, and v_k is the vector of the class k center, x_i is the feature vectors from multi-weighted MR images, N_i stands for the 8 neighboring pixels of x_i for 2D images, and the parameter p is a weighting exponent and is selected as 2. The objective function is the sum of three terms, where α and β are scaling factors to maintain balance between them. The first term is the standard fuzzy c-means object function that assigns a high membership to the voxel whose intensity is close to the center of the class. If only this term is used, we call this standard FCM method. The second term allows the membership in neighborhood pixels to regulate the classification toward piecewise-homogeneous labeling. If both the first and second terms are used, it was called the modified FCM method. The third term is to incorporate the supervision information from the classification of the previous scale, $u_{ik}^{\prime }$ is the membership obtained from the classification in the previous scale. If all three terms are used, we call this multiscale FCM (MFCM) method. $u_{ik}^{\prime }$ is determined as:

$$u_{ik}^{\prime }=\{\begin{array}{c}\hfill u_{ik}^{t+1},\text{if}\underset{k}{\text{max}}\left(u_{ik}^{t+1}\right)>\kappa \hfill \\ \hfill 0,\text{otherwise}\hfill \end{array}\phantom{\}}$$ (4)

Where κ is the threshold to determine the pixels with known class in the next scale classification, and is set as 0.85 in our implementation. The classification is implemented by minimizing the object function J. The minimization of J happens when first derivative of J with respect to u_ik and v_k are zero.

From

$$\frac{\partial J}{\partial v_k}=\sum _{i=1}^N{u}_{ik}^2(x_i-v_k)+\frac{\alpha }{N_R}\sum _{i=1}^N{u}_{ik}^2\left(\sum _{x_r\in N_i}(x_r-v_k)\right)+\beta \sum _{i=1}^N{(u_{ik}-u_{ik}^{\prime })}^2(x_i-v_k)=0$$ (5)

The class center is updated as:

$$v_k=\frac{{\sum }_{i=1}^N{u}_{ik}^2(x_i+\frac{\alpha }{N_R}{\sum }_{x_r\in N_i}{x}_r)+\beta {\sum }_{i=1}^N{(u_{ik}-u_{ik}^{\prime })}^2{x}_i}{(1+\alpha ){\sum }_{i=1}^N{u}_{ik}^2+\beta {\sum }_{i=1}^N{(u_{ik}-u_{ik}^{\prime })}^2}$$ (6)

${\sum }_{k=1}^c{u}_{ik}=1$ for every pixel, according to Lagrangian approach define

$$Fm=\sum _{k=1}^c\sum _{i=1}^N{u}_{ik}^2{\Vert x_i-v_k\Vert }^2+\frac{\alpha }{N_R}\sum _{k=1}^c\sum _{i=1}^N{u}_{ik}^2\left(\sum _{x_r\in N_i}{\Vert x_r-v_k\Vert }^2\right)+\beta \sum _{k=1}^c\sum _{c=1}^N{(u_{ik}-u_{ik}^{\prime })}^2{\Vert x_i-v_k\Vert }^2+\lambda (1-\sum _{k=1}^c{u}_{ik})$$ (7)

for optimization with respect to u_ik which requires

$$\frac{\partial Fm}{\partial u_{ik}}=u_{ik}{\Vert x_i-v_k\Vert }^2+\frac{\alpha }{N_R}{u}_{ik}\left(\sum _{x_r\in N_i}{\Vert x_r-v_k\Vert }^2\right)+\beta (u_{ik}-u_{ik}^{\prime }){\Vert x_i-v_k\Vert }^2-\lambda =0$$ (8)

The membership of every pixel i belong to class k is updated according to below equation

$$u_{ik}=\frac{1+\beta {\sum }_{m=1}^c\frac{u_{ik}^{\prime }{\Vert x_i-v_k\Vert }^2-u_{im}^{\prime }{\Vert x_i-v_m\Vert }^2}{(1+\beta ){\Vert x_i-v_m\Vert }^2+\frac{\alpha }{N_R}\left({\sum }_{x_r\in N_i}{\Vert x_i-v_m\Vert }^2\right)}}{{\sum }_{m=1}^c\left(\frac{(1+\beta ){\Vert x_i-v_k\Vert }^2+\frac{\alpha }{N_R}\left({\sum }_{x_r\in N_i}{\Vert x_r-v_k\Vert }^2\right)}{(1+\beta ){\Vert x_i-v_m\Vert }^2+\frac{\alpha }{N_R}\left({\sum }_{x_r\in N_i}{\Vert x_r-v_m\Vert }^2\right)}\right)}$$ (9)

2.3 MFCM Algorithm

MFCM is an iterative algorithm that requires initial estimation of the class prototypes. Generally proper selection of the initial classification will improve clustering accuracy and reduces the number of iterations. The k-means method is used on the coarsest image to estimate the initial class prototypes because noise and inhomogeneities have been effectively attenuated by anisotropic filtering at the coarsest image.

The proposed MFCM algorithm for classifying MR images can be summarized in the following steps:

1. Anisotropic diffusion filtering the images to the scale level t = n

2. Obtain the initial class prototypes using k-means clustering method in the coarsest level image. Set $\left\{u_{ik}^{\prime }\right\}=0$

3. Run clustering in the filtered image at level t =n-1

   1. Update the membership using (9)
   2. Every class centroids {V_k} are computed using (6)
   3. Repeat a, b until the convergence which is defined as

      $$\Vert v_{new}-v_{old}\Vert <\epsilon$$ (10)

      The iteration is terminated when Euclidean distance of class centers between iterations is less than a small number ε (ε = 0.01).

4. Threshold membership functions according to (4) and get matrix $\left\{u_{ik}^{\prime }\right\}$.

5. Repeat step 3-4 on the next scale image until completing the classification at original image (t=0)

2.4 PDT Experiments and Image Acquisition

RIF(Radiation-induced fibrosarcoma)-1 tumor cells were grown as monolayers in E-MEM supplemented with 15% fetal bovine serum. Two RIF-1 tumors were initiated in C3H/HeN mice by injection of RIF-1 cells subcutaneously on the back. Tumors were treated and imaged when they reach 6-10 mm in diameter. The photosensitizer Pc 4 was injected to tumor-bearing mice via tail vein by 0.6 mg/kg of body weight. After 48 hours, one of the tumors was exposed to red laser light (672 nm) from a diode laser with a dose of 150 J/cm² and a fluence rate of 100 mW/cm². For monitoring the whole process of the PDT treatment, MR images of the mice were acquired before, immediately after and 24 hours after the therapy. During each imaging session, the mice were mounted on a plastic holder and were provided with a continuous supply of 2% isoflurane in oxygen to minimize motion artifacts in MR images.

T1-,T2- and FLASH weighted MR images were acquired for 3 tumor-bearing mice. The mouse MR images were acquired using a Siemens Sonata 1.5 T scanner (Siemens Medical Systems, Erlangen, Germany). A custom-designed whole-body mouse coil (2-element phased-array, ID = 32 mm) was used to minimize noise levels. The acquired coronal scan has 256X120 matrix, 80X36 mm FOV and 1mm slice thickness. The number of signal averages was typically set at six to obtain images with low noise.

3. RESULTS

To evaluate the classification method, we simulated a 3D tumor model that consists of four classes. Every class is given a gray level between 0 and 255 and added with 10% Gaussian noises. The images were filtered by 3 pixel gaussian filter for partial volume effects. The visual assessment of classification by the algorithm is illustrated in Figure 2. In Table 1, a more detailed tissue-dependent quantitative analysis is performed by computing the sensitivity and specificity that is evaluated by the ground truth. For the simulated images, the algorithm can correctly classify 96.0 ± 1.1% tissues indicating the method's excellent performance.

Fig.2.

Visual assessment of classification results for a 3D simulated image volume. Three image slices (Slice 6, 21, and 52) are shown to cover the whole tumor volume. (a) and (b) are simulated MR image volumes with different weightings. (c) is the classification result, (d) is the ground truth. In (c) and (d), four gray levels show different classes, bright is the class 4.

Table. 1.

Confusion table on simulated data. Values are the percentages computed over all voxels of each class (C1-C4) in the reference. False positive (FP) and false negative (FN) rates are computed in percentages using the reference.

		Reference (Ground Truth)				FP
	Class	C1	C2	C3	C4
Classified	C1	97.5	4.0	0.0	2.6	6.2
	C2	2.0	96.0	0.0	0.0	8.7
	C3	0.0	0.0	95.6	2.5	5.8
	C4	0.5	0.0	4.4	94.9	3.6
FN		2.5	4.0	4.4	5.1

We also applied our method to the digital brain phantom data generated by the BrainWeb MR simulator[14]. We applied the classification method to T1 and T2 weighted MR images with different noise levels and 20% intensity non-uniformity. Prior to the classification, the extracranial tissue such as skull, meninges and blood vessels have been removed, so the brain MRI consists of 3 types of tissue: gray matter, white matter and CSF. The classification is evaluated by the overlap ratio between the classification result and the realistic model for every class, which is defined as twice the number of corrected classified pixels divided by the total number of pixels in the ground truth and classified results for each class. Figure 3 illustrates the classification results and corresponding ground of truth. Figure 4 demonstrates the overlap ratio for each class between classified results and ground of truth, which decrease by less than 6.0% with added noises. We compared our MFCM method with the standard FCM method and the modified FCM method. This was implemented by setting the constant α= β =0 for the standard FCM method, α = 0.85 and β=0 for the modified FCM method, and α=0.85 and β =0.80 for our multiscale FCM method, respectively. Figure 5 shows the overlap ratio change with respect to different methods applied on T1 and T2 MRI with 9% noise and 20% intensity non-uniformity.

Fig.3.

a) is a slice from a T2 weighted volume with 9% noise level from the digital phantom data. b) is the image at the scale 6 after anisotropic diffusion filtering. c) is the classification result using our method. d) is the ground truth.

Fig.4.

Classification evaluation of the digital phantom data. Overlap ratios of classification results with the ground truth are plotted for each class at different noise levels.

Fig.5.

Comparison of classification of the standard FCM, modified FCM and our MFCM method on BrainWeb MRI data where 9% noise and 20% inhomogeneity T1 and T2 volume are used for classification.

For real MR images from treated mice, the tumors were first manually segmented on each slice and then classified into three classes (live, necrotic, and intermediate tissues). The classification results were evaluated by histology images which have been labeled into 3 classes by a pathologist. Figure 6 illustrates the T1, T2, FLASH weighted MR images and the classification results of a real mouse tumor MR images 24-hour after PDT. Fig 6.d is the classification result showing 3 classes which are corresponding to the regions labeled in the histology (Fig6.e). We run the classification method on the MR images of 3 mice. The overlap ratios were 85.6 ± 5.1%, 82.4 ± 7.8%, and 80.5 ± 10.2% for the live, necrotic, and intermediate tissues, respectively.

Fig.6.

Classification results of real mouse tumor MR images 24-hour after PDT. (a), (b), and (c) are the original T1, T2, FLASH tumor MR images, respectively. d) is the classification result with showing 3 classes. e) is the corresponding histology, the necrotic (right arrow) and intermediate (left arrow) regions are labeled on the images. f) shows the overlap of the histologic labeling and the classified result.

4. DISCUSSIONS AND CONCLUSIONS

We developed a multiscale fuzzy c-means classification method for a new application in photodynamic therapy. We use an anisotropic filter to effectively attenuate the noise within regions but preserve edges between different tissue types. A scale space is generated by anisotropic filtering, and the general structure information is kept in the images at a coarser scale. Therefore, a k-means method on the coarsest images is used for the initial guess. The classification is advanced along the scale space to include local information in fine-level images and to compensate the partial volume effects due to smoothing. The result from a coarser scale provides the initial parameter for the classification in the next scale. Meanwhile, the pixels with a high probability of belonging to one class in the coarse scale will belong to the same class in the next level. Therefore, these pixels in the coarser images are considered as points with a known class and are used as training data to constrain the classification in the next scale. In this way, we get accurate classification step by step and avoid being trapped into local minima. Furthermore, we include a term that constrains a pixel that can be influenced by its immediate neighborhoods so as to achieve a piecewise-homogeneous solution. The method is accurate and robust for both simulated and real MR images.

Field inhomogeneity is smooth compared to MR images and we classify segmented tumors with a small volume on the whole MR images, therefore, field inhomogeneity is neglected in our classification. However, heavy field inhomogeneity can not be attenuated by anisotropic filtering and will corrupt the result severely. Similar to [15], we can incorporate a gain field term into the objective function of FCM methods. By this way, the tissue classification and the bias field can be estimated simultaneously.

Small animal imaging provides a new technique to study the therapeutic effects of cancer treatment. The proposed imaging and classification method could provide a useful tool to differentiate necrosis from viable tumor cells on MR images. It could be used for early assessment of therapeutic effects in the future.