Segmentation and Classification of Breast Tumor Using Dynamic Contrast-Enhanced MR Images

Abstract

Accuracy of automatic cancer diagnosis is largely determined by two factors, namely, the precision of tumor segmentation, and the suitability of extracted features for discrimination between malignancy and benignancy. In this paper, we propose a new framework for accurate characterization of tumors in contrast enhanced MR images. First, a new graph cut based segmentation algorithm is developed for refining coarse manual segmentation, which allows precise identification of tumor regions. Second, by considering serial contrast-enhanced images as a single spatio-temporal image, a spatio-temporal model of segmented tumor is constructed to extract Spatio-Temporal Enhancement Patterns (STEPs). STEPs are designed to capture not only dynamic enhancement and architectural features, but also spatial variations of pixel-wise temporal enhancement of the tumor. While temporal enhancement features are extracted through Fourier transform, the resulting STEP framework captures spatial patterns of temporal enhancement features via moment invariants and rotation invariant Gabor textures. High accuracy of the proposed framework is a direct consequence of this two pronged approach, which is validated through experiments yielding, for instance, an area of 0.97 under the ROC curve.

1 Introduction

Dynamic contrast-enhanced MR imaging (DCE-MRI) is emerging as an important complementary diagnostic tool for early detection of breast cancer [1]. It involves characterizing temporal response of a tumor to a contrast agent prior to analyzing discriminating features between various tumor types. High sensitivity of DCE-MRI to breast cancer detection is, however, confounded by its relatively low specificity [2]. Existing approaches attempt to improve low specificity through better segmentation [3] and/or complete characterization of tumor (using architectural and dynamic features [4,1]).

Expert manual segmentation, regarded as gold standard for tumor segmentation, is usually tedious and time consuming. In addition, it suffers from inaccuracy due to imprecision, which results in high inter- and intra-rater variability. Numerous segmentation methods [3,5,6] have recently been proposed to address these limitations. These algorithms are driven by a simple assumption that considers enhancements within tumor to be uniform, limiting only one class per tumor. Here, we propose a graph cut [7] based segmentation algorithm that accounts for spatial variations of enhancements, and allows association of multiple classes to the tumor and background for a more accurate tumor extraction.

Tumor classification employs segmented tumor to extract appropriate features, which are eventually used for cancer diagnosis. Two such features, namely dynamic and architectural features [4,1,2,3], have extensively been investigated for quantifying tumor properties. For instance, an early strong enhancement with rapid washout has been observed in malignant tumors. On the other hand, a slow increase in the temporal enhancement followed by a tapering off is typically exhibited by benign tumors. Majority of the existing dynamic features exploit these simple enhancement differences within a region of interest taken inside a tumor. Architectural features, on the other hand, are driven by morphological differences, with a spiculated border and irregular shape attributed to malignancy versus a smooth border and regular shape related to benignancy. The information in such dynamic and architectural features is, however, limited in the scope in the sense that their combination lacks comprehensive spatial variations of pixel-wise temporal enhancements (TE). These spatial patterns have been shown to be fundamentally important for distinguishing malignant and benign tumors [4,1]. Although the spatial information incorporated in [4,1] offers some improvement, their dependence on qualitative ratings by experts limits their utility for automatic diagnosis.

In this paper, we propose a framework for cancer diagnosis that combines temporal, spatial, and morphological attributes of an automatically segmented tumor in a Spatio-Temporal Enhancement Pattern (STEP). STEP views a serial contrast-enhanced images as a single spatio-temporal image and consequently models its temporal variations through Fourier transform coefficients. Spatial and morphological features are then accounted for by moment invariants [8] and Gabor texture features [9]. Our segmentation refinement algorithm coupled with STEP features provides a robust framework for cancer diagnosis, which is validated with tumor classification using a linear classifier. Instead of using the entire set of STEP features in the classifier, we employ a simple ranking-based feature selection method that helps in finding the most discriminating features between malignancy and benignancy.

2 Methods

This section describes our breast cancer diagnosis framework, which consists of tumor segmentation, STEP feature extraction, and tumor classification.

2.1 Segmentation

Discrimination between benign and malignant breast tumors may be greatly enhanced by accurate segmentation [3] that precisely identifies spatial domain of a tumor. To this end, we develop a graph-cut based algorithm [7] for tumor segmentation, which improves on a coarse manual segmentation, thereby eliminating the need of an expert rater. It involves assigning same labels to pixels with similar TE vectors by minimizing an energy functional.

TE vector C_i = [C (i, 1) ···C (i, T − 1)] of a pixel i is defined as:

$$C(i,t)=\frac{I(i,t)-I(i,0)}{I(i,0)},t=1,\cdots ,T-1$$ (1)

where I(i, t) denotes the intensity of ith pixel at scanning time t.

The energy function is defined as below, which consists of four terms:

$$E=\sum _{i\in \mathrm{\Omega }}{E}_1(l_i)+{\lambda }_1\sum _{<i,j>\in \mathcal{N}}{E}_2(l_i,l_j)+{\lambda }_2\sum _{<i,j>\in {\mathcal{N}}_d}{E}_3(l_i,l_j)+{\lambda }_3\sum _{<i,j>\in {\mathcal{N}}_{tb}}{E}_4(l_i,l_j)$$ (2)

where l_i is the label of pixel i. Factors λ₁, λ₂, λ₃ are used to adjust the relative importance of the four terms, and are empirically set to 1 in this paper.

E₁ ensures the statistical similarity of pixel-wise TEs within each class, and is defined as E₁(l_i) = (1 − Pr (C_i|μ_li, σ_li)) for pixels in image Ω, where Pr measures probability of C_i belonging to class l_i, and the class l_i is represented by a Gaussian model with mean μ_li and variance σ_li.

E₂ penalizes different label assignment to neighboring pixels, which in fact introduces a Markovian property. It is defined for neighboring pixel-pairs as E₂(l_i, l_j) = (1 − δ(l_i − l_j)), where δ is a Kronecker delta function.

E₃ introduces fidelity in segmentation by forcing boundary to regions of large enhancement gradients. It is defined for all neighboring pixel-pairs having different labels as E₃(l_i, l_j) = g₁ (||C_i − C_j||), where || · || means L2-Norm and $g_1(\zeta )=\frac{1}{\zeta +1}$.

E₄ attempts to find the tumor boundary in the vicinity of manually placed contour, and is defined for the pixel pairs which are neighboring and belong differently to tumor and background as E₄(l_i, l_j) = g₂ (β · D_i,j), where D_i,j is the distance from the center point between pixel i and pixel j to the manually delineated boundary, and β is a control parameter for D_i,j, and $g_2(\zeta )=\frac{1}{\zeta +1}$.

In order to initialize the graph-cut based segmentation algorithm, tumor is first specified and roughly segmented by a manual rater. Then, a rectangle region, larger than the bounding box of segmented tumor, is specified as a domain for segmentation. After that, tumor and background are both classified into 3 classes using k-means clustering. The energy functional described above is then minimized by the expansion move algorithm [7] with the output of the k-means algorithm as initialization. After convergence, the union of all tumor classes found by the algorithm is taken as the refined tumor region.

Fig. 1 compares segmentation refinement with expert manual segmentations (ground-truth). It can be observed that the refined segmentations yield a result very close to expert manual segmentations.

Fig. 1.

Rough manual segmentations by a manual rater, corresponding refined results, and manual segmentations by an expert

2.2 Extraction of STEP Features

To extract STEP features, segmented tumor samples (from Section 2.1) are first spatially normalized and then temporally modeled. Finally, both spatial and temporal properties of TE are combined to construct the STEP features.

Tumor Normalization

n order to extract spatial and temporal properties, tumors are first rigidly registered using an approach similar to Procrustes analysis, which aligns principal directions corresponding to the distribution of pixels of each tumor sample and scales tumor sizes for equal largest principal modes. As a result, all tumor samples are normalized to have similar predefined principal directions, in addition to the same “largest eigenvalues” as shown in Fig. 2.

Fig. 2.

Tumor samples before and after normalization, and corresponding TE map of the 1st DFT coefficient. For each panel, top row is malignant, while bottom row is benign.

Temporal Enhancement (TE) Modeling

While several signal decompositions, such as Fourier, wavelet transforms, and wavelet packets, may be exploited to model temporal response of a breast tissue, we adopt Fourier transform to model pixel-wise temporal enhancement, due to its simplicity. Discrete Fourier Transform (DFT) of TE (given by Eq. (1)) yields T − 1 coefficients for each pixel. Combining all pixels in the tumor results in N_t = T − 1 DFT coefficient maps which provide a frequency domain representation of the spatio-temporal enhancement image. An example of a DFT based TE map is given in Fig. 2.

Spatial Description of TE Maps

DFT based TE maps given in Fig. 2 indicate that both global morphological attributes of a tumor and local variations in the TE map can form distinguishing characteristics of malignant and benign tumors. Accordingly, we employ two types of rotation-invariant features, namely, moment invariants [8] for global properties, and Gabor textures [9] for local attributes. Consequently, we compute Hu’s H_m = 7 moment invariants for each of N_t TE maps in addition to H_g = Z × K rotation-invariant Gabor texture features for K orientations within a period of π and Z radial frequencies. We set K = 4 and Z = 8 in our experiments.

The resulting $N_t^{\prime }\times (H_m+H_g)$ STEP features for $N_t^{\prime }$ TE maps out of a total of N_t TE maps provide a rich characterization of a tumor, thereby incorporating spatial, temporal, and morphological attributes.

2.3 Tumor Classification

We validate tumor segmentation and the resulting STEP features by classifying tumors into benign and malignant. It should be noted that STEP features are derived for complete characterization of a tumor, some of which may not be discriminative between the two classes. It is, therefore, important to select a smaller set of most distinctive STEP features before tumor classification. In this paper, we employ Fisher Linear Discriminant based Linear Discriminant Analysis(LDA) [10] classifier, along with a simple t-score ranking-based feature selection method. Although one may obtain better performance by using advanced features selection [11] and nonlinear classification [12], the results provided serve as the baseline performance. Due to limited sample size, leave-one-out cross-validation framework is employed.

3 Experimental Results

DCE-MRI data of bilateral fat suppressed T2 weighted image for 31 subjects were acquired from UPenn under PO1CA085424 (Clinical evaluation of multimodality breast imaging). Sequential post contrast acquisitions were acquired for approximately 6 minutes following contrast injection. There are 22 malignant and 9 benign tumors, which were histologically verified. All the subjected were aligned before performing our algorithms.

Two experiments were performed to evaluate the proposed segmentation algorithm and the STEP features. Classification performance was consequently compared through receiver operating characteristic (ROC) curves (fitted with [13]), sensitivity, specificity, and accuracy for various tumor features.

• Evaluation of Segmentation Algorithm

We compare the performances of tumor classification on roughly-segmented tumor samples by manual rater, segmentation-refined tumor samples, and expert-segmented tumor samples.

As indicated by ROC curves in Fig. 3, the segmentation refinement algorithm improves tumor classification, which is consistent with the results reported in [3]. Highly coincident ROC curves for refined segmentation based- and expert segmentation based-classification given in Fig. 3 indicate that our segmentation algorithm yields a performance comparable to that for expert segmentations.

Fig. 3.

ROC curves of tumor classification on rough segmentation, refinement segmentation, and expert segmentation, along with their AUC values. Notice that ROC curves of ‘refinement’ and ‘expert’ are overlapped together, as identical curves.

Besides, based on all our testing samples, the mean and the standard deviation of boundary distances between our segmentation and expert segmentation are 4.10 and 5.62 pixels, which are much less than the mean and the standard deviation of boundary distances obtained between rough segmentation and expert segmentation, i.e., 7.67 and 8.24 pixels.

• Validation of STEP Features

Performance of STEP features was compared with various existing features. Seven dynamic features (D = {D₁, ···, D₇}), six architectural features (A = {A₁, ···, A₆}), and nine features of spatial variations of TE (V = {V₁, ···, V₉}) were selected (See Appendix for details). Classification performance of individual features {A, D, V }, their combinations {A ∪ D, A ∪ D ∪ V}, and the proposed STEP features was compared using feature selection and leave-one-out classification procedure explained in Section 2.3.

ROC curves and their AUC values for various features are given in Fig. 4. Best classification accuracy, corresponding sensitivity and specificity, along with features selected by our feature selection method, are listed in Table 1. Four selected STEP features (shown in the last row of Table 1) include 3 moment invariants and 1 local Gabor texture feature.

Fig. 4.

ROC curves of tumor classification using different sets of features. The corresponding AUC values are also provided. Notice that ROC curves of A and A ∪ D are overlapped together, as identical curves.

Table 1.

Best classification accuracy, along with corresponding sensitivity and specificity, for different sets of features used in tumor classification

Feature	Accuracy(%)	Sensitivity	Specificity	Selected Features
A	87.1	0.91	0.78	A3,A5,A6
D	64.5	0.68	0.56	D6
V	90.3	0.91	0.89	V3,V5
A ∪ D	87.1	0.91	0.78	A3,A5,A6
A ∪ D ∪ V	90.3	0.91	0.89	A3,A5,V3,V5
STEP	96.8	0.95	1.00	3(moment)+1(Gabor)

Combining general architectural and dynamic features (A ∪ D) for tumor diagnosis, did not improve the AUC value or the best classification rate as compared to the architectural features. This may be due to the simple ranking-based feature selection, which may fail to optimally combine architectural and dynamic features. On the other hand, spatial variation of contrast enhancement was proved very effective in distinguishing malignant and benign tumors. Although the combination of this feature with architectural and dynamic features (A ∪ D ∪ V) fell short of providing an outright improvement in classification rate (compared to V) as shown in Table 1, it improved AUC from 0.89 to 0.93 as shown in Fig. 4. This clearly demonstrates that TE, architectural structure, and spatial variation of TE all play an important role in distinguishing malignant and benign tumors.

In all experiments, STEP features offered the best performance, with AUC, classification rate, sensitivity, and specificity all showing improvements. The fact that both moment invariants and local Gabor texture features were selected as STEP features for tumor classification validates our hypothesis that capturing both global and local variations of contrast enhancement is important in distinguishing between malignant and benign breast tumors. In particular, the 6th moment invariant of the 2nd DFT coefficient map, the 1st moment invariant of the 1st DFT coefficient map, the 2nd moment invariant of the 4th DFT coefficient map, and one texture feature were selected, respectively.

4 Conclusion

In this paper, we have proposed a framework for extracting Spatio-Temporal Enhancement Pattern (STEP) that completely characterizes three tumor properties, namely, temporal enhancement, architectural structure, and spatial variations of pixel-wise temporal enhancement. STEP features were validated through tumor classification, where experimental results show better tumor classification performance with STEP features, than that with individual and pairwise features. We have also shown that STEP features are benefited greatly by the proposed segmentation refinement algorithm as indicated by the tumor classification results, which are consistent with those for expert segmentations. Future work involves extensive evaluation of our methods with a larger database.

5 Appendix

For the dynamic features, we selected the standard deviation of enhancement (D₁), maximum washout (D₂) [14], the maximum uptake (D₃), uptake rate (D₄), washout rate (D₅) [15], and the two features (D₆) and (D₇) extracted from the enhancement curve modelled by the Hayton-Brady pharmacodynamic model in [16]. Notice that the dynamic features were all computed from the average intensities over the tumor area at every time point.

For the architectural features, we selected the compactness (A₁) [14], circularity (A₂) [15], irregularity (A₃), eccentricity (A₄), rectangularity (A₅), and entropy of radial length distribution (A₆) [3].

The features that can account for spatial variations of TE are the variance of uptake (V₁), change in variance of uptake (V₂), margin gradient (V₃), variance of margin gradient (V₄), variance of radial gradient histogram (V₅) [2], the maximum variation of enhancement (V₆), the enhancement-variance increasing rate (V₇), the enhancement-variance decreasing rate (V₈), and the enhancement-variance (V₉) at the first post-contrast frame [15].