Learning Shape and Texture Characteristics of CT Tree-in-Bud Opacities for CAD Systems

Abstract

Although radiologists can employ CAD systems to characterize malignancies, pulmonary fibrosis and other chronic diseases; the design of imaging techniques to quantify infectious diseases continue to lag behind. There exists a need to create more CAD systems capable of detecting and quantifying characteristic patterns often seen in respiratory tract infections such as influenza, bacterial pneumonia, or tuborculosis. One of such patterns is Tree-in-bud (TIB) which presents thickened bronchial structures surrounding by clusters of micro-nodules. Automatic detection of TIB patterns is a challenging task because of their weak boundary, noisy appearance, and small lesion size. In this paper, we present two novel methods for automatically detecting TIB patterns: (1) a fast localization of candidate patterns using information from local scale of the images, and (2) a Möbius invariant feature extraction method based on learned local shape and texture properties. A comparative evaluation of the proposed methods is presented with a dataset of 39 laboratory confirmed viral bronchiolitis human parainfluenza (HPIV) CTs and 21 normal lung CTs. Experimental results demonstrate that the proposed CAD system can achieve high detection rate with an overall accuracy of 90.96%.

1 Introduction

As shown by the recent pandemic of novel swine-origin H1N1 influenza, respiratory tract infections are a leading cause of disability and death. A common image pattern often associated with respiratory tract infections is TIB opacification, represented by thickened bronchial structures locally surrounded by clusters of 2–3 millimeter micro-nodules. Such patterns generally represent disease of the small airways such as infectious-inflammatory bronchiolitis as well as bronchiolar luminal impaction with mucus, pus, cells or fluid causing normally invisible peripheral airways to become visible [1]. Fig. 1 shows TIB patterns in a chest CT.

Fig. 1.

(Left) CT image with a significant amount TIB patterns. (Right) Labelled TIB patterns (blue) in zoomed window on the right lung.

The precise quantification of the lung volume occupied by TIB patterns is a challenging task limited by significant inter-observer variance with inconsistent visual scoring methods. These limitations raise the possibility that radiologists’ assessment of respiratory tract infections could be enhanced through the use of computer assisted detection (CAD) systems. However, there are many technical obstacles to detecting TIB patterns because micro-nodules and abnormal peripheral airway structures have strong shape and appearance similarities to TIB patterns and normal anatomic structures in the lungs.

In this work, we propose a new CAD system to evaluate and quantify respiratory tract infections by automatically detecting TIB patterns. The main contributions of the paper are two-fold: (1) A candidate selection method that locates possible abnormal patterns in the images. This process comes from a learning perspective such that the size, shape, and textural characteristics of TIB patterns are learned a priori. The candidate selection process removes large homogeneous regions from consideration which results in a fast localization of candidate TIB patterns. The local regions enclosing candidate TIB patterns are then used to extract shape and texture features for automatic detection; (2) another novel aspect in this work is to extract Möbius invariant local shape features. Extracted local shape features are combined with statistical texture features to classify lung tissues. To the best of our knowledge, this is the first study that uses automatic detection of TIB patterns for a CAD system in infectious lung diseases. Since there is no published work on automatic detection of TIB patterns in the literature, we compare our proposed CAD system on the basis of different feature sets previously shown to be successful in detecting lung diseases in general.

2 Methodology

The proposed CAD methodology is illustrated in Fig. 2. First, lungs are segmented from CT volumes. Second, we use locally adaptive scale based filtering method to detect candidate TIB patterns. Third, segmented lung is divided into local patches in which we extract invariant shape features and statistical texture features followed by support vector machine (SVM) classification. We extract features from local patches of the segmented lung only if there are candidate TIB patterns in the patches. The details of the proposed methods are presented below.

Fig. 2.

The flowchart of the proposed CAD system for automatic TIB detection.

I. Segmentation

Segmentation is often the first step in CAD systems. There are many clinically accepted segmentation methods in clinics [2, 3]. In this study, fuzzy connectedness (FC) image segmentation algorithm is used to achieve successful delineation [2]. In FC framework, left and right lungs are “recognized” by automatically assigned seeds, which initiate FC segmentation.

II. Learning characteristics of TIB patterns

From Fig. 1, we can readily observe that TIB patterns have intensity characteristics with high variation towards nearby pixels, and such regions do not usually exceed a few millimetre(mm) in length. In other words, TIB patterns do not constitute sufficiently large homogeneous regions. Non-smooth changes in local gradient values support this observation. As guided by these observations, we conclude that (a) TIB patterns are localized only in the vicinity of small homogeneous regions, and (b) their boundaries have high curvatures due to the nature of its complex shape.

III. Candidate Pattern Selection

Our candidate detection method comes from a learning perspective such that we assign every internal voxel of the lung a membership value reflecting the size (i.e., scale) of the homogeneous region that the voxel belongs to. To do this, we use a locally adaptive scale based filtering method called ball-scale (or b-scale for short) [2]. b-scale is the simplest form of a locally adaptive scale where the scene is partitioned into several scale levels within which every voxel is assigned the size of the local structure it belongs. For instance, voxels within the large homogeneous objects have highest scale values, and the voxels nearby the boundary of objects have small scale values. Because of this fact and the fact in II.(a), we draw the conclusion that TIB patterns constitute only small b-scale values, hence, it is highly reasonable to consider voxels with small b-scale values as candidate TIB patterns. Moreover, it is indeed highly practical to discard voxels with high b-scale values from candidate selection procedure. Fig. 2 (candidate selection) and Fig. 3(b) show selected b-scale regions as candidate TIB patterns. A detailed description of the b-scale algorithm is presented in [2].

Fig. 3.

a. CT lung, b. selected b-scale patterns, c. mean Curvature map (H), d. Gaussian Curvature (K), e. Willmore energy map, f. zoomed (e). g. Multi-phase level set segmentation based on the proposed shape features is shown in three different slices from the same patient’s chest CT scan.

3 Feature Extraction

For a successful CAD system for infectious lung diseases, there is a need to have representative features characterizing shape and texture of TIB patterns efficiently. Since TIB is a complex shape pattern consisting of curvilinear structures with nodular structures nearby (i.e., a budding tree), we propose to use local shape features (derived from geometry of the local structures) combined with grey-level statistics (derived from a given local patch).

It has been long known that curvatures play an important role in the representation and recognition of intrinsic shapes. However, similarity of curvature values may not necessarily be equivalent to intrinsic shape similarities, which causes a degradation in recognition and matching performance. To overcome this difficulty, we propose to use Willmore energy functional [4] and several different affine invariant shape features parametrically related to the Willmore energy functional.

Willmore Energy

The Willmore energy of surfaces plays an important role in digital geometry, elastic membranes, and image processing. It is closely related to Canham-Helfrich model, where surface energy is defined as

$$\mathcal{S}={\int }_{\mathrm{\sum }}\alpha +\beta {(H)}^2-\gamma \mathit{KdA}.$$ (1)

This model is curvature driven, invariant under the the group of Möbius transformations (in particular under rigid motions and scaling of the surface) and shown to be very useful in energy minimization problems. Invariance of the energy under rigid motions leads to conservation of linear and angular momenta, and invariance under scaling plays a role in setting the size of complex parts of the intrinsic shapes (i.e., corners, wrinkles, folds). In other words, the position, grey-level characteristics, size and orientation of the pattern of interest have minimal effect on the extracted features as long as the suitable patch is reserved for the analysis. In order to have simpler and more intuitive representation of the given model, we simply set α = 0 and β = γ = 1, and the equation turns into the Willmore energy functional,

$${\mathcal{S}}_w={\int }_{\mathrm{\sum }}(H^2-K)dA={\int }_{\mathrm{\sum }}{\mid H\mid }^{2}dA-{\int }_{\partial \mathrm{\sum }}\mid K\mid ds,$$ (2)

where H is the mean curvature vector on Σ, K the Gaussian curvature on ∂Σ, and dA, ds the induced area and length metrics on Σ, ∂Σ (representing area and boundary, respectively). Since homogeneity region that a typical TIB pattern appears is small in size, total curvature (or energy) of that region is high and can be used as a discriminative feature.

In addition to Willmore energy features, we have included seven different local shape features in the proposed CAD system. Let κ₁ and κ₂ indicate eigenvalues of the local Hessian matrix for any given local patch, the following shape features are extracted: 1) mean curvature (H), 2) Gaussian curvature (K), 3) shape index (SI), 4) elongation (κ₁/κ₂), 5) shear ((κ₁ − κ₂)²/4), 6) compactness (1/(κ₁κ₂)), and 7) distortion (κ₁ − κ₂). Briefly, the SI is a statistical measure used to define local shape of the localized structure within the image [5].

Elongation indicates the flatness of the shape. Compactness feature measures the similarity between shape of interest and a perfect ellipse. Fig. 3(c) and (d) show mean and Gaussian curvature maps from which all the other local shape features are extracted. Fig. 3(e) and (f) show Willmore energy map extracted from Fig. 3(a).

Based on the observation in training, TIB patterns most likely occur in the regions inside the lung with certain ranges (i.e, blue and yellow regions). This observation facilitates one practically useful fact in the algorithm that, in the feature extraction process, we only extract features if and only if at least “one” b-scale pattern exists in the local region as well as Willmore energy values of pixels lie in the interval observed from training. Moreover, considering the Will-more energy has a role as hard control on feature selection and computation, it is natural to investigate their ability to segment images. We present a segmentation framework in which every voxel is described by the proposed shape features. A multi-phase level set [6] is then applied to the resulting vectorial image and the results are shown in Fig. 3(g). First and second columns of the Fig. 3(g) show segmented structures and the output homogeneity maps showing segmented regions in different grey-level, respectively. Although segmentation of small airway structures and pathological patterns is not the particular aim of this study, the proposed shape features show promising results due to their discriminative power.

Texture features

Spatial statistics based on Grey-Level Co-occurrence Matrix (GLCM) [7] are shown to be useful in discriminating patterns pertaining to lung diseases. As texture can give a lot of insights into the classification and characterization problem of poorly defined lesions, regions, and objects, we combine our proposed shape based invariants with GLCM based features. We extract 18 GLCM features from each local patch including autocorrelation, entropy, variance, homogeneity, and extended features of those. Apart from the proposed method, we also compare our proposed method with well known texture features: steerable wavelets (computed over 1 scale and 6 orientations with derivative of Gaussian kernel), GLCM, combination of shape and steerable wavelets, and considering different local patch size.

4 Experimental Results

39 laboratory confirmed CTs of HPIV infection and 21 normal lung CTs were collected for the experiments. The in-plane resolution is affected from patients’ size and varying from 0.62mm to 0.82mm with slice thickness of 5mm. An expert radiologist carefully examined the complete scan and labeled the regions as normal and abnormal (with TIB patterns). As many regions as possible showing abnormal lung tissue were labeled (see Table 1 for details of the number of regions used in the experiments). After the proposed CAD system is tested via two-fold cross validations with labeled dataset, we present receiver operator characteristic (ROC) curves of the system performances.

Table 1.

Accuracy (A_z) of the CAD system with given feature sets.

Features	Dimension	Patch Size	# of patches (TIB)	# of patches (Normal)	Area under ROC curve: Az
Shape & GLCM	8+18=26	17×17	14144	12032	0.8991
Shape & GLCM	8+18=26	13×13	24184	20572	0.9038
Shape & GLCM	8+18=26	9×9	50456	42924	0.9096
Shape	8	17×17	14144	12032	0.7941
Shape	8	13×13	24184	20572	0.7742
Shape	8	9×9	50456	42924	0.7450
Steer. Wavelets& Shape	6×17×17+8=1742	17×17	14144	12032	0.7846
Steer. Wavelets& Shape	6×13×13+8=1022	13×13	24184	20572	0.7692
Steer. Wavelets& Shape	6×9×9+8=494	9×9	50456	42924	0.7908
Steer. Wavelets	6×17×17=1734	17×17	14144	12032	0.7571
Steer. Wavelets	6×13×13=1014	13×13	24184	20572	0.7298
Steer. Wavelets	6×9×9=486	9×9	50456	42924	0.7410
GLCM	18	17×17	14144	12032	0.7163
GLCM	18	13×13	24184	20572	0.7068
GLCM	18	9×9	50456	42924	0.6810

Table 1 summarizes the performance of the proposed CAD system as compared to different feature sets. The performances are reported as the areas under the ROC curves (A_z). Note that shape features alone are superior to other methods even though the dimension of the shape feature is only 8. The best performance is obtained when we combined shape and GLCM features. This is expected because spatial statistics are incorporated into the shape features such that texture and shape features are often complementary to each other. In what follows, we select the best window size for each feature set and plot their ROC curves all in Fig. 4. To have a valid comparison, we repeat candidate selection step for all the methods, hence, the CAD performances of compared feature sets might perhaps have lower accuracies if the candidate selection part is not applied. Superiority of the proposed features is clear in all cases. To show whether the proposed method is significantly different than the other methods, we compared the performances through paired t-tests, and the p-values of the tests are summarized in Table 2. Note that statistically significant changes are emphasized by p < .01 and p < .05.

Fig. 4.

Comparison of CAD performances via ROC curves of different feature sets.

Table 2.

p-values are shown in confusion matrix.

p-Confusion Matrix	Shape	Steer. & Shape	Steer.	GLCM
Shape& GLCM	0.0171	0.0053	0.0056	0.0191
Shape	–	0.0086	0.0094	0.0185
Steer.& Shape	–	–	0.0096	0.0175
Steer.	–	–	–	0.0195

5 Conclusion

In this paper, we have proposed a novel CAD system for automatic TIB pattern detection from lung CTs. The proposed system integrates 1) fast localization of candidate TIB patterns through b-scale filtering and scale selection, and 2) combined shape and textural features to identify TIB patterns. Our proposed shape features illustrate the usefulness of the invariant features, Willmore energy features in particular, to analyze TIB patterns in Chest CT. In this paper, we have not addressed the issue of quantitative evaluation of severity of diseases by expert observers. This is a challenging task for complex shape patterns such as TIB opacities, and subject to further investigation.