A modified gradient correlation filter for image segmentation: Application to airway and bowel

Abstract

The segmentation of structures of interest from medical images may incorrectly include adjacent structures in the segmented image (i.e., false positives). This study introduces a family of gradient correlation filters that reduce false positives in the segmented image by comparing the segmented region gradients with a user-defined model. A gradient correlation filter was applied to a database of clinical computed tomography scans for the task of differentiating airway from lung regions and bowel from lung regions. The results were evaluated using receiver-operating characteristic analysis and demonstrated excellent results for both the airway∕lung and bowel∕lung classification tasks.

INTRODUCTION

The first step of many automated and semiautomated computer-aided diagnosis systems is the segmentation of structures of interest from the patient data set. Often, segmentation is also the most important step since all other aspects of the system (e.g., disease detection and classification of disease severity) assume that the structure of interest is correctly segmented. The application of any segmentation technique (e.g., thresholding, seeded region growing, and active contours) to medical images results in the segmentation of regions other than the structure of interest (i.e., false positives). For example, both seeded region growing and active contour segmentation of the trachea may “leak” into the lungs and incorrectly include lung parenchyma in the trachea segmentation. Segmentation results can be improved through a stage of filters placed after the initial segmentation by characterizing segmented structures with a set of features and eliminating as false positives those structures with feature values beyond the accepted feature bounds. Similarly, if the initial segmentation technique produces a false positive, a feedback loop can be used to alter parameters of the initial segmentation technique to produce an acceptable segmentation of the structure of interest and eliminate false positives. For example, if seeded region growing incorrectly includes lung when segmenting the trachea, then the segmentation can be rerun with increased intensity thresholds to prevent leakage.

Segmentations of lung, airway, and bowel structures of interest in thoracic computed tomography (CT) scans are interrelated tasks. CT sections containing both lung and airway (or lung and bowel) can confound segmentation techniques due to the similar gray-level values and spatial proximity of these structures (Fig. 1), which can lead to the incorrect segmentation of lung regions as airway (bowel) or the incorrect inclusion of airway or bowel during lung segmentation. In pseudo-three-dimensional segmentation schemes, where segmentation in an adjacent axial section is dependent on the segmentation in the current section, the error compounds as each false positive results in the segmentation of more false positives.

Figure 1.

(A) An example of a lung region (black arrow) and airway regions (white arrows) with similar shape and HU values. (B) An example of a lung region (black arrows) and bowel region (gray arrow) in the same section. (C) A lung region (black arrow) demonstrating morphology similar to bowel region. Note that while the shape characteristics are significantly different between lung and bowel in (B), the anterior lung region in (C) has a very similar morphology to the bowel region in (B).

Several methods have been introduced to decrease the number of false positives in lung, airway, and bowel segmentation tasks. Three dimensional connected component analysis was implemented in several studies to differentiate lung from bowel.^{1, 2} This method is only effective when the lung parenchyma constitutes a single 3D region. If the morphology and topology of the lung are significantly altered by disease (e.g., diffuse disease, mesothelioma, or effusion) this method cannot be used. Gray-level value and a rule set based on size, circularity, and location were applied to eliminate lung regions incorrectly identified as airway and eliminate bowel regions incorrectly identified as lung.^{3, 4} Region size has also been included with pseudo-three-dimensional seeded region growing to keep false-positive regions from propagating into the lungs during airway segmentation.¹

The purpose of this study is to introduce a family of gradient correlation filters for eliminating false positives from segmented images. Specifically, a modified directional gradient correlation filter is applied to the task of differentiating lung from bowel and airway regions.

MATERIALS AND METHODS

The gradient correlation filter

Let S be a continuous region defined in the plane and S^′ the gradient of the gray-level values defined over S. Let F(S) be a vector-valued function defined over S that models the gradient of the tissue of interest. The gradient correlation (GC) filter is defined as

$$\text{GC}=S^{\prime }\circ F\left(S\right){\mid }_{u=0}={\int }_S{s}^{\prime }\cdot F\left(s\right)ds,$$ (1)

where s^′∊S^′ and u=0 indicates that only the correlation value corresponding to complete overlap of the domains of S^′ and F(S) is calculated. Thus, GC is a scalar similarity measure of the test image gradient (S^′) and the model gradient (F(S)) and has a range (−∞,∞). The infinite range of GC makes the choice of a fixed region classification threshold difficult since the GC value can be dependent on the number of elements in S and demonstrates a strong dependence on the magnitudes of individual vectors at a point in S. The GC value may be constrained to the range [−1,1] by defining the constrained gradient correlation (CGC) filter:

$$\text{CGC}=\frac{1}{n}{\int }_S\frac{s^{\prime }\cdot F\left(s\right)}{\mid s^{\prime }\mid \mid F\left(s\right)\mid }\text{min}\left(\frac{\mid s^{\prime }\mid }{\mid F\left(s\right)\mid },\frac{\mid F\left(s\right)\mid }{\mid s^{\prime }\mid }\right)ds,$$ (2)

where n is the size of S. Note that the first term is the cosine of the angle between the vectors s^′ and F(s) and measures the similarity in the gradient directions with a range [−1,1]. The second term is a weighting function that measures the similarity between the magnitudes of s^′ and F(s) and has a range [0,1]. Thus, CGC will be a similarity measure similar to GC but constrained to the range [−1,1]. The second term makes CGC insensitive to individual vector magnitudes, instead making it dependent on their ratio, and the 1∕n normalization makes CGC insensitive to region size.

If the gradient magnitude is unable to be accurately modeled a priori but gradient direction can be modeled, then a gradient direction correlation (GDC) filter can be defined by setting the second term in Eq. 2 to the value of 1:

$$\text{GDC}=\frac{1}{n}{\int }_S\frac{s^{\prime }\cdot F\left(s\right)}{\mid s^{\prime }\mid \mid F\left(s\right)\mid }ds,$$ (3)

which creates a similarity measure based only on the direction of the gradients and is constrained to a range [−1,1].

The modified directional gradient correlation filter for air-filled cavities

Let S be an air-filled structure in a thoracic CT scan. In the ideal case, the segmentation only contains cavity pixels, the gradient is zero inside the cavity, and the gradient is oriented normal to the cavity boundary for each boundary point. If F(S) is defined by this description, then in the case of an ideal image and ideal segmentation, the GDC could be applied to S and would yield a value of 1 for an air–filled cavity and <1 for any other structure (due to the presence of nonzero interior gradients). Unfortunately, such a model cannot be applied to clinical images. The point spread function and partial volume effects create a blur in the cavity boundary causing it to be defined over several pixels with varying gray-level values. Further, air does not have a single value but is better defined by a range, roughly −1000 to −800 HU (Hounsfield unit). This range is due to partial volume effect, noise in acquisition and reconstruction, and the fact that CT scanners are calibrated to water and not air. Correct definition of both S and F(S) and a nonlinear gray-level transformation on S can overcome these difficulties.

Transforming the gray-level values of S such that any value in the range listed above is set to −800 HU eliminates the effect of multiple HUs representing air. Next, let S_dist be the distance map of S; that is, for each point s in S_dist, the value of s is set to the Euclidean distance from s to the nearest boundary point of S, and let $S_{\mathrm{dist}}^{\prime }$ be the gradient of S_dist. Setting $F\left(S\right)=S_{\mathrm{dist}}^{\prime }$ causes the GDC filter to be insensitive to edge blurring because the blurred edges of the region have gradients similar to the gradients of the region’s distance transform (Fig. 2). This model asserts that the gradient for any pixel in the cavity should be directed normal to its closest boundary point. Finally, the current model would produce low GDC values since F(S) is nonzero everywhere, but the nonlinear gray-level value threshold caused most interior values to be −800 HU (and thus have a zero gradient magnitude). This final problem is overcome by defining the domain of integration to be constrained to those points in S with a gradient magnitude value greater than some cutoff G_cutoff. Note that if G_cutoff is set to some value greater than zero, then this modification also can eliminate possible degradation of the GDC by small image gradients caused by noise. G_cutoff was set to 0.001 for this experiment and thus only eliminated zero gradient points from the domain of integration.

Figure 2.

(A) An airway region (dark gray outline). (B) Gradient magnitude image of the trachea region. Note that blurring causes a substantial gradient to exist in a band that is several pixels thick (white line). (C) Airway region with distance transform gradient direction, $S_{\mathrm{dist}}^{\prime }$, superimposed (white arrows). (D) Airway region with image gradient direction superimposed (white arrows). Note the strong similarity in image and distance transform gradient direction along the edge of the region.

Database and experiments

A database of patient scans was collected from The University of Chicago Medical Center. Each of the 20 patients (age: 71±9 years; 16 men and 4 women) underwent a helical thoracic CT scan on either a Brilliance 16 or a Brilliance 64 CT scanner (Philips Medical Systems, Cleveland, OH) at our institution. Each CT section was reconstructed as a 512×512 pixel matrix. The scans were acquired at 414±67 kVp with a section thickness of 1 mm and pixel sizes from 0.64 to 0.86 mm (mean: 0.73±0.07 mm).

For each scan, a section containing trachea and lung, a section containing bronchi and lung, and a section containing higher level bronchi and lung (Fig. 3) were arbitrarily selected for testing and resulted in n=60 sections containing both lung and airway. Each CT section was thresholded at values of −500 and −700 HU and region boundaries were simplified using the Douglas–Peuker algorithm⁵ with a tolerance of 1 pixel. For each set of regions produced by thresholding, all airway regions greater than 10 pixels were retained resulting in n=140 and n=159 airway regions, respectively. For each airway region, a lung region with an identical boundary was then extracted from an arbitrary position in the lung on the same CT section. The modified directional gradient correlation (MDGC) filter was applied to all segmented regions and receiver-operating characteristic (ROC) analysis⁶ (ROCKIT software⁷) was applied to the data set to test the ability of the MDGC to differentiate between lung and airway. An optimal cutoff value for airway∕lung classification was determined from the graphs of true-positive fraction and false-positive fractions. As a means of comparison, the maximum HU value was calculated for all segmented regions and ROC analysis was applied to the data set to test the ability of the maximum value to differentiate between lung and airway.

Figure 3.

CT sections demonstrating (A) bronchi (white arrows) inside the lung parenchyma (black arrow) and (B) bowel (white arrows) adjacent to lung parenchyma (black arrows). Note that images are displayed using a lung window.

For each scan, a section containing lung and bowel (n=18) was arbitrarily selected for testing the modified gradient correlation filter. Each CT section was thresholded with values of −500 and −700 HU and region boundaries were simplified using the Douglas–Peuker algorithm with a tolerance of 1 pixel. For each set of regions produced by each threshold value, the MDGC was applied to all segmented regions greater than 10 pixels, and all lung regions with area outside the size range of bowel regions were eliminated. This resulted in n=45 and n=34 bowel regions and n=46 and n=68 lung regions, respectively. Each region was manually identified as either lung or bowel, and ROC analysis was applied to determine the ability of MDGC to differentiate between bowel and lung and to determine the optimal threshold for false-positive reduction. As a means of comparison, the maximum HU value was calculated for all segmented regions and ROC analysis was applied to the data set to test the ability of the maximum value to differentiate between lung and airway.

RESULTS

Airway region size ranged from 1.7 to 403.3 mm² (mean: 119±103 mm²) for the −500 HU threshold and ranged from 2.7 to 377.4 mm² (mean: 105±97 mm²) for the −700 HU threshold. The average MDGC values for lung regions in sections also containing airway was 0.004±0.17 and 0.011±0.20 for −500 and −700 HU thresholds, respectively. The average MDGC values for airway regions were 0.546±0.209 and 0.524±0.207 for −500 and −700 HU thresholds, respectively. An example of an airway region and lung region with corresponding MDGC values is illustrated in Fig. 4. The area under the ROC curve (A_z) was 0.98±0.01 and 0.97±0.01 for −500 and −700 HU thresholds, respectively (Fig. 5). The true-positive fraction and false-positive fraction were graphed against MDGC values for both thresholds. The graph for the −700 threshold is illustrated in Fig. 6 (the graph for the −500 HU threshold demonstrated a similar trend). Based on these graphs, an optimal MDGC feature threshold of 0.20 was determined for both the −500 and −700 HU thresholds. The A_z value using the maximum HU value was 0.83 at −700 HU (Fig. 5).

Figure 4.

(A) Segmented trachea region (gray outline) with a MGDC value of 0.55. (B) Contour corresponding to airway contour in (A) placed in lung (white outline) with a MGDC value of −0.35. (C) Trachea region (gray outline) demonstrating a low MGDC value of 0.19. (D) Lung region (white outline) demonstrating a high MGDC value of 0.61. Note that the low MGDC value in (C) and high MGDC value in (D) misclassifies both regions if the optimal feature threshold of 0.2 is applied.

Figure 5.

Results of the ROC analysis for images thresholded at −700 HU. (A) ROC curve for airway vs lung regions. (B) ROC curve for bowel vs lung regions. Note that the ROC curves for images thresholded at −500 HU were similar to those demonstrated here.

Figure 6.

True-positive fraction (TPF) and false-positive fraction (FPF) vs modified gradient directional filter value. Top: Application to airway and lung regions thresholded at −700 HU. Bottom: Application to bowel and lung regions thresholded at −700 HU.

Bowel region sizes ranged from 2.7 to 1298.6 mm² (mean: 149.1+248.5 mm²) for the −500 HU threshold and lung region sizes ranged from 4.2 to 4391.0 mm² (mean: 996.3±1285.1 mm²) for the −500 HU threshold. The bowel region sizes ranged from 1.7 to 1215.0 mm² (mean: 166.7±255.9 mm²) for the −700 HU threshold. The lung region sizes in sections also containing bowel ranged from 2.4 to 4115.3 mm² (mean: 584.7±993.8 mm²) for the −700 HU threshold. The average MDGC values for lung regions in sections also containing bowel were 0.282±0.172 and 0.220±0.150 for −500 and −700 HU thresholds, respectively. The average MDGC values for bowel regions were 0.475±0.22 and 0.457±0.21 for −500 and −700 HU thresholds, respectively. An example of an airway region and lung region with corresponding MDGC values is illustrated in Fig. 7. The A_z was 0.75±0.05 and 0.80±0.05 for −500 and −700 HU thresholds (Fig. 5), respectively. The true-positive fraction and false-positive fraction were graphed against MDGC values for both −500 and −700 HU threshold regions. The graph for the −700 HU threshold is illustrated in Fig. 6 (the graph for the −500 HU region threshold demonstrated a similar trend). Unlike the lung∕airway classification task, an optimal MDGC feature threshold was not clear from the graphs. The A_z value using the maximum HU value was 0.50 at −700 HU (Fig. 5).

Figure 7.

(A) A segmented lung region (white outline) with a MGDC value of 0.26. (B) A bowel region (gray outline) located in the same section with a MGDC value of 0.60. (C) A segmented lung region with a MGDC value of 0.69. (D) A segmented bowel region with a low MGDC value of 0.25. Note that the high MGDC value of (C) and low value of (D) would result in one or both regions being misclassified by the MGDC filter depending on the feature threshold applied.

Finally, it should be noted that a slight dependence of the MDGC value on the region area was observed for airway and bowel regions. This trend is illustrated in Fig. 8 for all three tissue types (lung, bowel, and airway).

Figure 8.

Filter value vs region area. Top: lung regions; middle: bowel regions; bottom: airway regions. The linear trendline (black line) indicates a slight dependence of filter value on region area for bowel and airway regions.

DISCUSSION

This study introduces the gradient correlation filter as a feature for the reduction of false positives in image segmentations. The GC filter has an unconstrained range and a strong dependence on both region size and the individual gradient magnitude values at each pixel in the region. GC values are unsuitable as a classifying feature since the choice of a feature threshold is complicated by these strong dependencies and the unconstrained range. A CGC filter was then created to address these problems. The CGC filter was normalized by the region size and the magnitude of the vector at each point in the region was constrained to [−1,1] by inclusion of a minimum function in the integration, thus reducing the impact of any single magnitude value on the final CGS value.

The CGC filter may have applications outside medical imaging; however, it is rare that the gradient magnitudes may be modeled a priori due to the variable nature of human anatomy. Although the magnitudes are difficult to accurately model, there are medical image analysis tasks for which the gradient direction can be modeled a priori with some certainty (e.g., lung nodules or air-filled cavities). In such cases the GDC filter can be applied. Unfortunately, the results of the GDC filter are very sensitive to the modeled gradient and thus the modeled gradients should describe gradients found in actual image regions (including variable shape due to anatomic variations, partial volume blurring, etc.) and not the gradients of an “ideal” image without blurring or noise. The MDGC filter and air-filled cavity model make no assumptions about the shape of the structure of interest (e.g., modeling airway regions as circular or ovoid), instead specifying the direction of the model gradient based on the shape of the initially segmented region.

The MGDC filter was applied to the task of differentiating air-filled cavities (specifically, airway and bowel) from lung regions. The MGDC filter demonstrated excellent classification accuracy for both classification tasks. The maximum HU value for each initially segmented region was also examined as a feature for differentiating the airway and bowel regions from lung regions. The MGDC filter performed substantially better than maximum HU (Fig. 5). Further, the filter was robust with respect to the initial segmentation, producing accurate results at both the −500 and −700 HU segmentation thresholds. Lower A_z values were observed for lung∕bowel differentiation than for the lung∕airway differentiation. The lower classification accuracy is likely due to the similar appearance of lung regions in the base of the lungs and bowel regions. Base regions of the lung demonstrate large positional variations at the boundaries as a function of axial position. These changes produce increased partial volume effect along the lung boundary. Since base lung regions at the axial position of bowel are usually small, a large portion of the segmented lung regions demonstrates image gradients dependent on partial volume. The gradients produced by partial volume blurring mimic the gradients produced by the distance transform and thus result in a more difficult classification task.

The MGDC values demonstrated a slight dependence on size for the air-filled cavities. This dependence represents a higher order effect associated with CT resolution and is independent of the mathematical calculation of MGDC. The impact that any single vector has on the overall MGDC value increases as region size decreases. For very small regions, partial volume blurring makes up a larger portion of the region and thus increases the agreement between the gradient model and the image value. Fortunately, this trend is not seen in most lung regions and thus has a negligible impact on classification accuracy.

This study evaluated the MGDC filter on 2D image regions; however, the family of gradient correlation filters can be applied to images of arbitrary dimensions. The application of MGDC to a 3D region requires the extra step of modeling axial gradient. Although CT section thickness is constantly improving, current clinical practice does not routinely acquire isotropic voxels. As the section thickness increases, the partial volume effects increase and will reduce the accuracy of the MGDC filter. High axial resolution scans (i.e., thin sections) can reduce this problem and facilitate application of the MGDC filter to higher dimensional regions. Finally, though not tested in this study, the GC filter and its variants could be applied to vector-valued images (such as those produced as external energy fields for active contours or tensor diffusion images) to measure the direction and∕or magnitude of flow relative to a user-defined model or identify the spatial positions of vector flows similar to the gradient model.

CONCLUSION

This study derived a gradient correlation filter and its variants for the purpose of medical image processing. The filters were applied to the task of false-positive reduction in the segmentation of air-filled cavities. Task-specific modifications to the filter were introduced and the modified filter was applied to a database of patients for lung∕airway and lung∕bowel classification. Results for both sets of experiments were evaluated with ROC analysis and demonstrated high A_z values. The GC filter and its variants demonstrate flexibility in application and can be used in conjunction with other similarity features in larger segmentation and∕or classification schemes.