Interactive iterative relative fuzzy connectedness lung segmentation on thoracic 4D dynamic MR images

Abstract

Lung delineation via dynamic 4D thoracic magnetic resonance imaging (MRI) is necessary for quantitative image analysis for studying pediatric respiratory diseases such as thoracic insufficiency syndrome (TIS). This task is very challenging because of the often-extreme malformations of the thorax in TIS, lack of signal from bone and connective tissues resulting in inadequate image quality, abnormal thoracic dynamics, and the inability of the patients to cooperate with the protocol needed to get good quality images. We propose an interactive fuzzy connectedness approach as a potential practical solution to this difficult problem. Manual segmentation is too labor intensive especially due to the 4D nature of the data and can lead to low repeatability of the segmentation results. Registration-based approaches are somewhat inefficient and may produce inaccurate results due to accumulated registration errors and inadequate boundary information. The proposed approach works in a manner resembling the Iterative Livewire tool but uses iterative relative fuzzy connectedness (IRFC) as the delineation engine. Seeds needed by IRFC are set manually and are propagated from slice-to-slice, decreasing the needed human labor, and then a fuzzy connectedness map is automatically calculated almost instantaneously. If the segmentation is acceptable, the user selects “next” slice. Otherwise, the seeds are refined and the process continues. Although human interaction is needed, an advantage of the method is the high level of efficient user-control on the process and non-necessity to refine the results. Dynamic MRI sequences from 5 pediatric TIS patients involving 39 3D spatial volumes are used to evaluate the proposed approach. The method is compared to two other IRFC strategies with a higher level of automation. The proposed method yields an overall true positive and false positive volume fraction of 0.91 and 0.03, respectively, and Hausdorff boundary distance of 2 mm.

1. INTRODUCTION

Due to its excellent soft-tissue contrast and lack of radiation exposure, dynamic magnetic resonance imaging (dMRI) of the thorax plays an important role in the study of respiratory dynamics, respiratory diseases, and radiotherapy planning [1], especially in pediatric populations. Lung segmentation is a necessary first step in these applications for quantitative analysis. Our area of application is the study of pediatric Thoracic Insufficiency Syndrome (TIS) – the inability of the thorax to support normal respiration or lung growth [2]. Segmentation of the lungs in dMRI is particularly challenging, especially in pediatric patients with TIS, due to the absence of MRI signal from hard and connective tissues (cortical bone and ligaments) adjacent to the pleural space, motion, and high respiratory rate in patients who are often very sick. Manual segmentation by outlining the lung on every slice in every 3D volume of each time point is extremely labor-intensive and is impractical for conducting studies based on many patient scans. Currently, except for the method of using 4D probabilistic atlases for the segmentation of cardiac structures [3], almost all reported lung segmentation methods on dMRI use a segmentation propagation strategy wherein the image is segmented at one time point of the 4D volume first by using various techniques including manual segmentation, and then propagated via image registration to images at other time points [4, 5]. This approach, besides being somewhat inefficient, may not produce accurate delineations [6] especially when the change in lung space from one time point to the next is large or non-uniform. Usually adjacent volumes are therefore used for registration, but then the propagated and accumulated registration error may become significant for volumes at later time points. The purpose of this paper is to present a practical solution for lung segmentation on 4D dMRI images for the TIS application.

Our solution consists of an adaptation of the iterative relative fuzzy connectedness (IRFC) algorithm in an interactive manner as a potential practical tool. We will refer to the method as interactive IRFC or i-IRFC. The method gives users effective control on the segmentation process, which is needed on a slice-by-slice basis for challenging segmentation tasks like this, and yet keeps the process efficient enough. In its spirit, it resembles the iterative live wire approach [7] in that user control is at the slice level, where the process is designed in such a manner that, in practice, user action is needed only once in a few slices. The method is compared to two other strategies, also based on IRFC. Repeatability of the proposed approach from inter-user and intra-user experiments on TIS dMRI data sets is also evaluated.

2. MATERIALS & METHODS

2.1. Image data

This retrospective study was conducted following approval from the Institutional Review Board at the Children’s Hospital of Philadelphia along with a Health Insurance Portability and Accountability Act waiver. Image data sets utilized in our evaluation all pertain to pediatric thoracic dMRI. In our image acquisition protocol, for each coronal or sagittal slice position, 2D slice images are acquired continuously at a rate of about 200 ms/slice over several natural breathing cycles. The data sets consist of 5 dMRI scans (TrueFISP with TR/TE ~ 4.3/2.2 msec, magnetic field strength of 1.5 T) from 5 subjects, including 2 normal adults (in coronal imaging plane) and 3 TIS patients (in sagittal imaging plane) with voxel size ranging from 2.21×2.21×4.8 mm³ to 1.17×1.17×5.0 mm³ and 3D scene size varying from 192×192×31 to 224×256×34, the number of time points varying from 6 to 10. In total, 39 3D volume images are involved in this study considering all time points. Manual segmentation is done via CAVASS software and used as ground truth for evaluation [8]. All three IRFC algorithms are implemented within the CAVASS software system as well.

2.2. Methods

4D dMRI imaging

In this study, 4D thoracic dMRI images are constructed retrospectively by using a graph-based combinatorial optimization solution [9] to form the best possible 4D scene from 1000s of 2D slice images acquired as described above. The 4D image construction approach is purely image-based and does not need breath holding or any external surrogates or instruments to record respiratory motion or tidal volume [9]. Because of the non-standardness (lack of consistent tissue-specific numeric meaning for the intensity values) of the MR images, all 3D images of every time point are standardized for image intensity [10]. Prior to standardization, intensity non-uniformities arising from magnetic field inhomogeneity are corrected [11].

Iterative relative fuzzy connectedness (IRFC)

IRFC is a top-of-the-line algorithm in the fuzzy connectedness (FC) family which operates with the basic principles of FC but by iteratively reinforcing the segmentation evidence in a conservative manner. The FC framework is graph-based [12]. Let I = (C, f) denote a 3D image where C is a rectangular array of voxels and f is the MR image intensity function defined on C. A graph (C, α) is associated with image I = (C, f) where α is a voxel adjacency relation on C such as 6-, 18-, and 26-adjacency. Each pair (c, d) of adjacent voxels in α is assigned an affinity value κ(c, d) which expresses the strength of the bond between c and d in belonging to the same object. To each path π in the graph (or equivalently in I) in the set of all possible paths Π_a,b between any two voxels a and b of C, a strength of connectedness Κ(π) is determined, which is the minimum of the affinities between successive voxels along the path.

A fuzzy connected object is defined with a threshold on the strength of connectedness for the basic FC method called Absolute FC. Relative fuzzy connectedness overcomes the need for a threshold and leads to more effective segmentations by setting up object and background seeds to compete for membership of voxels with the object and background [14]. The central idea is that an object gets defined in an image because of the presence of other co-objects. IRFC uses an iterative strategy for fuzzy connectedness calculation wherein the strongest relative connected core parts are first defined and iteratively relaxed to conservatively capture the fuzzier parts subsequently. In IRFC, two seed sets A _O and A_B are indicated for an object O and its background B, respectively [14]. A voxel in I will be decided as belonging to the object or background by comparing the connection strength between the voxel and A_O and the voxel and A_B.

IRFC-based approaches for lung segmentation on 4D dMRI images

P4D-IRFC:

This is a pseudo-4D method based on our previous work developed for upper airway segmentation on 4D dMRI images [6], where seeds are propagated automatically along the time dimension. In this approach, seed specification is needed in only the 3D image corresponding to the first time-instance of the 4D volume, and from this information the 3D volume corresponding to the first time-point is segmented. Seeds are then automatically generated for the next time-point from the segmentation of the 3D volume corresponding to the previous time-point, and the process continues without human interaction and completes segmenting the airway structure in the whole 4D volume. We will follow the same procedure for lung segmentation on 4D dynamic MR images under this approach.

S3D-IRFC:

This is a spatial 3D approach where each 3D volume corresponding to a time point is segmented independently. Instead of propagating the seeds from a previous time point to the following time point as in the P4DIRFC algorithm, this approach deals with every 3D volume image at each time-point separately on its own, and seeds are also set up individually for each time-point. Once seeds are specified, IRFC computation is carried out in 3D space leading to lung segmentation for every 3D volume image.

i-IRFC:

This is an interactive IRFC approach which operates in a manner similar to iterative Live Wire [7]. Livewire and iterative live wire are boundary-based approaches while all IRFC approaches are region-based. i-IRFC operates in a slice-by-slice manner. In one slice of the 4D volume which is strategically selected, seed sets A_O and A_B are first specified for the object and the background. The IRFC engine then operates in the 2D space of the current slice and delineates the object in the slice. The user then selects NEXT or PREVIOUS slice. The seeds are propagated automatically to the next/previous slice, the IRFC algorithm is executed in the next/previous slice, and the results are displayed. Since the delineation takes place almost instantly, the results are displayed instantly and the user can visually verify the delineation and accept if fully agreeable by selecting NEXT or PREVIOUS slice again. If the result is not acceptable the seeds are specified again or the propagated seeds are modified, and the process continues. This process provides fine control to the user in placing seeds for both the object and various background tissue components at the slice level which is difficult to implement in the S3D-IRFC and P4D-IRFC approaches. If we observe enough examples, we may be able to learn from user-specified seeds in the i-IRFC approach and devise seeds more intelligently for the S3D-IRFC and even the P4DIRFC approaches in the future.

The level of automation of P4D-IRFC is higher than that of S3D-IRFC, which in turn is greater than that of i-IRFC. Clearly, i-IRFC trades off efficiency for accuracy. As the degree of automation is lowered in this manner for improving accuracy, the precision (repeatability) of the method often declines as well.

3. RESULTS

3.1. Qualitative evaluation

Figure 1 illustrates seed setting in the i-IRFC approach in sagittal image and the resulting fuzzy connectedness map computed by the i-IRFC algorithm which leads to the final segmentation. Object seeds are denoted by a disc painted inside the lung space, and the background seeds are painted in blue on the background components. For IRFC approach, there is no need to use a lot of seeds as shown in Figure 1. One seed for the foreground seems enough and the number of background seeds can change, for example, from one background seed to several and then to more seeds roughly around the boundary. It is interesting that very few (foreground and background) seeds lead to the similar fuzzy connectedness maps as that from more seeds setting.

Figure 1.

Illustration of seed specification for object and background tissues in i-IRFC.

Figure 2 shows segmentation results obtained via i-IRFC for a 4D reconstructed image over one breathing cycle. Figure 3 shows segmentation results from the three approaches as well as from manual delineation. P4D-IRFC and S3D-IRFC approaches lead to over or under segmentation. i-IRFC achieves better results which most closely resemble the ground truth delineations.

Figure 2.

Lung segmentations obtained via i-IRFC from one 4D dMRI data set of one subject for one slice location over one full breathing cycle.

Figure 3.

Lung segmentation from different approaches. Top: From a patient image. Bottom: From dMRI of a normal adult. From left to right: Original image, manual segmentation, P4D-IRFC, S3D-IRFC, and i-IRFC.

3.2. Quantitative evaluation

Quantitative evaluation of the segmentations expressed in terms of true positive volume fraction (TPVF), false positive volume fraction (FPVF), and Hausdorff boundary distance (HD) in mm is shown in Table 1.

Table 1.

Mean (sd) of TPVF, FPVF, and HD measures for the three segmentation approaches.

	P4D-IRFC	S3D -IRFC	i-IRFC
TPFV	0.84 (0.09)	0.82 (0.02)	0.91 (0.03)
FPFV	0.04 (0.025)	0.01 (0.004)	0.03 (0.004)
HD (mm)	1.71 (0.19)	1.86 (0.32)	2.15 (0.33)

3.3. Reproducibility of i-IRFC

To assess the precision of i-IRFC, two operators who have been trained to recognize lungs in dynamic MRI, specified seed sets on 5 data sets twice. The intra- and inter- operator variability (precision) is assessed from those delineations following the framework described in [15], where precision is defined as R= |(S1 ⋂ S2)/(S1 ⋃ S2)|, where S1 and S2 are binary segmentations in two repeated trials, and |.| denotes volume. Precision results are listed in Table 2.

Table 2.

Mean (sd) of intra-and inter-observer repeatability of segmentation with i-IRFC.

Precision	Intra-observer	Inter-observer	Overall
R (%)	94.1 (0.3)	89.6(0.9)	91.9 (2.7)

4. CONCLUSIONS

Lung segmentation on thoracic dynamic 4D MRI images is a necessary first step for the quantitative analysis of respiratory dynamics in restrictive lung diseases of pediatric TIS patients. In this paper, we propose an interactive IRFC-based approach as a potential practical solution to this challenging problem. Although less efficient compared to other more automated IRFC strategies evaluated, the level of detailed interaction in the i-IRFC method seems necessary to reliably segment the lung space in the TIS application with acceptable accuracy for the type of dMRI images that are currently obtainable.

The interdependency of accuracy, precision, and efficiency of segmentation methods [15] when using the same delineation engine often allows intentionally favoring one factor over others, as illustrated for i-IRFC in this paper.

Learning from the user interaction input supplied over many data sets may lead us to methods in the future where more efficient specification of the same information from the user may become feasible for this really challenging but important segmentation problem. For example, interaction is needed at certain locations in the anatomy. If the locations and nature of interaction required can be predicted with sufficient accuracy from learned experience, we may be able to improve efficiency without sacrificing accuracy and precision.