Abstract
Purpose:
There are several disease conditions that lead to upper airway restrictive disorders. In the study of these conditions, it is important to take into account the dynamic nature of the upper airway. Currently, dynamic magnetic resonance imaging is the modality of choice for studying these diseases. Unfortunately, the contrast resolution obtainable in the images poses many challenges for an effective segmentation of the upper airway structures. No viable methods have been developed to date to solve this problem. In this paper, the authors demonstrate a practical solution by employing an iterative relative fuzzy connectedness delineation algorithm as a tool.
Methods:
3D dynamic images were collected at ten equally spaced instances over the respiratory cycle (i.e., 4D) in 20 female subjects with obstructive sleep apnea syndrome. The proposed segmentation approach consists of the following steps. First, image background nonuniformities are corrected which is then followed by a process to correct for the nonstandardness of MR image intensities. Next, standardized image intensity statistics are gathered for the nasopharynx and oropharynx portions of the upper airway as well as the surrounding soft tissue structures including air outside the body region, hard palate, soft palate, tongue, and other soft structures around the airway including tonsils (left and right) and adenoid. The affinity functions needed for fuzzy connectedness computation are derived based on these tissue intensity statistics. In the next step, seeds for fuzzy connectedness computation are specified for the airway and the background tissue components. Seed specification is needed in only the 3D image corresponding to the first time instance of the 4D volume; from this information, the 3D volume corresponding to the first time point is segmented. Seeds are automatically generated for the next time point from the segmentation of the 3D volume corresponding to the previous time point, and the process continues and runs without human interaction and completes in 10 s for segmenting the airway structure in the whole 4D volume.
Results:
Qualitative evaluations performed to examine smoothness and continuity of motions of the entire upper airway as well as its transverse sections at critical anatomic locations indicate that the segmentations are consistent. Quantitative evaluations of the separate 200 3D volumes and the 20 4D volumes yielded true positive and false positive volume fractions around 95% and 0.1%, respectively, and mean boundary placement errors under 0.5 mm. The method is robust to variations in the subjective action of seed specification. Compared with a segmentation approach based on a registration technique to propagate segmentations, the proposed method is more efficient, accurate, and less prone to error propagation from one respiratory time point to the next.
Conclusions:
The proposed method is the first demonstration of a viable and practical approach for segmenting the upper airway structures in dynamic MR images. Compared to registration-based methods, it effectively reduces error propagation and consequently achieves not only more accurate segmentations but also more consistent motion representation in the segmentations. The method is practical, requiring minimal user interaction and computational time.
Keywords: 4D MR imaging, upper airway, segmentation, fuzzy connectedness
1. INTRODUCTION
1.A. Background
Several important medical conditions in childhood, such as obesity, polycystic ovary syndrome, adenotonsillar hypertrophy, and craniofacial and genetic disorders that are associated with upper airway restriction during sleep, often lead to obstructive sleep apnea syndrome (OSAS).1 When left untreated, pediatric OSAS carries significant sequelae including neurocognitive and behavioral deficits along with other cardiovascular and metabolic derangements including hypertension, glucose intolerance, and diabetes mellitus.2 When trying to understand the mechanisms of airway obstruction during sleep in these conditions and the impact of various effects of treatment, it is imperative to take into account the dynamic changes that occur in the upper airway during the respiratory cycle under natural breathing conditions as much as possible. As such, various dynamic imaging protocols have been investigated in the study of the upper airway in these subjects including magnetic resonance imaging (MRI), computerized tomography (CT), and optical coherence tomography (OCT). While OCT offers high spatial resolution, it is intrusive, has poor depth of penetration, and has shadowing effects. CT affords good spatial and reasonable temporal resolution but has poor contrast resolution for soft tissue structures and has radiation concerns especially in the dynamic mode and in imaging children. Therefore, at present, dynamic MRI is the modality of choice in studying the upper airway in these disorders.3,4
Two types of approaches have been used to study OSAS with imaging. The first is one of the syntheses wherein a patient-specific biomechanical model of the upper airway is established5,6 to simulate airway dynamics by making use of the anatomic information derived from the patient images. The model parameters and behavior are used to characterize OSAS. Its effectiveness depends on the data used for building the model and the fidelity of the model. The second approach is one of the analyses wherein image data are processed to harness optimally OSAS-specific information that may reside in the images.5 Its effectiveness also depends on the information content of the image data and the processing methods. Whatever is the approach taken to study OSAS from images, a fundamental step that becomes necessary is the segmentation of the upper airway in the acquired images.
Because of the sparse, tubular nature of the upper airway and the surrounding low-contrast hard and soft structures, inadequate contrast resolution obtainable in the MR images leaves many challenges for an effective segmentation of the dynamic airway in 4D MR images. Due to the 4D nature of the images, manual or interactive segmentation becomes impractical in studying patient populations because of the extensive human labor involved. Any practical method should be either fully automatic or call for minimal user interaction, with minimal user-related bias, in segmenting each 4D MRI data set. The only 3D methods for upper airway segmentation that we are aware of are those described in previous Refs. 7 and 8. A method based on absolute fuzzy connectedness (FC) whose goal was to perform segmentation on static 3D MRI images was previously proposed.8 Another general methodology called automatic anatomy recognition (AAR), based on body-wide fuzzy models, demonstrated its ability to locate and delineate objects in 3D static images from different parts of the body including the neck and upper airway region where 14 objects in the vicinity of the upper airway were considered.7 No viable methods have been currently demonstrated in the literature for the segmentation of the upper airway in 4D dynamic MR image data sets.
1.B. 4D image segmentation in other areas
4D image segmentation approaches for brain,9 breast,10 lungs,11 heart,12–16 aorta,17 and kidneys,18 and time-varying vocal tract outline19 have been studied. In general, approaches to 4D image segmentation can be grouped into two categories. The first category is true 4D segmentation where all volumes are considered at the same time. Several segmentation frameworks including graph-based 4D optimal surface detection,17,20 hidden Markov model,9 level sets,10 and 4D probabilistic atlases15 have been proposed in this category. As an example, a 4D probabilistic atlas approach for heart segmentation was proposed15 which includes spatial and time-varying probability maps for the left and right ventricles, using which 3D volumes at all phases are segmented at the same time. The 4D atlas (multiple 3D atlases over time) was constructed by manually segmenting all 3D sequences from 14 volunteers. The disadvantage of this approach is that much labor is required for building the 4D atlas. The second category is to adopt registration related techniques11–14,16–19 which are the common approaches for 4D image segmentation by performing segmentation individually in each 3D volume. A 4D image is segmented by first manually carrying out segmentation of a 3D spatial volume corresponding to one time point (TP) and then propagating the manual segmentation to other time points to achieve segmentations at those time points. This is done by morphing the manual segmentation by the deformation needed to register the 3D reference volume to the volume at other time points. The rationale behind this approach is that the motion between successive time points is small, smooth, and consistent. Since this is the paradigm considered in this paper, some examples of this approach from the literature are reviewed below briefly.
Deformable image registration was used to propagate manual segmentations of the heart, left and right lung, and spinal cord in CT images at end inspiration to other respiratory phases.12 An algorithm for 4D cardiac micro-CT segmentation using histogram and region sampling technique was previously proposed.13 The method yielded consistent functional measurements for the left ventricle but it was not readily applicable to the myocardium and the other chambers of the heart. Another example of segmentation of 4D cardiac gated micro-CT images of the mouse is to make manual segmentation first in one phase and then propagate to other phases via registration.14 Yang et al.11 also follow a similar approach for segmenting the lungs via 4D thoracic MRI. The only difference is that they use spatiotemporal information about diaphragm movement to optimally select one 3D volume as reference instead of using the volume at end inspiration. Song et al.18 performed 4D MR image analysis of dynamic kidney images by combining registration and 4D time-series segmentation techniques. Given a 4D MR renography image, kidneys are first manually cropped from the whole body image and then rough rigid alignment is done by registering every 3D volume to a reference volume. The aligned image series is then segmented using the 4D time series algorithm. Registration, time series estimation, and segmentation are then performed iteratively. Bresch and Narayanan19 proposed a region segmentation algorithm in the frequency domain and applied to upper airway real-time MRI slice images. The goal is to extract the time-varying vocal tract outline and the position of the articulators to facilitate the study of the time-varying shape of the vocal tract during speech production. The segmentation algorithm builds an anatomically informed object geometrical model, and then the model with its intensity information is fitted to the image by registration between the model and every 3D image in the MRI sequence. Different strategies are used, such as, for tongue, translation, rotation, and scaling, but for epiglottis, only translation and rotation. A registration-based framework for whole heart segmentation by propagating the segmentation from a template 3D ultrasound volume to other volumes in a 4D image through registration was proposed16 which utilizes a new similarity measure combining local phase and intensity information, and local geometry.
Several limitations of the approaches based on registration in the second category motivated us to develop the proposed method for upper airway segmentation. These approaches require a reference volume, a similarity metric, and an optimal searching method to be carefully chosen that are appropriate for the problem at hand. The registration-based approach to 4D images is somewhat inefficient. It is also known that the larger the difference between the two 3D volumes, the more iterations required for registration to converge. Further, when this difference is large, registration may not produce accurate results. Usually adjacent volumes are therefore used for registration, but then the propagated and accumulated registration error may become significant for volumes at later time points. Most importantly, for sparse objects like the upper airway structure, image registration may fail to bring the focus of registration to these thin, subtle, tubular structures, resulting in registration errors that are substantially compared to the thickness/width of these structures, which can lead to unacceptable segmentation results and motion estimation.
In this paper, a nonregistration based segmentation approach is proposed for 4D dynamic upper airway MR images based on an iterative relative fuzzy connectedness (IRFC) algorithm21 which calls for minimal user interaction. IRFC is a top-of-the-line algorithm in the FC family which operates with the basic principles of FC but by iteratively reinforcing the segmentation evidence in a conservative manner.22 IRFC leads to more effective segmentations using relative connectedness to minimize moderately strong paths seeping through the object of interest. The basic idea is to identify the “core” of the object through relative connectedness in the first iteration. Then this region is excluded from being considered by other co-objects for tracking their connectivity path through.23,24 Like other FC members, IRFC requires seeds to be specified in the object as well as in the background components. While we are developing automatic means of generating these seeds guided by anatomic models,7 in this paper, we study an approach wherein the seeds are specified interactively on the images corresponding to only one time point, and the rest of the 4D segmentation process proceeds without requiring human interaction, as described in Sec. 2. Due to the theoretically provable robustness property of FC methods to varying seeds,21–26 only a few seed points are needed for the object and the co-objects surrounding the object of interest. Once seeds of the object and background co-objects are correctly identified on the remaining time point images and affinity functions are correctly set up as described in Sec. 2, IRFC can automatically and efficiently segment the target object at all time points. We demonstrate the performance of the method on T1-weighted MRI sagittal images of the upper airway region of patients with OSAS in Sec. 3 where we compare its performance quantitatively to a registration-based segmentation propagation method. We discuss the results and our conclusions in Sec. 4. Some preliminary results along the lines of the study in this paper appeared previously in the proceedings of the SPIE Medical Imaging 2014 conference.27 This paper is a significant extension over the conference paper in the following aspects: more details on preprocessing and the 4DIRFC engine and a formal pseudocode presentation of the algorithm which makes it easy for others to implement the algorithm; extensive background and literature review which was missing in the SPIE paper; greatly enriched experimental results by describing data sets and patients in detail, testing the repeatability of the 4DIRFC method, and animating 4DIRFC results; and expanded concluding remarks.
2. MATERIALS AND METHODS
2.A. Image data and preprocessing
4D dynamic MR images utilized in this paper were acquired by a retrospective gating method.3 In this approach, image data acquisition was triggered only if the input respiratory signal was within predefined temporal tolerances. Abnormal volumes arising due to events such as swallowing and deep inhalation are discarded. Images were collected on a 3 T Philips Achieva scanner. A 3D, T1-weighted, inversion-prepared gradient echo sequence, acquired in the sagittal plane and reconstructed in the axial and coronal planes, was used for the 4D dynamic study. Thirty-six 1.1-mm thick sagittal slices were acquired. The slices were 240 × 240 pixels with a pixel size of 1 × 1 mm. 4D image data from 20 female subjects with OSAS and each 4D image with ten equally spaced time points over the respiratory cycle (a total of 200 3D volumes) were used in our experiments. Subjects were between 14 and 18 yr of age. Table I describes the patient condition (OSAS or PCOS) for each of the 20 subjects studied.
TABLE I.
Disease condition of the patients included in the study.
| Disease | Total | ||||
|---|---|---|---|---|---|
| OSAS | Yes | No | Yes | No | — |
| PCOS | Yes | Yes | No | No | — |
| Subjects | 9 | 6 | 3 | 2 | 20 |
Note: PCOS, polycystic ovary syndrome; OSAS, obstructive sleep apnea syndrome.
MR image analysis methods are challenging because of two phenomena—image intensity nonuniformity and nonstandardness. Nonuniformity refers to the presence of a slow-varying background component of intensity which may make the same tissue appear with widely different intensities in different locations of the same acquired image set. Nonstandardness refers to the significant variation of intensities of the same tissue region in images of different subjects acquired on the same scanner with the same imaging sequence. Correction to properly address these two phenomena is essential in order to set the values of the parameters of any segmentation algorithm in a consistent manner for meaningful performance. As recommended in previous research,28 we perform nonuniformity correction first29 followed by intensity standardization30 of all images before performing any segmentation operation.
2.B. 4DIRFC engine
IRFC is a top-of-the-line segmentation engine in the FC family.21 The FC framework is graph-based. We first provide a broad description of the basics of FC and then delineate its adaptation to the problem at hand. All FC methods aim at defining and delineating “objects” in an image via the concept of the strength of connectedness of voxels or how voxels “hang together” in the image when compared to surrounding co-objects. The FC family has some unique characteristics compared to other image based delineation methods.23 These include theoretically provable robustness of segmentation to the number and position of seeds, computational efficiency, and the ability to capture a variety of image characteristics including blur, noise, background nonuniformity, and prior information about the geographic layout of objects.
Let I = (C, f) denote a 3D image, where C is a rectangular array of voxels and f is the MR image intensity function which assigns a value to each voxel in C. A graph (C, α) is associated with image I = (C, f), where α is an adjacency relation on C such as 6-, 18-, or 26- adjacency. Each pair (c, d) of adjacent voxels in α is assigned an affinity value κ (c, d). To each path π in the graph (or equivalently in I) in the set of all possible paths Πa,b between two voxels a and b of C, a strength of connectedness K(π) is determined, which is defined to be the minimum of the affinities along the path. The connectivity measure K*(a, b) between a and b is then defined to be K*(a, b) = max{K(π) : π ∈ Πa,b}. The notion of connectivity measure between two voxels can be generalized to the case of “between a set A and voxel b” by a slight modification: K*(A, b) = max{K(π) : π ∈ Πa,b and a ∈ A}. By using a fast algorithm to compute K*(A, b),25 the machinery of FC allows a variety of approaches to define and compute objects in images by specifying appropriate affinity functions and seed sets for objects and co-objects, and setting up a competition among all objects. The central idea is that an object gets defined in an image because of the presence of other co-objects. Each object is initialized by a seed voxel (or a set of seed voxels). Any voxel v in the image is considered to belong to that object with respect to whose seed (seed set) v has the highest strength of connectedness. IRFC uses an iterative strategy for fuzzy connectedness computation wherein the strongest connected core parts are first defined and iteratively relaxed to conservatively capture the more fuzzy peripheral parts subsequently.
In the IRFC algorithm, two seed sets AO and AB are indicated for an object O (in our case, upper airway) and the set of background objects B, respectively. Then the object indicated by AO is separated from the background indicated by AB by an iterative competition in connectivity measure between AO and every voxel c ∈ C and AB and c. In the proposed interactive IRFC method, AO and AB are specified with human interaction in the 3D image corresponding to the first time instance of the 10 time point 4D image. Affinities κO(c, d) and κB(c, d) for O and B are designed separately. Subsequently they are combined with affinity κ by taking a fuzzy union of κO and κB: κ(c, d) = max{κO(c, d), κB(c, d)}. Each of κO and κB has two components. The description below is for κO. The same applies to κB,
| (1) |
Here, ψO(c, d) represents a homogeneity component of affinity, meaning, the more similar image intensities f(c) and f(d) are at voxels c and d, the greater is this component of affinity between c and d. φO(c, d), the object feature component, on the other hand, describes the “degree of nearness” of the intensities at c and d to an intensity expected for the object O under consideration. For both ψO and φO, we use a Gaussian function as follows:
| (2) |
| (3) |
where σψO is a homogeneity parameter that indicates the standard deviation (Sd) of intensities within object O. Parameters mφO and σφO are the mean and standard deviation of object intensities which are estimated from a few sample object- and background-tissue regions and then fixed once for all. This is why MRI nonuniformity correction and standardization become crucial. For the upper airway object, a half-Gaussian form for Eq. (3) (that is, right half of the curve only) is chosen centered at mφO, the idea being that if f(c) and f(d) are both lower than mφO then this component of affinity should be maximum. This is because the airway regions appear dark in MR images. The background tissue regions considered constitute essentially the tissue regions surrounding the airway: air outside the body region, hard palate, soft palate, tongue, and other soft structures around the airway including tonsils (left and right) and adenoid. Based on our past experience with FC methods in other applications, we set w = 0.5 and σψO = σψB. Once the affinity functions are specified, the interactive IRFC process proceeds as follows:
| Procedure 4DIRFC |
| In: A 4D MRI/CT Image, affinity parameters for the object and co-objects. |
| Out: 4D binary image representing upper airway. |
| Begin |
| Step 1. Perform non-uniformity correction and standardization (for MRI only and not for CT); |
| Step 2. Specify seed sets AO and AB in 3D image of Time Point 1; |
| Step 3. Generate segmentation S of current 3D image by calling IRFC with seeds AO and AB and affinity κ; |
| Step 4. Generate new seed set AO for next time point from S; |
| Step 5. If all time points are not covered, go to Step 3; |
| Else, output 4D binary image; |
| End |
To start off, in the 3D image corresponding to the first time point, we specify seed sets AO and AB in the different tissue components mentioned above. The IRFC algorithm is then launched which results in a segmented binary volume. The algorithm then proceeds to the next time point by propagating seed sets from the previous to the next time point, and in this manner, the entire respiratory cycle is covered. Since the background tissue regions move very little with respiration, we reuse seed set AB from the previous time point for the next time point. AO for the next time point is modified by applying a morphological erosion operation to the binary segmented volume in the previous time point, the idea being that voxels in the core part of the airway structure remain in the same spatial position in all phases of the airway object over the breathing cycle. All affinity functions remain the same throughout all time points. Given the seed sets and affinity functions, delineation is completed automatically for the remaining time points. The procedure for the 4DIRFC method is summarized above. It calls the IRFC engine of Ref. 26.
3. EXPERIMENTS, EVALUATION, AND DISCUSSION
Experiments are carried out on image data from 20 subjects and each subject with 10 3D images. Evaluation utilized all 200 3D images and was based on comparison with both manually drawn ground truth segmentations as well as results produced by a segmentation propagation method that used registration.
3.A. Comparison with manually guided segmentations
The 4DIRFC algorithm has been integrated into the cavass (Ref. 31) software system32 with a friendly and flexible interface for interactively specifying seeds, setting affinity functions, and viewing results immediately. Object and co-object seeds can be placed on one or more 2D slices. Once seeds and affinity functions are set, the 3D fuzzy connectedness map is computed and displayed in seconds, see Fig. 1.
FIG. 1.
Left: seed selection for 4DIRFC at first time point. Yellow dots denote seeds for the object and red dots indicate seeds for the co-objects. Middle: Connectedness map from 4DIRFC at the time point indicated. Right: surface rendering of the 3D segmentation (color in online version only).
The fuzzy connectedness map indicates the strength of connectedness K*(AO, b) of the voxels b to the object seed set AO that are greater than the strength of connectedness to the background seed set. Voxels where the strength of connectedness to the background seed set is greater are indicated with 0 value in the connectedness map. The fuzzy connectedness map can be directly volume rendered or defuzzified at a threshold just above 0 and subsequently surface rendered as shown in the figure on right. Due to the robustness of seed selection of FC methods, just a few foreground seeds and background seeds (around the target organ—whole pharynx) are sufficient and the segmentation is provably insensitive to the actual location of the seeds.21,22,25 Note that since background seeds are propagated without change to subsequent time points, their setting should be chosen carefully to make sure that they are properly inside the co-objects in all time points. If the propagated background seeds are wrongly placed in some time points, the segmentation may fail due to inappropriate background tissues considered for FC competition between object and background. In our implementation, correct propagation can be easily and quickly checked by roaming through the ten slices along the time dimension of the 4D image.
In Fig. 2, we display the ten time-varying slices for one anatomic position along with the corresponding segmentations of an OSAS patient image set. White circles at different time points highlight the periodic motion of the airway over one breathing period. Figure 3 demonstrates surface renditions of the airway structure segmented from this entire 4D data set at ten time points. The segmentation results are visually acceptable in all ten time points in our inspection of the slices as well as 3D renditions for this as well as other data sets. We also visually observe the dynamic change of the axial cross-sectional area of the upper airway at several crucial anatomic locations as shown in Fig. 4. When displayed in cine mode, the area visually changed smoothly in a periodic manner over the respiratory cycle as expected.
FIG. 2.
Segmentation results are overlaid on MRI slices over a full respiratory cycle at one fixed slice position for data from one subject.
FIG. 3.
Surface renditions of the airway structure for all ten time points for the subject data set shown in Fig. 2.
FIG. 4.
Axial cross section of upper airway at different anatomic locations.
Nonuniformity correction and intensity standardization approaches can improve MR image segmentation results as illustrated in previous research.33 However some images with poor contrast resolution may offer significant challenges for upper airway segmentation. No practical techniques for handling such challenges have emerged. Figure 5 shows an example of the results from the proposed approach on one image with poor contrast resolution, and ground truth segmentation as well as the image after nonuniformity correction and intensity standardization.
FIG. 5.
An example of the image with poor contrast resolution. Left to right: original image, image after nonuniform correction and standardization, segmentation from the proposed approach, and ground truth segmentation.
Quantitative evaluation of the segmentation with measurements of true positive volume fraction (TPVF), false positive volume fraction (FPVF), and Hausdorff boundary distance (HD) is shown in Table II. Results from manually guided segmentation of all 200 3D data sets are used as ground truth which are achieved by using the livewire tools in cavass.34 (The reasons for choosing this tool for creating reference segmentations are the following. Livewire is a user-guided delineation tool where the user provides recognition help for boundary localization, the implemented algorithm performs delineation, and the two processes are tightly and synergistically integrated. The delineation performed is guaranteed to be always agreeable to the user. Its efficiency and precision have been shown to be better than manual contouring.) The mean and Sd values shown for each TP are over the 20 patients. Overall mean and Sd values are also listed in the last column. We observe that 4D MRI volumes of the upper airway can be segmented with the proposed approach with mean TPVF of 94%–95%, mean FPVF of 0.05%–0.08%, and mean HD of 0.5 mm. Note also that accuracy does not seem to change from one time point to the next.
TABLE II.
Quantitative evaluation of segmentation from 4DIRFC with TPVF, FPVF, and HD.
| Time points | TP1 | TP2 | TP3 | TP4 | TP5 | TP6 | TP7 | TP8 | TP9 | TP10 | Mean | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TPVF (%) | mean | 94.21 | 94.95 | 95.31 | 95.10 | 95.66 | 95.26 | 95.24 | 95.26 | 94.69 | 94.72 | 95.04 |
| Sd | 7.04 | 5.60 | 5.19 | 7.44 | 5.31 | 6.40 | 6.88 | 6.95 | 7.70 | 8.23 | 6.68 | |
| FPVF (%) | mean | 0.06 | 0.05 | 0.06 | 0.05 | 0.06 | 0.06 | 0.06 | 0.08 | 0.08 | 0.08 | 0.06 |
| Sd | 0.12 | 0.12 | 0.15 | 0.11 | 0.14 | 0.10 | 0.10 | 0.13 | 0.14 | 0.14 | 0.13 | |
| HD (mm) | mean | 0.49 | 0.49 | 0.49 | 0.49 | 0.49 | 0.49 | 0.49 | 0.49 | 0.49 | 0.49 | 0.49 |
| Sd | 0.01 | 0.00 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.01 | 0.01 | |
To illustrate the precision (repeatability) of 4DIRFC with respect to seed specification, we list in Table III the difference in segmentations resulting from two separate experiments conducted by an operator wherein the seeds were specified independently in two separate sessions. The difference in the two segmentations is expressed for each time point over 20 subjects by computing the exclusive or (EOR) between the segmentations. The quantity %EOR listed in Table II is the EOR volume between two segmentations expressed as a fraction of the volume of true segmentation. The mean %EOR value from 400 segmentations at ten time points is 4.3%. The mean HD measure between the two segmentations is also within 1 mm. As seen from the table, the segmentations are highly repeatable with respect to the only subjective action needed, namely, manual seed specification.
TABLE III.
Repeatability of 4DIRFC with respect to seed specification.
| Time points | TP1 | TP2 | TP3 | TP4 | TP5 | TP6 | TP7 | TP8 | TP9 | TP10 | Mean | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| EOR (%) | mean | 4.26 | 3.65 | 4.55 | 4.34 | 4.45 | 4.11 | 4.21 | 4.57 | 4.51 | 4.34 | 4.30 |
| Sd | 2.55 | 2.51 | 2.32 | 2.66 | 2.61 | 2.86 | 2.81 | 2.74 | 2.78 | 3.34 | 2.72 | |
| HD (mm) | mean | 1.01 | 0.94 | 0.95 | 1.00 | 1.00 | 0.95 | 0.94 | 0.95 | 0.89 | 0.94 | 0.96 |
| Sd | 0.52 | 0.52 | 0.51 | 0.52 | 0.52 | 0.52 | 0.52 | 0.52 | 0.50 | 0.52 | 0.52 | |
3.B. Comparison with a method of registration-based segmentation propagation
In the registration approach, the segmentation at the current time point is propagated to the next time point by using the transformation derived from an affine registration (involving three translation parameters, three rotation parameters, and three scale parameters) between the volumes at the current and next time points. Figure 6 shows the segmentation results from the three approaches—top row for manual segmentation, middle row for registration approach, and bottom row for 4DIRFC. The results are shown for time points 1, 3, 6, 8, and 10.
FIG. 6.
Segmentation results from five time points (from left to right, time points 1, 3, 6, 8, and 10 in one breathing cycle) from manual segmentation in top row, from registration in middle row, and from 4DIRFC in bottom row. Arrows indicate the site where IRFC and manual segmentations exhibit correct dynamic change but the registration propagation method does not.
The results from 4DIRFC appear to be closer to manual segmentation than the registration method, considering the periodic breathing motion. Table IV lists TPVF, FPVF, and HD measures for the registration method. Clearly 4DIRFC achieves higher TPVF and lower FPVF and HD than the registration method. Note that for the registration method, the volume at the first time point is used as reference and hence will be its result as well, and so for evaluation, results from only the remaining nine time points are considered. Since segmentation based on registration is implemented in a propagating manner, the segmentation error may also propagate. TPVF becomes lower and FPVF becomes higher starting from TP2. A paired t-test conducted for each of the three measures for all-time points (except the first time point where the segmentation from manual segmentation is same as ground truth) over all 20 data sets showed that all three measures were better for 4DIRFC than the registration method with statistical significance (P < 0.01). An unpaired t-test for the three measures conducted over all time points and data sets also showed a similar behavior, with the mean TPVF from 4DIRFC about 10% higher than that from the registration method with statistical significance, while improvements in FPVF and HD may not be considered substantial. The same HD value observed at every time point is due to the fact that HD is an average measure over the entire 3D surface.
TABLE IV.
Quantitative evaluation of the segmentation results from registration strategy.
| Time points | TP1 | TP2 | TP3 | TP4 | TP5 | TP6 | TP7 | TP8 | TP9 | TP10 | Mean | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TPVF (%) | mean | — | 88.07 | 87.11 | 86.24 | 84.66 | 84.66 | 85.07 | 84.65 | 85.17 | 86.22 | 85.76 |
| Sd | — | 6.37 | 7.36 | 8.43 | 8.51 | 9.33 | 9.32 | 9.56 | 8.58 | 8.36 | 8.42 | |
| FPVF (%) | mean | — | 0.22 | 0.24 | 0.23 | 0.25 | 0.25 | 0.27 | 0.27 | 0.26 | 0.21 | 0.25 |
| Sd | — | 0.12 | 0.12 | 0.13 | 0.13 | 0.14 | 0.15 | 0.15 | 0.15 | 0.11 | 0.13 | |
| HD (mm) | mean | — | 0.50 | 0.50 | 0.50 | 0.50 | 0.50 | 0.50 | 0.50 | 0.50 | 0.50 | 0.50 |
| Sd | — | 0.01 | 0.01 | 0.01 | 0.01 | 0.02 | 0.01 | 0.01 | 0.01 | 0.01 | 0.01 | |
Both manual segmentation and 4DIRFC seem to capture the change of the size of upper airway at different time points and the associated motion correctly. However, the results from registration seem to be not able to capture the proper motion in its magnitude or periodicity, as evidenced by cine displays of the 3D rendered surfaces over a breathing period. Three movies named “Manual.avi,” “4DIRFC.avi,” and “Registration.avi” animating the motion for the three segmentation approaches are included in “4D-DynamicMRI-Segmentations.pptx” and available at the link shown in Ref. 35, where the differences among the animations are highlighted by white arrows. Due to the robustness of seed selection for 4DIRFC and since segmentation is actually carried out on each time point, even small motions are captured by the method which is crucial for the OSAS application. However, the registration method fails to capture true motion due to registration errors and not performing actual delineation. TPVF and FPVF of all subjects over the entire breathing cycle are shown in Figs. 7 and 8, respectively, for the two methods. FPVF values from both methods are less than 0.6%, and 4DIRFC achieves lower average FPVF. TPVF of 4DIRFC is significantly higher (even up to 20%–30%, such as for subjects 8, 12, 15, and 18) than that of the registration method.
FIG. 7.
Average TPVF over the breathing period for 4DIRFC and registration method on 20 subjects.
FIG. 8.
Average FPVF over the breathing period for 4DIRFC and registration method on 20 subjects.
As seen from Figs. 7 and 8, low image quality becomes a challenge to both registration and 4DIRFC methods on some subjects (5, 6, and 10), although 4DIRFC still outperforms the registration method on those images. With image quality enhancement approaches and more effective nonuniformity correction and standardization approaches,36 intensity values should have more consistent tissue-specific meaning and the performance of IRFC may further improve.
Once seed sets are specified in the 3D image corresponding to the first time point, 4DIRFC takes on average 10 s to segment an entire 4D image on a 4-core Intel Xeon 3.6 GHz CPU with 8 GB RAM and running the linux-jb18.3.7.20-1.16 operating system. After segmenting the first time point 3D image, the registration method requires on average about 3 min on the same platform.
4. CONCLUSIONS
This paper demonstrates a practical solution by employing an iterative relative fuzzy connectedness delineation algorithm as a tool for upper airway segmentation on 4D dynamic MRI images. This problem has remained a challenge in the study of OSAS where it is considered important to study the architecture and dynamics of the upper airway in a state very close to the tidal breathing condition. No practical techniques for its segmentation have emerged as yet due to the poor contrast resolution obtainable in these images. In the proposed approach, after preprocessing to correct for background image nonuniformities and nonstandardness of intensities, seeds are specified for the airway and its crucial background tissue components. Seed specification is needed in only the 3D image corresponding to the first time instance of the 4D volume. Subsequently the process runs without human interaction and completes in 10 s for segmenting one whole 4D volume. The approach achieves a mean TPVF of about 95%, mean FPVF of about 0.1%, and mean HD around 0.5 mm. The method is also highly reproducible and can thus be used in a production mode. It seems to be the first demonstration of a viable approach for segmenting the upper airway structures in dynamic MR images. The method is practical requiring minimal user interaction and computational time. Compared with segmentation results from the registration approach, the proposed method is more accurate and efficient. Although more sophisticated (such as deformable) registration methods37,38 are available, it does not automatically imply that they are better for the task at hand. We believe our choice was reasonable since the changes are small from one time point to another. While deformable registration may improve in some sections of the boundary over affine, they are also known for overcorrecting and introducing extreme deformations. While we admit that we did not carry out comparison with deformable registration, another reason for not considering them was that they require much more computational time than affine methods.
As to possible future work, if the acquisition field of view of our current dynamic MRI sequence3 is expanded to include the entire neck region to cover other objects surrounding the upper airway as opposed to the upper airway region only in the current protocol, then 4DIRFC can be extended to handle important static objects such as tonsils, fat pad, adenoid, mandible, and soft palate in the neck region which are currently segmented using a 3D method.39 In this manner a single 4D algorithm can potentially handle both static and dynamic objects. The proposed method has potential applications in other areas such as 4D segmentation of dynamic MRI images of the thorax for delineating pleural space, ribs, and the diaphragm.40 It has applications also in segmenting 4D CT images of different dynamic objects in the body including the upper airway region.
ACKNOWLEDGMENT
The research reported here is partly supported by a DHHS Grant No. HL105212.
REFERENCES
- 1.Davidson Ward S. L., Amin R., Arens R., Chen Z., Davis S., Gutmark E., Superfine R., Wong B., Zdanski C., and Khoo M. C. K., “Pediatric sleep-related breathing disorders: Advances in imaging and computational modeling,” IEEE Pulse 5, 33–39 (2014). 10.1109/MPUL.2014.2339293 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Muzumdar H. and Arens R., “Physiological effects of obstructive sleep apnea syndrome in childhood,” Respir. Physiol. Neurobiol. 188(3), 370–382 (2013). 10.1016/j.resp.2013.05.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Wagshul M. E., Sin S., Lipton M. L., Shifteh K., and Arens R., “Novel retrospective, respiratory-gating method enables 3D, high resolution, dynamic imaging of the upper airway during tidal breathing 4D imaging,” Magn. Reson. Med. 70(6), 1580–1590 (2013). 10.1002/mrm.24608 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Kim Y. C., Joshi B., Loloyan S., Kato R., Khoo M. C. K., Ward S. L. D., and Nayak K. S., “Investigations of upper airway obstruction pattern in sleep apnea benefit from real-time 3D MRI,” Proceedings of ISMRM 22nd Scientific Sessions, Italy (2014), p. 4387, see https://mrel.usc.edu/pdf/ismrm2014/4387.pdf. [Google Scholar]
- 5.Xu C., Sin S., McDonough J. M., Udupa J. K., Guez A., Arens R., and Wotton D. M., “Computational fluid dynamics modeling of the upper airway of children with obstructive sleep apnea syndrome in steady flow,” J. Biomech. 39(11), 2043–2054 (2006). 10.1016/j.jbiomech.2005.06.021 [DOI] [PubMed] [Google Scholar]
- 6.Arens R., Sin S., Nandalike K., Rieder J., Khan U. I., Freeman K., Wylie-Rosett J., Lipton M. L., Wotton D. M., McDonough J. M., and Shifteh K., “Upper airway structure and body fat composition in obese children with obstructive sleep apnea syndrome,” Am. J. Respir. Crit. Care Med. 183, 782–787 (2011). 10.1164/rccm.201008-1249OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Udupa J. K., Odhner D., Zhao L., Tong Y., Matsumoto M. M., Ciesielski K. C., Falcao A. X., Vaideeswaran P., Ciesielski V., Saboury B., Mohammadianrasanani S., Sin S., Arens R., and Torigian D. A., “Body-wide hierarchical fuzzy modeling, recognition, and delineation of anatomy in medical images,” Med. Image Anal. 18, 752–771 (2014). 10.1016/j.media.2014.04.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Liu J., Udupa J. K., Odhner D., McDonough J. M., and Arens R., “System for upper airway segmentation and measurement with MR imaging and fuzzy connectedness,” Acad. Radiol. 10(1), 13–24 (2003). 10.1016/S1076-6332(03)80783-3 [DOI] [PubMed] [Google Scholar]
- 9.Solomon J., Butman J. A., and Sood A., “Segmentation of brain tumors in 4D MR images using the hidden Markov model,” Comput. Methods Programs Biomed. 84(3), 76–85 (2006). 10.1016/j.cmpb.2006.09.007 [DOI] [PubMed] [Google Scholar]
- 10.Qi W., Hui D., and Wang G. Z., “A multistage processing procedure for 4D breast MRI segmentation,” in Proceedings of IEEE Engineering in Medicine and Biology Society, Vancouver, BC (IEEE, 2008), pp. 3036–3039. 10.1109/IEMBS.2008.4649843 [DOI] [PubMed] [Google Scholar]
- 11.Yang Y., Van R. E., Poh C. L., Tan C. H., and Tham I. W., “A spatial–temporal based scheme for efficient registration-based segmentation of thoracic 4D MRI,” IEEE J. Biomed. Health Inf. 18(3), 969–977 (2014). 10.1109/JBHI.2013.2282183 [DOI] [PubMed] [Google Scholar]
- 12.Wijesooriya K., Weiss E., Dill V., Dong L., Mohan R., Joshi S., and Keall P. J., “Quantifying the accuracy of automated structures segmentation in 4D CT images using a deformable image registration algorithm,” Med. Phys. 35(4), 1251–1260 (2008). 10.1118/1.2839120 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Badea C. T., Johnston S., Johnson B., Lin M., Hedlund L. W., and Allan Johnson G., “A dual micro-CT system for small animal imaging,” Proc. SPIE 6913, 691342-1–691342-10 (2008). 10.1117/12.772303 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Clark D., Badea A., Liu Y., Johnson G. A., and Badea C. T., “Registration-based segmentation of murine 4D cardiac micro-CT data using symmetric normalization,” Phys. Med. Biol. 57(19), 6125–6145 (2012). 10.1088/0031-9155/57/19/6125 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Lorenzo-Valdés M., Sanchez-Ortiz G. I., Elkington A. G., Mohiaddin R. H., and Rueckert D., “Segmentation of cardiac MR images using the EM algorithm with a 4D probabilistic atlas and a global connectivity filter,” Med. Image Anal. 8(3), 255–265 (2004). 10.1016/j.media.2004.06.005 [DOI] [PubMed] [Google Scholar]
- 16.Zhuang X., Yao C., Ma Y. L., Hawkes D., Penney G., and Ourselin S., “Registration-based propagation for whole heart segmentation from compounded 3D echocardiography,” inIEEE International Symposium on Biomedical Imaging: From Nano to Macro, Rotterdam (IEEE, 2010), pp. 1093–1096. 10.1109/ISBI.2010.5490183 [DOI] [Google Scholar]
- 17.Zhao F., Zhang H., Wahle A., Thomas M. T., Stolpen A. H., Scholz T. D., and Sonka M., “Congenital aortic disease: 4D magnetic resonance segmentation and quantitative analysis,” Med. Image Anal. 13(3), 483–493 (2009). 10.1016/j.media.2009.02.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Song T., Lee V. S., Rusinek H., Wong S., and Laine A. F., “Four dimensional MR image analysis of dynamic renography,” in Proceedings of IEEE Engineering in Medicine and Biology Society, New York, NY (IEEE, 2006), Vol. 1, pp. 3134–3137. 10.1109/IEMBS.2006.260178 [DOI] [PubMed] [Google Scholar]
- 19.Bresch E. and Narayanan S., “Region segmentation in the frequency domain applied to upper airway real-time magnetic resonance images,” IEEE Trans. Med. Imaging 28(3), 323–338 (2009). 10.1109/TMI.2008.928920 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Li K., Wu X., Chen D. Z., and Sonka M., “Optimal surface segmentation in volumetric images-a graph-theoretic approach,” IEEE Trans. Pattern Anal. Mach. Intell. 28(1), 119–134 (2006). 10.1109/tpami.2006.19 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Udupa J. K. and Saha P. K., “Relative fuzzy connectedness and object definition: Theory, algorithms, and applications in image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell. 24(11), 1485–1500 (2002). 10.1109/TPAMI.2002.1046162 [DOI] [Google Scholar]
- 22.Udupa J. K. and Saha P. K., “Fuzzy connectedness and image segmentation,” Proc. IEEE 91(10), 1649–1669 (2003). 10.1109/JPROC.2003.817883 [DOI] [Google Scholar]
- 23.Ciesielski K. C. and Udupa J. K., “Region-based segmentation: Fuzzy connectedness, graph cut and related algorithms,” in Biomedical Image Processing, Biological and Medical Physics, Biomedical Engineering (Springer-Verlag, Berlin, 2011), pp. 251–278. [Google Scholar]
- 24.Saha P. K. and Udupa J. K., “Iterative relative fuzzy connectedness and object definition: Theory, algorithms, and applications in image segmentation,” in Proceedings of IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, Hilton Head, SC (IEEE, 2000), pp. 28–35. 10.1109/MMBIA.2000.852357 [DOI] [Google Scholar]
- 25.Udupa J. K. and Samarasekera S., “Fuzzy connectedness and object definition: Theory, algorithms, and applications in image segmentation,” Graphical Models Image Process. 58(3), 246–261 (1996). 10.1006/gmip.1996.0021 [DOI] [Google Scholar]
- 26.Ciesielski K. C., Udupa J. K., AFalcao A. X., and Miranda P. A. V., “Fuzzy connectedness image segmentation in graph cut formulation: A linear-time algorithm and a comparative analysis,” J. Math. Imaging Vision 44, 375–398 (2012). 10.1007/s10851-012-0333-3 [DOI] [Google Scholar]
- 27.Tong Y., Udupa J. K., Odhner D., Sin S., Wagshul M. E., and Arens R., “A novel non-registration based segmentation approach of 4D dynamic upper airway MR images: Minimally interactive fuzzy connectedness,” Proc. SPIE 9038, 90380Z-1–90380Z-6 (2014). 10.1117/12.2044473 [DOI] [Google Scholar]
- 28.Madabhushi A. and Udupa J. K., “Interplay between intensity standardization and inhomogeneity correction in MR image processing,” IEEE Trans. Med. Imaging 24(5), 561–576 (2005). 10.1109/TMI.2004.843256 [DOI] [PubMed] [Google Scholar]
- 29.Zhuge Y., Udupa J. K., Liu J., and Saha P. K., “Image background inhomogeneity correction in MRI via intensity standardization,” Comput. Med. Imaging Graphics 33(1), 7–16 (2009). 10.1016/j.compmedimag.2008.09.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Nyúl L. G. and Udupa J. K., “On standardizing the MR image intensity scale,” Magn. Reson. Med. 42(6), 1072–1081 (1999). [DOI] [PubMed] [Google Scholar]
- 31.Grevera G., Udupa J. K., Odhner D., Zhuge Y., Souze A., Iwanaga T., and Mishra S., “ cavass: A computer-assisted visualization and analysis software system–visualization aspects,” Proc. SPIE 6509, 650904-1–650904-12 (2007). 10.1117/12.711170 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.cavassis an open source software system developed and distributed by the Medical Image Processing Group, University of Pennsylvania, and can be downloaded from http://www.mipg.upenn.edu/Vnews/mipg_software.html.
- 33.Zhuge Y. and Udupa J. K., “Intensity standardization simplifies brain MR image segmentation,” Comput. Vision Image Understanding 113(10), 1095–1103 (2009). 10.1016/j.cviu.2009.06.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Falcao A. X., Udupa J. K., and Miyazawa F. K., “An ultra-fast user-steered image segmentation paradigm: Live wire on the fly,” IEEE Trans. Med. Imaging 19(1), 55–62 (2000). 10.1109/42.832960 [DOI] [PubMed] [Google Scholar]
- 35.Three movies displaying the motion of the 3D rendered surfaces derived from the three methods over a breathing period can be viewed at http://www.mipg.upenn.edu/Vnews/4DIRFC/4DIRFC.html.
- 36.Tong Y., Udupa J. K., Odhner D., Sharma S., and Torigian D. A., “Interactive non-uniformity correction and intensity standardization of MR images,” Proc. SPIE 9415, 94151N-1–94151N-7 (2015). 10.1117/12.2082878 [DOI] [Google Scholar]
- 37.Uneri A., Nithiananthan S., Schafer S., Otake Y., Stayman J. W., Kleinszig G., Sussman M. S., Prince J. L., and Siewerdsen J. H., “Deformable registration of the inflated and deflated lung in cone-beam CT guided thoracic surgery: Initial investigation of a combined model- and image-driven approach,” Med. Phys. 40, 017501 (15pp.) (2013). 10.1118/1.4767757 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Pekar V., Gladilin E., and Rohr K., “An adaptive irregular grid approach for 3D deformable image registration,” Phys. Med. Biol. 51, 361–377 (2006). 10.1088/0031-9155/51/2/012 [DOI] [PubMed] [Google Scholar]
- 39.Tong Y., Udupa J. K., Odhner D., Sin S., and Arens R., “Recognition of upper airway and surrounding structures at MRI in pediatric PCOS and OSAS,” Proc. SPIE 8670, 86702S-1–86702S-7 (2013). 10.1117/12.2007948 [DOI] [Google Scholar]
- 40.Tong Y., Udupa J. K., Ciesielski K. C., McDonough J. M., Mong A., and Campbell R. M., “Graph-based retrospective 4D image construction from free-breathing MRI slice acquisitions,” Proc. SPIE 9038, 9038O-1–9038O-7 (2014). 10.1117/12.2044419 [DOI] [Google Scholar]








