Automatic Segmentation of the Left Atrium from MR Images via Variational Region Growing With a Moments-Based Shape Prior

Abstract

The planning and evaluation of left atrial ablation procedures are commonly based on the segmentation of the left atrium, which is a challenging task due to large anatomical variations. In this paper, we propose an automatic approach for segmenting the left atrium from magnetic resonance imagery. The segmentation problem is formulated as a problem in variational region growing. In particular, the method starts locally by searching for a seed region of the left atrium from an MR slice. A global constraint is imposed by applying a shape prior to the left atrium represented by Zernike moments. The overall growing process is guided by the robust statistics of intensities from the seed region along with the shape prior to capture the entire atrial region. The robustness and accuracy of our approach are demonstrated by experimental results from 64 human MR images.

I. Introduction

Automatic segmentation of the left atrium (LA) from MR images is a challenging but major task in medical imaging analysis. An important application is concerned with the treatment of left atrial fibrillation [1]. Atrial fibrillation is a cardiac arrhythmia characterized by unsynchronized electrical activity in the atrial chambers of the heart. One of the treatments for such arrhythmia is the catheter ablation, which targets specific parts of the LA for radio-frequency ablation using an intracardiac catheter [2]. Application of radio-frequency energy to the cardiac tissue causes thermal injury, which in turn results into scar tissue. Successful ablation can eliminate, or isolate, the problematic sources of electrical activity and effectively cure atrial fibrillation. In order to perform such ablation, the extraction of the LA from the late gadolinium enhancement MR (LGE-MR) images is required and is often performed manually, which is a very time-consuming task. On the other hand, automatic LA segmentation is attractive but challenging due to the following factors: 1) the LA size is relatively small as compared to the left ventricle (LV) or lungs in cardiac MR images; 2) boundaries are not clearly defined when the blood pool goes into the pulmonary veins from the LA; 3) the shape variability of the LA is large across subjects.

A. Related Work

In the literature, heart segmentation has been investigated for decades, of which the main focus has been on extracting the ventricles [3], [4]. Advances have been made in whole heart segmentation from CT and MR images [5]–[8]. Heart models are used in many of the state-of-the-art methods for segmentation. In general, the anatomical structure of heart is modeled either by a triangulated surface or labeled images. A cost functional is commonly defined driven by the model parameters and image content. Once the model is globally localized, the cost function is optimized by deforming the model to match image information. These methods achieve high overall accuracy since the whole heart is modeled so that the components with strong features, e.g., the LV, help to reduce possible mis-alignments.

However, there has not been as much work reported for the automatic segmentation of the LA from MR images. Some nice research aimed at LA segmentation was focused on MR Angiography (MRA) data [9]–[11], where the LA blood pool is highlighted. The geometric characteristics of the heart chambers are used [9], [10] to identify the LA on the blood pool surface using a divide-and-merge strategy. In particular, the surface is decomposed into different subdivisions based on the observation that neighboring heart chambers can be separated by cutting the locations where the blood pool narrows. The challenges these surface-decomposition-based techniques face include the wide variability of intensity distributions and of the anatomic structures of the LA across subjects. To capture a patient specific LA from C-arm CT images, the LA chamber and pulmonary veins are modeled separately as parts of an LA model [12].

In contrast to surface-based modeling, atlas-based methods consider the LA segmentation from another perspective by using transformations found after registering training images to a target image to align the labels of the training images for segmentation. Different strategies, e.g., weighted voting [11], are used to build a consistent solution for final segmentation. The initial registered atlases can also be refined by using a region-based active contor model [13]. Another common way of representing an anatomical surface is by learning a subspace of the shapes using the principal component analysis (PCA) technique. In [14], the shape space of the LA is defined as a linear combination of eigen-shapes returned by applying the PCA to labeled training images. Then, segmentation is accomplished via evolving active contors in the shape space.

One common feature shared with most of the methods described so far is that they start with some global localization and then capture local details. This strategy works well when global features of a given object are well-defined, but may not hold for small structures such as the LA in an MR image, since the contribution to a cost functional may be overwhelmed by large structures such as the LV or strong artifacts associated with image quality. Instead of starting globally, region growing provides another perspective to the problem that begins locally to capture the entire target. This local property makes it more adaptive to variations of the dataset. In region-growing approaches [15], starting from seed regions, voxels neighboring to a given voxel are merged according to an aggregation criterion. Image intensity homogeneity is a widely used criterion in a growing process. For example, the statistics of intensities in a region are used to evaluate the homogeneity for region growing [16], [17]. The preliminary results reported in [18] shows promising results of segmenting the LA using the region-growing-based framework. Shape priors can be naturally incorporated into a growing process to segment complex objects. Typically, these priors are defined as some distance between reference and observed shapes. A point-wise metric such as the signed distance function between two shapes may be utilized with the prior information [19], [20]. To this end, the reference shape needs to be aligned to the observed shape before computing the shape distance. A more abstract way of defining shape distance employs shape moments, which removes the requirement of aligning shapes for the distance computation. Legender moments has been proposed to represent a shape prior for 2D images [21]. The results in [22] demonstrate the advantages of using the Zernike moments [23] in 2D image recognition over other moments-based priors in terms of accuracy and robustness. The generalization to the 3D case was reported in [24], which has been applied to 3D shape retrieval [25].

The purpose of this work is to develop an automatic segmentation technique that is robust to the large variability of the LA datasets. To this end, a variational region-growing framework was adopted which starts locally by searching for a seed region of the LA from a slice of an MR image. To prevent leakage in the growing process, a shape prior represented by the Zernike moments is employed. The overall process is fully automatic and easy to implement.

B. Contributions

The main contributions of the present work are as follows:

1. In seed region extraction, we present an effective way to locate the LA that is not sensitive to changes of volume coverage, intensity distribution, and atrial shapes.

2. We formulate the problem of LA segmentation in a variational region-growing framework, incorporating a shape prior that is invariant to rotation, translation, and scaling. The advantage of this formulation is that it explicitly separates segmentation from registration when applying a shape prior, rather than needing to optimize over additional pose parameters.

3. Instead of fixing the weights while including prior information, the proposed method adaptively adjusts the weights between the data-driven term and shape prior constraint. The rationale is that, when the volume of a region is very small, the data-driven force term plays the main role; as the volume increases, the shape prior force term gradually dominates the overall process.

The remainder of this paper is organized as follows. In Section II, we describe the proposed region-growing method, and show how to locate a proper seed region and formulate the LA segmentation problem using a variational framework with a shape prior. In Section III, we provide some experimental results elucidating the proposed methodology. Finally, in Section IV, we draw some conclusions and outline some future research directions.

II. Proposed Method

The proposed method consists of two key steps: (1) search for a seed region of the LA from an MR slice in the axial view. (2) explore the LA region using a variational region-growing process. A shape prior is employed to drive the growing process towards atrium-like shapes.

A. Seed Region Extraction

In region growing, it is common to initialize the seed region close to the center of the target to be segmented. Suppose an MR image is oriented along the axial plane. Then, as shown in Figure 1, two features may be utilized to facilitate this task: i) partial boundaries exist that distinguish the LA from other heart chambers, e.g., the mitral valve (MV) between the LV and LA (Figure 1 (b), (c), and (e)); ii) the spatial relation between the LV and LA in the axial view may imply how close a slice is to the center of mass of the LA, which we call the “centrality” of a slice with respect to the center of mass of the LA (see Figure 1 (d)~(f)).

Fig. 1.

(a) Diagram of the heart, including the left ventricle (LV), right ventricle (RV), left atrium (LA), right atrium (RA), mitral valve (MV), tricuspid valve (TV), and pulmonary veins (PVs). (b)~(c) Heart regions from two subjects with the LV and LA highlighted. (d)~(e) The spatial relation between the LA and LV in the axial view as tracing from the bottom of the LA (d), to the middle slice of the MR image (e), and the center of the LA (f).

Suppose the world coordinates XYZ trace from right to left, posterior to anterior, and inferior to superior, in which the origin of XY plane is defined as the bottom left of a slice. Denote the LA and LV regions extracted from a slice I_i as I_LA and I_LV, respectively. Then, the centrality of the slice I_i is approximated by

$$g\left(I_i\right)=\frac{A\left(I_{\mathrm{LV}}\right)}{A\left(I_{\mathrm{LA}}\right)}\underset{p\in I_{\mathrm{LV}}}{\max }d(p,I_{\mathrm{LA}}),$$ (1)

where A(·) is the area of a region, and d(p, I_LA) is the Euclidean distance from a point p to I_LA. The first part of this equation evaluates the significance of the LV with respect to the LA, and the second part measures the greatest distance from the LV region to the LA region. Figure 2 shows the values of g for the three different cardiac MR images in Figure 1 (b)~(d), where the LV and the LA are completely covered in the Z direction (about 110 mm). This indicates that a centralized slice has a relatively small g value and the central slices are close to their middle slices as well. Based on these observations, we propose an effective method for detecting a seed region inside the LA. It begins from the middle slice I_MID of an MR image and searches for the seed region in a specific range around I_MID (10 mm below and above) along the Z direction. On each slice, a seed region is detected, which is utilized to find approximations for the LA and the LV regions to measure the slice centrality g. The search process stops when termination conditions are satisfied. For MR images with wider coverages along the Z direction, g may be utilized as well to find a central slice by changing the initial search slice accordingly. The overview of the seed detection process is summarized in Algorithm 1. The key steps are described in the following sections.

Fig. 2.

Examples of the centrality for the three MR images as shown in Fig. 1(b), (c), and (d), as the MR slices tracing from bottom to up in the Z direction. The indices of central slice are marked by circles.

Algorithm 1.

Seed Region Extraction

1: Initialize Ii as the middle slice

2: Segment the heart region Iheart on Ii

3: Extract a seed region inside Iheart

4: Evaluate the centrality g(Ii)

5: If g(IMID) is above a threshold tg, go to a upper slice and repeat steps 2-4 until g(Ii) < tg; otherwise, go to a lower slice and repeat steps 2-4 until g(Ii) > tg. Both cases are terminated when Ii is out of the search range

6: Once the slice is centralized, adjust the seed radius

1) Heart Region Detection

To evaluate the centrality g of a slice I_i, the heart region that contains the LA and the LV needs to be segmented. As a pre-processing step, the low intensity background outside of the chest is removed by using the Ostu's method [26], which automatically determines the best separation of an image histogram. The largest bounding box of the foreground pixels is set as the region of interest (ROI) for further processing. Since the heart region has higher intensity values than its surrounding structures (such as the lungs), a coarse segmentation is first extracted via thresholding, which is then refined by utilizing a localized region-based active contor model [13], [27] due to the complexity of the heart region being viewed from an MR slice where global-based segmentation methods [28] may fail.

The threshold for a coarse segmentation is determined as

$$T_{heart}=\min \{T_{\text{global}},T_{\text{local}}\},$$ (2)

where Tglobal is the intensity threshold returned by applying the Ostu's method again inside the ROI and Tlocal is the mean intensity inside the LA mask obtained from training images (see Section III-A). This thresholding strategy reduces the possibility of missing the heart region while only the global threshold Tglobal is used when the liver or thoracic wall has very high intensities. After applying a morphological opening operator to remove noisy structures, we get a set of connected components L = {L_k}, k = 1, . . . , N_con, where N_con is the number of the components. The heart region is chosen as L_k* such that

$$k^{\ast }=\underset{0\le k\le N_{\mathrm{con}}}{\max }\frac{A\left(L_k\right)}{d(L_k,C_{\mathrm{ROI}})},$$ (3)

where A(L_k) is the area of component L_k, and d(L_k, C_ROI) is the Euclidean distance from the center of component L_k to the center of the ROI, denoted by C_ROI. This measurement favors large regions with small distance to the center of the ROI.

The localized region-based active contor energy functional [13], [27] used for refinement is defined as

$$E\left(\varphi \right)={\int }_{{\Omega }_x}\delta \left(\varphi \left(x\right)\right){\int }_{{\Omega }_y}B(x,y)F(I\left(y\right),\varphi \left(y\right))dydx+{\lambda }_{\mathrm{LG}}{\int }_{{\Omega }_x}\delta \left(\varphi \left(x\right)\right)\mid \nabla \varphi \left(x\right)\mid dx,$$ (4)

where δ(·) is the Dirac function, ϕ is the signed distance function of a contor defined over the image domain $\Omega \in {\mathbb{R}}^2$ with positive distances inside and negative distances otherwise. B(x, y) is a ball of radius R_LG centered at x. F(I (y), ϕ(y)) is a generic internal energy measure used to represent local adherence to a given model at each point along the contor. In this work, a localized Chan-Vese energy [13], [28] is used. The term weighted by λ_LG is a regularization term that measures the arclength of the contor.

2) Seed Region Extraction

Denote the coordinates of a point p in slice I_i as (p¹, p²). As illustrated in Figure 1, the LA lies at the central bottom of the heart region. To narrow down the search space, the seed region is explored within the detected heart region Iheart in the following manner. To begin with, the magnitude of the gradient inside the ROI is computed, which is then thresholded by using its mean gradient. After that, an edge map I_edge of this binarized image is obtained by applying a binary thinning operation. Next, a distance field image I_dist is constructed by performing distance transform [29] on the edge image. The distance values outside Iheart are set as zero. Finally, the location of a seed region is detected by searching for a circular region S_LA(p*, r) of a fixed radius r centered at point p* such that

$$p^{\ast }=\underset{p\in I_{\text{heart}}}{\max }\frac{{\int }_{B(p,r)}{I}_{\mathrm{dist}}\left(p\right)dA}{d(p,P_{\mathrm{ref}})},$$ (5)

where B(p, r) is a ball of fixed radius r centered at point p, and d(p, P_ref) is the Euclidean distance from p to a reference point P_ref. Denote the set of points inside I_heart as $\left\{(p_i^1,p_i^2)\right\}$, i = 1, . . ., N_ht, where N_ht is the cardinality of the set. The components of the reference point $(p_{\mathrm{ref}}^1,p_{\mathrm{ref}}^2)$ are determined as

$$\{\begin{array}{c}{p}_{\mathrm{ref}}^1=\mathrm{mean}\left\{p_i^1\right\}\hfill \\ p_{\mathrm{ref}}^2=\min \left\{p_i^2\right\}+t_{\mathrm{ref}},\hfill \end{array}\phantom{\}}$$ (6)

where t_ref is a constant to prevent the center of S_LA(p*, r) being attracted to heart boundary.

As can be seen from equ. (6), the reference point actually lies vertically below the center of I_heart, which is closer to the LA than to any other structures inside I_heart. The radius of S_LA(p*, r) is set so that the circular region is completely contained in the LA region, which is empirically set as r = 10 mm.

Figure 3 illustrates the process of detecting a seed region. Note that, although there does not exist closed contor that identifies the LA, partial edges, e.g., the mitral valve, were detected. Thus, a distinct cluster containing the LA was formed on each of the distance maps. Seed regions were successfully identified using the measurement defined in eq. (5).

Fig. 3.

First row from left to right: middle slices from three MR images. Second to fourth row from left to right: the corresponding heart region (yellow), edges inside I_heart, distance map (the ’jet’ colormap was used where blue is zero, the smallest value), and detected seed region with the same fixed radius r = 10 mm (red).

3) Slice Centralization

To check whether the current slice I_i is approximately centralized with respect to the LA, the appropriate areas of the LA and LV need to be determined. To this end, a morphological opening operator is performed on the distance image I_dist to isolate the LA region from other structures, which returns a group of connected regions. The one containing S_LA is selected as the mask of the LA. In this step, the size of the structure element in the opening operation needs to be smaller than the largest value of I_dist, denoted by d_max, but large enough to disconnect the LA from other structures. Thus, this value is empirically set as 0.8d_max.

The determination of the LV region is based on the fact that it resides to the top right of LA as shown in Figure 1. Thus, the center of a seed region p* is used as a reference origin, with respect to which to divide the top right quadrant equally into two sections. In each of these two sections, the closest point between the boundary of I_heart and p* is used as a start/end point for the LV boundary. Connecting these two points to p* gives a close contor for the LV region and the points inside I_LA are removed to get an approximation of I_LV. Figure 4 shows the extracted LV and LA regions for the three slices in Figure 3.

Fig. 4.

Extracted LV (green) and LA (blue) for the three slices in Fig. 3.

The threshold t_g used to guide the centralization process is obtained from training images (see Section III-B.1). Once a centralized slice is found, its seed radius is adjusted such that r* = max_r{I_dist(q) > 0, ∀q ∈ S_LA(p*,r)}.

B. Variational Region Growing With a Shape Prior

Given a properly determined seed region, the robust statistics [30] of intensities inside the region may be employed to explore the entire LA. However, evolution leakage is almost inevitable because the computation of the statistics does not provide a global shape constraint on evolving contors. Adding a regularization term can prevent leakage, but may fail to capture finer details of the LA. Hence, a shape prior is applied to attract the growing process towards an expected shape. The details of how to integrate this prior information into the growing process is described in the following sections.

1) Variational Region Growing

The robust statistics have been integrated into a generic variational region-growing framework for image segmentation [17]. Robust statistics of intensities are evaluated to measure the homogeneity among neighboring voxels. To this end, a feature vector is defined for each voxel $\mathit{x}=(x,y,z)\in {\mathbb{R}}^3$ in an image as

$$f\left(\mathit{x}\right)=(MED\left(\mathit{x}\right),IQR\left(\mathit{x}\right),MAD\left(\mathit{x}\right))\in {\mathbb{R}}^3,$$ (7)

where M E D(x), I Q R(x), and M A D(x) are the intensity median, inter-quartile range, and median absolute deviation around a neighborhood of x, respectively. Denote the label map of a seed region as G. Then, the seed region is characterized by the probability density function of the feature vectors as

$$p\left(\mathit{f}\right)=\frac{1}{\mid G\mid }\sum _{x\in G}{K}_h(\mathit{f}-\mathit{f}\left(\mathit{x}\right)),$$ (8)

where K is the kernel function and |G| is the cardinality of G. A Gaussian kernel is used in this work, the variance of which is chosen to be h times the MAD of the seed region. The region-growing energy functional is defined as

$$E_{\mathrm{RS}}\left(\varphi \right)={\int }_{\Omega }-p\left(\mathit{f}\left(\mathit{x}\right)\right)H\left(\varphi \left(\mathit{x}\right)\right)d\mathit{x},$$ (9)

which evaluates the intensity homogeneity inside a contor where ϕ > 0, and H is the Heaviside function. The value of h controls the intensity homogeneity favored by the growing process. The larger the value of h is, the more homogeneous regions the growing process tries to explore. A regularization term is typically used, while applying this energy for segmentation, to balance the growing force so that the evolution stops at an expected location.

2) Zernike Moments as Shape Prior

The Zernike moments are many times used as a shape descriptor because of their invariance to rotation. The 3D Zernike moments of an object defined by I(x) are

$${\lambda }_{nl}^m=\frac{3}{4\pi }{\int }_{\mid \mathit{x}\mid \le 1}I\left(\mathit{x}\right)\overline{Z_{nl}^m\left(\mathit{x}\right)}d\mathit{x},$$ (10)

where $\overline{Z_{nl}^m\left(x\right)}$ is the complex conjugate of the 3D Zernike functions

$$Z_{nl}^m\left(\mathit{x}\right)=R_{nl}\left(\tilde{r}\right)Y_l^m(\tilde{\vartheta },\tilde{\psi }).$$ (11)

R_nl is a radial function and $Y_l^m$ is the spherical harmonics, where n is the degree of polynomial in the components of $\mathit{x}=\tilde{r}(\sin \tilde{\vartheta }\cos \psi ,sin\tilde{\vartheta }\sin \tilde{\psi },\cos \tilde{\psi })$, and m ∈ [–l, l], l ∈ [0, n] such that n – l is an even number. Denote the component of a point x as (x, y, z). As shown in [25], eq. (10) can be rewritten as a linear combination of geometric moments

$${\lambda }_{nl}^m=\frac{3}{4\pi }\sum _{r+s+t\le n}\overline{{\chi }_{nlm}^{rst}}{M}_{rst},$$ (12)

where ${\chi }_{nlm}^{rst}$ is the complex coefficient of the monomial x^r y^sz^t when $Z_{nl}^m\left(\mathit{x}\right)$ is expanded as a linear combination of monomials, and M_rst is the geometric moments of order r + s + t computed inside a unit ball. A rotationally invariant 3D Zernike descriptor is obtained by taking the norms of vectors ${\lambda }_{nl}=({\lambda }_{nl}^{-l},{\lambda }_{nl}^{-l+1},\dots ,{\lambda }_{nl}^l)$ to get rid of phase terms. The invariance to translation and scale is achieved by computing the normalized central moments η_rst for an object. The normalized central moments are defined as

$${\eta }_{rst}={\int }_{\Omega }\frac{{(x-\stackrel{‒}{x})}^r{(y-\stackrel{‒}{y})}^s{(z-\stackrel{‒}{z})}^t}{{\left(\rho K_0\right)}^{(r+s+t+3)∕3}}H\left(\varphi \right)d\mathit{x},$$ (13)

where (x̄ , ȳ, z̄) is the center of mass of the object, $K_0={\int }_{\Omega }H\left(\varphi \right)d\mathit{x}$ is the volume of the object, and ρ is a scale factor so that the object is mapped into a unit ball by ((x – x̄)/(ρK₀)^1/3, (y – ȳ)/(ρK₀)^1/3, (z – z̄)/(ρK₀)^1/3). That is, as in [25], an object is first transformed to its center of mass and then rescaled into the unit ball before computing the geometric moments.

The proposed shape prior defined between an evolving shape, implicitly represented by ϕ, and a reference shape in terms of the Zernike moments is

$$J\left(\varphi \right)=d(\lambda ,{\lambda }^{ref})=\sum _{nl}{({\Vert {\lambda }_{nl}\Vert }^2-{\Vert {\lambda }_{nl}^{ref}\Vert }^2)}^2.$$ (14)

By using the chain rule, the variational derivative of J(ϕ) with respect to ϕ is

$$\begin{array}{cc}\hfill \frac{\delta J\left(\varphi \right)}{\delta \varphi }=& 2\sum _{nl}({\Vert {\lambda }_{nl}\Vert }^2-{\Vert {\lambda }_{nl}^{ref}\Vert }^2)\frac{\delta {\Vert {\lambda }_{nl}\Vert }^2}{\delta \varphi }\hfill \\ \hfill =& 2\sum _{nl}({\Vert {\lambda }_{nl}\Vert }^2-{\Vert {\lambda }_{nl}^{ref}\Vert }^2)\frac{\delta }{\delta \varphi }\sum _{-l\le m\le l}{\lambda }_{nl}^n\overline{{\lambda }_{nl}^m}\hfill \\ \hfill =& 4\sum _{nl}({\Vert {\lambda }_{nl}\Vert }^2-{\Vert {\lambda }_{nl}^{ref}\Vert }^2)\sum _{-l\le m\le l}\mathrm{Re}\left\{\overline{{\lambda }_{nl}^m}\frac{\delta {\lambda }_{nl}^m}{\delta \varphi }\right\}\hfill \\ \hfill =& \frac{3}{\pi }\sum _{nl}({\Vert {\lambda }_{nl}\Vert }^2-{\Vert {\lambda }_{nl}^{ref}\Vert }^2)\times \sum _{-l\le m\le l}\mathrm{Re}\left\{\overline{{\lambda }_{nl}^m}\sum _{r+s+t\le n}\overline{{\chi }_{nlm}^{rst}}\frac{\delta }{\delta \varphi }{\eta }_{rst}\right\},\hfill \end{array}$$ (15)

where n ∈ [0, N_ZM], l ∈ [0, n such that n – l be an even number, which N_ZM is the order of the Zernike moments for a given shape. The variational derivative of η_rst with respect to ϕ is given in the Appendix.

Gradient descent is employed to minimize J(ϕ). An example of deforming two canonical geometric shapes, i.e., a sphere and a hyperbola, to minimize the shape distance defined in eq. (14) is shown in Figure 5. Starting from those two significantly different shapes, the evolution process deforms these shapes towards an LA shape prior of order 12. This shows that the Zernike moments are powerful and suitable for representing the LA shapes.

Fig. 5.

Deformation of sphere and hyperbola to a given LA shape. The moments order is 12.

3) Region Growing With a Shape Prior

Combining the region-growing energy eq. (9) with the shape prior eq. (14), the energy functional to be minimized for the LA segmentation is defined as

$$E\left(\varphi \right)=f\left(V_j\right)E_{\mathrm{RS}}\left(\varphi \right)+(1-f\left(V_j\right)-\gamma )J\left(\varphi \right)+\gamma {\int }_{\Omega }\delta \left(\varphi \left(\mathit{x}\right)\right)\mid \nabla \varphi \left(\mathit{x}\right)\mid d\mathit{x},$$ (16)

where the term weighted by γ is a regularization term that measures the surface area of the LA. The function f(V_j) adjusts the weight between E_RS(ϕ) and J(ϕ) based on the volume of the evolving region V_j so that the robust statistic term dominates the growing process initially and its role decreases as the volume of the region increases. Since a seed radius is related to the volume of the LA to be segmented, the function f(V_j) is defined as

$$f\left(V_j\right)=\frac{1-\gamma }{1+e^{\alpha (V_j-({\mu }_V+\left(\beta {\sigma }_V\right)))}},$$ (17)

where μ_V and σ_V are the mean and standard deviation of the LA volumes from training images, respectively. β is introduced to reflect the relation between a seed radius and the volume of the LA to be segmented, which is defined as

$$\beta =sign(r^{\ast }-{\mu }_r)\lfloor \frac{\mid r^{\ast }-{\mu }_r\mid }{{\sigma }_r}\rfloor ,$$ (18)

where μ_r and σ_r are the mean and standard deviation of the radii of seed regions learned from the training images (see Section III-A). The use of the floor function ⌊·⌋ gives the flexibility to radius variation, i.e., for seed regions of radii within a range of σ_r have the same impact on β. As can be seen from eq. (17), f(V_j) → 1 – γ when $V_j\ll \mu v$ and f(V_j) → 0 otherwise as expected.

Starting from a given seed region, which is relatively small as compared to the average LA size, a small value of h defined in E_RS(ϕ) is preferred so that the growing force is strong enough to capture the whole LA region. A large h value imposes a high homogeneous constraint on the overall growing process, which may cause the process to stop before reaching the LA boundary. We found that h = 0.1 works well in all of our experiments.

Note that V_j = K₀, and thus the gradient descent flow of eq. (16) is

$$\begin{array}{cc}\hfill \frac{\partial \varphi }{\partial t}\left(\mathit{x}\right)=& -f^{\prime }\left(K_0\right)\delta \left(\varphi \left(\mathit{x}\right)\right)E_{\mathrm{RS}}+f\left(K_0\right)\delta \left(\varphi \left(\mathit{x}\right)\right)p\left(\mathit{f}\left(\mathit{x}\right)\right)\hfill \\ \hfill & +f^{\prime }\left(K_0\right)\delta \left(\varphi \left(\mathit{x}\right)\right)J\left(\varphi \left(\mathit{x}\right)\right)-(1-f\left(K_0\right)-\gamma )\frac{\delta J\left(\varphi \left(\mathit{x}\right)\right)}{\delta \left(\varphi \left(\mathit{x}\right)\right)}\hfill \\ \hfill & +\gamma \mid \nabla \varphi \left(\mathit{x}\right)\mid \nabla \cdot \left(\frac{\nabla \varphi \left(\mathit{x}\right)}{\mid \nabla \varphi \left(\mathit{x}\right)\mid }\right).\hfill \end{array}$$ (19)

The last term is the curvature at point x, which smooths evolving contors.

C. Implementation

Numerically, the sparse level set method [31] was employed for the computer implementation of the variational region-growing model because of its efficiency. In particular, the upwind scheme was used in discretizing |∇(·)| in eq. (19) (see [32] for details).

III. Experiments and Results

A. Prior Learning

There were 16 Delayed Enhanced MR images (DE-MRI) used for learning prior information and tuning parameters. These training data cover 110 mm in the Z direction, in which the LA has wide shape variations and volume sizes (ranging from 66 to 237 ml). Manual segmentations are available for each image.

The LA mask for determing the local threshold T_local may be obtained in the following manner. First, for each centralized slice from the training image, a ROI is detected as described in Section II-A.1. Then, the mask of its manual segmentation inside the ROI is normalized to [–1, 1]×[–1, 1] by setting the origin at the center of the ROI. Finally, all normalized masks are summed up and rescaled to [0, 1] as the spatial distribution of the LA (see Figure 6(a)). The points with values greater than 0.5 were chosen as the LA mask. When applying this mask to an MR slice to be segmented, it is rescaled to fit into the ROI of the slice (see Figure 6(b)). Normalizing coordinates can compensate the differences of volume coverages when applying the LA mask.

Fig. 6.

(a) Spatial distribution of the LA in the normalized coordinate. (b) The ROI (yellow) of the middle slice of an MR image and applied LA mask (red).

The scaling factor ρ in eq. (13) was computed as

$$\rho =\underset{1\le i\le 16}{\max }\frac{R_i^3}{V_i},$$ (20)

where R_i is the greatest distance between points of an LA and its center of mass, V_i is its volume. ρ = 31.3 was found from the training images. When applying this scale factor to new images, points outside the unit ball determined by ρ were set to zero.

To determine the parameters in eq. (17), the seed detection process was performed on each central slice. The statistics of the final seed radii were μ_r = 11.6 mm, σ_r = 3.1 mm. The average and standard deviation of manual segmentation volumes were μ_V = 106 ml and σ_V 45 ml, respectively. To compute the Zernike moments for=the LA, each manual segmentation was used individually, and then these moments were averaged componentwise as the shape prior. In the experiments, the order of the moments was set to be N_ZM = 12.

B. Parameter Determination and Sensitivity Test

1) Parameters in Seed Extraction

All the central slices from the training images were used to determine the parameters for seed extraction. All of these slices have a resolution of 0.625 × 0.625 mm². Specifically, the following three measures were used for the parameter determination and sensitivity test: i) the average distance between the centers of seed regions and their corresponding centers of manual segmentations, denoted by d̄(C_Seed, C_LA); ii) the average radius of seed regions from the training images, denoted as $\overline{r^{\ast }}$; iii) the average distance between the centralities from manual segmentations (g̃) and proposed method (g), denoted by d̄(g, g̃).

The parameters λ_LG and R_LG in eq. (4) and t_ref in eq. (6) were determined as follows: First, fix an empirical value t_ref = 50 pixels, then sweep the parameters λ_LG and R_LG in the range [0.2, 0.8] × [10, 20] to search for the minimum of d̄(g, g̃), since heart region detection may affect the value of g. The optimal values were found to be λ_LG = 0.6 and R_LG = 15 pixels (9.38 mm) and the threshold of t_g = 10 (6.25 mm) was chosen accordingly. Then, fix λ_LG and R_LG, and search for t_ref in the range from 2 to 120 pixels that minimizes d̄(C_Seed, C_LA), because the seed location directly depends upon t_ref. The optimal value of t_ref was found to be 58 pixels (36.25 mm).

The sensitivity of these three parameters to seed detection were examined by varying one parameter in a given range while keeping the other two fixed. As an example, the impact of t_ref on seed location and radius as it ranges from 2 to 120 pixels is shown in Figure 7. Initially, d̄(C_Seed, C_LA) is very large but $\overline{r^{\ast }}$ is small, which means seed regions were all attracted close to the heart boundary. When t_ref goes to infinity, i.e., eq. (5) is not normalized by the distance to the reference point, d̄(C_Seed, C_LA) is relatively large, which means some seed regions may not reside in the LA region. As can be seen from Figure 7, there is stable region (when t_ref varies from 58 to 106 pixels) such that the seed locations and radii are not sensitive to t_ref. The coefficient of variation [33], defined as the ratio of the standard deviation to the mean, was used to evaluate the sensitivity of parameters to seed location, radius, and slice centrality. The details of this test are summarized in Table I. It shows that the seed location and radius are controlled by t_ref, while the slice centrality is mostly affected by λ_LG and R_LG. Also, these results indicate that the radii of seed regions are not sensitive to parameters in the given range.

Fig. 7.

Impact of t_ref on d̄(C_Seed, C_LA) (Left) and $\overline{r^{\ast }}$ (Right).

TABLE I.

Coefficient of Variation for Parameters in Seed Extraction

Parameters	Range	d̄(CSeed, CLA)	$\overline{r^{\ast }}$	d̄(g,g̃)
λ LG	[0.2, 0.8]	4.0%	1.0%	20.6%
R LG	[10, 20]	5.5%	1.1%	16.0%
t ref	[58, 106]	12.5%	1.4%	1.0%

2) Parameters in Region Growing

Due to the computational complexity in implementing the Zernike moments, the training data were resampled to 2 × 2 × 2 mm³ when tuning parameters, where the LA shape is×well-preserved. The average of the Dice coefficient [34] between manual and automatic segmentations was used to determine parameters and test sensitivity of the region-growing process. The two parameters α and γ defined in eq. (16) and (17) were determined by first empirically setting α = 0.01 and searching for γ in the range [0.1, 0.9] that maximizes the average Dice coefficient. Then, similarly, by fixing γ, we searched for α in the range [0.01, 0.1]. The optimal parameters were found to be γ = 0.3 and α = 0.03. The coefficient of variation for the average Dice is 3% as α varies from 0.01 to 0.1 by fixing γ = 0.3, which means that α is robust in this range. The sensitivity of γ will be discussed in the next section.

The sensitivity of seed perturbations to the LA segmentation was tested by translating and scaling each seed region and comparing the changes of the average Dice values. For translations, random tests were conducted to shift each seed region from [–10%, 10%] to [–90%, 90%] of its radius r in the XY plane and along the Z direction separately. Regarding scale, each seed region was randomly shrunk from 3% to 30% of its radius (about the range of μ_r/σ_r = 27%, which is far larger than the sensitivity of parameters to seed radius (<2% as shown in Table I)). Each case was randomly tested for 10 times and the average Dice coefficient was computed. The testing results are given in Figure 8. Overall, perturbations in translation is less sensitive than scaling to final segmentations, while perturbing the seed region in XY plane affects segmentation more than along the Z direction. In the scaling test, a sharp decrease of Dice happens after 18% of shrinking as a smaller seed region may contain insufficient intensity information for capturing the whole LA region.

Fig. 8.

Variation of the average Dice coefficient as seed regions perturbed in translation (Left) and scaling (Right).

3) Importance of Prior Information

Figure 9 shows an example of the region-growing process, in which Figure 9(f) is the segmentation result without using the prior information by setting the coefficient of the prior to zero. It shows that the process explores homogeneous regions at the beginning when the robust statistic force is dominant, and gradually captures finer details when the shape prior starts to play a main role. The evolution leakage was avoided by using the shape prior information. A common practice to prevent evolution leakage in active contor models is by increasing the weight of the curvature term. Thus, a comparison was made between segmentations with and without using the prior information under varying γ from 0.1 to 0.9. The overall performance is shown in Figure 10. It was observed that, without using the prior, adding more weight to the curvature term reduces evolution leakage and thus improves segmentations up to γ = 0.8. Afterwards, the performance decreases as the curvature term causes under-segmentations since it favors smooth surfaces. Better segmentations may be achieved by tuning γ case by case. On the contrary, the curvature term had less effect when the shape prior was used which competes with the smoothing force and favors LA-like shapes.

Fig. 9.

Example of region-growing process. (a) Raw image. (b) to (e) The growing process with the shape prior. (f) Segmentation without using the shape prior. Manual segmentation is highlighted in red.

Fig. 10.

Comparison of the LA segmentation with and without using the prior information.

4) Moments Order and Computational Complexity

The proposed method was implemented in C++ and tested on a computer with Quad CPU 3GHz, 8G RAM. In this test, the training images were resampled to a resolution of 1×1×1 mm³ to test the time complexity for higher resolution segmentation. The parameters learned from training images at lower resolution were applied. Figure 11 shows the effect of moments order on segmentation accuracy and computational complexity. As can be seen from this figure, there is a slight improvement (about 3%) as the moments order goes from 10 to 20. On the other hand, the average computational time per image increases sharply by 30 times as moments order goes to 20. In applications, we found the moments order of 12 gives a good balance of time and accuracy.

Fig. 11.

Left: Effect of moments order on segmentation accuracy in terms of average Dice. Right: Average computational time per image (in minute).

C. Experiments With Cardiac MR Data

We evaluated the proposed segmentation method using 64 MR images from the Comprehensive Arrhythmia Research and Management (CARMA) Center that are publicly accessible at [35]. These data are Late-Gadolium Enhanced (LGE-MRI) from 32 patients with atrial fibrillation (AF) who have undergone radiofrequency (RF) catheter ablation for treatment of AF, of which 32 are images before RF ablation and others are after. Enhancement is hypothesized to fibrosis or endocardial tissue that is correlated with the progression of AF for the pre-ablation images, and that of post-ablation images is due mainly to the scars. Thus, these data have large variations in terms of LA shape, intensity range, and image quality.

In the experiments, the parameters learned from the training images were directly applied. Starting from the middle slice, the proposed method succeeded in finding the seed regions for all of the 64 images in presence of large intensity variations and noises, which provides good initialization for the following region growing process. After the slice centralization search, the average difference between the indices of automatic and manually determined central slices was 1.4, which was 3.1 as measured from middle slices. An example of the final segmentation results is shown in Figure 12 for two LAs with significantly different shapes and volume coverages. The final segmentations capture the overall LA structure, only missing parts of the pulmonary veins (PVs).

Fig. 12.

Segmentations of the LA with sharp shape variations and wide volume coverages (second and fourth columns). From top to bottom: the LA in axial, coronal, and sagittal views, respectively.

In addition, the proposed method was compared to the standard multi-atlas-based method [36], [37]. Let $\left\{I_{tr}^i\right\}$ and $\left\{L_{tr}^i\right\}$, i = 1 · · · 16 be a set of label images, respectively. The multi-atlas method works as follows: First, each training and images $I_{tr}^i$ is registered to a testing image I_j to be segmented by searching for an optimal affine transformation $T_{opt}^i$ using mutual information [38], [39] as a metric. Then, the corresponding label image $L_{tr}^i$ is transformed using $T_{opt}^i$. Let ${\tilde{T}}_{tr}^i$ and ${\tilde{L}}_{tr}^i$ be the transformed training and label images, respectively. Next, the non-zero region of ${\tilde{L}}_{tr}^i$ is selected as a ROI and refined by registering the ${\tilde{T}}_{tr}^i$ to I_j using the B-Spline registration method [40]. After registering all the training images to I_j, we get a set of transformed label images. Finally, these label images are combined by applying a majority voting method [41] to get the segmentation of the LA.

To quantify errors, the standard Dice coefficient [34], volume overlap [42], surface distance from the automatic to manual segmentations, and Hausdorff distance [43] (95% Hausdorff distance [44]) were used. The comparison results are summarized in Table II. Overall, the proposed method achieves better results in terms of accuracy and the worst cases. A case-by-case comparison in terms of Dice and volume overlap is shown in Figure 13 where only the results from the subjects with even ID indices are presented to save displaying space. These detailed results show that the proposed method outperformed the atlas-based method in all of the 32 cases in terms of the Dice measurement, and in 30 out of 32 cases (other than subjects 4 and 46) with respect to the volume overlap measurement.

TABLE II.

Comparison of Segmentation Accuracy Between the Proposed Method and Multi-Atlas-Based Method

Measurements	Proposed Method	Multi-Atalas
Dice [min]	0.79 ± 0.05 [0.67]	0.69 ± 0.11 [0.27]
Overlap [min]	0.65 ± 0.07 [0.50]	0.54 ± 0.13 [0.15]
Surface distance (mm)	2.79 ± 2.84	5.36 ± 4.60*
Hausdorff [95%] (mm)	14.40 ± 3.65 [8.39 ± 2.06]	23.10 ± 9.25* [13.01 ± 3.59]

^*Cases with Dice < 0.60 were regarded as completely failures and excluded from computation.

Fig. 13.

Comparison of segmentation accuracy in terms of Dice (Left) and Volume Overlap (Right) between the proposed method (green) and multi-atlas-based method (yellow).

In addition, the worst cases with respect to the Dice coefficient are given in Figure 14, which illustrates the characteristics of these two types of methods. The proposed method is more flexible to variations of the volume coverage and LA shapes because of its local characteristics, while under-segmentation may occur due to extensive noise where the information inside a seed region may not be able to characterize the intensities in the entire LA volume. In addition, because of the underlying smoothness requirement in variational derivations, the proposed method has lesser tendency to develop sharp changes over the shape, which results in missing some of the regions around the PVs. The atlas-based method suffers severely from changes in volume coverage since it is a global method. As shown in Figure 14, the existence of an outlier (an arm) on the left side of the MR image makes the final segmentation deviate away from the expected location, even though the overall segmentation has a LA-like shape.

Fig. 14.

Comparison of the worst results (in terms of the Dice measurement) obtained using the proposed method (second column) and the atlas-based method (fourth column). From top to bottom: the LA returned using the proposed method (yellow) and atlas-based method (green) in axial, coronal, and sagittal views, respectively.

Though it is hard to make a fair comparison between different methods because of the use of different datasets, the performance of our method seems to be competitive with some of the latest results. For example, one state-of-theart method [8] reports a result of 0.81 ± 0.10 [0.47] and 0.69 ± 0.12 [0.30] for the Dice and volume overlap measures, respectively. The referenced method achieves higher mean values since it uses a whole heart model that contains richer information while interpreting an MR image. On the other hand, the proposed method has a smaller standard deviation and better performance in the worst cases because of its localized property, which specifically focuses on the LA.

IV. Conclusion

We have described an automatic segmentation scheme for extracting the LA from MR images by using a variational region growing with a moments-based shape prior. The method starts from searching for a seed region of the LA by utilizing the anatomical characteristics of the LA. Then, it explores the LA region by employing locally computed robust statistics of seed intensities and a shape prior that enforces a global guidance. Thus, this method is robust to shape variations and image quality. The experimental results indicate the ability of the proposed approach to give very reasonable results.

As indicated by the experimental results, challenges arise with segmentations near the PVs and when the information in a circular seed can not characterize the whole LA region. The first issue may be solved by incorporating certain geometric information of the PVs with respect to the LA. For the second issue, we will investigate other seed shapes or extract seed regions from other image views. In applications, we will use the segmentation results as prior information to identify scars for MR images of post-ablations and apply the moments-based shape prior for segmenting other anatomical structures.

Biography

Liangjia Zhu is a Post-Doctoral Fellow with the Department of Electrical and Computer Engineering, University of Alabama at Birmingham, Birmingham, AL, USA. He received the Ph.D. degree in electrical engineering from the Georgia Institute of Technology, Atlanta, GA, USA, in 2013. His research interests include image processing, computer vision, and video analysis.

Yi Gao is an Assistant Professor with the Department of Electrical and Computer Engineering, University of Alabama at Birmingham (UAB), Birmingham, AL, USA. He is an Associate Scientist with the UAB Comprehensive Cancer Center. His research interests include computer vision, image, and shape computing.

Anthony Yezzi holds the position of Ken Byers Professor within the School of Electrical and Computer Engineering, Georgia Institute of Technology (Georgia Tech), Atlanta, GA, USA, where he directs the Laboratory for Computational Computer Vision. He has over 20 years of research experience in shape optimization via geometric partial differential equations. He received the Ph.D. degree in electrical engineering from the University of Minnesota, Minneapolis, MN, USA, in 1997, with a minor in mathematics. After completing a post-doctoral research appointment with the Massachusetts Institute of Technology, Cambridge, MA, USA, he joined the faculty of Georgia Tech in 1999. His research lies primarily within the fields of image processing and computer vision with a particular emphasis on medical imaging and 3D surface reconstruction. He has consulted for a number of companies including GE, 3M, MZA, Philips, Picker, and VTI. His work spans a wide range of image processing and vision problems including image denoising, edge detection, segmentation, shape analysis, multi-frame stereo reconstruction, visual tracking, and registration. His research interests include curve and surface evolution, differential geometry, partial differential equations, and shape optimization.

Allen Tannenbaum (F'09) has held faculty positions in Israel, Canada, Switzerland, and the U.S. He is currently a Professor of computer science and applied mathematics/statistics with SUNY Stony Brook, Stony Brook, NY, USA. He works in image processing, medical informatics, computer vision, and systems and control.

Appendix

Define a function Q(x, ϕ, K₀, K₁, K₂, K₃) as

$$Q=\frac{{(x-K_1∕K_0)}^r{(y-K_2∕K_0)}^s{(z-K_3∕K_0)}^t}{{\left(\rho K_0\right)}^{(r+s+t+3)∕3}},$$ (21)

where

$$K_1={\int }_{\Omega }xH\left(\varphi \right)d\mathit{x},K_2={\int }_{\Omega }yH\left(\varphi \right)d\mathit{x},K_3={\int }_{\Omega }zH\left(\varphi \right)d\mathit{x}.$$ (22)

Then eq. (13) can be rewritten as

$${\eta }_{rst}={\int }_{\Omega }Q(\mathit{x},\varphi ,K_0,K_1,K_2,K_3)H\left(\varphi \left(\mathit{x}\right)\right)d\mathit{x}.$$ (23)

By applying the chain rule, the variational derivative of η_rst with respect to ϕ is

$$\begin{array}{cc}\hfill \frac{\partial {\eta }_{rst}}{\partial \varphi }=& \sum _{0\le i\le 3}\frac{\partial {\eta }_{rst}}{\partial K_i}\frac{\delta K_i}{\delta \varphi }+\frac{\partial {\eta }_{rst}}{\partial H}\frac{\delta H}{\delta \varphi }\hfill \\ \hfill =& \sum _{0\le i\le 3}\frac{\partial K_i}{\delta \varphi }{\int }_{\Omega }\frac{\partial Q}{\partial K_i}H\left(\varphi \right)d\mathit{x}+\delta \left(\varphi \left(\mathit{x}\right)\right)Q,\hfill \end{array}$$ (24)

where δ(·) is the Dirac function. It can be shown that the variational derivatives $\frac{\partial {\eta }_{rst}}{\partial K_i}$ are

$$\frac{\partial {\eta }_{rst}}{\partial K_0}=\frac{r{K}_1{\eta }_{r-1,s,t}+s{K}_2{\eta }_{r,s-1,t}+t{K}_3{\eta }_{r,s,t-1}}{{K_0}^2{\left(\rho K_0\right)}^{1∕3}}-\frac{r+s+t+3}{3K_0}{\eta }_{r,s,t},$$ (25)

$$\frac{\partial {\eta }_{rst}}{\partial K_1}=-\frac{r{\eta }_{r-1,s,t}}{K_0{\left(\rho K_0\right)}^{1∕3}},$$ (26)

$$\frac{\partial {\eta }_{rst}}{\partial K_2}=-\frac{s{\eta }_{r,s-1,t}}{K_0{\left(\rho K_0\right)}^{1∕3}},$$ (27)

$$\frac{\partial {\eta }_{rst}}{\partial K_3}=-\frac{t{\eta }_{r,s,t-1}}{K_0{\left(\rho K_0\right)}^{1∕3}}.$$ (28)

The variational derivatives of K_i with respect to ϕ are

$$\begin{array}{cc}\hfill \frac{\delta K_0}{\delta \varphi }=& \delta \left(\varphi \left(\mathit{x}\right)\right),\frac{\delta K_1}{\delta \varphi }=x\delta \left(\varphi \left(\mathit{x}\right)\right),\hfill \\ \hfill \frac{\delta K_2}{\delta \varphi }=& y\delta \left(\varphi \left(\mathit{x}\right)\right),\frac{\delta K_3}{\partial \varphi }=z\delta \left(\varphi \left(\mathit{x}\right)\right).\hfill \end{array}$$ (29)

Note that the results for computing the variational derivatives for 3D moments, equs. (26)~(28), are similar to those of [21] for the 2D case.