Deformable Models with Sparsity Constraints for Cardiac Motion Analysis

Abstract

Deformable models integrate bottom-up information derived from image appearance cues and top-down priori knowledge of the shape. They have been widely used with success in medical image analysis. One limitation of traditional deformable models is that the information extracted from the image data may contain gross errors, which adversely affect the deformation accuracy. To alleviate this issue, we introduce a new family of deformable models that are inspired from the compressed sensing, a technique for accurate signal reconstruction by harnessing some sparseness priors. In this paper, we employ sparsity constraints to handle the outliers or gross errors, and integrate them seamlessly with deformable models. The proposed new formulation is applied to the analysis of cardiac motion using tagged magnetic resonance imaging (tMRI), where the automated tagging line tracking results are very noisy due to the poor image quality. Our new deformable models track the heart motion robustly, and the resulting strains are consistent with those calculated from manual labels.

Graphical Abstract

1. Introduction

Physics-based deformable models and their variations have been studied extensively in recent decades and widely used in computer vision, computer graphics and medical image analysis (Metaxas, 1992; McInerney and Terzopoulos, 1996; Metaxas, 1996; Nealen et al., 2006b). They are able to solve diverse types of problems, such as, but not limited to, image segmentation (Kass et al., 1988), image registration (Rueckert et al., 1999; Shen and Davatzikos, 2002), shape reconstruction (Terzopoulos et al., 1988; Metaxas and Terzopoulos, 1993), and motion analysis (Metaxas and Terzopoulos, 1991; Paragios and Deriche, 2000). The name “deformable models” is derived from the nonrigid body mechanics, which describes how elastic objects respond to applied forces. Starting from an initial shape, the model is usually deformed by two types of forces, i.e., internal and external forces. The external force drives the model to fit the observations, while the internal force constrains the geometric flexibility of the shape. For examples, in the image segmentation problem, the external force computed from the image intensity drives the model to the estimated boundary, while the internal force keeps the boundary smooth. In the motion analysis (e.g., cardiac motions (Park et al., 1996; Haber et al., 2000; Hu et al., 2003; Chen et al., 2008; Wang and Amini, 2012)) and the shape manipulation problems (Nealen et al., 2006b), control points are employed as the external force to drive the model, and the internal force maintains the smoothness and preserves shape details. The control points are tracked along a motion sequence, and then an initial model is deformed to fit the control points in each following frame. This is often measured by the distances between the control points and the corresponding points on the initial model. In fact, in the context of motion analysis and shape manipulation, many previous methods (Sorkine et al., 2004; Zhou et al., 2005; Yan et al., 2007; Wang et al., 2008a) use Euclidean distance or L2 norm as the distance metric for penalty functions. This assumes intrinsically that the errors of the target points follow a Gaussian distribution with small variances. Nevertheless, this is not always true in practice. Since the control points are usually from automated detections, they may contain not only Gaussian noise, but also some gross errors or outliers due to the erroneous detection. Therefore, the accuracy of the traditional deformable models depends heavily on the accuracy of the control point detection.

In this paper, we focus on improving the robustness of traditional deformable models, particularly for the problems of cardiac motion analysis. Inspired by the robust recovery power of the compressed sensing approach (Donoho, 2006; Candes et al., 2006), we propose a new class of deformable models using sparse regularization. Recent research in compressed sensing shows that using an L1 norm can dramatically increase the probability of accurate signal recovery, even when there are both sparse outliers and moderate Gaussian noise (Candes and Tao, 2006). Thus, we design a robust deformable model by integrating seamlessly an L1 norm regularization with a modified Laplacian deformable model (Sorkine et al., 2004; Yu et al., 2013). This new model is able to handle outliers or gross errors. In addition, it is designed as a convex optimization problem, and can be efficiently solved within a constrained solution space. However, when the variances of the Gaussian noise are large, solely using the L1 norm may cause overfitting problems due to its nature of pursuing the sparse structure (Candes et al., 2005). Therefore, we propose a deformable model using hybrid norm regularization that is able to handle both the Gaussian errors and gross errors. We also generalize these two models in a unified formulation, named as Sparse Deformable Models.

In the following section, we discuss the relevant work of deformable models and compressed sensing. Our proposed sparse deformable models (SDM) are presented in detail in section 3. In section 4, we validate our models on a clinically important and challenging problem, i.e., the left ventricle (LV) motion analysis in mouse cardiac tagged MRI. Section 5 presents experimental results demonstrating the robustness of our models on mouse heart motion tracking even with inaccurate results of control point detection. The last section draws conclusions on model advantages, and discusses directions of future work.

2. Related Work

2.1. Deformable Models

With the success of active contour models (Kass et al., 1988), many methods have been proposed to improve deformable models. Most of the work focuses on either internal force or external force. In this section, we introduce some relevant papers in these two aspects.

Internal force usually enforces the smoothness characteristics of deformable models, such as the local deformation similarity. An unconstrained deformable model may easily result in unrealistic shapes due to the weak or misleading image cues. Therefore, the internal force is critical for the robustness. The global parametric models (e.g., deformable superquadratics) were proposed to build models based on a few global shape parameters (Terzopoulos and Metaxas, 1990; Bardinet et al., 1996). Although these models reduce the degree of freedom dramatically, they have difficulty to present the shape details. The local geometry properties can be used as constraints to solve these problems. For examples, splines were used on image deformation to constrain the smoothness of the deformation field (Tustison and Amini, 2006). Piecewise-smooth finite element model (FEM) was employed to present the deformable boundary (Duan et al., 2009b, 2010), which achieved real-time myocardial segmentation in both ultrasound and MRI data. The Laplacian coordinates (Sorkine et al., 2004) have been also a well-known measurement for the local similarity. Comparing with spline- and FEM-based methods, Laplacian coordinates allow more flexible shape representation. Sorkine et al. (2004) employed it to constrain the smoothness and local similarity of the 2D mesh deformation in shape editing. Shen et al. (2011) decomposed the Laplacian coordinates into components in the perpendicular and tangential directions, to formulate a detail-preserved internal force. In this paper, we adapt the traditional Laplacian coordinates in a new setting of 3D volumetric and meshless deformable models to enforce the smoothness and local shape similarity.

External force matches the model to the observations derived from the image appearance. They are usually categorized as short-range and long-range forces. The short-range forces are defined based on the local information in a small neighborhood. For example, in segmentation problems, they drive the contour to the estimated boundary. The boundary may be defined by the intensity, gradient change, or high response of boundary detectors (Kass et al., 1988). In registration problems, the source image is deformed to match the target image according to the appearance similarity (Duan et al., 2009a). The pixels are matched based on textures in their neighborhoods. The long-range forces deform the model to match pre-calculated landmarks (Terzopoulos and Metaxas, 1991) or satisfy model priors (Cohen and Cohen, 1993). Region appearance features have also been used (Zhu and Yuille, 1996; Jehan-Besson et al., 2003; Huang and Metaxas, 2008) to augment the deformable models by leveraging the image intensity statistics. They discriminate the inside and outside region based on their intensities and textures. Recently, dictionary learning is also used to learn appearance characters (Huang et al., 2013a,b). Each pixel is classified into different regions based on their reconstruction residues under different dictionaries. Our deformable model uses control points as the external force, which is a natural choice for cardiac motion analysis.

2.2. Robust Shape Priors

Most deformable models assume that there is no outlier or gross error on the detected landmarks, while such error are very common due to the image noise or weak appearance cues. Statistical shape models, such as active shape models (Cootes et al., 1995) and their variants, can effectively handle outliers using shape priors. Some of them detect and eliminate the outliers explicitly before the deformation. Duta and Sonka (1998) proposed a method to detect outliers by hypothesis testing based on the point distribution model. The detected outliers are removed or replaced based on the mean shape of the model. Prastawa et al. (2004) proposed to detect the abnormal regions by registering with a standard atlas. The regions largely different from the normal intensities are determined as outliers. Lekadir et al. (2007) used a local shape dissimilarity measure, which is invariant to scaling, rotation and translation, to detect the outliers, and then displaced them based on the local valid points. Other researchers aimed to reduce the effect of the outliers during the model deformation. Rogers and Graham (2006) evaluated M-estimator, least median of squares and random sample consensus (RANSAC) (Fischler and Bolles, 1981) to handle outliers in active shape models. RANSAC showed the best performance in the quantitative evaluation. Davatzikos et al. (2003) utilized wavelet transformation to build a hierarchical shape model to improve the local robustness. The low-frequency bands carry global shape information, and the high-frequency bands serve as local smoothness constraints. Besides shape priors, image atlas-based methods also naturally have the properties of handling segmentation errors (Shiee et al., 2011). Priors can also be based on data-specific properties, e.g., the relative positions of multiple components, which are modeled by formulating the relation explicitly (Paragios, 2002) or learning shape priors from examples (Paragios, 2003).

Recently, compressed sensing methods have been intensively investigated. These methods aim to reconstruct a signal that is known to be compressible with certain transformation based on sparse measurements. Such sparse methods have been widely used in computer vision and image processing communities to deal with gross errors or outliers. Particularly, the sparse constraints have been employed to model shape priors effectively (Zhang et al., 2012a,b) and register shapes robustly (Hontani et al., 2012). In their settings, most of the control points generated from point detectors are roughly accurate, while a small number of points may contain large errors. These points are represented as sparse outliers and optimized with sparsity regularization.

Most above-mentioned methods rely on the training samples, which are not always available in practice for deformable models. In addition, they are based on the static images or shapes to form an over-complete dictionary and assume that the target model can be sparsely reconstructed by the dictionary. Different from these above-mentioned methods, we propose to seamlessly integrate sparsity prior with deformable models to handle outliers adaptively and dynamically for a sequence of images. Therefore, there is no need for training data.

3. Methodology

Consider a set of points 𝒱, where each point has a neighborhood structure¹, and a subset of 𝒱 as control points 𝒱_c that are computed from the observations (e.g., image information). Denote the homogeneous coordinate of the point i as v_i = [x_i, y_i, z_i, 1]^T and its position after deformation as ${\mathbf{v}}_i^{\prime }={[x_i^{\prime },y_i^{\prime },z_i^{\prime }]}^T$, where i = 1,2,⋯,n. Then the coordinates of all the deformed points are represented as:

$${\mathbf{V}}^{\prime }={[{{\mathbf{v}}_1^{\prime }}^T{{\mathbf{v}}_2^{\prime }}^T\cdots {{\mathbf{v}}_n^{\prime }}^T]}^T.$$ (1)

The goal of our deformable models is to track the motion of the whole shape, given the position ${\mathbf{V}}_c^{\prime }$ of control points 𝒱_c. Instead of calculate the deformed point positions V′ directly, we optimize the deformations T for each point. The deformed position ${\mathbf{v}}_i^{\prime }$ then can be derived by the relation ${\mathbf{v}}_i^{\prime }=T_i{\mathbf{v}}_i$. It is easier to incorporate the shape constraints into the deformations than the point positions. This will be further discussed in the internal force. The deformation of each point i is parameterized by an affine 3 ×4 transformation matrix T_i, where the first three columns are linear transformation and the last column is translation.

In our model, the internal force preserves the local shape structure by using the Laplacian coordinate, and the external force minimizes the difference between the shape and the control points via our proposed sparse constraints.

3.1. Internal Force

Our internal force ensures the local similarity by regularizing the differences of the transformation matrices between neighboring points. Instead of measuring the matrix distance directly, we measure the difference by the deformation they generate. Specifically, if a point is deformed using the transformation matrices of its neighboring points, the resulting displacements should be similar. In other words, for a point i, its displacement after applying its own transformation matrix T_i should be similar to applying its neighbor’s transformation matrix T_j. Thus, the energy function of the internal force is:

$$E_{\mathit{int}}=\sum _{i\in \mathcal{V}}\sum _{j\in \mathcal{N}(i)}{w}_{ij}{\Vert T_i{\mathbf{v}}_i-T_j{\mathbf{v}}_i\Vert }_2^2,$$ (2)

where ||·||₂ denotes the entry-wise matrix L2 norm², and weight w_ij is the strength of connection between points i and j. In the meshless model, it is defined by a kernel function, such as the polynomial kernel (Muller et al., 2003):

$$w(r,h)=\frac{315}{64\pi h^9}\{\begin{array}{cc}{(h^2-r^2)}^3& \text{if}r<h\\ 0& \text{otherwise},\end{array}$$ (3)

where r is the distance between points i and j, and h is the support of the kernel, which means that the kernel is non-zero only when r < h. The function only contains the square term of r. Since r is defined as the square root of $\mathrm{\sum }{r}_k^2$, where r_k is the distance in the kth direction, this kernel eliminates the requirement of square root calculation.

The unconstrained transformation matrix T_i is able to deform the local structure arbitrarily. This may produce some unnatural and unreasonable shapes to minimize the energy. For example, if all transformation matrices are zero matrices, points will be transformed to the origin. Thus, the whole model degenerates to a single point and the internal energy is zero. Although this complete degeneration may not happen in most applications because of the external force from the control points, the deformable models with unconstrained transformation will still lose geometry details in directions with less control points information, e.g., the direction perpendicular to the surface (Nealen et al., 2006a). To alleviate this problem, we restrict T_i to translations, rotations and isotropic scales. Particularly, we disallow the anisotropic scaling, which will remove local shape details.

The translations are defined explicitly in the last column of T_i, and rotations and isotropic scales are both represented in the linear transformation, which also contains other types of transformation that we do not expect in our model, e.g., anisotropic scales. Therefore, we define a special type of linear transformation containing only rotations and isotropic scales. Rotations are represented by multiplication with an orthogonal matrix, which is represented as the matrix exponential of a skew-symmetric matrix exp(H). In particular, 3 ×3 skew-symmetric matrices emulate a cross product with a vector, i.e. Hx = h ×x. Based on this property, one can derive the following expansion of the exponential above:

$$exp(H)=\alpha I+\beta H+\gamma {\mathbf{h}}^T\mathbf{h},$$ (4)

where I and H are linear terms, while h^T h is quadratic. Adding the isotropic scale s to the transformation, the class of matrices for linear part becomes T = s exp(H). Here, we keep only the linear term of the matrices and form the approximately constrained transformation as:

$$T_i=\left(\begin{array}{cccc}s& -h_1& h_2& p_x\\ h_1& s& -h_3& p_y\\ -h_2& h_3& s& p_z\end{array}\right).$$ (5)

In this matrix, s is the isotropic scalar, h is the non-zero term of the skew-symmetric matrix, and p is the translation part. It is a good linear approximation for rotations with small angles.

Within this setting, we expect to find T_i satisfies Eq.5, while minimizing the internal energy. In Eq.2, T_i only appears in the structure T_iv_k, where k = i or in the neighborhood of point i. T_i is a matrix and v_k is a vector in this function. In order to enforce the constraint, we reformulate it to a function of t_i = (s_i, h_i, p_i)^T, which is the vector of the unknowns in T_i. Thus, we substitute T_iv_k with $A_k{\mathbf{t}}_i^T$, where A_k contains the position of v_k. It is defined as:

$$A_k=\left(\begin{array}{ccccccc}{v}_{k_x}& 0& v_{k_z}& -v_{k_y}& 1& 0& 0\\ v_{k_y}& -v_{k_z}& 0& v_{k_x}& 0& 1& 0\\ v_{k_z}& v_{k_y}& -v_{k_x}& 0& 0& 0& 1\end{array}\right).$$ (6)

Thus, the internal force is reformed as

$$E_{\mathit{int}}=\sum _{i\in \mathcal{V}}\sum _{j\in \mathcal{N}(i)}{w}_{ij}{\Vert A_i{\mathbf{t}}_i-A_i{\mathbf{t}}_j\Vert }_2^2.$$ (7)

Since this is a summation of the quadratic forms of the transformation matrices t_i, we can represent the energy function as a quadratic form of all the unknown transformations t. In order to form the internal force, we encode the point position matrix A_i into matrix K_i as

$$K_i=M_i\otimes A_i,$$ (8)

where ⊗ is the Kronecker product, and M_i is a m_i × n node-arc matrix of v_i, where m_i is the number of its neighbors. For each neighbor v_j, there is one row in M_i where the ith element is w_ij and the jth element is −w_ij, while the other elements are all zeroes:

$$M_i=\left(\begin{array}{ccccc}& i\text{th}\text{column}& & j\text{th}\text{column}& \\ \cdots & w_{ij}& \cdots & -w_{ij}& \cdots \\ & ⋮& & ⋮& \end{array}\right),j\in \mathcal{N}({\mathbf{v}}_i).$$ (9)

K_it is the difference of the displacements based on different transformations of the neighborhood of point i. We concatenate the matrix K_i for all the points to form the matrix $K={[K_1^T{K}_2^T\cdots K_n^T]}^T$. Thus, the energy function of the internal force (Eq.2) is formulated as:

$$E_{\mathit{int}}={\Vert K\mathbf{t}\Vert }_2^2.$$ (10)

This energy function is a quadratic form of all the independent transformation unknowns. By solving this function, we directly get the transformation matrices for all the points, which contain only translation, rotation and isotropic scale. There is no need to regularize each transformation explicitly to the three types above. This setting of internal force is able to ensure the smoothness of the whole shape and preserve the local similarity. In addition, this internal force is intentionally designed as a convex optimization problem, so it can be coupled seamlessly with our sparsity-based external forces.

3.2. External Force

Besides the shape constraint from the internal force, the deformable model also aims to match the observations. For example, a point i on the model is expected to fit the position of corresponding control point ${\mathbf{v}}_i^{\prime }$ after deformation t_i. In other words, ${\mathbf{v}}_i^{\prime }=A_i{\mathbf{t}}_i$, where A_i is the position of point i defined above. We concatenate the point coordinates A_i into an 3n × 7n sparse matrix:

$$D=\left[\begin{array}{cccc}{A}_1& & & \\ & A_2& & \\ & & \ddots & \\ & & & A_n\end{array}\right],$$ (11)

where V′ = Dt is the model deformation based on the transformation parameters t. We use a control point indicator c to select the rows of D and V′ corresponding to the control points. Thus, this deformable model is defined as:

$$\underset{\mathbf{t}}{argmin}\{{\Vert K\mathbf{t}\Vert }_2^2+\lambda {\Vert D_c\mathbf{t}-{\mathbf{V}}_c^{\prime }\Vert }_2^2\},$$ (12)

where λ is the stiffness weight, which controls how much the model is able to deform to match the control points. Larger λ results in a better fitting, but the deformed shape may not be smooth. The L2 norm is used as a penalty function in this formulation. This is also known as Laplacian deformable models (LDM). Similar settings have been used in many applications, such as shape editing (Sorkine et al., 2004; Sorkine and Alexa, 2007) and cardiac modeling (Wang et al., 2008b). However, it may not be the most proper metric for certain problems, especially when there are gross errors or outliers.

3.2.1. Sparsity Constraint Using L1 Norm

Gross errors may easily happen in medical imaging problems, such as the erroneous detections in a noisy image. The L2 norm in Eq.12 follows a Gaussian distribution for residuals. It may overfit these sparse outliers, and hence adversely affect the deformation accuracy. It is desirable to model such sparse outliers during the deformation. The L0 norm counts the number of non-zero elements and can model such sparse errors exactly. Therefore, we can use it to capture the sparse outliers:

$$\begin{array}{l}\underset{\mathbf{t}}{argmin}{\Vert K\mathbf{t}\Vert }_2^2\\ s.t.{\Vert D_c\mathbf{t}-{\mathbf{V}}_c^{\prime }\Vert }_0<k,\end{array}$$ (13)

where ||·||₀ is the L0 norm and k is the pre-defined maximum number of outliers. However, the L0 norm is non-convex, and solving an L0 norm problem is NP-hard. Although greedy methods (Mallat and Zhang, 1993; Tropp and Gilbert, 2007) can be employed to solve such problem, we do not know the sparse number, i.e., the number of outliers, and different data may have different sparsity numbers. Therefore, in practice it is hard to use L0 norm as the sparsity constraint in deformable models. Recent developments in compressed sensing (Candes et al., 2006) show that minimizing an L1 norm problem can produce a nearly identical sparse solution as using the L0 norm. Thus, we use convex relaxation to define a sparse deformable model based on the L1 norm as (we use SDM-L1 to stand for Sparse Deformable Models with L1 norm regularization):

$$\underset{\mathbf{t}}{argmin}\{{\Vert K\mathbf{t}\Vert }_2^2+\lambda {\Vert D_c\mathbf{t}-{\mathbf{V}}_c^{\prime }\Vert }_1\},$$ (14)

where λ is a constant and controls how sparse the outliers are. Different from sparsity number k, this λ is more flexible as this is a soft constraint.

To illustrate intuitively the effectiveness of method, we test it on a toy example of shape deformation (Fig. 1). The initial model is a 4 ×4 grid with four-direction connection. Four corners are chosen as the control points, and the top-right one is an outlier, as shown in Fig. 1(a). The deformable model aims to fit these four points while maintaining the original grid shape. The result using the L2 norm is adversely affected by the outlier, since the quadratic form of the energy function results in a large penalty on this outlier. We obtain a much more accurate result by using the L1 norm, since the linear constraint has a higher tolerance for gross errors.

Figure 1.

The toy sample of the deformable model using L2 and L1 norm regularization. The circles are the observed control points with outliers. We fit a grid to control points. The stars are the desired result (ground truth).

3.2.2. Sparsity Constraint Using Both L1 and L2 Norms

Similar sparsity constraints have been successfully applied in many applications, such as face recognition (Wright et al., 2009), background subtraction (He et al., 2012), and shape prior modeling (Zhang et al., 2012a). However, in deformable models, the observations may still contain Gaussian errors with large variations. Solely using the L1 norm may not be able to handle them well. Therefore, we combine both L1 and L2 norms, which assist our sparse deformable models to handle more general cases (we use SDM-L1/L2 to stand for the Sparse Deformable Model with both L1 and L2 norm regularization):

$$\underset{\mathbf{t},\mathbf{e}}{argmin}\{{\Vert K\mathbf{t}\Vert }_2^2+\lambda \left({\Vert D_c\mathbf{t}-{\mathbf{V}}_c^{\prime }-\mathbf{e}\Vert }_2^2+\gamma {\Vert \mathbf{e}\Vert }_1\right)\},$$ (15)

where e represents the gross errors and is constrained by the L1 norm, and γ ∈ [0,1] controls how sparse e is. The new model combines the advantages of both L1 and L2 norms, and is able to handle both gross errors and traditional Gaussion errors with large variations. In fact, this model is closely related to the other models by adjusting the parameters λ and γ. If γ is extremely large, e will be all zeros. Thus, the model degenerates to a method with only an L2 norm, as in Eq.12, and it is sensitive to gross errors. If λ is extremely large and γ is small, the deformation errors will be approximately equal to e. Thus, the model is similar to the one with only sparse constraints, as in Eq.14. It will be robust to the outliers, but cannot handle large Gaussian noise. Therefore, Eq.15 is actually a general form of these models. In addition, it is also closely related to Huber norm, which has been used in statistics for robust estimation. It is defined as:

$$\rho (x)=\{\begin{array}{ll}{x}^2\hfill & \mid x\mid \le \gamma \hfill \\ 2\gamma \mid x\mid -{\gamma }^2\hfill & \mid x\mid >\gamma .\hfill \end{array}$$ (16)

Huber norm is similar to the L2 norm when the errors are smaller than γ, and similar to the L1 one when the errors are larger than γ. Therefore, it effectively models both the Gaussian noise and the outliers. Using the notation of Huber norm, SDM-L1/L2 can be rewritten as:

$$\underset{\mathbf{t}}{argmin}\{{\Vert K\mathbf{t}\Vert }_2^2+\lambda {\Vert D_c\mathbf{t}-{\mathbf{V}}_c^{\prime }\Vert }_{\text{Huber}}\}.$$ (17)

Besides handling different types of errors, another benefit of this deformable model is the convexity and the continuous first-order derivative. Therefore, the above problem can be solved efficiently using our proposed algorithm introduced below.

3.2.3. Optimization Framework

Although the above problem can be solved by the standard convex optimization algorithms, such as interior point method (Grant and Boyd, 2008), we propose an effective optimization algorithm that fully utilizes the special structure of this problem. We alternately optimize two variables t and e. These two subproblems both have analytical solutions. They can be solved efficiently in each iteration. The results converge fast in our experiments. We initialize the gross error e as zero and apply our alternating algorithm to the problem. In the first step, when e is fixed, the problem is reduced to the conventional L2 norm regularization:

$$\underset{\mathbf{t}}{argmin}\{{\Vert K\mathbf{t}\Vert }_2^2+\lambda {\Vert D_c\mathbf{t}-{\mathbf{V}}_c^{\prime }-\mathbf{e}\Vert }_2^2\}.$$ (18)

It can be solved by least square minimization. In the second step, t is fixed, and the optimization problems for each term e_i of the outlier e are independent:

$$\underset{e_i}{argmin}\{{({(D_c\mathbf{t})}_i-{\mathbf{V}}_{ci}^{\prime }-e_i)}^2+\gamma \mid e_i\mid \},$$ (19)

where (D_ct)_i is the ith element of the vector D_ct. The minima for the two parts can be achieved at ${(D_c\mathbf{t})}_i-{\mathbf{V}}_{ci}^{\prime }$ and 0, respectively. The minimum of the energy function must lie between them, since both of them are convex. Therefore, e_i has the same sign as ${(D_c\mathbf{t})}_i-{\mathbf{V}}_{ci}^{\prime }$. After determining the sign of e_i, the problem reduces to a constrained quadratic function of e_i, and the solution is:

$$e_i=\{\begin{array}{ll}max\{0,{(D_c\mathbf{t})}_i-{\mathbf{V}}_{ci}^{\prime }-\gamma /2\}\hfill & \text{if}{(D_c\mathbf{t})}_i-{\mathbf{V}}_{ci}^{\prime }\ge 0\hfill \\ min\{0,{(D_c\mathbf{t})}_i-{\mathbf{V}}_{ci}^{\prime }+\gamma /2\}\hfill & \text{otherwise}.\hfill \end{array}$$ (20)

Since there are analytical solutions for both of the substeps in our algorithm, the energy function will monotonically decrease to a minimum solution. The convexity of the whole problem makes sure that this is the global solution of the problem.

4. Motion Analysis of Mouse LV with Deformable Models

In this section, we apply our proposed deformable models to solve a clinically important and challenging problem, i.e., mouse LV motion analysis using tagged MRI (tMRI). Over the past two decades, clinical experiments utilizing transgenic and knockout mice have significantly advanced the research on cardiovascular diseases. In addition, these models have become indispensable tools to study the cardiovascular diseases in humans as well (Zhou et al., 2003). For the study of ventricular function in particular, tMRI offers a powerful non-invasive tool for making measurements of the beating heart that directly reflect its complex in vivo physiology. It has been widely used for the assessment of human heart diseases as well as in experimental heart disease models in mice. Compared to the human heart, the data acquisition from the mouse heart is more challenging, owing to the inadequate spatial and temporal resolutions and limited image quality. The mouse heart is about 1000th the size of a human heart and beats much faster, at 400–600 beats per minute (bpm), than the human heart, with 60–80 bpm. Currently available MRI instruments for mouse imaging operate at a higher magnetic field strength (4.7T or above) than clinical M-RI scanners, but they are still unable to provide temporal and spatial resolution in proportion with the mouse heart rate and size. Consequently, the tagging lines extracted from the mouse tMRI images contain more outliers than that from human data, as shown in Fig. 2(b). A compromise is obtained between tagging spatial and temporal resolution in order to complete the study in reasonable amount of time. As shown in Fig. 2(a), there are five SA slices, which can only cover the equatorial region of LV. There are few deformation information in the basal region from the SA slices. Meanwhile, the distances between the tagging lines are small in each image. The tagging lines are easily mistracked into adjacent ones due to the fuzzy image, as shown in Fig. 2(b).

Figure 2.

(a) The setting of the MRI images with fitted LV model. (b) Sample tagged SA image with the result of tagging line tracking. The tracking is inaccurate when there are large displacements.

Before introducing our framework with sparse deformable models, we first discuss related work for cardiac motion analysis, starting from the human data and then the mouse data. In recent studies, as the increasing of cardiac MRI data, many methods have been developed to track and measure the cardiac motion (Wang and Amini, 2012; Tavakoli and Amini, 2013; Tobon-Gomez et al., 2013). There are two main categories. The methods in the first category obtains the deformation field directly from tagged MRI. HARP(Harmonic Phase) (Osman et al., 2000) analyzed the image in the frequency domain. The tagging technique gave rise to spectral peaks in k-space. The tagging motions were extracted from the inverse Fourier transform of a single peak. Other methods use local sine wave modeling (Arts et al., 2010) or Gabor filter banks (Qian et al., 2003; Chen et al., 2010) to analyze the image in the frequency domain. These methods cannot handle large tagging movement, and estimate only in-plane motion. Additional information was needed to reconstruct 3D motion with only sparse tagged MR images. The second category aims to track the model motion based on feature extraction. The tagging lines and their intersections were usually detected first. Then the myocardial motion was estimated based on their displacements. Young et al. (1995) used the 2D active contour model to track the tagging lines semi-automatically. Then, the LV was represented as a volumetric deformable model, and driven by the displacement of the tag intersections. Chen et al. (2010) employed the Gabor filter banks to detect the local tag intersections, and robust point matching to track the myocardial motion sparsely. The deformable model was then refined by meshless deformation, which is initialized by the sparse tracking results.

Several methods developed for human data have been applied to murine cardiac motion analysis. The characterization of the murine cardiac mechanical function in normal, infarcted, or genetically engineered mice or rat model subjected to stem cell intervention were reported in (Liu et al., 2006; Young et al., 2006; Zhou et al., 2003), and 3D rendering of LV deformation was attempted. For example, recently Zhong et al. (2008) represented the 3D myocardial deformation based on the movement of material points near the sparse SA slices. By decomposing the heart movement in long-axis (LA) and SA directions, Chuang et al. (2010) reconstructed the motion of the whole heart based on the combination of the displacement of intersections in all these components.

However, all methods mentioned above assume that the tagging lines were labeled manually or tracked correctly. The tracking error is unavoidable in automatic methods, especially when they are applied to low quality MR images, such as mouse data. To solve this problem, we propose a robust motion analysis system from 2D image processing to 3D motion analysis. The sparse deformable models are employed to overcome the possible gross errors in the image-processing step. The system automatically calculates the model deformation without any user interaction.

Our system consists of four major components: 1) 2D image segmentation, 2) 3D surface model construction, 3) material point tracking, and 4) sparse model deformation. Fig. 3 provides a pipeline of the proposed system. The LV is segmented on both short and long axis non-tagged MR images with active contour models. The 3D LV model is initialized from these sparse 2D contours at the end of diastole (Zhang et al., 2009). We register a standard LV surface model, manually segmented from 3D CT data, to the sample specified boundary using coherent point drift (CPD) (Myronenko and Song, 2010). Then a Gabor filter bank (Chen et al., 2010) has been implemented to generate corresponding phase maps from low quality tagged MRI images. The 3D control points are generated, based on the tagging lines and the contours of separate slices. Finally, we generate a dense 3D meshless model based on the initial surface mesh. This model is driven by the control points to track the LV movement along a cardiac cycle with our proposed sparse deformable models, and the motion strains are calculated locally based on the tracking results. We have introduced the deformation module, which is our major contribution, and in the following we briefly introduce the model initialization and strain analysis modules.

Figure 3.

The pipeline of our system. Both the tagged and non-tagged MR images are used to track the control points in a cardiac cycle. Meanwhile, the non-tagged MR images are used to generate the boundary mesh for initialing meshless model. Driven by control points, the meshless model deforms based on our nonlinear Laplacian kernel.

Volume Model Initialization

A surface mesh at the initial frame is needed to build the meshless 3D heart model. Different from the 3D segmentation methods such as (Zhuang et al., 2010), the boundary information only distributes sparsely on the given slices in our data. Thus, we extract a normal heart mesh model segmented from 3D CT data as the standard heart model, and then apply CPD (Myronenko and Song, 2010) to fit this mesh model to the sparse contours. When many LV surface samples are available, methods based on active shape models, like SPASM (van Assen et al., 2006), are able to capture more model details. In contract to statistical shape models, our method generates data specific LV model based only on a standard LV model. It well handles the inconsistency among the contours, i.e., the LA and SA contours are not intersected exactly. The result surface fits all the 2D cues well, and maintains a reasonable shape. We use this data-specified surface mesh as the boundary, and generate a meshless model by evenly interpolating the material points inside it.

Strain Analysis

The strain is a description of the relative displacement in the body. It is represented as the displacement between points in the body relative to a reference length (i.e., a ratio between lengths). It is close related to the gradient of the displacement field ∇u, where u_i = v′_i − v_i is the displacement of points. Based on the moving least square method, the displacement field for the point i is defined in its neighborhood as:

$$\nabla \mathbf{u}{\mid }_{{\mathbf{x}}_i}={\mathbf{A}}^{-1}\left(\sum _j({\mathbf{v}}_j-{\mathbf{v}}_i){({\mathbf{u}}_j-{\mathbf{u}}_i)}^T{\omega }_{ij}\right),$$ (21)

where A = Σ_j(v_j − v_i)(v_j − v_i)^T ω_ij is the moment matrix, which is based solely on the initial model. Then, the Green-Lagrangian strain tensor ε is:

$$\mathit{\epsilon }=\frac{1}{2}(\nabla \mathbf{u}+\nabla {\mathbf{u}}^T+\nabla \mathbf{u}\nabla {\mathbf{u}}^T).$$ (22)

In the strain tensor ε, there are only six different terms because of the symmetry. The diagonal terms are the normal strains on three perpendicular directions, and the others are the shear strains.

5. Experiments

5.1. Validation on the Synthetic Data

Our methods are validated on synthetic 3D volumes with pre-defined deformation. Three methods are compared: 1) Laplacian deformable models (LDM) (Sorkine et al., 2004; Wang et al., 2008a), 2) sparse deformable model with L1 norm regularization (SDM-L1), and 3) sparse deformable model with both L1 and L2 norm regularization (SDM-L1/L2). First, an LV volume model is generated, with the internal points evenly distributed on SA and LA directions. Ten percent of SA slices with equal intervals are chosen as the control points. Then random global transformations are applied to this model, such as scaling, rotation and twisting. Two types of errors are also applied to the deformed model to simulate the noisy tracking results. Gaussian noise is added to all control points, and a few points are selected randomly and large displacements are applied to simulate gross errors. Based on the displacements of the control points, deformable models are employed to reconstruct the displacements of the other points. The deformable models are tested under different parameters and different noise intensities. In each parameter setting, we randomly generate 1000 samples and calculate the mean and variance of deformation errors.

We first analyze the deformation errors when changing the coefficient γ, reflecting the balance between the L1 and L2 norms (Fig. 4(a)). The deformation error is large when γ is close to zero, and reduces dramatically when γ is a little larger. Since SDM-L1/L2 is similar to SDM-L1 when γ is small, it indicates that SDM-L1 cannot handle Gaussian error properly. As γ becomes even larger, the mean error increases again. It becomes stable when γ is large enough, which is similar to the result of LDM. The results show that SDM-L1/L2 outperforms the other models that use only one type of norm. Theoretically, the model achieves the best result when the threshold γ is similar to the variance of Gaussian noise. However, it is hard to measure the noise variance exactly in real data. Therefore, we set it to one tenth of the median of the neighborhood distances empirically and it shows good results.

Figure 4.

(a) shows the deformation errors related to the coefficient γ of the L1 norm. (b) and (c) show the deformation errors of different deformable models (errors are measured in millimeters). They are compared under different Gaussian noise variances and numbers of outliers, respectively. SDM-L1/L2 out-performs the other two most of the time, while SDM-L1 norm is better when the Gaussian noise intensity is small.

We also test our sparse deformable model under different noise intensities. First, we increase the variance of Gaussian noise with fixed outliers. In Fig. 4(b), SDM-L1 performs the best when the noise intensity is low. As the variance increases, SDM-L1/L2 outperforms the others. LDM is always the worst, due to the outliers. It shows that our model is more stable when handling moderate Gaussian noise. Then we test the models with different numbers of outliers. In Fig. 4(c), the errors of all models increase almost linearly with the number of outliers. SDM-L1/L2, which is still the best among them, performs better than SDM-L1 when there are a few outliers. They achieve similar errors when the outliers are dominant. Both of the experiments show that SDM-L1/L2 is more stable under different noisy conditions.

5.2. Motion Analysis of Mouse LV

We also test our method on mouse myocardial strain analysis. The strain computation is especially sensitive to tracking outliers. Even when there are only small amounts of outliers on deformation, the strains on points near these outliers will be affected heavily. To obtain the ground truth, we manually label the tagging lines in each 2D image, and then use the tag motion to drive a 3D LV volume model based on finite element method (FEM). This approach is very accurate. However, manual labeling is time-consuming and tedious, and FEM is not very efficient. In this experiment, we use this method as reference, and compare our models using automatic tagging line tracking results that contain outliers. We compare the LDM, SDM-L1 and SDM-L1/L2. Table 1 shows the deformation errors of different models on 17 datasets. SDM with the combined norm has smallest average error owing to its robustness to outlier. Meanwhile, the results of SDM-L1 are much less stable than other two methods. The reason is perhaps that there is strong Gaussian noise in real data.

Table 1.

Quantitative evaluation of deformation errors (Unit: mm).

Method	Average	Min	Max	Median
LDM	1.036	0.724	1.635	0.927
SDM-L1	2.107	0.437	3.580	2.069
SDM-L1/L2	0.482	0.341	0.719	0.469

To further analyze the regional LV motion, we divide the LV into 17 segments, based on the standardized myocardial segmentation of the American Heart Association (Cerqueira et al., 2002). The heart is first sectioned into apical, mid-cavity, and basal parts perpendicular to the left ventricular long axis. Each part is further segmented based on different short-axis directions. We compare the deformation errors on all the segments. Fig. 5 shows 16 of them except the apex segment. The results show that SDM-L1/L2 always has the best accuracy, i.e., small average deformation errors and standard deviation. In contrast, SDM-L1 has worst accuracy in both aspects. Comparing among different parts based on the short axis directions, the lateral parts have relatively larger errors than the septal parts. This may because that lateral parts have larger movements. Comparing along the long axis direction, the basal part have relative larger errors in all the methods. The reason is that there is no short-axis slices in this part. All the point movements are calculated based on the control points from the middle part. The free ends at the basal boundaries produce more errors than other parts. The errors are smaller in the middle and apical parts in the results of LDM and SDM-L1/L2 due to the dense control points. However, they are still large for SDM-L1 in these parts. This shows that using L1 norm alone cannot represent the noisy tracking errors well, and L2 norm is necessary in this data.

Figure 5.

Box plot of the errors for different deformable models. In each box plot, the central mark is the median, the edges of the box are the 25th and 75th percentiles, and the whiskers extend to the extreme data points. The unit of Y-Axis is millimeter. X-axis means three compared methods: LDM, SDM-L1 and SDM-L1/L2.

We also compute the myocardial strains over a cardiac cycle, which are commonly used to describe the strength of the heart motion. The strains are decomposed into radial, circumferential, and longitude directions, and the shear strains among them. Fig. 6 provides the comparisons of the strains generated with different deformable models on several mouse datasets. The rows correspond to different types of strains, and the columns correspond to different mice. Each figure contains the strains generated from different models in a cardiac cycle. The numbers of frames in the cardiac cycles may be different on each mouse because of the acquisition procedures. The first column is from a healthy mouse, and the other two are from mice with myocardial infarction. It is shown that the strains generated from the healthy mouse are larger than from the unhealthy ones. For each individual dataset, the strains generated from the automatically tracked tagging lines are less stable than that from manually labeled ones, due to the tracking errors. The strains based on LDM are relatively smooth, but this method tends to underestimate the strains. The results from SDM-L1 have the worst stability because of its nature to pursue the sparse solution. Since the control points contain both outliers and strong Gaussian noise, the L1 norm cannot handle Gaussian noise stably. The results from SDM-L1/L2 best match the reference strains. This shows that our model performs well in the LV motion tracking even with inaccurate control points.

Figure 6.

Comparing the strains generated from different deformable models with the reference model in three mouse cardiac data. Each column represents one dataset, and each row represents one type of strain. In each figure, y-axis is the strain, and x-axis means the frame in a cardiac cycle.

The LV strain is estimated on dense sample points. Based on these motion results, strains at arbitrary positions inside the LV can be calculated by linear interpolation. In order to analyze the local heart motion properties, strains at the end of systole are projected to short-axis and long-axis slices in Fig. 7. We show the radial, circumferential, and longitudinal strains on the LV in the MR images. The strains change smoothly in each slice. Generally, the lateral side has larger strain than the septal side. The reason is that the right ventricle motion restricts the septal side motion.

Figure 7.

The first row is the short axis slice, and the second row is the long axis slice. The three columns are colored by the radial, circumferential and longitudinal strains, respectively.

We also visualize the strains on the external and internal surfaces of the LV. Since the points of the surface mesh are all in the initial volume model, where the strains are calculated, we use them as samples and linearly interpolate the strains on the LV surface. The circumferential strains are shown locally on the LV external and internal surfaces in Fig. 8. They indicate larger contraction near the endocardium than the epicardium. The high strain area starts from the apical endocardium and expands quickly toward the base, which is similar to human hearts.

Figure 8.

The deformations of the LV on a cardiac cycle are colored by the circumferential strain.

6. Conclusions

In this paper, we introduce a group of sparse deformable models. Benefitted from the sparsity techniques, these deformable models are able to handle outliers or gross errors. Therefore, they are robust to deal with noisy images or tracking errors. We have validated these methods on the synthetic data and the mouse cardiac motion tracking. Both qualitative and quantitative results demonstrate that our methods outperform and are more robust than previous ones. It is also noteworthy that the applications of our proposed methods are not limited to cardiac motion analysis. It is flexible enough for many other medical image problems.

In the future, we plan to extend the deformable models by using structured priors. The LV is conventionally separated into 17 segments. This inspires us to add group constraints to the current sparse model. The group sparsity and other structure sparsity constraints will further improve the robustness of the model. The current regularization term is only related to the external force based on noisy observations. It is also possible to extend the other parts of the model. The problem for modeling arbitrary internal force is that the resulting model may not be a convex problem. The traditional finite difference method can be employed to find a local minimum, while the performance should then be further analyzed.

1. We propose a new class of deformable models using sparse regularization.

2. It is able to handle gross errors or outliers robustly and adaptively during deformation.

3. It is formulated as convex optimization problems, and solved by harnessing sparse structures.

4. We apply this method to tackle a challenging problem, i.e., mouse LV motion analysis using tMRI.