3D Motion Modeling and Reconstruction of Left Ventricle Wall in Cardiac MRI

Abstract

The analysis of left ventricle (LV) wall motion is a critical step for understanding cardiac functioning mechanisms and clinical diagnosis of ventricular diseases. We present a novel approach for 3D motion modeling and analysis of LV wall in cardiac magnetic resonance imaging (MRI). First, a fully convolutional network (FCN) is deployed to initialize myocardium contours in 2D MR slices. Then, we propose an image registration algorithm to align MR slices in space and minimize the undesirable motion artifacts from inconsistent respiration. Finally, a 3D deformable model is applied to recover the shape and motion of myocardium wall. Utilizing the proposed approach, we can visually analyze 3D LV wall motion, evaluate cardiac global function, and diagnose ventricular diseases.

1 Introduction

Cardiovascular diseases, such as ventricular dyssynchrony, heart attack, and congestive heart failure, are one of the major causes for human death all over the world. A comprehensive analysis of 3D heart wall motion is fundamental for understanding the ventricular functioning mechanism, and essential for early prevention and accurate treatment of the related diseases. However, classical diagnosic tests, including electrocardiogram (ECG), echocardiography (echo), chest X-ray, and cardiac catheterization, are not able to provide sufficient spatial information with adequate resolution for motion modeling. The 3D echocardiography can be currently used to study cardiac motion and strains, but its visual appearance may not be clear due to limited imaging quality. In the present study, we adapt images from cardiac cine magnetic resonance imaging (MRI), which is an a non-invasive imaging technique to visualize the heart conditions both in time and space. The sequence of 2D MRI acquired along the long-axis (LAX) and the full cardiac cycle provides a complementary view of the shape and function of the left ventricle (LV) compared to the sequence of 2D MRI acquired across the short-axis (SAX) and time. Thus, analyzing a set of 2D cine MRI sequence can provide a feasible way to fully recover 3D LV wall motion.

In order to analyze the global function and the regional heart wall motion, the contours of the epicardium and endocardium of the LV must be annotated or delineated, either by human experts or machines. However, the annotation procedure is time-consuming and tedious for doctors and physicians, which then becomes a bottleneck for extraction of functional cardiac data in the clinical practice. An automatic method for LV segmentation (or contour extraction), which would reduce both manual labor and annotation time dramatically, has been sought for decades to increase the clinical efficiency of cardiac MRI. Recently, some scholars have proposed several methods to address them. For example, Paragios proposed a level-set method for cardiac MR segmentation [4] with the gradient vector flow and geodesic active contour model. Jolly also introduced an automatic segmentation method for both CT and MR images, using multi-stage graph cut optimization in the image plane [5]. In addition, Zhu et al. developed a statistical model, named subject-specific dynamic model (SSDM), to handle the cardiac dynamics and shape variation [6]. Although the ring-shaped structure formed by the paired epicardium and endocardium contours is fairly simple, the cardiac MR imaging quality can be inconsistent, because of factors such as different acquisition settings or potential artifacts introduced by respiration during the slow acquisition process. Furthermore, the endocardial contour is intrinsically somewhat ill-defined, due to the presence of the papillary muscles and trabeculations, which tend to be considered as part of the ventricular cavity. Thus, the contours of the LV wall segmentation may need to be estimated even when the local image contrast is partially corrupted; conventional intensity-based segmentation methods may fail in such cases. Moreover, the prevailing approaches [12–14] are mostly concerned with the SAX MR slices. Without further study of the LAX slices and slice alignment, the calculation of the global functions for the LV may not be accurate.

Removal of motion artifacts caused by varying respiration is another important issue to accommodate for analyzing the function of the heart. Although the cine MR sequences are captured at fixed spatial locations during breath-holding, it is unlikely that the respiration phase would remain the same at different slices of the cine MRI. The MR slices at different locations are inevitably misaligned with spatial offsets and in-plane deformation. Such misalignment issues can seriously affect the precision and representativeness of a 3D heart model that is built up on the unaligned MR sequences. Therefore, we need to solve this image registration problem between different MR slices. In [2], Lotjonen et al. proposed an alignment method maximizing normalized mutual information of image appearances between SAX and LAX slices. However, the optimization procedure is highly non-convex and easily falls into a local minimum. Garlapati et al. [11] proposed an effective method to solve the misalignment problems in brain imaging, based on the local boundary detection. It may not be applicable in our case because the boundary of LV wall is not always clear in cardiac MRI.

Assuming the in-plane segmentation and slice alignment are accommodated, 3D shape modeling and motion reconstruction of LV wall are the next steps in analysis. The 3D shape and motion provide quantitative and visual characteristics to study the normal and abnormal heart functioning mechanisms in a more comprehensive way, compared with echo. Park et al. studied the shape and motion of the LV using a volumetric deformable model based on tagging MRI [1]. The dynamic deformation of the ventricular wall is computed with Lagrangian dynamics and finite element method.

In this paper, we present a novel approach to reconstruct 3D shape and motion of the LV wall for understanding ventricular functioning mechanisms. First, we adopt a fully convolutional network (FCN) to extract epicardium and endocardium contours from the MR slices. Second, we develop a new algorithm to align MR slices in space, compensating the respiration effect. Finally, a deformable model is utilized to recover the 3D shape and motion of the LV wall with Lagrangian dynamics.

2 Myocardium Contour Extraction

In our approach, LV segmentation is defined as a pixel-wise semantic classification problem, that is, segmentation with class labels. The pixels of myocardium muscle within a semi-ring shape (formed by the epicardium and endocardium) are labelled as one class; pixels of blood pool and other contents are labelled as another class. We adopt the fully convolutional network (FCN), U-net [3], as the learning model following the end-to-end convention during the training and testing. The initial segmentation results are shown in Figure 1, which don’t all resemble the golden standard.

Fig. 1.

For example segmentation: the raw images (left) are segmented by FCN (middle), which are close to the gold standard (right).

We enforce strong shape constraints for the segmented contours resulting from the previous step, since the ring-shaped structure of the LV contours is an important prerequisite and the smoothness of contours needs further refinement. However, the raw prediction from the FCN sometimes forms ring-shapes with unreasonable patterns, e.g., zig-zag curves or the intersection of two contours, as shown in the fourth example of Figure 3. The initial shape is generated from the shape pool and can be reliably placed in the image plane even when the appearance cue is misleading. The shapes of the LV wall vary from the phase of end-diastole (ED) to that of end-systole (ES), and from the slices near the aorta to those near LV apex. For instance, the contours close to the aorta may be partially merged together, particularly in the membranous portion of the interventricular septum, and the myocardium muscle close to the LV apex is thinner compared to the muscle at other locations (although the typically oblique intersection of the image plane with the apical LV wall in SAX images can result in apparent increased wall thickness, due to volume averaging). We cluster training shapes into different groups by the geometry and muscle thickness, and compute the refined contours of testing data by optimizing the dictionary learning formulation with the group sparsity constraints shown in Equation 1:

$$\underset{x,e,\beta }{\text{minimize}}\left\{{\Vert T(y,\beta )-Dx-e\Vert }_2^2+{\lambda }_1\sum _{s\subseteq S}{\Vert x_s\Vert }_2+{\lambda }_2{\Vert e\Vert }_1\right\}$$ (1)

where T(y, β) is the similarity transformation with parameter β for aligning the initial shape y, generated by FCN, to the mean shape of the shape pool. Matrix D = [d₁, d₂, ···, d_k] represents the training shape pool, column vector d_i ∈ ℝ³ⁿ contains the coordinates of n vertices on the contours. S = {1, 2, ···, k} is the set of indices of x. The clustering process divides S into several non-overlap subsets, S = ∪_i S_i, S_i ∩ S_j = ∅, ∀i ≠ j. Vector x ∈ ℝ^k contains the weights for the linear combination of shapes in the pool. x_Si is the sub-vector for the group S_i ∈ S, and the term Σ_Si⊆S||x_Si||₂ is a standard group-sparsity regularization (l_2,1 norm). Vector e models the non-Gaussian error in the case that partial contour information is missing. λ₁ and λ₂ control the weights of two sparsity terms. After solving the optimization, the myocardium contours are refined with the most correlated shapes from a small group of shapes from the training pool. The sample results are shown in Figure 3. The similar process is conducted for the LAX slices as well.

Fig. 3.

Four sample results before and after applying the group sparsity constraints: red contours are the results from the proposed fully convolutional network (FCN), green ones are the refined results after applying group sparsity constraints.

3 Rigid Image Registration for Spatial Alignment

The heart motion under respiration is mainly a rigid-body translation in the craniocaudal (CC) direction, with minimum deformation[7]. Therefore, we assume that in-plane rigid translation is sufficient to compensate the respiration effect for SAX. We also assume the offset of one cardiac phase in a slice can be applied for all cardiac phases at the location, because the respiration phase is almost identical in one-slice acquisition with breath-holding. For simplicity, the registration is carried out only at the state of end-diastolic (ED) for all slices simultaneously.

We propose a novel slice alignment algorithm, described in Algorithm 1, to adjust both SAX and LAX slices, using the contours from the previous step and slice intersection relations. Since SAX slices are almost parallel to each other, we take intersections between SAX and LAX slices, or different LAX slices, into consideration. At SAX slice s, the corresponding image plane is T_s and 2D contours (epicardium and endocardium) are v_s. Contours v_l in LAX slice l have intersection points p_l with slice s. Then, the closest points p_s ∈ v_s are computed corresponding to all points in p_l. ||p_s – p_l||₂ = 0 ideally if no respiration effect exists during the acquisition. However, as shown in Figure 4, p_s and p_l may not intersect with each other. The difference p_s – p_l provides the direction to shift the image plane (or shift the contours equivalently). Computing all the intersection points from LAX contours, the final translation displacement can be determined by taking the average on p_s – p_l. The procedure is analogous for LAX slices. The whole procedure is repeated if the marginal update of alignment is greater than a fixed threshold. The complete algorithm, shown in Algorithm 1, is guaranteed to converge to a stable condition where most intersection points are on the in-plane contours among all slices and frames.

Algorithm 1.

Joint alignment of 2D MR short- and long-axis slices

Fig. 4.

Results (before and after) MR slice alignment. (a,b): SAX myocardium contours and intersection points with LAX contours; (c,d): LAX myocardium contours and intersection points with SAX contours; (e,f): all contour points in 3D space; bottom: four sample slices with intersection points before and after alignment.

4 3D Shape Modeling and Motion Reconstruction

Deriving 3D shape and motion of LV wall from the well-aligned contours of different slices is essential for understanding heart functioning mechanism. Analyzing motion of a sequence of 2D contours along an axis and time is able to show some characteristics of heart motion. However, 2D image slices, at the same location but at different phases of the cardiac cycle, actually may present different parts of heart, due to the 3D ventricular motion. Thus, the sequence of 2D MRI slices does not show the true pattern of heart dynamics (shape, strain, etc.). In order to achieve better analysis, we recover the 3D LV wall shapes over the whole cardiac cycle from the sparse in-plane contours. We propose a new method, shown in Algorithm 2, to reconstruct 3D LV shapes and motion, adopting the deformable model. We use the rigid point-wise registration method, coherent point drifting (CPD) [8], to initialize the 3D shape for the cardiac phase of end-diastolic (ED) from a reference shape towards the aligned contours in space. The shapes for the whole cardiac cycle are computed along the direction from ED to end-systolic (ES). Next we construct the deformable model directly on the triangular mesh from results of CPD registration. The point locations of the deformable model are a function of time t and vector q:

$$x(q,t)=c+R(s+d)$$ (2)

where c is the origin of local coordinates, R is a rotation matrix, s and d are global and local deformation, respectively. q is defined as a vector of parameters in kinematics and dynamics and ẋ = Lq̇, where matrix L is derived from Equation 2. According to Lagrangian dynamics, we have the following equation:

$$D\stackrel{.}{q}+Kq=f_t$$ (3)

where D is the damping matrix, and K is the stiffness matrix. The external force f_t at phase t is proportional to the Euclidean distance between contour points and initial shape S within a local neighborhood. Once we have the initial shape, we can update the deformable model and the corresponding mesh by solving Equation 3. Therefore, the shape at each phase can be computed using the computed shape of the previous phase as initialization for deformation (Figure 5). Then, we can recover the whole motion of the LV wall phase-by-phase with proper smoothness (guaranteed by the deformable model).

Algorithm 2.

LV wall motion computation over the whole cardiac cycle.

Fig. 5.

3D yellow models are the LV models, red curves are the 2D aligned contours from SAX and LAX slices in space. (a) Initial model from the referenced LV model at the phase of ED using CPD; (b) fitted model for the phase of ED using deformable model based on the contours; (c) LV model at the phase k – 1 and contours at the phase k; (d) final fitted model at phase k.

5 Experiments

We used a cardiac MRI dataset containing MR image sets of 22 normal volunteers and 3 patients for the initial study. The patients all had heart failure with dyssynchrony, and were scheduled for cardiac resynchronization therapy (CRT). We manually annotated LV contours for all LAX and SAX images at each location over different cardiac phases, except the slice planes that did not cut through the LV. Image size varied between 224 × 204 pixels and 240 × 198 pixels, and its resolution varied from 1.17 mm to 1.43 mm. In total, 25 subjects (approximately 5625 images, both SAX and LAX) were used from our dataset, randomly divided into training set (20 subjects) and testing set (5 subjects). The SAX and LAX network models were trained separately. A 5-fold cross-validation was used in the training set. We compared the results for FCN and the proposed methods, using Dice’s coefficient as the evaluation metric for segmentation. The result in Table 1 shows that the proposed method has better performance than FCN, because the shapes of output contours are regularized. For the motion reconstruction, some manual adjustment of segmented contours is necessary in terms of accuracy, which takes a few minutes for each case, on average. Once the adjustment of contours is finished, we conduct the processing steps without any further update for the contours.

Table 1.

Evaluation results

Dice’s coefficient	our dataset	challenge dataset
U-Net (mean)	0.70	0.53
U-Net (std)	0.07	0.15
proposed method (mean)	0.86	0.70
proposed method (std)	0.04	0.12

We also evaluated our methods with the public dataset from the cardiac MRI segmentation challenge of MICCAI 2009 [10], as well. The dataset, from the Sunnybrook Health Sciences Center, contains 45 cine SAX slices, covering both normal and abnormal cardiac conditions. Image size is 256 × 256 pixels, and its resolution varies from 1.2500 mm to 1.3672 mm. Expert annotations of endocardium and epicardium contours are provided for some slices at ED-V and ESV phases. We only evaluated the cases where both endocardium and epicardium annotations are given for the same image. The dataset is divided into three subsets: training, validation, and online, following the standard nested cross-validation. We trained our model with the training set (135 images), evaluate the model with the evaluation set (138 images) and tested with the online set (147 images). Accuracy was measured with the Dice’s coefficient, as well shown in Table 1. The accuracy is slightly less than previous experiments since the training set is fairly small.

The average distance between the contour points and the reconstructed model is utilized as the metric to evaluate the performance of the rigid alignment. The result for the whole dataset along the cardiac phase is shown in the Figure 7. We find that the distance at each time point is much smaller when applying alignment than that without any alignment. This means our alignment strategy well improves the consistency of contours in 3D space well. The model from the aligned contours is also improved, as shown in Figure 8.

Fig. 7.

The average distance (in mm) between contour points and reconstructed model along the full cardiac cycle for the whole dataset.

Fig. 8.

Left: the model reconstructed from the contours without aligned; right: the model from the contours with alignment. The model shape with alignment becomes more proper and smooth comparing result without alignment.

Figure 6 shows the reconstructed shapes at different frames of the cardiac cycle. There is a clear difference between LV motions of normal volunteers and those of patients with heart dyssynchrony. In the ES phase, the LV contracts well to pump the blood out for normal people; whereas, it does not deform as much for patients, which means the patients’ hearts are unable to function properly. Based on the reconstructed model, we can study the LV volumes along time for normal volunteers and patients shown in Figure 9. Comparing with normal people, the patient’s LV contains more blood and it does not contract much during the cardiac motion (which also can be proved by the ejection fraction rate: 55% for a normal volunteer and 28% for a patient). 2D myocardium contours in tagged MR slices, which are useful for further studying the interior dynamics of the LV wall, can also be located and mutually registered, based on the reconstructed 3D LV model and its intersection with the MR planes (as shown in Figure 10).

Fig. 6.

Two views of LV model at three frames: first row for a volunteer, second row for a patient.

Fig. 9.

LV volume change along time within a full cardiac cycle for a normal volunteer and a patient with heart dyssynchrony.

Fig. 10.

Intersected contours of fitted model on a tagged MRI slice.

6 Conclusion

In the paper, we proposed a novel approach to reconstruct 3D shape and motion of LV wall from cardiac cine MRI. The approach is effective and efficient with few user interactions. We may extend the proposed approach to the tagged MRI, whose alignment is still challenging due to the imaging artifacts. We call for the future extension to explore our approach to other potential applications.

Fig. 2.

Left: clusters in the shape pool; right: mean shape.