Automatic estimation of aortic and mitral valve displacements in dynamic CTA with 4D graph-cuts

Abstract

The location of the mitral and aortic valves in dynamic cardiac imaging is useful for extracting functional derived parameters such as ejection fraction, valve excursions, and global longitudinal strain, and when performing anatomical structures tracking using slice following or valve intervention’s planning. Completely automatic segmentation methods are still challenging tasks because of their fast movements and the different positions that prevent good visibility of the leaflets along the full cardiac cycle. In this article, we propose a processing pipeline to track the displacement of the aortic and mitral valve annuli from high-resolution cardiac four-dimensional computed tomographic angiography (4D-CTA). The proposed method is based on the dynamic separation of left ventricle, left atrium and aorta using statistical shape modeling and an energy minimization algorithm based on graph-cuts and has been evaluated on a set of 15 electrocardiography-gated 4D-CTAs. We report a mean agreement distance between manual annotations and our proposed method of 2.52±1.06 mm for the mitral annulus and 2.00±0.69 mm for the aortic valve annulus based on valve locations detected from manual anatomical landmarks. In addition, we show the effect of detecting the valvular planes on derived functional parameters (ejection fraction, global longitudinal strain, and excursions of the mitral and aortic valves).

1. Introduction

Detecting and tracking cardiac valve annuli and cardiac valvular planes in dynamic cardiac imaging is a necessary task to derive functional parameters, such as the ventricular ejection fraction, valve excursions, or the global longitudinal strain. In addition, it allows slice following to track dynamically the same anatomic slice through the cardiac cycle and allows the automatic selection of data around the valves, e.g., for surgery planning, or for three-dimensional (3D) printing of a model of the valve (Giannopoulos et al., 2016; Vukicevic et al., 2017). Furthermore, the geometry, tracking, and movement analysis of the cardiac valves are crucial to assess the functional implications of several cardiovascular conditions in early stages, such as hypertension and dilated cardiomyopathy (Carlsson et al., 2007; Helein and Gibson, 1999; Romano et al., 2018).

To provide an accurate assessment, the cardiac valves must be anatomically and functionally characterized. This can be achieved by applying a heart model that integrates the anatomy of the valve located by the best-fitting plane as has been widely used in manual and semi-automatic segmentation and quantification protocols (Lang et al., 2015). A more elaborate solution would be a dynamic model that includes the complete anatomy of the valve, including the movements of leaflets along the entire cycle (Sun et al., 2014).

Regarding the mitral valve, an intermediate approach is the modeling of the mitral annulus, defined as the junctional zone providing attachment to the mitral valve, separating the left atrium (LA) from the left ventricle (LV) (McCarthy et al., 2010). The mitral annulus is traditionally modeled as a saddle-shaped ring (Levine et al., 1989), although alternatives such as the D-shaped model (Blanke et al., 2014) have also been proposed. As with the mitral valve, there are different approaches to modeling the aortic valve. It is commonly approximated by a ring delimited by different anatomical structures, such as the sinotubular junction, ventriculo-arterial junction, or virtual ring joining the hinge points (Piazza et al., 2008). Furthermore, the aortic valve annulus is also described as the semilunar crown-like structure demarcated by the hinges of the leaflets (Piazza et al., 2008). Regardless of the approach taken to model and characterize them, accurate detection and delineation of mitral and aortic valves in dynamic cardiac imaging is a complex task because they contain rapidly moving structures. Specifically, valve leaflets experience rapid movements through the cardiac phases that are not always visible.

The challenge of segmenting or tracking cardiac valves in 3D dynamic images (denoted as 3D+t or 4D) can be found as an individual objective or as a part of a more general proposal that also encompasses the automatic segmentation of heart chambers. Because mitral and aortic valves limit the upper part of the LV, a segmentation of the LV leads directly to the associated valvular model. However, generally, LV models do not follow the LV’s complex shape in the area near the valves (Buckberg et al., 2008); thus, the LV outflow tract (LVOT) is left outside the model. Therefore, the state-of-the-art methods for shape modeling the LV do not track mitral and aortic valves independently because the upper part of the LV is modeled as a single opening with circular topology (Suinesiaputra et al., 2018). To have a complete model, it is necessary to include at least the LA and ascending aorta (AA) to be able to model the mitral and aortic valves as a secondary result of a multichamber segmentation approach. The multichamber cardiac segmentation has traditionally been performed by active shape models (ASMs) and deformable methods to extract the anatomy and contractile function of the heart. A complete survey can be found in (Tavakoli and Amini, 2013). More recently, convolutional neural networks (CNNs) have also been proposed for multichamber cardiac segmentation because of their outstanding performance in segmentation problems (Bai et al., 2018; Payer et al., 2018).

In addition, 4D computed tomographic angiography (4D-CTA) is a very useful tool to complement echocardiography and magnetic resonance imaging (MRI) in cardiac imaging (Maurer et al., 2012; McVeigh et al., 2018; Ohnesorge et al., 2000; Storz et al., 2017), providing 3D+t high-resolution scans covering a whole heartbeat cycle, making it especially recommended in the task of assessing the heart valves (Chheda et al., 2010). However, although cardiac structures can be best visualized in 4D-CTA relative to echocardiography and cardiac MRI, providing a superior depiction of its anatomy and dynamics, segmentation, and tracking of cardiac chambers and valves in 4D-CTA usually requires user interactions, as anatomical landmarks need to be manually located to preregister the shape model (Von Berg and Lorenz, 2005). Moreover, high sensitivity to initialization and subjective landmarks annotations in training may reduce the performance of automatic methods (Ecabert et al., 2008; Peters et al., 2010; Zheng et al., 2008). More recently, a semiautomated method based on the registration of a template mesh has been proposed for canine studies (Vigneault et al., 2018).

Because of the arduous task of manual segmentation of cardiac valves, especially when complex models are involved, several works have been proposed to achieve this goal automatically despite its inherent difficulty. A patient-specific constrained shape model of both mitral and aortic valves was proposed by Ionasec et al. (Ionasec et al., 2010) for 4D-CTAs transesophageal echocardiographic images that started from a landmark model refined by a full physiological surface valve model constrained by the previously defined landmarks and also used steerable features (Zheng et al., 2008). This work was later extended to the location and study of the motion of the four heart valves (Grbic et al., 2012). The mitral valve alone has been studied in (Voigt et al., 2011) from transesophageal echocardiographic images, where the valvular model is achieved by combining discriminative learning and leaflet biomechanics. The dynamic segmentation of aortic valve, including the root wall and leaflets, has also been proposed (Auricchio et al., 2011) and uses a combination of elliptical shapes to model the leaflet-sinus complexes (Rankin et al., 2013).

These aforementioned methods require basic training that must be manually determined across the cardiac phases, but given the complex nature and variability of cardiac data, methods based on ASM or active appearance models have problems with the accurate 3D+t representation of size variations of cardiac chambers, as well as the limited spatiotemporal representation of the 4D behavior. In addition, models have limitations in the representation of pathological cases (Hoogendoorn et al., 2013). A database correctly labeled by experts is also necessary to create the model and should include pathological cases if the latter were also to be segmented. New proposals based on CNNs and deep learning need even larger labeled data sets to train the network, but these large data sets are not always available.

To overcome these limitations, graph-cut methods (Boykov et al., 2001) have been used to solve different segmentation problems in cardiac imaging. Graph-cuts were developed to address a wide range of problems in computer vision using graph theory. The main idea is to minimize an energy cost function by finding the minimum cut of a graph. For an in-depth understanding of the graph-cut segmentation approach, see (Boykov and Funka-Lea, 2006).

As mentioned above, graph-cuts have been used to segment the endocardial wall of the LV in MRI and echocardiographic 3D images (Bernier et al., 2017). If we focus on 4D data, several methods have been proposed: a coarse 4D-graph-cut, later refined using competitive region growing, was used for segmentation of cardiac chambers in dynamic 4D MRI data (Lombaert et al., 2011). Other 4D methods use a spatiotemporal watershed cut for LV myocardium segmentation (Cousty et al., 2010), as well as LV segmentation in MRI with smoothness and interslice constraints (Albà et al., 2014). Apart from cardiac imaging, 4D-graph-cuts have also been proposed for the segmentation of brain structures in longitudinal MRI data (Wolz et al., 2010).

The main contribution of this work is the proposal of a processing pipeline to locate and track the displacement of the aortic and mitral valve annuli and associated planes from cardiac electrocardiography (ECG)-gated 4D-CTAs based on a 4D-graph-cut algorithm. Concretely the pipeline targets to locate the posterior segment of the mitral valve (D-shape plane) (Blanke et al., 2014) and the virtual ring defined by the aortic hinges (Piazza et al., 2008). The acquisition protocol of 4D-CTA ensures that the injected contrast is limited to the left chambers and aorta; therefore, a mesh can be built by isolating regions with contrast. A 4D graph is then constructed by encompassing these meshes and the temporal information between time frames. We can detect and track the mitral and aortic valvular annuli after segmenting LV, LA, and aorta using a unique 4D-graph-cut algorithm for the whole cardiac cycle. Our method aims to be fully automatic and not dependent on a large training database, implying that no ASM should be used in the final step (the refinement stage). However, because of the need for robust initialization, we use an ASM for the initial location of the cardiac chambers.

We have tested our processing pipeline with 15 patients comprising a total of 278 3D scans. To validate the usefulness of our proposal, functional derived parameters that are influenced by the correct valve position during the cardiac cycle are computed and compared with the results obtained from manually detected valves. In addition, the error in the valve location is assessed using manually selected landmarks.

2. Methods

In this section, we present a new method to locate and automatically estimate the displacement of mitral and aortic valve annuli in ECG-gated 4D-CTAs. Left chambers are also segmented as an auxiliary result. The proposed workflow follows the main steps summarized in Fig. 1. The following sections detail the different steps in the process.

Fig. 1.

Basic steps of the proposed workflow: A) a bimodal tissue vs. blood gamma mixture model (GMM) is estimated from a 4D-CTA to differentiate tissues vs. contrasted blood in the cardiac region, followed by the segmentation, for every time frame, of a binary mask comprising the blood pool of the left chambers, with high values of intensity because of the contrast; B.1) an active shape model (ASM) is used to fit the left ventricle (LV) and atrium-aorta separation plane to ensure a robust initialization of the graph-cut; B.2) triangular meshes of the binary mask (gray color) are calculated for all the time frames in the sequence (shown for one single frame) that are processed to remove structures not belonging to the left chambers (red color); C) a surface-based temporal registration of the meshes allows the construction of a 4D graph containing spatial edges from the meshes and temporal edges from the registration. Seeds for the graph-cut are shown as color points; D) a final 4D-graph-cut segmentation is performed, separating ascending aorta, left atrium, and LV, to obtain the mitral and aortic valve annuli in all frames and associated planes (shown for one single frame).

2.1. GMM and binary mask

The contrast agent highest concentration is found in the oxygenated arterial blood when the ECG-gated images are acquired. Therefore, left chambers, aorta, pulmonary veins, and coronary arteries present high values of Hounsfield units (HU). After removing background and lungs regions with Hounsfield values (also referred as CT numbers) closer to air values (HU = −1000), the remaining voxels are grouped in clusters values associated with soft tissues and blood without contrast, oxygenated blood with contrast, and bone. To differentiate between tissues and regions with the contrast agent, the image is modeled as a GMM.

GMMs have been extensively used for the characterization of tissues in ultrasound imaging (Vegas-Sanchez-Ferrero et al., 2014). Recently, the GMMs have also been studied for the characterization of CT numbers in CT imaging and were proven to be excellent models to describe the statistical properties of CT numbers for different doses and reconstruction kernels (Vegas-Sánchez-Ferrero et al., 2017). The characterization with GMMs allows us to calculate the posterior probability of belonging to each component of the mixture model. In our context, we take advantage of the GMM characterization to calculate the probability distribution associated with the contrast agent instead of the original CT numbers. For brevity, in the rest of the proposed workflow, we will refer to this posterior probability as the gamma map.

The proposed workflow uses a triangle mesh of the surface of the left chambers (LV and LA) and AA obtained from the binary mask. This mask is calculated with an automatic thresholding algorithm of the gamma map (Otsu, 1979) that was used to separate high-intensity structures (chambers and arteries with contrast agent and bone) from the rest of the tissues.

Some postprocessing is required to obtain the binary mask comprising the LV, LA, and AA: first, a morphological closing is performed to soften the surface to eliminate irregularities caused by image noise and complex topology caused by the LV trabeculae and papillary muscles; second, the largest segment containing the left chambers and the AA is selected, discarding other subregions that include bones and other structures with high values in the gamma map; third, a morphological opening removes the coronary arteries. Lastly, all possible holes in the remaining mask are filled to get a closed and topologically simple isosurface prior to the extraction of a triangle mesh.

2.2. ASM of the LV

To obtain a first automatic location of the LV, the blood pool of the LV is detected and delineated using a statistically based shape model of the endocardium of the LV, and this model is derived from the model of the whole human heart (Hoogendoorn et al., 2013).

We first locate the LV model by using Horn's quaternion-based method (Horn, 1987) to find the rotation, translation, and scaling that optimally map the vertices of the shape model to candidate locations extracted from the gamma map. The fitting is performed by the weighted least squares method where the candidate points are selected in the normal directions of the surface that show the maximum gradient of the gamma map (i.e., the most likely border). The weights are also derived from the gamma map. For a more detailed explanation, we refer the reader to (Haak et al., 2015). The fitting method is performed iteratively until the estimated error reaches a given threshold. In the next step, the modes are projected using a weighted regularized ASM (Zhao et al., 2004) to estimate the LV pose.

2.3. Triangle mesh

In the following step, we extract the surface of the binary mask described in Section 2.1 parametrized as an unordered triangular mesh (Qianqian and Boas, 2009). The meshing algorithm implementation is described in (Rineau and Yvinec, 2007) based on the previous work of (Boissonnat and Oudot, 2005), choosing 1.5 mm as the maximum radius of the Delaunay sphere. The mesh is corrected to remove existing duplicated, isolated, and nonmanifold nodes to obtain a single watertight triangle mesh without singularities, self-intersections, and degenerate elements (Attene, 2010; Attene et al., 2013). A mesh of approximately 2 mm edges is later obtained using a remeshing algorithm that iteratively cuts small edges and subdivides larger ones.

The resultant mesh may also contain remaining structures associated with pulmonary veins and coronary arteries with contrast. These structures, although small in volume, can contain a high percentage of triangles; therefore, these structures are removed before the main graph-cut algorithm to ensure robustness and efficiency. These structures are segmented using the shape diameter function (SDF) introduced in (Shapira et al., 2008), which estimates the neighborhood diameter of the mesh as an approximation of the medial axis transform. On a smooth surface, the diameter can be defined by the distance to the antipodal surface point along the inward-normal direction. To reduce the variance because of noisy and low-resolution surfaces, the SDF (Shapira et al., 2008) takes the mean value of intersection distances over directions inside a cone centered in the inward-normal direction. A clustering algorithm then segments the surface regions with small shape diameter values, e.g., coronary arteries.

A coherent point drift (CPD) algorithm for nonrigid point set registration (Myronenko and Song, 2010) was used to register pairs of meshes in consecutive frames, being the barycenter of the faces the points used in the CPD algorithm.

2.4. Graph-cut formulation

To solve the separation of the different structures in the sequence, a 4D (also referred as 3d+t) graph-cut approach is proposed, establishing both spatial and temporal edges to impose both spatial and temporal coherence by combining the information of all the frames. An undirected graph G = ⟨V,E⟩ is composed of a set of nodes V and a set of edges E , where each edge connects two nodes {p,q}∈E : p,q∈V. Therefore, the topology of a triangle mesh can be formulated as an undirected graph if the triangle faces are identified as nodes and the neighboring ones are connected by edges. In addition, the triangle meshes of all the time frames can be joined in a single undirected graph to achieve the 4D structure in such a way that a set of nodes V is composed of all triangle faces from all N frames: $V\equiv {\bigcup }_{n=1}^N{V}_n$, where V_n is the set of nodes (i.e., triangle faces) obtained from the time frame n.

We define the set of edges E by considering two types of connections: edges connecting two neighboring faces of the same triangle mesh associated to a time frame (E^s), and edges connecting two triangle faces belonging to two consecutive time frames that were registered by using the previously described CPD algorithm (E^T). For clarity, we call them spatial and temporal edges, respectively, and we will use superscript S for spatial terms and T for temporal terms. Decomposing by time frames, the set of nodes and edges G = ⟨V,E⟩ can be expressed as:

$$\begin{gathered} V\equiv \bigcup _{n=1}^N{V}_n, \\ E\equiv \left\{\bigcup _{n=1}^N{E}_n^S\right\}\cup \left\{\bigcup _{n=1}^{N-1}{E}_{n,n+1}^T\right\}, \\ where{E}_n^S\equiv \{\{p,q\}\in E:p,q\in V_n\}\text{and}{E}_{n,n+1}^T\equiv \{\{p,q\}\in E:p\in V_n,q\in V_{n+1}\}. \end{gathered}$$ (1)

In the preceding expression, $E_n^s$ is the subset of spatial edges connecting nodes of the time frame n, and $E_{n,n+1}^T$ is the subset of temporal edges connecting nodes of consecutive time frames n and n+1.

The complete graph G = ⟨V,E⟩ encodes a 4D structure because it includes spatial edges associated with the 3D triangular meshes and temporal edges encoding the temporal relation between 3D meshes extracted from the different frames. Fig. 2 schematizes this 4D graph.

Fig. 2.

4D Graph extracted from triangle meshes and the coherent point drift (CPD) algorithm comprising spatial and temporal information: triangle meshes corresponding to three time frames are represented at the top of the figure, and a small portion of them is schematized at the bottom for clarity. The nodes of the graph, represented by red dots, correspond to the triangle faces, while the edges of the graph can be of spatial type (red lines in the figure) when connecting nodes of the same frame or temporal type (orange lines in the figure) when connecting nodes of consecutive frames. Examples of edges are shown for nodes {a,b,c,d} are as follows: $\{a,b\}\in E_n^S$, $\{d,e\}\in E_{n+1}^S$ are static edges, while $\{b,c\}\in E_{n,n+1}^T$ and $\{c,d\}\in E_{n+1,n+2}^T$ are temporal edges.

The segmentation of the graph G is formulated as a graph-cut algorithm (Boykov et al., 2001) to find a labeling f of some finite set that assigns a label f_p to each p∈V by minimizing an energy function E(f) with two cost terms:

$$\mathrm{E}(f)={\mathrm{E}}_{bound}(f)+{\mathrm{E}}_{reg}(f).$$ (2)

The boundary term, denoted as E_bound, accounts for the edges connecting the nodes in which f is not piecewise smooth, while the regional term E_reg describes the disagreement between f and the observed data. These terms can be calculated as a summation of costs over edges and nodes:

$$\begin{gathered} {\mathrm{E}}_{bound}(f)=\sum _{\{p,q\}\in E}{B}_{\{p,q\}}\left(f_p,f_q\right), \\ {\mathrm{E}}_{reg}(f)=\sum _{p\in V}{R}_p(f_p), \end{gathered}$$ (3)

where B_{p,q}(f_p, f_q) is the boundary cost of assigning labels {f_p, f_q} to the edge {p,q} and R_p (f_q) is the regional cost of assigning the label f_p to the node p. The detailed sum of boundary costs, distinguishing between temporal and spatial edges, is expressed as:

$${\mathrm{E}}_{bound}(f)=\sum _{n=1}^N\left(\sum _{\{p,q\}\in E_n^S}{B}_{\{p,q\}}^S\left(f_p,f_q\right)\right)+\sum _{n=1}^{N-1}\left(\sum _{\{p,q\}\in E_{n,n+1}^T}{B}_{\{p,q\}}^T\left(f_p,f_q\right)\right),$$ (4)

where the boundary cost terms $B_{\{p,q\}}^S$ are for spatial edges and the boundary cost terms $B_{\{p,q\}}^T$ are for temporal edges. Using expressions (4) and (3), the components of (2) can be expressed as:

$$\mathrm{E}(f)=\sum _{n=1}^N\left(\sum _{\{p,q\}\in E_n^S}{B}_{\{p,q\}}^S\left(f_p,f_q\right)+\sum _{p\in V_n}{R}_p\left(f_p\right)\right)+\sum _{n=1}^{N-1}\left(\sum _{\{p,q\}\in E_{n,n+1}^T}{B}_{\{p,q\}}^T\left(f_p,f_q\right)\right).$$ (5)

Expression (5) is general for graph-cut algorithms with boundary and regional costs defined over 4D graphs (i.e., graphs with spatial and temporal edges). Next, we describe the specific values and expressions of the temporal boundary terms $B_{\{p,q\}}^T$, spatial boundary terms $B_{\{p,q\}}^S$, and regional terms R_p used in our proposal.

The temporal boundary terms $B_{\{p,q\}}^T$ have a constant value independent of the edge {p,q}, (however, their relative value with respect to the rest of the edges and their topological relation affect the solution). Consequently, every two triangle meshes temporally connected by the CPD registration algorithm have the same probability of belonging to the same anatomical regions at different frames. In contrast, spatial boundary terms $B_{\{p,q\}}^S$ consider the curvature of the edge, denoted as $B_{\{p,q\}}^{curv}$, and the gamma map in the spatial neighborhood of the edge location, denoted as $B_{\{p,q\}}^{gamma}$. This dual contribution is a sum where constants α and β are weighting each contribution:

$$B_{\{p,q\}}^S=\alpha B_{\{p,q\}}^{curv}+\beta B_{\{p,q\}}^{gamma}.$$ (6)

In the following sections, the spatial components $B_{\{p,q\}}^{curv}$ (shape curvature cost) and $B_{\{p,q\}}^{gamma}$ (gamma value cost), as well as the regional costs R_p, are described. The temporal boundary term is constant and only dependent on the CPD algorithm output.

2.4.1. Shape curvature cost function term

The curvature of the triangle mesh can be used to identify the valve locations, as shown in Fig. 3. Note that the valve location is usually characterized by an indentation of the endocardial surface. Therefore, the cost associated with curvature should be minimal in concave areas of the surface. Each term $B_{\{p,q\}}^{curv}$ is calculated as follows: first, we estimate the curvature tensor in the surface, as described in (Alliez et al., 2003); then, we regularize spatially the resulting tensor field though a local average of the curvature tensor (elementwise) in a local neighborhood defined around each vertex; lastly, the lowest valued eigenvalue of the curvature tensor is scaled to the range [0,1] in the entire triangle mesh, and the value in each edge is then calculated as the mean value in its two vertices. The minimum curvature was chosen as it presented a greater contrast at the valve indentation areas (see supplementary Fig. S3).

Fig. 3.

Surface maps of boundary costs in a single time frame: (a) gamma value cost function terms B^gamma; (b) shape curvature cost function terms B^curv. Small function cost terms are detected along the mitral and aortic valves. Displayed values have been normalized to the [0,1] range.

2.4.2. Gamma value cost function term

The information around the surface can provide useful information to identify the location of the valve. Therefore, we consider the surrounding samples of the gamma map.

To retain the advantages of the fast segmentation method provided by the triangle mesh strategy, we perform a fast sampling strategy of the gamma map around each edge, searching for minimal values along directions randomly distributed around the normal of the edge. This search aims at finding valve leaflets in open or closed positions, which have small values in the gamma map. An example of this procedure is shown in Fig. 4, and the detailed procedure is as follows:

Fig. 4.

The gamma value cost function term $B_{\{p,q\}}^{gamma}$ is calculated by sampling the volumetric gamma map around each spatial edge {p,q}. The sampling scheme is performed inside a cone oriented along the vector normal to the triangle mesh surface. (a): an example of the sampling cone for one single edge is drawn in violet color, along with the triangle mesh of a single time frame; (b) zoomed view of (a) showing the normal vector (red arrow) to the edge marked with a black bold segment, as well as the random directions inside the cone, shown with blue arrows, in which the gamma map is sampled; (c) CTA slice of a systolic frame showing a cone encompassing leaflets of the mitral valve; (d) CTA slice of a diastolic frame, where the open leaflets of the mitral valve are not clearly visible inside the cone; (e) surface map of B^gamma cost function terms in a systolic frame, where the closed mitral valve causes low B^gamma values, while the opened aortic valve slightly decreases the B^gamma values; (f) surface map of B^gamma cost function terms in a diastolic frame showing slightly lower B^gamma values in the edges associated with the opened mitral valve and showing a more clearly distinguishable aortic valve.

1. Given the outward-pointing normal vectors n_p and n_q of the faces of the triangle mesh associated with the edge {p,q}, the vector n_{p,q} = −(n_p + n_q) is normal to the edge and points toward the interior of the mesh.

2. The central point of the edge {p,q}, calculated from the coordinates of its two associated vertices in the mesh, and the vector n_{p,q} define the vertex axis of a cone with an aperture angle δ between 30 degrees and 60 degrees.

3. N vectors r_n are randomly calculated inside the cone to avoid undesirable patterns derived from uniform sampling when N is small. Then, the gamma map is sampled across N segments of length d following directions of the vectors from the central point of the edge.

4. The average of the samples is calculated per segment, and the cost is set to the minimal average across edges per node.

2.4.3. Regional cost function term

To calculate the regional terms R_p in expression (5), a seed belonging to each chamber (i.e., AA, LA, and LV) located away from the valve planes that are going to be determined is placed automatically given the ASM fit. The triangle faces selected as seeds and their local neighborhood contribute to a negligible regional cost when the correct chamber label is used, and the maximum regional cost is when the wrong label is assigned corresponding to one of the classes (sink or source). The regional cost for the rest of the triangle mesh is calculated using the geodesic distance from each seed. Geodesic distances are computed using the fast marching method (Sethian, 1996) over a triangle mesh (Kimmel and Sethian, 1998), where the metric is induced by the embedding of the 3D surface in ${\mathbb{R}}^3$.

We strive to provide a robust method to select the chamber seeds, so we use the ASM to define the most likely location of the chambers and valves. Then, we locate the seeds according to the location and pose of the ASM. The adjusted mesh of the LV using the ASM model has an open aperture composed of a loop of points, as represented in Fig. 5. A plane is fitted to those points. Because the model is expected to be reasonably adjusted, this plane is close to the mitral valve, while AA and LA are supposed to fall above this plane, whereas the ventricular apex is in the opposite direction. Accordingly, the point of the triangle mesh with maximum distance to this plane below the plane is chosen as the ventricular apex of the triangle mesh, which is the seed of the LV.

Fig. 5.

(a) Active shape model (ASM) of the left ventricle (LV) depicted as a red triangle mesh, left atrium (LA) as a green mesh, and ascending aorta (AA) as a blue mesh, also representing the fitted plane to the upper LV aperture (violet plane) and separation plane between LA and AA (brown plane); (b) when the ASM model of the LV is fitted to the study (gray mesh), both planes are useful when selecting the initial seeds of the LV (red point) LA (green point), and AA (blue point).

The rigid transformation obtained in the estimation of the ASM model of the LV is also applied to place a separation plane between LA and AA, and this transformation was previously calculated in the ASM model (Fig. 5). The parts of the triangle mesh that have a large distance to that plane in the right direction and simultaneously are very distant from the ventricular apex have a high probability of belonging to AA. In addition, this part of the triangle mesh, because of the tubular shape of the AA, is intersected by the plane defined by the Z-slices of the CT with an elliptical well-defined shape and is therefore adjusted with an ellipse fitting to ensure that the AA is correctly detected. The central point of the fitted elliptical shape is selected as the AA seed.

Once LV and AA seeds are detected, the upper LV plane and LA–AA planes of the ASM model are also used to choose the seed of the LA as the point that maximizes the sum of the squares of distances from both planes in the right direction, excluding points where the geodesic distance from the AA seed is smaller than a threshold to avoid selecting another point in the AA as the LA seed.

2.4.4. Graph-cut algorithm

A graph-cut algorithm (Boykov et al., 2001; Kolmogorov and Zabih, 2004) is used to segment the 4D graph to minimize the energy expression (5). The graph-cut algorithm optimizes multilabel energies via the α-β swap (Boykov and Kolmogorov, 2004).

To simplify the problem of multilabel segmentation further, the triangle meshes in LV, AA, and LA are divided by two independent bilabel segmentations: first, an AA–non-AA graph-cut algorithm is performed in which the LV and LA are merged together as a unique label. In this way, the graph-cut detects the aortic valve as the separation between segments. This is followed by a second stage where the subgraph of the AA can be removed, and an LV–LA bilevel graph-cut segmentation is performed to detect the mitral valve.

2.5. Extraction of valvular annuli location

As a result of the 4D-graph-cut, the surfaces of the left chambers are segmented. The mitral and aortic valvular annuli locations are identified in a secondary step and extracted as the vertices of the mesh connecting two faces belonging to two different chambers. Because this contour has a 3D shape, the valvular plane is approximated by fitting this contour to a plane.

In the case of the mitral valve, its characteristic saddle shape causes that a fitted plane does not properly represent the annulus location. Instead, the posterior annulus segment is better approximated by a plane, according to the D-shape model (Blanke et al., 2014). We use this segment and the fitted plane to determine the displacement of the mitral valve along the cardiac cycle and to extract derived parameters.

The posterior annulus segment is calculated from the obtained mitral and aortic complete annuli estimated locations, extracting the approximated D-shape of the mitral annulus by selecting the principal axis of the two annuli, and the two most distant points in the mitral annulus to this axis. Fig. 6 illustrates this procedure.

Fig. 6.

This figure illustrates the automatic selection of the posterior segment of the mitral annulus (shown in cyan): after the localization of the aortic valvular annulus (blue color) and the complete mitral annulus (cyan/red), the principal axis of both valves can be traced, allowing the automatic extraction of the posterior segment of the mitral valve, delimited by the points with maximum distance to the principal axis (shown in yellow). The manual landmarks used for validation are illustrated in magenta.

3. Experiments and results

The evaluation of the proposed methodology was conducted in 4D-CTA scans acquired from 11 subjects with normal LV function and 4 patients with low ventricular ejection fraction. The acquisition was performed during an inspiratory breath-hold, full R-wave to R-wave, covering a complete cardiac cycle. To reduce the heart rate, beta blockers were administered to those subjects with no contraindications. The average intensity values of the ascending aorta given in HU were used in the bolus-tracking method to trigger the acquisition. For this process, 60–100 mL iodinated contrast (Iopamidol, Isovue 370, Bracco) was injected at a rate of 4.5–5.0 mL/s, followed by a 50 mL saline flush.

The studies where reconstructed using the vendor-provided methods. CT scanners used, acquisition data and reconstruction parameters using vendor-provided methods are shown in Table 1. Detailed pixel size, field of view (FOV), and cardiac cycle duration of the studies are listed in Table 2. Study identifiers (IDs) are maintained in all the figures and tables of this work to facilitate the identification of the reported results. IDs #1 to #11 are identifiers of healthy subjects, while IDs #12 to #15 identify pathological cases.

Table 1.

CT scanners and parameters used for acquision of healhy subjects and patient with low ventricular ejection fraction.

	Subjects with normal LV function Study IDs #1 to #11	Patients with low LV function Study IDs #12 to #15
CT Scanner	Toshiba Aquilion ONE	GE Revolution CT
Gantry rotation	275–350 ms	280 ms
Wide-area detector	320 slices	256 slices
Acquisition mode	“Half-scan”	Cardiac, 16 cm slab in the Z direction
X-ray energy		80–100 kV
Timeframes in cardiac cycle	20	13–16
Reconstruction method	AIDR 3D STD, FC03 filter	ASIRv = 0.5, STANDARD kernel
Pixels (axial planes)	512x512	512x512
Pixel size (axial planes)	0.334 × 0.334 – 0.432 × 0.432 mm2	0.415 × 0.515 – 0.528 × 0.528 mm2
Slice spacing	0.25–1.0 mm	0.625 mm

Table 2.

Fifteen ECG-Gated 4D-CTA studies were processed. This table summarizes the pixel size, field of view (FOV), and cardiac cycle duration of these studies. IDs #1 to #11 are healhy subjects, while IDs #12 to #15 are patients with low ventricular ejection fraction.

Study ID	Pixel size (mm)	FOV (mm)	Cardiac cycle (s)	Frames
#1	0.339×0.339×0.5	174×174×120	1.12	20
#2	0.395×0.395×1	202×202×120	1.40	20
#3	0.429×0.429×0.5	220×220×120	1.19	20
#4	0.339×0.339×1	174×174×100	1.15	20
#5	0.344×0.344×0.5	176×176×120	1.04	20
#6	0.334×0.334×1	171×171×120	0.91	20
#7	0.432×0.432×0.25	221×221×120	1.08	20
#8	0.376×0.376×1	193×193×100	1.17	20
#9	0.339×0.339×1	174×174×120	1.13	20
#10	0.407×0.407×0.25	208×208×120	1.14	20
#11	0.415×0.415×1	212×212×100	1.35	20
#12	0.528×0.528×0.625	270×270×160	0.98	14
#13	0.422×0.422×0.625	216×216×160	1.05	15
#14	0.510×0.510×0.625	261×261×125	1.12	16
#15	0.441×0.441×0.625	226×226×160	0.91	13

The CT data from subjects with normal cardiac function were conducted in the Department of Cardiology of the National Heart, Lung, and Blood Institute under IRB approved protocols. The CT data from subjects with reduced LV function were conducted at the UCSD Thornton Hospital (La Jolla, CA) under IRB approved protocols.

The number of distributions of the GMM used for the estimation of the gamma map was initially fixed to two for healhy subjects, and was tuned to three in the case of patients with contrast in the right cavities.

Fig. 7 shows the segmented surfaces of the left chamber and their intersections with a slice of the LVOT. This cardiac view allows us simultaneously to visualize the location of the mitral and aortic valves as the points where the color changes. The rendering of segmented chambers is also superimposed onto Fig. 7. The result shown corresponds to study ID #1 in end-systolic and end-diastolic frames. The color codes used are red for the LV, green for the LA, and blue for the AA.

Fig. 7.

Right column: segmented surfaces of left chambers superimposed in left ventricular outflow tract (LVOT) slice view (red: left ventricle; green: left atrium; blue: ascending aorta). The surface edges between the segmented regions determine the mitral and aortic annuli. Left column: intersection of the segmented surface with the LVOT slice represented as color contour polylines. This result corresponds to Study ID #1, time frame #6 (upper images), and frame #20 (lower images).

The complete result of the 4D-graph-cut segmentation, covering all the time frames of the cardiac cycle is depicted in Fig. 8 for study ID #5 and in Fig. 9 for all studies covering the end-systolic and end-diastolic frames. The surface mesh segmentation is labeled with red, green, and blue corresponding to the LV, LA, and AA, respectively. The animations of all the studies have been submitted as supplementary material.

Fig. 8.

4D-graph-cut segmentation of surface mesh, covering all time frames of the cardiac cycle for study ID #5. The left ventricle is represented in red, the left atrium is represented in green, and the ascending aorta is represented in blue.

Fig. 9.

4D-graph-cut segmentation of surface mesh, covering end-diastolic and end-systolic time frames of the cardiac cycle for all studies. The left ventricle is represented in red, the left atrium is represented in green, and the ascending aorta is represented in blue.

The main parameters of the 4D-graph-cut algorithm were optimized using a small cohort of 3 initial healthy subjects and a simulated annealing algorithm. The table S1 of supplementary material specifies the parameters used in the experiments. Average time to process each case is less than 20 minutes, using a not fully optimized MATLAB^® implementation running on an Intel(R) Core(TM) i7-6700 CPU, 3.40GHz.

To test the performance of the preprocessing steps and the role of the gamma map, we compare the ASM with the gamma map to the ASM on the original CT numbers for all the cases. As a result, the ASM fitting using the gamma map achieved 100% success in all frames, while trying with the original CT numbers, only 39 frames out of 278 were adjusted acceptably.

3.1. Evaluation of the location estimation of the mitral and aortic valves annuli

The error in the location of the mitral and aortic valve annuli is defined as the agreement between the manual annotations and our automated method. To define the reference location of the mitral valvular annulus, all acquisitions were inspected manually, and three anatomical point landmarks along the mitral annulus were selected in all frames and studies. Two landmarks were selected near the anterior lateral and posterior medial commissures of the mitral valvular annulus, while the remaining landmark was selected approximately in the middle of the posterior leaflet. These three manually selected anatomical landmarks define the mitral valvular plane, as represented in Fig. 10.

Fig. 10.

Example of posterior mitral annulus localization and method of error estimation. Three manually selected landmarks (blue dots) are used to define a mitral plane (translucent plane in the image on the right) which intersects with the triangular mesh of the left chambers forming a contour, drawn as a blue polyline in the image on the right. The contour obtained by the proposed method is depicted as a red polyline in the figure. The performance in each frame is measured by the mean Euclidean distance between this red contour and the blue plane.

The mean Euclidean distance from the extracted segment of the posterior mitral annulus and the plane defined by the manually selected landmarks is considered in the reported errors. Fig. 10 represents the method of error estimation, as described previously, illustrated with the details of mitral annular valve localization.

Because the contours are constituted by line segments connecting points, we calculate the Euclidean distance between each point of the contour with respect to the plane obtained from manually selected landmarks, reporting the mean value of this set of distances. We define the mean value of this measurement over all time frames as the mean absolute error (MabsE) in the localization of the mitral valvular annulus, which is reported in Table 3. The errors along the time frames are indicated in the box plots of Fig. 11.

Table 3.

Mean absolute error (MabsE) and standard deviation (SD) in the localization of the mitral and aortic valve annuli. Mitral valve: for every frame, the estimated error is calculated as the mean Euclidean distance between all points in the estimated contour and the contour determined by the intersection between the plane defined by three manual markers and the mesh. Aortic valve: the estimated error by frame is calculated as the mean Euclidean distance between three manually selected landmarks and the corresponding closest points in the estimated contour. For both valves, the MabsE is then calculated as the mean error distance for all frames. The last row contains the overall values for all 15 studies.

Study ID	Mitral valve annulus		Aortic valve annulus
	MabsE (mm)	SD (mm)	MabsE (mm)	SD (mm)
#1	2.37	0.71	1.65	0.48
#2	2.78	1.14	1.85	0.63
#3	3.83	1.37	1.89	1.11
#4	3.33	1.32	1.24	0.64
#5	1.61	0.82	3.61	0.85
#6	3.30	1.01	1.21	0.60
#7	3.25	1.46	1.48	0.61
#8	2.23	1.28	2.17	1.35
#9	2.17	0.81	1.13	0.58
#10	4.24	1.97	1.64	0.50
#11	3.81	1.54	2.90	1.98
#12	1.33	0.34	2.55	0.60
#13	1.12	0.32	2.11	0.78
#14	1.23	0.44	1.93	0.70
#15	1.26	0.49	2.57	0.96
Mean values	2.52	1.06	2.00	0.69

Fig. 11.

Left: agreement distances in the determination of the location of the mitral valve annulus, measured with respect to the valvular plane obtained by three manually-selected landmarks. The distance is calculated for each frame and averaged along the sequence. Right: agreement distances in the localization of the aortic valve annulus. The distance is calculated as the mean Euclidean distance between the manually-selected landmarks and the corresponding closest points in the contour extracted from the segmented mesh.

In contrast, for the manual location of the aortic valvular annulus, the basal attachments of the three aortic cusps were manually selected as landmarks, defining a virtual ring that is slightly below the anatomic atrioventricular junction but clearly detectable in the image. The error of the location method, in this case, was calculated as the mean Euclidean distance between the manually selected landmarks and the corresponding closest points in the contour extracted from the segmented mesh. In this case, we did not use the distance between plane and contour, as reported for the mitral valvular annulus location because the aortic valvular annulus is far from being an ideal plane because of the anatomical shape of the cusps. Aortic valve localization MabsE is reported in Table 3, and agreement distances along the time frames are reported in Fig. 11. The total mean value of MabsE was 2.52±1.06 mm for the mitral annulus and 2.00±0.69 mm for the aortic valve annulus. Data are expressed as the mean values ± standard deviation.

In order to assess interobserver variability we asked a second observer to delineate the posterior mitral annulus in the frames #8 and #16 (approximately corresponding to end-systolic and end-diastolic moments) in studies IDs #1 to #11. The delineation was performed between the posteromedial commissure and the anterolateral commissure, covering the posterior leaflet the mitral valve. This part can be approximated by a plane using the D-shaped model as previously reported (Blanke et al., 2014). The agreement between plane defined by the three landmark points marked by the first observer and this delineation resulted in a final distance of 1.62±0.67 mm. For comparison, our reported agreement in the mitral valve is of 2.72±1.30 mm in the same 11 cases and two frames. The paired mean difference between the reported errors and the interobserver variability is 1.11 mm [95%CI: [0.57, 1.86] mm]. The P value of the two-sided permutation t-test is 0.0022. Therefore, these results assess a low (1.11mm) and significant difference with respect to the interobserver variability.

On the other hand, the overall quantitative effect of the temporal edges has been evaluated by calculating the 3D-graph-cuts segmentations independently for each frame using the same terms and parameters. The comparison respect to the 4D result is shown in Fig. S1 and Fig. S2 of supplementary material. It is emphasized that the improvement is mainly noted in the decrease of outliers thanks to the continuity obtained by the temporary edges. In the mitral annulus, the 10th percentile of worst frames have a mean agreement distance of 5.8 mm (worst frame, 7.2 mm) for the 4D-graph-cut method, while in the 3D case, the mean agreement distance of the 10th percentile of worst frames is 7.4 mm (worst frame, 13.7 mm). In the case of the aortic annulus, the comparison is more beneficial for the 4D method: the mean agreement distance for 10% percentile of worst frames is 4.4 mm (4D-graph-cut) and 11.0 mm (3D-graph-cut).

3.2. Excursion in mitral and aortic valves

The mitral annular plane systolic excursion (MAPSE) represents the total displacement of the mitral annular plane toward the apex between systole and diastole and assesses the size variation of the LV in the direction of the apex (Hu et al., 2013). Given the wide use of this descriptor, we performed an automatic measurement experiment to test the performance of our method. Apart from the displacement at systolic frames, the time evolution of the displacement between frames has also been quantified. We denote this dynamic mitral annular excursion as MAE. We have compared the measured MAE from the manually selected point landmarks with respect to the estimated MAE calculated from the mitral annulus extracted using the proposed 4D-graph-cut method.

Because of the complex movements of the mitral valve annulus and angular variations of MAE, we follow the approximation of measuring the linear excursion projected on the axis normal to the mean mitral valvular plane (considering the whole cardiac cycle) and consider the relative local coordinate along such axis.

The mean value of the projected excursion of the manual landmarks is considered as the measured MAE at a certain time frame, while the mean value of the projected estimated excursion of the anatomically equivalent points in the estimated mitral annulus is considered as the estimated MAE. Once the excursion estimations are obtained for all time frames, the time frame corresponding with the lowest mean value is taken as reference (diastolic frame), and the rest of the frames imply a negative displacement in mm. Fig. 12 shows this dynamic excursion, frame-by-frame, for all cases. The systolic excursion corresponds to the total mitral excursion (MAPSE) and is reported in Table 4.

Fig. 12.

Mitral annulus excursion (MAE) along the normal axis to the mean mitral plane, comparing estimated automatic results using the 4D-graph-cut (blue lines), the active shape model (ASM) depicted in red lines, and the manual landmark-based result (dotted gray lines) for all the cases under study.

Table 4.

Systolic excursion of mitral and aortic annuli: results from the manually selected landmarks, from 4D-graph-cut (GC) method and from the ASM in the case of the mitral valve. The absolute error is the unsigned difference between the manual values and the automated values. The total excursion is obtained along the normal axis to the mean annular averaging the whole cardiac cycle.

Study ID	Mitral annular plane systolic excursion (MAPSE) (mm)					Total aortic annular excursion (TAAE) (mm)
	Manual	GC	Error (GC)	ASM	Error (ASM)	Manual	GC	Error (GC)
#1	−14.29	−11.59	2.69	−18.30	4.02	−6.86	−5.91	0.95
#2	−13.10	−12.98	0.12	−15.20	2.10	−8.10	−7.55	0.55
#3	−11.19	−7.25	3.95	−9.00	2.19	−8.76	−9.79	1.03
#4	−9.20	−10.08	0.89	−16.43	7.23	−7.75	−7.84	0.09
#5	−13.76	−14.04	0.28	−10.85	2.91	−10.65	−8.61	2.04
#6	−9.89	−12.59	2.70	−16.00	6.12	−8.01	−7.32	0.69
#7	−11.61	−10.31	1.30	−14.05	2.44	−9.09	−9.08	0.01
#8	−15.46	−15.10	0.36	−16.70	1.24	−10.38	−9.46	0.92
#9	−9.39	−12.00	2.61	−8.85	0.55	−6.12	−7.11	0.99
#10	−17.44	−11.71	5.73	−14.66	2.79	−6.03	−5.29	0.74
#11	−12.76	−11.87	0.89	−14.67	1.91	−6.83	−6.38	0.45
#12	−8.26	−9.25	0.99	−8.18	0.08	−4.76	−5.46	0.70
#13	−14.40	−11.59	0.34	−34.45	20.05	−9.69	−10.03	0.34
#14	−10.96	−12.98	0.27	−20.88	9.92	−5.96	−7.95	1.98
#15	−6.68	−7.25	0.57	−26.34	19.66	−6.95	−6.27	0.68
Mean values			1.58±1.63		5.55±6.37			0.81±0.57

A second estimate of MAE and its total displacement (MAPSE) has been computed using the ASM model used in the initialization, approximating the upper ring of the modeled LV as the mitral annulus. The results are also shown in Table 4 and Fig. 12.

The aortic annulus excursion (AAE) is calculated in the same manner as MAE for the mitral annulus, i.e., measured along the normal axis to the mean aortic plane. In Fig. 13, the frame-by-frame AAE using manually selected landmarks is compared with AAE extracted from automatically estimated aortic annulus. The maximum displacement of AAE along the cardiac cycle is denoted as total AAE (TAAE) and is reported in Table 4.

Fig. 13.

Aortic annulus excursion (AAE) along the normal axis to the mean aortic valvular plane, comparing estimated automatic results (blue lines) using the 4D-graph-cut and the manual landmark-based result (dotted gray lines) for all the cases under study.

Estimated total MAE differences between systolic and diastolic phases (i.e., MAPSE) ranged from −7.25 mm for study ID #3 to −15.1 mm for study ID #8. Comparing with the results from manually marked landmarks, the mean absolute error of the total mitral excursion is 1.58±1.63 mm, with only one case with the error above 5 mm. As a comparison, the error using the ASM fitting was 5.55±6.37 mm, with two cases of patients reporting errors around 20 mm. TAAE measurements are of lower range than MAPSE, as expected: varying from −5.3 mm for study ID #10 to −10.0 mm for study ID #3. The mean absolute error is 0.81±0.57 mm.

We analyze the differences between the MAE estimates for the different methods depicted in Fig. 12 by considering a mixed effects model described in the following equation:

$$MA{E}_{i,j,k}=Subjec{t}_i+(Method{)}_k+(Method\cdot Frame{)}_{j(i),\mathrm{k}}+{\epsilon }_{j(i),\mathrm{k}}$$ (7)

where Subject accounts for the random effect given by the subject sampling (i=1,…, 15), Method is the fixed effect given by the different approaches (proposed method, manual landmarks, and ASM), and Frame is the temporal phase (where j(i) is the frame index for the i-th subject whose range is described in Table 2). The fixed effects test showed no statistical significance in the difference between methods (p-value = 0.1278). Similarly, we performed the same mixed effects model analysis for the aortic AAE estimates shown in Fig. 13. The fixed effect test showed no statistically significant differences between the proposed method and the measurements using manual landmarks (p-value = 0.6151).

In the case of MAE, the interaction term (Method · Frame) does not show any significant effect (p=0.9254). However, if removed from the model, the Method term becomes statistically significant (p<0.0001). This means that the measures vary with the time frame similarly, but there are some multicollinearities between the Method term and the time frame, an expected behavior considering that the interaction term is the method itself times the time intervals that are likely to be equispaced. A more detailed analysis performed with the Tuckey’s Range test for multiple comparisons shows that the proposed and manual methods differ from the ASM method in around 2 mm (Proposed – ASM = 2.14 mm, p<0.0001, [95%CI: [1.53, 2.76] mm]; Manual – ASM = 2.45 mm, p<0.0001, [95%CI: [1.84, 3.07] mm]). However, there are no significant differences between the proposed and manual methods (Proposed – Manual = −0.31 mm, p=0.4586, [95%CI: [−0.92, 0.30] mm]).

3.3. Global longitudinal strain of the LV

The estimated mitral valve annulus locations along the cardiac cycle have been used to report measurements of another relevant image-based descriptor: the global longitudinal strain of the LV (LVGLS) (Reisner et al., 2004), which is defined here as:

$$\text{LVGLS}=\frac{L_{ES}-L_{ED}}{L_{ED}}\times 100\%,$$ (8)

where L_ED and L_ES represent the length of the LV over the myocardium in the end-diastolic frame and end-systolic frames, respectively. This definition is an approximation of the LVGLS typically defined as the mean value of the local longitudinal strain over the myocardium. L_ED and L_ES are approximated as the distance between the mitral valve and the ventricular apex.

In this experiment, we evaluated the validity of the proposed method to compute this descriptor. In our 4D-CTA, the ventricular apex is approximated as the most distant point of the ventricular cavity to the mitral annulus. The mitral valve location considered for estimating L_ED and L_ES is the center of mass of the estimated posterior segment of the mitral annulus.

The ventricular apex is calculated using the gamma map, highlighted by the administered contrast inside the ventricular cavity, which makes detecting the most distant point in the vicinity of the ventricle mesh a robust process. Therefore, manual marking was not used.

We denote the distance between the valve location and the ventricular apex as L_MA. Therefore, in the end-systolic frame, it follows that L_MA(ES) = L_ES, while in the end-diastolic frame, it follows that L_MA(ED) = L_ED. We can extend the LVGLS to all time frames as the longitudinal strain (LS) as follows:

$$\text{LS}(t)=\frac{L_{MA}(t)-L_{ED}}{L_{ED}}\times 100\%.$$ (9)

In Fig. 14, the evolution of LS along the cardiac cycle is depicted, comparing the result obtained when the mitral location is estimated by the proposed segmentation using 4D-graph-cuts with respect to the mitral valve location obtained using manually selected landmarks.

Fig. 14.

Longitudinal strain (LS) evolution during the cardiac cycle. The blue line represents the estimated values obtained from the automatically derived mitral valve, while the gray-dotted lines are the values obtained from manually selected landmarks of the mitral valve for all the cases under study.

The mixed effects analysis performed for MAE and AAE was also employed for the LS measure of Fig. 14. As in the previous analyses, the fixed effect test showed no statistically significant differences between the proposed method and the manual landmarks (p-value = 0.8903).

Based on the maximum and minimum values of the evolution of L_MA in Fig. 14, LVGLS is reported for both the automatic estimated method and for the manual measurements in Table 5. The mean absolute error is 1.57±1.24%.

Table 5.

Left: left ventricle global longitudinal strain (LVGLS) measured from the automatically derived location of the mitral valve and from manually selected landmarks. The absolute error is the unsigned difference between both values. Right: left ventricle ejection fraction (LVEF) using the automatic displacements of the valves and using the upper plane of the active shape model (ASM) of the left ventricle.

Study ID	LVGLS (%)			LVEF (%)
	Estimated	Landmark based	Absolute Error	Estimated valves	ASM plane
#1	−22.10	−23.95	1.85	62.38	69.47
#2	−15.88	−17.66	1.78	59.23	62.44
#3	−25.18	−26.26	1.08	67.42	69.03
#4	−18.64	−17.02	1.63	65.41	71.18
#5	−18.60	−18.84	0.24	63.74	54.41
#6	−22.57	−19.07	3.50	71.65	73.86
#7	−19.85	−22.83	2.98	65.99	72.34
#8	−19.40	−19.75	0.35	67.13	69.11
#9	−25.54	−22.46	3.08	69.47	66.74
#10	−26.77	−30.26	3.49	69.38	76.39
#11	−23.07	−23.67	0.60	70.62	81.08
#12	−10.76	−8.61	2.15	40.90	37.86
#13	−11.70	−11.44	0.26	47.81	56.02
#14	−12.42	−12.55	0.13	44.77	50.74
#15	−4.03	−4.48	0.44	36.70	51.47
Mean values			1.57±1.24

3.4. Left ventricular ejection fraction

The left ventricular ejection fraction (LVEF) measured as LVEF = 1−V_ES / V_ED, where V_ED corresponds to the maximal ventricular cavity volume, achieved at the expansion phase in the end-diastolic frame, and V_ES is the minimal cavity volume reached at the contraction phase at end-systole. The LVEF is usually measured based on the LV cavity and a fixed selected plane at the base, but this value can be refined by including the total ventricular volume the mitral and aortic valvular annuli. Table 5 (right column) shows the LVEF, in percentage. Results using the automatically derived mitral and aortic valve planes are compared with those of the LVEF using the mitral plane resultant of the ASM model fitted to each time frame. The left ventricular volume fraction (LVVF) is defined (in percentage) as LVVF = (1−LVEF)×100%. LVVF can be extended to all time frames as LVVF(t) = (V_F(t)/V_ED)×100%, and the results for all studies are shown in Fig. 15.

Fig. 15.

Evolution of left ventricle volume fraction (LVVF) along the cardiac cycle, as a percentage of maximal volume with respect to the end-diastolic frame. The blue lines represent the calculated the ventricular volumes from the automatically derived mitral and aortic valve planes along the cardiac cycle, while the red lines consider the estimated upper ventricular plane of the active shape model (ASM).

4. Discussion and conclusion

In this work, we propose an automatic method based on 4D-graph-cuts to determine the displacement of the mitral and aortic valvular annuli in 4D-CTA images. This method has been tested on a set of 15 dynamic studies, comprising a total of 278 3D images, without substantial errors. The proposed pipeline has been designed to be fully automatic and not dependent on a large training database. Initial alignment and scaling of the ASM model of the LV were employed in one reference study, while the remaining patients were correctly segmented using the same initialization regardless of substantial differences in the heart sizes and FOV of the images. The robustness in the initialization provided by the ASM shows that the method is sufficient without needing manual alignment.

The displacement is successfully completed throughout the time frames even when only partial LA and a small portion of the AA is found within the FOV, as shown in study ID #2 and in Fig. 9. The proposed procedure succeeded in both healthy subjects and pathological cases.

The robustness of the ASM fitting to each frame was ensured with the use of the gamma map instead of the original CT values present in the data because there is a clear difference between the probabilities of the gamma map with high values (close to one) in the blood pool with injected contrast, and the probabilities of the gamma map with low values (close to zero) in the surrounding tissues. Therefore, a clear gradient can be detected around the cavity of the LV filled with contrasted blood during the ASM fitting process. If the original CT numbers were used instead of the gamma map, there would also be clear additional gradients, e.g., between the epicardium and lungs, that would hinder the ASM fitting. The ASM model of the LV uses improved weighted optimization algorithms in both registration and projection steps. This way, points with negligible gradient values along the scanned line segments have close-to-zero weights and are not really involved in the ASM fitting process. In contrast, the most relevant points have higher weights and reliably guide the ASM fitting. Our experiments on pathological cases demonstrated that, even though the ASM was built from a healthy population and its performance might be sub-optimal for these cases (e.g., Fig. 12, study IDs #13, #14 and #15), it was enough to detect the initial location of the ventricle and therefore the workflow performed properly in the unhealthy patients.

The role of the gamma map extracted from the GMM is not only confined to the fitting of the ASM model of the LV. It also allows us to create a binary mask containing the contrast regions (e.g., right chambers, aorta, and pulmonary veins). The unprocessed mask has an irregular surface and complex topology because of the presence of image noise and small structures, such as papillary muscles, LV trabeculae, or coronary arteries, and needs to be softened by morphological closing and opening operations to obtain a surface without singularities, degenerate elements, or topological artifacts. In contrast, excessively smoothed surface meshes are counterproductive because information on the surface curvature would have been lost and unable to adjust faithfully to the edges of the gamma map.

The proposed method could have benefited from a multiscale scheme in which the first coarser segmentation would be refined in an iterative process, obtaining a series of increasingly finer meshes, but the total processing time is increased. An alternative use of the increase in processing time is to use the multiscale approach only in a region around the valvular annuli segmented in coarser scales. We did not implement this multiscale approach because our dataset gives accurate estimates applying just one scale. Nevertheless, a multiscale implementation will be considered in future work.

The surface-based weighting factors of the cost function in the 4D-graph-cut, denoted as $B_{\{p,q\}}^{curv}$, were designed to include information about local curvature of the surface mesh because the valvular annuli can, in some cases, be identified by concave regions, as shown in the right picture of Fig. 3. This fact is particularly true for the aortic valve because of the presence of clearly visible semilunar cusps, although they are less prominent in some patients (see, for instance, study ID #8 in Fig. 9. In other cases, the entire perimeter of the mitral valve cannot be recognized by this characteristic concave zone. Moreover, other structures, especially the ventricular trabecula, produce concavities. Therefore, a graph-cut based solely on curvature factors would not be robust enough for our purpose.

To overcome this problem, we consider the information related to the gray level of the image. Similar to the ASM fitting, instead of using the original CT numbers of the image, we use the gamma map and the sampling scheme in conic regions, as shown in Fig. 4. These gray-level-based weighting factors $B_{\{p,q\}}^{gamma}$ significantly change during the cardiac cycle. In the systolic phase, when the mitral valve is closed and falls inside the sampling cone, $B_{\{p,q\}}^{gamma}$ decreases; in contrast, during the diastolic phase, when the mitral valve is open, it is more difficult to detect the posterior leaflet because it can be nearly tangent to the surface and outside the sampling cone. The opposite effect can be seen in the aortic valve.

The CPD registration between time frames, which adds new temporal dynamic edges to create a 4D graph, helps to localize the valvular annuli in the cardiac phases correctly where there are no low weights in the curvature components $B_{\{p,q\}}^{curv}$ or in the gamma components $B_{\{p,q\}}^{gamma}$ of the cost function. In those cases, the vertices that are difficult to segment are connected to anatomically equivalent points in other frames in which these characteristics appear to present temporal consistency. This triple contribution to the weights of the cost function (curvature, gamma map value, and temporal registration) increases the robustness of the 4D-graph-cut algorithm.

We have compared our results against manual measurements to report the error. However, this was a difficult task: 1668 points had to be manually selected on the 4D-CTAs (three markers per valve in 13 to 20 time frames for 15 studies). Completely delineating the valves to measure the localization error along its entire contour would have been even more arduous, although it would have made possible using a saddle-shaped mitral annular assessment and reduce the reported errors. Without this complete delineation, the error can only be measured at the manually marked points on the valve, as was done for the aortic valve in Table 3. However, for the case of the mitral valve, this measurement based on only three points could lead to apparently good estimates in cases in which the resultant mitral annulus segmentations differ substantially from the real mitral annulus (saddle-shaped) approximated by a plane if it was coincident in the three marked points but not in the rest of the valves. Therefore, the error was reported based on the mitral annulus defined by the intersection of the mesh and a plane derived from the three manually marked points with respect to the resultant annulus from the proposed pipeline. It is important to note that this procedure has some limitations because of the manual marking of the reference points and the mentioned saddle-shape of the mitral annulus, and these limitations both imply an overestimation of the error.

Regarding the parameters directly related with mitral excursions, MAE and AAE, we used for simplicity the typical approach that considers the movement of the valve as a single plane, although mitral annulus motion can be broken down into three components (Silbiger, 2012): translation toward and away from the LV apex, folding across the intercommissural axis, and circumferential contraction.

Derived parameters (LVGLS and LVEF) are in the range of values reported in the literature. Regarding the LVGLS, the average value in our experiments with healthy subjects was −21.6±3.4%. This result is within the expected range for normals. A study of 1266 subjects from the general population using echocardiography (Biering-Sørensen et al., 2017) reported a total average of −18.0±3.7%. The foundational article proposing LVGLS (Reisner et al., 2004), obtained −24.1±2.9% in 12 control patients. A Meta-analysis reported normal values of LVGLS varied from −15.9% to −22.1% (Yingchoncharoen et al., 2013). The obtained LVGLS values in our small cohort of patients were substantially smaller: −9.7±2.9%, which is associated with higher risk of heart failure (Biering-Sørensen et al., 2017).

The LVEF value for healthy individuals was 66.6±3.75%. As a comparison, (Suzuki et al., 2012) reported values of 68.9±6.8% for 84 control individuals (age 67.5±13.4). As expected, measured LVEF values for patients diagnosed with low LVEF were substantially smaller: 42.5±14.8%.

In conclusion, we have proposed a robust and fully automatic pipeline for the estimation of the location and displacement of the aortic and mitral valve annuli in high-resolution 4D-CTA. The method includes the calculation of a GMM to obtain a binary mask that is then processed to obtain a triangulated mesh for each time frame. Then, a robust initialization using an ASM model of the LV determines a priori locations, and a CPD registration algorithm connects the meshes of all time frames to yield a unique graph on which graph-cuts are calculated to obtain a temporally consistent segmentation of the left heart cavities that identify the mitral and aortic annuli. The method has been evaluated on a set of 11 healthy subjects and 4 patients, and our results have been compared against the manual identification of anatomical landmarks at the valve annuli. The reported agreement between the manual annotations and our method (MabsE) was 2.52±1.06 mm for the mitral annulus and 2.00±0.69 mm for the aortic valve annulus. We have also shown the impact of the valve plane detections on derived functional parameters (LVVF, LVGLS, MAE, and AAE) that are evaluated throughout the cardiac cycle. Valve excursions and the LVGLS were compared with manually detected valves, reporting mean errors of 1.58±1.63 mm for MAPSE, 0.81±0.57 mm for TAAE, and 1.57±1.24% for the LVGLS. The MAE results were also compared with results obtained from the automatic ASM fitting: our method reported a mean error below 1.6 mm, while the ASM mean error was larger than 5 mm. These results suggest that we have achieved a robust method of localization and determination of the displacements of the mitral and aortic valve to evaluate derived functional parameters without the need for a large training database. Its use in both pathological and healthy data is therefore direct without the requirement for extra training.