Extracting cardiac shapes and motion of the chick embryo heart outflow tract from four-dimensional optical coherence tomography images

Abstract.

Recent advances in optical coherence tomography (OCT), and the development of image reconstruction algorithms, enabled four-dimensional (4-D) (three-dimensional imaging over time) imaging of the embryonic heart. To further analyze and quantify the dynamics of cardiac beating, segmentation procedures that can extract the shape of the heart and its motion are needed. Most previous studies analyzed cardiac image sequences using manually extracted shapes and measurements. However, this is time consuming and subject to inter-operator variability. Automated or semi-automated analyses of 4-D cardiac OCT images, although very desirable, are also extremely challenging. This work proposes a robust algorithm to semi automatically detect and track cardiac tissue layers from 4-D OCT images of early (tubular) embryonic hearts. Our algorithm uses a two-dimensional (2-D) deformable double-line model (DLM) to detect target cardiac tissues. The detection algorithm uses a maximum-likelihood estimator and was successfully applied to 4-D in vivo OCT images of the heart outflow tract of day three chicken embryos. The extracted shapes captured the dynamics of the chick embryonic heart outflow tract wall, enabling further analysis of cardiac motion.

1. Introduction

This work is motivated by the need to analyze the motion of the embryonic heart and to elucidate relationships between altered cardiac mechanics during early development and congenital heart defects. Blood flow is essential for cardiac development, and alterations in blood flow dynamics can lead to heart defects,¹ which affect about 1% of newborn babies. The interaction between cardiac tissue and blood flow dynamics determines the mechanical stimuli (e.g., stresses/strains) to which cardiac cells are subjected, stimuli that modulate cardiac growth and development. Images of the beating heart can then be used not only to visualize cardiac motion but also to quantify mechanical stimuli that modulate intrinsic cardiac genetic programs.

Optical coherence tomography (OCT) has been extensively used to study embryonic and cardiac development.^2,3 Recently, the development of spectral domain OCT (SDOCT)⁴ made it possible to acquire in vivo image sequences of the developing heart^5,6 and to obtain four-dimensional (4-D) [three-dimensional (3-D) images over time] OCT images of the beating embryo heart^7,8 and blood flow velocities.^9,10 Although OCT allows micro-structural and Doppler velocity visualization, quantification and analyses of cardiac motion and blood flow have been hampered by a lack of reliable, automatic procedures to extract the shape and motion of the developing heart from OCT images.

To analyze cardiac motion from in vivo OCT images, most previous studies used manual segmentation and manually extracted measurements.^10,11 However, thorough manual analysis of OCT cardiac image sequences, which comprise large volumes of data, can be very time consuming and subject to operator errors. Therefore, to facilitate quantification and comparison of image datasets there is a dire need to develop automated analysis of cardiac motion from OCT images. Here, we present a semi-automatic, robust procedure to extract the dynamic shapes of the beating embryonic heart from 4-D OCT images.

We applied our segmentation procedure to 4-D images of a cardiac segment, the heart outflow tract (OFT), of chick embryos at the Hamburger-Hamilton¹² developmental stage 18 (HH18), approximately 3 days of incubation. At HH18, the heart OFT has a tubular structure, with an external myocardial layer, an endocardial layer that is in contact with blood, and a cardiac jelly layer in between the myocardium and endocardium (Fig. 1). From OCT images, the myocardium, cardiac jelly and the blood can be readily distinguished. The endocardium that lines the lumen, however, cannot be distinguished from blood. This is, however, not an important limitation as the endocardium is a thin monolayer of endocardial cells and thus can be assumed to coincide with the lumen-tissue interface. During the cardiac cycle, the OFT myocardium contracts and expands approximately preserving a tubular shape. In contrast, upon contraction, the shape of the endocardium becomes irregular as it “folds” along lines that are approximately aligned with the direction of flow.¹¹ Our developed procedure successfully detects and segments the OFT myocardium and endocardium over the cardiac cycle.

Fig. 1.

Structure of the heart outflow tract (OFT). (a) 3-D image set acquired with OCT, showing a longitudinal OFT section. (b) Sketch of the OFT showing its wall layers, as well as proximal and distal connections. The arrow shows the direction of blood flow. (c) and (d) Cross-sectional images at the inlet of the OFT during maximal contraction and maximal expansion. (e) and (f) Cross-sectional images at the outlet of the OFT during maximal contraction and maximal expansion. M: myocardium; C: cardiac jelly; L: lumen. Scale bars: 100 μm.

Automatic extraction of myocardium and endocardium shapes from 4-D OCT images presents several difficulties, including:

• (1)the presence of tissues that are adjacent to the myocardial layer;
• (2)image noise and variations in intensity due to weakening of OCT signals with tissue depth [see Fig. 1(e) and 1(f)];
• (3)the irregular, folding motion of the endocardium, with a shape that deviates from tubular upon OFT contraction; and
• (4)large displacements of OFT tissue layers over time.

To overcome these difficulties, a segmentation procedure must distinguish target tissue from adjacent tissue, even as intensity lowers with tissue depth and must successfully track the large and irregular displacements of target tissues.

Algorithms have been developed to automatically extract the shape of tubular structures from vascular images.^13–19 In particular, deformable models,^20,21 such as active-contour (snake), have been widely used to detect the boundary of target tissues and extract their shapes.^22–25 These models, however, typically fail to correctly detect target tissues when adjacent tissues with similar intensity are present. To correctly segment target tissue layers, piece-wise deformable models (a type of template-matching model) adjust their shape to match the shape of the tissue layer by using intensity values from inside and outside of the model.^26–28 Here, we designed a robust two-dimensional (2-D) piece-wise deformable model, the double-line model (DLM), to locally detect the myocardial layer and the endocardial layer from 2-D cross-sectional images of the developing heart. A robust likelihood estimator is used to achieve accurate detection of the target tissue layer even with weakening intensities of OCT signals (lower signal-to-noise ratios) with tissue depth, and in the presence of adjacent tissues. Tracking of tissue layers was performed in space and time to achieve 4-D segmentations. The extracted cardiac shapes accurately captured the geometrical configuration of the OFT tissue layers and their motion over the cardiac cycle.

2. 4-D OCT Imaging of the Chick Embryonic Heart Outflow Tract

In this section we briefly describe the acquisition of in vivo imaging data using OCT, and image data reconstruction into 4-D images. Prior to imaging, white leghorn eggs were incubated to HH18 ($\sim 72\mathrm{h}$). The eggs were then removed from the incubator and placed in an organic glass cube in which temperature was controlled to approximately 37.5°C to keep the cardiac cycle at a physiological range (around 370 ms). The egg shell and part of the membrane overlaying the embryo were removed to provide optical access to the chick heart. A spectral domain OCT system was then used to image the OFT of the embryonic chick heart.^8,9 The system acquired $256\times 512\text{pixel}$ images (256 A-scans) at a rate of 140 frames per second (fps), with axial and lateral resolutions of $\sim 10\mu \text{m}$ and $\sim 16\mu \text{m}$. Following our previously described 4-D imaging strategy,⁸ we acquired 2-D image sequences that included about four cardiac cycles (200 frames at speed of 140 fps) at different image planes along the OFT, separated by 7.5 μm. Our imaging procedure resulted in 120 frame sequences along the OFT (see Fig. 2). We also imaged longitudinal frame sequences (for reconstruction purposes). Raw image acquisition took about 20 minutes.

Fig. 2.

Schematic representation of OCT imaging strategy and reconstruction procedure. Cross-sectional OCT images were obtained by scanning the heart at 120 positions (separated by 7.5 μm) along the OFT. At each position, 200 sequential 2-D scans were obtained at 140 fps (about four cardiac cycles). A transverse position was also imaged (not shown). Acquired image sequences, however, start at different phases (timings) over the cardiac cycle. To synchronize the images, cross-sectional sequences are first pooled to one cardiac cycle, phases between image sequences are calculated, and then image sequences are interpolated so that sequences from all positions start at the same time and have the same number of image frames (196 frames over the cardiac cycle). 3-D images that show the position of the OFT over the cardiac cycle are then constructed to obtain 196 sets of 3-D images, with each 3-D image consisting of 110 positions along the OFT. $t$ in the right image is time, and $T$ is the period of the cardiac cycle (370 ms).

To obtain 4-D images of the OFT, the acquired OCT image frames were combined using our developed reconstruction procedure.⁸ Since image acquisition was not gated, we used similarity of lines from neighboring images to synchronize frame sequences acquired at different image planes along the OFT, and then we calibrated the synchronization using time-point lags estimated from longitudinal image frames. To generate the 4-D image dataset, we pooled the acquired image frames to one cardiac cycle interpolating the images to obtain several 3-D reconstructions over the cardiac cycle (see Fig. 2). Each reconstructed 4-D image dataset consisted of 3-D image datasets at 196 different time points (over one cardiac cycle), with each 3-D image dataset consisting of $180\times 265\times 110\text{\hspace{0.17em}\hspace{0.17em}voxels}$. We applied our shape extraction algorithm to these reconstructed 4-D OFT image datasets.

3. Developed 4-D Shape Extraction Procedure

We designed an algorithm to extract the dynamic shapes of the heart myocardium and endocardium from reconstructed 4-D OCT images. We extracted three surfaces over time: the external and internal surfaces of the myocardium layer and the endocardium (or tissue-lumen interface) surface. Here, we first present a brief overview of the extraction procedure. Then, we explain in more detail the DLM model and the segmentation strategy employed.

3.1. Overview of the 4-D Extraction Algorithm

To extract the shape of the cardiac tissue layers, we used 2-D cross-sectional planes swept along the centerline of the heart OFT (see Fig. 3 and Sec. 3.4 for a detailed description of how the centerline and cross-sectional planes are obtained), and a robust 2-D segmentation algorithm. The advantages of using 2-D cross-sectional planes are:

• (1)for tubular structures tissue layer surfaces form a closed contour on the plane and
• (2)image processing is simpler on 2-D frames (than 3-D or 4-D volume data).

The 2-D segmentation algorithm was successively applied to 2-D cross-sectional planes over space and time to render 4-D segmentations. We used this strategy, together with smoothing and regularization procedures, to extract the shapes of the myocardium and endocardium over the cardiac cycle.

Fig. 3.

Sketch showing the OFT shape extraction procedure from OCT images. (a) Cross-sectional plane depicting the myocardium and endocardium layers, the myocardium mid-layer contour, a layer DLM placed on the myocardium and an edge DLM placed on the endocardium. (b) Initial cross-sectional plane. (c) Extraction of the cardiac shape is performed by sweeping cross-sectional planes along the OFT.

The 2-D segmentation algorithm was based on a local, piece-wise deformable double-line model [DLM; see Fig. 3(a)]. Two slightly different DLM formulations were developed: the layer DLM, and the edge DLM. To detect the myocardium layer (internal and external surfaces) we used the layer DLM, and to capture the endocardium layer (essentially a thin interface between the tissue and the lumen), we used the edge DLM [see Fig. 3(a)]. In our algorithm implementation, to detect cardiac tissues, several layer DLMs are placed around the myocardium and several edge DLMs are placed around the endocardium. The DLMs on each tissue layer are then linked together using active-contour techniques, and detection of the myocardial and endocardial layers is guided by a robust maximum-likelihood estimator. Linking the DLMs with active-contour facilitates detection and tracking of tissue motion.

Our extraction procedure segments the myocardium layer and then the endocardium layer (see Fig. 4). Extraction of 3-D surface shapes from the first 3-D image volume dataset (time $t=0$) starts with a manual setting of the myocardium mid-layer contour [a contour that divides the thickness of the myocardium layer in half, see Fig. 3(a)] on an initial plane. Layer DLMs are arranged around the mid-layer contour, and linked with an active-contour model. Then, guided by a maximum-likelihood estimator, the layer DLM thickness and position are adjusted to match the myocardium layer [see Fig. 3(b)]. The procedure is repeated for each cross-sectional plane until the entire 3-D volume data is spanned [see Fig. 3(c)]. To segment the myocardium at the next time-step $t=\mathrm{\Delta }t$, the configuration of the layer DLMs at $t=0$ is copied to the 3-D volume data corresponding to $t=\mathrm{\Delta }t$, and the myocardium layer is detected in the new time step. The procedure is repeated for all time steps, until the entire cardiac cycle is spanned. Smoothing in space and time is then performed to yield the myocardial shape.

Fig. 4.

Steps to extract the 4-D OFT myocardial and endocardial layer surfaces.

Once the myocardium has been segmented, we used the segmentation to mask the regions of the cross-sectional images that lie outside the inner myocardial surface (see Fig. 4). This is done to facilitate detection of the endocardium. To reduce image noise, we then generate a binary image that increases lumen contrast. Edge DLMs are then placed around the lumen-tissue interface, and linked with an active-contour. The position of the edge DLMs is then adjusted to match the endocardium layer on the preprocessed image. This is repeated in space and time until the endocardium from the whole 4-D dataset is segmented. Smoothing of the lumen/endocardium surface is then performed in space and time.

3.2. 2-D Detection of the Myocardial Layer

The layer DLM is a local 2-D template deformable model that we designed to track the myocardium layer, and is key to our myocardial shape extraction procedure. In this section, we will introduce the layer DLM, its main parameters, and the maximum-likelihood estimator used to match the DLM to the myocardial layer. We will also describe how DLMs are linked together with an active-contour model to smooth the contour and guide myocardium detection.

3.2.1. Layer DLM basic parameters

The DLM consists of two parallel lines: AB and CD [Fig. 5(a)], which are the outer and inner DLM edges, respectively. Basic parameters of the DLM describe its size and location: width $w_m$; length $l_m$; position ${\mathbf{p}}_m$ (DLM center point position vector); and direction ${\mathbf{t}}_m$ (a unit vector parallel to the DLM edges). These parameters also define a binormal vector ${\mathbf{n}}_m$, a unit vector perpendicular to ${\mathbf{t}}_m$ and pointing from the DLM center to the DLM outer edge. $M=\{w_m,l_m,{\mathbf{p}}_m,{\mathbf{t}}_m\}$ represents the state of the DLM. To detect the OFT myocardium, layer DLMs are scaled by changing $w_m$ and $l_m$; rotated by adjusting ${\mathbf{t}}_m$; and shifted by changing ${\mathbf{p}}_m$.

Fig. 5.

The layer double-line model (layer DLM). The layer DLM consists of two parallel lines AB (outer edge) and CD (inner edge), which can be adjusted to detect a tissue layer. (a) DLM parameters that define the DLM state : width, $w_m$; length, $l_m$; position, ${\mathbf{p}}_m$; and direction, ${\mathbf{t}}_m$. (b) Regions of the layer DLM: the DLM is divided into an internal region, $R_{mi}$, which is in turn subdivided into two subregions, $R_{mh}$ and $R_{ml}$; and two external regions, $R_{bo}$ and $R_{bi}$. These regions are used to calculate foreground and background intensity levels from OCT images.

The layer DLM was designed to

• (1)detect and track target, high-intensity tissue layers and
• (2)accurately capture the thickness of the tissue layers (avoiding adjacent tissues).

Local DLM intensity values are used to determine adaptive intensity levels, which, within a maximum-likelihood estimator, guide adjustment of DLM parameters to match the myocardial layer locally.

3.2.2. Foreground and background intensity estimation

To distinguish the target tissue from adjacent tissues, foreground and background intensity levels are estimated. The foreground intensity ($I_f$) is defined as the intensity of the target tissue; the background intensity ($I_b$) is defined as the intensity around the target tissue. In OCT images, the intensity at the center of the myocardial tissue layer is higher than the intensity near the tissue boundaries; also the intensity around the outer edge of the DLM may be different from the intensity around its inner edge. Thus, we define two levels of foreground intensity: high ($I_{fh}$), and low ($I_{fl}$); and two levels of background intensity: outer ($I_{bo}$), and inner ($I_{bi}$). To estimate $I_f$ and $I_b$ we use pixel intensities from regions inside and outside of the layer DLM, respectively [see Fig. 5(b)]. We divide the internal DLM region, $R_{mi}$, into two subregions, $R_{mh}$ and $R_{ml}$, corresponding to the middle portion of the DLM, and the portions adjacent to the edges, respectively [see Fig. 5(b)]. Thus $R_{ml}$ is a disjoint region. Similarly, we define two external regions, $R_{bo}$ and $R_{bi}$, which are adjacent to the DLM outer and inner edges, respectively. The equations used to calculate foreground and background intensities are listed in Table 1. Foreground and background intensities are adaptive and depend on the local intensities of the region where the DLM is located. This allows successful detection of cardiac layers even when image intensity varies.

Table 1.

Equations used to calculate foreground and background intensity levels.

Intensity level	Formula
$I_f$	$\mathrm{Med}[I(x)]$, $x\in R_{mi}$
$I_{fh}$	$\mathrm{Med}[I(\mathbf{x})]$, $I(\mathbf{x})>I_f$ and $\mathbf{x}\in R_{mi}$
$I_{fl}$	$\mathrm{Med}[I(\mathbf{x})]$, $I(\mathbf{x})<I_f$ and $\mathbf{x}\in R_{mi}$
$I_{bo}$	$\mathrm{Med}[I(\mathbf{x})]$, $\mathbf{x}\in R_{bo}$
$I_{bi}$	$\mathrm{Med}[I(\mathbf{x})]$, $\mathbf{x}\in R_{bi}$

$I(\mathbf{x})$: intensity of the pixel at position $\mathbf{x}$.

$\mathrm{Med}()$: median value.

3.2.3. Maximum-likelihood estimator

It has been shown that maximizing a likelihood estimator that uses intensity information from inside and outside of a deformable model can be successfully used to differentiate target tissues from adjacent tissues.^17,26,29 Inspired by these studies, we define the normalized likelihood function $L_n$ in discrete form by:

$$L_n(M|I)=\frac{1}{I_{fh}}[\sum _{\mathbf{x}\in R_{mi}}\frac{|I(\mathbf{x})-I_{bo}|+|I(\mathbf{x})-I_{bi}|}{A(R_{mi})}+\sum _{\mathbf{x}\in R_{mh}}\frac{-|I(\mathbf{x})-I_{fh}|}{A(R_{mh})}+\sum _{\mathbf{x}\in R_{ml}}\frac{-|I(\mathbf{x})-I_{fl}|}{A(R_{ml})}],$$ (1)

where $I(\mathbf{x})$ is the intensity of a pixel located at position $\mathbf{x}$ in the cross-sectional image, and $M$ is the state of the layer DLM. To reduce mis-detection of target tissue layers, the likelihood function was normalized with $I_{fh}$ and the DLM area, $A(R)$, defined as the number of pixels in the DLM region considered.

To detect the target tissue, we find the DLM state $M=\{w_m,l_m,{\mathbf{p}}_m,{\mathbf{t}}_m\}$ that maximizes $L_n(M|I)$. Because the likelihood function uses local intensity information from inside and outside of the DLM, the detection algorithm is robust and can locally find the target tissue even when the local image is noisy and intensity is low.

3.2.4. Linking layer DLMs with the active-contour model

The layer DLMs are used to detect the myocardium locally. To mitigate local uncertainties, and facilitate smoothing, the DLMs placed around the myocardium layer (see Fig. 6) are linked with an active-contour (snake) model. The snake model is a classical method to detect the contour of an object in an image by means of energy minimization.²² The energy function, $E_{\text{snake}}$, of the active-contour model that we used here is,

$$E_{\text{snake}}={\int }_0^1[\alpha {|v_s(s)|}^2+\beta {|v_{ss}(s)|}^2+\gamma E]\mathrm{d}s,$$ (2)

where $v(s)$ is the parametric curve of the myocardium mid-layer contour, with $s\in [0,1]$; $v_s(s)$ and $v_{ss}(s)$ are the first-order and second-order derivatives of $v(s)$; $E$ denotes an external energy term, which in our implementation was equal to the negative of the intensity of the local pixel (maximum in the center of the myocardium layer, where the mid-layer contour of the DLM is placed); $\alpha$, $\beta$ and $\gamma$ are constant weights.

Fig. 6.

Localization of DLMs around the myocardial layer and lumen interface.

3.2.5. Implementation and constraints

The layer DLM can be used to locally find the target tissue, by changing its size, position and direction to approach the tissue layer and, finally, match the tissue layer [see Fig. 7(a)–7(c)]. However, sometimes detection fails, especially in situations when there is: 1. close proximity of target and adjacent tissue layers [e.g., Fig. 7(i)]; 2. incorrect initial orientation of the layer DLM [e.g., Fig. 7(h)]; and 3. overlap of layer DLMs placed around the myocardium layer. To avoid these issues and increase the robustness of the extraction procedure, we constrained the layer DLM state parameters. Constraint values were found empirically from images, but we could use them in several image datasets without change.

Fig. 7.

Detection of the myocardial layer (target tissue) from OCT images of the embryonic heart OFT using the layer DLM. (a–c) Detection process. (d–g) Successful segmentation results: the layer DLM can successfully detect the myocardial layer even when there are adjacent tissues (in d and e) and intensity is weak (in f and g). (h) and (i) Examples of failed detection of the myocardium: adjacent and target tissues detected together (in i) and the orientation of the DLM is incorrect (in h). Scale bars: 100 μm.

To prevent simultaneous detection of both the target and adjacent tissue layers, we set bounds on the value of the DLM width $w_m$. The maximum $w_m$ was constrained to be larger than the actual thickness of the target tissue layer, but smaller than about twice the target layer thickness. From analysis of OCT cross-sectional images, the thickness of the OFT myocardial layer was about 14 pixels (70 μm) near the OFT inlet, and 10 pixels (50 μm) near the OFT outlet. Thus, we set the maximum $w_m$ to be 16 pixels near the OFT inlet, and 14 pixels near the OFT outlet. We also set the minimum $w_m$ to be 4 pixels (20 μm) everywhere. To avoid incorrect orientation of layer DLMs, the DLM direction, ${\mathbf{t}}_m$, was aligned to the tangent of the active-contour curve that links the DLMs around the myocardium (see Fig. 6). Finally, to prevent overlapping of layer DLMs, DLM shifts were allowed only in the direction of the DLM binormal ${\mathbf{n}}_m$ (that is perpendicular to the active-contour curve). Placing 40 layer DLMs around the myocardial layer in a cross-sectional image (see Fig. 6) was found to be enough to achieve an accurate segmentation of the myocardium.

Parameters of the active-contour model [see Eq. (2)] that linked the layer DLMs were also found empirically. Because the second term in Eq. (2), $v_{ss}(s)$, is related to the contour curve smoothness, $\beta$ was chosen to be larger than $\alpha$ and $\gamma$. Setting $\beta$ about 1.5 times larger than $\alpha$ and $\gamma$ ($\alpha =15$, $\gamma =15$) worked well in implementation in several image datasets.

3.3. 2-D Detection of the Endocardial Layer

The procedure to extract the endocardium layer from the cross-sectional images is similar to the procedure used to extract the myocardium layer. In OCT images, the endocardium layer cannot be distinguished, but the lumen-wall interface appears as the “edge” between the low-intensity cardiac jelly and the high-intensity endocardium and lumen. However, the lumen edge is frequently blurred in OCT images and to facilitate detection, we preprocessed the images to increase contrast and reduce noise. In the preprocessed images, the lumen-wall edge shows as a region of finite width in which intensity changes continuously (from dark to bright), therefore suited for detection using a DLM. Changes to the DLM formulation to detect the lumen-wall interface (the “edge” DLM) included redefining the foreground and background intensities and the DLM constraints and changing the definition of the associated maximum-likelihood estimator.

3.3.1. Image preprocessing

To generate preprocessed OCT images, we used an adaptive intensity threshold on the previously identified 2-D cross-sectional images and created a binary image with intensities either set to a fix high value, $I_h$, or 0, with lumen areas showing with high-intensity. Because only the tissues inside the region delineated by the inner contour of the myocardium layer (the inner-layer contour, see Fig. 6) are useful for extracting the lumen-wall interface, the intensity of the pixels that are outside of the inner-layer contour are set to 0 (masked out) in the binary image. To increase image contrast, the intensity of the pixels within the inner-layer contour is calculated using the median intensity of a small image region (window) centered at each pixel. If the intensity of the pixel is smaller than the median intensity of the region, pixel intensity is set to 0; otherwise, it is set to $I_h$. To reduce noise, we then calculate the mean intensity of three binary images: the image of interest and the two images that are adjacent to it in a 3-D image dataset (see Fig. 8). This operation reduces the sharpness of the lumen “edge” allowing use of a formulation that employs DLMs. We then use this preprocessed image to extract the lumen-wall interface.

Fig. 8.

Image preprocessing for extracting the 2-D lumen contour. Scale bars: 100 μm.

3.3.2. Edge DLM and maximum-likelihood estimator

To detect the lumen-wall interface, we slightly modified the formulation of the DLM, resulting in what we called an edge DLM formulation. The edge DLM is similar to the layer DLM, except that:

• (1)its width $w_m$ is set to a small, fixed constant;
• (2)background intensities are constant ($I_{bo}=0$, $I_{bi}=I_h$,);
• (3)only one foreground intensity level is used, and its value is constant ($I_f=I_h$) rather than adaptive.

The maximum-likelihood estimator function used for detection of the lumen-wall interface, $L_e$, is defined as follows:

$$L_e(M|I)=\frac{1}{A(R_{mi})}\sum _{\mathbf{x}\in R_{mi}}[|I(\mathbf{x})-I_{bo}|+|I(\mathbf{x})-I_{bi}|-|I(\mathbf{x})-I_f|].$$ (3)

The edge DLMs on the lumen contour are connected by an active-contour model. Similar to how layer DLMs are used to detect the myocardium, in detecting the lumen-wall interface at each cross-sectional preprocessed image, the likelihood function $L_e$ is locally maximized.

3.3.3. Implementation and constraints

The values of parameters needed to obtain an accurate segmentation of the lumen were determined empirically and tested on several image datasets. For the binary image, we used $I_h=120$. Further, to make the lumen visible and yet accurate for extraction, the size of the window used to calculate the binary intensity of pixels was set to 31 pixels (that is about 165 μm) in the image vertical and horizontal directions, centered on the pixel of interest. These values resulted in preprocessed images that accurately showed the lumen domain (see Fig. 8).

To detect the lumen-wall interface, we set $w_m$ to be 4 pixels (20 μm), and background and foreground intensities to be constant: $I_{bo}=0$, $I_{bi}=120$, $I_f=120$. Due to the irregular shape of the lumen contour, edge DLMs easily overlap when using an active-contour model to smooth the contour. Instead, we smooth the contour by averaging the position of adjacent DLMs on the lumen contour and, like for the layer DLM, use the active-contour to constrain edge DLM orientation and shifts. We found that placing 80 edge DLMs around the lumen contour in a cross-sectional plane was sufficient to accurately extract the lumen-wall interface.

3.4. Extraction of 4-D OFT Shapes

Extracting 4-D OFT shapes is performed as summarized in Sec. 3.1. We first determine the orientation of cross-sectional planes along the OFT, and extract the contours of the myocardial layer from cross-sectional images, using layer DLMs linked by active-contour models (see Fig. 4). Then, the lumen contours are extracted using edge DLMs applied to preprocessed cross-sectional images. It is worth noting here that we have not implemented a global optimization function on our procedure. Instead, our approach is to find an overall solution by successively solving local optimization problems:

• (1)Maximum-likelihood estimators to find local parameters of each DLM (DLM states are independent of each other at this step), and then
• (2)An active-contour algorithm to smooth and further refine the contour made by linking individual DLMs.

Step 2 changes the position of DLMs, and thus the procedure is iterated to find accurate solutions. Further, smoothing in space (outside of the plane) and time is also performed, and the successive procedure described above applied again after smoothing.

In this section, we describe in detail how we determine the orientation of cross-sectional planes, how the iterative segmentation procedure is applied, and how we smooth the extracted contours in space and time.

3.4.1. Determination of cross-sectional planes

A cross-sectional plane is defined here as a plane approximately perpendicular to the 3-D centerline of the OFT myocardium, and is represented by a position vector, ${\mathbf{p}}_s$, and a normal vector, ${\mathbf{n}}_s$ (see Fig. 3). Changing ${\mathbf{p}}_s$ shifts the plane, and changing ${\mathbf{n}}_s$ rotates the plane. In our implementation, ${\mathbf{p}}_s$ is the centroid of the myocardium mid-layer contour. Identification of cross-sectional planes in 4-D starts with a manual setting of the myocardium mid-layer contour on an initial plane on the first 3-D image volume dataset ($t=0$). Layer DLMs are arranged around the mid-layer contour, linked with an active-contour model, and used to detect the myocardium layer on the plane. The initial plane chosen, however, is not necessarily a cross-sectional plane. To find a cross-sectional plane, the initial plane is rotated. DLMs are used to match the myocardium layer after each rotation, until a plane that has the minimum mid-layer contour perimeter (the cross-sectional plane) is found. The DLM configuration is then copied to a new plane that is parallel to the cross-sectional plane found but slightly shifted in the direction of the plane normal. DLMs then are used to match the myocardium layer, and the plane is rotated until a new cross-sectional plane is found. Plane shifting and rotation are repeated until the entire OFT from the first 3-D volume data is spanned [see Fig. 3(c)]. The centroids of the cross-section mid-layer contours are then extracted to form the OFT centerline, which is smoothed in space. After centerline smoothing, cross-sectional planes are adjusted, so that they remain perpendicular to the centerline and detection of the myocardium layer is performed on adjusted planes. This procedure results in an initial segmentation of the myocardium layer at time $t=0$.

To segment the myocardium at the next time-step, $t=\mathrm{\Delta }t$, the configuration of the layer DLMs at $t=0$ is copied to the 3-D volume data corresponding to $t=\mathrm{\Delta }t$, and the segmentation procedure is used to change the DLM thickness and position to match the myocardial layer in the new volume dataset. The OFT centerline is then extracted, smoothed in space, and cross-sectional planes are adjusted. This is followed by detection of the myocardial layer on the adjusted cross-sectional planes. The procedure is then repeated for all time steps until the entire cardiac cycle is spanned. This strategy provides a good initial estimation of the position and orientation of cross-sectional planes in 4-D.

After the initial estimation of cross-sectional planes described above, the location and orientation of the planes is adjusted once more. To avoid spurious oscillations near the boundaries (the inlet and outlet of the outflow tract), the first and last cross-sectional planes along the OFT are fixed in space (${\mathbf{p}}_s$ and ${\mathbf{n}}_s$ do not change over time). For these planes, ${\mathbf{p}}_s$ and ${\mathbf{n}}_s$ are the mean of the previously estimated normal and position vectors over time. Plane positions in between the first and last planes are adjusted by first smoothing the centerlines in space and time. Smoothing in space and time uses data from adjacent planes: suppose that $S_{ij}$ denotes a quantity to smooth on the $i$’th cross-sectional plane at time $j$; the new, smoothed value of $S_{ij}$ is then caculated as the mean of $S_{(i-1)j}$, $S_{(i+1)j}$, $S_{i(j-1)}$, and $S_{i(j+1)}$. This procedure is performed for each component of ${\mathbf{n}}_s$ and ${\mathbf{p}}_s$, and for each plane (except the first and last), and each time-point. Cross-sectional planes are then arranged so that distance within planes is as uniform as possible, while avoiding cross-sectional planes from crossing each other (such that a nonoverlapping mesh can be constructed directly from extracted results). In regions where the OFT centerline is rather straight, plane positions are approximately equidistant along the centerline, and in regions where the radius of curvature of the OFT centerline is small, distance between planes is larger (proportional to the contour mean radius and the angular change of ${\mathbf{n}}_s$). This last step determines the final position and orientation of cross-sectional planes in 4-D.

3.4.2. Correction of inaccurate segmentations

Inaccurate segmentation could happen in our procedure because, like with other deformable models, layer detection using DLMs depends on the DLM initial state. Even after combining DLMs with active-contour models, and setting constraints for the DLM parameters (thickness, orientation), we still found a few sections that were inaccurately segmented (especially close to the OFT outlet, where signal-to-noise ratios diminish due to tissue depth). To solve this problem, an interactive interface was designed to allow the user to manually adjust cross-sectional contours and thus reset the state of the DLMs when needed. With this procedure, accurate segmentation of the 4-D cardiac OFT wall layer was achieved. In our implementation, inaccurate segmentations occurred at a very low frequency, with at most 0.2% frames showing failed segmentations. Nevertheless, even after detection failed, the DLM accurately found the target tissue again at the next adjacent section or time step.

3.4.3. Extraction and smoothing of the myocardium layer

Myocardium layer detection presents local fluctuations in space and time that need to be further smoothed. We proceed by first smoothing the myocardium mid-layer surface, which is described by the position vectors, ${\mathbf{p}}_m$, of the layer DLMs. Smoothing in space and time was performed (as in Sec. 3.4.1) by calculating the mean position of four spatiotemporally adjacent layer DLMs: the corresponding positions of two adjacent cross-sectional planes of the 3-D image set and the corresponding positions of two adjacent time points (in plane smoothing was performed using the active-contour). After smoothing the myocardium mid-layer, layer DLMs are used once more to accurately find the myocardium thickness at the new mid-layer positions. Outer and inner surface layers of the myocardium are then obtained by linking the outer and inner edges of the layer DLMs (see Fig. 6). Smoothing of the outer-layer and inner-layer surfaces is also carried out by calculating the mean positions of the four spatiotemporally adjacent points on the surfaces.

3.4.4. Extraction and smoothing of the endocardium/lumen surface

Once the myocardium layer is extracted, we proceed to extract endocardium/lumen surfaces. To this end, we first generate preprocessed images of each cross-sectional plane as described in Sec. 3.3.1. For the very first cross-sectional plane, edge DLMs are placed manually around the lumen edge on the preprocessed image, and the DLM parameters are optimized by maximizing $L_e$, Eq. (3). Edge DLM parameters are then copied to the adjacent cross-sectional plane and then optimized. This is repeated until the whole 3-D dataset is segmented. For subsequent 3-D datasets, edge DLM parameters are copied from the previous time step, and then optimized. Like in the case of myocardial surfaces, after obtaining the lumen surfaces, surface smoothing is performed by calculating mean positions of four spatiotemporally adjacent edge DLM positions. This step completes the extraction procedure of 4-D surfaces from 4-D OCT images of the chicken heart outflow tract.

4. Results

We semi-automatically extracted the shapes of HH18 OFT cardiac tissue layers from 4-D in vivo OCT images of the chicken heart. More precisely, we extracted 4-D surfaces of the myocardium and tissue-lumen interface over 196 time points spanning one cardiac cycle (approximately 370 ms), with each 3-D OFT shape consisting of 50 cross-sections spanning around 700 μm along the OFT centerline. While we analyze here the extraction from one set of 4-D images in detail, we have so far successfully extracted data from seven HH18 OFTs.

To evaluate the accuracy of the extracted results, we compared segmentations of 2-D cross-sectional images performed with our extraction algorithm and manually. For this study, manual segmentation was performed at three positions along the OFT: near the inlet, approximately in the middle, and near the outlet; at three different time points during the cardiac cycle: when the OFT walls are closed, half-closed, or opened (see Fig. 9, first to third rows). To compare results we used the areas enclosed by contours obtained by manual segmentation, $A_{ma}$; contours obtained by automatic segmentation, $A_{au}$; and $A_{co}$, which is the area of the intersection between the automatic and manually extracted regions. Two metrics, recall and precision, were used to evaluate the accuracy of the automatic segmentation. The recall, defined as $A_{co}/A_{ma}$, measures the ability of the automatic extracted result to match the target tissue; the precision, defined as $A_{co}/A_{au}$, measures the ability of the automatic extracted result to contain only the target tissue. The average recall of the myocardium from the nine selected 2-D cross-sectional images was 84.7%, with a standard deviation of 3.3%, and its precision was 92.8%, with a standard deviation of 4.4%. The average recall of the lumen from the nine selected cross-sectional images was 91.9%, with a standard deviation of 3.3%, and its precision was 93.2%, with a standard deviation of 5.1%. To further confirm the overall accuracy of the segmentation, we visually inspected automatic segmentation results on OFT longitudinal sections at three different time points (see last row in Fig. 9). We found that in most places deviations between visual estimations of the location of a boundary and segmentation lines were smaller than 1 pixel (around 5 μm). The comparison of segmentation results on the 2-D cross-sections and longitudinal sections demonstrated that our segmentation strategy can accurately extract the OFT surfaces of the myocardium and lumen from 4-D OCT images of the developing heart even when images are noisy and intensity levels (as well as signal-to-noise ratio) diminish with depth.

Fig. 9.

Automatic extracted myocardial layer and lumen surfaces. (First to third rows) Segmentations of cross-sectional planes perpendicular to the 3-D centerline of the OFT at three time points over the cardiac cycle. (Bottom row) Segmentations of OFT longitudinal sections. Scale bars: 100 μm.

Figure 10 shows extracted OFT surfaces from 4-D OCT images (myocardial surfaces and the lumen-tissue interface surface) as well as volume data (see also Videos 1, 2, and 3). The extracted OFT shapes showed the peristaltic-like motion of the myocardial and endocardial layers of the OFT over the cardiac cycle, characterized by a contraction wave that travels through the OFT. During contraction, the OFT lumen appears to close, preventing blood flow, and folding of the endocardium layer becomes apparent. Folding and unfolding of the endocardium layer are observed from the extracted endocardium surfaces (as well as in raw images). The extracted shapes are therefore useful not only to visualize cardiac motion over the cardiac cycle but also in the analysis of cardiac motion.

Fig. 10.

Extracted 4-D OFT shapes over one cardiac cycle. (a) Extracted OFT layers (represented as surface meshes) versus the original image at time 0 ms; see also (Video 1, MOV, 7.96 MB) [URL: http://dx.doi.org/10.1117/1.JBO.17.9.096005.1] showing the steps of the segmentation procedure. (b) and (c) Extracted results at times 95 ms, 190 ms, 285 ms, and 370 ms (cardiac cycle is 370 ms). Due to the periodic motion of the heart, the shape of the OFT at time 370 ms was the same as at time 0 ms. (b) OFT myocardial and endocardial surfaces; see also (Video 2, MOV, 9.30 MB) [URL: http://dx.doi.org/10.1117/1.JBO.17.9.096005.2] showing the motion of extracted surfaces (green: outer myocardium surface; blue: inner myocardium surface; red: endocardium/lumen surface). (c) 3-D volume data images of the OFT; see (Video 3, MOV, 7.67 MB) [URL: http://dx.doi.org/10.1117/1.JBO.17.9.096005.3] showing the motion of the heart from the extracted volume data (red: lumen; blue: myocardium, showing intensity levels in shades of blue).

We successfully extracted OFT shapes from six other 4-D image datasets of embryonic chick OFTs at HH18 (not shown). These chick embryo heart OFTs had similar size, but 4-D OCT images differed in intensity and level of noise. In segmenting the OFTs from these datasets, and due to the robustness of our extraction procedure, there was no need to adjust DLM parameters in our algorithm. However, a parameter of the active-contour model, $\beta$ [see Eq. (2)] was adjusted on a case-by-case basis to avoid deviations of the active-contour from the target tissue layer. The range of $\beta$ values used was from about 1.3 to 1.9 times the value of the other two parameters $\alpha$ and $\gamma$ ($\alpha =\gamma =15$), which were not modified. Larger values of $\beta$ were used when adjacent tissues outside of the myocardium had higher intensity levels than the myocardium tissue because this choice of $\beta$ penalized deviations from the myocardium layer by reducing curvature changes and increasing smoothing. For other cases, the smaller values of $\beta$ were chosen. All seven extracted 4-D chick heart shapes showed similar peristaltic-like motion of the myocardial and endocardial layers of the OFT, and endocardial folding upon contraction.

5. Discussion

In this paper, we present a semi-automatic procedure to extract the shape and motion of developing heart tissues from 4-D OCT images. The extracted tissue layers included the myocardium (internal and external surfaces) and the endocardium (which also represents the lumen-tissue interface). While the method has been applied to 4-D OCT images of the heart outflow tract of HH18 chick embryos, the procedure could also be applied to other stages of development, other portions of the heart, and also to other animal models and images acquired with different imaging modalities. This is, however, outside the scope of this work. The main characteristic of the presented procedure is that it can accurately track the dynamics of tissue layer motion over time, even when motion and deformation are large, and in the presence of image noise, gaps, and variations in intensity.

The main drawbacks of the extraction procedure presented are that the procedure is sensitive to initial conditions, and it has a relatively high computational cost. Like with other deformable models, procedures to match the layer and edge DLMs to tissue layers are sensitive to the initial state (e.g., position, orientation). To increase the robustness of the procedure, we linked DLMs with active-contour models and constrained some of its parameters (thickness, orientation). To increase the accuracy of the endocardium detection, which presents “folding” patterns, in our procedure we preprocessed cross-sectional images. Nevertheless, we found a few sections that were inaccurately segmented ($<0.2\%$). We solved this problem by designing an interactive interface that allows the user to first check the accuracy of the segmentations, and, when needed, manually correct segmentations that are not accurate. Manual correction allows “resetting” the state of DLMs, improving the initial conditions and facilitating accurate detection.

Our segmentation strategy differs from other methods to extract tissue layers. Most methods^13,14 detect tissue layers using intensity variations, but detection becomes difficult when adjacent tissue layers are present. Unlike other template-matching deformable models used to detect tissue layers,²⁰ which only consider intensities inside of the model, our procedure uses intensities from inside and outside of the DLM; further intensity levels are adaptive to adjust for the diminishing intensities with tissue depth.

Robustness of the procedure was achieved at the cost of a modest increase in computations. To achieve robustness, the layer DLM uses adaptive foreground and background intensity levels. Further, layer DLMs are linked with an active-contour model. Nevertheless, in our system (Dell T7400 workstation with eight core CPUs at 2.0 GHz and 10.0 GB RAM), extracting the OFT cardiac shape and motion from 4-D OCT images took about 3 h. Comparing this performance to that of manual segmentations, which can take more than a week for the whole 4-D image dataset, the proposed procedure enables faster and accurate analysis of cardiac motion. Thus, while computationally expensive, 4-D segmentation of the OFT of HH18 chick embryos is feasible.

The main advantage of our proposed semi-automatic procedure, is that it achieves robust segmentation of cardiac tissues from highly dynamic 4-D OCT images. This was confirmed by applying the algorithm to several 4-D OCT image reconstructions of the chick heart OFT, images that differ in the level of noise, image intensity and embryo orientation, and were also affected by biological variations, and possibly inaccuracies in embryo staging. The presented extraction procedure successfully segmentated all datasets. We are confident that we could also successfully apply the algorithm to other developmental stages and sections of the heart, enabling the study of changes in cardiac motion over developmental stages and after interventions.

The extracted 4-D results provide detailed information on the motion of the OFT wall, which could be used to quantify motion, tissue strains, or as input for subject-specific computational models of the developing heart OFT, among other applications. This work then provides a foundation to later work that aims at understanding how biomechanical stimuli affect cardiac development.