Sequential Turning Acquisition and Reconstruction (STAR) method for four-dimensional imaging of cyclically moving structures

Abstract

Optical coherence tomography allows for dynamic, three-dimensional (3D+T) imaging of the heart within animal embryos. However, direct 3D+T imaging frame rates remain insufficient for cardiodynamic analysis. Previously, this limitation has been addressed by reconstructing 3D+T representations of the beating heart based on sets of two-dimensional image sequences (2D+T) acquired sequentially at high frame rate and in fixed (and parallel) planes throughout the heart. These methods either require additional hardware to trigger the acquisition of each 2D+T series to the same phase of the cardiac cycle or accumulate registration errors as the slices are synchronized retrospectively by pairs, without a gating signal. Here, we present a sequential turning acquisition and reconstruction (STAR) method for 3D+T imaging of periodically moving structures, which does not require any additional gating signal and is not prone to registration error accumulation. Similarly to other sequential cardiac imaging methods, multiple fast image series are consecutively acquired for different sections but in between acquisitions, the imaging plane is rotated around the center line instead of shifted along the direction perpendicular to the slices. As the central lines of all image-sequences coincide and represent measurements of the same spatial position, they can be used to accurately synchronize all the slices to a single inherent reference signal. We characterized the accuracy of our method on a simulated dynamic phantom and successfully imaged a beating embryonic rat heart. Potentially, this method can be applied for structural or Doppler imaging approaches with any direct space imaging modality such as computed tomography, ultrasound, or light microscopy.

1. Introduction

Studies of cardiac development and function increasingly rely on the ability to image the beating embryonic heart in three dimensions at high frame rate. Optical coherence tomography (OCT) allows for non-destructive imaging through several millimeters of biological tissue with single cell resolution. It is being adopted as an accurate imaging method to study in vivo systems, as it provides both information on structure and dynamics. OCT has been used to image whole mammalian and avian embryos [1, 2], measure flow in the vasculature [3, 4], and image the mammalian and avian embryonic hearts in 4D [5–7].

Direct acquisition of volumetric (3D) OCT data, which involves scanning a beam along two directions and remains too slow for imaging fast moving structures such as cardio-vascular structures or the beating heart. Insufficient frame rates lead to image artifacts, such as motion blur, which prevent fine characterization of morphology and function. However, since cardiac structures move periodically, indirect imaging methods can be used to produce 3D+T renderings at high frame rates. One approach consists in acquiring 2D+T image bursts in fixed planes throughout the heart, each lasting the duration of a few heartbeat cycles. When the imaging planes (all parallel) are separated by a small distance, each sequence can be synchronized to its neighbors to finally produce a 3D+T data set of the beating heart with a frame rate identical to that of the 2D+T sequences (Fig. 1). This approach has been used in confocal microscopy [8,9] and OCT [5,6]. This technique is particularly attractive because reconstruction is possible based only on the imaging data itself. However, since the reconstruction proceeds by synchronizing image sequences recursively from one imaging plane to the next, this method is susceptible to error propagation.

Fig. 1.

Sequential acquisition using parallel slice geometry [8,9,5]. (a) Direct 3D volume is obtained by collecting multiple B-scans (y-planes) (b) Sequential imaging of fast B-scans at a fixed y-positions (c) If triggered at the same time in the heart cycle, or subsequently synchronized, the frames can be re-assembled to form a 3D+time series at high frame rate.

Alternatives to this image-intrinsic reconstruction is to use a separate gating signal so imaging can be triggered at a precise time in the cardiac cycle. Known as prospective gating in magnetic resonance imaging (MRI) and X-ray computed tomography (CT) [10–14], gated cardiac imaging does not rely on post-acquisition synchronization but usually requires additional hardware to collect the gating signal. In OCT, gated imaging has been implemented by at least two research groups [15, 16], yet it is particularly challenging since two independent OCT scanning heads have to be aligned around the tiny embryo.

Techniques that can take advantage of a single gating signal yet do not require additional hardware are therefore desirable. In CT, Dinkel et al. have proposed to use an intrinsic gating signal obtained from the projections restricted to an ROI that includes the heart [17] and so-called kymograms [11]. ECG-free algorithms have also been applied to ultrasound imaging [18, 19]. ECG-free algorithms have also been applied to ultrasound imaging [18, 19]. In MRI, Spraggins [20] describes a method in which two image series are acquired alternatively, one in a fixed common area, serving as the gating signal. Alternative k-space sampling procedures have been proposed, which use an interleaved radial k-space sampling trajectory such that each readout samples the k-space center, providing an intrinsic gating signal [21–23].

In OCT, Liu et al. proposed to adjust the phase lags between set of parallel planes, by acquiring, in addition to the series of parallel slices, one orthogonal slice [7]. The parallel planes are first aligned with each other then the alignment of each parallel plane is adjusted based on the line in the orthogonal plane with which each plane intersects. Since all parallel planes share a different line with the orthogonal reference plane, the orthogonal slice must be chosen carefully and may not intercept the beating heart in its entirety.

Here, we present a method—sequential turning acquisition and reconstruction (STAR)—for imaging periodically moving structures that does not require measuring an additional gating signal yet still overcomes the error accumulation limitation introduced by gating-free recursive synchronization procedures. Our STAR approach proceeds similarly to nongated approaches where multiple fast images series are consecutively acquired, but instead of considering a set of parallel imaging planes, it involves 2D+T series acquired in imaging planes that share a common axis (Fig. 3(a)). In STAR, all planes share the same common line. This line is chosen, during acquisition, to be of good quality (e.g. in the center of the heart). This geometry is made possible by the fact that traditional OCT data are acquired as a collection of one-dimensional in-depth scans (axial scans or A-scans) serving as single units of information. Two-dimensional images (or B-scans) are obtained by laterally moving the OCT beam with two galvanometer-controlled mirrors (see Fig. 2). Since the direction of the A-scans is fixed along the z-direction the B-scan planes are always vertical. However, the user can freely adjust the orientation and lateral position of the plane by controlling the mirrors’ oscillations.

Fig. 3.

Sequential turning acquisition and reconstruction (STAR) method. (a) Image sequences are acquired (without gating) along planes oriented radially. Note that all sequences share a common line, the rotation axis, where the radial planes intersect. (b) Intensity of the central line of slice sequences versus time (kymograph) acquired sequentially at different orientations θ_i. (c) Synchronization of the sequences, with respect to the sequences θ₀ is carried out by time-warping the kymographs. (d) The warping function obtained from the intensity values on the common axis is then applied to the other pixels in the sequences, resulting in synchronized radial sequences that cover one (or possibly more) period(s). (e) the synchronized sequences can either be visualized as cutting planes or (f) interpolated onto a Cartesian grid for volume rendering.

Fig. 2.

X–Y scanner allows recording vertical planes arbitrarily rotated around a vertical axis. The signal emanating from the fiber of one arm of an OCT system is sent through an X–Y scanner (consisting of two scanning mirrors SM1 and SM2 and two relay lenses RL1 and RL2) via a collimating lens lens (CL), into a microscope objective (MO) and to the heart (H) inside the sample (S). The inset shows how one 2D OCT image frame (B-scan) is assembled from a collection of 1D signals (A-scans) that record the scattering within the sample along the vertical direction. The double arrow indicates the scanning motion produced by the galvanometer-mounted mirrors that can oscillate around a fixed axis.

Also, since the central line of all 2D+T time-sequences represents measurements taken over the same physical line (the rotation axis), the central 1D+T signal from each sequence can be used for synchronization. The central lines serve as intrinsic temporal gating signals for post-acquisition synchronization. After the synchronization, the data are transformed from cylindrical into orthogonal coordinates to reveal the 3D+T representation of the periodically moving structure.

Our method bears some similarity with a recently presented method by Happel et al. [24] for OCT of chicken embryos. There, images are acquired in a geometry identical to ours (with a common A-scan) after which each frame in a sequence at a given angle is assigned a new time stamp (based on an algorithm that assumes perfect, yet unknown, periodicity in the sequence). Although this algorithm can handle variations in the heart frequency that occur in between angles, it does so by reshuffling the images, producing a new sequence in which images from multiple heartbeats are interleaved. Our reconstruction algorithm differs in that it applies a nonuniform, elastic registration that preserves the order of the frames and allows reconstruction even in the presence of slight, local variations in the heartbeat that occur during acquisition at a single angle. By preserving the sequence order, our algorithm produces reconstructions in which aperiodic motion features within a sequence at a given angle, such as moving red blood cells, could be analyzed to infer lateral blood flow velocities within one plane (e.g. via block matching or optical flow techniques).

In Section 2 we formally introduce the acquisition and reconstruction method. Then, in Section 3, we present validation reconstructions obtained with the parallel-slice and proposed STAR method. We also present experimental results from data collected in the beating heart of an embryonic rat.

2. Method

We consider a dynamic, three-dimensional object, whose measured intensity I(x,y,z,t) is periodic, translating to

$$I(x,y,z,t)\approx I(x,y,z,t+T),\forall t\in \mathbf{R},$$ (1)

where T is the period of the cardiac cycle. Since, in practice, signals are never perfectly periodic, we relax this constraint by assuming that, given a reference period T, the length of other periods may only vary by an estimated percentage α of the reference period such that the minimal and maximal lengths of the cycles are T_min = (1 – α)T and T_max = (1 + α)T, respectively.

For cardiac imaging in live embryos, intensity is varying rapidly with time (the embryonic heart rate is approximately 2–3 beats per second) and instantaneous acquisition of three-dimensional data in the spatial dimension (corresponding to ideal sampling at a fixed point in time) is technologically challenging. Rather, data along the spatial dimensions are acquired sequentially to build a multi-dimensional image. Images are built by sequentially scanning individual voxels, lines, or, at best, full planes. Here we assume that a 2D plane (B-scan) can be acquired in a single shot with high repetition rate. Specifically, we assume that the imaging system allows for instantaneous acquisition of a full two-dimensional plane parametrized by the coordinates (r,z), defined by a rotation of angle θ ∈ [0,π) around the z-axis of the plane y = 0 (see Fig. 3 (a)). We consider the set of measurements

$$I_{\theta }(r,z,t)=I(r\text{cos}\theta ,r\text{sin}\theta ,z,t+t_{\theta }),$$ (2)

for rotation angles θ ∈ [0,π), radial coordinates r ∈ [−r_M, r_M], and axial coordinates z ∈ [0,z_m], over time t ∈ [0,L), each sequence being triggered at a time offset t_θ. This set of measurements corresponds to the sequential acquisition of 2D+T image sequences of duration L and is schematically depicted in Fig. 3(b). The acquisition of these sequences is not triggered at any particular time in the cardiac cycle (non-gated imaging), but to ensure that at least one complete cycle of the heartbeat (starting at any arbitrary time in the cycle) can be extracted from each sequence, the acquired sequences must be of length L ≥ 2T_max, that is, they must be twice the duration of the longest heartbeat period estimate.

These sequences are acquired in a radial, star-like pattern, with all planes intersecting at and sharing the z-axis (see Fig 3 (a)). The first period of the sequence—I₀(r,z,t) for t = [0,T) (in the plane y = 0)—is considered the reference and the other sequences are synchronized to it. The nonrigid synchronization problem is equivalent to finding a set of angle-dependent warping functions w_θ(t) for all rotation angles θ ∈ [0,π) such as to minimize, along the shared z-axis (that is, the axis defined by r = 0, see Fig. 3(c)), the functional

$$\begin{array}{ll}{Q}_{\theta }\left\{w\right\}=\hfill & (1-\lambda ){\int }_0^{z_m}{\int }_0^T|I_0(0,z,t)-\hfill \\ \hfill & I_{\theta }(0,z,w(t))|\text{d}t\text{d}z+\lambda {\int }_0^T\left|\frac{\text{d}}{\text{d}t}w(t)-1\right|\text{d}t.\hfill \end{array}$$ (3)

The first double integral over time and the axial direction z compares a temporally warped sequence (a sequence whose temporal axis is deformed) I_θ(0,z,w(t)) with the template I₀(0,z,t), while the second integral keeps the extent of this deformation in check by penalizing warping functions w(t) that stretch or compress the time axis. These two contributions to the cost function are balanced by the parameter 0 ≤ λ < 1 to favor either good matching of the warped and reference sequences (λ = 0) or the temporal integrity of the warped sequence (λ → 1). We note that the warping functions w are in the set 𝓜 = {w ∈ C¹([0,T))|0 ≤ w(t) < L and w(t₁) < w(t₂),t₁ < t₂} of continuous, non-negative, and strictly increasing functions bounded by L and defined over the interval [0,T). We then determine a warping function that minimizes the cost function. This minimization is carried out using a previously described dynamic programming algorithm, which traces a minimum-cost path in a two-dimensional similarity graph [25]. For each angle, the optimal warping function produced by this algorithm is w_θ ∈ {w ∈ 𝓜|Q{w} = min_w′∈𝓜 Q_θ {w′}}. For example, if λ = 1, the matching is rigid and warping functions are of the form w(t) = t₀ + t, a uniform offset in time. As λ < 1 decreases, the resulting warping functions can become non-linear to better match the data.

Once the warping functions are determined, we compute time-interpolated series (according to the temporal warping functions w_θ) for the measured intensity series I_θ for each angle θ (see Fig. 3(d))

$$\overline{I_{\theta }}(r,z,t)=I_{\theta }(r,z,w_{\theta }(t)),t\in [0,T).$$ (4)

Although the synchronized series could be visualized per se (see Figs. 3(e) and 6(a)), it is usually desirable to have a volumetric representation of the data in Cartesian coordinates (see Fig. 3(f)). The final step of the reconstruction therefore consists in converting, at each time point t ∈ [0,T) from a cylindrical geometry (θ,r,z) to a Cartesian grid (x,y,z). Given samples $\overline{I_{{\theta }_i}}(r_j,z,t)$ for uniformly spaced angles θ_i = iΔθ, 0 ≤ i < N_θ and uniformly-spaced radial coordinates r_j = −r_M + jΔr, 0 ≤ j < N_r (where Δθ = π/N_θ and Δr = 2r_M/(N_r − 1) are the angular and radial spacings, respectively), we can define a cubic spline interpolating function

$${\tilde{I}}_{\theta }(r,z,t)=\sum _{i=0}^{N_{\theta }}\sum _{j=0}^{N_r}{c}_{i,j}(z,t){\beta }^3\left(\frac{\theta -{\theta }_i}{\mathrm{\Delta }\theta }\right){\beta }^3\left(\frac{r-r_j}{\mathrm{\Delta }r}\right),$$ (5)

where the centered cubic B-spline function β³ is defined as the 4-fold convolution of β⁰, the indicator function over the interval [−1/2,1/2],

$${\beta }^3(x)={\beta }^0*{\beta }^0*{\beta }^0*{\beta }^0(x),$$ (6)

and where the B-spline coefficients c_i,j(z,t) in the expansion (5) are obtained by recursive filtering [26] such that the interpolating function interpolates the data at the sample points (θ_i, r_j)

$${\tilde{I}}_{{\theta }_i}(r_j,z,t)=\overline{I_{{\theta }_i}}(r_j,z,t).$$ (7)

Fig. 6.

Images of the beating embryonic rat heart at 10.5 days of gestation cultured on the imaging stage obtained via Sequential Turning Acquisition and Reconstruction (STAR). (a) Representative cross-sections through the synchronized reconstruction demonstrating the positions of the imaging planes (6 out of 60 angles). (b) XY cross-section of STAR data after polar-to-Cartesian interpolation. See also Media 2 ^{(22.5MB, MOV)} , Media 3 ^{(2.6MB, MOV)} , and Media 4 ^{(2.4MB, MOV)} .

Finally, the recovered intensity I_r on a Cartesian grid is given by

$$I_{\text{r}}(x,y,z,t)={\tilde{I}}_{\theta (x,y)}(r(x,y),z,t),$$ (8)

where the Cartesian to polar coordinate transform is given by

$$\begin{array}{l}(x,y)\mapsto (r(x,y),\theta (x,y))=\\ \{\begin{array}{ll}\left(\sqrt{x^2+y^2},\text{atan2(}y,x)\right)\hfill & \text{if\hspace{0.17em}0}\le \text{atan2(}y,x)\le \pi \hfill \\ \left(-\sqrt{x^2+y^2},-\text{atan2}(y,x)\right)\hfill & \text{if\hspace{0.17em}}-\pi <\text{atan2}(y,x)<0\hfill \end{array}\end{array}$$ (9)

where −π < atan2(y,x) = Prarg(x + iy) ≤ π is the principal argument of the complex number x + iy, which is restricted to the interval (−π,π].

3. Results

To evaluate the reconstruction accuracy of STAR and illustrate the advantages over sequential parallel-slice acquisition and reconstruction, we carried out a simulation experiment with a synthetic, periodically deformed heart tube phantom, presented in [8] and shown in Fig4(a). Briefly, the intensity of a static tube is defined for any real value of x, y, and z in three-space by a parametric analytic equation. To generate intensity volumes over time, an affine transform of the spatial coordinates at which the intensity is to be evaluated is carried out, and the heart-tube function is evaluated at the transformed coordinates. The affine transform is defined by a matrix whose coefficients are given by a weighted sum of sinusoidal and cosinusoidal functions at harmonic frequencies. Here, acquisition of data was simulated either by evaluating the phantom values sequentially for different y-sections (see Fig. 4(d)) or by acquiring sections in a radial geometry (Fig. 4b) to match our proposed acquisition conditions. For each time-series, we introduced a random time-shift (uniform distribution in [−T,T]), similarly to the approach in [8].

Fig. 4.

Still of heart phantom (a) Reference image of synthetically generated heart phantom. (b–d) STAR (b) sequentially acquired slices in radial geometry (rotation axis is perpendicular to page), before alignment (c) STAR reconstructed volume (d) absolute difference between reference and reconstruction (e–g) Parallel geometry (e) sequentially acquired slices in parallel geometry, before alignment (f) post-acquisition aligned reconstruction (g) absolute difference between reference and reconstruction. See also Media 1 ^{(7.7MB, MOV)} .

We next reconstructed the data in the STAR and conventional geometries (Fig. 4(c) and (f), respectively). We compared the resulting data volumes with the reference data volumes and computed the absolute difference voxel by voxel resulting in a difference data volume (see Fig. 4(d) and (g) for the difference images corresponding to reconstructions from sequential parallel-section imaging and STAR imaging, respectively).

By randomly selecting (following a zero-mean normal distribution with σ = 0.1) the weights of sine and cosine functions up to the third harmonic (higher harmonics were set to zero), we generated 100 dynamic reference phantoms as well as corresponding sequential parallel-slice and radial measurement series (50 slices in each case), introducing random time-shifts (following a uniform distribution in [−T,T]) for each section. We then ran our algorithms to synchronize the slices and recover volumes for both acquisition geometries. Since the ground-truth shifts were known, we could accurately estimate reconstruction errors: for each synchronized slice time series, we compared the local time-shifts of the frames produced by the warping functions, and subtracted them from the actual shift introduced during the simulation. We then calculated the mean of the absolute value of the errors of all frames in a given section. Figure 5 shows the average synchronization error for image series at a given depth or orientation (computed over the 100 simulated experiments) as a function of distance from the reference slice (in the case where volumes were reconstructed from parallel slices) and angular distance from the reference slice (in the case of STAR).

Fig. 5.

Synchronization error does not accumulate with STAR reconstruction (which synchronizes radial slices that all share common line to the same line in the reference slice) unlike methods that sequentially record parallel-slices at increasing distance (on both sides) of a reference slice then proceed recursively to synchronize the slices. Each point corresponds to the absolute value of the synchronization error for one image sequence, obtained by averaging the absolute errors at all time-points. Error bars correspond to the standard deviation over 100 simulations, divided by $\sqrt{100}$.

In the case of recursive reconstruction, the average time-synchronization error increases as the distance to the reference slice increases. This is visible in Figure 5. Visually, this translates to high difference images toward the top and bottom in Fig. 4(g)). This degradation can be explained by the fact that the parallel section algorithm proceeds recursively: with 50 slices to align, the error propagates (the central slice was used as a reference).

Conversely, for the STAR reconstruction the synchronization proceeds non-recursively (only the central portion of each slice is aligned to the central portion of the reference slice) the error remains in check (Fig. 5), even for large number of slices and for angles very different from that of the reference. Large differences visible in the STAR reconstruction error image near the top and bottom of the tube (see Fig. 4(d)) are due to the fact that the reconstruction is limited to a cylinder. Within the cylinder, the difference image does not show a systematic error. Rather, remaining visible errors are mostly due to the lower sampling step away from the center in the STAR geometry.

To evaluate the feasibility of our method in practice, we acquired image sequences of a live 10.5 day rat embryo using a swept-source OCT (SS-OCT) system as described previously in [27] (procedures were approved by the Animal Care and Use Committee of the University of Houston). Briefly, the experimental system was customized from the Thorlabs SL1325-P16. It utilizes a broadband swept-source laser with a central wavelength of λ₀ = 1325 nm and a bandwidth of Δλ = 110 nm; output power is P = 12 mW; the A-line scanning frequency was 16 kHz. An interferogram is detected by a balanced-receiver configuration that reduces source intensity noise as well as autocorrelation noise from the sample Thorlabs, PDB140C and is digitized using a 14-bit digitizer. A Mach-Zehnder interferometer MZI-based optical frequency clock is used to calibrate the OCT interference signals from the optical time-delay domain in the frequency domain before application of fast Fourier transform FFT algorithms. An FFT is used to reconstruct an in-depth profile A-scan of OCT intensity from a single scan over the operating wavelength range. By 2-D mechanical scanning over the area of up to 10 ×10 mm, the 3-D structure of the embryo can be revealed.

Each time lapse consisted of 200 frames of 256 × 512 pixels (512 pixels per A-scan) acquired at 51 frames per second. The whole data set consisted of 60 image sequences; the angle between the neighboring time lapses was set to 3°. The central line (axis of the imaging plane rotation) was positioned over the highly moving structure to ensure strong reference signal for synchronization. We were able to successfully reconstruct the dynamics of the beating heart in the embryo using the proposed algorithm (Fig. 6 and Media 2 ^{(22.5MB, MOV)} , Media 3 ^{(2.6MB, MOV)} , and Media 4 ^{(2.4MB, MOV)} ).

4. Conclusion

In this paper, we have provided an acquisition and reconstruction algorithm for imaging of dynamic cardio-vascular structures whose motion is cyclical. The STAR algorithm, relies on the intrinsic signal provided by an axis of data common to all slices. Our algorithm has several advantages, in particular over previously proposed non-gated imaging and reconstruction algorithms in which data are acquired over parallel sections instead of radial sections. Our method achieves accurate reconstructions without requiring any additional hardware for acquiring a gating signal. Unlike existing nongated synchronization approaches where slices are synchronized to neighboring slices and errors may accumulate with each round of synchronization, in the STAR method, there is no accumulation of the error since all slices are synchronized to the same signal. If the sample drifts during acquisition, this becomes obvious after reconstruction, since after 180° rotation, the imaging plane is a mirror-image of the initial position. We demonstrated the method’s improved accuracy on simulated data and its practical applicability for in vivo imaging of embryonic rats.

Since sampling occurs in polar coordinates, the spatial resolution is non-uniform: it is finer towards the axis of rotation and coarser as the radial distance increases. In our experiment, where the extent of the OCT scan is 400 μm (and represents the size of a 10.5 dpc embryonic rat heart) and the imaging plane is rotated by 3° for each time series, the distance between neighboring slices at the radially most distant border is about 10 μm, which is similar to the step used for parallel-slice imaging approaches.

Use of an elastic synchronization method that does not assume the interval between heartbeats to remain constant ensures robustness to slight variations in heartbeat period, which we previously showed to be beneficial in the case of parallel-slice imaging [9].

Our method is general and could be applied for structural or any other (e.g. Doppler) imaging of periodically moving structures with optical coherence tomography, or other modalities that allow acquiring images according to the proposed geometry.