Temporal Enhancement of Three Dimensional Echocardiography by Frame Reordering

Abstract

In this work we describe a method to increase the frame rate for three dimensional ultrasound sequences of periodically moving cardiac structures by reordering the acquired volume series. Frame rate is especially important in studying intracardiac structures such as valve leaflet motion where valve closure times are on the order of milliseconds. Current commercially available systems for volumetric ultrasound imaging are limited to approximately 10–20 volumes per second. While this frame rate is sufficient for real time observation of basic cardiac morphology, to understanding cardiac dynamics requires faster frame rates. The presented work achieves higher frame rates by sampling over several beats and using a simultaneous ECG signal to accurately place the frame within the cardiac cycle. The proposed method relies on periodicity of the heart motion, and that within the temporal regions of highest velocity, structural motions of interest have the least beat to beat variability.

Description of Problem

Current commercially available volumetric ultrasound imaging systems have limited temporal resolution. While their frame rates are often sufficient for real time observation of basic cardiac morphology, to understanding cardiac dynamics, especially in pediatric imaging, requires faster frame rates.

We have developed a method to significantly increase the frame rate of three-dimensional echocardiograms. This methodology employs standard clinical three-dimensional echocardiographic equipment and standard computer technology, and thus, it should be readily adaptable to the clinical setting. This method demonstrates a marked improvement in visualization of the three-dimensional echocardiograms at increased frame rates, using this computer based algorithm for frame rate enhancement.

This algorithm re-orders image volumes (frames) of a periodically moving structure taken at a large number of instances over several periods (cardiac cycles), to reconstruct a higher frame rate sequence than the imaging system could produce on its own in a single period. This is not an interpolation method, where intermediate frames are constructed from adjacent frames. In our method, each frame is exactly as acquired, with no synchrony of frame acquisition times needed (i.e. Triggering/gating). Instead, intermediate frames come from different periods and are acquired ad-hoc. This allows for fast acquisition times on the order of 10's of second as opposed to the several minutes required for gating based methods [1].

The objective of the high frame rate algorithm is to find the temporal sequence of groups and interleave their frames into a coherent ordering with respect to the peak of the R wave obtained from the EKG. Groups are the collection of frames that span on either side of the QRS complex. This location of the frames was chosen as these frames contain all or nearly all of the images of valve opening and closing. The maximum width of this window (group size) is determined by the R-wave to R-wave interval and regularity. The actual width is manually selected so that the motions of the most rapidly intra-cardiac structures (e.g heart valves) are captured, since slower moving parts of the cardiac cycle, by their nature; do not require this type of enhancement for visualization.

Temporal Re-sequencing Algorithm

For each heart beat (i) in the acquired sequence, G_i is defined as the group of frames (F) {F_-N . . . F₀ . . . F_N } where F₀ contains a R-wave peak (i.e. Key Frames) and F_j are frames that occur before (−j) or after (+j) the key frame F₀ in the group, where j is an element of [1, N ]. The value for N is determined by the desired widow size around the QRS complex of the EKG. The offset times (the difference between the end time of the key frame to the time of the R wave peak within the key frame) are computed for the key frames in the acquisition (F₀G_i for all i) by subtracting each frames’ end time from the time of the contained R-wave peak. Sorting the resulting offset times from shortest (S) to longest (L) gives the relative times of the key frames with respect to the R wave peak. Applying the same reordering to corresponding frames, F_j, from each group gives a time coherent frame reordering: {F_{-N GS} . . . F_{-N GL}} --- {F_0GS . . . F_0GL} --- {F_NGS . . . F_NGL}.

In Figure 1, we present an illustrative example looking at only frames that contain the R-wave peaks. Within these frames, the actual peak of the R wave occurs at different offset times from the beginning of the frames (shown in the second row of figure 1). Displaying the sequence red-key-frame, blue-key-frame, green-key-frame in a cine mode will therefore result in an incoherent cardiac motion, not the desired physiologic one. We should also point out that this effect is most prominent with fast heart rates, i.e. those seen in children; and while observing rapidly moving structures i.e. valve leaflets. This effect has little impact on the display of slow moving structures, for example the ventricular wall. While these intra-frame shifts hamper the creation of stationary cine sequences, they are ideal for generating high frame rate sequences if they are ordered with respect to the EKG. In this way, frame rate can be enhanced by a factor equal to the number of heart beats (R-waves) in the sequence, assuming the EKG R-peak offset is random and uniformly distributed within the key frames.

Figure 1.

ECG and exaggerated frame durations for hypothetical ultrasound acquisition.

This work adopts a model of 3D volume formation where the data is a set of 3D volumes (consisting of voxels) acquired over time. This 4D data could be constructed either through mechanical steering of a b-mode type probe or with a matrix array probe. Using this simplified model of volume formation allows the proposed method to be generally applicable. Although not necessary for our method, additional enhancements may be possible by incorporating information about system specific volume formation, such as scan line sequence and timing, when they are available from the system vendor.

A number of events, such as manipulation of the heart, motion of the probe and respiration can cause non-periodic motion within the ultrasound frame. By using frame to frame registration [2], we can compensate for some of these motions. Using methods presented in [2], we extracted a number of co-registered feature points from subsequent frames post re-sequencing. If the motion was due to respiration or probe manipulation the relationship between these features is a rigid body transformation. Non-linear cardiac motions between two frames will be small due to re-sequencing and the subsequent high frame rate. Rigid body registration of 3D ultrasound was used successfully in [2] where frame-to-frame motions were small. With the frame-to-frame registration the sequence is stabilized using the inverse of frames registration transformations.

Within this work the term “frame rate” is used as the average number of frames in a second and not as a uniform sampling rate. Since frames are ordered with respect to the offset time from the peak of the R-wave in the EKG, there will be slight variations in peak R-wave time between frames. This leads to a sampling of frames within the cycle across beats that is dense but nonuniform. These variations are not perceivable by a human observer, but their existence should be taken into account if the re-ordered frames are used for quantitative measurements involving time.

STUDY

The three-dimensional echocardiograms were performed pre-bypass, in the operating room, while the patients were intubated and anesthetized prior to cardiopulmonary bypass. An EKG was obtained in standard fashion, and the signal relayed from standard monitors to the echo machine. The acquisitions were obtained using a matrix array three dimensional echocardiographic transesophageal probe and 3D ultrasound system, ( X7-2T, and IE33, Philips Medical Imaging, Andover, Mass). In addition to outputting a series of Cartesian volumes, this implementation platform samples the EKG at 2 kHz. R-waves are detected internally on the machine and their matching sample is marked.

The fast frame rate data acquisition was performed in the zoom-mode, which is not gated, and allows a wide field of view at low frame rates. Prior to the acquisition, orthogonal two dimensional images provided feedback ensuring the entire valve was in the three-dimensional volume data set. In the example presented here, the echocardiographic study was performed in a young patient with a severe sub-aortic membrane. There were no complications during the one minute acquisition. Mechanical breath hold was not performed, as gross volume motion distortions due to respiration were later compensated for by image stabilization.

Figure 2A shows an acquisition en face view of the aortic valve in a patient with a sub-aortic membrane. The IE33 reported an acquisition rate of 10 frames/sec. At this slow frame rate only two frames are seen between end diastole when the valve leaflets are closed, and at end systole, when the valve leaflets are seen completely open. At end diastole three equal size and equally shaped aortic leaflets are seen, without commissural fusion. At end systole the noncoronary cusp, above the effective orifice of the sub-aortic membrane, opens completely to the sides of the aortic root. The left and right cusps seem to open partially to the edges of the aortic root. Figure 2B shows the fast frame enhancement, 540 Hz, in the same patient in figure 2A. Again, this is a three dimensional echocardiograph en-face view of the aortic valve leaflets looking on the superior surfaces of the valve. Figure 2B shows the multiple frames (20 of the 54), now seen during ventricular systole. At the immediate beginning of systole the noncoronary cusp opens completely as the right and left cusps, overlying the membrane, are closed. Frame by frame the right/left commissure slowly divides from left to right, and the left cusps open to the edges of the aortic root. During this systolic interval a folding of the right leaflet can be seen.

Figure 2.

(a): This shows two consecutive frames from the original image set, acquired at 10Hz. The AOV can be seen as having opened sometime between the two consecutive frames.

(b): This shows volume renderings of reordered volumes from the first part of the ejection phase. Only alternating frames showing the leaflet opening are shown, and the native frames between which they lay are shown in Figure 2(a). Figure 2(b) starts with all three leaflets closed. Then shows the right most leaflet opening. This leaflet was not thickened by pathology and was positioned directly in the path of the outflow jet, as such it opens the fastest. A several frame delay can be seen before the thickened left leaflets begin opening.

Figure 3 is the fast frame rate acquisitions in the same patient as Figure 2, now from the beginning of aortic valve fully opened to full aortic valve closure. There are 54 intervening frames available during this diastolic time interval (show in 3A), 20 alternating are shown. An undulating appearance of the valve is perceptible at this much faster frame rate. The high frame rate sequence shows marked difference between aortic valve opening and closing for this patient, in that all three leaflets symmetrically close at the same time. From both figures 2B and 3B the multiple frames provide a clear view of the sub-aortic membrane and the anatomic position as it relates to the aortic valve leaflets. This was not possible when few frames were available during systole with the slow frame rate.

Figure 3.

(a): This shows two consecutive frames from the original image set, acquired at 10Hz. The AOV can be seen as having closed sometime between the two consecutive frames.

(b): This shows volume renderings of reordered volumes from the first part of the ejection phase. Only alternating frames showing the leaflet closing are shown, and the native frames between which they lay are shown in figure 3(a).

The frame rate increase for this case is consistent with our observation on other clinical data where frame rates were also increased in proportion to the beats in the sample. Frame rate improvement is based solely on the number of heartbeats recoverable from the US sequence with “legible” EKG peaks.

Relevance

We report the first clinical application of high frame rate three- dimensional echocardiography. As shown in this clinical example, the number of frames from the re-ordered registered study is markedly increased from the standard three-dimensional echocardiograph study (from 10fps to 540fps). In the example, the sequential time points in aortic valve opening and closure are clearly seen at fast frame rates, but they were difficult to discern in the standard frame rate imagining. Moreover, the increased number of frames allows for much improved ability to visualize juxtaposed anatomic structures, in this example the sub-aortic membrane.

The potential importance of fast frame rate three dimensional echocardiography is far reaching and yet to be explored. Future research will be to apply this advancement to pediatric and adult congenital and acquired valve disease, to enhance our understanding of valve pathology and guide planning of surgical valve repairs. As shown in this example, this fast frame rate technology will aid in the understanding of complex interactions of adjoining pathologies, i.e the interaction of a subaortic membrane and the more precise effects of aortic valve function. Moreover, this technology will certainly improve the ease to appreciate cardiac structures in the neonate and infant with fast heart rates.

Importantly, this technological advancement can be readily adaptable to standard three-dimensional echocardiographic machines, since standard three dimensional echocardiographic acquisitions are used as the basis for this method, thus allowing for expedient application in the clinical setting.

Although this study used an example from a transesophageal application, it is equally effective for transthoracic studies.

The proposed algorithm for increasing frame rate in 3D ultrasound imaging offers a way to enhance visualization of rapidly moving structures in a way that is not limited by native ultrasound imaging speeds. The clinical impact of this technological advancement requires further study. However, as valve repair techniques become more sophisticated, a better understanding of an individual patent's tissue dynamics may result in improved outcomes. The presented method has the following advantages: short acquisition times of under a minute, the ability to use any existing 3DUS machine, and the ability to analyze retrospective data of sufficient duration unlike other hardware based frame rate enhancement methods i.e. Handke et al [3].

The potential pitfalls of this method are the violations of assumptions made when applying our enhanced frame rate algorithm to 3D ultrasound data sequences. These assumptions are: periodic motion of the heart and specifically, the particular cardiac structure of interest; low beat-to-beat variability within the QRS complexes; and other factors that can affect temporal coherence.

It is well understood that heart rate and the subsequent periodicity of cardiac structure motion are variable. Even small shifts in rate could cause problems in building an accurate temporal reconstruction of the complete cycle. The solution we have adopted is to only reorder frames close to the QRS complex to remove beat to beat time interval variability. Within this window of the cardiac cycle the motions are regular from beat to beat, thus satisfying our periodicity requirement. This windowing shows the most relevant motion of intracardiac structures, including the aortic and mitral valve leaflet motions during valve opening and closing. Additionally, for frames further away in time from the QRS complex the less high speed motion is occurring within the heart; so slight re-sequencing errors in this part of the cardiac cycle are hard to detect.

In the case of 3DUS, different parts of the volume are acquired at different times, not instantaneously, due to how scan lines are sent, read and composed into a Cartesian volume. Since frames are not acquired instantaneously, it is possible for individual elements (voxels) in a frame to be slightly out of sequence. A solution to this limitation would be to reorder subcomponents (if available) instead of whole frames. In our experience with clinical data, any temporal distortions were small compared to the benefits of the improved frame rate. The time to execute the presented algorithm on desktop pc hardware is very long, in excess of 20 hours. To make the method clinically useful the stabilization part of method (the most time consuming component) is executed in parallel on off the shelf graphics processing units (GPUs). Using GPUs, we have reduced the processing time to 20 min per sequence and we expect this value to fall below 5 min with the next generation of hardware.

Based on the success of our initial results, our laboratory [4, 5] is continuing to explore the feasibility and application of fast frame rate three-dimensional echocardiography in congenital heart disease.