A Fast Normalized Cross-Correlation Calculation Method for Motion Estimation

Abstract

High-precision motion estimation has become essential in ultrasound-based techniques such as time-domain Doppler and elastography. Normalized cross-correlation (NCC) has been shown as one of the best motion estimators. However, a significant drawback is its associated computational cost, especially when RF signals are used. In this paper, a method based on sum tables developed elsewhere is adapted for fast NCC calculation in ultrasound-based motion estimation, and is tested with respect to the speed enhancement of the specific application of ultrasound-based motion estimation. Both the numerator and denominator in the NCC definition are obtained through pre-calculated sum tables to eliminate redundancy of repeated NCC calculations. Unlike a previously reported method, a search region following the principle of motion estimation is applied in the construction of sum tables. Because an exhaustive search and high window overlap are typically used for highest quality imaging, the computational cost of the proposed method is significantly lower than that of the direct method using the NCC definition, without increasing bias and variance characteristics of the motion estimation or sacrificing the spatial resolution. Therefore, high quality, high spatial resolution, and high calculation speed can be all simultaneously obtained using the proposed methodology. The high efficiency of this method was verified using RF signals from a human abdominal aorta in vivo. For the parameters typically used, a real-time, very high frame rate of 310 frames/s was achieved for the motion estimation. The proposed method was also extended to 2-D NCC motion estimation and motion estimation with other algorithms. The technique could thus prove very useful and flexible for real-time motion estimation as well as in other fields such as optical flow and image registration.

I. Introduction

Motion estimation is performed in several ultrasound-based techniques such as blood flow imaging [1]–[4], elastography or elasticity imaging [5], [6], phase-aberration correction [7]–[9], and strain compounding [10]. Many methods have been proposed for motion estimation, including phase-domain methods [11], [12], time-domain or space-domain methods [13], [14], and spline-based methods [15], [16]. Time-domain (1-D) or space-domain (2-D) methods have been widely and frequently used because of their high accuracy, precision, and resolution, and relative simplicity in implementation [3]. Typically, an observation reference signal/image window (i.e., kernel) is defined in one frame of ultrasound data (either B-mode image, RF, or envelope signal) and subsequently compared with different candidate windows (i.e., comparison windows) from another frame of ultrasound data, within a pre-defined search range. A cost function is calculated to quantify the similarity, or matching, between the reference and comparison windows. The motion is estimated from the temporal or spatial shift between the reference window and the best-match comparison window, which gives the maximum (or minimum) cost function value.

Numerous cost functions have been developed for different motion estimation methods. These methods include, but are not limited to, non-normalized cross-correlation (CC), normalized cross-correlation (NCC), zero-mean normalized cross-correlation (ZNCC, i.e., normalized covariance), sum of absolute differences (SAD), and sum of squared differences (SSD) [13], [14]. Numerous efforts have been concentrated on the performance comparison of different methods [13], [14], [17]–[20].

The ultimate objective of motion estimation techniques is to obtain images at high accuracy, high spatial resolution, and low computational cost. However, there exist several trade-offs between accuracy, spatial resolution, and computational cost: 1) compared with B-mode-based motion estimation, RF signal-based motion estimation is more accurate [21] and provides higher spatial resolution, but is associated with higher computational cost because of the higher RF sampling rate; 2) compared with other estimators, the NCC algorithm is considered as one of the most accurate motion estimators [13] but at a higher computational cost because of its complex definition; 3) high window overlap (or, small shift between successive reference windows) in NCC improves the spatial resolution for a given set of acoustic parameters [22], but increases the computational cost because it incurs a larger number of calculations; 4) within a certain range, a larger window size reduces the jitter error of motion estimation [21], [23], [24] but at the expense of increased computational cost. Because of these trade-offs, estimation quality is typically sacrificed to obtain real-time motion estimation, e.g., through the use of B-mode instead of RF frames or alternative estimators and parameters.

The NCC is generally considered to be the gold standard of motion estimation because it compensates for local variation (such as scaling) of the signal energy [13]. However, its higher computational cost is a significant drawback in its real-time application, especially when highly sampled RF signals and an exhaustive search are used. An exhaustive search, or, full search, entails that the reference window is contrasted with every possible comparison window within a pre-specified search range.

An intuitive approach to reducing the computational cost in motion estimation is to use a guided, instead of an exhaustive, search [25]–[27]. In a guided search, the estimated motion of neighboring regions is used as a priori information, thus significantly reducing the search region and its computational cost [25]–[27]. Methods with a guided search have been successfully applied to strain imaging of breast tumors [25], [27], where an external, static compression is used and the a priori information is easier to obtain. Motion estimation of cardiovascular tissues has been playing an increasingly important role in the diagnosis of cardiovascular disease [28]–[30]. The distribution of the internal, physiologic motion within cardiovascular tissues is too complex for any a priori information to be used. An exhaustive search is thus required. This more general methodology also acts as a benchmark for evaluating the efficiency of all fast algorithms.

The NCC and ZNCC have been widely used in the fields of template/block matching, object recognition, image registration and image/video compression. As will be shown later [see (1)], the definition of NCC consists of a denominator (i.e., energy terms) and a numerator (i.e., standard CC term). A relatively efficient way of calculating the NCC is to compute the standard CC (numerator) in the frequency domain using the fast Fourier transform (FFT) and then compute the energy terms in the denominator using sum tables (i.e., tables that store cumulative sums, in this case, cumulative sums of the squared signal) [31]. Sum tables have also been used in the calculation of mean and energy terms in ZNCC [32]. However, the calculation of the numerator dominates the computational cost of the NCC. As a result, even though the FFT is used, it remains computationally intensive. Basis functions have been proposed to approximate the template (i.e., reference window) and obtain substantial computational gains over the FFT-based methods [33], [34]. This method relies on selecting a suitable set of basis functions and only provides an approximation of NCC or ZNCC. Tsai and Lin proposed a fast ZNCC method for defect detection which used the sum tables to calculate both the numerator and denominator [35]. In their application, both reference and comparison windowed sub-images were simultaneously shifted pixel by pixel across the images, with low correlation between them indicating the defect. However, the calculation of the ZNCC in their application did not include a search region, as emphasized in this paper, because the reference and comparison windows were always coincident in their locations. In motion estimation, the location of the maximum correlation within a search range indicates the shift or, motion, between the reference and comparison windows. The search range is therefore essential for the purpose of motion estimation.

In this paper, a fast method for NCC with an exhaustive full search is hereby proposed for motion estimation by using the sum-table scheme [31], [35]. This method, referred to as the sum-table method, includes the construction of one sum table for the reference signal energy, one sum table for the comparison signal energy, and several sum tables for the non-normalized CC term between the reference and comparison signals. Both the denominator and numerator of the NCC are calculated using sum tables, unlike the previously reported method for which only the denominator was taken into account [31]. Compared with [35], this method also consists of a search region, as required by the principle of motion estimation. An example of motion estimation using high-frame-rate RF signals from a human abdominal aorta in vivo is used to test the high efficiency of the proposed method. As will be shown in this paper, the proposed method significantly reduces the computational cost of the NCC algorithm, thus allowing for the simultaneous use of RF signals, large window sizes, and high window overlaps for real-time imaging. Therefore, high-quality motion estimation can be achieved at both high spatial resolution and low computational cost.

II. Methods

A. Normalized Cross-Correlation (NCC)

The reference and comparison signals are referred to as f(n) and g(n), respectively, where n is the sample index (1 ≤ n ≤ M, M is the total number of samples). The NCC between the reference and comparison windows, R_NCC, is defined as [13]

$$R_{\text{NCC}}(u,\tau )=\frac{{\sum }_{n=u}^{u+W-1}f(n)g(n+\tau )}{\sqrt{{\sum }_{n=u}^{u+W-1}{f}^2(n)·{\sum }_{n=u}^{u+W-1}{g}^2(n+\tau )}},({\tau }_1\le \tau \le {\tau }_2),$$ (1)

where the reference window is located within the interval of [u, u + W − 1], u is the origin of the reference window, W is the window size, as shown in Fig. 1, τ is the shift between the comparison and reference windows, and [τ₁, τ₂] is the search range determined by the range of physiologic displacements.

Fig. 1.

A schematic diagram of redundant calculation of the (a) energies of the (upper) reference and (lower) comparison windows, and (b) non-normalized cross-correlation between the reference and comparison windows. The overlapping regions (i.e., redundant calculations) were indicated by the gray areas.

As shown in (1), the NCC calculation consists of three terms, i.e., the energy of the reference window $\left({\sum }_{n=u}^{u+W-1}{f}^2(n)\right)$ in the denominator, the energy of the comparison window $\left({\sum }_{n=u}^{u+W-1}{g}^2(n+\tau )\right)$ in the denominator, and the standard (i.e., non-normalized) CC between these two windows $\left({\sum }_{n=u}^{u+W-1}f(n)g(n+\tau )\right)$ in the numerator. For each reference window, both the non-normalized CC term between the reference and comparison windows and the energy of the comparison window need to be calculated for every search τ within the range of [τ₁, τ₂]. This calculation is repeated for each reference window (i.e., each value of u) across the entire signal length. Therefore, NCC-based motion estimation with an exhaustive search is extremely time-consuming.

Motion estimation consists of a series of template matching. In the entire process of motion estimation, the reference window (i.e., the template) varies, by sliding it across the entire signal length. Typically, the separation between successive reference windows in space, i.e., the window shift (ΔW), is smaller than the window size (W), as shown in Fig. 1. In other words, the reference windows overlap.

B. Fast NCC Calculation Using the Sum-Table Method

The efficient NCC calculation method based on sum tables relies on the fact that most calculations are redundant because of the exhaustive search of the comparison windows and high overlap between the reference windows. When the search is equal to τ and τ + 1, the comparison windows overlap, as shown in Fig. 1(a). Therefore, the energy calculation within the overlapping region becomes redundant. According to this, the sum table for the comparison window, $s_g^2(u)$, is constructed as follows:

$$s_g^2(u)=\{\begin{array}{ll}{g}^2(u)+s_g^2(u-1)\hfill & (1\le u\le M)\hfill \\ 0\hfill & (u=0)\hfill \end{array},$$ (2)

where M is the total length (in samples) of the signal used in motion estimation.

The energy of the comparison window [i.e., the first term of the denominator in (1)] can then be directly calculated as

$$\sum _{n=u}^{u+W-1}{g}^2(n+\tau )=s_g^2(u+W-1+\tau )-s_g^2(u-1+\tau ).$$ (3)

Similarly, the calculations of different reference windows become redundant when there is an overlap between them (i.e., ΔW < W), as shown in Fig. 1(a). Similar to (2), the following sum table, $s_f^2(u)$, is constructed as follows:

$$s_f^2(u)=\{\begin{array}{ll}{f}^2(u)+s_f^2(u-1)\hfill & (1\le u\le M)\hfill \\ 0\hfill & (u=0)\hfill \end{array}.$$ (4)

The energy of the reference window [i.e., the second term of the denominator in (1)] can be calculated as

$$\sum _{n=u}^{u+W-1}{f}^2(n)=s_f^2(u+W-1)-s_f^2(u-1).$$ (5)

Because successive reference windows usually overlap, the entire calculation of the numerator in (1) is also redundant. For a search τ, as shown in Fig. 1(b), the CC between the reference window at u and the comparison window at u+τ, and the CC between the successive reference window at u+ ΔW and the comparison window at u+τ+ΔW share some redundant information, because both reference (and comparison) windows substantially overlap (i.e., ΔW < W). This holds for every search τ. Based on this fact, the following sum table, s_f,g(u, τ), is constructed for each search τ (τ₁ ≤ τ ≤ τ₂),

$$s_{f,g}(u,\tau )=\{\begin{array}{ll}f(u)·g(u+\tau )+s_{f,g}(u-1,\tau )\hfill & (1\le u\le M)\hfill \\ 0\hfill & (u=0)\hfill \end{array}.$$ (6)

Therefore, the numerator in NCC given by (1) can be calculated from the sum tables through

$$\sum _{n=u}^{u+W-1}f(n)g(n+\tau )=s_{f,g}(u+W-1,\tau )-s_{f,g}(u-1,\tau ).$$ (7)

In summary, the calculation of the NCC is converted to two sum tables, i.e., $s_f^2(u)$ and $s_g^2(u)$, for the denominator and τ₂ − τ₁ + 1 sum tables, i.e., s_f,g(u, τ), for the numerator, as defined by (2), (4), and (6), respectively.

$$R_{\text{NCC}}(u,v,{\tau }_x,{\tau }_y)=\frac{{\sum }_{m=u}^{u+W_x-1}{\sum }_{n=v}^{v+W_y-1}f(m,n)g(m+{\tau }_x,n+{\tau }_y)}{\sqrt{{\sum }_{m=u}^{u+W_x-1}{\sum }_{n=v}^{v+W_y-1}{f}^2(m,n)·{\sum }_{m=u}^{u+W_x-1}{\sum }_{m=v}^{v+W_y-1}{g}^2(m+{\tau }_x,n+{\tau }_y)}}$$ (8)

C. 2-D Motion Estimation Using the Sum-Table Method

In the previous section, a 1-D kernel in a 1-D search was used to estimate the 1-D motion. To estimate 2-D motion, a 1-D kernel in a 2-D search can be used [36]. The RF signals are interpolated along the lateral direction of the ultrasound beam to perform sub-beam lateral displacement estimation. The expansion of the sum-table method in this case is straightforward. For both original and interpolated beams within the lateral search range, sum tables given by (4) and (6) are calculated.

The sum-table method can be easily extended to a 2-D/3-D kernel in a 2-D/3-D search, where the windows overlapped in two/three directions, thus suggesting high efficiency of the proposed method. The NCC between the 2-D reference window f(m,n) and comparison window g(m,n) is defined as (8), see above, where the reference window is located in the interval of [u, u + W_x − 1] and [v, v + W_y − 1] in x and y directions, W_x and W_y denote the window sizes in x and y directions, τ_x and τ_y are the searches along the x and y directions, respectively. Similar to (1), most calculations in (8) are redundant when an exhaustive search and high window overlap are used. The sum-table method for fast 2-D NCC calculation (i.e., motion estimation with a 2-D kernel in a 2-D search) is described and tested in Appendix A.

D. Computational Complexity

Table I lists the numbers of arithmetic operations required for the direct method and the proposed sum-table method for the NCC calculation, respectively. The direct method calculates the NCC by directly using its definition given by (1). Assuming the search range R = τ₂ − τ₁ + 1 ≫ 1, its computational complexity is approximately O(nWR), where O(·) denotes the order of magnitude, n is the total number of reference windows across the entire signal length, and W is the window size.

TABLE I.

The Comparison of Arithmetic Operations Between the Direct Method and the Proposed Sum-Table Method for the NC Calculation.

	Direct method	Sum-table method
Construction of $s_f^2$ in (4)	0	M multiplications M additions
Construction of $s_g^2$ in (2)	0	M multiplications M additions
Construction of sf,g in (6)	0	MR multiplications MR additions
Calculation of Σf2 in (1) or (5)	nW multiplications nW additions	n subtractions
Calculation of Σg2 in (1) or (3)	nWR multiplications nWR additions	nR subtractions
Calculation of Σf · g in (1) or (7)	nWR multiplications nWR additions	nR subtractions
Normalization in (1)	nR multiplications nR root square operators nR divisions	nR multiplications nR root square operators nR divisions

W = window size; ΔW = window shift; R = search range; n = number of reference windows.

The computational complexity of the sum-table method is mainly determined by the construction of sum tables in (2), (4), and (6). Assuming R ≫ 1, the computational complexity is approximately O(MR), where M is the total signal length. Therefore, the complexity of the sum table construction mainly depends on the signal length (M) and the search range (R), and is independent of the window size (W).

Assuming M ≫ W and M ≫ R, the number of windows is approximated by n = M/ΔW. Therefore, the complexity of the direct method is approximately O(M(W/ΔW)R). Compared with the complexity of the sum-table method, i.e., O(MR), the direct method is significantly more computationally expensive in the case of high overlap (i.e., W/ΔW ≫ 1). In other words, higher overlaps are associated with higher computational savings of the sum-table method.

E. In Vivo Performance Evaluation

The proposed method was compared with the direct calculation method [i.e., using (1) without sum tables] using in vivo RF signals. The RF frames were acquired from a human abdominal aorta using a Sonix RP system (Ultrasonix Medical Corp., Burnaby, BC, Canada) with a phased array (3.3 MHz) at a frame rate of 194 Hz [37], [38]. Each frame consisted of 32 beams with 2592 samples per beam (i.e., 10-cm depth) using a sampling frequency of 20 MHz.

C++ (Visual Studio .NET 2003, Microsoft Corp., Seattle, WA) was used to implement both the direct and proposed sum-table methods; 64-bit, double precision floating-point numbers were used for all computations. The computational time of both methods was compared at different window sizes (W), window shifts (ΔW), and search ranges (R = τ₂ − τ₁ + 1). The average and standard deviation (SD) of computational time (in milliseconds) for the motion estimation of each frame was calculated from a total of 323 frames, with 32 beams per frame. The data processing was performed on a PC workstation (Intel Pentium D CPU, 3.4 GHz, 2 GB RAM).

At the high frame rate used, a search range of nine samples in the axial direction (i.e., −4 to 4 samples, or −0.15 to 0.15 mm displacement range) was deemed sufficient to cover the maximum inter-frame displacement experienced by the aorta over a cardiac cycle. The search range was also varied between 5, 17, 33, and 41 samples to investigate its effect on the computational cost. The window size used had 64, 128, or 256 samples, i.e., 2.46, 4.93, or 9.86 mm in length, respectively. The window shift was varied to include 16, 32, 64, or 128 samples.

Using the RF signals of a human left-ventricle in a parasternal long-axis view [39], a preliminary comparison was also performed on the same PC workstation. The RF signals had 2848 samples (or, 11 cm at a sampling frequency of 20 MHz) and there were 64 beams in each frame, acquired at the high frame rate of 481 Hz. At a window size of 128 samples (i.e., 4.9 mm) and window shift of 32 samples (i.e., 75% overlap), an axial search range of ± 4 samples, a lateral search range of ± 1 beam and an interpolation factor of 10 were used.

III. Results

Fig. 2 shows the axial displacement estimation using the sum-table method (W = 128 samples, ΔW = 32 samples, and R = 9 samples). Using these parameters, the computational time was equal to 3.2 ± 0.1 ms/frame, resulting in a frame rate of motion estimation of approximately 310 frames/s. The sequence of displacement images [Figs. 2(a)–(c)] depicts the pulse-wave propagation from the rightmost (proximal) to the leftmost (distal) side of the image [37]–[40]. Although not shown here, it should be noted that the direct and sum-table methods yielded identical estimates, because they were mathematically equivalent, i.e., both computed (1).

Fig. 2.

The estimated pulse wave imaging (PWI) displacements (motions) of a human aortic aorta overlaid onto the B-mode images [37], [38]. (a), (b), and (c) are the image frames at consecutive time points, with 10 ms (i.e., 2 frames) between images [40]. The time lapsing after the ECG R-wave peak is indicated above each image. The color scale of the displacement is [−3 3] samples, equivalent to [−0.12 0.12] mm or velocities range of [−2.2 2.2] cm/s. The positive and negative displacements represent motion toward and away from the phased array on the top of all images, respectively. Only the displacements on the anterior aortic wall and the peri-aortic tissue are shown for better visualization. The thick arrows indicate the propagation of the pulse wavefront. With the parameters used (W = 128 samples, ΔW = 32 samples and R = 9 samples), the frame rate of motion estimation is approximately 310 frames/s.

Fig. 3 shows the computational time of the direct and sum-table methods at different window sizes. As shown, the sum-table method reduces the computational time of the direct method by 53 to 90% depending on the parameters used. The computational time of the direct method linearly increases with the window size when the window shift is fixed. However, the computational time of the sum-table method remains virtually the same at different window sizes for both small and large search ranges (Fig. 3). The computational savings of the sum-table method over the direct method are more pronounced at larger window sizes (i.e., higher window overlaps), in accordance with the computational complexity analysis.

Fig. 3.

The computational time (ms) and percentage of computational time of the sum-table relative to the direct method at different window sizes (W). (a) search range R = 5 samples (b) search range R = 41 samples. (ΔW = 32 samples). Error bars represent 10 × standard deviation (SD). *P < 0.001.

Fig. 4 compares the computational times of the direct and sum-table methods at different search ranges. The sum-table method reduces the computational cost of the direct method by 73 to 81% at the window overlap of 75% and the search range used (5 to 33 samples). The computational time of both methods increases with the search range, in agreement with the computational complexity.

Fig. 4.

The computational time (ms) and percentage of computational time of the sum-table relative to the direct method at different search ranges (R). (W = 128 samples, ΔW = 32 samples). Error bars represent 10 × SD *P < 0.001.

Fig. 5 shows the computational time of both the direct and sum-table methods at different window shifts for a fixed window size. As shown, the computational time of the direct method steadily decreases with the window shift, because the number of reference windows used across the entire signal length decreases (from 152 to 19). However, compared with the direct method, the computational efficiency of the sum-table method was observed in all cases.

Fig. 5.

The computational time (ms) and percentage of computational time of the sum-table relative to the direct method at different window shifts (ΔW) (W = 128 samples, R = 41 samples). Error bars represent 10 × SD. *P < 0.001.

The computational cost gain of the sum-table method decreases from 88% to 47% when the window shift increases from 16 to 128 samples (Fig. 5). In other words, the efficiency of the sum-table method decreases with the window shift. This is mainly because of the reduced window overlap when the window shift increases, and thus is in agreement with the complexity analysis. However, even when there is no overlap (i.e., maximum window shift, W = ΔW = 128 samples), the computational cost of the sum-table method is still substantially lower than that of the direct method (Fig. 5). This is because the comparison window slides sample-by-sample within the search range while still overlapping with successive comparison windows, as shown in Fig. 1(a).

In the motion estimation of the human left ventricle using a 1-D kernel in a 2-D search, the computational time of the direct and sum-table methods was equal to 0.63 ± 0.01 and 0.13 ± 0.01 s/frame, respectively. Therefore, the sum-table method achieved computational savings of 79%, thus ensuring quasi-real-time 2-D estimation at a frame-rate higher than 7 frames/s. The frame rate was further increased to 30 frame/s by using a more powerful PC workstation (Intel Core 2 Quad Q9650 CPU, 3.00 GHz, 8 GB RAM) and parallel computation (4 parallel threads). To the authors’ best knowledge, this is the fastest reported speed for 2-D motion estimation using RF signals with an exhaustive search.

IV. Discussion

Motion estimation with an exhaustive search can be computationally intensive. The proposed method used pre-calculated sum tables to avoid repeating redundant computations in the definition of the NCC given by (1). After construction of the sum tables, the NCC can be efficiently calculated through table lookup. For the parameters typically used (W = 128 samples, ΔW = 32 samples, R = 9 samples), the frame rate of the motion estimation in a human abdominal aorta can be increased from 70 frames/s in the direct method to 310 frames/s in the proposed method. The frame rate of the motion estimation is higher than the frame rate of the data acquisition (i.e., 168 frames/s), thus guaranteeing real-time data processing.

By assuming that NCC needs to be calculated at three search points to obtain the sub-sample precision using interpolation on the NCC function [41], the guided search [25]–[27] could potentially reduce the computational cost of the direct method to a minimum ratio of 3/R. The sum-table method is faster than the method with a guided search, when the search range is small, e.g., R = 5 and R = 9 samples (Fig. 4). Such small search ranges are typically used in motion estimation in cardiovascular applications, where data are acquired at very high frame rates to obtain high-quality estimation, e.g., in the cases of [37]–[40] and the case presented in this paper. The comparison of the sum-table method and the guided search with a larger search range remains to be further investigated.

The reduced computational cost of the sum-table method incurs, however, larger RAM requirements, because the sum tables are pre-calculated and stored in RAM. In the in vivo data verification with a 1-D kernel in a 1-D search, the RAM needed for the sum tables was only 142 (= 2592 × (5 + 2) × 8/1024), 223, 385, and 709 KB at 64-bit (i.e., 8-byte) precision and a search range of 5, 9, 17, and 33 samples, respectively. Using a 1-D kernel in a 2-D search, the RAM required increased linearly with the interpolation factor and search range in the lateral direction, and was significantly higher (e.g., on the order of 1 MB) than that with a 1-D kernel in a 1-D search.

The computational cost of the proposed method was significantly lower than the direct method, without sac-rificing the high-quality motion estimation that can be obtained at high window overlaps (i.e., high spatial resolution), large window sizes (i.e., high signal-to-noise ratio), and high frame rates. In vivo data were used to verify the high efficiency of the proposed method, while maintaining the high precision of motion estimation by using RF signals [21]. The computational savings of this method are more significant at higher window overlaps, thus guaranteeing the high resolution of motion estimation with specific acoustic parameters [22]. In addition, because the computational cost of this method is independent of the window size, a larger window size (within the spatial resolution limit) can be used to reduce the jitter errors of motion estimation [21], [23], [24], without the tradeoff of increased computational cost of conventional methods. It should be noted that a larger window size may include larger intra-window deformation and can thus reduce the spatial resolution of motion estimation [22]. Therefore, the largest possible window size that can be used depends on the tradeoff between accuracy and spatial resolution [42], and will thus have to be adapted to the specific application. If needed, a large window can be used in the proposed method to improve the estimation quality without increasing the computational time.

Dynamic programming has also been previously proposed to reduce the computational cost of NCC in 3-D motion estimation [43]. In that method, intermediate row sums were stored in a circular buffer. The calculation of the CC of sliding windows was obtained by adding in the new row sums and subtracting out the previous ones, similar to the approach of box filtering in the field of template matching [44]–[50]. That method also took advantage of the overlap between successive windows in the axial direction. By using dynamic programming combined with signal downsampling (i.e., reducing the number of samples in the RF signals) and two-path processing (i.e., coarse-to-fine estimation), a reduction in the computational time by a factor of 6 was obtained [43]. However, dynamic programming was implemented in only one direction (i.e., the axial direction), although it was applied to 3-D motion estimation. Extension of the dynamic programming method [43], or the approach of box filtering [44]–[50], in more than one direction has not been reported in the literature and, if possible, may introduce higher programming complexity and overhead time. The sum-table method proposed in this paper is deemed more straightforward and more easily extendable to 2-D and 3-D cases, by taking advantage of high window overlaps in all directions. In Appendix A, an example of 2-D motion estimation has shown a reduction in the computational cost by a factor of about 110 (i.e., computational savings of 99%), demonstrating thus the high efficiency and advantage of the proposed method.

In addition to its expansion to 2-D/3-D, the sum-table method can be extended to other algorithms, such as non-normalized CC, ZNCC, SSD, and SAD. Fig. 6 compares the computational time of the direct and sum-table methods for different algorithms in the 1-D case using RF signals of the human abdominal aorta. As shown, the sum-table method performs faster than the direct method for all algorithms. The computational efficiency of the sum-table methods over the direct methods is more significant in the ZNCC algorithms (i.e., computational savings of 85%), because the signal means, energies, and CC terms are all efficiently calculated from the sum tables.

Fig. 6.

The computational time (ms) at different estimation algorithms and percentage of computational time between the sum-table and direct method. (W = 128 samples, ΔW = 32 samples, R = 41 samples.) Error bars represent 10 × SD. *P < 0.001.

As shown in Fig. 6, the ZNCC is the slowest method because its definition is more complex than the other methods. The CC method is the fastest for both direct and sum-table methods. The SAD was unexpectedly found to be slower than the CC. This could be explained by the fact that the computational cost between addition and multiplication is similar in some new computer architectures (e.g., a pipeline processor) [51], and, compared with CC, the SAD method needs an additional relational test operation and an assignment operation. The efficiency of the methods thus depends on the workstation used. The SAD can be implemented in parallel by using Intel CPU instruction sets, as previously shown [52], [53], but was typically limited to a window size of 4, 8, or 16 samples and 8-bit data precision [54]. In contrast, the sum-table method can still work with an arbitrary window size, data precision, and motion estimator, and hence has a more general impact in its applications. It is also worth mentioning that the computational costs of different motion estimators are on the same order of magnitude with the use of the sum-table method because of its high efficiency (Fig. 6).

V. Conclusion

A time-efficient sum-table method was implemented in the field of ultrasound-based motion estimation to rapidly calculate the normalized cross-correlation (NCC). By taking advantage of the exhaustive search and high overlap between windows required for high-quality imaging, this method can avoid redundant calculation in motion estimation and thereby significantly reduce the computational cost. This method allows for the use of RF signals, together with large window sizes and high window overlaps, for high-quality motion estimation at low computational cost, without the multiple associated tradeoffs in conventional methods between accuracy, spatial resolution, and computational cost. This method was also extended to 2-D motion estimation and motion estimation with methods other than NCC, achieving lower computational time in all cases. The proposed method could thus prove very useful in real-time motion estimation in medical ultrasound as well as other medical imaging modalities or computational fields.

Biographies

Jianwen Luo (S’02–M’06) was born in Fujian Province, China, in 1978. He received the B.S., M.S., and Ph. D. with honors degrees in biomedical engineering from Tsinghua University, Beijing, China, in 2000, 2002, and 2005, respectively.

His research interests include ultrasound imaging and biomedical signal processing. He joined the Ultrasound and Elasticity Imaging Laboratory as a postdoctoral research scientist in 2005, and is currently an associate research scientist in the Department of Biomedical Engineering at Columbia University, New York. He is researching high-resolution cardiovascular imaging, including myocardial elastography, electromechanical wave imaging, and pulse wave imaging. He is a member of IEEE, Sigma Xi, and the American Institute of Ultrasound in Medicine, and is an editorial board member of the Journal of Ultrasound in Medicine.

Elisa E. Konofagou received her B.S. degree in chemical physics from Université de Pierre et Marie Curie, Paris VI in Paris, France, and her M.S. degree in biomedical engineering from Imperial College of Physics, Engineering and Medicine in London, UK, in 1992 and 1993, respectively. In 1999, Dr. Konofagou received her Ph.D. degree in biomedical engineering from the University of Houston, Houston, TX, for her work on multidimensional elastography for breast cancer diagnosis at the University of Texas Medical School, Houston, TX, then pursued her postdoctoral work in elasticity-based monitoring of ultrasound therapy at Brigham and Women’s Hospital, Harvard Medical School, Boston, MA.

Dr. Konofagou is currently an associate professor of biomedical engineering and radiology and Director of the Ultrasound and Elasticity Imaging Laboratory at Columbia University, New York. Her main interests are in the development of novel elasticity imaging techniques and therapeutic ultrasound methods, such as myocardial elastography, breast elastography, ligament elastography, harmonic motion imaging, and ultrasound-induced brain drug delivery, with several clinical collaborations in the Columbia Presbyterian Medicine Center, New York. Dr. Konofagou is a technical committee member of the Acoustical Society of America and a technical standards committee member of the American Institute of Ultrasound in Medicine. She has served on peer-review committees for the National Institutes of Health, the National Aeronautics and Space Administration, and the National Science Foundation. She also serves as an associate editor for the journal Medical Physics and as an editorial board member of the Ultrasound in Medicine and Biology journal, and is recipient of several awards, including from the Acoustical Society of America, the American Heart Association, the American Institute of Ultrasound in Medicine, the National Institutes of Health, the National Science Foundation, the Radiology Society of North America, and the Wallace H. Coulter Foundation. She is also a member of the IEEE Ultrasonics, Ferroelectrics and Frequency Control Society, the International Society of Therapeutic Ultrasound, the Acoustical Society of America, the American Institute of Ultrasound in Medicine, and the American Heart Association.

Appendix A. Fast 2-D NC Calculation Using the Sum-Table Method

The sum tables are defined as follows:

$$\begin{array}{l}{s}_f^2(u,v)=\sum _{m=0}^u\sum _{n=0}^v{f}^2(m,n)\\ s_g^2(u,v)=\sum _{m=0}^u\sum _{n=0}^v{g}^2(m,n)\\ s_{f,g}(u,v,{\tau }_x,{\tau }_y)=\sum _{m=0}^u\sum _{n=0}^{v}f(m,n)·g(m+{\tau }_x,n+{\tau }_y).\end{array}$$ (A1)

The sum tables can be constructed efficiently in two steps. First, the summation is performed in the u direction for each v. Then, the summation was performed in the v direction for each u. An example for constructing $s_f^2(u,v)$ is given as follows:

$$\begin{array}{r}{s}_{f,\text{temp}}^2(u,v)=f^2(u,v)+s_{f,\text{temp}}^2(u-1,v)\\ s_f^2(u,v)=s_{f,\text{temp}}^2(u,v)+s_f^2(u,v-1).\end{array}$$ (A2)

Therefore, the three terms in (8) can be efficiently computed from the sum tables given by (A1) as

$$\begin{array}{l}\sum _{m=u}^{u+W_x-1}\sum _{n=v}^{v+W_y-1}{f}^2(m,n)\\ =s_f^2(u+W_x-1,v+W_y-1)-s_f^2(u-1,v+W_y-1)\\ -s_f^2(u+W_x-1,v-1)+s_f^2(u_x-1,v-1),\end{array}$$ (A3)

$$\begin{array}{l}\sum _{m=u}^{u+W_x-1}\sum _{n=v}^{v+W_y-1}{g}^2(m+{\tau }_x,n+{\tau }_y)\\ =s_g^2(u+W_x-1+{\tau }_x,v+W_y-1+{\tau }_y)\\ -s_g^2(u-1+{\tau }_x,v+W_y-1+{\tau }_y)\\ -s_g^2(u+W_x-1+{\tau }_x,v-1+{\tau }_y)\\ +s_g^2(u_x-1+{\tau }_x,v-1+{\tau }_y),\end{array}$$ (A4)

$$\begin{array}{l}\sum _{m=u}^{u+W_x-1}\sum _{n=v}^{v+W_y-1}f(m,n)g(m+{\tau }_x,n+{\tau }_y)\\ =s_{f,g}(u+W_x-1,v+W_y-1,{\tau }_x,{\tau }_y)\\ -s_{f,g}(u-1,v+W_y-1,{\tau }_x,{\tau }_y)\\ -s_{f,g}(u+W_x-1,v-1,{\tau }_x,{\tau }_y)\\ +s_{f,g}(u-1,v-1,{\tau }_x,{\tau }_y).\end{array}$$ (A5)

A preliminary test was also performed on the same PC workstation (Intel Pentium D CPU, 3.4 GHz, 2 GB RAM) using the high-resolution (30 MHz), high framerate (1000 Hz) B-mode data of a mouse left-ventricle in a short-axis view acquired from a Vevo 770 system (Visual-Sonics Inc., Toronto, ON, Canada). Each B-mode image consisted of 432 × 192 pixels (axially × laterally) in a region of interest of 12 × 12 mm². A 2-D kernel of 64 samples (i.e., 1.78 mm) in the axial direction and 32 samples (i.e., 2 mm) in the lateral direction and a search range of ± 2 pixels in the axial direction (i.e., ± 0.056 mm) and ± 1 pixel in the lateral direction (i.e., ± 0.063 mm) were used. Fig. 7 shows the estimated 2-D displacements in the systolic phase. All the anterior, posterior, septal, and lateral walls of the left ventricle move toward the center of the blood cavity. The computational time of the sum-table method was 42 ± 1 ms for each frame, approximately 110 times lower than that of the direct method (4.68 ± 0.01 s). Therefore, a frame rate of 24 Hz was achieved for the 2-D motion estimation, with significant computational savings of 99% because of the high overlaps required in both axial the lateral directions. Compared with the 1-D case, the 2-D/3-D extension of the sum-table method takes advantage of the window overlap in more than one direction. Therefore, the computational savings of the 2-D case of the sum-table method are much more significant than those of the 1-D case.

Fig. 7.

The estimated (a) axial and (b) lateral displacements of a mouse left ventricle overlaid onto the B-mode images. The images were taken from the systolic phase (25 ms after R-wave of the ECG). The positive (red) displacements represent upward or rightward motion, while negative (blue) displacements represent downward or left motion. Only the displacements of the left ventricle are shown. (ANT: anterior wall; LAT: lateral wall; LV: left ventricle; POS: posterior wall; RV: right ventricle; SEP: interventricular septum)