2-tier in-plane motion correction and out-of-plane motion filtering for contrast-enhanced ultrasound

Abstract

Objectives

Contrast-enhanced ultrasound (CEUS) cines of focal liver lesions (FLL) can be quantitatively analyzed to measure tumor perfusion on a pixel-by-pixel basis for diagnostic indication. However, CEUS cines acquired freehand and during free breathing cause non-uniform in-plane and out-of-plane motion from frame to frame. These motions create fluctuations in the time-intensity curves (TIC), reducing accuracy of quantitative measurements. Out-of-plane motion cannot be corrected by image registration in 2D CEUS and degrades the quality of in-plane motion correction (IPMC). A 2-tier IPMC strategy and adaptive out-of-plane motion filter (OPMF) are proposed to provide a stable correction of non-uniform motion to reduce the impact of motion on quantitative analyses.

Materials and Methods

22 cines of FLLs were imaged with dual B-mode and contrast specific imaging to acquire a 3-minute TIC. B-mode images were analyzed for motion, and the motion correction was applied to both B-mode and contrast images. For IPMC, the main reference frame was automatically selected for each cine, and subreference frames were selected in each respiratory cycle and sequentially registered toward the main reference frame. All other frames were sequentially registered toward the local subreference frame. Four OPMFs were developed and tested: subsample Normalized Correlation (NC), subsample Sum of Absolute Differences (SAD), mean frame NC, and histogram. The frames that were most dissimilar to the OPMF reference frame using one of the four above criteria in each respiratory cycle were adaptively removed by thresholding against the low-pass filter of the similarity curve. OPMF was quantitatively evaluated by an out-of-plane motion metric (OPMM) that measured normalized variance in the high-pass filtered time-intensity curve within the tumor region-of-interest with low OPMM being the goal. IPMC and OPMF results were qualitatively evaluated by two blinded observers who ranked the motion in the cines before and after various combinations of motion correction steps.

Results

Quantitative measurements showed that 2-tier IPMC and OPMF improved imaging stability. With IPMC, the NC B-mode metric increased from 0.504 ± 0.149 to 0.585 ± 0.145 over all cines (p < 0.001). 2-tier IPMC also produced better fits on the contrast-specific TIC than industry standard IPMC techniques: (p < 0.02). IPMC and OPMF were shown to improve goodness-of-fit for pixel-by-pixel analysis (p < 0.001). OPMF reduced variance in the contrast-specific signal as shown by a median decrease in the OPMM of 49.8%. 2-tier IPMC and OMPF were also shown to qualitatively reduce motion. Observers consistently ranked cines with IPMC higher than the same cine before IPMC (p < 0.001) as well as ranked cines with OPMF higher than when they were uncorrected.

Conclusion

The 2-tier sequential IPMC and adaptive OPMF significantly reduced motion in 3-minute CEUS cines of FLLs thereby overcoming the challenges of drift and irregular breathing motion in long cines. The 2-tier IPMC strategy provided stable motion correction tolerant of out-of-plane motion throughout the cine by sequentially registering subreference frames that bypassed the motion cycles, thereby overcoming the lack of a nearly stationary reference point in long cines. OPMF reduced apparent motion by adaptively removing frames imaged off-plane from the automatically selected OPMF reference frame, thereby tolerating non-uniform breathing motion. Selection of the best OMPF by minimizing OPMM effectively reduced motion under a wide variety of motion patterns applicable to clinical CEUS. These semi-automated processes only required user input for region-of-interest selection and can improve the accuracy of quantitative perfusion measurements.

I. INTRODUCTION

Contrast-enhanced ultrasound (CEUS) is a real time imaging modality that allows functional imaging of tissue perfusion. Intravenously injected microbubbles enhance all functional vasculature, including capillaries, causing enhancement of perfused tissues.^1-3 Highly specific time-resolved imaging of tumor vasculature and perfusion has been widely used to detect and characterize tumors noninvasively.^4-6 Tumors possess characteristic perfusion patterns that can be quantitatively characterized with CEUS.^7-10 However, analysis of CEUS cines requires highly experienced radiologists to accurately and reliably diagnose tumors.^{11, 12} Quantitative analysis of time-intensity curves (TIC) provides hemodynamic data that can be used to aid characterization or quantify tumor response to therapy.^13-19 Traditionally, TICs are acquired as the change in mean intensity over time within an ROI positioned over a tumor.^{12, 20} More recently, TICs are being analyzed on a pixel-by-pixel (P×P) basis to provide localized measures of perfusion and heterogeneity within the tumor.^{16, 18, 21-24} It was shown that P×P TIC analysis was superior at discriminating benign from malignant breast tumors in a rat model which incorporated the variance among pixels as an independent measure.²⁵

One of the key challenges of performing P×P analysis of clinically acquired CEUS is motion.^{12, 20} Its elimination is required for accurate quantitative analysis, which is difficult to achieve when 3-minute long observations are necessary to capture the wash-in and wash-out of contrast media. Strategies for in-plane motion correction (IPMC) and out-of-plane motion filtering (OPMF) are therefore required.

Liver lesion characterization is the most commonly used indication for CEUS, however, the liver offers unique challenges. It abuts the diaphragm creating respiratory motion. Furthermore, since the liver is shielded by the ribcage, imaging the liver through the intercostal space forces the plane of section to be oblique to respiratory excursion creating out of plane motion. Since CEUS is a 2D imaging modality, software motion correction can only correct motion that occurs within the imaging plane. In addition to the enhancements of in-plane motion correction algorithms, novel systems of filtering motion that occurs across the imaging plane were developed and evaluated.

For IPMC, image registration is classified into either non-rigid or rigid registration, distinguished by the types of transformations allowed. In non-rigid image registration, different regions within an image are allowed to be warped independently of one another under certain continuity constraints. Woo et al. developed a non-rigid ultrasound registration method using both image intensity and local phase information.²⁶ Lu et al. developed a non-rigid image registration that optimized a combination of local and global landmarks.²⁷ While non-rigid image registration techniques benefit tasks such as multimodal image registration or construction of an anatomical atlas, it may result in biased estimates of motion in dynamic 2D CEUS due to changes in microbubble uptake and out-of-plane motion.²¹

Rigid image registration techniques are restricted to affine transformations, e.g., translation and rotation. Renault et al. tested six conventional rigid registration techniques on contrast uptake images.²⁸ Motion correction for all frames was performed using a single reference frame selected within an imaging plane at the end of inspiration. However, due to changes in gray-level intensity with microbubble concentration and out-of-plane motion, motion correction failed to produce accurate results. Rognin et al. also implemented rigid image-registration by maximizing the normalized mutual information with a reference frame.²¹ They produced satisfactory results after manually editing the cines to remove frames imaged out-of-plane relative to the reference frame.

Out-of-plane motion is a major hurdle for IPMC and quantitative analysis of 2D ultrasound. In the presence of out-of-plane motion, it is often ineffective to use a single reference frame to register an entire 2D ultrasound cine. Frames imaged at distances near or greater than the ultrasound beam width away from the reference frame may contain tissue that appears completely different from the reference frame. Without similar features in each frame to align, image registration fails. Alternatively, it is possible to co-register each sequential pair of frames as suggested by Woo et al.²⁶ If out-of-plane motion between consecutive frames is small, both frames will image the majority of the same tissue scatterers, resulting in similar images. This improves the probability that the image registration technique can find corresponding features between each pair of frames. However, this strategy is difficult to implement accurately for a 3 minute cine with over 1500 frames. Small errors in each iteration propagate to all following iterations and accumulate towards the ends of the cine. Bouhlel et al. developed a dual-mode registration technique where a cost function combined tissue images registered to a single reference frame and contrast images registered sequentially.²⁹ This dual-mode registration improved registration accuracy over tissue-only registration but required prior removal of frames off-plane from the selected reference frame.

Since out-of-plane motion in 2D ultrasound results in the imaging of different slices of the anatomy with different vascularity,^{18, 30} image post-processing cannot correct out-of-plane motion. Frames out-of-plane relative to the reference frame must be removed prior to quantitative analysis to ensure the data are extracted from the same tissue slice. Renault et al. applied independent component analysis to contrast uptake images to estimate the principal components of respiratory motion and extract frames corresponding to the end-of-inspiration plane and end-of-expiration plane.²⁸ They relied on the assumption that the target was in the same locations in each respiratory cycle. However, this assumption is unlikely to hold true in long imaging sequences due to changes in breathing depth or unintentional motion of the transducer or patient. Averkiou et al. implemented respiratory gating by manually rejecting all frames where the position of the diaphragm deviated from its position in the reference frame.³¹ Unfortunately, manually filtering frames is extremely time consuming and relies on the presence of a reliable landmark within the field of view. Zhang et al. semi-automatically filtered out off-plane frames by running a double-selection method against a reference frame.³² Only those frames passing under a global sum of absolute differences (SAD) threshold and were the local minima within specified time intervals were retained for analysis. However, this technique required manual selection of a reference frame, omitted major portions of the cine from analysis, and did not perform well with irregular breathing motion, which is common in long ultrasound acquisitions.

To overcome the issues discussed above, a 2-tier sequential image registration strategy and adaptive out-of-plane motion filter were developed. The 2-tier sequential image registration strategy provided the same advantage as sequential image registration in handling out-of-plane motion while minimizing the risk of accumulating errors by sequentially registering subreference frames that bypassed the respiratory motion cycle. The adaptive OPMF automatically selected a reference frame and detected the best matching frames in each respiratory cycle, thereby tolerating irregular breathing motion and other nonuniform motion from the patient or transducer.

II. MATERIALS AND METHODS

II.A. Imaging

For validation, a retrospectively collected dataset of 22 CEUS cines of 21 focal liver lesions (FLL) acquired following the administration of SonoVue (Bracco Imaging, Milan, Italy) as part of a clinical trial sponsored by Bracco Diagnostics, Inc. with approval by the Institutional Review Board. Sonovue was administered intravenously as a 2.4 mL bolus injection with a 20-gauge catheter followed immediately by a 5 mL saline flush. B-mode and contrast specific imaging were displayed side by side on the screen and were recorded for 3 minutes with a Philips iU22 Ultrasound System (Philips Healthcare, Andover, MA) and C5-1 transducer between 7-11 frames/sec. Patients were imaged during quiet breathing to reduce respiratory motion. Whenever possible, the imaging window and transducer orientation were optimized to minimize out-of-plane motion.

II.B Motion correction: overview

The following motion correction strategy was designed to eliminate both in-plane and out-of-plane motion while minimizing user input. A flow chart of the overall process is shown in Figure 1 and is briefly described below. In addition, each step is described in detail in sequential sections of the manuscript. First, a fast in-plane motion correction step was applied to estimate translational motion. Since different regions of the image frame presented differing motion characteristics, the user selected an ROI for IPMC analysis encompassing the tumor and any surrounding tissue with the same motion profile. IPMC analysis was performed within this user selected region, but the motion correction was applied to the entire frame. Following IPMC, the user defined a second ROI that tightly encompassed the tumor. OPMF was performed on the second ROI to track the motion of the tumor and provide the most consistent imaging of the tumor. After OPMF, the cines could be used for quantitative analysis.

Figure 1.

Flow chart showing the overall motion correction process. Motion analysis was performed on the B-mode imaging and subsequently applied to the contrast imaging.

IPMC and OPMF were performed on the B-mode images to more accurately correct both translational and rotational motion since B-mode images retain anatomical and structural details that are subtracted out of the contrast image. Further, tissue signal on B-mode is not strongly affected by microbubbles, providing stable signal for more accurate motion detection. Once the correction parameters were defined on the B-mode images, they were applied to the contrast images. All processes were implemented in MATLAB R2011a (The MathWorks Inc., Natick, MA) on a laptop with an Intel Core i7-2720QM CPU and 6 GB of memory.

II.C In-plane motion correction

To overcome the problem of out-of-plane motion and drift for IPMC, a 2-tier image registration process was developed (Fig. 2). In the first tier, subreference frames selected from each motion cycle were sequentially registered toward the main reference frame. On the second tier, all other frames were sequentially registered to the nearest subreference frame. This motion correction strategy allowed IPMC even in the presence of out-of-plane motion while minimizing the risk of propagating errors. 2-tier IPMC consisted of three main steps: 1) selection of a main reference frame; 2) selection of subreference frames in each motion cycle; and 3) image registration for motion correction.

Figure 2.

Schematic of three in-plane motion correction strategies. a) Single reference frame. All frames in a cine are registered directly to a single reference frame. b) Sequential image registration. Each pair of consecutive frames in a cine are registered sequentially toward the main reference frame. c) 2-tier sequential image registration. Subreference frames from each motion cycle are sequentially registered together toward the main reference frame. Afterwards, all other frames are sequentially registered toward the local subreference frame.

II.C.1 Selection of reference frame

For IPMC, the main reference frame was the single frame to which all other frames were aligned; it established an anchor point for the anatomy within the image coordinate system. The goal of automatically selecting the main reference frame was to choose a frame that had similar features to the majority of the cine to increase the likelihood of finding a similar subreference frame in each respiratory cycle for optimal alignment. To find the frame that best matched the entire cine, the normalized correlation (NC) was calculated between the temporal mean frame and all other frames (Fig. 3). The frame that produced the greatest NC with the mean frame was selected as the main reference frame.

Figure 3.

Reference frame selection using the mean frame method for cine #4. a) The normalized correlation was calculated between every frame and the mean frame in B-mode. This correlation curve has two notable features. 1) Due to respiratory motion, the correlation curve has a high frequency cyclical fluctuation. 2) The correlation curve exhibits slow shifts over time due to a motion of the handheld transducer. The frame with the greatest correlation is marked with a square. b) The mean frame was a temporal mean of the image intensities of each pixel throughout the cine. c) The frame with the strongest correlation to the mean frame was selected as the reference frame.

II.C.2 2-tier sequential image registration

In order to produce reliable registration between respiratory cycles, subreference frames near the same imaging plane were selected from each respiratory cycle. The algorithm for selecting subreference frames was a repetitive search for additional subreference frames starting from the main reference frame and expanding towards the beginning and the end of the cine (Fig. 4). Figure 4 shows four iterations of the subreference frame (SRF) selection algorithm: iteration #1 (Fig. 4a-b), iteration #2 (Fig. 4c-d), iteration #6 (Fig. 4e-f), and iteration #7 (Fig. 4g-h). Iterations #1-2 are shown to demonstrate the initiation of the algorithm and a typical iteration. Iterations #6-7 are shown to demonstrate an iteration where multiple motion cycles were processed together to handle large, temporary excursions from the reference imaging plane.

Figure 4.

Four iterations of the subreference frame (SRF) selection algorithm from cine #4: iteration #1 (a and b), iteration #2 (c and d), iteration #6 (e and f), and iteration #7 (g and h). Iterations #1-2 are shown to demonstrate the initiation of the algorithm and a typical iteration. Iterations #6-7 are shown to demonstrate an iteration where multiple motion cycles were processed together to handle large excursions from the reference imaging plane. a,c,e,g) The left side shows normalized correlation between the mean frame and each frame throughout the B-mode cine. The right side shows the normalized correlation within the small subregion of the box on the left side. Circles indicate all SRFs found prior to the current iteration. In the first iteration (a), the search starts from the main reference frame. The box indicates the 30 second search window for additional SRFs. b,d,f,h) The correlation between the “current” SRF (square) and all other frames within the current search window. The low-pass filter of the correlation curve (red line) allows identification of the individual motion cycles. The frame with the greatest correlation to the “current” SRF was identified (“peak” SRF, diamond). The frames with the greatest correlation in each motion cycle up to and including the “peak” SRF were selected as SRFs (circles) and registered directly to the “current” SRF. The “peak” SRF was set as the “current” SRF for the following iteration (arrows from b to d and from f to h). i) All SRFs selected throughout the cine are marked by circles on the correlation curve.

The algorithm was initiated by setting the “main” reference frame as the “current” subreference frame (Fig. 4a). In each iteration of the algorithm, the following steps were performed. The NC was calculated between the “current” SRF and all other frames within a 30 second search window (Fig. 4b,d,f,h). A 0.1 Hz 8-order Butterworth low-pass filter (LPF) was applied to the correlation curve (red line in Fig. 4b, d, f, h). The frames with NC above the LPF were grouped together as individual motion cycles. The single frame within the search window with the greatest NC to the “current” SRF was identified as the “peak” SRF (Fig. 4b,d,f,h - diamond) of the current iteration and registered directly to the “current” SRF. If there were any motion cycles between the “current” SRF and the “peak” SRF, the frame with the greatest NC to the “current” SRF in each motion cycle was also selected as a SRF and registered directly to the “current” SRF (Fig. 4d,f - circles). For the next iteration of the algorithm, the “peak” SRF of the previous iteration was assigned as the new “current” SRF, and the cycle was repeated with a new 30 second search window (arrows from Fig. 4b to 4d and from 4f to 4h). These steps were applied starting from the main reference frame and moving towards both the beginning and the end of the cine. Once SRFs were selected and registered throughout the cine, all of the remaining frames in the cine, i.e., those not selected as SRFs, were sequentially registered towards the nearest SRF as depicted in Figure 2c.

II.C.3 Image registration

As shown in the process flow chart (Fig. 1), IPMC was performed twice, and the approaches described above for the selection of a reference frame and the 2-tier image registration strategy were applied in both steps. In the first IPMC process, single-step discrete Fourier transform image registration quickly estimated translational registration parameters.³³ This provided a more stable cine to allow the user to easily select a region around the tumor for the second and more accurate IPMC step. In the second IPMC step, translational and rotational transformation parameters were estimated by minimization of the SAD using Powell’s method for minimization within the user selected region for motion correction to ensure that only motion of the target lesion was detected. This second IPMC step was slower but provided more accurate motion correction. All steps were performed on B-mode (tissue) images.

II.D Out-of-plane motion filtering

The goal of OPMF was to focus TIC analysis on the same tissue slice throughout the cine. With respiratory motion, the tumor may move in and out of the imaging plane. Combined with motion of the ultrasound transducer, the tumor may appear at different phases of the respiratory cycle (Fig. 5). OPMF will find the frames in each motion cycle where the imaged anatomy is the same and filter out the frames that imaged different tissue slices. The removal of portions of the CEUS cine from analysis is acceptable with contrast bolus imaging since the timescale of the characteristic enhancement kinetics is much larger than that of OPMF. OPMF consists primarily of two steps: 1) select an optimal OPMF reference frame; and 2) filter out the frames in each motion cycle that are the most dissimilar to the OPMF reference frame. Out-of-plane motion analysis was performed on B-mode images within the user selected ROI and, subsequently, applied to the contrast specific images on the cine.

Figure 5.

Out-of-plane motion filtering (OPMF). a) Motion of the transducer or the subject perpendicular to the ultrasound imaging plane is out-of-plane motion. b) Schematic plot of OPMF. Motion of the transducer (slow) and patient breathing (fast) cause the tumor to move in and out of the imaging plane. OPMF selects certain frames for analysis where the tumor is in the most similar positions within each motion cycle (red). Due to the slow motion of the transducer, the frames filtered may occur at different phases of the respiratory cycle (black). c) The correlation curve between the reference frame and all other B-mode frames for cine #4. The low-pass filter of the correlation curve established an adaptive threshold. Frames with correlation above this adaptive threshold were the frames in each motion cycle that were the most similar to the reference frame.

II.D.1 Selection of the reference frame

The goal of selecting an OPMF reference frame was to find the frame with the most commonly imaged tissue slice within the cine to allow the OPMF to find the maximum number of matching frames throughout the cine. Note the OPMF reference frame was distinct from the IPMC “main reference frame” and was chosen by a different algorithm. To efficiently find an OPMF reference frame that best matched the maximum number of frames throughout the cine, multiple approaches were developed and tested: 1) subsampled image comparison; 2) mean frame image comparison; and 3) image intensity histogram comparison.

II.D.1.a Subsampled image comparison

Frames were subsampled at 1 Hz throughout the cine, and subsampled frames were all compared to one another with an image similarity metric. The frame that optimized the total image comparison metric with all subsampled frames was selected as the OPMF reference frame. Two image comparison metrics were tested with this approach: NC and SAD; these techniques will be referred to as “Subsampled (NC)” and “Subsample (SAD)”. These techniques were more accurate than the reduced dimensionality techniques (described below), but were relatively computationally expensive, hence requiring subsampling in order to maintain an acceptable computational time.

II.D.1.b Mean frame image comparison

Similar to the technique described above for selecting an IPMC main reference frame, the frame that optimized the NC with the mean frame was selected as the OPMF reference frame. This method will be referred to as “Mean Frame (NC)”.

II.D.1.c Image intensity histogram comparison

To allow efficient comparison of all pairs of frames within the cine, histograms of the image intensities were used. Histograms are simple and effective techniques of compressing large datasets into smaller and more tractable datasets that reduce computational burden. A histogram of the image intensity within the ROI was calculated for each frame, i.e. for each timepoint. Histograms were compared between all frames by the L₂-norm metric, and the frame with the minimum total L₂-norm was selected as the OPMF reference frame. This allowed the comparison of all frames within a 3 minute cine (~9 fps, ~1600 frames total) to be performed within ~10 seconds.

II.D.2 Frame filtering to remove motion

After the OPMF reference frame was selected, the OPMF reference frame was compared with all B-mode frames in the cine using either NC or SAD to find the frames within each motion cycle that look the most similar to the OPMF reference frame. The NC between the OPMF reference frame and all other frames within the cine is shown in Figure 5c. The high frequency fluctuations in the correlation curve were caused by respiratory motion. The correlation curve also exhibited slow shifts in the NC over time. These low frequency changes were caused by either motion of the handheld transducer or non-respiratory motion from the patient. Due to these shifts, a single threshold could not be applied to filter out the worst matched frames in each motion cycle. Instead, a low-pass filter (0.1 Hz 8-order Butterworth filter) was applied to the similarity curve to establish an adaptive threshold (Fig. 5c). The OPMF removed frames that had a similarity score with the OPMF reference frame below the adaptive threshold. This effectively chose the frames within each motion cycle with the greatest similarity to the OPMF reference frame.

II.D.3 Out-of-plane motion metric

To quantitatively analyze the performance of each configuration of the OPMF, an out-of-plane motion metric (OPMM) was developed by measuring the influence of motion on the mean CEUS TIC within the tumor ROI. In the unfiltered cine, respiratory motion caused the body to move underneath the imaging probe, which in turn created fluctuations in the CEUS TIC (Fig. 6). When OPMF is successful, these fluctuations are reduced. However, the strength of these fluctuations was proportional to the microbubble concentration, which changed over time with the bolus injection. To measure the motion-induced variability within the TIC with minimum bias from microbubble concentration, the high-pass filtered TIC (the fluctuations in the TIC) (Fig. 6b) was normalized by the low-pass filtered TIC (the strength of the contrast signal) (Fig. 6c). The normalized high-pass filtered TIC signal was consistent over the enhanced portion of the cine (Fig. 6d). The percent change of variance in this normalized signal pre- and post-OPMF quantified the performance of the various OPMF methods.

Figure 6.

The motion-induced variance in the contrast-enhanced ultrasound (CEUS) image. a) Time-intensity curve (TIC) from the mean CEUS intensity over the tumor in each frame. Respiratory motion caused the body to move back and forth underneath the imaging probe, which in turn created fluctuations in the TIC signal at the same rate as the breathing. b) A high-pass filtered TIC. The magnitude of the fluctuations changed over time due to changes in the microbubble concentration from the bolus injection. c) The low-pass filtered TIC. The low-pass filtered TIC was proportional to microbubble concentration. d) The high-pass filtered TIC normalized by the low-pass filtered TIC. This signal carried the motion-induced variance of the CEUS TIC independent of microbubble concentration.

II.E Observer validation of motion correction and filtering

To determine if the IPMC was effective, two radiologists (ME and RFM) independently graded the 22 focal liver lesion cines with and without IPMC. For each of the cines, the observers were shown a 2×2 composite video. The original and the in-plane motion corrected B-mode cines were displayed side-by-side on the top row and the contrast mode cines on the bottom row. To blind the observers to the processing, the positions of the original and IPMC cines were randomized independently for the B-mode and contrast cines and for each FLL cine. The observers compared the uncorrected and IPMC cines and ranked them according to the amount of motion in the cine. The B-mode and contrast cines received separate rankings; with 22 FLL cines and two observers, 88 rankings were collected. The Kruskal-Wallis test was performed to determine statistical significance of the ranking results.

To determine if the OPMF was effective, the two observers were also shown videos with different combinations of IPMC and OPMF: 1) No IPMC and Standard OPMF; 2) IPMC and No OPMF; 3) IPMC and Standard OPMF; and 4) IPMC and Best OPMF. Since OPMF cines contain noticeably fewer frames than non-filtered cines, a control without OPMF was simulated by randomly removing frames evenly throughout the cine. Based on the results shown below in section III.B, the Subsample (SAD) OPMF technique was chosen as the Standard OPMF technique since this technique had the lowest median OPMM over all cines. The Best OMPF processing was a combination of four OPMF techniques: 1) Subsample (NC); 2) Subsample (SAD); 3) Mean Frame (NC); and 4) Histogram. The technique that yielded the lowest OPMM for each cine was selected as the best OPMF technique for that cine. When the Standard OPMF method was selected for Best OPMF, the Standard OPMF cine was shown twice, and the motion grades for the Standard OPMF and Best OPMF were assigned the average of the two. The Kruskal-Wallis test was performed and Tukey’s least significant difference method was performed for multiple comparisons.

II.F Statistical Analysis

T-tests and Kruskal-Wallis tests were performed using MATLAB R2011a. Power analyses were performed using R (The R Foundation for Statistical Computing, Vienna, Austria).

III. RESULTS

Figure 7 shows the correlation curve against the mean frame in the original B-mode image for all 22 cines. There was a wide variety of motion profiles across all of the cines due to non-uniform respiratory motion, transducer orientation, and transducer motion. In some cases, there was a large range of respiratory motion, e.g., cine #2. In other cases, there may have been a combination of motion from the patient and the imaging probe throughout the cine, e.g., cines #16 and #22.

Figure 7.

Correlation curves with the mean frame for all raw B-mode cines. All cines show the characteristic high frequency fluctuation of correlation due to breathing motion. A few cines have relatively little shifting in the correlation curve (e.g., cines #6 and #15), indicating little to no motion from the transducer. However, the majority of cines have some major shifts over the course of the 3 minute acquisition, indicating movement from the transducer or other non-respiratory movement from the patient.

III.A In-plane motion correction

To quantitatively evaluate IPMC on the B-mode images, the mean of the NC between all frames and the mean frame was calculated for each of the 22 FLL cines before and after IPMC. Prior to motion correction, the mean and standard deviation over all cines was 0.504 ± 0.149. After IPMC, the mean NC increased to 0.585 ± 0.145. These results were statistically significant (p < 0.001) with more than 99% power (α = 0.05) according to the two-tailed paired t-test, indicating that IPMC was effective at producing more stable cines. Although a statistically significant improvement was shown, this improvement of 16.1% undervalued the impact of IPMC since out-of-plane motion still caused different tissue sections to come into frame, substantially lowering the NC from the ideal value of 1.0.

The impact of IPMC can be more readily appreciated by viewing the temporal mean image. Figure 8 displays the temporal mean image from cine #4 before and after IPMC; note this was a cine that had much larger than average in-plane motion and relatively low out-of-plane motion, so the effect of the IPMC was especially large. If the tissue was perfectly stationary within the imaging plane, the mean image would appear sharp since each part of the anatomy would appear at the same pixel location over time. However, motion of the tissue forced the anatomy to move to different pixel locations over time, causing the mean images to appear blurry. Without motion correction, very few features can be identified. IPMC eliminated the majority of the motion and produced sharper mean images in both the B-mode and contrast specific images. For this cine, the mean NC before and after IPMC was 0.52 and 0.76, respectively. Although the temporal mean images were drastically sharper after IPMC, the mean NC was still degraded by out-of-plane motion. In other cines where the out-of-plane motion was more severe, the mean NC was negatively impacted even further.

Figure 8.

Temporal mean from cine #4 before and after in-plane motion correction for B-mode and contrast mode. In each image, the mean of the pixel intensity over time is shown. If the tissue was perfectly stationary within the imaging plane, the mean image would appear sharp; however, motion of the tissue caused the images to appear blurry. Without motion correction, very few features can be identified. In-plane motion correction eliminated the majority of the motion and produced sharper mean images.

To compare the 2-tier sequential IPMC against standard IMPC techniques, all cines were motion corrected with direct IPMC, sequential IPMC, and 2-tier sequential IPMC (Fig. 2). For each of the IPMC corrected cines, the TIC of the mean contrast-specific signal within the ROI was fitted to equation 1.³⁴

$$I(t)=a_0+a_1\frac{e^{-a_{2}t}}{1+e^{-a_3(t-t_0)}}$$ (1)

The coefficient of determination (R²) was selected to represent the goodness-of-fit (GoF) to quantitatively compare the three IPMC techniques (see Figure, Supplemental Digital Content 1, which shows the GoF results). The mean ± standard deviation of R² for direct IPMC, sequential IPMC, and 2-tier IPMC were 0.84 ± 0.16, 0.78 ± 0.20, and 0.89 ± 0.08, respectively. While the direct IPMC and sequential IPMC methods performed well across some cines, the 2-tier sequential IPMC performed well across all 22 cines (R² > 0.7). A statistically significant difference was detected between direct IPMC and 2-tier IPMC (p < 0.02) with 76% power (α = 0.05 using two-tailed paired t-test) and between sequential IPMC and 2-tier IPMC (p < 0.02) with 78% power. Despite the more complicated registration scheme, 2-tier sequential IPMC required similar processing times to the standard techniques. Direct IPMC, sequential IPMC, and 2-tier IPMC required an average of 2.10 sec/frame, 1.91 sec/frame, and 1.93 sec/frame, respectively.

III.B Out-of-plane motion filtering

All four approaches to the OPMF (Subsampling, Mean Frame, Histograms, and Best) were quantitatively evaluated using the OPMM on the contrast-specific images. The percent change of OPMM between pre- and post-OPMF were calculated for all 22 FLL cines and summarized in Figure 9 and Table 1. Overall, all techniques performed similarly and had significant percent changes of OPMM according to two-tailed t-test results (p < 0.001) and power > 97% (α = 0.05), however, no individual technique was superior to the others in a majority of the cines. Subsampling (SAD) produced the greatest median percent decrease OPMM (43.4%) over all cines. Mean Frame (NC) and Subsampling (NC) produced the next best average percent decrease OPMM of 36.8% and 31.5%, respectively. The Mean Frame approach was approximately 5X faster than the corresponding Subsampling approach. Since each of the four individual methods performed better in different cases, by trying all methods and choosing the best performing filter, the OPMM was substantially reduced in almost all cines.

Figure 9.

Percent change of the out-of-plane motion metric (OPMM) pre- and post- out-of-plane motion filtering on all 22 FLL contrast mode cines (more negative is better). By trying all methods and choosing the best performing filter, the OPMM was substantially reduced in almost all cines. In this figure, the middle of the box is the median, the top/bottom of the box are the 1^st and 3^rd quartiles, the whiskers show the range excluding outliers, and the markers indicate possible outliers. Note that while the Histogram method had the worst median performance, it had the best performance in a few cases due to the large standard deviation.

Table 1.

Results of out-of-plane motion filtering. No single method consistently performed better than all of the other methods according to the out-of-plane motion metric. The mean frame technique was the fastest due to having the fewest image comparisons to perform.

Out-of-plane motion filtering method	Median Change (%)	25% – 75%	# Best Cines	Average time (sec)
Subsampling (NC)	-31.5	-4.1 – -51.0	5	7.63
Subsampling (SAD)	-43.4	-29.3 – -51.7	7	4.32
Mean Frame (NC)	-36.8	-10.4 – -46.7	6	1.51
Histograms	-19.6	-6.1 – -46.1	4	8.90
Best	-49.8	-39.8 – -60.7	22	22.4

Figure 10 shows the effect of IPMC and OPMF on the TIC for cine #14 in contrast mode. The means of the signal intensity within the tumor ROI with or without motion correction were calculated for all frames and shown as circles. The TIC without motion correction, with IPMC only, and with IPMC and OPMF are shown in red, green, and black, respectively. For OPMF, the Subsampling (SAD) OMPF method was performed. After IPMC only, the fluctuations in the TIC decreased but were still present due to out-of-plane motion. The OPMF consistently selected frames within a certain spatial window throughout the entire cine. Although OPMF was performed by analyzing the B-mode images only, the deviant frames in the corresponding contrast images were removed. OPMF resulted in a TIC with noticeably less variance than the original.

Figure 10.

Effect of in-plane motion correction (IPMC) and out-of-plane motion filtering (OPMF) on the contrast-specific time-intensity curve (TIC) from cine #14. The means of the signal intensity within the tumor ROI for all frames without motion correction (red), with IPMC only (green), and with IPMC and OMPF (black) are shown. The TIC fluctuated drastically with each motion cycle even after IPMC. The TIC was noticeably tightened by OPMF. Out-of-plane motion analysis was performed on the B-mode images but was effective at removing deviant frames from the contrast signal.

The visual results of OPMF for B-mode and contrast mode are demonstrated in Figure 11, where the temporal mean images are shown for cine #16 with 1) no motion correction, 2) IPMC only, and 3) IPMC and OPMF. IPMC eliminated the majority of the motion and produced sharper mean images. The outline of the tumor can be seen in the B-mode image, and blood vessels can be identified in the contrast specific image. OPMF further reduced motion within the cines and increased the sharpness of the mean images. Compared to IPMC alone, the details in both the B-mode and the contrast specific images became more prominent after OPMF. The blood vessels appeared sharper in the contrast image, and even the speckle in the B-mode image became more defined. Image contrast was improved after OPMF, indicating a reduction of mixing between adjacent planes in the mean image. These improvements are more readily observable in the line traces. With each additional processing step, the line traces show more peaks and valleys with steeper slopes, indicating a sharper image with greater contrast.

Figure 11.

Temporal mean from cine #16 before motion correction, after in-plane motion correction (IPMC) only, and after both IPMC and out-of-plane motion filtering (OPMF). Line traces of the intensities along the locations marked by the red line are shown at the bottom of each image. Without motion correction, only large features can be identified from the mean image. IPMC reduced the majority of the blurring, allowing discrimination of the tumor boundary in the B-mode image and blood vessels in the contrast specific image. Out-of-plane motion filtering further reduced motion within the cines and increased the sharpness and image contrast of the mean images. The line traces show that more detail was present with each extra processing step. All line traces are shown on the same scale.

III.C Observer evaluations

The results of the motion evaluations with and without IPMC on B-mode and contrast mode cines are summarized in Table 2. The results show a highly significant difference between the original cine and the IPMC cine. The IPMC videos were never graded to have worse motion than the original video. One cine was graded by both observers to have equal motion (cine #9), and a second cine was graded by one observer to have equal motion in the uncorrected and corrected cines (cine #6).

Table 2.

Observer grading of in-plane motion corrected B-mode and contrast mode cines. The Kruskal-Wallis test was performed and the mean rank sums are displayed along with 95% confidence intervals. p < 0.001.

In-Plane Motion Correction	Mean Rank Sum	95% C.I.
No IPMC	131.2	124.5 – 138
IPMC	45.8	39.1 – 52.5

The results of the motion evaluations on IPMC and OPMF are summarized in Table 3. The results indicate that performing IPMC and OPMF together produced videos with significantly less observed motion than when either of these two techniques were left out. Performing OPMF without first applying IPMC had the worst results, suggesting that IPMC usually contributed more to the overall motion correction than OPMF. Comparing the full set of graded IPMC and Standard OPMF against IPMC and Best OPMF cines, there was a small overlap in the 95% confidence intervals with Tukey’s least significant difference test for multiple comparisons. However, the Best OPMF was implemented as a selection among four other methods, and the standard OPMF method was selected in 7 out of 22 cines. Comparing just the grades from the cines where the Best OPMF method did not use the Standard OPMF method (15 cines, 60 total grades) with the Kruskal-Wallis test showed statistically significant results (p = 0.0012).

Table 3.

Observer grading of in-plane motion corrected and out-of-plane motion filtered B-mode and contrast mode cines. The Kruskal-Wallis test was performed and the mean rank sums are displayed along with 95% confidence intervals. For the out-of-plane motion filtering grades, Tukey’s least significant difference method was performed for multiple comparisons.

In-Plane Motion Correction	Out-of-Plane Motion Filtering	Mean Rank Sum	95% C.I.
No	Standard	277.3	262.6 – 292
Yes	None	240.7	226 – 255.4
Yes	Standard	107.6	92.9 – 122.4
Yes	Best	80.4	65.6 – 95.1

III.D TIC fitting with motion corrected cines

Fitting TICs to a model is a common method of analyzing CEUS cines. To quantitatively determine if the 2-tier IPMC and OPMF techniques presented here impact quantitative analysis, the CEUS TICs were fitted to equation 1 on a P×P basis within the user defined ROI for the uncorrected cine, in-plane motion corrected cine, and in-plane motion corrected and out-of-plane motion filtered cine. The coefficient of determination was selected to represent the goodness-of-fit for comparison across the different levels of motion processing, and the results are displayed in Figure 12. The mean ± standard deviation of R² for the uncorrected cine, IPMC cine, and IPMC + OMPF cine were 0.24 ± 0.08, 0.28 ± 0.09, and 0.31 ± 0.09, respectively. In general, R² improved with each added motion correction step, implying that the 2-tier IPMC and OPMF each improved the accuracy of fitting TICs on a pixel-by-pixel basis. Increases in R² at each step were statistically significant (p < 0.001) with more than 99% power according to the two-tailed paired t-test with α = 0.05. Improving the GoF for TICs should help diagnostic accuracy of quantitative CEUS.

Figure 12.

Mean goodness-of-fit (GoF) from fitting CEUS time-intensity curves on a pixel-by-pixel basis for each cine as calculated by the coefficient of determination (R²). GoF was determined for the uncorrected cine (No M.C.), the in-plane motion corrected cine (IPMC), and the in-plane motion corrected and out-of-plane motion filtered cine (IPMC + OPMF). Except for cine #11 where IPMC causes scale markers to enter the region-of-interest, R² improves with each added motion correction step. Increases in R² are statistically significant (p < 0.001) according to the Wilcoxon sign-rank test.

IV. DISCUSSION

IV.A In-plane motion correction

The quantitative and subjective observer evaluations showed that IPMC cines had markedly less motion than the uncorrected cines. This 2-tier scheme of registering frames sequentially allowed the use of sequential image registration without the risk of propagating errors to the ends of the cine. Sequential image registration performed better at handling out-of-plane motion than directly registering to the reference frame. This allowed the entire cine to be motion corrected, not just those frames that were sufficiently similar to the reference frame to allow direct image registration. With the entire cine motion corrected, the entire cine could be smoothly viewed without a dropout of frames coinciding with out-of-plane motion.

Since motion transformation parameters were propagated along with sequential image registration approaches, there was a potential for errors from the misregistration of two adjacent subreference frames to severely degrade the motion correction results. This type of error could occur when the ultrasound transducer was accidentally moved far from the reference image slice. The algorithm to search for SRFs was designed to account for these situations. The 30 second search window was established to allow sufficient time for the image probe to be repositioned near its original location, and the direct registration of all SRFs within the search window to the “current” SRF prevented the propagation of errors. The 2-tier IPMC strategy provided stable motion correction throughout the cine by registering sequential subreference frames that effectively bypassed the respiratory cycles and other major motion events.

Sequential image registration works well under the assumption that out-of-plane motion between each frame is small relative to the width of the ultrasound beam. This assumption guarantees that there is mutual information between each pair of frames to allow proper alignment. This assumption was rarely violated when capturing at 7-11 FPS, and the only instances when this assumption was noticeably violated were those instances where a fast motion event occurred, such as a cough. In such cases, the 2-tier motion correction strategy easily handled the situation by containing the errors within the local motion cycle.

There were only two cines where the observers judged the IPMC cine to have an equal amount of motion as the uncorrected cine and no cases where the motion corrected cine was worse than the uncorrected cine. In cine #6, there was very little in-plane motion present in the original cine. Since there was such a small amount of motion, one observer graded the uncorrected cine to be equal to the corrected cine. In cine #9, however, there was an ample amount of motion, but both observers saw no improvement with IPMC. The tumor in cine #9 was hypoechoic and appeared featureless in the B-mode images, preventing image registration from finding the correct alignment. This situation can be ameliorated by instructing the observer to select an ROI larger than the tumor that includes surrounding tissue.

IV.B Out-of-plane motion filtering

No single OPMF technique performed dominantly better than the others according to the out-of-plane motion metric OPMM (Table 1). Each method appeared to perform better at different levels of out-of-plane motion. Both of the Subsampling methods were generally selected more often when there were lower levels of out-of-plane motion, and the Mean Frame method was selected more often when there were high levels of out-of-plane motion. This may have occurred because the Subsampling methods compared individual frames, which each only carry information from a single imaging plane, with the other subsampled frames throughout the cine to pick the best OPMF reference frame. When there was little out-of-plane motion, more frames in each motion cycle could be found at the same imaging plane, allowing this method to optimally choose the most accurate OPMF reference frame. In contrast, the Mean Frame method compared the mean frame, which contained information from multiple imaging planes, with all other frames within the cine. In cines where substantial out-of-plane motion was present, each respiratory cycle may contain different sets of imaging planes. The Mean Frame method allowed choosing an OPMF reference frame with an imaging plane that had greatest presence throughout the cine. Furthermore, since all frames were compared to the mean frame, there was a larger set of frames from which to choose an ideal imaging plane.

Since each OMPF method appeared to perform best under different circumstances, a combinatorial OPMF technique was applied instead of a single technique to reliably reduce out-of- plane motion. Applying all four techniques took a total of 22.4 sec (0.014 sec/frame) on average across all cines. However, the application of the Subsample (SAD) OPMF as the standard OPMF method still produced good results on its own.

The overall motion correction processing, including the fast Fourier transform IPMC step, 2-tier IPMC, and combinatorial OPMF but excluding manual ROI selection steps (Fig. 1), required 2.09 sec/frame to complete on average across all cines with unoptimized and single threaded code in MATLAB. Individually, the Fourier transform IPMC, 2-tier IPMC, and combinatorial OPMF required an average of 0.15 sec/frame, 1.93 sec/frame, and 0.01 sec/frame of processing time. With multithreading on a modern quad-core processor or GPGPU programming, these timings can be dramatically reduced. For example, the image comparisons during the selection of the ideal reference frame, during image registration, and while performing OPMF are highly parallelizable. The four OPMF techniques can be performed in parallel as well.

According to the observer evaluations, the combination of IPMC and Best OPMF provided the most overall motion correction. Because the method selected for the Best OPMF was based on the OPMM, the observer rank results also show correlation between lower OPMM and reduced out-of-plane motion. Although measuring variance in the CEUS TIC was not as sensitive in detecting motion as using a similarity metric on the B-mode image, this strategy allowed measurement of the impact of OPMF on a signal that was separate and independent from the one analyzed by OPMF. Furthermore, this strategy is more relevant since the TIC is heavily used throughout CEUS literature.

The 2-tier IPMC and adaptive OPMF techniques developed here performed very well over the large majority of the 22 FLL cines. Previously published techniques rely on regular motion patterns where the tumor is in approximately the same positions at the same phases of each respiratory cycle.^{28, 32} These conditions were rarely present among long 3 minute sequences for the FLL cines. As seen in Figure 7, there was a wide variety of motion profiles observed across all cines. Respiration was rarely uniform throughout the entire sequence, and motion of the transducer caused different tissue slices to be imaged at corresponding phases of the respiratory cycle across respiratory cycles. Since the IPMC and OPMF techniques developed here were designed to account for these variable motion profiles, they should provide robust motion correction for broad clinical ultrasound imaging applications. Indeed, while the direct IPMC and sequential IPMC methods performed well across some cines, the 2-tier sequential IPMC performed well across all 22 cines (see Figure, Supplemental Digital Content 1). By applying these IPMC and OMPF techniques, the accuracy, reliability, and clinical utility of quantitative CEUS techniques can be consistently improved.

IV.C Limitations

OPMF automatically selected the most commonly observed anatomy for use as the reference. This approach made the basic assumption that a good cross section of the tumor for analysis was captured in the imaging plane for the majority of the cine. If the sonographer is unable to image an appropriate cross section of the tumor for the majority of the cine due to heavy motion or other imaging restrictions, then this method of automatically selecting an OPMF reference frame may fail. However, the adaptive OPMF method is compatible with a manual selection of an OPMF reference frame.

In an effort to provide an even selection of data points for the TIC throughout the cine, only the most deviant frames from each motion cycle were filtered out by the OPMF. By using the low-pass filter of the correlation curve as the cutoff for filtering, roughly 50% of the frames from each motion cycle were allowed through the filter. This method made the assumption that 50% of the frames in each motion cycle were in similar positions to the OPMF reference frame. In cases where there is a large component of out-of-plane motion, this assumption may not be valid and the OPMF may allow too much motion to pass through the filter. In such cases, it may be worthwhile to select fewer frames per respiratory cycle. For example, the OPMF could be adapted to select the desired percentage (e.g., 10%) of frames in each motion cycle with the best NC to the OPMF reference frame. Also, a global threshold could easily be added to the OPMF to ensure no drastically deviant frames are allowed through the filter.

V. CONCLUSION

Contrast-enhanced ultrasound sequences of focal liver lesions acquired under free breathing are subjected to in-plane and out-of-plane motion from respiration, patient and transducer movements that degrade the accuracy of quantitative measurements, particularly when performed on a P×P basis. The approach presented here corrects both in-plane and out-of-plane motion on ultrasound real-time images acquired free hand and for an extended period. It handled non-uniform respiratory motion, major sudden motion, and transducer motion while minimizing error accumulation characteristic of techniques that simply register each frame to the subsequent frame.

The 2-tier sequential in-plane motion correction that uses subrefrence frames to correct for motion over the entire cine and then within each respiratory cycle along with adaptive out-of-plane motion filtering were consistently superior to the original cine both subjectively and quantitatively. The OPMF reduced apparent motion by adaptively removing frames imaged off-plane from the automatically selected OPMF reference frame. These techniques performed well under a wide variety of breathing motion patterns. Since each of the OPMF methods performed best under different scenarios, adopting the best method selected by the out-of-plane motion metric (for each cine proved to be the superior approach across all cines. Combining the 2-tier IPMC and adaptive OPMF motion compensation approaches improved the stability of tissue positioning over the entire length of the CEUS cines. The semi-automated processes only required user input to select the ROI, optimizing workflow. These algorithms can be applied prior to quantitative pixel-by-pixel analysis to improve the accuracy of perfusion measurements.

Supplementary Material

1. Supplemental Digital Content 1.

Figure that shows the results of the goodness-of-fit tests for in-plane motion correction comparison. pdf