Sparse matrix beamforming and image reconstruction for real-time 2D HIFU monitoring using Harmonic Motion Imaging for Focused Ultrasound (HMIFU) with in vitro validation

Abstract

Harmonic Motion Imaging for Focused Ultrasound (HMIFU) is a recently developed High-Intensity Focused Ultrasound (HIFU) treatment monitoring method. HMIFU utilizes an Amplitude-Modulated (f_AM = 25 Hz) HIFU beam to induce a localized focal oscillatory motion, which is simultaneously estimated and imaged by confocally-aligned imaging transducer. HMIFU feasibilities have been previously shown in silico, in vitro, and in vivo in 1-D or 2-D monitoring of HIFU treatment. The objective of this study is to develop and show the feasibility of a novel fast beamforming algorithm for image reconstruction using GPU-based sparse-matrix operation with real-time feedback. In this study, the algorithm was implemented onto a fully integrated, clinically relevant HMIFU system composed of a 93-element HIFU transducer (f_center = 4.5MHz) and coaxially-aligned 64-element phased array (f_center = 2.5MHz) for displacement excitation and motion estimation, respectively. A single transmit beam with divergent beam transmit was used while fast beamforming was implemented using a GPU-based delay-and-sum method and a sparse-matrix operation. Axial HMI displacements were then estimated from the RF signals using a 1-D normalized cross-correlation method and streamed to a graphic user interface. The present work developed and implemented a sparse matrix beamforming onto a fully-integrated, clinically relevant system, which can stream displacement images up to 15 Hz using a GPU-based processing, an increase of 100 fold in rate of streaming displacement images compared to conventional CPU-based conventional beamforming and reconstruction processing. The achieved feedback rate is also currently the fastest and only approach that does not require interrupting the HIFU treatment amongst the acoustic radiation force based HIFU imaging techniques. Results in phantom experiments showed reproducible displacement imaging, and monitoring of twenty two in vitro HIFU treatments using the new 2D system showed a consistent average focal displacement decrease of 46.7±14.6% during lesion formation. Complementary focal temperature monitoring also indicated an average rate of displacement increase and decrease with focal temperature at 0.84±1.15 %/ °C, and 2.03± 0.93%/ °C, respectively. These results reinforce the HMIFU capability of estimating and monitoring stiffness related changes in real time. Current ongoing studies include clinical translation of the presented system for monitoring of HIFU treatment for breast and pancreatic tumor applications.

Introduction

HIFU is a noninvasive, extracorporeal, radiation-free thermal ablation technique for treatment of oncologic and fibrotic diseases across a wide range of applications including liver, breast, muscle, prostate and kidney, and bone metastatic disease, serving as a valuable alternative option to standard surgical procedure [1]. Despite the unique variety of advantages offered by HIFU treatment, currently standard clinical guidance and monitoring are limited to either Magnetic Resonance Imaging (MRI) guided Focused Ultrasound (MRgFUS) [2-4] or Ultrasound Imaging (B-mode) [5-8]. Some commercially available scanners capable of monitoring HIFU treatment include the ExAblate® 2000 system (Insightec Ltd.), Sonalleve MRHIFU system (Koninklijke Philips N.V.) and Model JC and Model JC200 system (Chongqing Haifu (HIFU) Technology Co., Ltd.). MRI is capable of providing high resolution guidance [9, 10] and monitoring based on either displacement induced by HIFU beam or mapping of the thermal dosage from the focal temperature rise detected through the change in proton resonance frequency. On the other hand, B-mode qualitatively detects the appearance of hyperechoic regimes following or during the formation of thermal lesions as bubbles formed during boiling [6, 7, 11, 12]. Nevertheless, neither MRgFUS nor B-mode can provide quantitative information on the mechanical property of the lesion throughout the HIFU treatment.

In the past decade, therapy guidance and monitoring have also become an emerging application branch of tissue elasticity imaging [13-15] with feasibilities shown in treatment applications. Several ultrasound-based elasticity imaging have also been developed for guidance and monitoring of ablative treatment: Past studies include but are not limited to static [16] and dynamic [17] elastography for HIFU treatment [18-22] and RF ablation [23], Vibroacoustography for cryo-ablation treatment[23], Magnetic Resonance Elastography (MRE) for HIFU treatment [24] and laser ablation treatment [25], Supersonic Shear Imaging (SSI) for Histotripsy treatment [26] and HIFU [27, 28] treatment monitoring, and Acoustic Radiation Force Impulse (ARFI) Imaging for RF ablation [29, 30] and HIFU treatment [31]. In addition, several other ultrasound or acousto-optic based guidance techniques have shown feasibility in guidance and HIFU monitoring applications such as echo-shift imaging [32, 33], and passive cavitation imaging [34]. Nevertheless, with respect to monitoring of HIFU treatments, several fundamental limitations still remain amongst the aforementioned techniques: Both static (e.g., Elastography) and dynamic elasticity imaging (e.g., Sonoelasticity Imaging) relies on external global excitation forces, which limit their accuracy in estimating the local mechanical property distributions, especially under unknown and asymmetric boundary conditions. MR-based monitoring techniques are limited in frame rate (0.1- 1Hz) and requires the costly MRI platform. In addition, the MRgFUS method usually underestimates the focal temperature during enhanced HIFU treatment where cavitation bubbles are present [35, 36]. On the other hand, echo-shift based imaging is also bound by the upper limit of 50 °C in estimated temperatures when the tissue coagulation changes the assumed properties of the backscattered ultrasound signals [37]. Other acoustic radiation force-based modalities require the HIFU to be interrupted during radiation force interrogation, which significantly increases the treatment window and cumulatively reduces the efficiency of an already extensive HIFU treatment procedure which usually requires a series of raster-treatment locations across the entire tumor volume.

Harmonic Motion Imaging for Focused Ultrasound (HMIFU) utilizes a single HIFU transducer emitting an amplitude-modulated (AM) beam for inducing both thermal therapy while inducing a stable oscillatory tissue displacement at its focal zone. The oscillatory response, namely HMI displacement, is estimated using the radio-frequency (RF) signals recorded during the HIFU treatment through a confocally-aligned pulse-echo imaging transducer [38] [39]. The localized tissue response is monitored continuously from the onset of HIFU treatment and aims to provide the onset of treatment termination to the surgeon based on the change in local tissue stiffness in order to prevent any overtreatment. Several studies have been published in order to investigate the principle in silico [40, 41], as well as feasibilities in vitro [42], ex vivo [43], and in vivo [44] using a 1D [42] and 2D [44] system. However, until now the systems used have required separate acquisition and processing units, where displacement estimation were performed offline in a different hardware unit. Therefore, in order to fully translate the HMIFU technique towards clinical use, it is necessary to implement a clinically-oriented, fully-integrated high frame rate platform capable of analyzing and streaming real time feedback of HMI assessment back to the user.

In order to develop a high-frame-rate ultrasound imaging modality, it is necessary to build a system with fast and efficient imaging acquisition, reconstruction (i.e., beam forming), and displacement estimation algorithm. Historically, the emergence of high frame rate imaging stems from the concept of parallel beamforming, which was initially proposed with reconstruction of an entire image following a single acoustic transmission with frame rates up to 1000 frames per second through parallel beamforming with fast analog multiplexing [45]. Later, the parallel processing technique was successfully implemented and validated in vivo using a phased array configuration, namely “Explososcan”, where the data acquisition rate was quadrupled with simultaneously reconstructing four receiving beams per a wide single transmit beam [46-48]. Recently, several Graphical Processing Unit (GPU) based beamforming approaches have been developed and implemented onto commercial scanners to further increase the imaging framerate and resolution. These have also been developed to achieve high frame rate imaging such as Synthetic Aperture (SA) imaging [49-53], and Short-lag Spatial Coherence Imaging (SLSC) [54].

In the field of ultrasound elasticity imaging, numerous software beamforming techniques utilizing various transmit sequences have also been developed and implemented into commercial scanners to achieve high imaging rates and resolution such as composite imaging [55], plane-wave [56] or divergent transmit beam [57, 58]. High frame rate elasticity imaging has demonstrated a promising clinical value in quantitative imaging of tissue viscoelasticity with estimation of motion generated by external compression or acoustic radiation force such as Transient Elastography [59, 60], Shear Wave Imaging (SSI) [56, 61], Elastography [62], ARFI imaging [63], and Harmonic Motion Imaging [64].

Conventional ultrasound elasticity imaging techniques rely on previously beamformed RF signals which in turn requires the beam reconstruction of the entire field of view through the entire imaging depth. In localized elasticity imaging for HIFU monitoring (e..g, HMIFU), only the focal spot is considered as the region of interest. Therefore, a more effective beamforming strategy for HIFU treatment monitoring would be to reconstruct only the focal region, which in turn reduces computational cost and allows real-time streaming of elasticity maps throughout the entire treatment window. Nevertheless, there has yet to be a novel beamforming algorithm developed for such ultrasound elasticity imaging based HIFU monitoring application, where simultaneously high frame rate, high spatial resolution, real-time feedback are achieved over continuous monitoring of several minutes. Therefore, a fast parallel beamforming algorithm with further improvement in reconstruction speed is required for real time, 2D elasticity imaging based HIFU monitoring in order to detect the onset of effective treatment (i.e., cell necrosis) and stop the treatment to spare as much normal surrounding regions as possible. More importantly, monitoring feedback abilities in HIFU treatment is also the key to maximizing the surgical efficiency because HIFU treatment is known to be a lengthy procedure especially when the target tumor volume is large compared to the focal spot size of the HIFU transducer, i.e., a series of treatment sequence is applied across the entire tumor volume in a raster scan manner. Therefore, the ability to detect the onset of lesion formation at each individual treatment location can decrease the total time of the ablation procedure.

Our objectives in this study are first to develop and implement a sparse-matrix algorithm for parallel beamforming and scan conversion in order to achieve real-time 2D HMIFU monitoring. Although the sparse matrix beamforming and reconstruction techniques can be applied to a wide range of applications, our aim here is to implement the developed algorithm onto a fully-integrated, clinically relevant commercial ultrasound scanner with high frame rate real time imaging capability by incorporating a GPU-based algorithm. The completed platform is expected to provide a quantitative real time 2D monitoring feedback during the HIFU treatment directly back to the user. We also aim at demonstrating initial feasibility as well as reproducibility of our system in elasticity imaging and HIFU treatment monitoring through phantom and in vitro tissue experiments. We expect to observe under a real-time streaming mode, a reproducible range of displacement across our imaging measurement on the phantom study, as well as detect the onset and decrease of displacement throughout HIFU treatment, which indicates for stiffening of tissue upon lesion formation.

Materials and Methods

1. Therapy and Imaging Unit (Hardware)

A 93-element, PZT-4 ceramic HIFU Array (H-178, Sonic Concept Inc., Bothell WA, U.S.A, Diameter _{individual element} = 10 mm, Diameter_{overall outer} = 110 mm, Diameter_{overall inner} = 41 mm, f_center = 4.5 MHz, Focal depth = 70 mm) was used in this study. The geometric and acoustic parameters of the HIFU transducer were designed for clinical application of localized HIFU treatment on superficial organ applications. The transducer surface is covered with a polyurethane based membrane, which is coupled with a custom sterilization and degassing system (WDS-104, Sonic Concept, Bothell, WA, U.S.A.) with control of both volume and circulation flow of degassed cooling water within the transducer-tissue interface during HIFU treatment. All channels for the 93 elements were synchronously excited by an AM-HIFU signal (f_carrier = 4.5 MHz, f_AM = 25Hz) generated through a dual-channel arbitrary waveform generator (AT33522A, Agilent Technologies Inc., Santa Clara, CA, U.S.A.). The emitted HIFU beam is capable of inducing an oscillatory motion at the focal zone in addition to inducing the conventional thermal ablation. The oscillatory motion is estimated based on the RF signals acquired by a confocally aligned diagnostic transducer in order to achieve real-time HMIFU monitoring during HIFU application. The extrapolated in situ focal acoustic pressure and intensity (I_sptp) was extrapolated to be 6.5 MPa and 9067 W/cm², respectively, based on prior hydrophone (HGN-0200; Onda Corporation, Sunnyvale, CA, U.S.A.) calibration procedure [40]. The diagnostic transducer is a 64-element phased array (ATL., Bothell, WA, U.S.A., f_center = 2.5 MHz) has been confocally fitted through a circular void or the HIFU transducer aperture through a custom built water-proof mechanical gasket with rotational degree of freedoms, i.e. the confocally-aligned imaging probe can be adjusted rotationally for adaptive targeting and monitoring at 10 steps with individual step of 36°. The phased array transducer is operated through a 4-board VDAS system (Verasonics, Bothell, WA, U.S.A.) and a 260-pin header connector. The coupled transducer pair was mounted and maintained stationary onto a 3D positioning system (Velmex Inc., Bloomfield, NY, U.S.A.) during both imaging and treatment protocols. The transducer pair was mechanically moved using the positioning system only between the imaging or therapy protocols for positioning and alignment adjustment purposes as needed. In order to synchronize the acquisition of the monitoring signals (i.e., the pulse-echo imaging sequence) with the onset of HIFU treatment, the therapeutic transducer was simultaneously triggered with the VDAS imaging system through a MATLAB-based (Mathworks, Natick, MA, U.S.A.) custom algorithm on a host PC (Precision T7500, Dell Inc., Austin, TX, U.S.A.) (Figure 1).

Figure 1.

a 2D HMIFU system. The focal depth of the HIFU is 70 mm. Height of degassed water interface was adjusted around 35 mm, i.e., focal spot localized at 25 mm from surface of the target. The degassing circulation system was turned off during the experiment and turned on between trials to maintain degassing quality and transducer cooling. 1b. Acquisition PC controls both transducers through waveform generator, Verasonics System, and the translational stage. 1c. Upgraded 2D-HMIFU framework for thermocouple temperature monitoring using two acquisition PCs controls both transducers through waveform generator, Veraonics System, thermometer, and the translational stage, respectively. 1d. Photo of the multi-element HIFU array with encasing of the confocally aligned phased array imaging probe.

2. Therapy and Imaging Unit (Software)

2a. Data acquisition and storage

The channel data signals were individually acquired through a 64-element phased array and the Verasonics system using a custom single-transmit based divergent wavefront imaging sequence [58]. The acquisition frame rate was set at 1000 frames/sec, the analog-to-digital (A/D) sampling was chosen to be 10 MHz since a 2.5 MHz diagnostic probe was used for this study. The acquisition sequence is repeated continuously while the acquired frames were transferred in a stacked set of 10 frames through an external function operated within the host computer, where additional reconstruction algorithms will be applied. In this study, we have also chosen to store all beam-formed radio frequency (RF) frames in order to demonstrate the reliability of our technique.

2b. GPU implementation of linear operators

GPU-based algorithms are formidable tools to drastically improve processing speeds, often by orders of magnitude when compared to standard MATLAB implementations. However, translating MALAB codes that often rely on pre-compiled proprietary functions to the Compute Unified Device Architecture (CUDA) language requires a significant amount of time and skill. Here, we have developed a simple approach to execute efficiently linear operations on the GPU with minimal effort and straightforward MATLAB integration. We have built our algorithm through the sparse matrix option of JACKET package (AccelerEyes, Atlanta, GA. U.S.A.) that can perform highly optimized sparse matrix-vector products on the GPU seamlessly in a MATLAB environment. As any linear operation can be recast as a matrix (from now on we will refer to this matrix as a “function matrix”), it suffices to find that function matrix to obtain a high performance GPU function of said linear operation by using the JACKET package. The developed algorithm uses only interpreted MATLAB algorithms, which is advantageous in terms of both flexibility and ease-of-use. More specifically, let's consider a function f with input x and output y, which can be a combination of any number of linear operations, including compiled functions such as interp2 in MATLAB. In the following equation, the x and y are vectors (or matrices) containing a total of N and M elements, respectively. We have

$$y-f\left(x\right),$$ (1)

Because f is linear, there exists a matrix A such that

$$y=Ax,$$ (2)

where y and x are recast as vectors in R^M and R^N, respectively, without loss of generality. To find A, we apply f to the k^th standard basis vector e_k (i.e., a vector with zeros everywhere except in the k^th position) and obtain:

$$y=f\left(e_k\right),$$ (3)

$$y=\sum _j{A}_{ij}{e}_k.1\le i\le M,$$ (4)

$$y=A_{ik},1\le i\le M,$$ (5)

or, in other words, f(ek) is the kth column of the function matrix. By simply repeating this operation for all k, one can recover the matrix A. The reconstruction matrix can be used for beamforming a set of any amount of frames, and is also designed to host data with varying depth and sampling resolution. Detailed implementation schematic has been provided in Figure 2 with descriptions of y, A, and x in order to demonstrate the implementation of a sparse matrix based beamforming and reconstruction procedure. When working with images, the function matrix A can be very large, e.g., in our case ranging from 6 ×109 to 48 ×109 elements, depending on the up-sampling rate and spatial size of displacement map reconstruction, hence the importance of using sparse matrix formats when possible, i.e., a regimen in which only non-zero elements are allocated. This case is very common, e.g., in the case of 2D linear interpolation: let vector x contain N elements corresponding to N pixels of a given image, and y contain M>N elements corresponding to M pixels of the interpolated image. For a typical 4-neighbor interpolation scheme, an interpolated pixel y_i is given by a linear combination of 4 pixels of vector x. Therefore, the i^th line of A_ij, used to compute pixel y_i, contains 4 non-zero values and N-4 zeros, with N typically larger than 10,000. It is therefore highly beneficial both in terms of memory requirements and computational speeds to represent the matrix A in its sparse form.

Figure 2.

Representation of reconstruction sparse matrix A (a), channel data matrix (b), and reconstructed RF data matrix (c). The channel data matrix was reshaped to perform the reconstruction for all the frames in a single operation. The reconstructed RF data matrix obtained was then reshaped to put beam lines in the second dimension and the frames in the third dimension.

Generating the function matrix itself can be computationally expensive both in terms of time and memory; however, the function matrix has to be computed only once, which expedites the process of generating the function matrix to code compilation. Additionally, in many cases, smaller matrices can be obtained from larger matrices in a straightforward manner by removing appropriate lines of the function matrix, since each column of the function matrix correspond to one pixel of x, and each line of the function matrix corresponds to one pixel in y. This property can be used to adjust the angle field-of-view and the depth in real-time, without re-computing the function matrix.

2d. GPU based sparse matrix RF data reconstruction

In the present work, we used this approach to recast the delay-and-sum beamforming [65] as well as the scan conversion in matrix-vector products since these operations are linear. In order to obtain every beamformed RF frame, two sparse matrices were generated for reconstruction and scan conversion, respectively. More specifically, every frame of RF data was reconstructed by first multiplying the channel data matrix with the reconstruction sparse matrix, then multiplying the product matrix by another sparse matrix for scan conversion (Figure 3), each performed as a single operation. The RF signals were up-sampled to 80 MHz and reconstructed on a 90 degrees field of view with 128 beam lines for the gelatin phantom imaging studies, and reduced to 30 degrees with 32 beam lines for the in vitro liver HIFU treatment monitoring studies due to transfer and storage efficiency for HIFU treatment monitoring studies, respectively. The reconstructed field of view was defined to be larger than the focal excitation region, because it is expected that the excitation region may increase with the formation and growth of the thermal lesion. In addition, a larger field of view can provide additional insightful information due to the propagation of shear waves. It is also noteworthy that the process of constructing the sparse matrix function matrix was itself performed on the GPU by using MATLAB GPU-compatible operations to limit processing times.

Figure 3.

Flow-chart of displacement image reconstruction algorithm. Note that frames were acquired and transferred from VDAS to host computer in a synchronous way, i.e., every acquisition is triggered by the completion of the previous processing.

2e. Displacement estimation

In order to filter out the HIFU frequency in the received echo from diagnostic transducer, HMIFU system usually incorporates a low-pass filter [66] or band-pass filter [39, 40] depending on the center frequency of the diagnostic probe with respect to that of the therapeutic probe. In this study, a 6^th order low pass filter with a cutoff frequency at 4 MHz was applied to the beamformed RF signals in order to remove the interference HIFU frequency component without affecting the receiving bandwidth of the diagnostic transducer (2-4MHz).

Next, a 1-D normalized cross-correlation technique [67] was used to estimate the axial displacement along each lateral beam lines between two pre-selected frames within the acquired frames (window size of 3.85 mm and 90% overlap). Another 6^th order low pass filter at 100 Hz cutoff frequency was also applied along the temporal space before the 2D HMI displacement images were constructed using the sparse-matrix based scan conversion as described in section 2b (Figure 3). In order to verify the reliability of the streamed data in this study, we have utilized another acquisition method where a separate set of 200 frames was acquired, transferred, and only beamformed before stored in the host computer.

3. Phantom and in vitro experiment

3a. Gelatin phantom experiment

A gelatin phantom (n = 1, location = 3, measurement = 3) using gelatin bloom 50 powders (MP Biomedicals LLC., Santa Ana, CA, U.S.A.) and scatterers using 10% agar powders were manufactured based on prior literatures [68]. The acoustic attenuation was 0.5 dB/MHz/cm and speed of sound was 1551.7 m/s while the gelatin concentration was 4.9g/L [68]. The constructed phantom was designed to cure with a cylindrical shape (diameter 120 mm, height 60 mm) with a Young's Modulus of 10 kPa. The phantom was placed on an acoustic absorber in order to minimize any interface reflection interference and degassed echo gel (AQUASONIC®100, Parker Laboratories, Inc., Fairfield, NJ, U.S.A.) was placed above the phantom between the transducer membrane for impedance matching (Figure 1a). The imaging sequence consisted of a continuous 1.2 seconds excitation, and data were transferred back to the host PC for a set of 400 ms, equivalent to 20 cycles of HMI excitation. The water in the coupling membrane of the HIFU transducer was degassed for 2 hours prior to the experiment using the circulation system, and acoustic gel was also degassed for one hour prior to the experiment.

3b. In vitro canine liver experiment

Initial feasibility studies (subject=2, lobes = 2, treatment location = 3) and reproducibility studies (subject = 6, lobe = 6, treatment location = 19) were performed using canine livers excised immediately upon animal sacrifice and immersed into degassed Phosphate buffered saline (PBS) solution bath maintained at temperature of 25 °C. The specimens were degassed for two hours prior to the HIFU in order to prevent any air trapped inside. Each specimen was fixed using metallic needles onto an acoustic absorber submerged in a de-ionized and degassed PBS tank (Figure 1a). The HIFU treatment sequence consisted of a continuous 120-seconds excitation, and beamformed RF data frames were transferred back to the host PC at a rate of 100 frames per second, equivalent to 20 cycles of HMI excitation. Focal temperature monitoring was also performed by inserting a T-type bare wire thermocouple with diameter of 25 μm (Omega Inc., Stamford, CT) inside the tissue. The diameter of the thermocouple was chosen to be smaller than 1/10 of the carrier wavelength in order to minimize reflection and viscous heating artifacts [69]. For thermocouple monitoring, the setup framework of the 2-D HMIFU system was optimized by utilizing two acquisition PCs in order to synchronously acquire both the thermocouple reading and the RF signals, where the VDAS system was used as a trigger source (Figure 1c).

Results

1. Gelatin Phantom experiment

Five displacement maps were obtained at three separate locations inside the gelatin phantom. B-mode imaging was performed before each imaging in order to optimize the field of view. For each displacement image, a 1.2 seconds continuous HMIFU excitation was applied and the RF signals were recorded at sets of 20-cycles (400 ms). The focal excitation zone was clearly imaged for every location investigated and also centered at the focusing depth of HIFU transducer, which is 70 mm with -6 dB boundaries encompassing an ellipsoidal shape with diameters of 10 mm (axial) by 5 mm (lateral) (Figure 4). The distribution and magnitude range of the displacement profile at maximum excitation (Figure 4a, 4b), relaxation (Figure 4e, 4f), and zero (Figure 4c, 4d) force phase all remained reproducible for every cycle across the entire imaging sequence. Any estimated motion beyond the edges of phantom should not be considered due to lack of scattering. The average peak-to-peak HMI displacement at each location was estimated to be 21.9 ± 7.98 μm, 23.9 ± 8.7 μm, and 21.6 ± 2.4 μm, respectively (mean ± standard deviation). A separate movie file (Movie 1) is attached, where a full set of displacement frames are shown during a 200 ms excitation period, it is noteworthy to see both focal displacement as well as propagation of shear waves generated from the focal excitation.

Figure 4.

Displacement imaging using HMIFU on a gelatin phantom. Peak negative (a,b), zero (c,d), and peak positive (e,f) displacement during a 50 Hz-cycle across two independent periods. (g) B-mode image of the gelatin phantom. (h) Focal oscillatory displacement produced by the HMIFU system, note that the oscillation frequency is 50 Hz, twice that of the AM frequency. Shear wave propagation can also be imaged in the phantom. Any estimated motion outside of the phantom should not be considered.

2. In vitro canine liver experiment

For each HIFU treatment, conventional B-mode imaging was used in order to target the focal zone within the region of interest inside the tissue. In the initial feasibility study, three HIFU treatments were performed across two liver lobes with HMIFU monitoring. The B-mode images were acquired before and after HIFU treatment (Figure 5, 6) to provide anatomical information and used to overlay with peak-to-peak HMI displacement images. The peak-to-peak HMI displacements within the focal excitation region (Figure 7) were monitored and processed using the same algorithm as for the gelatin phantom experiment throughout the entire 2-min long treatment period. A separate movie file (Movie 2) is attached, where a full set of displacement frames are shown during a 120-s HIFU treatment period. For each of the three initial feasibility case studied, a clear decrease in peak-to-peak HMI displacement of 40%, 30%, and 33% was observed, respectively. For the reproducibility cases studied, 18 of the 19 reproducibility study HIFU treatment cases exhibited average displacement decrease of 45.2±20.8% (Figure 8). The difference between monitoring of displacement at the end of the HIFU treatment was found to be significantly lower than that of the beginning of HIFU treatment (P-value = 0.0003). In addition, 16 cases exhibited increase-then-decrease of displacement change during HIFU treatment, with average rate of increasing slope of 0.73±0.69 %/s, and average rate of decreasing slope of 0.60±0.19 %/s, respectively. Based on the thermocouple monitoring, the corresponding average rates of displacement increase and decrease with focal temperature were found to be 0.84±1.15 %/ °C, and 2.03± 0.93%/ °C, respectively. The same decrease trends were also clearly imaged in 2D, where individual single-cycle frame set consisting of maximum and minimum displacement profiles were shown in Figure 7 as a representative treated locations. In the initial feasibility cases, the detected thermal lesion size were also imaged as 251, 254, and 226 mm² from gross pathology with an expected consistency given that the HIFU treatment parameters remained the same for all cases. In addition, the estimated diameter of HMI focal region from the displacement images across the three treatment cases increased both in axial and lateral direction from before (9.8 mm × 8.2 mm, 9.3 mm × 7.6 mm, 9.2 mm × 6.6 mm, respectively) to after (13.0 mm × 11.3 mm, 10.9 mm × 8.4 mm, and 10.0 mm × 10.5 mm, respectively) HIFU treatment, leading to an estimation of the confirmed thermal lesion diameter from gross pathology (9.0 mm × 8.0 mm, 9.0 × 8.5 mm, 7.5 × 6.5 mm, respectively) (Table 1). The average size of all of the treated thermal lesion size amongst the reproducibility study cases was 236.6±140.2 mm².

Figure 5.

B-mode images with overlayed peak-to-peak HMI displacements (a) before, and (b) after HIFU treatment in the same cases as in Figure 3 (case 1) and the corresponding gross pathology image (c), respectively, along with (d) focal displacement monitoring during the course of treatment.

Figure 6.

B-mode images with overlayed peak-to-peak HMI displacement overlay (a) before and (b) after HIFU treatment in a separate treatment location case with (c) the corresponding gross pathology image, along with the (d) focal monitoring displacement during the course of treatment.

Figure 7.

In vitro liver peak-to-peak displacement imaging and monitoring of HIFU treatment using the 2D HMIFU platform. The peak-to-peak HMI displacement frames during a 50 Hz-cycle at representative time points of (a) 4 s, (b) 50 s, (c) 87 s, and (d) 117 s were selected from the HIFU treatment monitoring sequence to show the decrease of focal displacement as the thermal lesion forms.

Figure 8.

Statistical analysis of HIFU treatment monitoring studies using the developed 2DHMIFU platform. 18 of the 19 reproducibility study HIFU treatment cases exhibited displacement decrease of 45.2±20.8% with statistical significance (P=0.0003).

Table 1.

Comparison table of HMI focal excitation region and the diameter of thermal lesion size from gross pathology analysis following in vitro experiment.

Treatment Case	Focal excitation diameter at T = 5s (Axial vs. Lateral) (T = 5s)	Focal excitation diameter at T = 120s (Axial vs. Lateral) (T = 120s)	Thermal lesion diameter from gross pathology (Axial vs. Lateral)
1	9.8 mm vs.8.2 mm	13.0 mm vs. 11.3 mm	9.0 mm vs. 8.0 mm
2	9.3 mm vs. 7.6 mm	10.9 mm vs. 8.4 mm	9.0 mm vs. 8.5 mm
3	9.2 mm vs. 6.6 mm	10.0 mm vs. 10.5 mm	7.5 mm vs. 6.5 mm

3. GPU vs. CPU streaming speed

For both sets of experiments, the processing speed of GPU-based sparse matrix reconstruction algorithm, CPU-based sparse matrix reconstruction algorithm was compared against that of the CPU-based conventional reconstruction algorithm. In the gelatin phantom experiment, the motion display (i.e., processing time from data acquisition to displacement estimation) frame rate was 1 Hz using the GPU-based sparse matrix algorithm, 0.4 Hz using the CPU-based sparse matrix algorithm, and 0.01Hz using the CPU-based conventional reconstruction algorithm when reconstructing on a 90° field of view (128 lines) image from 50 to 90 mm deep (9.6 μm axial grid step). In the in vitro canine liver experiment, the motion display (i.e., processing time from data acquisition to displacement estimation) frame rate was 15 Hz and 5 Hz with reconstructing 32 and 64 RF lines, respectively, using the GPU-based sparse matrix, 2.6 and 1 Hz using the CPU-based sparse matrix algorithm, respectively, and 0.09 and 0.05 Hz using the CPU-based conventional reconstruction algorithm for a 40 mm range (9.6 μm axial grid step) and 30 degrees angle field of view image (Table 2).

Table 2.

Online streaming frame rate using CPU-based conventional reconstruction algorithm, CPU and GPU-based sparse matrix reconstruction algorithm under HMIFU imaging settings for a 40 mm range image with 9.6 μm axial grid step.

Field of view	Conventional CPU reconstruction	CPU-based sparse matrix reconstruction	GPU-based sparse matrix reconstruction
30°, 32 Beams	0.09 Hz	2.6 Hz	15 Hz
30°, 64 Beams	0.05 Hz	1 Hz	5 Hz
90°, 128 Beams	0.01 Hz	0.4 Hz	1 Hz

Discussion

HIFU treatment has shown great promise as a noninvasive, and non-ionizing, extracorporeal, and cost-effective thermal ablation technology for treatment of a variety of disease applications [1]. The ability to efficiently monitor the progress of HIFU treatment in real time remains as a challenging aspects and hinders its broad clinical translation. MRgFUS [2-4] and B-mode ultrasound [6, 7, 11, 12] are currently used in the clinic but lack the capability of providing quantitative lesion size mapping or real-time lesion formation detection. Recently, developed techniques such MRE [24], static [20] and dynamic Elastography [22], Acoustic Radiation Force Impulse Imaging [31], and Acousto-optical Imaging [70, 71] have been implemented to monitor HIFU treatment. However, despite the improvement in HIFU monitoring capabilities in appreciation to the development of aforementioned techniques, there still lacks 2D quantitative monitoring method which can be localized, in real time and does not further delay the HIFU treatment procedure. HMIFU is an acoustic radiation force based dynamic elasticity imaging technique using a HIFU transducer for transmitting an AM-HIFU beam to induce a stable focal oscillatory motion, which is related to the local tissue mechanical property, tracked by 1D cross correlation of RF signal acquired using a confocally-aligned diagnostic transducer. Although in principle HMIFU enables the technique to perform localized HIFU monitoring without interrupting the treatment, the system used in all past studies requires offline processing. In order to show the core value of real-time HMIFU with capability to stream displacement during the treatment window, it is necessary to develop a fast beamforming and reconstruction algorithm. Recently, GPU-based beamforming has becoming an emerging area within the field of parallel beamforming [45-48], where previous work have developed GPU-based beamforming techniques for applications of Synthetic Aperture (SA) imaging [49-53], real-time small displacement estimation [62, 63], and Short-lag Spatial Coherence Imaging (SLSC) [54]. In this study, we have designed and built a 2D HMIFU system equipped with a real-time feedback capable of streaming the displacement image during the ablation procedure by incorporating a novel sparse matrix beamforming algorithm implemented on GPU. Both phantom and in vitro tissue experiments were performed in order to assess the quality and stability performance of the completed system. We hypothesized that our system was capable of producing reproducible high frame rate, 2D HMI displacement mapping in gelatin phantoms, as well as displacement variation during treatment upon and detecting thermal lesion formation.

There are three main challenges behind monitoring of HIFU treatment: Detecting the onset of lesion formation, providing quantitative mapping of the treated region (i.e., thermal lesion), and performing efficient monitoring without delaying the HIFU procedure. To our knowledge, the capability of real-time monitoring and quantitatively mapping thermal lesion formation at a frame rate of 5 to 15 Hz is currently the fastest amongst existing 2D HIFU monitoring modalities. This approach facilitates the physicians with an enhancing temporal resolution to monitor and detect the onset of thermal lesioning indicating effective point of termination. MRgFUS provides temperature maps at sub-millimeter resolution but only at a frame rate between 0.1-1 Hz depending on slice thickness and spatial resolution. Echo shift imaging can offer capability of monitoring temperature rise up to 7 °C up at 400 Hz [32] but lacks the ability of MRgFUS to spatially map the temperature distribution. Nevertheless, both modalities currently lack the ability to monitor the onset of lesion formation and provide quantitative real-time imaging indicating the size of the formed thermal lesion. On the other hand, HMIFU is capable of streaming the focal displacement map which quantitatively delineates the region of thermal lesion based on the stiffness contrast. Both ARFI and SSI methodologies implemented a cost-effective, all-ultrasound-based HIFU with monitoring system capable of receiving beamformed RF signals between 11 to 17 kHz [27, 31]. However, ARFI utilizes a single transducer which must be excited at a low duty cycle (6%) in order to prevent transducer damage and the ARFI image is displayed interspersed between HIFU treatments at 0.2-Hz frame rate following a single mechanical excitation [31], whereas SSI also requires the interruption of the treatment for HIFU beam during the its plane shear wave excitation, limiting its frame rate to 0.333 Hz [27]. On the other hand, HMIFU is capable of continuously streaming focal displacement maps at up to 15 Hz throughout the entire HIFU treatment duration. The HMIFU system utilizes the same HIFU beam for both treatment and elasticity monitoring, thus can operate in a more efficient monitoring manner since it does not require the stoppage of HIFU treatment to perform the monitoring/imaging sequence. It is also important to note that even without implementing on GPU, the proposed sparse matrix based beamforming algorithm still improved the frame rate by 20 to 40 times from that of the conventional delay-and-sum beamforming algorithm between field of view of 30 ° to 90 °.

The HMI displacement images across the gelatin phantom were very reproducible, with the largest variance across locations to be under 9.6 %. In addition, the focal excitation region was clearly imaged across all cases, where ellipsoidal shaped displaced regions were centered around 70 mm, in agreement with the expected geometrical focal depth of the HIFU transducer. The displacement profile maps measured across different locations showed a strong consistency, thus validating the reproducibility of our beamforming and motion estimation algorithm and ensuring the performance reliability of the new 2D HMIFU system. Despite the fact that the HMIFU excitation was continuous for 1.2 seconds, tissue heating and the associated changes such as in speed of sound were expected to be negligible within such a short time window and the associated low temperature changes [43]. For monitoring of HIFU treatment studies, the focal excitation region was also clearly imaged across all the cases, where focal displacement decreased by 40%, 30%, and 33% for each initial feasibility study cases as well as decreased by 45.2±20.8% amongst the reproducibility study cases upon lesion formation with statistical significance (P=0.0003). These findings are also consistent with our previous work [39] as well as prior literature [24, 31]. The displacement decrease began around 60 to 80 seconds upon treatment initiation and progressively continued until the end. It is noteworthy that both the slopes of focal displacement increase with time and focal temperature were less than the slope of decrease. This can be related to the findings from previous ultrasound elasticity imaging based studies investigating the softening-then-stiffening phase change [27, 72], which also discussed that the collagen fibers denaturation took place around 60 °C where the hydrogen bonding maintaining the collagen helices will break [72, 73]. For the cases that did not exhibit the same displacement decrease or increase-then-decrease cases, we speculate that the underlying structural differences across the liver tissue specimens could induce such variation in mechanical property change under thermal treatment. The average size of the treated thermal lesions estimated from gross pathology was 236.6±140.2 mm² under the same treatment parameters, which also validated the consistency of our developed 2D HMIFU platform. This study proved the significance of fast, continuous monitoring and lesion imaging of HIFU treatment, which allows physicians to identify the onset of lesion formation and the ability to either terminate the treatment or continue to monitor lesion growth. Steady decrease in the HMI focal displacement, which suggests the onset of thermal lesion formation due to stiffening of the treated region, was observed throughout HIFU monitoring window in all of the completed treatment cases. The overlay of peak-to-peak HMI displacement map onto the B-mode image allows to perform quantitative mapping of the mechanical response change in the tissue while maintaining the anatomical information provided by the B-mode (Figure 5, 6). The growth of the focal displacement region is believed to be associated with the growing and stiffer thermal lesion (Table 2, Movie 2). In addition, the displacement images reproducibly mapped the changes in mechanical property upon lesion formation.

Despite our successful implementation and feasibility study using the 2D system with streaming feedback ability, there are several limitations of the aforementioned HMIFU platform. All of the single variable sparse matrices used in this study were constructed offline using a separate algorithm prior to the experiment, and the matrix computational cost can vary between few minutes to several hours, depending on the of up-sampling rate, beam density, as well as well as field of view. However, it is possible to circumvent the computational cost by first generating a single matrix at highest sampling rate and larger fields of view, and adapt the reconstruction matrices with reshaping and down-sampling in respective to the specific imaging parameter. The reconstruction speed will also influence the streaming speed, where a larger field of investigation with high sampling rate than the presented case will have lower streaming frame rate. The data transfer rate from the VDAS to the host computer has also been a bottleneck issue in this platform (e.g. 930 ms is required to transfer every 200 frames acquired at 1 kHz frame rate), which is also another limiting factor to the streaming rate of HMI displacement because the upper limit of data transfer rate per channel in VDAS system is 18 MB/s. Despite the additional hardware improvements, streaming rate of 10-15 Hz deemed a suitable rate for HIFU guidance.

It is also important to emphasize that the proposed sparse-matrix based reconstruction method is very simple for rapid-prototyping and implementing on any conventional ultrasound system. Also, the nature of matrix-based algorithms allows for flexible adaptation of other types of linear functions. The frame rate of 1 kHz has been selected which provides a good compromise between displacement quality (i.e., correlation coefficient) and streaming framerate. In addition, under the very fast frame rate (1 kHz), we are capable of monitoring not only focal displacement, but also capturing the propagation of shear waves generated through our focal excitation (please see attached displacement movie). The ability to track shear waves will further unlock other application potentials for HMIFU such as simultaneous focal and peripheral-focal region shear-wave based elasticity imaging of lesion formation, as well as assessment of 2D viscoelasticity change during HIFU treatment using a previously developed method [64]. Other ongoing and future work includes implementation of a raster-ablation sequence for treatment of large tissue volume through electronic beam steering of the 93-element HIFU array, as well as a composite imaging with real time overlaying displacement image onto B-mode to perform simultaneous beam guidance and lesion assessment. Finally, clinical translation of this system is currently being explored and tailored towards breast and pancreatic tumor ablation.

Conclusion

In this study, a fully-integrated, clinically relevant ultrasound imaging platform was built for 2D real-time HIFU monitoring using HMIFU. A GPU-based sparse-matrix algorithm has been developed and implemented for beamforming and scan conversion, which is capable of providing 2D real-time streaming during HIFU treatment up to 15 Hz without interruption. Reproducible HMI displacements were imaged and highlighted a clear excitation region across multiple locations from a gelatin phantom with a 10 kPa Young's modulus. HIFU treatment monitoring was also performed and a clear decrease in focal displacements upon lesion formation was obtained along with complementary focal temperature monitoring using T-type barewire thermocouple measurement in canine livers in vitro. Ongoing investigations will aim at raster ablation using beam steering with HIFU array, shear wave imaging, and the clinical translation of HMIFU for our upcoming studies.