Varying ultrasound power level to distinguish surgical instruments and tissue

Abstract

We investigate a new framework of surgical instrument detection based on power-varying ultrasound images with simple and efficient pixel-wise intensity processing. Without using complicated feature extraction methods, we identified the instrument with an estimated optimal power level and by comparing pixel values of varying transducer power level images. The proposed framework exploits the physics of ultrasound imaging system by varying the transducer power level to effectively distinguish metallic surgical instruments from tissue. This power-varying image-guidance is motivated from our observations that ultrasound imaging at different power levels exhibit different contrast enhancement capabilities between tissue and instruments in ultrasound-guided robotic beating-heart surgery. Using lower transducer power levels (ranging from 40 to 75% of the rated lowest ultrasound power levels of the two tested ultrasound scanners) can effectively suppress the strong imaging artifacts from metallic instruments and thus, can be utilized together with the images from normal transducer power levels to enhance the separability between instrument and tissue, improving intraoperative instrument tracking accuracy from the acquired noisy ultrasound volumetric images. We performed experiments in phantoms and ex vivo hearts in water tank environments. The proposed multi-level power-varying ultrasound imaging approach can identify robotic instruments of high acoustic impedance from low-signal-to-noise-ratio ultrasound images by power adjustments.

1. Introduction

Over the past decade, intracardiac surgery has seen many technological innovations, ranging from new interventional devices, such as robotic catheters [1], concentric tube robots [2–8], 3D ultrasound [9], and cardioscopy [10]. Overall, advances in instrumentation and imaging have enabled minimally invasive image-guided surgery even without stopping the beating heart [11]. Without direct vision of the surgical field or direct manipulation of the distal intra-cardiac surgical instrument, however, minimally invasive intra-cardiac surgery challenges the surgeon’s limits of visual recognition, spatial-temporal imagination, and manipulation dexterity.

Robotic instruments comprised of concentric tubes (concentric tube robots or CTRs) have been introduced to minimally invasive beating-heart surgeries and have shown great future potential from animal studies [12]. However, the imaging artifacts and low signal-to-noise-ratio of ultrasound make it a challenge for surgeons to distinguish the surgical instruments from the surrounding tissues. As shown in Fig. 1, it is always difficult for even very experienced surgeons to tell if a robotic instrument is touching the tissue or not when the instrument is introduced into the atrium and approaching the surface. Passive markers for tracking surgical instruments in real-time 3D ultrasound imaging have also been proposed [13]; however, further study is required to realize the full potential of marker-based instrument tracking.

Fig. 1.

Ultrasound image (3D) of intracardiac continuum robot

While marker-free computerized tomography (CT) and magnetic resonance imaging (MRI) can provide acceptable instrument detection, the radiation exposure, operational cost, and the requirement for specially designed CT- or MRI-compatible instruments make them less viable in image-guided robotic procedures. As the fluoroscope brings radiation exposure to the patient and surgical team, real-time ultrasound is the major radiation-free real-time imaging modality for image-guided intracardiac robotic surgeries. Ultrasound-based surgical instrument detection and tracking play increasingly important roles primarily due to ultrasound’s low cost, radiation-free, and portability. Ultrasound imaging is, however, susceptible to high levels of speckle noise and imaging artifacts, which make it hard to identify the instruments from noisy images. The ultrasound imaging artifact patterns arising from metallic surgical instruments have been investigated in our earlier research [14].

Most ultrasound-based instrument tracking systems employ preprocessing steps for noise removal and contrast enhancement to increase the visibility before actually moving on to the detection and tracking steps. Speckle noise is a multiplicative noise and many non-linear order statistic-based filters have been proposed to eliminate the noise while preserving the image details [15, 16].

Despite the various image enhancement approaches [17], surgeons face tremendous challenges in manually identifying the relative spatial coordinates between the instrument and surgical target due to imaging artifacts, and thus have to carefully move the instrument in a trial and error way, as shown in Fig. 1. It is even harder for the robotic instrument to perform semiautomatic manipulations with respect to the target because of the difficulties in tracking instrument and tissue automatically.

Towards ultrasound (US) image guidance for robot-assisted minimally invasive intracardiac beating-heart surgery, this paper investigates a new method based on the power-varying characteristics of ultrasound imaging, for tracking surgical instruments for further navigation in robotic surgeries. The primary contributions of the proposed method are threefold: (1) instrument region identification by analyzing the varying intensity profile during ultrasound transducer power adjustment, (2) estimation of optimal power level for instrument detection: simple image processing methods based on the varying transducer power levels were investigated to estimate an optimal transducer power level for instrument detection, and (3) image overlay of the detected surgical instrument from lower-power-level images on the normal-power-level images in pseudo color to provide visual navigation. The system can run in parallel with the conventional ultrasound system and so provide navigation information without changing the existing protocol while also providing the potential for surgical robotic servoing under ultrasound guidance. In this study, we present the experimental setup and results of investigating instrument detection under varying power levels in ultrasound images. To the best of our knowledge, this is the first report that has utilized the approach of ultrasound power-varying characteristics for instrument detection and navigation especially for intracardiac surgery.

2. Methods

The current image-guided interventional system consists of a tubular robotic end-effector, ultrasound imaging equipment, and a computer workstation with the developed instrument detection and guidance software.

The experiments were conducted in different media including water and ex vivo porcine hearts. Simple medical image analysis including statistical intensity profile analysis is applied to investigate the efficacy of power-varying ultrasound image-based instrument tracking and the results are compared. We investigated the power-varying ultrasound image guidance framework on a commercial real-time ultrasound imaging system, Philips SONOS 7500 and an open-source programmable Ultrasonix imaging system (for more details about these two ultrasound imaging devices, please refer to the Appendix). Without involving complicated image processing, we develop a custom and efficient ultrasound image overlay mechanism under real-time ultrasound system to get tube robotic instrument detection. After acquiring multi-level power-varying ultrasound images of the mock-up robotic procedures (Section 2.2), we characterize the ultrasound images during different transducer power modes, analyze the statistical intensity profile (Section 3.4), and investigate tissue-instrument separability (Section 3.5).

2.1. Surgical instruments

Concentric tube robots [18] are catheter-like instruments which have been used for intra-cardiac surgery in our prior research. These robots consist of multiple telescoping segments of precurved elastic tubes that can translate and rotate with respect to each other. In this way, the pose (position and orientation) of the distal tool tip is remotely controlled. The inner diameter of the distal tubular device is typically smaller than 2 mm and the outer diameter of tubes is smaller than 4 mm for passing through the vascular structures. The hollow lumen of the inner tube serves as working channel to deliver various interventional instruments, such as tissue removal tools [19] or tissue approximation devices [20]. The benefits of using the curved robot includes its curvilinear capability to access intracardiac cavity through jugular vein, exert substantial force at the tool tip, accurate tip positioning control, and delicate intracardiac manipulations as illustrated in [21].

2.2. Imaging at varying power levels

Our approach to varying the power levels of the transmitted ultrasound waves exploits the physics of the ultrasound imaging system. Transducer probes usually consist of an array of piezo-electric elements, which are used to generate the ultrasound waves and also to measure the amplitude of the reflected waves.

The ultrasound imaging system has settings to adjust for visualizing the surgical field, such as frequency, fusion mode, gain, depth, smooth mode, brightness, resolution, and transducer power level, among others. Increasing the transmission frequency changes the resolution in the B-mode post-process image but reduces the penetration depth. Decreasing the transmitted frequency increases the penetration depth but at reduced resolution. Therefore, a high-frequency signal may be used for close-to-surface scans or for organs like the eye and skin. Most of the other adjustments such as smoothing and gain settings are for volume-rendering and visualization effects for the same volumetric data. However, we identified that the adjustment of transducer power level physically changes the acoustic interaction with tissues or instruments and thus, creates a physically different volumetric data set. Furthermore, we found that the contrast between tissue and a surgical instrument varies with power adjustment.

The degree of mismatch between the characteristic acoustic impedance of media determines the intensity of the ultrasound signal that is reflected. The strength of the echoes received by the transducer array from a specular reflection from a given point determines the brightness of that point in the image. Now, if we reduce the ultrasound-transmitted power, the amplitude of the reflected signals is attenuated to a such a degree that most of the tissue looks dark. Metallic instruments reflect the ultrasound waves more effectively than the tissue or other body parts and appear much brighter in these low-power images. Hence, there is higher contrast between the tissue and instruments than at high-power levels.

A set of 16 ultrasound images were acquired from the Ultrasonix machine at different power levels from −15 to 0 dB. This is a simple method to visualize and enhance the metallic instrument by analyzing images from different transducer power levels.

An example of ultrasound image of the pig’s heart taken in power 0 and its respective metal region is shown in Fig. 2. The metal region is marked in red. An example of the changes in varying ultrasound power level in pig’s heart is shown in Fig. 3. It can be observed that artifacts are less obvious as the power level is reduced. All the images are taken using these parameters: frequency of 10 MHz, depth of 5 cm, 65 dB of dynamic range, gain of 55%, and frames per second (FPS) of 12 Hz. These parameters were selected after several experiments in order to identify a good image visually.

Fig. 2.

Ultrasonix ultrasound image of pig’s heart in power 0 (left) and its respective marked metal region (right)

Fig. 3.

Ultrasound images at varying power levels, in a pig’s heart, taken using Ultasonix

2.3. Instrument enhancement and image overlay method

Multiple transducer power levels essentially provide a collection of images for the same target region. Utilizing simple pixel intensity processing among these image sets can effectively reduce the artifacts and enhance the instrument visibility, which thus enhances detection and tracking performance. Based on these observations, the following sections further investigate approaches for instrument enhancement and image overlays based on different power level images.

3. Results

3.1. Instrument enhancement by intensity comparison

As shown in Algorithm 1, by comparing the pixel values of the adjacent transducer power level images, the instrument region is well detected. For each of the adjacent power level, the pixel values are compared with the next for any similarity. If the pixel value is not similar, variable ‘z’ is made as 255 pixel value, i.e., white-colored pixel. Otherwise, ‘z’ is made as 0, i.e., a black pixel. Variable ‘z’ is reshaped to its original image size to construct variable ‘B1.’ Next, MATLAB function bwareaopen is executed for B1 to remove small objects (artifacts) from the image. Bwareaopen removes all connected components (objects) that have fewer than 100 pixels from the binary image B1, producing another binary image, B. The instrument region in the instrument-detected image, B, was detected effectively as shown in Fig. 4. The instrument-detected region in pig heart experiment is shown in Fig. 5. The instrument region was identified without any complex image processing procedures and simple computations were used. This is one of the novel contributions in this paper.

Algorithm 1.

Steps for analyzing different power levels

1:	for Each of the 16 transducer power level images do
2:	Crop the images accordingly;
3:	Put into a vector;
4:	Compare each column of the vector (each image) with the next column to check if there is similarity;
5:	if pixel value is not the same then
6:	‘z’ value is 255 (White);
7:	else ‘z’ value is 0 (Black);
8:	end if
9:	end for
10:	‘z’ is reshaped to variable B1;
11:	MATLAB function bwareaopen executed for B1 to remove small objects;
12:	Instrument detected image, B, displayed;

Fig. 4.

Power level 0-dB image from Ultrasonix water-tank experiments with instrument region marked as red (left). Instrument-detected image (right)

Fig. 5.

Pig heart image from Ultrasonix. Power level 0-dB image with instrument region marked as red (left). Instrument-detected image (right)

3.2. Image overlays with instrument-enhanced image

Based on the aforementioned instrument enhancement approach, we better visualize the instrument by overlaying instrument-enhanced image onto images taken at different power levels. The results of the pseudo colored overlay are shown in Fig. 6. Similarly, it is also shown in the pig heart image in Fig. 7. As can be seen, the resulting overlaid image gives a very good visual differentiating the two images taken at different power levels. This is done so that the instrument possibly occluded at high power levels among tissue can be made visible at low power levels when the tissue almost gets invisible but the metallic instrument is still reflected.

Fig. 6.

Low-power image overlaid onto high-power image in Ultrasonix water-tank experiments. a High-power image (power = 0 dB) with instrument region marked as red region. b Overlaid image

Fig. 7.

Low-power image overlaid onto high-power image in a pig’s heart image from Ultrasonix. a High-power image (power = 0 dB) with instrument region marked as red region. b Overlaid image

Further, based on the separability analysis of power-varying images in Sections 3.3 and 3.5, the approach here can effectively overlay the image taken at optimal power level onto the image taken at high power level in pseudo color.

3.3. Estimating an optimal power level using pixel-level comparison

We discuss a method to estimate an optimal power level to visualize the instrument by analyzing images from different transducer power levels of Ultrasonix scanner, as shown in the Algorithm 2. Power level images from −15 to 0 dB were collected from the Ultrasonix machine. Both stainless steel and nitinol tubes were used for this experiment.

Algorithm 2.

Steps for identifying an estimated optimal power level

1:	Convert each image to binary by Algorithm 1;
2:	for Each of the 16 transducer power level images do
3:	Compare pixel-wise with the instrument detected image (e.g. Fig. 4b or Fig. 5b);
4:	if The pixel value is the same then
5:	counter = counter + 1;
6:	end if
7:	end for

The plot of counter vs transducer power level for both stainless steel and nitinol is shown in Fig. 8. The pixel values of the different power level images were compared against the instrument-detected image (e.g., Fig. 4b or 5b). It was found that at low power level, the counter goes to a minimum stable value, showing that the power level is most similar to the detected image. This is shown in Fig. 8a, b, c. The minimum stable value of counter is at a power level of −6, −14, and −11 dB, for stainless steel in plane (experiment 1), for stainless steel in plane (experiment 2), and for nitinol in plane, respectively. Ultrasound images for the respective low-power images for both stainless steel and nitinol are shown in Fig. 9. We can view the at the minimum stable value of the curve, the optimal operating point or optimal power level is found approximately from 2/5 to 3/4 of the rated lowest ultrasound operating power level of Ultrasonix scanner and it is applicable for metallic instrument detection.

Fig. 8.

Counter vs transducer power level for Ultrasonix water-tank images

Fig. 9.

Ultrasound images at optimal low power levels for Ultrasonix water-tank images

3.4. Statistical analysis of power-varying ultrasound images

To investigate the intensity statistical profile and separability with power-varying overlay approaches, image analysis is performed in the following sections for the ultrasound images during power-level adjustment.

Due to the different acoustic impedances between tissue and instruments in the experiment environments, their imaging responses to the transducer power change are significantly different. For the normal imaging power setting, by default 0 dB, both tissue and instrument show up but severely blurred by the artifacts mainly caused by the metallic instrument. Reducing the transducer power level reduces the artifacts and increases the contrast between the tissue and instrument. This power level continues decreasing until a threshold value, where only the instrument is visible and all the surrounding tissue disappears, since only the high-impedance instrument returns acoustic echo back to the transducer.

In order to get insight on how the contrast between the tissue and instrument is changing during power level adjustment, we studied the intensity profile of several targets in the field of view, termed VOIs (volume-of-interests, in short, V) hereafter, as shown in Fig. 10. To determine a simple means of extracting the instrument, we investigated many statistical measures including the maximum intensity value, max(V); mean intensity value, mean(V); median intensity value, median(V); and intensity histogram, hist(V) of specific VOIs.

Fig. 10.

VOIs (volumes of-interest, V) of different target regions for SONOS7500 images. The rectangular frames with different colors stand for distinct targets or blended targets in the field of view. Blue: entire volume; pink: heart tissue and instrument (CTR); light blue: heart tissue alone; yellow: CTR instrument alone

A quantitative illustration is plotted in Fig. 11 to show the intensity profile change versus the transducer power level, including the comparative maximum, mean, and median intensity of specific VOIs that are manually identified. The image volumes were acquired with transducer power level increments of 0.3 dB from SONOS7500.

Fig. 11.

Statistical intensity measures (maximum: green circle line, mean: red cross line, median: black dash line) of selected target VOIs

In our experiments, it was determined that at a threshold power level (Pt) of −23.1 dB (about 3/4 of the rated lowest ultrasound operating power level, −30 dB) for the SONOS echo machine, the tissue in the image will fade out, and only the high-impedance instrument shows up due to its stronger acoustic echo back to the transducer. The experiments imply that the instrument location in the image can be distinguished from surrounding tissues by tuning the power up and down around the threshold power (Pt).

By comparing the maximum intensity of tissue-only VOI with the other three subfigures in Fig. 11, we can see the tissue’s maximum intensity is dropping down gradually until Pt, but the maximum intensity of the other three subfigures is always saturated because of the existence of high-acoustic-impedance instrument. The intensity saturation at power levels higher than 0 dB produces more imaging artifacts and makes it difficult to distinguish the instrument from the surrounding tissue, because both tissue and instrument have saturation effects at this level.

Another statistical measure, the volume intensity distribution or intensity histogram is further evaluated for the VOIs comprising only the CTR and the combination of CTR and tissue, as shown in Fig. 12. The intensity histogram provides the occurrence frequency of specific intensity values or ranges at a given power level. The histograms of the CTR alone demonstrate that the fraction of saturated voxels decreases with decreasing transducer power level. In the histograms of mixed CTR and tissue, the overall distribution shifts left with decreasing the transducer power in correspondence with the decreasing mean(V) and median(V) in Fig. 12. As long as transducer power is greater than Pt, decreasing it from 0 dB improves contrast between instrument and tissue as indicated by the clearer valley in the histograms. When the transducer power is less than Pt, only the instrument will show up in the image.

Fig. 12.

Voxel intensity distributions for five transducer power levels. The units for each histogram are intensity bins and frequency for x and y axes, respectively

3.5. Separability evaluation

Given the qualitative capability to distinguish robot instruments and tissues via intensity histogram, we seek a quantitative approach to selecting the optimal power level based on a separability metric between class distributions. The class separability evaluation problem is to separate the instrument voxel class from the surrounding tissue class. We have evaluated several widely studied class separability measures to determine optimal transducer power settings for the purpose of instrument-tissue identification, as there is no such a commonly accepted best measure yet [22].

For completeness, we plot the four measures for both working modes as follows, but only the curves in normal working mode are comparable. As the tissue fades out when P ≤ Pt, the cropped tissue (T) voxels turn out to be background of the image. Therefore, the class separability measures exhibit discontinuities profile over the entire power range, which implies two working modes: (1) normal working mode at a power level, p > Pt, with both instrument and tissue visible in the ultrasound scan; and (2) instrument-enhanced mode at a power level, p ≤ Pt.

3.5.1. Histogram-based class separability measure

The class separability can be directly evaluated from the intensity histograms of different objects [23], with the assumption that the probability distribution of each class is known priori. The objective here is to find the optimal power level to minimize the mis-classification between two object classes: robotic instrument, R, and tissue, T. The probability distributions of class R or T can be estimated as shown in the previous sections.

Let E1 denote the probability of an element from class R misclassified to class T, and E2 as an element from class T misclassified to class R. As only a limited number of discrete power levels is physically available from the ultrasound machine, we can use a brute-force search method to find the optimal power setting. As a smaller mis-classification probability implies better separability within the normal working mode, class separability evaluation can be performed using the Algorithm 3 below.

Algorithm 3.

Evaluation of histogram-based class separability measure

1:	for each power level p do
2:	Compute the histograms (Hist(R), Hist(T)) from the respective data samples R, T;
3:	for each bin i in histogram Hist(R) do
4:	if Hist(R, i) < Hist(T, i) then
5:	E1 = E1 + Hist(R, i);
6:	end if
7:	end for
8:	for each bin j in histogram Hist(T), do
9:	if Hist(T, j) < Hist(R, j) then
10:	E2 = E2 + Hist(T, j);
11:	end if
12:	end for
13:	Normalize E1 and E2 to get E where $\mathit{E}=\frac{({\mathit{E}}_1+{\mathit{E}}_2)}{2}$.
14:	end for

Figure 13 plots the normalized classification errors against available transducer power levels. In the normal working mode, the value of E reduces to a stable minimum value for power levels in the range of −7.5 to −23 dB. By comparing with the maximum intensity plot of tissue VOI in Fig. 11, we can see that for power levels lower than −7.5 dB, the images are not subject to saturation, which implies that tissue image saturation is a negative factor to separability.

Fig. 13.

Histogram-based class separability measure for SONOS7500 images

3.5.2. Fisher’s discriminant ratio for separability measure

In pattern classification, Fisher linear discriminant analysis [24] is another technique for optimal discrimination between two classes, i.e., R and T. The degree of discrimination is evaluated by the Fisher’s discriminant ratio (FDR):

$$\mathrm{FDR}={({\mu }_{\mathrm{R}}-{\mu }_{\mathrm{T}})}^2∕[{\sigma }_{\mathrm{R}}^2+{\sigma }_{\mathrm{T}}^2],$$ (1)

where μ_R and μ_T denote the means of classes R and T respectively, and σ_R and σ_T denote their variances. Similar to the histogram-based separability evaluation, we evaluate the FDR for each power level by Algorithm 4 and the resulting values are plotted in Fig. 14. Within the normal working mode, a larger FDR denotes higher separability indicating that the lowest power level at which the tissue remains visible is best.

Algorithm 4.

Evaluation of FDR-based separability

1:	for each power level p do
2:	Get the data samples R and T;
3:	Calculate μR, μT, σR and σT;
4:	Calculate FDR by Eq. 1;
5:	end for

Fig. 14.

FDR versus power level as a class separability measure for SONOS7500 images

3.5.3. Kullback-Leibler divergence

The Kullback-Leibler divergence (KLD) [25] is another measure widely used in information theory to quantify the separability of two probability distributions. It is closely related to mutual information, a similarity measure over different class domains. Assuming the investigated classes R and T are statistically independent Gaussian distributions, in shorthand denoted by N(μ_R, σ_R2) and N(μ_T,${\sigma }_{\mathrm{T}}^2$), the KLD of images at specific transducer power p, is given by

$$\mathrm{KLD}(\rho )=1∕2({\sigma }_{\mathrm{R}}^2∕{\sigma }_{\mathrm{T}}^2+{\sigma }_{\mathrm{T}}^2∕{\sigma }_{\mathrm{R}}^2-2)+1∕2{({\mu }_{\mathrm{R}}-{\mu }_{\mathrm{T}})}^2(1∕{\sigma }_{\mathrm{T}}^2+1∕{\sigma }_{\mathrm{R}}^2).$$ (2)

The Kullback-Leibler divergence values are plotted in Fig. 15 for the available power settings based on the estimated probability distributions of R and T. Since a larger divergence value stands for higher separability, this measure also indicates that the lowest power level at which the tissue remains visible is best for discrimination of metallic surgical instruments from the surrounding tissue and fluid environment.

Fig. 15.

Kullback-Leibler divergence as a class separability measure

3.5.4. Bhattacharyya distance

Bhattacharyya distance (BD) [22, 26] is another measure in statistics to quantify the similarity as well as separability of two probability distributions. It has been used as a contrast parameter for statistical processing of noisy optical images and for image segmentation. Assuming the investigated two classes R and T are independent Gaussian distributions over the same domain, similar to the assumptions in the former sections, the Bhattacharyya distance (BD) measure is given by

$$\mathrm{BD}=1∕8{({\mu }_{\mathrm{R}}-{\mu }_{\mathrm{T}})}^2∕[{\sigma }_{\mathrm{R}}^2+{\sigma }_{\mathrm{T}}^2]+1∕2\mathit{ln}[({\sigma }_{\mathrm{R}}^2+{\sigma }_{\mathrm{T}}^2)∕2{\sigma }_{\mathrm{R}}{\sigma }_{\mathrm{T}}].$$ (3)

The Bhattacharyya distance values for all power settings are shown in Fig. 16, where within the normal working mode, larger values indicate higher separability. Thus, this measure also indicates that minimizing power level within the normal working-mode range optimizes discrimination capability to separate metallic surgical instruments from the surrounding tissue and fluid environment, when both classes have nonzero intensity distributions.

Fig. 16.

Bhattacharyya distance as a class separability measure for SONOS7500 images

4. Discussion

Optimal working power levels are identified from the above numerical analysis of the ultrasound imaging characteristics. By incorporating multi-power level imaging, the image guidance system shows its potential for clinical usage, while employing only simple image processing at the pixel intensity level and parallel image streaming, and providing a navigation capability.

Equations 1, 2, and 3 show that the separability measures not only depend on the difference of between-class mean values but also are related to individual class distribution variance. Each of the four measures are information-theoretic metrics to study the distance between probability distributions, and they are computationally simple and efficient for the classes considered here.

Within normal working mode, the figures of the four separability measures (Figs. 13, 14, 15, and 16) share a common trend that reducing transducer power level from the default 0 dB will increase the separability from the perspective of pattern recognition. From the histogram-based separability (Fig. 14), intensity saturation is a key negative factor for class separation and can be avoid by reducing the power lower than −7.5 dB for SONOS7500 scanner. Further reduction in transducer power level ranging from 2/5 to 3/4 of the rated lowest ultrasound operating power levels as suggested by the other three measures can increase the separability from an image processing perspective. Note, however, that this further reduction does not imply that the resulting image is optimal from the perspective of the surgeon.

Based on the above observations, we can utilize different power levels for the intended goals and combine them to create an enhanced image. Normal operation power level is typically set at 0 dB for visualizing both tissue and instrument, but subject to imaging artifacts. Lower power level for enhancing metallic instruments with high acoustic response.

The research here is motivated by ultrasound (US) image-guided robot-assisted minimally invasive intracardiac beating-heart surgery, where effective instrument detection and tracking under ultrasound are significant and challenging. Moving forward, the proposed methodology will be validated from ex vivo experiment to the in vivo experiment, from animal trials to pre-clinical human trials. In the long run, the proposed multi-power image overlay approach can provide navigation with intracardiac concentric tube robot instrument detection capability, which is of great significance for surgeons to determine the relative position between the robotic manipulator and surrounding tissues under intraoperative ultrasound guidance. As robotic instruments are typically programmable and predictable by kinematic models, prior robotic motion knowledge can be further incorporated for robust instrument detection, which would be the future work and an essential step for image-based robotic servoing.

5. Conclusion

Aiming at the challenging problem of navigating robotic instruments using ultrasound images for beating-heart surgery, we proposed a novel, effective, and simple framework of instrument detection by transducer power adjustment. We identify the characteristics of acquired images under different working power levels and propose a standalone parallel instrument detection system. In order to determine the optimal transducer power settings for contrast enhancement and instrument detection, a number of class separability measures are evaluated, including Fisher’s ratio, Kullback-Leibler divergence, Bhattacharyya distance, histogram-based separability, and pixel-level comparisons.

The novel contributions in this paper are originated from expanding the ultrasound imaging spectrum by tuning the transducer power levels and specifically can be summarized as follows: (1) simple computation and image processing is used by varying transducer power level to identify the instrument region, (2) low-power ultrasound image was overlaid on the high-power ultrasound image in pseudo color to identify the instrument region visually, and (3) simple image processing methods used by varying transducer power level to estimate an optimal transducer power level.

Biographies

Hongliang Ren is currently an assistant professor and leading a research group on medical mechatronics in the Biomedical Engineering Department of National University of Singapore (NUS). He is an affiliated Principal Investigator for the Singapore Institute of Neurotechnology (SINAPSE) and Advanced Robotics Center at National University of Singapore. His main areas of interest include Biomedical Mechatronics, Computer-Integrated Surgery, and Robotics in Medicine.

Banani Anuraj has been a research engineer in the National University of Singapore. She has graduated with Masters Degree from the National University of Singapore in December 2015 and Bachelors Degree (Hons.) from the National Technological University in June 2011. Her research interest includes image processing, telecommunications, signal processing, and information security.

Pierre E. Dupont received the B.S., M.S., and Ph.D. degrees in mechanical engineering from Rensselaer Polytechnic Institute, Troy, NY, in 1982, 1984, and 1988, respectively. He is currently the Chief of Pediatric Cardiac Bioengineering and holder of the Edward P. Marram Chair at Boston Children’s Hospital, Harvard Medical School.

Appendix

Ultrasound imaging setups

We utilized both a clinical ultrasound imaging system, the Philips SONOS 7500 at Boston Children’s Hospital, and a lab-based ultrasound imaging system, Ultrasonix Sonix Touch, at the National University of Singapore, for cross-validating the consistency of power-varying experiments at both research labs. Both transducers allow users to adjust the transducer power levels either by graphical user interface or programming.

Philips SONOS 7500 ultrasound platform

The real-time interventional images were acquired through a Philips SONOS 7500 (www.philips.com) ultrasound imaging system. In addition to the imaging user interface of the SONOS system, a parallel stream of the same images was transferred to a custom workstation through the SONOS research interface. The image acquisition rate is about 28 volumes per second and varies slightly with the frequency settings. In the experiments of this article, standard settings of the image acquisition parameters were used, including 50% overall gain, 50% compression rate, frequency fusion mode 2, and high-density scan line spacing. The acquired gray-level ultrasonic volume image, V, can be defined as an M × N × P matrix. The individual voxel v(i, j, k) typically has an anisotropic spacing and represents the voxel intensity at the ith row, jth column, and kth slice, in the image volume space, which corresponds to the Cartesian coordinate, x = (x, y, z)^T, of the ultrasound transducer system, in the physical spatial space. Here, x represents increasing azimuth, y represents increasing elevation, and z indicates increasing distance from the transducer. The resulting anisotropic spacing varies with acquisition depth settings, for example, 10-cm acquisition depth will produce an anisotropic volume with spacing {0.542 mm, 0.706 mm, 0.451 mm} in the x, y, and z directions, respectively.

The transducer power level of SONOS 7500 can be varied from −30 to 0 dB, in which 0 dB is a default level for visualizing both tissue and instruments. As shown in Fig. 17, we performed comparative imaging experiments for two different intra-cardiac instruments: the concentric tube robot end-effector, fabricated from NiTi, and braid-reinforced polyimide catheter tubing manufactured by Microlumen (Microlumen Inc.). For qualitative analysis, the instruments were imaged inside an ex vivo pig heart submerged in the water tank, and the images were recorded using different transducer power levels.

Fig. 17.

Comparison of acquired images from SONOS7500 at normal 0 dB power and at instrument-enhancing −23.1 dB in an ex vivo heart. a Imaging of a concentric tube robot; b imaging of a Microlumen catheter tube; c photographs of instruments used in panels a and b. Left: concentric tube robot; middle: microlumen catheter; right: experiment setup

Ultrasound experimental setup

The experimental setup is as shown in Fig. 18 for scanning a tube in a water tank and pig heart tissue environment.

Fig. 18.

Experimental setup using Ultrasonix machine for scanning (left) the water tank and (right) pig heart tissue