Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Apr 18.
Published in final edited form as: Proc Int Conf Adv Robot. 2015 Jul;2015:53–59. doi: 10.1109/ICAR.2015.7251433

System Integration and Preliminary In-Vivo Experiments of a Robot for Ultrasound Guidance and Monitoring during Radiotherapy

H Tutkun Şen *, Muyinatu A Lediju Bell *, Yin Zhang †, Kai Ding †, John Wong †, Iulian Iordachita ‡, Peter Kazanzides *
PMCID: PMC4834973  NIHMSID: NIHMS751668  PMID: 27099871

Abstract

We are developing a cooperatively-controlled robot system in which a clinician and robot share control of a 3D ultrasound (US) probe. The goals of the system are to provide guidance for patient setup and real-time target monitoring during fractionated radiotherapy. Currently, there is limited use of realtime US image feedback during radiotherapy for lower abdominal organs and it has not yet been clinically applied for upper abdominal organs. One challenge is that placing an US probe on the patient produces tissue deformation around the target organ, leading to displacement of the target. Our solution is to perform treatment planning on the deformed organ and then to reproduce this deformation during radiotherapy. We therefore introduce a robot system to hold the US probe on the patient. In order to create a consistent deformation, the system records the robot position, contact force, and reference US image during simulation and then introduces virtual constraints (soft virtual fixtures) to guide the clinician to correctly place the probe during the fractionated treatments. Because the robot is under-actuated (5 motorized and 6 passive degrees-of-freedom), the guidance also involves a graphical user interface (adjustment GUI) to achieve the desired probe orientation. This paper presents the integrated system, a proposed clinical workflow, the results of an initial in-vivo canine study with a 3-DOF robot, and the results of phantom experiments with an improved 5-DOF robotic system. The results suggest that the guidance may enable the clinician to more consistently and accurately place the US probe.

I. Introduction

Radiation therapy is commonly used as a treatment option for cancer. The goal is to direct sufficient radiation to kill the tumor cells, without harming the healthy surrounding tissue. This treatment is usually fractionated; that is, the patient receives multiple radiation treatments, typically over several days. We consider image-guided radiation therapy (IGRT), which involves two main steps: (1) planning/simulation and (2) treatment delivery. Planning is performed using a 3D CT image (sometimes together with MRI) to identify the target, arrange radiation beams to optimize target coverage and spare normal tissue, and compute the resulting dose. In the simulation phase, the patient is placed in a big-bore CT scanner to obtain the image that will be used for planning. The simulation CT image also guides patient setup for subsequent radiation treatments.

The radiation treatment phase is performed with a linear accelerator (LINAC); modern LINACs include on-board cone beam CT (CBCT) imaging to show the bony anatomy of the patient in the treatment room frame of reference. Nevertheless, two major deficiencies have become apparent when CBCT is applied to verify radiotherapy: (1) CBCT only provides a snapshot of patient information at the time of imaging, but not during actual radiation delivery, and (2) CBCT often does not provide sufficient contrast to discriminate soft tissue targets. US imaging can overcome these deficiencies as in [1]–[3], but the ionizing radiation precludes an ultrasonographer from holding the US probe on the patient during treatment. This has motivated the development of passive probe holders [4] and at least one telerobotic system [5], [6] for US monitoring during radiotherapy. Robotic systems for ultrasonography have also been developed for other applications [7]–[15]. Because US imaging requires contact between the probe and patient, all of these robotic systems include a force sensor for monitoring and/or controlling the contact force. Our system differs from these prior works by focusing on using the robot to achieve reproducible probe placement, and therefore reproducible soft-tissue deformation, with respect to the target organ.

Our goal in this study is to construct a robotically-controlled, integrated 3D x-ray and US imaging system to guide radiation treatment of soft-tissue targets in the abdomen. The first requirement for our robotic system is to enable an expert ultrasonographer to place an US probe during simulation, record the relevant information (e.g., position, force, and reference US image). Then, the ultrasonographer substitutes a model probe [3] and acquires a planning CT (the model probe does not contain metal components and therefore does not cause CT artifacts). The second requirement is to enable an inexperienced user to reproduce this placement (and tissue deformation) during the subsequent fractionated radiotherapy sessions. We do not attempt to move the robot autonomously, but rather employ a cooperative control strategy, where the robot shares control of the US probe with the human operator. We previously presented a cooperative control strategy, using virtual springs, and demonstrated its utility in phantom experiments considering only the 3 translational degrees-of-freedom (DOF) [16].

The novelty of this study, however, is the inclusion of the US imaging modality in the radiation treatment process to assist with patient setup and to monitor the beam delivery. In our previous in-vivo experiments [3], we acquired a 3D US image (reference US image) on the planning day and recorded the position of the US probe (goal position) at this instant. We then reproduced this probe position to obtain a real-time US image to assist with patient setup. This paper extends the prior work by incorporating a novel US probe placement procedure that minimizes the discrepancy in soft tissue deformation between simulation and treatment. This procedure, cooperative control with virtual springs, was first demonstrated in phantom experiments with a 3-DOF robot [16] and is now utilized for in-vivo experiments with the 3-DOF robot. Additionally, two active rotary joints are added to the 3-DOF robot and the orientation adjustment GUI is developed to guide the clinician to correctly set all 3 orientation DOF. With the new 5-DOF robot, two phantom experiments are performed.

II. Materials and Methods

A. Radiation Therapy Environment

The planning/simulation (SIM) and treatment environments are both represented by Fig. 1, with the primary difference being that the SIM room contains a large-bore CT scanner rather than a LINAC. We assume that each room contains an optical tracking system, attached to the ceiling of the room, that is calibrated to provide a common world reference frame. In our case, this is provided by the Clarity System (Elekta AB, Stockholm, Sweden).

Fig. 1.

Fig. 1

Open in a new tab

Radiation therapy environment: SIM room (with CT scanner) and LINAC room; each contains an optical tracking system that is typically calibrated to place the isocenter at the origin.

The robotic manipulator that holds the US probe is attached to the table via a bridge, as shown in Figs. 1 and 2. The bridge contains passive linear axes and can also slide on the couch rails to provide coarse positioning of the robot base. Once the approximate position is reached, the bridge is locked to the couch. The robotic manipulator is attached to the bridge and consists of three parts: a Base Robot (3-axis active linear stages), a 6 DOF passive arm, and a Tip Robot (2 DOF active rotary stages), as shown in Figs. 2 and 3. As illustrated in Fig. 1, optical markers (reference frames) are placed on the Base Robot and Tip Robot; these enable the optical tracking system to measure the pose of the unencoded passive arm and to provide direct end-point feedback of the US probe.

Fig. 2. Robotic System for radiation therapy.

Fig. 2

Open in a new tab

Fig. 3. 2 DOF rotary stages.

Fig. 3

Open in a new tab

B. System Transformations

In both rooms (SIM and LINAC), there are 6 basic coordinate frames: FR (room), Fc (camera), Fb (robot base marker), Ft (robot tip marker), Fu (US image) and F0 (robot origin) which is defined as the world coordinate frame on which the 3 DOF linear stage (Robot Base) positions are measured. There are several transformations that convert data between these coordinate frames. A comprehensive review of the transformations is given in [17]. The tracking camera directly measures the transformations between Fb and Fc and between Ft and Fc. The Clarity System includes calibration procedures that establish the transformation between the camera coordinates, Fc, and the room coordinates, FR, as well as the transformation between the US image coordinates, Fu, and the marker frame attached to the US probe (robot tip marker), Ft. The transformation (rotation) between robot coordinates, F0, and camera coordinates, Fc, is determined via a simple calibration procedure. Specifically, the robot is first moved along its x axis, and then along its y axis, while the camera measures the robot base marker frame, Fb, with respect to the camera frame, Fc. This produces two direction vectors that are normalized and orthogonalized to form the first two columns of the rotation matrix between the camera and the robot coordinate frames. The third column vector is found as the cross product between the first and second column vectors. The translation component is not required so it is arbitrarily set to zero.

C. Proposed Workflow

In the treatment planning phase of the traditional radiotherapy process, the patient is first positioned on the SIM room couch. Then, depending on the organ to be scanned, either some markers are attached on the patient body or a mask which conforms to the shape of the patient body area is prepared. Now that the target area is coarsely immobilized, the planning CT scan of the target organ is obtained.

In the treatment delivery phase, the goal is to deliver radiation beams to the target according to the plan created during the planning phase. This process is accomplished in the LINAC room shown in Fig. 1. In this phase, the patient is initially positioned on the couch either based on the markers placed during simulation or based on the patient-specific mask generated before the CT scan was taken. Then, a CBCT scan is acquired and the image is compared to the simulation CT image based on bony anatomy. Via this comparison, the couch is shifted in the x, y and/or z directions to align the CT image with the CBCT image. The delivery day can be a single day or multiple days. The most crucial part of this procedure is the patient setup on the treatment couch which should ideally be identical between the treatment planning and the treatment delivery days.

We have two main goals in the US-guided radiation therapy process: 1) during delivery days, the tumor should be located at the LINAC isocenter (intersection point of the radiation beams), and 2) between the simulation and radiation delivery days the US probe should be located at the same position relative to the target, thereby producing consistent soft tissue deformation and reproducible US images. To accomplish these, as seen in Fig. 4, the user first brings the US probe near the target location with unconstrained cooperative control, following by cooperative control with virtual springs between the US probe and the target goal position and orientation [16]. The virtual springs pull the US probe, which is attached to the robot and held by the user, toward the simulation day goal while giving the user the flexibility to override the goal position and orientation. There are three main reasons why the user may need this flexibility: 1) possible patient setup errors between treatment simulation and treatment delivery days, 2) possible changes to the patient's anatomy (e.g., due to gas in the abdominal area or weight loss), and 3) discrepancies between the calibration of the room coordinate system, which is supposed to accurately identify the treatment isocenter in both rooms. If the robot forces the user to return to exactly the same position recorded during simulation (e.g., by using a “hard virtual fixture”), none of the above discrepancies between treatment phases would be addressed.

Fig. 4.

Fig. 4

Open in a new tab

Proposed workflow for US probe placement on the treatment day to help patient setup and treatment monitoring. Purple boxes represent functions not yet implemented.

Our current implementation relies on the user to visually compare the real-time US image to the reference US image recorded during simulation, and then override the preferred position indicated by the virtual springs. Fig. 4 also shows our future work (dashed boxes), which is to use US image feedback to adjust the virtual springs so that they guide the user to the correct probe placement. This would enable clinicians with limited ultrasound experience to more accurately place the probe. We have previously shown, in phantom experiments, that the virtual fixtures help the user to find the target more accurately and faster compared to free hand US probe placement [16].

D. Orientation Adjustment

As noted in Section II-C, on the treatment day the US probe should be placed in the same position and orientation as in the planning day. However, the proposed system is underactuated (i.e., it has only two motorized rotary joints) and the passive arm is the only way to set the third rotation angle. To help the user to correctly orient the passive arm, a GUI has been designed (Fig. 5). The Adjustment GUI displays the reference orientation (i.e. the SIM day orientation) and the current orientation of the US probe as graphical icons and as Euler angles (ZYX convention). It also displays the difference between the current and reference Euler angles. The camera measures the orientation by tracking the robot tip marker, Ft, but to make the adjustment more intuitive, the Euler angles are expressed with respect to the robot coordinate system, F0. This requires an additional calibration procedure, where the first rotary joint is moved to three different positions and its axis of rotation determined by the normal to the plane defined by these three points (the points are defined by the origin of the robot tip marker, which is tracked by the camera). This is repeated for the second rotary joint, to get a second unit vector. These two unit vectors are orthogonalized, and used to define the end-effector rotation with respect to the robot tip frame, Ft.

Fig. 5. Adjustment GUI to help the user to orient the passive arm.

Fig. 5

Open in a new tab

Thus, the user's goal is to set the leftmost Euler angle difference as close to zero as possible, because it corresponds to the unmotorized rotation angle. The user is not required to accurately set the other two Euler angles because the motorized axes will zero out the discrepancy. We have found, however, that there is some coupling between the Euler angles (i.e., moving one DOF can cause small angle changes in the other two rotary DOF), probably due to inaccuracies in determining the axes of rotation as well as the possibility that the mechanical axes of rotation are not perfectly orthogonal. Thus, in practice, we attempt to set all angle differences as close to zero as possible, to minimize this effect. Section III presents experimental results that indicate that we can generally adjust the system to bring the current orientation within 1.5 degrees of the reference orientation.

III. Experiments and Results

We previously evaluated the use of the 3-DOF robot with virtual fixtures to recreate soft tissue deformation on a phantom [16]. This paper first presents the results of an in-vivo canine experiment using this same 3-DOF robot. Based on the results of this canine experiment, we upgraded our system to a 5-DOF robot by including a 2 DOF rotary stage, as described in Section II. We then performed phantom experiments to test the performance of the 5-DOF robot, where the virtual springs now include 2 DOF torsional springs and an adjustment GUI to help set the US probe orientation by adjusting the passive arm. The following subsections present the in-vivo experiments with the 3-DOF robot and the phantom experiments with the 5-DOF robot.

A. In-Vivo Experiments with 3-DOF Robot

For this experiment (see Fig. 6), we implanted metal markers that are clearly distinguishable in CT and US images into several organs of a dog (e.g., prostate, liver, kidney, and pancreas); these markers serve as reference fiducials for subsequent measurements.

Fig. 6. The experiment setup with the canine. On the left is the 3 DOF robot scanning the canine prostate with the 3D US probe.

Fig. 6

Open in a new tab

As mentioned in section II-C, the experimental procedure consists of two phases: treatment planning/simulation and treatment delivery. In the experiment, we followed the radiotherapy workflow proposed in [3] and in section II-C:

  1. In the SIM room, the anesthetized dog is placed in a supine position on a special cradle that conforms to the shape of the dog.

  2. The cradle and dog are marked with non-erasable markers that are aligned with the SIM room lasers.

  3. The ultrasonographer uses the robot in free hand cooperative control mode to find the markers in the organ and saves the 3D US image as the reference US image.

  4. The US probe position (i.e., the goal position) in the room coordinates frame and the reaction force along the probe axis are recorded.

  5. The US probe is replaced with the model probe and the model probe is brought back to the goal position.

  6. A CT scan of the dog, with the model probe in place, is acquired.

Now that the CT scan is obtained, the marker positions in CT coordinates as well as in room coordinates are obtained. In the next phase of the experiment, the dog is brought to the LINAC room for the radiation therapy delivery process. The goal of this phase of the experiment is to determine how well we can reproduce the reference 3D US image, which indicates how well we can produce the same soft tissue deformations as in treatment planning/simulation. For this reason, it was not necessary to actually deliver radiation during this experiment. The procedure in the LINAC room is as follows:

  1. The anesthetized dog is placed in a supine position on the same cradle that conforms to its shape.

  2. The cradle and the dog skin markers are aligned with the LINAC room lasers.

  3. A CBCT scan of the dog is acquired with the probe in place. Based on the bony anatomy, the CBCT image is compared with the SIM CT to calculate the couch shift required to compensate for system setup errors.

  4. After the couch is shifted, another CBCT scan of the dog is acquired to confirm the alignment.

  5. Using the robot in cooperative control mode with hard virtual fixtures, the US probe is brought to the goal position defined in the room coordinate frame (the robot does not allow the user to deviate from the recorded goal position).

  6. The hard virtual fixtures are changed to soft virtual fixtures (virtual springs) and the user moves the US probe until the acquired image matches the reference US image recorded during simulation, based on the user's subjective visual assessment. For practical reasons, even though a 3D US probe was used, the operator performed the visual assessment with 2D images.

Fig. 7 shows the contours of the dog prostate extracted from 3D US image slices. The red contours represent the prostate cross-sections on the treatment planning day, the green contours represent the prostate image taken on the treatment delivery day with hard virtual fixtures (i.e., when the US probe is placed at exactly the same room coordinates as the simulation) and the blue contours represent the prostate contours when the user relocates the US probe with the help of the soft virtual springs. To remove the effect of possible transformation errors between the various coordinate frames discussed in section II-B, the results in Fig. 7 were analysed in the US image coordinate system where the axial dimension is normal to the probe face. We assess the accuracy with which the US probe was positioned during treatment by computing the percentage of overlap between the prostate contours from the reference US image and the treatment day prostate contours obtained with both hard virtual fixtures and soft virtual fixtures, using the following Eq. 1:

Fig. 7.

Fig. 7

Open in a new tab

Contours of the prostate (in pixels) based on acquired ultrasound images from the planning day (red), treatment day with hard virtual fixtures (green), and treatment day with soft virtual fixtures (blue). The hard virtual fixtures produced an overlap of 38%, whereas the soft virtual fixtures produced an overlap of 57%.

%Overlapp=100(∑i=1N|Ai∩Bi|/∑i=1N|Ai|) (1)

where N is the number of slices, Ai and Bi correspond to prostate cross-section areas of the ith reference US image slice and treatment day US image slice, respectively. The hard and soft virtual fixtures produced overlaps of 38% and 57%, respectively. Although neither is near the ideal value of 100%, the soft virtual fixture enabled the user to better compensate for the various sources of error.

B. Accuracy of Setting Orientation of 5-DOF Robot

The first phantom experiment checks how accurately the passive arm can be set to a desired orientation using the GUI described in Section II-D. In this experiment, three different reference orientations are recorded and the user is asked to reproduce these orientations. For each reference orientation, four trials are performed, where the user starts from a random initial position and reconfigures the passive arm using the Adjustment GUI. Then, the two active rotary DOF are used to zero the error in those directions. The final orientation errors (for each individual Euler angle and the total angle), as measured by the optical tracker, are shown in Fig. 8. The results indicate that the probe orientation can be set within 1.4 degrees of the desired orientation.

Fig. 8. Robot orientation error measured by optical tracker, μ and σ are the mean and the standard deviation, respectively.

Fig. 8

Open in a new tab

C. Phantom Experiments with 5-DOF Robot

For this experiment, we used the Polaris optical tracker, but substituted a Micron Hx40 optical tracker (Claron Technologies, Toronto, Canada) for the US probe, as shown in Fig. 9. This provides two advantages: (1) it is more convenient to use camera images instead of US images, and (2) by attaching tracking markers (checkerboard patterns) to the phantom (Fig. 10), the Micron tracker can provide a direct “ground truth” measurement of the robot end-effector position with respect to the phantom, which would not be possible with an US probe.

Fig. 9.

Fig. 9

Open in a new tab

Setup for Phantom Experiments with 5-DOF Robot; Micron Tracker used instead of US probe.

Fig. 10.

Fig. 10

Open in a new tab

Camera images (substitute for US images), showing the reference image (left) and the image after the phantom is translated and rotated in 3D (right)

This experiment focuses on the position and orientation error compensation capabilities of the system when there is a mismatch between the reference image and the image obtained on the treatment day (i.e., to simulate an inter-fraction setup error). As introduced in [16], virtual springs are used to compensate for the 3D position and 2D orientation error. In [16], it was assumed that the target organ had undergone only a translational displacement; however, this study evaluates the ability to compensate for a 6 DOF error. This tests both the Adjustment GUI and the use of the torsional virtual springs around the 2 DOF rotation stage (as well as the linear springs, which were already tested in [16]). Similar to the radiation therapy procedure explained in Section II-C, we initially record the reference image of the phantom (Fig. 10-left). Then, we disturb the position and orientation of the phantom (to emulate patient setup error, Fig. 10-right) and ask the user to move the robot system, first using the Adjustment GUI to set the passive arm and then using cooperative control with soft virtual fixtures (virtual springs), until the current image is as close as possible to the reference image. As described in [16], the linear springs are engaged when the user crosses the plane that passes through the recorded position, at which point they guide the user towards this position. In this case, however, the recorded position is no longer accurate due to the applied displacements. Thus, it is necessary for the user to apply force to counteract the virtual springs and thereby deviate from this position in order to obtain the desired image. Once all the linear virtual springs are activated, depending on the need, the torsional springs may also be activated.

For the first five trials, the phantom is disturbed in the MicronTracker x-y plane (the plane facing the MicronTracker) and for the last five trials, it is disturbed in all 6 DOF. To better visualize the orientation, additional features were attached to the phantom, as shown in Fig. 10. The phantom position and orientation tracked by the MicronTracker are used to evaluate the performance of the system. During the experiment, two different data sets are recorded. The first data set is the position and orientation error of the phantom, based on MicronTracker measurements, before the user attempts to counteract the virtual springs (i.e., it essentially measures the applied displacement, which corresponds to the emulated patient setup error). These results are shown in the left columns of Tables I and II. The right columns of Tables I and II show the phantom's position and orientation error, based on the MicronTracker measurements, after the user has attempted to reproduce the reference image (i.e., by counteracting the virtual springs as needed).

Table I. Orientation Error of Phantom (In Degrees) Before and After Use of Virtual Springs. Experiments 1-5 are with X-Y Plane Disturbance and 6-10 are with 6 DOF Disturbance.

Before Virtual Springs After Virtual Springs
ΔRx ΔRy ΔRz Total ΔRx ΔRy ΔRz Total
1 -0.26 0.12 0.62 0.68 -0.10 0.04 0.71 0.72
2 0.06 0.06 0.79 0.80 0.35 -0.18 0.91 0.99
3 -0.07 0.19 0.35 0.40 0.20 1.71 -0.11 1.73
4 0.21 -0.07 0.99 1.01 -0.17 -0.18 1.00 1.03
5 -0.09 0.15 0.58 0.60 -0.25 1.57 0.69 1.73
6 -15.15 1.28 -5.62 16.21 -7.06 7.91 -2.07 10.80
7 -15.10 1.11 -5.81 16.22 -5.40 3.83 -0.45 6.64
8 -13.63 1.20 -5.20 14.64 -8.92 3.54 -2.15 9.83
9 -9.21 1.04 -1.93 9.47 -6.60 3.66 -0.09 7.55
10 -11.04 0.94 -3.28 11.56 -4.20 -2.55 0.29 4.92
Open in a new tab

Table II. Position Error of Phantom (In mm) Before and After Use of Virtual Springs. Experiments 1-5 are with X-Y Plane Disturbance and 6-10 are with 6 DOF Disturbance.

Before Virtual Springs After Virtual Springs
Δx Δy Δz Total Δx Δy Δz Total
1 10.27 4.45 -0.15 11.19 -1.24 -1.06 0.16 1.64
2 11.18 2.55 -0.91 11.50 0.05 -0.08 -0.66 0.67
3 8.54 4.37 0.38 9.60 -0.61 0.33 -0.37 0.79
4 11.84 3.60 -0.70 12.39 -0.23 0.15 -0.11 0.30
5 8.92 4.10 0.07 9.82 0.19 -0.44 0.20 0.52
6 -0.58 3.83 7.11 8.09 0.52 0.86 -0.91 1.36
7 32.38 4.32 12.25 34.89 1.50 1.22 -0.13 1.93
8 27.61 5.25 11.20 30.25 -0.68 -0.21 0.12 0.72
9 6.22 4.70 5.56 9.57 -0.20 0.15 -0.22 0.34
10 13.96 4.39 7.27 16.34 -0.02 0.55 -0.10 0.56
Open in a new tab

In Tables I and II, for trials 1-5 with x-y plane disturbance, the total mean error before the virtual springs is 0.7 degrees in orientation and 10.9 mm in translation and after the virtual springs it is 1.2 degrees in orientation and 0.8 mm in translation. Additionally, for trials 6-10 with 6 DOF disturbance, the total mean error before the virtual springs is 13.6 degrees in orientation and 19.8 mm in translation and after the virtual springs it is 7.9 degrees in orientation and 1.0 mm in translation.

IV. Discussion and Conclusions

We developed a robot system for integrating 3D US imaging with x-ray (CT) guidance for external beam radiotherapy. Because ultrasound provides better soft tissue contrast, it allows the clinician to better visualize the treatment target and surrounding organs. Furthermore, US does not introduce additional radiation and can be used to continuously monitor the treatment. In particular, this enables the clinician to ensure that the target remains in the x-ray beam path.

We proposed a new workflow that incorporates US probe placement in the simulation (SIM) and radiation treatment (LINAC) phases and canine experiments were performed to verify this workflow. The canine experiment results suggest that the proposed cooperative control method with soft virtual fixtures (implemented as virtual springs) can improve the reproducibility of probe placement during the fractionated radiation treatments. Our phantom and in-vivo results indicate that soft virtual fixtures can lead to more reproducible soft tissue setup by enabling the system to deviate from the recorded probe position to achieve a more consistent US image. While our in-vivo results with the 3-DOF robot (Fig. 7) demonstrated a small improvement with the use of soft virtual fixtures, we believe that a more dramatic difference would occur with a 5-DOF (or 6-DOF) robot that enables correction for orientation errors.

We performed phantom experiments with the 5-DOF robot system to reproduce the position, orientation, and applied force of the US probe with respect to the target anatomy (of course, it is not always possible to satisfy both position and force constraints, so the clinician must make the final decision). Because the robot system is under-actuated, we developed an Adjustment GUI to guide the user to set the desired orientation using a passive arm. We demonstrated that this can be done with less than 1.5 degrees of total orientation error, which we expect to be clinically acceptable. We previously proposed a cooperative control method with soft virtual fixtures (implemented as virtual springs) and tested it for 3D translations. In this paper, we provide experimental results from extending the soft virtual fixtures to 5 DOF (three translations and two rotations). The results, presented in Tables I and II, show that the virtual springs enable the user to reduce the simulated patient setup error by over-riding the recorded position and orientation to obtain the best image match. The first 5 trials did not introduce orientation setup errors, so in some cases the user slightly increased the orientation error. The main point, however, is that all large position and orientation setup errors were significantly reduced.

Our future work includes the use of US image feedback to adjust the virtual springs so that they more accurately guide the clinician to the correct location. This is especially important because it can eliminate the need for clinicians with US expertise during treatment. In this case, an experienced ultrasonographer would only be needed during simulation and the robot system would enable the novice user to reproduce the probe position defined by the expert.

Acknowledgments

Martin Lachaine and Rupert Brooks at Elekta (formerly, Resonant Medical) provided and supported the research version of the Clarity System that was used for this project. Emad Boctor contributed to the canine study protocol. Judy Cook administered anesthesia and Robert DeJong provided ultrasound assistance during the canine experiments. This work was supported by NIH R01 CA161613.

Contributor Information

H. Tutkun Şen, Email: hsen1@jhu.edu.

Muyinatu A. Lediju Bell, Email: mledijubell@jhu.edu.

Yin Zhang, Email: yzhang54@exchange.johnshopkins.edu.

Kai Ding, Email: kding1@jhmi.edu.

John Wong, Email: jwong35@jhmi.edu.

Iulian Iordachita, Email: iordachita@jhu.edu.

Peter Kazanzides, Email: pkaz@jhu.edu.

References

  • 1.Harris EJ, Miller NR, Bamber JC, Symonds-Tayler JRN, Evans PM. Speckle tracking in a phantom and feature-based tracking in liver in the presence of respiratory motion using 4d ultrasound. Physics in Medicine and Biology. 2010;55(12):3363. doi: 10.1088/0031-9155/55/12/007. [DOI] [PubMed] [Google Scholar]
  • 2.Bell MAL, Byram BC, Harris EJ, Evans PM, Bamber JC. In vivo liver tracking with a high volume rate 4d ultrasound scanner and a 2d matrix array probe. Physics in Medicine and Biology. 2012;57(5):1359. doi: 10.1088/0031-9155/57/5/1359. [DOI] [PubMed] [Google Scholar]
  • 3.Bell MAL, Sen HT, Iordachita I, Kazanzides P, Wong J. In vivo reproducibility of robotic probe placement for a novel ultrasound-guided radiation therapy system. Journal of Medical Imaging. 2014;1(2):025 001–025 001. doi: 10.1117/1.JMI.1.2.025001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Lachaine M, Falco T. Intrafractional prostate motion management with the Clarity Autoscan system. Medical Physics International. 2013;1(1):72. [Google Scholar]
  • 5.Schlosser J, Salisbury K, Hristov D. Telerobotic system concept for real-time soft-tissue imaging during radiotherapy beam delivery. Medical Physics. 2010;37:6357–6367. doi: 10.1118/1.3515457. [DOI] [PubMed] [Google Scholar]
  • 6.Schlosser J, Salisbury K, Hristov D. Tissue displacement monitoring for prostate and liver IGRT using a robotically controlled ultrasound system. Medical Physics. 2011;38:3812. [Google Scholar]
  • 7.Pierrot F, Dombre E, Degoulange E, Urbain L, Caron P, Gariepy J, Megnien JL. Hippocrate: A safe robot arm for medical applications with force feedback. Medical Image Analysis. 1999;3:285–300. doi: 10.1016/s1361-8415(99)80025-5. [DOI] [PubMed] [Google Scholar]
  • 8.Abolmaesumi P, Salcudean SE, Zhu WH, Sirouspour MR, DiMaio SP. Image-guided control of a robot for medical ultrasound. IEEE Trans on Robotics and Automation. 2002;18(1):11–23. [Google Scholar]
  • 9.Vilchis A, Troccaz J, Cinquin P, Masuda K, Pellissier F. A new robot architecture for tele-echography. IEEE Trans on Robotics and Automation. 2003;19(5):922–926. [Google Scholar]
  • 10.Mitsuishi M, Warisawa S, Tsuda T, Higuchi T, Koizumi N, Hashizume H, Fujiwara K. Remote ultrasound diagnostic system. IEEE Intl Conf on Robotics and Automation (ICRA) 2001;2:1567–1574. [Google Scholar]
  • 11.Koizumi N, Warisawa S, Hashizume H, Mitsuishi M. Impedance controller and its clinical use of the remote ultrasound diagnostic system. IEEE Intl Conf on Robotics and Automation (ICRA) 2003;1:676–683. [Google Scholar]
  • 12.Conti F, Park J, Khatib O. Experimental Robotics. Springer; 2014. Interface design and control strategies for a robot assisted ultrasonic examination system; pp. 97–113. [Google Scholar]
  • 13.Salcudean S, Bell G, Bachmann S, Zhu WH, Abolmaesumi P, Lawrence PD. Medical Image Computing and Computer-Assisted Intervention–MICCAI99. Springer; 1999. Robot-assisted diagnostic ultrasound–design and feasibility experiments; pp. 1062–1071. [Google Scholar]
  • 14.Monfaredi R, Wilson E, Azizi Koutenaei B, Labrecque B, Leroy K, Goldie J, Louis E, Swerdlow D, Cleary K. Robot-assisted ultrasound imaging: Overview and development of a parallel telerobotic system. Minimally Invasive Therapy & Allied Tech. 2014:1–9. doi: 10.3109/13645706.2014.992908. [DOI] [PubMed] [Google Scholar]
  • 15.Kuhlemann I. Master's thesis. Institute for Robotics and Cognitive Systems; 2013. Force and image adaptive strategies for robotised placement of 4D ultrasound probes. [Google Scholar]
  • 16.Şen HT, Bell MAL, Iordachita I, Wong J, Kazanzides P. A cooperatively controlled robot for ultrasound monitoring of radiation therapy. IEEE/RSJ Intl Conf on Intelligent Robots and Systems (IROS) 2013 doi: 10.1109/IROS.2013.6696791. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Bourland J. ch 2: Ultrasound-Guided Radiation Therapy. Taylor and Francis; 2012. Image-Guided Radiation Therapy. [Google Scholar]

RESOURCES