Prostate brachytherapy seed localization using a mobile c-arm without tracking

Abstract

The success of prostate brachytherapy depends on the faithful delivery of a dose plan. In turn, intraoperative localization and visualization of the implanted radioactive brachytherapy seeds enables more proficient and informed adjustments to the executed plan during therapy. Prior work has demonstrated adequate seed reconstructions from uncalibrated mobile c-arms using either external tracking devices or image-based fiducials for c-arm pose determination. These alternatives are either time-consuming or interfere with the clinical flow of the surgery, or both. This paper describes a seed reconstruction approach that avoids both tracking devices and fiducials. Instead, it uses the preoperative dose plan in conjunction with a set of captured images to get initial estimates of the c-arm poses followed by an auto-focus technique using the seeds themselves as fiducials to refine the pose estimates. Intraoperative seed localization is achieved through iteratively solving for poses and seed correspondences across images and reconstructing the 3D implanted seeds. The feasibility of this approach was demonstrated through a series of simulations involving variable noise levels, seed densities, image separability and number of images. Preliminary results indicate mean reconstruction errors within 1.2 mm for noisy plans of 84 seeds or fewer. These are attained for additive noise whose standard deviation of the 3D mean error introduced to the plan to simulate the implant is within 3.2 mm.

1. INTRODUCTION

Brachytherapy accounts for a significant and growing proportion of prostate cancer treatments. It involves implanting radioactive seeds into the prostate via needles according to a preoperative plan. A schematic is shown in Fig. 1(a). The success of brachytherapy relies on faithful delivery of the dose plan, thus avoiding insufficient dose to the cancer and/or inadvertent radiation of the rectum, urethra, and bladder. The former permits the cancer to recur, while the latter causes adverse side effects like rectal ulceration, incontinence, and impotence. According to a comprehensive review by the American Brachytherapy Society, the major existing limitation of brachytherapy is the inability to perform treatment optimization on the fly on the operating room, due to the inability to localize the implanted seeds in relation to the prostate.¹

Figure 1.

(a) Prostate brachytherapy schematic. (b) X-ray image of brachytherapy seeds.

In current clinical practice, c-arm fluoroscopy imaging is often used in conjunction with transrectal ultrasound (TRUS) for gross visual assessment of the implanted seeds. With a rapid and inexpensive solution, it may be feasible, through interactive 2D imaging, to provide accurate quantitative measurement of the 3D positions of the implanted seeds during the operation. This, together with the registration to ultrasound, allows surgeons to modify the plan on the spot to improve the efficacy of brachytherapy. However, two major difficulties exist for attaining a rapid and accurate 3D reconstruction using the x-ray images captured by mobile c-arms that are available in the operating room (OR) (see Fig. 1(b)). First, the most commonly available c-arms do not have encoded rotational joints and they are allowed to move between shots, which means that pose estimation is required. External trackers and tracking fiducials have been used for pose estimation in the past, and each method has its own drawbacks. External trackers, such as optical and EM trackers have proven to be both expensive and cumbersome in the OR. Optical trackers require a line of sight, whereas EM trackers are sensitive to the presence of metal objects in the work area. Alternatively, different tracking fiducials^{2, 3} have been used and have provided reasonable results. However, these require proper positioning and segmentation which is sometimes difficult and time-consuming.

The second major difficulty is the limited angular separation between the acquired fluoroscopy images, as required by the surgical setup, which makes seed reconstruction more difficult. Seed reconstruction approaches can be categorized as either correspondence-based techniques^4–10 or tomosynthesis techniques.^11–13 All of these methods assume that the c-arm poses are known to relatively high accuracy. Murphy et al.¹⁴ proposed a forward iterative method that uses a 3D model to reconstruct the seeds. However, they assume prior knowledge of the intrinsic and extrinsic camera parameters in order to refine those parameters and reconstruct the seed configurations. We make use of this model for our current algorithm but with no prior estimates for the extrinsic parameters, i.e., we employ the model for initial pose estimation as well. Tutar et al.¹⁵ have also used a model-based technique for registration of ultrasound to fluoroscopy.

Jain et al.¹⁶ have proposed a unified framework for pose estimation and reconstruction using randomly distributed points in the imaging volume. However, for proper convergence, this method requires the availability of some prior correspondences of the points. More recently, Chintalapani et al.¹⁷ have presented a reconstruction approach that uses a simple elliptical fiducial and an auto-focus approach as well.

In this paper we propose a strategy to enable seed reconstruction using fluoroscopy images from mobile c-arms with no reliance on any tracking devices or fiducials for pose estimation. We use the preoperative plan in collaboration with the 2D images in an iterative fashion for both pose estimation and reconstruction. The central idea of our approach is that prior knowledge about the 3D configuration of the seeds can provide reasonable initial estimates for the image poses. Those, in turn can enable us to find a better model for the true seed locations from the captured images. Iteration using this enhanced model improves both the pose estimates and the reconstruction.

2. METHODS

2.1 Assumptions and preliminaries

We assume that the c-arm geometry is known and that its x-ray images have been corrected for geometrical distortion. The algorithm uses pre-segmented images in which the entire seed collection is specified by their 2D coordinates. We use six parameters (three for rotation and three for translation) to specify the unknown pose.

2.2 Fluoroscopic imaging

X-ray imaging involves three different coordinate systems, as shown in Fig. 2: (1) the camera coordinate system F_X, which refers to the source of the x-ray in this case, (2) the x-ray image coordinate system F_I, and (3) the model coordinate system F_M. Let P_M be a 3D point in the F_M frame (in homogeneous coordinates), and let P_I be its corresponding 2D-projection in the F_I frame (in homogeneous coordinates as well). Define ^XF_M to be the 4 × 4 rigid transformation matrix that transforms a point in F_M to F_X and define ^IF_X to be the 3 × 4 perspective projection matrix.

Figure 2.

Fluoroscopic imaging geometry and the involved coordinate systems.

Image acquisition follows a perspective projection model that is described by the following equation:

$${\mathbf{P}}_{\mathbf{I}}={}^I{F}_X{}^X{F}_M{\mathbf{P}}_{\mathbf{M}}$$ (1)

$${}^I{F}_X=\left[\begin{array}{cccc}\hfill f/s_u\hfill & \hfill 0\hfill & \hfill o_u\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill f/s_v\hfill & \hfill o_v\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \end{array}\right]$$ (2)

$${}^X{F}_M=\left[\begin{array}{cccc}\hfill r_{11}\hfill & \hfill r_{12}\hfill & \hfill r_{13}\hfill & \hfill T_x\hfill \\ \hfill r_{21}\hfill & \hfill r_{22}\hfill & \hfill r_{23}\hfill & \hfill T_y\hfill \\ \hfill r_{31}\hfill & \hfill r_{32}\hfill & \hfill r_{33}\hfill & T_z\\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \end{array}\right]$$ (3)

where s_u and s_v are pixel sizes of the image in the x- and y-directions respectively, o_u and o_v are the coordinates of the image center O in the x- and y-directions respectively, f is the focal length, r_ij’s are the elements of the rotation matrix, and T_x, T_y, and T_z are the translation parameters.

Since we assume that the c-arm is calibrated, it follows that ^IF_X is known. For non-encoded c-arms, however, the pose (which is embedded in the transformation ^XF_M) is unknown and must be determined. The pose parameters are stacked into a vector

$$\mathbf{s}={(a,b,c,T_x,T_y,T_z)}^T$$ (4)

where a, b, and c are the Rodrigues parameters derived of the quaternion representation of the rotation matrix.¹⁸

2.3 Algorithm

A pose estimation algorithm using the Hungarian method¹⁹ and a correspondence-based reconstruction technique are used iteratively. The input is a set of pre-processed observed images and a prior 3D model of the implanted seeds. After each iteration, we have a new pose for each image and an updated 3D model.

2.3.1 Pose estimation

Pose estimates are retrieved for each image through a multivariable constrained optimization procedure that minimizes an objective function based on the Hungarian algorithm.^{19, 20} The optimization uses a sequential quadratic programming (SQP) method and estimates the Hessian of the Lagrangian at each iteration using the BFGS formula.²¹

Let p_I and p_M be the 2D and 3D points (in non-homogeneous coordinates) that correspond to P_I and P_M in Eq. (1) respectively. Now let M_k be the set of all 3D model points after iteration k, i.e.,

$${\mathbf{M}}_k=\{{\mathbf{p}}_{{\mathbf{M}}_e}\},e=1,2,..,n,$$ (5)

where n is the number of seeds. To be concrete, we think of M₀ as the initial 3D model consisting of points from the preoperative plan, but it could also be a 3D model derived from an intraoperative record of the locations of the dropped seeds. Similarly, let I_r be the set of all 2D segmented seed positions in the r^th observed image:

$${\mathbf{I}}_r=\{{\mathbf{p}}_{{\mathbf{I}}_{r,e}}\},r=1,2,..,m,e=1,2,..,n,$$ (6)

where m is the number of images.

Consider the r^th image at iteration k. The Hungarian matching algorithm is applied in 2D between the seed locations in I_r and those obtained from projections of M_k−1 with a tentative pose s̃ using Eqs. (1), (2) and (3); we denote this latter set Ĩ. Note that each pose vector s̃ maps uniquely to a transformation ^XF̃_M.

The Hungarian algorithm both solves the assignment problem and computes a corresponding cost T for this matching based on an input cost matrix A given to the algorithm. This input cost matrix is n × n

$$A=\{a_{\mathit{\text{ij}}}\},$$ (7)

where a_ij is the 2D Euclidean distance between element i in Ĩ that corresponds to a candidate pose s̃ and element j in I_r.

The output cost is given by

$$T={\displaystyle \sum _{i=1}^n{a}_{(i,c(i))},}$$ (8)

where c(i) is the index of the element in I_r that corresponds to element i in Ĩ. The optimization procedure finds s that minimizes T for each image. In this case, the optimal pose for image r at iteration k, denoted sŝ_r,k, is the one whose respective set Î gives the minimum matching cost T̂_r,k. Thus, the pose estimation step also yields the seed correspondences ĉ_r,k = [c(1), c(2), ‥, c(n)], which we use in the reconstruction step.

2.3.2 Reconstruction

After independent pose estimation for each of the observed images, the estimated poses ŝ_r,k, r = 1, 2, ‥, m are used collectively with the correspondences ĉ_r,k and the sets I_r, r = 1, 2, ‥, m, to reconstruct M_k. The reconstruction accuracy (RA) is used as a cost metric to compute a so-called symbolic 3D intersection point based on the back-projected lines.^{8, 22} This symbolic intersection point is the one that minimizes the sum of the squared distances to the lines.

Consider the reconstruction of the e^th point in M_k; let us call it Q. Corresponding projections in the images are p_I1,ĉ1,k(e), p_I2,ĉ2,k(e), ‥, and p_Im,ĉm,k(e). Let q₁, q₂, ‥, qm be the same points after transforming them to F_M using the pose estimates ŝ_r,k, r = 1, 2, ‥, m. Let ℓ₁, ℓ₂, ‥, ℓ_m be the unit directional vectors from each source position (the origin of F_X) to the points, and they are in F_M as well. Q is computed such that it minimizes a function

$$G={\displaystyle \sum _{i=1}^m{d}_i^2={\displaystyle \sum _{i=1}^m{\Vert (\mathbf{Q}-{\mathbf{q}}_i)\times {\ell }_i\Vert }^2}}$$ (9)

where d_i is the Euclidean distance between Q and line i, as shown in Fig. 3 for the case when m = 3.

Figure 3.

The symbolic intersection of three lines.

2.3.3 Model update and iteration

The set of points of the implant reconstructed in the previous step is then used as an updated 3D model to re-estimate the poses of each image and refine the reconstruction as in Sec. 2.3.1 and Sec. 2.3.2. This is done in an iterative fashion until the pose estimation cost is almost unchanged between successive iterations for all the images.

3. EXPERIMENTS AND RESULTS

3.1 Effect of various factors

Simulation studies were conducted to examine the effect of several factors on the accuracy of the pose estimation and the reconstruction. For the following experiments, the range of pose parameters is assumed to be known within 60 degrees, 55 degrees and 30 degrees for the three Euler angles (Z-Y-Z notation) and 40 mm for the translations in each of x- and y- directions and 400 mm for the translation in the z-direction. These assumptions are supported by the fact that the motion of the c-arm is constrained within a relatively small cone during the clinical procedure. Random multistart optimizations are used to initialize the pose estimates for each image where the pose chosen is the one corresponding to the minimum Hungarian cost. It is to be noted that the pose optimization is done with respect to the Rodrigues parameters for the rotation as explained in the methods section, whereas the Euler angles are just used for initial range specification.

3.1.1 Effect of noise level

Simulated plans of 84 seeds each were used. In order to simulate seed implantation errors, we added independent identically distributed (i.i.d.) additive white Gaussian noise to the seed coordinates of the generated plans. The standard deviation of the 3D mean error introduced by the noise simulator is denoted σ, which was varied from 0.8 mm to 4 mm in increments of 0.8 mm. Five different plans were used and ten different noise instances were added to each plan providing a total of 50 different experiments for each noise level. Six images were simulated using the synthetic implant and used to run the algorithm. Table 1 shows the mean number of matched seeds (over all the experiments) and means and standard deviations of the reconstruction error (RE). The latter were computed once based on all of the seeds and another time based on only the matched ones. It is to be noted that a seed is declared to be matched only if it is correctly matched in all images. Table 2 shows the mean and standard deviation of rotation and translation errors in the pose. Since translation errors in depth are always significantly greater than those parallel to the plane, large errors in translation do not reflect that badly on reconstruction errors.

Table 1.

Effect of noise level on reconstruction accuracy.

σ (mm)	Number of matched seeds	Overall RE (mm) mean ± std	Matched RE (mm) mean ± std
0.8	81.4(96.9%)	0.2 ± 0.3	0.2 ± 0.2
1.6	83.5(99.5%)	0.3 ± 0.2	0.3 ± 0.2
2.4	78.6(93.6%)	0.7 ± 0.8	0.5 ± 0.3
3.2	76.0(90.5%)	1.0 ± 1.4	0.7 ± 0.4
4.0	67.5(80.4%)	1.7 ± 2.4	0.8 ± 0.5

Table 2.

Effect of noise level on pose estimates.

σ (mm)	Rotation error (degrees) mean ± std	Translation error (mm) mean ± std
0.8	0.7 ± 3.8	1.9 ± 3.9
1.6	0.7 ± 0.4	3.1 ± 2.6
2.4	1.5 ± 3.5	4.5 ± 4.1
3.2	1.9 ± 2.5	6.8 ± 4.8
4.0	2.3 ± 3.3	7.1 ± 5.9

3.1.2 Effect of seed density

We generated plans with seed densities of 1, 1.5, 2, and 2.5 seeds/cc for a 40 cc prostate, resulting in 36, 60, 84, and 96 seeds, respectively. Gaussian noise of σ equal to 3.2 mm was added to the plans and the experiments were carried out using the same scheme explained in Sec. 3.1.1. Results are shown in Table 3 and Table 4.

Table 3.

Effect of seed density on reconstruction accuracy.

Total number of seeds	Number of matched seeds	Overall RE (mm) mean ± std	Matched RE (mm) mean ± std
36	32.8(91.1%)	1.2 ± 1.1	1.0 ± 0.6
60	57.1(95.2%)	1.0 ± 1.2	0.8 ± 0.5
84	76.0(90.5%)	1.1 ± 1.4	0.7 ± 0.4
96	74.2(77.3%)	1.3 ± 1.9	0.6 ± 0.3

Table 4.

Effect of seed density on pose estimates.

Total number of seeds	Rotation error (degrees) mean ± std	Translation error (mm) mean ± std
36	3.7 ± 10.4	10.0 ± 7.4
60	2.2 ± 3.6	8.5 ± 7.5
84	1.9 ± 2.5	6.8 ± 4.8
96	2.3 ± 6.0	7.0 ± 5.4

It is observed that we attain relatively lower mean reconstruction errors for seed clouds of middle-range densities (60 and 84 seeds). This can be explained as follows: for small number of seeds (36 seeds), the errors in the pose estimates tend to be relatively high (due to the presence of fewer seeds to constrain the pose) and therefore degrade the reconstruction accuracy. As for more dense clouds (96 seeds), the errors in correspondence returned by the Hungarian increase as reflected in the low percentage of correctly matched seeds, which in turn increases the reconstruction error.

3.1.3 Effect of number of images

The number of images used was varied from three to six images. Simulated plans of 84 seeds each were used. Gaussian noise with σ = 3.2 mm was added to the plans and the experiments were conducted using the same scheme explained in Sec. 3.1.1. Results are shown in Table 5 and Table 6.

Table 5.

Effect of number of images on reconstruction accuracy.

Number of images	Number of matched seeds	Overall RE (mm) mean ± std	Matched RE (mm) mean ± std
3	52.8(62.8%)	1.8 ± 1.9	0.8 ± 0.5
4	70.1(83.4%)	1.2 ± 1.6	0.7 ± 0.4
5	74.3(88.5%)	1.1 ± 1.6	0.7 ± 0.4
6	78.2(97.6%)	1.0 ± 1.5	0.7 ± 0.4

Table 6.

Effect of number of images on pose estimates.

Number of images	Rotation error (degrees) mean ± std	Translation error (mm) mean ± std
3	2.4 ± 4.8	6.2 ± 6.0
4	1.7 ± 0.8	5.8 ± 4.2
5	1.8 ± 3.6	6.3 ± 6.0
6	1.6 ± 1.0	6.3 ± 5.2

Increasing the number of images results in a noticeable improvement in reconstruction as reflected by the mean reconstruction error (MRE) dropping from 1.8 mm (using 3 images) to 1.0 mm (using 6 images). The same behavior is observed for rotation error. Furthermore, the number of matched seeds increases dramatically.

3.1.4 Effect of angular separation of images

For this experiment, six images were generated from a noisy plan of 84 seeds on a cone of angle 10°, 15°, 20° and 25° around the AP axis for different experiments and the six images were used to run the algorithm. Gaussian noise with σ = 2.4 mm was added to all the plans using the same scheme explained in the previous section. Results over 50 different runs for each case are shown in Table 7.

Table 7.

Effect of angular separation on reconstruction accuracy.

Cone angle (degrees)	Number of matched seeds	Overall RE (mm) mean ± std	Matched RE (mm) mean ± std
10	79.1(94.2%)	0.9 ± 1.5	0.6 ± 0.3
15	80.7(96.1%)	0.7 ± 1.0	0.5 ± 0.3
20	78.7(93.6%)	0.7 ± 0.8	0.5 ± 0.3
25	79.2(94.3%)	0.6 ± 0.7	0.5 ± 0.3

The overall MRE drops from 0.9 mm to 0.6 mm when the cone angle is increased from 10° to 25°; however for all 4 experiments the matching rate was fairly good.

3.2 Convergence of the algorithm

A simulation was done to test the convergence of the algorithm. For this experiment, a plan of 84 seeds was used with six images. The cone angle was 20°. Figure 4 shows the Hungarian cost and the matching rate plotted for each iteration of the algorithm for different values of noise levels with σ varying from 0.8–4.0 mm. The Hungarian cost was averaged over all six images.

Figure 4.

An example of the convergence rate of (a) the Hungarian cost and (b) the matching rate for the iterative algorithm for different noise levels.

4. DISCUSSION AND CONCLUSION

We proposed the use of the preoperative plan to localize the implanted seeds from multiple c-arm fluoroscopy images with no tracking. We rely on the fact that adequate similarity between the plan and the reality may allow us to use the seeds themselves as fiducials. The other fact we rely on is that reasonably accurate pose estimates for the 2D fluoroscopy images (within a few degrees for the rotation and several mm for the translation) are sufficient to initialize a clinically satisfactory reconstruction of the implanted seeds in 3D. Iteration is used to refine the reconstruction through the use of the 2D x-ray projections and prior 3D reconstructions. Furthermore, the explicit correspondence assigned to each seed in the prostate volume simplifies the task of cold- and hot-spot identification intraoperatively, thus contributing to the overall improvement of the dose delivery scheme throughout the execution of the brachytherapy treatment procedure.

It is worth mentioning that in some cases, the seed drop positions are recorded intraoperatively, for example using ultrasound. In this case, the marked seed locations may be more accurate and should therefore be used instead of the preoperative plan to initialize the 3D model for our algorithm.

We demonstrated feasibility using preliminary simulation studies and are currently pursuing to apply this approach for phantoms. However, we must address several issues before the method will be clinically feasible, including the problems of hidden seeds, false positives, and seed segmentation errors. Another important aspect that yet needs to be studied is the different error sources that might possibly make the plan and the actual implant significantly different from each other. These are errors that are due to edema, faulty seed placement, seed splaying, patient movement and others. Once these are properly modeled, these can easily be incorporated into our algorithm. In summary, this is a first step towards removing all types of trackers from the procedure. It provides proof of concept and encourages further investigation of this approach.