Robust surface registration using salient anatomical features for image-guided liver surgery: Algorithm and validation

Abstract

A successful surface-based image-to-physical space registration in image-guided liver surgery (IGLS) is critical to provide reliable guidance information to surgeons and pertinent surface displacement data for use in deformation correction algorithms. The current protocol used to perform the image-to-physical space registration involves an initial pose estimation provided by a point based registration of anatomical landmarks identifiable in both the preoperative tomograms and the intraoperative presentation. The surface based registration is then performed via a traditional iterative closest point (ICP) algorithm between the preoperative liver surface, segmented from the tomographic image set, and an intraoperatively acquired point cloud of the liver surface provided by a laser range scanner. Using this more conventional method, the registration accuracy can be compromised by poor initial pose estimation as well as tissue deformation due to the laparotomy and liver mobilization performed prior to tumor resection. In order to increase the robustness of the current surface-based registration method used in IGLS, we propose the incorporation of salient anatomical features, identifiable in both the preoperative image sets and intraoperative liver surface data, to aid in the initial pose estimation and play a more significant role in the surface-based registration via a novel weighting scheme. Examples of such salient anatomical features are the falciform groove region as well as the inferior ridge of the liver surface. In order to validate the proposed weighted patch registration method, the alignment results provided by the proposed algorithm using both single and multiple patch regions were compared with the traditional ICP method using six clinical datasets. Robustness studies were also performed using both phantom and clinical data to compare the resulting registrations provided by the proposed algorithm and the traditional method under conditions of varying initial pose. The results provided by the robustness trials and clinical registration comparisons suggest that the proposed weighted patch registration algorithm provides a more robust method with which to perform the image-to-physical space registration in IGLS. Furthermore, the implementation of the proposed algorithm during surgical procedures does not impose significant increases in computation or data acquisition times.

INTRODUCTION

The determination of an accurate image-to-physical space registration is a fundamental step in providing meaningful guidance information to surgeons via image-guided surgery (IGS). A significant body of research has been dedicated to the use of IGS techniques for neurosurgical applications and has resulted in several commercially available systems (e.g., StealthStation, Medtronic Navigation, Louisville, CO). A common feature of the developed IGS technology for neurosurgery is the use of point-based landmarks, via bone-implanted or skin-affixed fiducial markers, to provide the registration of image and physical space. The use of such point-based techniques is greatly facilitated in neurosurgical IGS by the rigid anatomy (i.e., skull) surrounding the organ of interest. Unfortunately, the use of such point-based techniques is not applicable for open abdominal IGS due to the lack of rigid anatomical landmarks and the inability to preoperatively attach a set of extrinsic fiducials that will remain rigid relative to the organ of interest after laparotomy and organ mobilization.

Since the use of rigid, point-based landmarks is not feasible in image-guided liver surgery (IGLS), surface-based techniques were proposed to determine the registration between the preoperative images and the intraoperative presentation.^{1, 2} Specifically, the iterative closest point (ICP) algorithm proposed by Besl and McKay³ has traditionally been used to determine the transformation between the image-space surface of the liver, derived from preoperative image segmentations and the intraoperative liver surface. Intraoperative data were initially acquired using an optically tracked probe, while more recent efforts have utilized a laser range scanner (LRS) to provide spatially dense, textured delineations.^{4, 5} In addition to being used for IGLS, LRS technology has also been employed to provide surface data in neurosurgical procedures for the purpose of tracking intraoperative brain shift.^{6, 7, 8, 9} Several groups have also explored the use of intraoperative ultrasound (iUS) to acquire sparse data for use in abdominal IGS.^{10, 11}

The current protocol for surface-based image-to-physical space registration in IGLS (described in detail by Cashet al.^{5, 12}) begins with the selection of anatomical fiducial points in the preoperative image sets prior to surgery. The homologous physical-space location of these anatomical fiducials are then digitized during the surgical procedure such that a point-based initial alignment registration can be performed. The point-based registration serves to provide a reasonable initial pose for the ICP algorithm, which is used to register the liver surface derived from preoperative images and LRS data acquired intraoperatively.

Being that the surface alignment provided by the ICP algorithm is highly dependent on the initial pose of the surfaces, gross errors in the initial alignment provided by the point-based registration can result in erroneous surface alignments. A failed surface-based registration not only compromises the guidance information relayed to the surgeon, but also impairs deformation correction efforts due to inaccurate surface displacement data that are used to drive mathematical models.¹³ In IGLS, the quality of the initial alignment registration can be compromised by the large fiducial localization errors (FLE) inherent in using anatomical landmarks that undergo deformation relative to the preoperative images. Additionally, gravity and the effects of the liver mobilization and packing performed prior to open liver resections can lead to liver deformations that can compromise the results of a rigid ICP surface registration. Figure 1 shows an example clinical dataset where a poor initial alignment registration due to high FLE of the anatomical fiducials and large liver deformations resulted in the convergence of the rigid ICP algorithm to a gross misalignment.

Figure 1.

Example of poor initial alignment (a),(b) and resulting misregistration (c),(d) of clinical data obtained using a traditional ICP algorithm. Note that the LRS scan of the anterior surface of the liver is registered to the posterior liver surface via ICP.

In order to circumvent erroneous surface registrations due to gross misalignments in the initial pose, we propose the incorporation of reliably identifiable, salient anatomical features into the ICP algorithm. As shown in Fig. 2, the falciform ligament region is one such feature. This ligament divides the medial and lateral segments of the left lobe and can be identified on the preoperative image surface via the ligament’s distinctive groove in the surface. The falciform ligament region can be delineated in the intraoperative LRS surface presentation via the difference in texture between the ligament and liver parenchyma. In addition to the falciform groove, the inferior ridge of the liver between the falciform and right triangular ligaments along sections 4, 5, and VI would also be a potential salient feature to utilize. For the purposes of this study the salient anatomical features were delineated in the preoperative computer tomography (CT) image space via manual segmentation of liver surface (generated via the Marching Cubes Algorithm).¹⁴ With regards to intraoperative LRS salient feature segmentation, the homologous regions were delineated by the surgeon via optically tracked pen probe. These point sets, as well as the texture information provided by the LRS, were then used to guide the manual segmentation of the salient features in the intraoperatively acquired sparse data. A preliminary formulation allowing the incorporation of a single salient anatomical feature has been described previously.¹⁵

Figure 2.

Anatomical schematic (left) and examples of preoperative image (middle) and intraoperative LRS liver data (right) with corresponding falciform ligament regions outlined. Note that the falciform ligament region can be located on the preoperative image surface via the groove in the surface and texture can be used to delineate the falciform region in the LRS surface.

The proposed use of salient anatomical features to weight the ICP registration is similar to the weighted geometrical features (WGF) algorithm described by Maurer et al.¹⁶ This method described a way to incorporate multiple surfaces and point sets within an iterative matching process whereby each of the surfaces or point sets (called features) were assigned a particular weight within the closest point cost function. The WGF algorithm was shown to facilitate the registration of CT and T2-weighted magnetic resonance image volumes of the head. The work also provides the closed form solution to perform a weighted point-based registration (PBR) that is utilized in this work. A similar approach where point sets and surface regions are combined within an ICP based approach was also proposed by Collignon et al.¹⁷

In order to curb some of the local minima convergence issues with the traditional closest point operator used in the ICP algorithm, an interpolated closest point transform was proposed by Cao et al.¹⁸ Building on the work of Ge et al.¹⁹ and Kapoutsis et al.,²⁰ the interpolated closest point method first computes a closest point transform, which is a variation on the distance transform whereby each voxel in a target image volume analog contains the location of its closest point. The proposed method computes the closest point transform using a variation on the Fast Marching Method proposed by Sethian.²¹ In order to circumvent discretization error of standard closest point transform methods, a novel interpolation scheme is implemented on the closest point transform.

In addition to the use of geometrical information in surface-based registration methods, a number of studies have also incorporated texture information to drive the matching process. Miga et al.⁸ proposed the SurfaceMI algorithm, which incorporated the mutual information metric^{22, 23, 24} into the registration of textured cortical LRS data to textured brain surfaces extracted from MR volumes. Johnson and Kang propose the incorporation of color information to improve point correspondence determination in the registration of textured 3D data.²⁵ In addition to the incorporation of texture information to bias point correspondence, other groups have proposed the use of geometric invariant features to guide correspondence determination.^{26, 27, 28}

Additionally, a deformation identifying rigid registration (DIRR) has been proposed by Cash et al.¹³ The DIRR provides a marked improvement in surface alignments relative to ICP with respect to the facilitation of deformation compensation algorithms. However, the algorithm relies on a Powell’s method optimization scheme to determine the rigid-body transformation and thus is more time consuming to perform and may not be feasible for intraoperative implementation at the present time. Additionally, the ability of the DIRR to provide reasonable alignments using clinical data has not yet been demonstrated.

The objective of this work is to implement a surface-based registration method that utilizes the homologous, salient anatomical features to ensure convergence to reasonable solutions under conditions of poor initial alignment. Similar to our preliminary work, we propose that these extracted anatomical regions be used to bias both the point correspondence determination as well as guide the traditional ICP via a dynamic weighting scheme such that convergence to an extremum is avoided. Furthermore, we seek to demonstrate that the use of multiple salient features will allow the surface registration algorithm to converge to favorable solutions in the absence of initial pose information. The ability to provide robust, favorable alignments in the absence of an initial PBR presents a significant advancement to the performance of intraoperative image-to-physical space registrations in IGLS.

METHODS

Weighted patch ICP algorithm

The algorithm proposed in this work is an extension to the WGF algorithm proposed by Maurer et al.¹⁶ The homologous anatomical features, or patches, will be used to both bias point correspondence determination as well as play a more significant role in the PBR performed at each iteration of the algorithm. The weighting scheme used to bias the PBR is dynamic over the course of the algorithm where the homologous patch regions play an overwhelming role early in the registration process to ensure the patches are initially aligned and a more supportive role at later iterations in the algorithm.

For the following explanation, let S={s_m} for m=1,…,N_s be the source point set and T={t_n} for n=1,…,N_T be the target point set. Assume that the point sets S and T each contain a number of patch point sets (N_p) that describe the homologous anatomical features that are used to drive the registration. Further, let $\left\{p_m^S\right\}$ and $\left\{p_n^T\right\}$ be integer arrays that contain values along the interval [0,N_p], where an array value of 0 corresponds with the nonpatch point indices and a value greater than 0 indicates the patch point indices (i.e., a value of 1 indicates that the particular point index refers to a point within the first anatomical feature). Let {w_m} be a set of weights where w_m=1 for $p_m^S=0$ and w_m=w_PBR, a dynamic weighting factor used to bias the PBR at each iteration, for $p_m^S>0$.

Point correspondence determination

In order to bias the point correspondence determination for the patch point sets, we introduce a weighting factor w_PC, where 0<w_PC≪1. The weighting factor is used to bias the closest point operator, C_m, by significantly decreasing the Euclidian distances (d) between patch point pairs via the following relationship:

$$d_{m,n}=\{\begin{array}{cc}{w}_{PC}\parallel {\mathbf{s}}_m-{\mathbf{t}}_n\parallel \hfill & \mathrm{if}{p}_m^S=p_n^T\hfill \\ \parallel {\mathbf{s}}_m-{\mathbf{t}}_n\parallel \hfill & \mathrm{otherwise}\hfill \end{array}.$$ (1)

In other words, Euclidean distances identified as being between source and target patch points are multiplied by the fractional weighting factor (w_PC). Since the weighting factor is presumably a very small fraction, the corresponding point found for a source patch point will primarily be contained within the target patch point set. This method of point correspondence determination is different, and to a degree more general, than that proposed by Maurer et al.¹⁶ Specifically, there is no constraint placed on the points that are determined as nonpatch points and these points are allowed correspondence to the identified patch regions as well. Furthermore, the fractional weight factor (w_PC) does not impose a hard correspondence constraint and hence even patch regions are allowed to correspond with other regions under particular circumstances.

Figure 3 shows a pictorial representation of the weighted point correspondence method in the case where only a single patch region is used. Biasing the point correspondence determination alone, however, will not be enough to facilitate a robust surface alignment under conditions of poor initial pose and soft tissue deformation. As described in the next section, biasing the rigid PBR performed at each iteration will provide increased robustness in the proposed algorithm. For clarification, we will use the notation of $C_m^{*}$ to represent the closest point operator biased by Eq. 1.

Figure 3.

Graphical depiction of the weighted point correspondence method. Only the Euclidean distances computed from source patch point to target patch point are biased by the weighting factor w_PC (i.e., dashed lines). The point correspondence determination for nonpatch points is not affected by the weighting (i.e., solid lines). Note that the graphical depiction represents the case where only a single patch region is used.

Weighted point-based registration

Once point correspondence has been determined, the weighted rigid PBR method described by Maurer et al.¹⁶ is implemented. This method seeks to find the rigid-body transformation (Ω) that minimizes the following objective function (f):

$$f\left(\Omega \right)=\sqrt{\sum _{m=1}^{N_M}{w}_m{\parallel C_m^{*}({\mathbf{s}}_m,T)-\Omega \left({\mathbf{s}}_m\right)\parallel }^2},$$ (2)

where {w_m} is a set of weights letting w_m=1 for $p_m^S=0$ and w_m=w_PBR, where w_PBR⩾1, for $p_m^S>0$. The weighting factor (w_PBR) serves to increase the role of the patch points within the determination of the transformation, Ω. A closed form solution for the special case of w_m=1∕N_m for m=1,…,N_m has been presented by Arun et al.²⁹ The solution is based on the singular value decomposition of the covariance matrix of the position vectors in the two spaces. The closed form solution presented by Maurer et al.,¹⁶ which is valid for all w_m>0, is an extension of the aforementioned solution.

In the WGF algorithm, the weights used within the PBR for the geometrical features used in the registration (i.e., w_PBR) remain constant throughout the registration process. We seek to modify this implementation by creating a dynamic scheme by which the patch point weight, w_PBR, is dynamic as the algorithm progresses.

Dynamic weighting scheme

Being that FLE and soft tissue deformation impose error in the accurate selection of homologous anatomical points, the initial alignment provided by the anatomical fiducial based PBR can be quite poor. In order to circumvent incorrect local minima convergence issues, the alignment of the homologous patch regions is made to play a very strong role early in the weighted patch ICP algorithm. However, due to segmentation inaccuracies and the fact that a one-to-one correspondence between source and target patch regions most likely will not exist, it is important that the bias in the PBR toward the patch regions be less significant as the registration continues. In other words, since patch regions identified in the source data will not likely contain the entire target patch point set, biasing the registration too heavily throughout the registration process could lead to convergence to an incorrect local minima. To address this problem we allow for the remainder of the surface data to play a more significant role as the registration proceeds. By employing this dynamic weighting, the patch regions serve as an anchor at later iterations within the algorithm such that deformation will not cause a divergence in the final registration result. The following equation describes the behavior of the dynamic weighting scheme, where w_PBR is described as a function of iteration (i,i⩾1):

$$w_{PBR}\left(i\right)=w_{PBR,\max }{e}^{-\alpha (i-1)}+w_{PBR,\mathrm{base}}(1-e^{-\alpha (i-1)}).$$ (3)

In the above equation, w_PBR,max is the maximum patch PBR weight factor and corresponds to the patch weight at the very first iteration of the algorithm. The weight factor w_PBR approaches w_PBR,base, the baseline patch weight where w_PBR,max⩾w_PBR,base⩾1, as i becomes significantly large. The rate at which w_PBR approaches w_PBR,base is determined by the relaxation constant α, where α∊[0,1]. Decreasing the value of α increases the length of time that the patch region dominates the PBR at each iteration. As the value of α approaches zero the PBR weighting scheme becomes more akin to that proposed by Maurer et al.¹⁶

Phantom validation

Silicon liver phantom and phantom data acquisition

In order to quantitatively compare the developed weighted patch ICP algorithm with the traditional ICP method, the imaging phantom shown in Fig. 4 was used. Poly (dimethyl) siloxane (rubber silicone) was used to fabricate the liver model. The liver model was surrounded by seven Teflon spheres (Small Parts Inc., Miami Lakes, FL), which served as a set of point-based fiducials for the performed experiments. A more detailed description of the imaging phantom can be found in the publications of Cashet al.^{4, 5, 30} Imaging data of the described phantom were acquired using both CT (Mx8000, Phillips Medical Systems, Bothell, WA) and LRS (RealScan 200C, 3D Digital Corporation, Bethel, CT) modalities. The RealScan 200C is capable of acquiring spatially dense 3D point cloud surface representations of 500 lines per scene with as many as 512 samples per line and a spatial resolution on the order of 0.5 mm. The scanner specifications state that the average deviation from planarity is 300 μm at a 300 mm depth and 1000 μm at an 800 mm depth. In addition to the geometrical data, a digital image of the LRS field of view is simultaneously acquired and texture mapped to the point cloud via a predetermined calibration. A detailed validation of the imaging capabilities of the LRS system used has been provided by both Sinha et al.⁹ and Cash et al.^{4, 5}

Figure 4.

Digital photograph (left), raw LRS scan (center) and sample CT slice (right) of imaging phantom. The silicon liver model, located in the center of the phantom, is surrounded by a set of seven white Teflon spheres. These spheres, which can be localized in both LRS and CT image spaces, are used in the determination of the gold standard ICP registration and serve as targets in the robustness studies.

Once imaging data were acquired, the sphere points were localized in the LRS scan using a least squares sphere fitting method described by Ahn et al.³¹ and the sphere centroids were computed in the CT image volume using a region growing algorithm implemented within the ANALYZE software package (ANALYZE AVW Version 6.0, Mayo Clinic, Rochester, MN). Once the fiducial points and surfaces were extracted from both CT and LRS images, a PBR was computed using the seven sphere fiducial points via Horn’s quaternion method.³² This PBR served as an initial alignment registration from which the “gold standard” ICP registration was computed using the entire LRS surface. Additionally, the sphere fiducials were also used as targets for computation of target registration error (TRE), defined by Fitzpatrick et al.³³ in the validation experiments.

Simulated falciform and inferior ridge patch regions were manually selected from the full LRS data, as shown in Fig. 5. The gold standard ICP registration was then used to extract the analogous region on the CT image surface of the liver phantom. The CT image falciform region contained all the points within a 3 mm radius of each of the LRS surface falciform points. This was done to simulate segmentation errors in accurately delineating the homologous patch region on the image surface (shown in Fig. 5). Additionally, only a subregion of the LRS data was used in the robustness studies since the LRS scans acquired intraoperatively very rarely contain the amount of surface information shown in the complete LRS scan. The region was selected based on the authors’ experience of the most scanned regions during the observed surgical procedures (shown in Fig. 5).

Figure 5.

Phantom LRS simulated falciform patch selected from full scan (left) was used to delineate the homologous region in the CT image surface (center). To more accurately simulate the typical LRS surface field of view obtained during surgery, a subregion of the LRS was manually selected for use in the robustness trials (right).

For reference, the number of points contained within the CT image liver surface, simulated falciform region, and simulated inferior ridge region were 106, 661, 2066, and 1393, respectively. The number of points contained within the full and partial liver LRS scans were 34 546 and 12 376 with 1125 and 802 falciform points in the respective full and partial scans. The number of inferior ridge points in the full and partial scans were 404 and 303, respectively.

Phantom data robustness trials

In order to describe the robustness of the proposed algorithm, a series of registration experiments were performed that involved perturbing the LRS data from the gold standard ICP alignment with a random 6°-of-freedom, rigid-body transformation. The random transformations were computed by generating a set of six random parameters (three translation and three rotation). In order to simulate the variety of initial alignments corresponding with those provided by an anatomical fiducial PBR and without performing any initializing registration, two magnitudes of perturbation (termed “small scale” and “large scale”) were utilized. The significance of performing robustness trials over two different magnitudes of perturbation is to test the hypothesis that utilizing salient feature information within the proposed algorithm provides the ability to reliably obtain reasonable surface registrations without the use of anatomical fiducial points to provide an initial pose. By alleviating the need to use an anatomical fiducial based PBR as an initial alignment, a primary error source in the current IGLS registration procedure will have been eliminated.

Three different surface registration methods were used for comparison within the robustness trials: Traditional ICP, patch ICP using a single feature (falciform), and patch ICP using multiple features (falciform and inferior ridge). The large scale and small scale robustness trials were run over 250 perturbations per registration method and the data were compared in terms of sphere TRE and surface root-mean-square (RMS) residuals (i.e., the RMS of the closest point distances between the source and target surfaces) provided by the registration algorithms. The parameters used for the ICP implementation for these trials were a maximum iteration number of 1000 and convergence criterion of 1e⁻⁴ mm RMS residual difference between iterations. The parameters used for the weighted patch ICP registration were as follows: 1000 maximum iterations, w_PBR,max=1000, w_PBR,base=25, w_PC=1e⁻⁴, α=0.01, and a convergence criterion of 1e⁻⁴ mm RMS residual difference between iterations.

A uniformly distributed random number generator was used to supply the rotation parameters (θ^x,θ^y,θ^z) and translation parameters (t^x,t^y,t^z) for the perturbation transformation matrices. For the large scale perturbation trials, the rotation parameters were generated on the interval [−180°,180°] (μ=−0.7±106.1) and the translation parameters were generated on the interval [−200 mm, 200 mm] (μ=−3.4±119.3). For the small scale perturbation trials the intervals for the rotation and translation parameters were set to [−45°, 45°] (μ=0.6±26.6) and [−50 mm, 50 mm] (μ=−1.0±28.9), respectively.

Clinical validation

Clinical image and intraoperative data acquisition

Using an institutional review board (IRB) approved patient protocol, CT or MR image sets and intraoperative data were acquired for six patients undergoing hepatic tumor resections at Barnes–Jewish Hospital in St. Louis, MO. The intraoperative protocol involved a series of preplanned apneic periods during the acquisition to minimize errors in the data due to respiratory liver motion. The apneic periods were part of the IRB approved protocol and were performed at the same point of the respiratory cycle (end-expiration) such that the liver would reside approximately in the same location during each period of data acquisition. Specifically, the intraoperative protocol involved the acquisition of both anatomical point fiducial data using an optically tracked probe (OPTOTRAK 3020, Northern Digital, Waterloo, Ontario) and LRS surface data. Furthermore, the LRS unit used was optically tracked (description of the tracked LRS design provided by Sinha et al.⁹ and Cash et al.^{5, 30}) such that the anatomical fiducial data and the LRS surface data were both acquired relative to the same reference coordinate system.

Clinical data registration experiments

The six clinically acquired datasets were then used in a set of registration trials to determine the effectiveness of the proposed patch ICP registration algorithm. For each dataset, falciform and inferior ridge regions were segmented from both the preoperative image surface and intraoperative LRS data. Comparisons were performed between the results obtained using a traditional ICP method, patch ICP using a single feature (falciform), and patch ICP using multiple features (falciform and inferior ridge). Additionally, registrations were performed under conditions of no initial pose transformation and an initial alignment provided by the anatomical fiducial based PBR. The parameters used for the ICP implementation for these trials were a maximum iteration number of 1000 and convergence criterion of 1e⁻⁴ mm RMS residual difference between iterations. The parameters used for the weighted patch ICP registration were as follows: 1000 maximum iterations, w_PBR,max=3000, w_PBR,base=25, w_PC=1e⁻⁴, α=0.005, and a convergence criterion of 1e⁻⁴ mm RMS residual difference between iterations.

Clinical data robustness trials

Finally, one of the clinical datasets (patient 4) was used to perform robustness tests similar to those described for the phantom data. The particular clinical dataset was chosen for the robustness trials due to the minimal soft tissue deformation in this particular case and the fact that the ICP registration provided a particularly good alignment (shown in Fig. 6), based on visual inspection. For reference, the number of points containing the preoperative liver, falciform, and inferior ridge regions derived from CT images were 57 873, 2220, and 1291, respectively. The number of points in the LRS scan representation of the liver, falciform, and inferior ridge regions for this clinical dataset were 17 848, 594, and 101, respectively.

Figure 6.

Traditional ICP registration results (a) and overlaid image and falciform patch regions (b) for the clinical data used in the robustness trials (Patient 3). The falciform and inferior ridge regions delineated in the intraoperative LRS and preoperative CT data are shown in panels (c) and (d), respectively. Note the large contrast in the accuracy of the alignment in this case than that shown in Fig. 1. The RMS residual for this registration was 3.4 mm.

As in the phantom trials, three different surface registration methods were used for comparison within the robustness trials: Traditional ICP, patch ICP using a single feature (falciform), and patch ICP using multiple features (falciform and inferior ridge). The large scale and small scale robustness trials were run over 250 perturbations per registration method and the robustness data are reported in terms of the RMS residual relative to the gold standard ICP registration. The parameters used for the ICP implementation for these trials were a maximum iteration number of 1000 and convergence criterion of 1e⁻⁴ mm RMS residual difference between iterations. The parameters used for the weighted patch ICP registration were as follows: 1000 maximum iterations, w_PBR,max=3000, w_PBR,base=25, w_PC=1e⁻⁴, α=0.005, and a convergence criterion of 1e⁻⁴ mm RMS residual difference between iterations.

As with the phantom robustness trials, a uniformly distributed random number generator was used to supply the rotation parameters (θ^x,θ^y,θ^z) and translation parameters (t^x,t^y,t^z) for the perturbation transformation matrices. For the large scale perturbation trials, the rotation parameters were generated on the interval [−180°, 180°] (μ=−0.5±105.4) and the translation parameters were generated on the interval [−200 mm, 200 mm] (μ=5.5±113.3). For the small scale perturbation trials the intervals for the rotation and translation parameters were set to [−45°, 45°] (μ=0.4±26.1) and [−50 mm, 50 mm] (μ=0.3±28.5), respectively.

RESULTS

Phantom data robustness trials

The results of the small scale perturbation experiments over all 250 trials for each registration algorithm with respect to both RMS residual and sphere TRE values are summarized in Table 1. For reference, the PBR calculated between the CT and LRS sphere point sets yielded an FRE of 1.4 mm. The gold standard ICP registration based off this PBR gave a TRE of 2.3 mm and an RMS residual of 0.6 mm. Based on the distributions of the TRE values shown, a “failed” registration was defined as that which yielded a sphere TRE value of greater than 5.0 mm. The mean TREs for failed registrations for the ICP and patch ICP using a single feature were found to be 149.8±60.7 mm (N=118) and 256.2±95.8 mm (N=7), respectively.

Table 1.

Summary of results for the small scale perturbation robustness trials using the phantom dataset shown in Fig. 5. The number of successful trials (out of 250), RMS residual and TRE overall trails, and RMS residual and TRE over successful trials are reported for each registration method. A successful trial is determined as that which yields a TRE of less than 5.0 mm. For reference, the gold standard ICP registration for the phantom yielded RMS residual and TRE values of 0.6 mm and 2.3 mm, respectively.

				Successful reg.
Registration method	Success (No.)	Residual (mm)	TRE (mm)	Residual (mm)	TRE (mm)
ICP	132 (52.8%)	3.0±2.9	72.1±81.4	0.6±0.002	2.7±0.2
PICP	243 (97.2%)	0.9±1.15	9.9±44.5	0.6±0.01	2.8±0.2
PICP2	250 (100%)	0.7±0.01	3.0±0.5	0.7±0.01	3.0±0.5

Using the aforementioned criterion to determine registration success, it can be seen in Table 1 that the traditional ICP algorithm had a significantly higher failure rate than the patch ICP algorithm using both single and multiple features. Furthermore, the weighted patch ICP algorithm using multiple patches provided successful registrations over all trials, whereas using a single feature yielded failures for seven trials, suggesting that multiple features is more robust. It is also notable that the average RMS residual and sphere TRE values over the “successful” registrations is higher for the patch ICP method than those provided by the traditional ICP registration.

Table 2 shows a summary of the results from the large scale perturbation robustness trials over all 250 trials with respect to both RMS residual and sphere TRE values. As with the small scale perturbation trials, a failed registration was determined as that which yielded a sphere TRE value larger than 5.0 mm. The mean TRE values for failed registrations for the ICP and patch ICP using a single feature were found to be 240.1±43.5 mm (N=239) and 293.9±24.0 mm (N=127), respectively. Similar to the results for the small scale perturbation trials, the rate of failure of the patch ICP registration method is significantly lower than that of the traditional ICP method and the use of multiple features provides successful registration methods over all large scale perturbation trials.

Table 2.

Summary of results for the large scale perturbation robustness trials using the phantom dataset shown in Fig. 5. The number of successful trials (out of 250), RMS residual and TRE over all trails, and RMS residual and TRE over successful trials is reported for each registration method. A successful trial is determined as that which yields a TRE of less than 5.0 mm. For reference, the gold standard ICP registration for the phantom yielded RMS residual and TRE values of 0.6 and 2.3 mm, respectively.

				Successful reg.
Registration method	Success (No.)	Residual (mm)	TRE (mm)	Residual (mm)	TRE (mm)
ICP	11 (4.4%)	5.4±2.4	229.7±64.7	0.6±0.002	2.5±0.3
PICP	123 (49.2%)	4.6±4.9	150.7±146.8	0.6±0.01	2.9±0.3
PICP2	250 (100%)	0.7±0.003	2.9±0.5	0.7±0.003	2.9±0.5

Clinical data registration experiments

Summaries of the clinical data registration experiments in terms of the RMS residual obtained by the performed registration method and over the six patients are shown in Tables 3, 4. The results are shown for the ICP, patch ICP with single feature, and patch ICP with multiple features using both the anatomical fiducial based PBR initial alignment (Table 4) and with no initial pose transformation (Table 3). In addition to reporting the RMS residuals over the entire surfaces, “feature errors” were computed for each registration result as RMS residuals of the homologous patch regions. For the feature errors, feature 1 indicates the falciform ligament region and feature 2 represents the inferior ridge. The registrations that yielded gross misalignments are indicted with a superscript, which was evaluated via visual inspection. The most notable result shown in Tables 3, 4 is the fact that given no initial alignment, the traditional ICP method was unable to provide a reasonable alignment for any of the clinical datasets. However, the multiple feature patch ICP algorithm yielded reasonable alignment for all cases even without any initial alignment. Furthermore, it is apparent that the patch ICP algorithm with a single patch is not quite as robust as that using multiple patches since the single patch ICP trials yield gross misalignments for two of the patients when no initial pose is provided.

Table 3.

Summary of the registration results for the six clinical datasets using no initial alignment transformation. The results are shown for the ICP, patch ICP registration with a single feature (PICP), and patch ICP registration with multiple features (PICP2) in terms of the RMS residual between the entire surfaces as well as the homologous patch regions. Feature 1 represents the falciform ligament region and feature 2 denotes the inferior ridge region. Grossly misaligned registrations are noted with a superscript ^(†) and were determined by visual inspection.

	RMS residual (mm)			Feature 1 error (mm)			Feature 2 error (mm)
Patient	ICP	PICP	PICP2	ICP	PICP	PICP2	ICP	PICP	PICP2
1	5.2†	4.5	4.7	40.2†	1.7	1.8	107.1†	3.0	1.7
2	5.7†	6.2	7.0	43.9†	3.8	4.9	61.8†	10.9	3.4
3	7.3†	5.5	6.2	86.3†	2.6	4.2	106.3†	12.5	7.2
4	7.5†	10.8†	3.7	64.4†	7.1†	3.5	140.6†	46.3†	5.9
5	11.6†	6.6	6.4	138.6†	3.2	3.5	11.6†	5.4	5.0
6	11.3†	10.8†	5.9	74.5†	4.1†	3.7	37.5†	104.4†	3.3

Table 4.

Summary of the registration results for the six clinical datasets using the anatomical fiducial-based PBR initial alignment. The results are shown for the ICP, patch ICP registration with a single feature (PICP), and patch ICP registration with multiple features (PICP2) in terms of the RMS residual between the entire surfaces as well as the homologous patch regions. Feature 1 represents the falciform ligament region and feature 2 denotes the inferior ridge region. Grossly misaligned registrations are noted with a superscript ^(†) and were determined by visual inspection.

	RMS Residual (mm)			Feature 1 error (mm)			Feature 2 error (mm)
Patient	ICP	PICP	PICP2	ICP	PICP	PICP2	ICP	PICP	PICP2
1	2.8	4.6	4.7	35.9	1.8	1.8	5.0	2.7	1.7
2	5.2†	5.7	7.0	63.5†	3.1	4.9	39.9†	38.6	3.4
3	5.2	5.5	6.2	7.1	2.8	4.2	10.4	11.5	7.2
4	3.4	3.5	3.7	3.9	3.7	3.5	7.1	7.2	5.8
5	3.4	6.5	6.4	30.2	3.1	3.5	5.3	8.6	5.0
6	5.4	5.6	5.9	4.3	2.9	3.7	5.9	6.5	3.3

In addition to reporting numerical summaries, visualizations of two of the clinical data registrations for all are shown in Fig. 7 (patient 1) and Fig. 8 (patient 5). The visualizations shown are the results of the ICP [panels (a) and (b)] and patch ICP using a single feature [panels (c) and (d)] given the anatomical fiducial PBR initial alignment. The results of the patch ICP registration with multiple features given no initial alignment is also shown [panels (e) and (f)] for the two patients. For each of the registrations performed, the patch regions are highlighted in both the preoperative image surface (red) and intraoperative LRS data (blue).

Figure 7.

Clinical results for Patient 1 showing visualizations of the ICP registration (a–b) and patch ICP registration using a single (falciform) patch (c–d) initialized using the anatomical fiducial PBR, as well as patch ICP registration using multiple patches (falciform and inferior ridge) given no initial alignment registration (e–f). The LRS and ICP patches are highlighted for the ICP and patch ICP registrations in (b,d,f). The ICP registration shows an apparent misalignment which is corrected via the proposed method. For reference, the number of points containing the preoperative liver, falciform, and inferior ridge regions derived from CT images were 53 459, 3074, and 1522, respectively. The number of points in the LRS scan representation of the liver, falciform, and inferior ridge regions for this clinical dataset were 16 675, 15 074, and 605, respectively.

Figure 8.

Clinical results for Patient 5 showing visualizations of the ICP registration (a–b) and the patch ICP registration using a single (falciform) patch (c–d) initialized using the anatomical fiducial PBR, as well as patch ICP registration using multiple patches (falciform and inferior ridge) given no initial alignment registration (e–f). The LRS and ICP patches are highlighted for the ICP and patch ICP registrations in (b, d, f). For reference, the number of points containing the preoperative liver, falciform, and inferior ridge regions derived from CT images were 53 413, 3139, and l548, respectively. The number of points in the LRS scan representation of the liver, falciform, and inferior ridge regions for this clinical dataset were 19 900, 923, and 358, respectively.

Similar to that shown in Tables 3, 4, the most notable result is the fact that the patch ICP algorithm using multiple features with no initial alignment was able to provide similar results to those obtained by both the ICP and single feature patch ICP given the anatomical fiducial PBR initial pose. Furthermore, for several of the patients, the patch ICP registration provides a much more reasonable alignment as compared with that provided by the traditional ICP method. Most noticeably, for patient 2 the ICP registration resulted in a gross misalignment of the surfaces, even given the anatomical fiducial PBR initial alignment, where the LRS scan of the anterior liver surface was aligned with the posterior surface (the ICP registration results for patient 2 are shown in Fig. 1). The improvement in the surface alignment provided by the weighted patch ICP algorithm is also visible in the registration for patient 1 shown in Fig. 7. Specifically, the alignment near the umbilical fissure is significantly improved relative to the ICP registration result. Similar to the results for patient 1, the alignment provided by the weighted patch ICP algorithm for patient 5 seems to be an improvement in comparison of that provided by the traditional ICP method (see Fig. 8). This is shown, specifically, by the alignment near the region of the umbilical fissure between segments III and IV of the liver surface. The improved alignments for patients 1 and 2 using the weighted patch ICP algorithm is also supported by the lower feature error results shown in Tables 3, 4.

Clinical data robustness trials

The clinical robustness results for the small scale perturbation trials are summarized in Table 5. For reference, the RMS residual of the gold standard ICP registration in this case was found to be 3.4 mm, which is shown in Fig. 6. Based on the distribution of RMS residuals shown, a failed registration was determined to be one which yielded an RMS residual of greater than 5.0 mm. The mean RMS values >5.0 mm for the ICP and patch ICP using a single feature were found to be 6.6±0.9 mm (N=63) and 15.9±4.8 mm (N=22), respectively. Similar to that shown by the results of the phantom small scale robustness trials, the traditional ICP algorithm was shown to have a higher “failure” rate than both the single feature and multiple feature patch ICP algorithm. Furthermore, over the successful registrations the mean RMS residual provided by the ICP algorithm is lower than either the patch ICP using a single feature and using multiple features. The higher RMS residuals over the successful trials is expected for the patch ICP algorithm based on the fact that utilizing the salient features imposes constraints on the final alignment.

Table 5.

Summary of results for the small scale perturbation robustness trials using the clinical dataset shown in Fig. 6. The number of successful trials (out of 250), mean residual over all trails, and mean residual over successful is reported for each registration method. A successful trial is determined as that which yields a RMS residual of less than 5.0 mm over the entire surface. For reference, the gold standard ICP registration (shown in Fig. 6) yielded an RMS residual of 3.4 mm.

Registration method	Success (No.)	Residual (mm)	Residual (success) (mm)
ICP	187 (74.8%)	4.2±1.4	3.4±5e−4
PICP	228 (91.2%)	4.7±3.8	3.6±0.01
PICP2	250 (100%)	3.7±0.05	3.7±0.1

Table 6 shows a summary of the results of the 250 large scale perturbation trials for the clinical data over the three registration implementations. As with the small scale trials, a failed registration was defined as that which yielded an RMS residual of greater than 5.0 mm. The mean RMS values for failed registrations for the ICP and patch ICP using a single feature were found to be 7.9±2.2 mm (N=220) and 12.0±4.9 mm (N=157), respectively. The results shown in Table 6 provide similar results as those shown in both the phantom and clinical robustness trials. Based on the incidence of the failed registrations, the patch ICP implementation, specifically the one that utilized multiple features, provides a much more robust method with which to achieve reasonable alignments. However, in the large scale trials, the multiple feature weighted patch ICP implementation failed on one of the trials. By modifying the algorithm parameters (w_PBR,max=4000 and α=0.001) for the particular random perturbation transform that resulted in a failure, it was found that the multiple feature patch ICP algorithm was able to achieve a reasonable alignment for this case.

Table 6.

Summary of results for the large scale perturbation robustness trials using the clinical dataset shown in Fig. 6. The number of successful trials (out of 250), mean residual over all trails, and mean residual over successful trials is reported for each registration method. A successful trial is determined as that which yields a RMS residual of less than 5.0 mm over the entire surface. For reference, the gold standard ICP registration (shown in Fig. 6) yielded an RMS residual of 3.4 mm.

Registration method	Success (No.)	Residual (mm)	Residual (success) (mm)
ICP	30 (12.0%)	7.3±2.6	3.3±0.2
PICP	93 (37.2%)	8.9±5.6	3.6±0.01
PICP2	249 (99.6%)	3.7±0.5	3.7±0.01

DISCUSSION

Weighted patch ICP robustness and validation

The data presented from both the phantom and clinical studies provide strong evidence that the proposed weighted patch ICP algorithm is more robust to poor initial alignment than the traditional ICP method. Furthermore, it is reasonable to conclude from this work that by including the multiple features (falciform and inferior ridge) and the correct algorithm parameters, the weighted patch ICP algorithm can provide alignments under virtually any possible initial pose routinely experienced during surgery. The ability to circumvent the need to provide an initial alignment registration is quite powerful in the case of IGLS, since the determination of this transformation is the most error prone step within the current process. As mentioned before, the accurate determination of homologous, rigid anatomical landmarks is complicated in the case of IGLS due to the amount of deformation and nonrigid movement of the liver upon laparotomy and mobilization. A success or failed registration for the phantom robustness trials is much easier to determine than for the clinical experiments since the liver phantom data set includes a set of target points from which the TRE of the transformations can be determined. Being that we do not currently have the ability to acquire accurate subsurface targeting data in a clinical setting, the RMS residual between the two surfaces is the only metric that can be used to evaluate the alignments in the clinical data experiments.

While the RMS residual between two surfaces is not the most objective measure of registration accuracy, it is highly unlikely that registrations resulting in large RMS residuals correspond with reasonable alignments. While it is quite possible that incorrect alignments may still provide small RMS residuals (as shown by the RMS residuals of the ICP alignments for patient 2 in Tables 3, 4), the comparatively large number of high RMS residual alignments resulting from the traditional ICP implementation under conditions of both small scale and large scale perturbation in initial pose (shown in Table 5 and Table 6) suggests that the proposed weighted ICP algorithm is much more robust. Furthermore, the patch ICP registration algorithm provided much improved registrations for three of the sets (patients 1, 2, and 5) of clinical data where the traditional ICP method resulted in obvious misalignments as determined by visualization and further indicated by the significantly lower feature error measurements indicated in Tables 3, 4.

Algorithm parameter selection and optimization

One of the primary advantages of the proposed algorithm is the use of the dynamic PBR weighting scheme described by Eq. 3. This dynamic weight factor allows for the registration to be significantly biased toward patch alignment at early iterations, while utilizing this patch alignment as an anchor at later iterations. The fact that the registration is so heavily biased toward the alignment of the patch regions at the early iterations of the algorithm provides the means by which variations in initial alignment are rendered less significant to the final outcome. Furthermore, by lowering the PBR weight factor of the patch points at later iterations the remaining surface information is utilized to provide a more unbiased alignment of the surfaces. Additionally, the dynamic weighting scheme also compensates for segmentation errors in the delineation of exactly homologous patch regions. Since the nonpatch regions of the surfaces play a more significant role later in the registration process, the registration is given the opportunity to converge to a more globally correct alignment.

For the phantom and clinical robustness trials and registrations performed in this work, all of the algorithm parameters were determined empirically via real-time visualization of the behavior of the proposed registration method and retrospective analysis of the registration results for the sample initial alignments of the phantom and clinical datasets. Ultimately, the factors that dictate the parameters required for the proposed weighted patch ICP algorithm to achieve reasonable alignments are the quality of the initial pose and the relative fraction of source (i.e., LRS scan) feature points to total source points. It stands to reason that the algorithm will require a higher maximum PBR weighting factor (w_PBRmax) and smaller relaxation parameter (α) when the fraction of source patch points is relatively small and∕or the initial alignment is extremely poor.

In terms of parameter determination, the primary difference between the clinical and phantom datasets is the relative fraction of the source (i.e., LRS scan) data that the anatomical features comprise. For example, the salient patch regions (falciform and inferior ridge) for the phantom data comprise approximately 9% of the total source points while these features represent only 4% of the total source points in the clinical dataset used in the perturbation studies. Based on the differences in the relative fraction of patch points to total size of the LRS data between the phantom and clinical datasets, it can be seen that if a smaller fraction of the source data is comprised of patch regions then a larger value of the maximum patch PBR weight factor (w_PBR,max) and smaller value of the relaxation constant (α) are required to achieve similar algorithm robustness.

In order to optimize the parameter selection for a given dataset it is important to take several points into consideration. As discussed in the previous paragraph, the values of the maximum patch PBR weight factor (w_PBR,max) and the relaxation constant (α) are directly dependent on the relative fraction of points that are contained in the patch point datasets. It is important that α not be too small and that w_PBR,base not be too large, less the algorithm be too heavily biased to patch regions at later iterations. Biasing toward the patch regions too heavily throughout the registration process could lead to less optimal alignments since in most cases, the source data (i.e., LRS scan) will not contain data to represent the entire region delineated from the preoperative image set. Optimizing the value for the point correspondence weight factor (w_PC) is a bit more obvious, as the only negative effect of an extremely small value for this factor would be to potentially increase negative effects of oversegmentation of the anatomical features in the LRS data or outliers contained in the source patch data. These effects are more appropriately minimized by conservative segmentation of the salient anatomical features in the LRS data.

Segmentation effects on algorithm performance

While the preliminary data are promising, a number of caveats exist with the proposed algorithm in its current form. In contrast to the ease of accurately delineating the falciform region within the LRS data, the ability to accurately segment the falciform region, based on the surface groove, is highly dependant on patient anatomy, image quality, and the quality of segmentation. As one would expect, if the segmentations of the salient anatomical features are grossly inaccurate, then the algorithm will most likely provide grossly inaccurate alignments. Based on the current implementation, however, favorable results can be facilitated by being a little more conservative in the segmentation of the LRS anatomical patches while being a bit more liberal in the preoperative anatomical feature delineation. As long as homologous target patch points exist for all source patch points (the opposite does not have to be true), the current implementation will not cause a bias toward an incorrect registration.

Concerns regarding intraoperative implementation

Being that the proposed algorithm requires additional point searches to be performed at each iteration of the algorithm, one of the potential concerns for intraoperative implementation is the increase in computation time. In order for the guidance information provided by IGLS to be relevant and useful, the ability to compute the registration must be as fast as possible. In order to address this concern, k−d dimensional trees were used to decrease point search times^{28, 34} for both the ICP and weighted patch ICP implementations used in the aforementioned studies.

To more accurately characterize the effects of the computational overhead imposed by additional point searches, the time to solution for each trial within the robustness studies performed for both the clinical and phantom datasets were recorded. A summary of the timing data is shown in Table 7, which displays the average times to solution both in terms of total time (in, seconds) and in time per iteration (in seconds per iteration). In order to remove bias from failed registrations, only the times to solution for the registrations that were determined as successful (based on the aforementioned criteria) were reported. For reference, the robustness trials were performed on a Dell XPS with Pentium D 3.20 GHZ CPU and 2 GB RAM, which is not unlike what would be used during an IGLS procedure.

Table 7.

Comparative summary of the time to solution of each algorithm under the condition of large scale and small scale perturbations for both phantom and clinical datasets. The reported solution times were averaged over the successful registration runs for each trial and reported both as mean total time as well as mean time per iteration for each algorithm.

	Small scale trials		Large scale trials
Registration method	Phantom (sec∕sec∕itr)	Clinical (sec∕sec∕itr)	Phantom (sec∕sec∕itr)	Clinical (sec∕sec∕itr)
ICP	60.77∕0.24	70.73∕0.37	59.06∕0.24	92.047∕0.37
PICP	76.77∕0.34	88.99∕0.54	108.09∕0.38	141.87∕0.62
PICP2	43.60∕0.36	52.70∕0.51	52.08∕0.38	54.77∕0.54

The results in Table 7 which show the increased computation for the weighted patch ICP algorithm using both single and multiple patch regions normalized per iteration, with respect to the ICP results, are modest. The increase in total computation time imposed by the weighted ICP algorithm using a single patch is only significantly greater than the ICP results for the large scale perturbation experiments. Additionally, the total time to solution for the weighted patch ICP algorithm using multiple patch regions over all trials was lower than the results provided by ICP. While the computation time per iteration is greater for the weighted patch ICP algorithm, when multiple patch regions are used the number of iterations required to reach a given convergence criterion is lowered, thus facilitating faster solution times when compared with ICP.

A second concern of utilizing the proposed method in the clinical setting is the additional time required to delineate the pertinent anatomical features from either LRS data or via digitization with a tracked probe. Since the utilization of the proposed method with two salient features (falciform ligament and inferior ridge) alleviates the need for an anatomical fiducial-based initial alignment, we feel that the manual selection of the salient anatomical features in the LRS data will have a negligible influence on the time and work flow for intraoperative data acquisition. In essence, there will be no increase in the OR time requirements for data acquisition while a considerable increase in algorithm robustness will be achieved. Furthermore, utilization of differential geometry similar to the crest line extraction work published by Monga et al.³⁵ may provide an avenue for the automatic delineation of the salient anatomical features, particularly the inferior ridge regions.

CONCLUSION

The results of the proposed weighted patch ICP algorithm suggest that this method is more robust to poor initial alignments than the traditional ICP based approach. As shown in several of the clinical data sets, the proposed weighted ICP method was able to achieve reasonable alignments under conditions where the traditional ICP method failed. Additionally, the use of multiple anatomical features (i.e. falciform ligament and inferior ridge) showed increased robustness in both the clinical and phantom perturbation trials, provided reasonable alignments for all clinical data sets under conditions where no initial pose transformation was provided. As such, the proposed method, using multiple anatomical features, allows the ability to neglect the use of an anatomical fiducial PBR initial registration used in current IGLS methods. Further, The incorporation of the proposed algorithm does not impose any additional computational time and requires a trivial additional effort, in terms of intra-operative data collection and preoperatively data processing, relative to the current technique.