Non-rigid Registration of Serial Intra-operative Images for Automatic Brain Shift Estimation

Abstract

Measurement of intra-operative brain motion is important to provide boundary conditions to physics-based deformation models that can be used to register pre- and intra-operative information. In this paper we present and test a technique that can be used to measure brain surface motion automatically. This method relies on a tracked laser range scanner (LRS) that can acquire simultaneously a picture and the 3D physical coordinates of objects within its field of view. This reduces the 3D tracking problem to a 2D non-rigid registration problem which we solve with a Mutual Information-based algorithm. Results obtained on images of a phantom and on images acquired intra-operatively that demonstrate the feasibility of the method are presented.

1 Introduction

Image-guided surgery aims at bringing pre-operative information to the surgeon during the procedure. Most often, this involves registering pre-operative images with the patient in the OR. A number of methods have been developed for this purpose but until recently these have involved rigid body registration techniques. Although rigid body techniques have proven clinically useful, there is a body of literature that shows that brain deforms during the procedure [1,2,3]. When this is the case, rigid body transformations are not sufficient to register accurately pre- and intra-operative information. This has lead several research groups to develop methods and techniques that can compensate for intra-operative brain shift. These methods fall into two broad categories. The first involves intra-operative imaging (for instance interventional MR [4,5], ultrasound [6,7], or CT [8]). When available, intra-operative images can be registered to the pre-operative images using a number of non-rigid intra- or inter-modal registration methods developed over the years (see for instance Meyer et al. [9], Rueckert et al. [10]). Although attractive, this solution is only possible at a few sites that have the required imaging equipment. As an alternative, others have proposed to use physical models [11,12]. Displacements measured at the surface of the brain can then be propagated through the entire volume based on these models. Surface displacements can be measured with a tracked probe [13] or, as is the case at our institution, a tracked laser range scanner (LRS) [14]. When a tracked probe is used, fiducial points such as the intersection between sulci or vessels’ branching points need to be identified at the brain surface and their position tracked over time intra-operatively. This is not only tedious but also error prone. Laser range scanners on the other hand offer the possibility to track points on the brain surface automatically and thus estimate their 3D displacement automatically as well. A laser range scanner as the one we use permits the simultaneous acquisition of the x, y, and z physical coordinates of the objects within its field of view and of a 2D RGB picture of these objects. The scanner also provides a mapping between each point whose physical coordinates are acquired and its position in the RGB image. If a series of images with their associated 3D physical coordinates can be acquired and if a correspondence between the pixels in these images can be established, the temporal displacement of any point within the field of view of the LRS can be computed. Thus, tracking brain surface deformation only requires establishing a correspondence between points in a series of 2D RGB images which can be cast as a registration problem. Clearly, as the surgical procedure evolves, the surface of the brain changes. At times these changes may be drastic, e.g., when a resection is made to remove a tumor close to the brain surface or when an incision is made to access a deeper one. To accommodate these changes non-rigid registration methods are called for. One possibility would be to identify homologous points in the images to be registered and compute a transformation between these images based on these points. This approach is not automatic and thus undesirable. In this paper, we propose and evaluate a method that permits the automatic registration of these images. In the remainder of the paper we present the method we have used as well as results we have obtained both with phantom and real images acquired intra-operatively.

2 Methods

2.1 Data Acquisition

The data used in this study has been acquired with a laser-range scanning device (RealScan3D USB, 3D Digital Corp, Bethel, CT, USA). This scanner is mounted on a vibration-damped monopod that is brought into and out of the surgical field-of-view (SFOV) manually. A thorough discussion of the range scanner can be found in [14]. Intra-operative images are acquired as follows. After dural opening, the monopod and scanner are brought into the SFOV and the laser scanning extents (left and right margins) are calibrated to cover the width of the craniotomy. A laser stripe is then passed over the brain’s surface and range data is collected using the principle of optical triangulation. After acquisition, the scanner and monopod are moved out of the SFOV. The entire data acquisition process adds approximately 1.5 minutes per scan to the operating time and has been approved for clinical use by the Vanderbilt University IRB (VUIRB). A 480×640 pixels RGB bitmap image registered to the range data is acquired at the time of scanning.

2.2 Serial Image Registration

Prior to registration, the RGB images are transformed into gray level images. These images are then first registered using a rigid body transformation (three degrees of freedom: rotation and translations in the horizontal and vertical directions). To do so we use a Mutual Information-based method as proposed by Maes et al. [15]. Our implementation of this algorithm permits registration at multiple resolutions. Here we use two levels (240×320 and 480×640 pixels). We estimate the probability density functions required for the computation of the Mutual Information from the joint histogram of the images. We use 32 bins to build these histograms. In a second step, we refine the results obtained after rigid body registration with a non-rigid registration algorithm we have recently proposed (Rohde et al. [16]). This method has been inspired by the work of Meyer et al. [9] and Rueckert et al. [10]. In this approach, the deformation field that registers one image to the other is modeled with a linear combination of radial basis functions with finite support. The similarity measure used to drive the registration process is the Mutual Information between the images. Our method differs from others in several ways. In our approach we can work on an irregular gird, we adapt the compliance of the transformation locally, we optimize our transformation sequentially on separate regions to speed up the process, and we have derived constraint schemes on the transformation coefficients to enforce the topological correctness of the transformation. Space precludes giving detailed information on this algorithm but these details can be found in [16]. Our algorithm computes the final deformation field iteratively across scales and resolutions (in this context, resolution means the spatial resolution of the image while the scale is related to the transformation itself). A standard image pyramid is created to apply the algorithm at different resolutions. At each resolution, the scale of the transformation is adapted by modifying the region of support and the number of basis functions. The final deformation field is computed as the sum of deformation fields computed at a series of levels, with one level referring to a particular combination of scale and resolution. For the images presented herein, we have used two resolutions (120×160 and 240×320 pixels). At the lowest resolution, we used 3 transformation scales (basis functions with region of support ranging from 40 pixels to 24 pixels). At the higher resolution, we use 5 transformation scales (basis functions with region of support ranging from 30 pixels to 8 pixels). As was the case for the rigid body registration algorithm, the probability density functions required for the computation of the Mutual Information are estimated from the joint histograms built with 32 bins.

2.3 Data Sets Used in This Study

We have tested our approach on one phantom and three in vivo cases. A silicon impression of a cortical surface mold was used as the scanning phantom. This phantom was placed in a clamp and scanned three times. The first time the phantom was not compressed, the second it was compressed from the top, and the third it was compressed from the top and the bottom. The three in vivo cases each involve a pair of intra-operative images. In each case the first image has been acquired early in the procedure (a short time after craniotomy) and the second later in the procedure, typically after tumor resection. Figure 1(a) shows the phantom in its custom made clamp. Figure 1(b) and 1(c) show one of the in vivo cases. Panel 1(b) is the early image, panel 1(c) the later image. Note the large whole in the image shown on panel 1(c) that is the site of the resection.

Fig. 1.

Example of 2D images used in this study.

3 Results

Qualitative and quantitative results obtained with the phantom and the in vivo images are presented in figures 2 and 3 and in tables 1 to 4. In figure 2, the left panels show the phantom in its original state. The right panels show the phantom after compression. The top row illustrates compression on one side, the bottom row compression on both side. Sulcal lines have been drawn on the compressed images and copied on the other ones. The middle panels show the results obtained when the images shown on the left panels are registered to the images shown on the right panels. Correct placement of the sulcal lines on the deformed images indicates a good registration between the deformed and undeformed images. The middle images also show a regular grid defined on the undeformed image to which the deformation field is applied to show the smoothness and regularity of the transformation. Seven homologous landmarks (respectively x_i and y_i) also shown on the figure have been identified on the undeformed and deformed images for quantitative evaluation of the method we propose. These points have been chosen to be easily identifiable in the images and repeated selection of these landmarks resulted in a negligible localization error. Quantitative evaluation was performed as follows. The deformation field ( ) found using the previously described method was used to project the points x_i onto the deformed image to find the deformed points.

Fig. 2.

Registration results for the phantom case.

Fig. 3.

In vivo undeformed and deformed images after rigid and non rigid registration. Landmarks used for validation have been superposed as well as corresponding contours.

Table 1.

Phantom registration errors. Left, compression from the top; Right, compression from the top and the bottom. d_in, before registration; ε, after registration.

Landmarks	din[pixels]	ε[pixels]	din[pixels]	ε[pixels]
1	13.00	0.45	2.00	0.41
2	13.00	0.25	1.41	0.38
3	15.00	0.18	7.62	0.38
4	14.00	0.17	4.47	0.40
5	10.00	0.10	1.00	0.10
6	15.00	0.67	8.60	0.36
7	12.00	0.23	4.47	0.33
Mean ± SD	13.14 ± 1.77	0.29 ± 0.20	4.23 ± 3.00	0.34 ± 0.11

Table 4.

Registration error for the third in vivo case; d_in, prior to registration; ε_r, after rigid body registration; ε_nr after non rigid registration.

Landmarks	din[pixels]	εr [pixels]	εnr [pixels]
1	38.60	2.24	0.11
2	39.29	1.00	0.51
3	40.52	1.00	0.44
4	42.72	2.82	0.53
5	40.52	1.00	2.18
6	41.98	2.24	0.24
7	39.56	1.41	2.20
Mean ± SD	40.46 ± 1.47	1.67 ± 0.75	0.89± 0.90

$$x_i^{\prime }=\mathcal{T}(x_i)$$ (1)

The error for each pair of points (ε_i) is computed as the Euclidian distance between the manually selected points y_i on the deformed image and the corresponding transformed points $x_i^{\prime }$ as follows

$${\epsilon }_i=\Vert (y_i-\mathcal{T}(x_i))\Vert$$ (2)

Table 1 shows the registration errors before (d_in) and after (ε) non-rigid registration. Figure 3 shows the results obtained for the three in vivo cases. The top panels show the images acquired early in the procedure and the bottom panels show the second images acquired later. The second row shows the results obtained after rigid-body registration of the images shown on the top to those shown on the bottom. The third row shows the final results obtained after both rigid and non-rigid registration. Sulcal lines have been drawn on the bottom images and copied on the other ones. This figure shows that serial intra-operative images can be very different from each other because of large resections (other factors include the appearance and/or disappearance of surgical instruments within the field of view or the application of clamps). This presents particular challenges to intensity-based registration algorithms as the one we use. In this work we found it necessary to outline manually Regions of Interest (ROIs) to specify regions over which the transformations are computed. The dashed lines shown on the figure define these ROIs. Homologous landmarks have also been selected on the top and bottom images to permit quantitative evaluation of the registration results. Tables 2, 3, and 4 present the quantitative results for the in vivo data sets. In each of these tables, d_in refers to the registration error prior to registration, ε_r is the registration error after rigid body registration, and ε_nr is the registration error after both rigid and non-rigid registration. The large error prior to rigid body registration is due to the fact that the scanner was not placed at the same position for the first and second image acquisition.

Table 2.

Registration error for the first in vivo case; d_in, prior to registration; ε_r, after rigid body registration; ε_nr after non rigid registration.

Landmarks	din[pixels]	εr [pixels]	εnr [pixels]
1	16.13	6.83	0.38
2	33.54	6.93	0.22
3	19.31	7.15	0.25
4	14.21	8.51	0.34
5	17.46	9.99	0.50
6	25.55	5.02	0.54
7	36.77	0.11	0.30
Mean ± SD	23.28 ± 8.90	6.36 ± 3.16	0.36 ± 0.12

Table 3.

Registration error for the second in vivo case; d_in, prior to registration; ε_r, after rigid body registration; ε_nr after non rigid registration.

Landmarks	din[pixels]	εr [pixels]	εnr [pixels]
1	66.29	9.10	0.40
2	65.80	10.23	0.32
3	64.82	11.79	0.31
4	62.80	13.32	2.25
5	61.22	12.08	0.65
6	59.67	11.87	0.51
7	56.22	12.27	0.25
Mean ± SD	62.40 ± 3.64	11.52 ± 1.40	0.67 ± 0.71

4 Discussion

The results presented in this paper indicate that automatic intra-operative tracking of brain motion using a LRS is feasible. Despite large differences in the images due to resection and different viewing angles the approach we propose is robust enough to lead to sub-pixel registration errors. For the in vivo cases, the algorithm still requires manual intervention to delineate regions of interests over which the transformations are computed but these ROIs do not need to be delineated very carefully. Further development will address this issue. Also, the validation performed in this study is incomplete. It evaluates the quality of the 2D registration between the images but it does not quantify the overall 3D tracking error. In a companion paper [17] we evaluate this error for phantom data using an OPTOTRACK 3020 (Northern Digital Inc, www.ndigital.com) localization system and we show sub-millimetric differences between these measurements and the ones obtained with the method we propose. Intra-operative validation of the overall tracking accuracy of this method is ongoing. Should this study confirm our current results it would be an important step toward using LRS technology for real-time intra-operative brain tracking.