PAIRWISE REGISTRATION OF IMAGES WITH MISSING CORRESPONDENCES DUE TO RESECTION

Abstract

Registration of images with missing correspondences, such as in the alignment of preoperative and postresection brain data, is a difficult task. To simplify this problem, we introduce an indicator map to segment valid correspondence regions from areas with missing data. The registration problem is posed in a marginalized maximum a posteriori (MAP) estimation framework in which the transformation and correspondence regions are simultaneously estimated using the expectation-maximization (EM) algorithm. The E-step calculates the weights of the possible indicator maps while the M-step updates the transformation. A spatial prior based on principal component analysis (PCA) is used to guide indicator map selection. We demonstrate the promise of our approach on synthetic and real data by comparing results using our algorithm to those from a standard non-rigid registration method.

1. INTRODUCTION

To make meaningful comparisons between medical images, they must first be aligned according to some registration process. In particular, the registration of preoperative and postresection brain data is challenging due to nonlinear deformations and missing correspondences in the resection volume. While a myriad of non-rigid registration methods exist (see [1]), these would likely fail to produce accurate results due to the assumption of a one-to-one correspondence of features between the images.

Some have improved the alignment of images with large differences by creating a hybrid registration similarity measure. In [2], a metric that weights voxel and semi-automatic point and surface information was defined to better register pre- and post-resection MR images of the brain. A similar method in [3] combined intensity and feature information and considered large variations in cortical anatomy as outliers in the robust point matching procedure. Before registering a pair of images, both methods require a segmentation step to extract the points or surfaces to be matched.

Other approaches directly model the vast changes between images. To aid registration of pre- and post-tumor brain data, biomechanical models have been used to simulate brain deformation due to tumor growth [4]. The correspondence problem in the alignment of pre- and post-contrast enhanced images was handled in [5] by first “de-enhancing” the contrast image before registration. In [6], a more general registration algorithm was presented which included an explicit model for missing or partial data. They assumed a priori that a voxel in the source image was equally likely to correspond to a point in the target or belong to an outlier model for missing data.

The key idea behind our proposed method is that, given a segmentation of the regions of valid correspondence, image alignment would be a simpler task; we could use one of the many standard registration algorithms which assume that all voxels in the source have a correspondence in the target. On the other hand, if the images are already properly aligned, we could easily determine which regions of the images do not match and label those voxels as missing correspondences.

Thus, our approach is to simultaneously estimate both the correspondence regions and the registration parameters. We use a joint segmentation and registration method similar to [7]. Here, we introduce an indicator map into the registration problem to segment brain tissue in the postresection image that has a corresponding match in the preoperative image. The indicator map allows us to incorporate different models for observing the data under different correspondence assumptions. We pose the registration problem in a Bayesian framework and search for the MAP estimate of the transformation parameters with the help of an indicator map spatial prior.

2. METHODS

2.1. Registration Estimation Framework

Given a postresection image U and a corresponding preoperative image V , the goal is to find the optimal transformation T mapping each voxel x in U to corresponding point T(x) in V . In a MAP estimation framework, we search for

$$\widehat{T}=\underset{T}{\text{arg max}}\log p(T\mid U,V).$$ (1)

We now define a “hidden” indicator map I which takes the value 0 at voxels in U which have no correspondence in V and takes the value 1 at voxels in U which have valid feature information present in V . We then rewrite (1) by marginalizing over the indicator map:

$$\widehat{T}=\underset{T}{\text{arg max}}\log \left[\sum _{I}p(T,I\mid U,V)\right].$$ (2)

Solving this optimization problem using the EM algorithm leads us to the following update equation for the transformation at the (k + 1)th iteration using the transformation T^k from iteration k:

$$\begin{array}{cc}\hfill T^{k+1}=& \underset{T}{\text{arg max}}{E}_{I\mid U,V,T^k}[\log p(U,V\mid T,I)\hfill \\ \hfill & +\log p(T\mid I)+\log p\left(I\right)].\hfill \end{array}$$ (3)

Assuming we have a set M of all the possible indicator maps I_m to choose from, we can expand the expectation in (3) as

$$\begin{array}{cc}\hfill T^{k+1}=& \underset{T}{\text{arg max}}\sum _{m\in M}p(I_m\mid U,V,T^k).\hfill \\ \hfill & [\text{log}p(U,V\mid T,I_m)+\text{log}p(T\mid I_m)].\hfill \end{array}$$ (4)

In our instance of the EM algorithm, the E-step computes the probability p (I_m | U, V, T^k). To calculate these weights, this probability can be rewritten using Bayes’ rule as

$$p(I_m\mid U,V,T^k)=\frac{p(U,V\mid T^k,I_m)p(T^k\mid I_m)p\left(I_m\right)}{\sum _{m\prime }p(U,V\mid T^k,I_{m\prime })p(T^k\mid I_{m\prime })p\left(I_{m\prime }\right)}.$$ (5)

The M-step then updates the transformation parameters using (4). From (4) and (5), we see that we need probability models for the likelihood of the images given the transformation and indicator, the prior on the transformation parameters given the indicator function, and the prior on the indicator map.

2.2. Likelihood Models

The likelihood p (U, V | T, I) is essentially the similarity metric of the registration algorithm. Assuming the voxels in image U are independent, we can write the likelihood as

$$\begin{array}{cc}\hfill p(U,V\mid T,I)=& \prod _{\text{x}:I\left(\text{x}\right)=0}p(U\left(\text{x}\right),V\left(T\left(\text{x}\right)\right)\mid T,I\left(\text{x}\right)=0).\hfill \\ \hfill & \prod _{\text{x}\prime :I(\text{x}\prime )=1}p(U(\text{x}\prime ),V\left(T(\text{x}\prime )\right)\mid T,I(\text{x}\prime )=1).\left(6\right)\hfill \end{array}$$ (6)

We thus need to define two likelihood models: one for when the indicator function labels voxel x in U as missing corresponding data, and one for when the indicator function labels x as having a valid correspondence in V .

While our algorithm could accommodate any similarity metric, here we present two possibilities. For one model, we assume the intensities in U and V are directly comparable and employ a squared distance metric. If a voxel in U is labeled as missing data, we assume its intensity can be aligned with any intensity in V with equal probability. For a voxel labeled as having a valid correspondence, the intensities in U and V should match, so we model the difference U(x) − V(T(x)) as following a normal distribution with 0 mean and a standard deviation of σ. To summarize, the likelihood under each indicator function condition is

$$p(U\left(\text{x}\right),V\left(T\left(\text{x}\right)\right)\mid T,I)=\{\begin{array}{cc}1∕c\hfill & ,I\left(\text{x}\right)=0\hfill \\ \frac{1}{\sqrt{2\pi }\sigma }\text{exp}\left(-\frac{{[U\left(\text{x}\right)-V\left(T\left(\text{x}\right)\right)]}^2}{2{\sigma }^2}\right)\hfill & ,I\left(\text{x}\right)=1\hfill \end{array}\phantom{\}}$$ 7

where c is a constant representing the number of intensity levels in the image.

Another similarity metric is the correlation coefficient (CC), which assumes a positive linear relationship between the intensities in the two images. In this case, for voxels in U labeled as missing data, we assume their intensities are not correlated with those in V and use a uniform distribution. The likelihood for voxels labeled as having valid correspondences is modeled as having higher probability with higher CC. Thus, the probability models based on CC are

$$p(U\left(\text{x}\right),V\left(T\left(\text{x}\right)\right)\mid T,I)=\{\begin{array}{cc}k\hfill & ,I\left(\text{x}\right)=0\hfill \\ \frac{1}{Z}\text{exp}\left(\rho \right)\hfill & ,I\left(\text{x}\right)=1\hfill \end{array}\phantom{\}}$$ (8)

where Z is a normalizing constant, ρ is the CC computed using only those voxels whose indicator function has value 1, and $k=\frac{1}{Z}\text{exp}\left(0\right)$.

2.3. Prior Models

To design a model for p (T | I), we first define the registration parameters. Our transformation model uses free-form deformations (FFDs) based on uniform cubic B-splines as in [8]. Thus, the transformation parameters consist of the control points t_i of the B-splines. We assume these control points are independent and that brain tissue will deform more in areas nearer the resection than farther away. We thus model component t of point t₁ as t | I_m ~ N (μ, σ² (d_i)), where μ is the starting location of the point on the uniform grid and ${\sigma }^2\left(d_i\right)\propto \frac{1}{d_i}$ depends on the distance d_i between control point t_i and the boundary of resection in map I_m.

Finally, we address the indicator map prior. Assuming we have a training set of pre- and post-operative image pairs with resections performed in similar areas of the brain (such as in temporal lobe epilepsy), we can construct a spatial prior on the indicator maps. We align the images using an affine transformation and segment the resection and remaining tissue volumes. Our prior utilizes PCA, following the methods in [9] and [10]. Keeping the first q modes, we model an indicator map using a q-dimensional vector w whose components are the weights w_i for each mode. Assuming a normal distribution on a map I represented by w, the prior probability p(I) is computed using w ~ N (0, Σ_q), where Σ_q is a diagonal matrix of the largest q eigenvalues. We then create a set of possible indicator maps by only allowing the weights w_i to fall within a range governed by the eigenvalues.

3. RESULTS

3.1. Synthetic Data

We first created a 2D dataset of 11 preoperative and postresection images by taking a slice from a normal brain, “resecting” some tissue in the left hemisphere by filling in those voxels with an intensity of 0, and warping the image using a physical model. An example image pair is shown in Fig. 1(a) and (b). These deformation simulations give us the ground truth of displacement vectors with which to compare registration results. Since the intensities do not change between the images, we used the likelihood model in (7) based on squared distance. We tested our algorithm by performing leave-one-out cross-validation and kept the first 3 modes from PCA results on each set of 10 images.

Fig. 1.

Sample results for a synthetic image pair. (a) Preoperative image. (b) Simulated postresection image. (c) Difference between pre and post images registered using BIS software. (d) Difference between images registered using our method.

Fig. 1(c) shows the difference image for voxels with valid correspondences between the postresection image and the preoperative image non-rigidly registered using BioImage Suite (BIS) [11]. For BIS registration, we used the squared distance metric and the same spacing for the B-spline control points as in our method. High errors are present especially near the resection. The result using our approach is shown in Fig. 1(d); the overall difference image is flatter, with areas close to the resection showing the most improvement.

The average taken over the synthetic dataset of the minimum, maximum, mean, and standard deviation of the difference between the true displacements and the displacements produced by the registration algorithms for each image pair is shown in Table 1. Our method reduced all displacement error measurements. We performed paired one-tailed t-tests comparing each error metric under the two methods and found the decrease in maximum, mean, and standard deviation of the errors was significant at a level of p < 0.0007. Thus, there is strong evidence that our method produced better registration results than those using the standard algorithm in BIS.

Table 1.

Displacement field errors after registration of synthetic data and p-values for paired one-tailed t-tests comparing the methods

	Min	Max	Mean	Std Dev
Using BIS (vox)	0.0022	4.2555	0.5361	0.6578
Using our method (vox)	0.0012	3.0010	0.3034	0.3360
p-value	< 0.03	< 0.0007	<4E-5	< 2E-6

3.2. Real Data

We applied the registration methods to 7 3D pre- and post-resection brain MR image pairs resampled to 128×128×60. Since each image was acquired using an SPGR sequence, we assumed intensities were correlated and employed the likelihood model in (8) based on CC. Due to the small number of available images, we artificially enlarged the dataset by randomly warping the true indicator maps from the real images using FFDs based on B-splines. It has been shown that artificially enlarging a small training set can improve shape modeling capabilities [12]. We created 5 artificial images per real image, resulting in 30 images for each training set.

Fig. 2 shows an example slice of the results using real image data. Fig. 2(a) is a postresection image, (b) is the warped preoperative image from BIS registration, and (c) shows the result using our method, in which we see better alignment of various sulci. On average, registration using BIS increased CC, computed just for the valid correspondence regions, by 19%, while our method increased CC by 51%. A paired one-tailed t-test comparing the CC after BIS registration and registration using our approach was performed and showed our results significantly improved CC at a level of p < 0.004.

Fig. 2.

Real data examples. (a) Postresection image with resection in right temporal lobe (arrow). (b) Preoperative image registered using BIS. (c) Preoperative image registered using our method. (d) Estimated valid correspondence region overlaid on postresection image.

Lastly, Fig. 2(d) shows the estimated indicator map for valid correspondences overlaid on the postresection image. Note the resection in the temporal lobe is excluded from the indicator map. The estimated valid region leaves out the skull due to using stripped brains during training. The average dice coefficient computed between the estimated and true valid correspondence regions was 0.91, while the average dice coefficient for the overlap of the best reconstructed indicator maps using PCA components and the true regions was 0.92.

4. CONCLUSIONS

We have presented a registration method to handle missing correspondences between preoperative and postresection images and shown it performs better than a conventional non-rigid registration technique. A “hidden” indicator map allowed different probability models to be used depending on whether a voxel was considered to contain missing or valid data. The PCA-based prior, created from a training set of images with resections in similar regions of the brain, guided the indicator map selection.

Future work will explore using indicator maps with more discrete labels or continuous indicator maps to allow variable confidence in region boundaries. In addition, we will incorporate different similarity metrics such as mutual information into the likelihood models to enable registration across imaging modalities. Different prior models will also be investigated. For example, shape models based on independent component analysis may improve performance. Also, while a spatial prior can be useful, it may be difficult to define what it means for the resections to be located in “similar” regions. Furthermore, the estimated indicator map is limited by the training data used to build the library of possible maps; thus, a small training set is problematic since it will not be able to capture the full range of indicator map shapes. We will therefore reformulate the problem to incorporate an intensity-based prior, since it is known that the resection volume will contain dark voxels in the MR images.