Summarizing and Visualizing Uncertainty in Non-Rigid Registration

Abstract

Registration uncertainty may be important information to convey to a surgeon when surgical decisions are taken based on registered image data. However, conventional non-rigid registration methods only provide the most likely deformation. In this paper we show how to determine the registration uncertainty, as well as the most likely deformation, by using an elastic Bayesian registration framework that generates a dense posterior distribution on deformations. We model both the likelihood and the elastic prior on deformations with Boltzmann distributions and characterize the posterior with a Markov Chain Monte Carlo algorithm. We introduce methods that summarize the high-dimensional uncertainty information and show how these summaries can be visualized in a meaningful way. Based on a clinical neurosurgical dataset, we demonstrate the importance that uncertainty information could have on neurosurgical decision making.

1 Introduction

An important, but somewhat neglected, topic in the field of non-rigid registration is: how can we quantify and visualize the registration uncertainty. The importance can be easily understood in the realm of neurosurgery. Important functional areas of the brain are commonly defined in the pre-operative domain. Whenever these eloquent areas are located adjacent to a tumor, it is critical that their location are accurately mapped intra-operatively. Current image guided navigation systems use rigid registration to establish the mapping between the pre- and intra-operative space, but we are beginning to see a shift towards non-rigid registration which is necessary to accommodate for the non-rigid movement of brain-tissue caused by for instance brain-shift. Conventional registration methods estimate the most likely deformation and consequently the most likely location of a functional area. However, non-rigid registration can behave chaotically in the sense that small changes of input parameters can yield dramatically different results. Some results may be categorized as ”registration errors” while others represent viable alternate explanations of the underlying physical reality based on incomplete imaging data. Furthermore, factors such as resection of tissue, degraded intra-operative image quality and blood and edema around a tumor, can all contribute to an increase in the registration uncertainty in the vicinity of tumors. Hence, for a neurosurgeon, the uncertainty in the estimated location of functional areas can be just as important information as the most likely estimate of the location.

Few authors have attempted to quantify the full uncertainty of the estimates produced by non-rigid registration methods. Hub et al. [1] developed a heuristic method for quantifying uncertainty in B-spline image registration by perturbing the B-spline control points and analyzing the effect on the similarity criterion. Kybic [2] consider images to be random processes and proposed a general bootstrapping method to compute statistics on registration parameters. It is a computationally intensive approach, but is applicable to a large set of minimization based registration algorithms. In [3], Kybic assumes normally distributed registration parameters and use the Hessian of the similarity criterion to compute confidence intervals of the registration parameters. Some reported applications of uncertainty estimation in other medical applications include; visualization of uncertainty in image guided needle insertions [4], uncertainty in dose control in radiation oncology [5] and determining uncertainty in fiber orientation of diffusion tensors [6].

In contrast to the frequentist approach of Kybic [2], we presented in [7] an elastic non-rigid Bayesian registration framework that estimates the posterior on both deformation and elastic parameters. We were, firstly, interested in distinguishing different tissue types based on the posterior distribution on elastic parameters, secondly, in analyzing how simultaneous estimation of elastic and registration parameters could potentially improve registration results.

The emphasis of this paper is that, 1) non-rigid registration results come with a level of uncertainty and 2) conveying the uncertainty, and not only the most likely estimate, is clinically important. We characterize the posterior distribution on deformation parameters with the framework in [7] and show several techniques for summarizing and visualizing the uncertainty of the estimated posterior distribution.

2 A Bayesian Framework for Elastic Image Registration

This section provides a short summary of the registration framework in [7]. Let f(x); m(x); x ∈ Ω be a fixed (pre-operative) and moving (intra-operative) d-dimensional image respectively with Ω ⊂ ℝ^d. We estimate, with a Bayesian model, a deformation t(x); x ∈ Ω, such that m(t(x) + x) is similar to f.

2.1 Bayesian Generative Model

Both the moving image and the deformation are modeled as random variables, while the fixed image is a model parameter. This leads to the following joint probability model, p(m, t) = p(m|t)p(t), where we assume that the observed moving image is dependent on the deformation. According to the theorem of conditional probability, we can write the posterior as:

$$\mathrm{p}(\mathbf{t}|\mathbf{m})=\frac{\mathrm{p}(\mathbf{m},\mathbf{t})}{\mathrm{p}(\mathbf{m})}=\frac{\mathrm{p}(\mathbf{m}|\mathbf{t})\mathrm{p}(\mathbf{t})}{\mathrm{p}(\mathbf{m})}.$$ (1)

We model the likelihood, p(m|t), with a Boltzmann distribution using a temperature, T_s, and a Sum of Squared Difference (SSD) similarity energy function. Similarly, the prior on the deformation, p(t), is modeled with a Boltzmann distribution with temperature, T_r, and a linear elastic energy function. The elastic energy acts as a regularizer on the deformation and is controlled by the Lamé parameters µ and λ. In [7], we were predominantly interested in generating posterior distributions on the elastic parameters, while this work focuses on the uncertainty of the deformation parameters. Hence, we keep the elastic parameters fixed as model parameters. The energy functions were computed with a linear elastic Finite Element (FE) model.

2.2 Markov Chain Monte Carlo (MCMC) Simulation

Unfortunately, it is not possible to analytically compute the posterior in Eq. (1), nor feasible to draw deformation samples directly from it. A common approach to generate samples from intractable posterior distributions is by way of the Metropolis-Hastings (MH) algorithm [8]. Candidate samples are drawn from a simple proposal distribution, t* ~ N(t, σ_t), in our case a Normal distribution, and accepted or rejected according to an acceptance criteria which guarantees that we generate random samples from the posterior distribution. In contrast to [7], where deformations were only sampled for boundary nodes, we sample deformations for all nodes in the FE-mesh. Consequently, we build up a posterior distribution on deformations, as well as marginal posterior distributions on the movements of individual nodes.

The sampling can easily be extended to reject samples that lead to improbable configurations of the deformation field, for example by rejecting samples that move a node in the mesh into an improbable region, or rejecting samples that lead to folding of the elements.

3 Uncertainty Visualization in Non-Rigid Registration

From the marginal posteriors on node deformations we can estimate the most likely deformation, as well as the uncertainty of this estimate. The most likely deformation is a simple object and is easily visualized, e.g. in the form of deformed versions of the pre-operative image data. Conversely, the full posterior on deformations is a high-dimensional object and is difficult to visualize. In this section we present; 1) ways of summarizing the uncertainty using robust statistics and, 2) how these low-dimensional uncertainty summaries can be visualized.

3.1 Inter-Quartile Ranges

The difference between the third and first quartile of a distribution, often called the inter-quartile range (IQR), is a robust statistic that conveys the dispersion of a distribution. It is robust in the sense that it provides meaningful information even for non-Gaussian distributions. We propose to use IQRs of the marginal distributions as a measure of uncertainty and visualize them, either as ellipsoids, or as a scalar map:

• IQR Ellipsoids At each node in the FE-mesh we compute the IQRs of the x, y and z samples and fit an ellipsoid to these ranges. An IQR ellipsoid provides a 3D view of the extent of the marginal distribution on deformations at this node.

• Maximum IQR Scalar Map Sometimes it is easier to interpret a scalar measure of the statistical dispersion. A maximum IQR scalar map can be generated by visualizing the maximum of the three IQRs (there is one IQR for each dimension) at a point. This measure, however, is not able to convey any bias towards dispersion in a single dimension.

3.2 Summarizing and Visualization of Marginal Distributions

In neurosurgery, a non-rigid mapping between the pre- and intra-operative image spaces is established mainly to update the intra-operative space with functional information from the pre-operative space. The functional information is commonly in the form of either a surface model outlining eloquent functional areas extracted from functional Magnetic Resonance Imaging (fMRI), or streamlines describing fiber tracts extracted from Diffusion Tensor Imaging (DTI). For each deformation sample, we deform both the surface models and line data into the intra-operative space. If we draw N samples to characterize the posterior, then we have N deformed versions of the pre-operative models in the intra-operative space. Depending on whether we have surface models (fMRI) or line data (DTI), we apply different techniques for reducing the high-dimensional data to a low-dimensional space that is more suitable for visualization:

• fMRI: Marginal Volume We construct a marginal integer image volume, or a histogram volume [9], such that for each deformed surface model (fMRI activated volume) we increment the value at each voxel that is inside the surface model and divide by N. This is the probability for a voxel to be in the fMRI activation volume.

• DTI: Marginal Visitation Volume For the deformed fiber tracts we use a technique called visitation count [6]. A visitation count volume is constructed by tracing the deformed tractography streamlines through the volume and incrementing values in each voxel a streamline crosses. This is the marginal distribution that a fiber crosses a voxel.

• Marginal Confidence Bounds Given a marginal volume, we can extract iso-contours and thereby render marginal confidence bounds. For instance, the 25% confidence bound is the contour where voxels outside the bound were inside the deformed model in less than 25% of the samples.

The marginal volume and the marginal visitation volume can be visualized either using volume rendering to provide a 3D view of the uncertainty, or with a colormap that conveys the marginal density overlaid on a 2D slice of the anatomical data.

4 Results

From registration of a clinical neurosurgical dataset, we present results in the form of visualizations of the registration uncertainty. The dataset consists of pre-operative anatomical MRI, fMRI and DTI, and a post-operative anatomical MRI used as a proxy for an intra-operative image of a patient with a large glioma lesion in the left frontal region near speech areas.

4.1 Dataset

The anatomical images consisted of a pre-operative 512×512×144 T1 MR image with spacing (0.5mm, 0.5mm, 1.0mm) and a post-operative 256×256×130 T1 MR image with spacing (0.94mm, 0.94mm, 1.4mm) acquired two days post surgery. We extracted tractography streamlines from DTI images and an iso-surface from an fMRI activation volume of the language center located closest to the tumor. A FE mesh covering the brain was constructed with 933 nodes and 4262 tetrahedral elements. The dataset can be seen in Fig. 1. Notice that the post-operative image is contrast-enhanced which makes it difficult to identify the size of the resection due to the bright appearance of blood and edema around the resected area.

Fig. 1.

[a b c; d e]: Dataset. (a)–(b) Pre- and post-operative image. (c) A 3D composite view of the pre-operative volume augmented with the tumor (blue), speech area (red) and the diffusion fiber tracts. (d) Absolute difference image between the two corresponding slices in (a)–(b) before registration. (e) Absolute difference image after registration. Notice the large intensity disagreement around the tumor because of edema and blood. Also notice that, due to contrast enhancement, blood vessels are bright in the post-operative image but dark in the pre-operative image which might lead to an increase in registration uncertainty.

4.2 Deformation Parameter Estimation

To speed up computations, three parallel MCMC samplers, each generating approximately 11 samples per second, were used. We assumed the chains had mixed, and samples could be pooled together, when the potential scale reduction [8] dropped below 1.2 for each scalar estimand. The elastic parameters were set to λ = 25; µ = 1, and the temperatures and the size of the jumping kernels (T_s = 1.5, T_r = 800 and σ_t = 0.03) were set to achieve an acceptance rate of approximately 25%. We let each sampler generate 300 000 samples, discarded the first 100 000 samples, used a thinning factor of 10 and pooled together the remaining samples for a total of N = 60 000 samples that were used for the estimations. The difference image between the pre- and intra-operative image after registration using the most likely deformation can be seen in Fig. 1(e).

4.3 Uncertainty Visualization

In Fig. 2 we visualize IQRs computed from the marginal posteriors in the form of scalar maps and ellipsoids. Notice that the uncertainty is large around the tumor site. The uncertainty, as well as the most likely location of the deformed functional information, is visualized in Fig. 3 using marginal volumes, marginal visitation count volumes and marginal confidence bounds. In Fig. 3(a) we see that the most likely boundary of the activated functional area is far from the area of resection, however, the probability map in Fig. 3(b) shows that there is some probability that parts the functional area might have been touched during surgery. This suggests the importance of uncertainty visualization in surgical decision making.

Fig. 2.

[a, b, c]: Inter-quartile visualizations overlaid on the post-operative image. (a)–(b) Axial and coronal maximum IQR maps. Pink areas have maximum IQR of 7.5 mm which decreases down to 1.7 mm for red. Notice the large maximum IQRs around the tumor. We suspect the reason is twofold: 1) the large intensity difference around the tumor due to blood and edema and 2) after resection there is no longer a one-to-one correspondence between the pre- and post-operative image. (c) Composite visualization of intersecting axial, coronal and sagittal planes overlaid with the maximum IQR map and a few IQR ellipsoids. Notice the elongation of the ellipsoid at the tumor.

Fig. 3.

[a, b, c; d, e]: Marginal volumes. (a) Sagittal slice with the most likely estimate of the fMRI activated area outlined with a white contour. (b)–(c) Close up of (a). In (b) we render the marginal fMRI activation volume using a colormap. The probability density decreases across the iso-contours from purple to red. (c) Rendering of the marginal quartile bounds. The most likely estimate in (a) is close to the 50% marginal quartile bound. (d) A coronal slice overlaid with marginal confidence bounds of the fMRI activation area as well as the density colormap of the marginal visitation count volume. (e) A composite view of three orthogonal planes, with volume rendering of the marginal visitation count volume and the density of the marginal activation volume as a colormap. In the conventional approach to registration, in which only the optimal deformation is presented, the display would only show (a), which would give the impression that the resection did not touch on the fMRI activated area (the high intensity area is due to blood and edema caused by the resection). However, if we take the uncertainty information in (b) into account, we may conclude that there is moderate probability that the surgeon touched the fMRI activated area. This additional information might have significant impact on surgical decision making.

5 Discussion

While the specific registration uncertainty presented in the examples is clearly a preliminary result, we believe, firstly, that in the future, with increased computational power, better modeling, and more highly developed algorithms, it will be practical to obtain dense information about the uncertainty of the estimated registration, and secondly that it will be important to share that information with the surgeon in a meaningful way. Our goal in this paper has been to explore the latter issue, and to promote the importance of these issues to the research community. We introduced several visualization methods to convey registration uncertainty, and the experiments show that it can be important to convey this information, especially for functional areas located close to tumors. Some comments from neurosurgeons that were shown the uncertainty visualizations were: 1) this is really important information that should be integrated in their surgical guidance systems and 2) the slice renderings of the marginal volumes are very informative (e.g. Fig. 3(b)) because they convey information similar to the original fMRI activation volumes which clinicians know how to interpret. Future work should include the functional uncertainty in the registration model.