MRI RESOLUTION ENHANCEMENT USING TOTAL VARIATION REGULARIZATION

Abstract

We propose a novel method for resolution enhancement for volumetric images based on a variational-based reconstruction approach. The reconstruction problem is posed using a deconvolution model that seeks to minimize the total variation norm of the image. Additionally, we propose a new edge-preserving operator that emphasizes and even enhances edges during the up-sampling and decimation of the image. The edge enhanced reconstruction is shown to yield significant improvement in resolution, especially preserving important edges containing anatomical information. This method is demonstrated as an enhancement tool for low-resolution, anisotropic, 3D brain MRI images, as well as a pre-processing step to improve skull-stripping segmentation of brain images.

1. INTRODUCTION

MRI image quality is often a decisive factor for low-level algorithms such as feature detection, object extraction, and segmentation. Spatial resolution of image pixels is an important measure of image quality, and directly affects the discrimination of important anatomical features in images. Traditionally, a collective approach that tackles the problem of reconstructing a high-resolution image from one or more of the low-resolution observations is termed as super-resolution. Much of the earlier research work in this area has been developed in the transform domain, using (discrete) Fourier and wavelet-transform based methods. In particular, MRI acquisitions usually have a low-resolution in the inter-slice direction, and it is of considerable interest to “fill-in” the intermediate slices. For e.g. the approach of Tsai and Huang [11] first outlined the idea of super-resolution in their seminal paper. Peleg et al. [5] used the iterative back projection scheme to achieve image reconstruction. A hybrid approach by Elad and Feuer [3] uses projections on convex sets (POCS) and the maximum likelihood approaches for both motion-based and motion-free super-resolution. Carmi et al. [2] use sub-pixel shifted MR (Magnetic Resonance) images for high resolution reconstruction. Greenspan et al. [4] combine several low resolution images in the slice-select direction to achieve super-resolution (SR) reconstruction.

While super-resolution methods attempt to exploit the information redundancy in several low-resolution observations of images, at times, only a single low-resolution instance of the image is available. This is sometimes the case in MRI images, where due to economic or health reasons, a patient is scanned only once over a period of time, or the time elapsed between successive scans may be too large to preserve any temporal coherence. This is the idea explored in this paper, where we will focus mainly on the problem of single frame high resolution reconstruction of images. Our approach will be based on a variational model that uses the total variation (TV) norm [8] as a regularizing functional. Recently, Marquina et al. [6] have proposed a new variational model based on the TV norm [8] for super-resolution of multidimensional images. We will follow this approach to solve the more general super-resolution problem using the TV norm as regularizing functional. Our model uses a multi-frame dataset instead of a single image and Gaussian kernels of convolution allowing homogeneous Neumann boundary conditions. This paper is organized as follows: Section 2 outlines the super-resolution model using TV regularization. Section 2.1 outlines the Bregman iterative refinement scheme. Additionally, section 2.2 proposes a new edge-preserving up (down) sampling operator used in the model. Section 3 details the evaluation of the algorithm as a preprocessing step for a well known dataset of over forty brain MRI images, followed by the summary.

2. IMAGE OBSERVATION AND SYNTHESIS MODEL

The low resolution image observation model can be formulated in a standard fashion as a down-sampled degraded version of the original high resolution image. For the purpose of this paper, we will primarily concern ourselves with 3D images, although the method is generally application to multi-dimensional images. We define f to be the observed low-resolution image, and u be the unknown high resolution image to be estimated. Then given a linear down sampling operator D, we can write the observation model as,

$$f=D(h\ast u)+\eta ,$$ (1)

where η is an additive Gaussian white noise with zero mean and variance σ², and h is translation invariant convolution kernel corresponding to the point spread function of the imaging device. Throughout this paper, the kernel is assumed to be Gaussian and given as, $h(x,y,z)={\mathit{Ke}}^{-\frac{1}{2}\left[\frac{x^2}{{\sigma }_x^2}+\frac{y^2}{{\sigma }_y^2}+\frac{z^2}{{\sigma }_z^2}\right]}$, where K is a normalization constant, and σ_x, σ_y, σ_z are variances along the X, Y , and Z directions respectively. The problem in Eqn. 1 is usually solved as a constrained optimization problem that seeks to minimize the regularizer ∫_Ω∥∇u∥²dxdydz, while constraining the noise to be ${\parallel h\ast u-f\parallel }_{{\mathrm{L}}^2}^2={\sigma }^2$. This ensures that the reconstructed image u is free of discontinuities. However in order to recover the edges satisfactorily, Rudin and Osher [8] propose the total variation norm as the regularizing functional. The total variation norm is given as,

$$\mathrm{TV}(u)={\mathit{\int }}_{\mathrm{\Omega }}|\mathrm{\nabla }u|\mathit{dxdydz}$$ (2)

Using the regularizer in Eqn. 2, we can state the single frame image reconstruction model as the minimization problem as follows:

$$\widehat{u}=\underset{u}{\mathrm{argmin}}\{\mathit{TV}(u)+\frac{\mathrm{\lambda }}{2}[{\parallel f-D(h\ast u)\parallel }_{{\mathrm{L}}^2}^2-{\sigma }^2]\}$$ (3)

The Euler-Lagrange formulation for Eqn. 3 can be written as

$$\mathrm{\nabla }·\frac{\mathrm{\nabla }u}{|\mathrm{\nabla }u|}+\mathrm{\lambda }(\stackrel{\sim }{h}\ast S(f)-\stackrel{\sim }{h}\ast (S◦D(h\ast u)))=0$$ (4)

$$\Rightarrow \mathrm{\nabla }·\frac{\mathrm{\nabla }u}{|\mathrm{\nabla }u|}+\mathrm{\lambda }\stackrel{\sim }{h}\ast (\overline{g}-T(h\ast u))=0,$$ (5)

where ḡ = S(f), and the operator T is defined as T = S ◦ D. The Euler-Lagrange equation given by Eqn. 5 can be solved as a time-dependent equation

$$u_t=\mathrm{\nabla }·\frac{\mathrm{\nabla }}{|\mathrm{\nabla }u|}+\mathrm{\lambda }\stackrel{\sim }{h}\ast (\overline{g}-T(h\ast u))$$ (6)

with homogeneous Neumann boundary conditions and an initial condition, u₀ = S(f). The convergence of Eqn. 6 to the steady state yields a reconstructed high resolution image. However if one wishes to recover even finer scales from the reconstructed image, one can use the Bregman iterative refinement procedure [1] as outlined in the next section.

2.1. Bregman Iterative Method

If u₀ is the solution of Euler-Lagrange equation (5), then we have,

$$\mathrm{\nabla }·\frac{\mathrm{\nabla }{u}_0}{|\mathrm{\nabla }{u}_0|}+\mathrm{\lambda }\stackrel{\sim }{h}\ast (\overline{g}-T(h\ast u_0))=0$$ (7)

We will denote the image residual in the high resolution scale by υ₀ as,

$${\upsilon }_0=\overline{g}-T(h\ast u_0)$$ (8)

We now solve the Euler-Lagrange equation for the new image ḡ+ υ₀ to obtain a new solution, which we denote by u₁. Again, the solution u₁ will satisfy

$$\mathrm{\nabla }·\frac{\mathrm{\nabla }{u}_1}{|\mathrm{\nabla }{u}_1|}+\mathrm{\lambda }\stackrel{\sim }{h}\ast (\overline{g}+{\upsilon }_0-T(h\ast u_1))=0,$$ (9)

where the new residual is defined as

$${\upsilon }_1=\overline{g}+{\upsilon }_0-T(h\ast u_1)$$ (10)

and so on. The sequence of images u₀, u₁, ⋯ , u_j , ⋯ are also referred to as Bregman iterates. It is advisable to terminate this procedure when a satisfactory image quality is obtained, otherwise it has a tendency to recover noise after all the finer scales in the image are recovered. This iterative procedure was introduced for image restoration in [7].

2.2. Edge-preserving Up (Down)-sampling operator

There are various choices for the up (S) and down (D) sampling operators used in the observation model in Eqn. 1 and the synthesis model in Eqn. 6 respectively. Especially for images with prominent edges and interfaces, we need an appropriate interpolation operator that preserves these features. Accordingly, we propose a new piecewise-linear up (down) sampling operator that preserves such edges and boundaries. We describe the edge-preserving operator in detail below. The image is defined on a grid x_j = (j − 1)Δx, y_k = (k − 1)Δy and z_l = (l − 1)Δz, where Δx > 0,Δy > 0,Δz > 0 and j = 1, … , n, k = 1, … , m and l = 1, … , p. We define the domain E = [0,A] × [0, B] × [0, C], where A = (n − 1)Δx, B = (n−1)Δy, and C = (n−1)Δz. We consider the grid function u defined as

$$u_{j,k,l}:{ℝ}^3\to ℝ$$

We define the edge-preserving piecewise linear approximation of the grid function u as the function L(x, y, z)∣E_jkl = L_jkl(x, y, z) where the computational voxel E_jkl is given by

$$E_{\mathit{jkl}}=[x_j-\frac{\mathrm{\Delta }x}{2},x_j+\frac{\mathrm{\Delta }x}{2}]\times [y_k-\frac{\mathrm{\Delta }y}{2},y_k+\frac{\mathrm{\Delta }y}{2}]\times [z_l-\frac{\mathrm{\Delta }z}{2},z_l+\frac{\mathrm{\Delta }z}{2}]$$

and

$$L_{\mathit{jkl}}(x,y,z)=u_{j,k,l}+a(x-x_j)+b(y-y_k)+c(z-z_l),$$

where a, b, and c are determined from

$$a=\mathrm{minmod}\left(\frac{{\mathrm{\Delta }}_{-}^x{u}_{j,k,l}}{\mathrm{\Delta }x},\frac{{\mathrm{\Delta }}_{+}^x{u}_{j,k,l}}{\mathrm{\Delta }x}\right)$$ (11)

$$b=\mathrm{minmod}\left(\frac{{\mathrm{\Delta }}_{-}^y{u}_{j,k,l}}{\mathrm{\Delta }y},\frac{{\mathrm{\Delta }}_{+}^y{u}_{j,k,l}}{\mathrm{\Delta }y}\right)$$ (12)

$$c=\mathrm{minmod}\left(\frac{{\mathrm{\Delta }}_{-}^z{u}_{j,k,l}}{\mathrm{\Delta }z},\frac{{\mathrm{\Delta }}_{+}^z{u}_{j,k,l}}{\mathrm{\Delta }z}\right),$$ (13)

where the operations in the term containing derivatives are understood component-wise, and ${\mathrm{\Delta }}_{\pm }^x{u}_{i,j,k}^n=\pm (u_{i\pm 1,j,k}^n-u_{i,j,k}^n)$, ${\mathrm{\Delta }}_{\pm }^y{u}_{i,j,k}^n=\pm (u_{i,j\pm 1,k}^n-u_{i,j,k}^n)$, ${\mathrm{\Delta }}_{\pm }^z{u}_{i,j,k}^n=\pm (u_{i,j,k\pm 1}^n-u_{i,j,k}^n)$, where i, j, k are the indices of the 3D grid. The minmod(d, e) function is defined as,

$$\mathrm{minmod}(d,e)=\frac{\mathrm{sgn}(d)+\mathrm{sgn}(e)}{2}\min (|d|,|e|),$$ 14

where sgn(d) = 1 if d ≥ 0 and sgn(d) = −1 otherwise. The function L_jkl(x, y, z) is defined on the computational voxel E_jkl. We want to up-(down) sample the grid function u with a spatial resolution of h_x > 0, h_y > 0, h_z > 0. Then the up-(down) sampled grid function υ is defined on a new grid υ(q, r, s) for q = 1, … , n_x, r = 1, … , m_y, and s = 1 … , p_z where

$$n_x=\mathrm{floor}\left(\frac{A}{h_x}\right),m_y=\mathrm{floor}\left(\frac{B}{h_y}\right),p_z=\mathrm{floor}\left(\frac{C}{h_z}\right),$$

where floor(d) is the maximum of all integers i such that i ≤ d. The new grid is then defined as x_hq = (q – 1)_hx, y_hr = (r – 1)h_y, and z_hs − (s – 1)h_z. Based on this grid, the function υ is defined as υ (q, r, s) = L(x_hq, y_hr, z_hs).

For the numerical implementation of the algorithm, the Lagrange multiplier λ was chosen to be the maximum value for which the algorithm was stable. It was empirically determined to be λ = 10, and was not changed thereafter. Figure 1 shows results of the reconstruction algorithm for two anisotropic 3D MRI image volumes. The dimensions of the two images were 256 × 256 × 160 image, with voxel widths given by 1 × 1 × 1.25 mm³. The images were first subsampled to half the resolution at 128 × 128 × 80 (2 × 2 × 2.5 mm³) and then super-resolved to a full isotropic 256 × 256 × 160 image with 1 × 1 × 1 mm³ resolution. Results due to sinc interpolation are also shown for comparison. As expected, we can see an improvement in the resolution plus an increase in the detail simultaneously across all X, Y, and Z dimensions. Furthermore the Bregman refined image shows a presence of additional details recovered by the process. In this experiment, we used an anisotropic Gaussian kernel with the variances proportional to the voxel dimensions. Furthermore the grid dimensions for the edge-preserving up sampling and down sampling operators were taken to be $\mathrm{\Delta }x=\frac{h_x}{2},\mathrm{\Delta }y=\frac{h_y}{2},\mathrm{\Delta }z=\frac{h_z}{2}$, where h_x, h_y, h_z are the voxel dimensions of the appropriate up sampled or down sampled image.

Fig. 1.

Comparison of 3D MRI images with the super-resolution and Bregman refinement scheme for two 3D MRI images. The odd rows shows a mid-axial slice of the 3D image. The even rows show a volumetric rendering of the MRI image.

3. PRE-PROCESSING ENHANCEMENT FOR SKULL STRIPPING OF MRI IMAGES

Skull stripping is a process that removes unwanted anatomical features from the image. For e.g., whenever any patient undergoes a brain MRI scan, the resulting volumetric image often shows the skull cap, eye sockets, sometimes the cerebellum, or even the brain stem. Often for diagnostic or research purposes, the physician is only interested in looking at the brain tissue comprising of the gray/white matter. For this purpose the MRI scan is processed to “skull-strip” unwanted anatomical features from the image. Even though there are a wide variety of automated methods that perform skull stripping of MRI images, not all of them yield satisfactory results on a wide range of image data sets. Such methods often require manual intervention to fine tune the parameters of the algorithm.

In this section we demonstrate the usefulness of our algorithm as a pre-processing step for skull stripping of brain MRI images. We select the LONI LPBA40 (LONI Probabilistic Brain Atlas) population atlas as our test dataset. The LPBA40 atlas (www.loni.ucla.edu/Atlases/LPBA40) is a series of maps of brain anatomic regions produced from a set of whole-head MRI of 40 human volunteers. Each MRI is manually delineated to identify a set of 56 structures in the brain, most of which are within the cortex. These delineations are then transformed into a common atlas space to produce a set of co-registered anatomical labels. Additionally, the 3D volumes contained within this data set also represent intensity averages of the co-registered skull-stripped MRI volumes. For the experiment, we used the Brain Extraction Tool (BET) [10] as a tool for skull stripping MRI images. As a first step, before using BET, we processed the entire dataset using our super-resolution algorithm to enhance edges and anatomical boundaries. We then used BET on the processed data set to obtain skull stripped versions of images. In order to test the accuracy of segmentation, we used an automated online segmentation validation resource [9] to test and evaluate segmentation algorithms. This system accepts segmentation results (segmented Analyze/Nifti images) and compares it with the manually delineated, skull stripped intensity volumes already present in the LPBA 40 data set. It computes false positive and false negative values for each segmentation result, and colors the values at each voxel to obtain a projection map. Since the data volumes are initially processed in their native MRI space, we re-map the false positive and false negative results to a common atlas space based on the LPBA40 mappings. We then average these maps across the 40 subjects. For each cardinal direction (axial, sagittal, and coronal), we then sum the average counts along projections that are orthogonal to the plane of section. Common similarity metrics such as Dice coefficient, and the Jacquard index are then computed. Given two sets A and B, the Jacquard(J(A,B)) and the Dice(D(A,B)) coefficients measure the degree of overlap between them. The Jacquard index is defined as the size of the intersection of the sets divided by the size of their union. The Dice coefficient represents the size of the union of two sets divided by the average size of the two sets. They are given by

$$D(A,B)=\frac{2|A\cap B|}{\left|A\right|+\left|B\right|},J(A,B)=\frac{|A\cap B|}{|A\cup B|},$$

where the magnitudes |A| and |B| depend upon the number of manually classified voxels of an image and vary from one image to the next. Table 1 shows the improvement in the results of skull stripping after the super-resolution algorithm is applied. Visually, these results are shown as colored average projection maps in Fig. 2. The first column labels the method and the measurement, for e.g. StdFP implies standard skull stripped, false positive, whereas SRFP implies superresolved skull stripped false positive. The last three columns show the average projection maps for false-positives and false-negatives. It is noticed that after super-resolution the average brightness intensity in all of the regions is reduced as compared to the standard result. This implies a lowering of the false positive rates, and consequently an improvement in segmentation.

Table 1.

Comparison of average values of metrics for skull stripped volumes before (Std) and after (SR) the application of the super-resolution algorithm.

Metric	Std	SR
Dice coefficient	0.93 ± 0.03	0.96 ± 0.007
Jacquard index	0.87 ± 0.05	0.92 ±0.01
Sensitivity	0.98 ± 0.005	0.97 ±0.01
Specificity	0.97 ± 0.01	0.99 ±0.003

Fig. 2.

Examples of average projection maps for false positive and negative values for the segmentation result for each of the axial, sagittal, and coronal views.

4. CONCLUSION

The proposed method shows an improvement in spatial resolution for anisotropic 3D anatomical MRI images. We have also demonstrated its usefulness as a preprocessing step for improving brain skull extraction from MRI images. The strengths of this approach lie in the i) TV norm as a regularizing functional in the variational model, and ii) a new piecewise-linear up(down) sampling operator that preserves edges. Although, the proposed method works in the spatial domain, the total variation prior does modify the amplitudes of the k-space data. The intuition in the improvement of skull stripped volumes after super-resolution processing stems from the fact that the total variation prior prefers edges, and tends to reduce uncertainty in their estimation. This implies that there is an improvement in the overall precision of the detection of edges will be improved by super-resolution. This translates in the lowering of the false positive rates and is evident from the experimental results.