PATCH BASED INTENSITY NORMALIZATION OF BRAIN MR IMAGES

Abstract

Magnetic resonance (MR) imaging (MRI) is widely used to study the structure of human brains. Unlike computed tomography (CT), MR image intensities do not have a tissue specific interpretation. Thus images of the same subject obtained with either the same imaging sequence on different scanners or with differing parameters have widely varying intensity scales. This inconsistency introduces errors in segmentation, and other image processing tasks, thus necessitating image intensity standardization. Compared to previous intensity normalization methods using histogram transformations–which try to find a global one-to-one intensity mapping based on histograms–we propose a patch based generative model for intensity normalization between images acquired under different scanners or different pulse sequence parameters. Our method outperforms histogram based methods when normalizing phantoms simulated with various parameters. Additionally, experiments on real data, acquired under a variety of scanners and acquisition parameters, have more consistent segmentations after our normalization.

1. INTRODUCTION

MRI is a non-invasive modality that has been widely used to gain understanding of the human brain. MR image processing techniques, such as segmentation, are needed to understand normal aging as well as the progression of diseases. For almost all MR image processing algorithms, image intensities are used as a primary feature. Unlike CT, MR intensities do not possess any tissue specific meaning. Thus a subject scanned in different scanners or with different pulse sequence parameters, will have variable image contrast between tissues (see Fig. 1 for example). It has been shown that the variability in tissue contrast gives rise to inconsistencies in segmentation [1].

Fig. 1.

Top row shows a subject’s SPGR (spoiled gradient recalled) from three different scanners [2], while the bottom row shows their histograms–scale is arbitrary.

Several intensity standardization algorithms have been proposed to bring MR intensities to a common scale. To normalize a subject T₁-weighted scan to another reference T₁-w scan, histogram matching algorithms try to match the subject histogram to the reference histogram by matching the intensities of manually or automatically chosen landmarks [3] or by minimizing some information-theoretic criteria [4] between the histograms. Another method uses intrinsic MR properties, such as longitudinal and transverse relaxation times (T₁ and T₂) or proton density (P_D), to normalize the subject intensities to a reference volume, acquired with a different pulse sequence [5]. A segmentation based technique has also been proposed to account for the scanner variations, by matching intensity distributions of similar tissues between two images [6]. Landmark based methods usually suffer from the fact that reliable landmarks are sometimes difficult to find as they are either manually chosen or based on a segmentation itself. While, using tissue properties can be difficult as one needs precise knowledge of the imaging equations and pulse sequence parameters, which are often not known or difficult to estimate accurately.

We have previously reported a patch-based intensity normalization method called MIMECS [7] to overcome these problems. In this paper, a generative model based intensity normalization technique is developed independent of but having strong parallels with the idea of coherent point drift [8]. We still use patches instead of intensities of a single voxel as features, because a patch around a voxel contains neighborhood information of that center voxel. We propose a patch matching technique that takes patches from the subject image and finds its best matching patches in a target image. We benefit from not having to choose landmarks and no pulse sequence parameters are required.

2. METHOD

We want to normalize the intensities of a normal subject to a normal atlas . We assume that and are acquired with similar imaging sequences, e.g. either T₁-w SPGR or MPRAGE (magnetization prepared rapid gradient echo), but differ in either or both the scanner or the imaging parameters (e.g., repetition time, echo time, or flip angle). At each voxel of an image, 3D patches are considered and stacked into 1D vectors of size d × 1. There are N subject patches x_i and M atlas patches y_k, where N and M are the number of non-zero image voxels in and , respectively. Define X = {x_i} and Y = {y_j } to be the collection of subject and atlas patches. To match the subject intensities to the atlas, we match X to Y. First, we make sure that the peak white matter (WM) intensities of both and are the same (which provides a rough normalization of the two data sets). We assume that each subject patch x_i is a realization of a Gaussian random vector whose mean is one of the atlas patches—i.e., x_i ~ (y_j, Σ_j) for some j. Then both the best matching atlas patch for each subject patch and the covariances of all atlas patches are defined by maximum likelihood and found using the expectation maximization algorithm (EM) [9], as described in detail below. The normalized image is produced by replacing the center pixel of each observed patch by the corresponding value of the matching atlas patch.

To motivate the key assumption of the Gaussian perturbation, we first observe that the image intensities are related to the intrinsic MR tissue properties, T₁, T₂, $T_2^{★}$, and P_D, by nonlinear imaging equations. However, we can use a linear approximation to write the intensities of a patch as x_i = HΘ(i) and y_j = HΘ(j), where H and H are linear approximations of the imaging equations and Θ(i) and Θ(j) are the underlying MR tissue properties of the patches. If patches x_i and y_j are from the same tissue, we can assume that their MR tissue properties follow a Gaussian distribution, Θ(j), Θ(i) ~ (μ, Ω). Then, ${\mathbf{x}}_i~\mathcal{N}(H_{\mathcal{S}}\mathit{\mu },H_{\mathcal{S}}\mathrm{\Omega }{H}_{\mathcal{S}}^T)$ and ${\mathbf{y}}_j~\mathcal{N}(H_{\mathcal{A}}\mathit{\mu },H_{\mathcal{A}}\mathrm{\Omega }{H}_{\mathcal{A}}^T)$. Similarly, x_i − y_j ~ ((H − H)μ, (H + H)Ω(H + H)^T). We note that it is not feasible to estimate H or H because neither the imaging equations nor the pulse sequence parameters are always exactly known. But we assume and are acquired with “similar” pulse sequences, so we can thus make the assumption that H ≈ H, implying x_i − y_j ~ (0, Ω′). Here, Ω′ is a tissue-specific constant dependent on the nature of Θ(j) and Θ(i), or in essence, the classification of x_i and y_j. However, it is very difficult to estimate intrinsic tissue MR properties (μ and Ω′) since the exact nature of the imaging equations (H or H) is usually not known. We avoid the classification and estimation problem of tissue MR properties by assuming that Ω′ is a characteristic of y_j, thereby creating an M-class Gaussian mixture model with means as the M atlas patches. We note that the same analysis can be extended even if a patch contains more than one tissue class. This idea of an M -class problem was explored previously for a registration algorithm [8].

To find the correspondence between patches, let z_ij be an indicator function having the value one when the i^th subject patch x_i originates from a Gaussian distribution having its mean as the j^th atlas patch y_j with co-variance matrix Σ_j, and is zero otherwise. The probability of observing x_i is written as

$$P({\mathbf{x}}_i\mid z_{ij}=1,{\mathrm{\sum }}_j,{\mathbf{y}}_j)=\frac{1}{\sqrt{2\pi }{\mid {\mathrm{\sum }}_j\mid }^{1/2}}exp\left\{-\frac{1}{2}{({\mathbf{x}}_i-{\mathbf{y}}_i)}^T{\mathrm{\sum }}_j({\mathbf{x}}_i-{\mathbf{y}}_j)\right\}.$$ (1)

For computational simplicity, we make the assumption that Σ_j is a diagonal matrix–i.e. ${\mathrm{\sum }}_j={\sigma }_j^{2}I$, where I is a d × d identity matrix. Assuming the i.i.d. nature of the patches and a uniform prior probability $P(z_{ij}=1\mid {\mathrm{\sum }}_j,{\mathbf{y}}_j)=\frac{1}{M}$ and, the joint probability distribution of all the subject patches is given by

$$P(\mathbf{X},\mathbf{Z}\mid \mathbf{Y},\mathrm{\Theta })=C\prod _{j=1}^M\prod _{i=1}^N{\left(\frac{1}{{\sigma }_j}exp\left\{-\frac{{\Vert {\mathbf{x}}_i-{\mathbf{y}}_j\Vert }^2}{2{\sigma }_j^2}\right\}\right)}^{z_{ij}}$$ (2)

where Z = {z_ij}, Θ = {σ_j: j = 1, …, M}, and C is a normalizing constant. An estimate of σ_j’s and posterior probabilities P (z_ij = 1|X, Y, Θ) can be obtained by maximizing the joint probability using EM.

EM is a two-step iterative process, that estimates the true z_ij’s based on the current estimates of the parameters Θ, and then updates Θ based on the estimates of z_ij. This is described as:

1. E-step: to find new update Θ^(m+1) at the m^th iteration, compute the expectation Q(Θ^(m+1)|Θ^(m)) = E[log P (Z|X, Y, Θ^(m+1))|X, Y, Θ^(m)].

2. M-step: find new estimates Θ^(m+1) based on the previous estimates by solving,

   $${\mathrm{\Theta }}^{(m+1)}=arg{max}_{{\mathrm{\Theta }}^{(m+1)}}Q({\mathrm{\Theta }}^{(m+1)}\mid {\mathrm{\Theta }}^{(m)}).$$

Assuming w_ij = E(z_ij |x_i, y_j, s_j ), the E-step gives

$$\begin{array}{l}{w}_{ij}=P(z_{ij}=1\mid {\mathbf{x}}_i,{\mathbf{y}}_j,{\sigma }_j),\\ =\frac{P({\mathbf{x}}_i\mid z_{ij}=1,{\mathbf{y}}_j,{\sigma }_j)P(z_{ij}=1\mid {\mathbf{y}}_j,{\sigma }_j)}{{\sum }_{k}P({\mathbf{x}}_i\mid z_{ik}=1,{\mathbf{y}}_k,{\sigma }_k)P(z_{ij}\mid {\mathbf{y}}_j,{\sigma }_j)},\\ =\frac{\frac{1}{{\sigma }_j}\phi \left(\frac{{\mathbf{x}}_i-{\mathbf{y}}_j}{{\sigma }_j}\right)}{{\sum }_{k=1}^M\frac{1}{{\sigma }_k}\phi \left(\frac{{\mathbf{x}}_i-{\mathbf{y}}_k}{{\sigma }_k}\right)},\phi (\mathbf{t})=\frac{1}{\sqrt{2\pi }}{e}^{-\frac{1}{2}{\Vert \mathbf{t}\Vert }^2}.\end{array}$$ (3)

The M-step gives the update equations for σ_j s as

$${\sigma }_j^2=\frac{{\sum }_{i=1}^N{w}_{ij}{\Vert {\mathbf{x}}_i-{\mathbf{y}}_j\Vert }^2}{{\sum }_{i=1}^N{w}_{ij}}.$$ (4)

The algorithm is deemed to have converged when the maximum difference between the posteriors in successive iterations is below a threshold. Then the central voxel of the subject patch x_i is replaced with the central voxel of y_t, where t = arg max_j (w_ij ).

For all of the experiments, we use 3 × 3 × 3 patches, thus d = 27. For a 256 × 256 × 199 MR brain image of resolution 1 mm³, M and N are typically ≈ 10⁷. It is therefore memory and time intensive to compute w_ij’s for every x_i’s. We observed experimentally that w_ij is very close to zero if ||x_i − y_j|| is large. Following the assumption that Θ(i) and Θ(j) should be the same tissue, we assume w_ij = 0 if ||x_i − y_j|| > δ, which is similar to a non-local criteria [10]. However, in our computation, for each i, we choose its closest D atlas patches to have non-zero w_ij, their indices denoted by Ψ_i, a set of cardinality D. Thus if any atlas index j is not in Ψ_i, we fix w_ij = 0. Eqn. 4 is modified as follows

$${\sigma }_j^2=\frac{{\sum }_{i;j\in {\mathrm{\Psi }}_i}{w}_{ij}{\Vert {\mathbf{x}}_i-{\mathbf{y}}_j\Vert }^2}{{\sum }_{i;j\in {\mathrm{\Psi }}_i}{w}_{ij}}.$$ (5)

3. RESULTS

3.1. Phantom validation

We first validate the algorithm on the Brainweb phantom data [11]. We use flip angle α as the varying imaging parameter of SPGR scans of a normal phantom. Phantoms with α = 30°, 45°, 60°, 75° with noise levels n = 0, 1, 3, 5% are used to normalize to a phantom with α = 90° and 0% noise. Our method is compared with histogram matching and a landmark based method [3] where the landmarks are found using a Gaussian mixture model algorithm. Fig. 2 shows the mean squared errors (MSE) between the atlas and the subjects before and after normalization with these three methods. Clearly the patch based method outperforms the other two for all values of α, and n = 1, 3, 5. For 0% noise, all three methods perform similarly, because the lack of any partial volume or noise makes the choice of landmarks accurate and histogram matching perfect. At higher noise levels, histogram matching becomes dependent on the number of bins and the estimation of landmarks becomes less robust. The atlas and a subject with α = 30° at 5% noise are shown in Fig. 3 the top row, the normalized images and their differences from the atlas are shown in the middle and bottom row, respectively. Clearly, both histogram matching and landmark based normalization leaves a bias in intensities in large regions, e.g., gray matter (GM), while our patch-based method fails only in recovering sharp edges such as at GM to WM transitions.

Fig. 2.

Mean squared errors (MSE) between atlas (α = 90°) and subject images (α = 30°, 45°, 60°, 75°) are shown for various noise levels.

Fig. 3.

Top row shows the atlas (α = 90°) and the subject (α = 30°). Middle row shows the histogram matched, landmark based normalized and patch based normalized images. Bottom row shows their differences with the atlas.

3.2. Experiments on real data

We next used five healthy subjects from the morphometry BIRN [2] calibration study to show the effects of normalization. Each subject has 14 SPGR scans, 7 each with α = 20° and 30° from 4 different scanners. Each scan of a subject was first rigidly registered to its GE 1.5T α = 30° scan and then skull-stripped [12] using the same mask generated from the GE 1.5T α = 30° scan. Then each image is corrected for any intensity inhomogeneity using N3 [13]. For every subject, we normalize every α = 20° scan to the corresponding α = 30° scan. Fig. 4 shows one set of α = 20° and α = 30° scans of a subject along with the normalized image in the top row.

Fig. 4.

Top row shows the subject (α = 20°), atlas (α = 30°) and the corresponding normalized image. Bottom row shows a plot of the histograms of the above three images.

Our method does not seek to match histograms, however it is useful to compare them after normalization. Fig. 4 bottom row, shows that the histogram of the normalized image more strongly resembles the atlas than the original subject image.

As described earlier, segmentations can become inconsistent with the variation of scanners and imaging parameters [1, 5, 6]. We demonstrate that the patch based normalization achieves more consistent segmentation between images acquired with different scanners and pulse sequence parameters. We compare our method with histogram matching and a landmark based intensity transformation [3] where three landmarks on the subject and the atlas are automatically chosen based on a 3-class Gaussian mixture model segmentation. After normalization, we use fuzzy C-means [14] to generate segmentations. We report Dice similarity coefficients in Table 1 between segmentations of the three primary tissues, cerebrospinal fluid (CSF), GM, and WM, and their weighted average. The values are averaged over five subjects, each with seven scans. Patch based normalization produces significantly higher Dice coefficients for GM and WM compared to the other two, while the CSF Dice segmentations are comparable. Histogram matching often decreases the similarity with the original, as it is heavily dependent on the number of bins. Landmark based matching can also deteriorate the segmentation consistency, when a suitable landmark is not found. This is often the case if the image histogram is not multi-modal (e.g., the blue line in Fig. 4). Both Figs. 3 and 4 reveal a smoothing effect on the reconstructed images. Though not unexpected in patch-based methods, this effect needs further investigation.

Table 1.

Average Dice coefficients of hard segmentations are obtained from 35 scans, before and after normalization, comparing our method with histogram matching and a landmark based matching [3].

	CSF	GM	WM	Mean
Original	0.882	0.788	0.927	0.884
Hist. Match	0.881	0.828	0.917	0.891
Landmark based	0.876	0.776	0.926	0.882
Patch based	0.874	0.845*	0.943*	0.902

^*Statistically significantly larger than the other three (p-value < 0.05).

4. DISCUSSION AND CONCLUSION

We have described a patch based MR intensity normalization framework that can normalize scans having similar acquisition protocol but acquired on different scanners or with different acquisition parameters. We validated our algorithm on phantoms and real scans and shown it gives rise to more consistent segmentations after normalization.