New Pseudo-CT Generation Approach from Magnetic Resonance Imaging using a Local Texture Descriptor

Abstract

Background:

One of the challenges of PET/MRI combined systems is to derive an attenuation map to correct the PET image. For that, the pseudo-CT image could be used to correct the attenuation. Until now, most existing scientific researches construct this pseudo-CT image using the registration techniques. However, these techniques suffer from the local minima of the non-rigid deformation energy function which leads to unsatisfactory results.

Objective:

We propose in this paper a new approach for the generation of a pseudo-CT image from an MR image.

Materials and Methods:

This approach is based on a dense stereo matching concept, for that, we encode each pixel according to a shape related coordinates method, and we apply a local texture descriptor to put into correspondence pixels between MRI patient and MRI atlas images. The proposed approach was tested on a real MRI data, and in order to show the effectiveness of the proposed local descriptor, it has been compared to three other local descriptors: SIFT, SURF and DAISY. Also it was compared to registration method.

Results:

The calculation of structural similarity (SSIM) index and DICE coefficients, between the pseudo-CT image and the corresponding real CT image show that the proposed stereo matching approach outperforms a registration one.

Conclusion:

The use of dense matching with atlas promises good results in the creation of pseudo-CT. The proposed approach can be recommended as an alternative to registration techniques.

Introduction

Attenuation correction (AC) is an important step in the analysis of PET images. In current PET/CT systems, the attenuation coefficients are calculated directly from CT images; this one provides a good separation between the bone and other tissues, but unfortunately the patient’s body is exposed to additional radiation. PET/MRI systems do not have that drawback, but they cannot distinguish between some tissues, since bone and air show the same signal on MR images. In addition, magnetic resonance imaging (MRI) cannot precisely define the attenuation coefficients of tissues [1].

Atlas-based methods are mostly used for estimating pseudo-CT using conventional MRI sequences. However, these techniques suffer from the local minima of the non-rigid deformation energy function leading to unsatisfactory results, and there is a non-negligible discrepancy in terms of quality of matching between non-rigid registrations techniques [2]. To overcome these issues, a new prediction of pseudo-CT images from a given MR image is proposed in this paper. This approach is based on a stereo matching of pairs of patient’s MRI and MRI atlas images.

After discussing related work, we introduce in section 2, our new dense stereo matching approach based on the shape related coordinates method to encode pixels, and on a local texture descriptor to put pixels into correspondence. In next sections, we describe other descriptors and the data set used in this work. Then, we present a quantitative and qualitative evaluation of our approach.

In order to obtain attenuation correction (AC) maps from MRI for PET/MRI systems, many studies have been reported. In general, there are two main methods to correct the attenuation: the segmentation-based methods and the atlas-based methods.

In segmentation-based methods, MRIs are divided into three classes (bone, soft tissue and air) after that, the attenuation coefficients at 511 kev corresponding to the biological tissues of each class, are assigned to the voxels belonging to that class. Unlike atlas-based methods, the main advantage of this method is its ability to give an interesting result even if there were anatomical differences between subjects (patients). However, the most challenging issue in this particular context is that direct bone segmentation, using MR image intensities, is a very difficult task [3]. A particular pulse sequence such as the ultra-short echo time (UTE) has been proposed recently in order to distinguish between bones and air cavities [4,5]. These authors have obtained an interesting result; nevertheless, they show that the classification of the bone using UTE is still subject to an over- or under-segmentation of bony structures. Some recent studies have exploited the information contained in PET emission data as well as the anatomical information from MRI, to estimate the attenuation maps. They have been mainly suggested to use the information from the time of flight (TOF) [6,7]. In general, we can conclude that the segmentation methods have the disadvantage of just providing discrete AC maps.

In atlas-based methods, the used atlas is a set of MRI/CT or MRI/PET images of the same subject. These methods consist of registering the MRI images atlas to MR image of the patient for whom the attenuation map must be derived. The resulting spatial transformations are then applied to all corresponding images in the CT atlas. Thereafter, a pseudo-CT image is generated, where each voxel is obtained as an average of the intensities of the registered CT atlas images. The pseudo-CT is then used for the correction of attenuation. These methods have the advantage of providing continuous values in addition to robustness, because they are less affected by the acquisition artefacts such as movement. However, the disadvantage of these methods is the possibility of making an erroneous representation of different pathological anatomical regions of the atlas image [8].

The first atlas approaches used PET template as the atlas [9,10]. Hofmann and Al [1] were the first to have developed an atlas approach with CT images, they proposed to combine the atlas approach and pattern recognition, consequently, for each voxel in the image of the patient, a pseudo-CT value based on the nearest voxels of images in the atlas database, is predicted using a Gaussian process regression. This one is considered as the first original approach which offers a significant improvement of the simple registration methods. Schreibmann [11] proposed an alternative method for constructing a pseudo-CT; however, they obtain significant errors on bone structures in the sinus area which presents great anatomical variability. Subsequently, several research works have been conducted in order to improve the quality of pseudo-CT, and to avoid local registration errors. The authors in [12] have recently proposed an improved Hofmann’s method for generating a pseudo-CT in the case of the study of whole body images. In [13], the authors developed a pseudo-CT from the fusion of special sequences of MRI, UTE and FLASH, but they did not give a quantitative evaluation of their approach. In [14], the authors proposed a fast pseudo-CT volume generation, using a group-wise patch-based approach, along with an MRI-CT atlas dictionary, for each voxel of the input MR image, the similarity between the patch containing that voxel and the patches of all MRI images existing in the database is calculated. The pseudo-CT image is obtained as a weighted linear combination of CT values. We notice that this method does not use the atlas-based registration technique, the problem of homogeneity of intensities in MR images penalizes that technique, because it is based on the use of pixel patches. More recent studies have used multi-atlas methods to improve the quality of pseudo-CT images, these methods are based on the fusion of results obtained by the registration of several atlases [15,16].

Image-registration methods often do not yield satisfactory results, because of local minima of the no-rigid deformation energy function [1]. To overcome this issue, we propose in this study, a new approach to create a pseudo-CT, it is based on a stereo correspondence concept in order to provide an increased robustness to handle the prediction local defects in registration.

Material and Methods

Principle of the Approach

Our proposed approach uses an atlas without registration. It is based on a stereo correspondence concept to find each corresponding pairs of pixels. Stereo matching methods are widely used in digital image processing field, we can cite for example image alignment, object recognition, remote sensing, motion tracking, indexing and content-based search [17-21]. A proposed method consists of finding the points of interest (salient points or vertices) in both images, each point is indicated by a vector (local descriptor), and an algorithm is used to find the corresponding pairs of points from the two images.

Compared with other types of imaging, images in MRI, CT or PET have got two specific properties: the first one is that these images can be in the same space (space of coordinates), and the second one is that their content sizes are approximately equal (Figure 1). These properties allow us to reduce the field of correspondence search to a local one, which is only around the position of the point of interest, and not in the entire image. Thus, our approach is based on the assumption that the pixel and its homologous (called corresponding point) are located roughly at two similar positions with the same spatial coordinates. So, grateful to this local search specificity and the local descriptor (the vector used to describe the pixel), it is possible to match each pixel of the first image to one pixel of the second image.

Figure1.

Left: MR image of patient 1, right: MR image of patient 2, middle: the two images juxtaposed.

If this approach is used with an atlas (MR images + CT images), it becomes possible to create a pseudo-CT image from an MR image of a patient. The proposed method uses a local correspondence search process with an atlas, in order to spread the intensities of the atlas to the relevant MR image.

In the remainder of this article, we use the following notations:

• A_MR: Atlas Magnetic resonance image

• A_CT: Atlas CT image

• P_MR: Patient magnetic resonance image

• P_PC: Patient pseudo-CT image

A local descriptor should be calculated for all pixels of both images P_MR and A_MR.Take a pixel p in image P_MR, defined by the descriptor vector V. Consider a search window of pixels [q₁...q_k] in the image A_MR of the atlas. These pixels are defined by the descriptor vectors [W₁...W_k]. This search window is around the same position of pixel p. Consider a pixel q in the search window, and defined by the vector W. This pixel corresponds to pixel p, when it meets the following formula (1):

W = arg min (Dist (V, [W₁...W_k])) (1)

A look-up table is built by calculating the matches for all pixels in image P_MR. This table is used with the CT image atlas A_CT to build the pseudo-CT P_PC. The intensity values in P_PC are deduced using the formula (2):

$P_{\mathrm{MR}}\left(\mathrm{x,y}\right)=A_{\mathrm{MR}}\left(x^{’},y^{’}\right)\Rightarrow P_{\mathrm{PC}}\left(\mathrm{x,y}\right)=A_{\mathrm{CT}}\left(x^{’},y^{’}\right)$ (2)

Shape-related Coordinates (SRC)

In the remainder of the article, the shape is the organ shown in the image. A new concept for encoding pixel positions in an image is defined in the present work. The principle is to encode the coordinates of a pixel within the shape, with the objective to have a relative position at the centre of the shape. This centre is the point from which a circle can be drawn around that shape. In the present article, this concept concerns images of the head.

Consider a pixel p with the coordinate (X,Y). The new shape-related coordinates (X_SRC,Y_SRC) are calculated (eq.3):

$\{\begin{array}{c}{X}_{\mathrm{SRC}}=\left(\frac{X_0-X}{\mathrm{LH}}\right)*100;\hfill \\ \hfill (3)\\ Y_{\mathrm{SRC}}=\left(\frac{Y-Y_0}{\mathrm{LV}}\right)*100;\hfill \end{array}$

Where (X₀,Y₀>) are the coordinates of the centre of the shape, LH is the distance from the centre of the shape to the horizontal boundary of the shape, and LV is the distance from the centre to the vertical boundary of the shape (Figure 2).

Figure2.

The principle of shape-related coordinates

Figure 3 shows a set of pixels in MR images of two subjects, each pixel in the first image is connected to the pixel in the other image which has the same SRC coordinates. We note that the pixels that are in the same anatomical areas in the two images are connected, although the differences in the size and shape of the two heads. This encoding method (SRC) allows a first mapping between two images.

Figure3.

Matches between points with the same shape-related coordinates

To find the exact matching of a pixel in the left image, just search around the pixel (with the same SRC coordinates) in the other right image. SRC method reduces the search field at the nearest neighbourhood, which promises a better match. The use of SRC allows us to achieve a guided local search.

Local Texture Descriptor for Matching (LTEMA)

Several local descriptors have been developed to put into correspondence pairs of images. A descriptor is used to identify each pixel in the image; it defines and distinguishes the pixels in the local neighbourhood. We use LTEMA descriptor in our proposed approach.

In magnetic resonance imaging (MRI), bone and air produce no signal, so additional information is needed to distinguish the two classes, i.e. bone and air. For that, we propose to use a texture analysis like LTEMA to define a set of measurements. We will show in the following that it allows an interesting performance.

We notice that the property of texture as an important characteristic for image analysis has been proved by a large majority of researchers. In [22], the authors noted that some texture features are very robust to changes in MRI acquisition settings, invariant to changes in image resolution, and unaffected by the corruption of the MRI image by magnetic field inhomogeneity. Moreover in [23], the authors concluded that the texture features extracted from MR images (sequence T1) from different MRI devices are statistically very similar for the same type of tissue.

Therefore, the texture properties should be used instead of the values of the neighbourhood. Since we have to calculate a descriptor for each pixel in the image, we have chosen to use measurements easy to calculate. First order textural properties are selected; these are statistics that are calculated from the pixel values such as variance; they do not include neighbourhood relations among the pixels. Our method uses three texture measurements:

• Mean: it is a measure of the brightness of pixel P_r,s located at position (r,s).

  $m_p=\frac{1}{n^2}{\sum }_{\mathrm{r=0}}^{\mathrm{n-1}}{\sum }_{\mathrm{s=0}}^{\mathrm{n-1}}{P}_{\mathrm{r,s}}$ (4)

• Standard deviation: it is a measure of contrast

  ${\sigma }_P={\lfloor \frac{1}{n^2}{\sum }_{\mathrm{r=0}}^{\mathrm{n-1}}{\sum }_{\mathrm{s=0}}^{\mathrm{n-1}}{[P_{\mathrm{r,s}}-m_P]}^2\rfloor }^{\mathrm{1/2}}$ (5)

• Entropy: it is a statistical measure of randomness

  $e=-{\sum }_{\mathrm{b=0}}^{\mathrm{L-1}}p\left(b\right){\log }_{2}p\left(b\right)$ (6)

Where p(b) = N(b)/n²for {0 ≤ b ≤ L-1}, where L is the number of different values which pixels can adopt, N(b) is the number of pixels of amplitude b in the pixel window of size (n × n).

To extract the most significant features of MR images, the textural properties (mean, standard deviation and entropy) are calculated from different neighbourhood window sizes. For the neighbourhood of a pixel in MR images, eight windows were selected, and the pixel is a corner, beside two windows centred on the pixel as shown in Figure 4. The normalized vector contains a total of 30 characteristics (3 measurements * 10 windows).

Figure4.

Distribution of used neighbourhood in LTEMA

In MR images, we faced a problem of the signal heterogeneity. Indeed, similar tissues do not give the same signal intensity, and each tissue does not have the same intensity. Our descriptor contains two types of data which were used to describe the pixel with the minimum of confusion. The first data include the information of close neighbourhood (windows centred on the pixel, with sizes 3 * 3 and 5 * 5), and the second are the information on the distant neighbourhood (windows of corners, with sizes 10 * 10 and 12 * 12). We mention that these values are obtained from a set of tests intended to determine the optimum values; the objective is to introduce one descriptor able to describe a pixel in both near and distant neighbourhoods. Therefore, if there is confusion with the close data (which is often the case with the problem of heterogeneity in MR images), the information in the distant description may help to eliminate that confusion. This descriptor was used for the segmentation of bone structures in a previous study [24].

Several local descriptors have been developed in a stereo matching process and some descriptors were tested in order to define the pixels in MR images. In this section, the chosen descriptors in this paper are introduced in the following:

─ SIFT (Scale-Invariant Feature Transform): it is one of the oldest local descriptors, it uses a vector having 128 dimensions to describe a point [25].

─ SURF (Speeded-Up Robust Features): this local descriptor is similar to SIFT. The main advantage of SURF over SIFT is its processing speed, because SURF uses a vector having only 64 dimensions [26].

─ DAISY is a new local descriptor, it is inspired from previous descriptors, and it can be calculated more efficiently in a dense matching case. Its fast computational time results from the replacement of the weighted sums, used by previous descriptors, by the sum of convolutions [27].

Main Steps of the Approach

• Algorithm: Construction of a pseudo-CT

• Input:

  • A_MR: MR Image atlas

  • A_CT: CT Image atlas

  • P_MR: MR Image patient

  • k: number of neighborhood search

• Output:

  • P_PC: pseudo-CT image

• Start:

• L = length of P_MR

• C = width of P_MR

• Calculate the shape-related coordinates of all pixels in the image P_MR

• Calculate the shape-related coordinates of all pixels in the image A_MR

• Calculate descriptors [V_1,1...V_L,C] of all pixels in the image P_MR

• Calculate descriptors [W_1,1...W_L,C] of all pixels in the image A_MR

• % Access to all pixels using shape-related coordinates% Access to all pixels using shape-related coordinates

• For x = 1 to L

  • For y = 1 to C

  • D = [] % initialization of table distances between descriptor vectors

  • % browse a neighbourhood in e A_MR

  • For i = (x - k) to (x + k)

    • For j = (y-k) to (y + k)

    • D[i,j] = distance (V_x,y,W_i,j) % calculate the distance

    • End for

  • End for

  • [x’,y’]=Min (D) % Select the candidate P(x’,y’) with minimum distance

  • Pseudo CT [x, y] = Act [x’, y’]

  • End for

• End for

• % A median filter is applied to the resulting image to harmonize intensities.

• Median_Filter (PseudoCT)

• End.

Results

Data Set

Our method was applied on MRI data of a patient’s brain. These data are obtained from the Vanderbilt database, called the Retrospective Image Registration Evaluation Project (RIRE). These datasets are now available as open-access data [28]. In this database, T1-weighted MR data were obtained using a Magnetization Prepared Rapid Gradient Echo (MP-RAGE) sequence. This is a quick gradient-echo technique in which a preparation pulse is applied before the acquisition sequence to enhance contrast. The MR data dimensions were 256 x 256 x 128 with an average resolution of 0.98mm x 0.99mm x 1.484mm. The corresponding CT scans had a 3mm slice thickness with slice dimensions 512 x 512 with an average resolution of 0.419mm x 0.419mm, the number of slices for each volume varied between 42 and 49.

Performance Assessment Metrics

The index of Structural Similarity (SSIM) is used as an evaluation measurement. SSIM measures the similarity between two images. It has been shown that SSIM is better suited than the traditional methods such as MSE and PSNR [29].

Dice similarity index (DSI) was also used to measure the quality of predictions of the three major classes of tissue, namely bone, voids and other tissues. DICE aims to quantify the intersection of our results with the ground truth masks [30]. The Dice similarity index is calculated as follows (eq7):

$D=\frac{2|M_A\cap M_B|}{\left(\right|M_A|+|M_B\left|\right)}$ (7)

Where MA is the result mask, and MB is the ground truth mask.

Note that the truth mask is obtained by thresholding the real CT images, and the result mask is obtained by thresholding the pseudo-CT images. Tissue classes were obtained by thresholding CT images and pseudo-CT (HU = Hounsfield units) [31]:

• - Air: < -500 HU

• - Soft tissue: [-500; 300] HU

• - Bone: > 300 HU

Qualitative and Quantitative Results

It should be noted that the proposed method is evaluated without the need for a pre-processing step. First to construct our atlas, CT images were registered to the corresponding MR images using SPM [32]. Secondly, several correspondence search windows were tested to adjust the proposed method. In our database, the value k = 7 was found as the optimum value, i.e. a neighbourhood window of 14 * 14 pixels. This value was used in the following tests, and the different descriptors were tested for the construction of pseudo-CT images.

Our approach proposes a new way for the atlas use and has been compared to a registration method. To check the quality of our proposed approach compared to other registration methods, Elastix was used as a registration tool [33]. As all classical methods based on atlas use for creating pseudo-CT, a rigid transformation was applied followed by a non-rigid transformation for the registration of the patient’s MRI image with the MRI atlas image, then these same transformations were applied to CT Atlas image in order to have a pseudo-CT by registration. The average values of SSIM and DICE calculated for all data are presented in Table 1. Figure 5 shows the qualitative results.

Table 1.

Test results on all data

	REGISTRATION	LTEMA	SIFT	SURF	DAISY
SSIM	0.685	0.721	0.708	0.608	0.606
DICE AIR	0.320	0.400	0.230	0.072	0.063
DICE SOFT	0.932	0.937	0.928	0.867	0.863
DICE BONE	0.760	0.774	0.752	0.360	0.202

Figure5.

Test of different descriptors. From left to right: MRI patient image, corresponding real patient CT image, pseudo CT image by registration method, pseudo-CT image by the proposed method, with the descriptors LTEMA ,SIFT, SURF, and DAISY.

Discussion

A quantitative and qualitative evaluation of our approach is presented in this article. We notice that the intensities of the pseudo-CT images obtained (the results) are computed with continuous values (Hounsfield unit). A proposed approach shows that the use of a stereo correspondence technique with an atlas is much better than the classical registration approach (Table 1), the choice of the appropriate descriptor is also required. Compared to other descriptors, our proposed LTEMA descriptor gives the best results, it allows the best identification of bone structures and voids.

In addition, LTEMA descriptor allows a good discrimination in local search matches. SIFT comes in the second position. However, SURF and DAISY descriptors give the poorest results because they are not discriminating in non-salient regions (lines and texture regions). The bad results of SURF are due to its reduced dimensions compared to SIFT. Unlike SURF, DAISY is a descriptor designed for a dense matching case but it does not seem suitable for MR images. DAISY descriptor is probably penalized by the nature of intensities in MR images. From Figure 5, it can be seen that our descriptor LTEMA is better suitable for a dense stereo matching in MR images.

In Figure 6, we notice the differences in bone thickness between the actual CT image and the Pseudo-CT resulting from registration and from the descriptor LTEMA. When the registration technique changes the shape of the atlas to conform to the shape of the patient’s image, without taking into account the local difference between the two images, our method with LTEMA descriptor can recognize that difference, because it compares the values of all pixels. This leads to a better estimation of the thickness of bony structures.

Figure6.

Difference in quality of pseudo-CT images. From left to right: real CT image, Pseudo-CT image by registration method, pseudo-CT image with LTEMA descriptor.

The purpose of the construction of a pseudo-CT is not only in its visual aspect, but in its use for CA. The best results were obtained in the images of the upper part of the skull, the average DICE value of bone structures is 0.774. It should be noted that the quality of the results depends on the atlas images used.

Conclusion

In this paper, a new approach is proposed to generate a pseudo-CT from an MRI image. It is a new way to use an atlas in medical image processing without the need of a registration step and pre-processing step. It has been demonstrated that it is possible to find correspondences between two MR images, pixel by pixel. Using an atlas (MRI images + CT images), it is possible to create a pseudo-CT with Hounsfield intensities. This pseudo-CT can be used for the attenuation correction in PET images.

This approach is distinguished by two main contributions. The first is SRC, a new concept for encoding pixel positions in a medical image, and the second contribution is LTEMA, the local descriptor based on texture properties. Obtained results with LTEMA are better than a registration method and other descriptors. The use of dense matching with atlas promises good results; however, we must choose the appropriate local descriptor for this task.

We can also deduce that with the use of an appropriate atlas, the proposed approach can be recommended in the segmentation of medical images as an alternative to registration techniques.