Pseudo CT Estimation from MRI Using Patch-based Random Forest

Abstract

Recently, MR simulators gain popularity because of unnecessary radiation exposure of CT simulators being used in radiation therapy planning. We propose a method for pseudo CT estimation from MR images based on a patch-based random forest. Patient-specific anatomical features are extracted from the aligned training images and adopted as signatures for each voxel. The most robust and informative features are identified using feature selection to train the random forest. The well-trained random forest is used to predict the pseudo CT of a new patient. This prediction technique was tested with human brain images and the prediction accuracy was assessed using the original CT images. Peak signal-to-noise ratio (PSNR) and feature similarity (FSIM) indexes were used to quantify the differences between the pseudo and original CT images. The experimental results showed the proposed method could accurately generate pseudo CT images from MR images. In summary, we have developed a new pseudo CT prediction method based on patch-based random forest, demonstrated its clinical feasibility, and validated its prediction accuracy. This pseudo CT prediction technique could be a useful tool for MRI-based radiation treatment planning and attenuation correction in a PET/MRI scanner.

1. INTRODUCTION

Computed tomography (CT), with its high availability and geometric accuracy, has proven an invaluable tool for cancer diagnosis and treatment. Since a CT voxel’s intensity directly relates to its attenuation of X-rays, CT images provide a means of estimating tissue electron density, which, in turn, is necessary to simulate radiation transport. For this reason, CT is a cornerstone of both radiotherapy planning and attenuation correction of Positron Emission Tomography (PET) images. Recently, there has been increasing interest in replacing CT with magnetic resonance (MR) in MRI-only radiotherapy treatment planning as well as in PET/MR attenuation correction. The first application is in certain types of radiotherapy planning, where CT’s limited soft tissue contrast stunts accurate tumor delineation whereas MR provides valuable guidance [1]; such cases include stereotactic radiosurgery of brain disorders and radiotherapy treatment of prostate cancer [2, 3]. The second application is in combined PET/MR imaging currently transitioning to reach its full clinical potential, where attenuation correction is necessary but CT acquisition would require an additional scan [4]. Therefore, there is an increasing clinical need to reliably estimate electron densities (called pseudo CT images) in these applications based on the MR images. The benefits of pseudo CT images include reducing medical cost, sparing the patient from X-ray exposure, eliminating registration errors between MR and CT images and simplifying clinical workflow. These issues greatly motivate the development of methods for estimate of CT image from the corresponding MR image.

Several methods have been investigated to estimate electron density information using MRI [5–19], which can be characterized into four categories: segmentation-based, atlas-based, pseudo-CT based and sequence-based methods. Each has its advantages and limitations. (1) In segmentation-based methods [6, 14, 16, 17, 19, 20], MR images are manually or automatically segmented into different tissue types (bone, soft tissue and air) with predefined electron density or attenuation coefficient. The main drawback of these approaches is the inter-observer inconsistency of the manual contours or the segmentation errors of the auto-contours [21]. (2) In atlas-based methods [15, 22], a deformable image registration is used to map a sample patient or atlas with known tissue label or electron density (typically derived from CT) to an individual’s MR image. The electron density map for this individual can then be obtained. These methods could suffer from inherent registration errors due to inter-patient anatomical differences. (3) In pseudo-CT based methods [5, 8, 10, 12, 18], tissue properties are directly characterized based on the MR image intensity and anatomy. However, due to the lack of a one-to-one relation between electron density and MR image intensity, this approach could lead to ambiguous results. In particular, the differentiation of bone from air has been challenging because of their similar and short T2 characteristics (both appear dark on MRI) [23]. (4) In MRI sequence-based methods [9, 13, 24], MR sequences such as the ultra-short echo time (UTE) sequence are used to visualize bony anatomy. However, the currently available image quality of UTE imaging is still far from satisfactory. Moreover, the non-standard MR sequence also adds considerable scan time, which may introduce more patient motion and discomfort. In this study we proposed to develop a learning-based method to derive electron density from standard clinical MRI. In this approach we integrated patch-based anatomical signature into machine learning framework to predict pseudo CT using MRI.

2. METHODS

Suppose we have a set of pairs of MR and CT training images. For each pair, the CT image is used as the regression target of the MR image. We further assume that the training data have been preprocessed by removing noise and uninformative regions, and have been aligned. The proposed prediction method consists of two major stages: the training stage and the prediction stage. During the training stage, patch-based anatomical features are extracted from the registered training images with patient-specific information, and the most robust and informative CT-MR features are identified by feature selection to train a random forest. During the prediction stage, we extract the selected features from the new (target) MR image and feed them into the well-trained forests for CT image prediction. The four major steps are briefly described below.

2.1. Image Alignment for Dataset

Before doing prediction, some preprocessing is performed for all CT and MR images in dataset, which includes noise reduction, inhomogeneity correction, and inter-subject intensity normalization. Such preprocessing steps are to improve the accuracy of the following prediction. The intra-subject registration is used to align each pair of CT and MR images of the same subject. We perform rigid and deformable registration with B-Splines [25], using mutual information as a similarity measure. The inter-subject registration is used to align each CT-MR pair images among the different subjects. During the inter-subject alignment processing of training set, we first select one MR-CT pair image as the template, and align other MR images to the template MR image. We use the corresponding transformation obtained from training MR image alignment to align the corresponding CT images of each subject to the template CT images. Since the CT images of each training image are available, we again align each training CT image to the template CT images to optimize the alignment of the training set, and then apply the corresponding transformation obtained from training CT image alignment to align the corresponding MR images to the template MR images. For each newly acquired MR image, all aligned training images in training set are rigidly registered to this new image. The rigid registration is used to obtain the spatial deformation field between the new MR image and training MR images. The same transformations are applied to the CT in training set.

2.2. Patch-based Feature Extraction

Patch-based representation has been widely used as voxel anatomical signature in computer vision and medical image analysis [5, 26–28]. The principle of the conventional patch-based representation is to first define a small image patch centered at each voxel and then use the voxel intensities of image patch as the anatomical signature of each voxel. However, due to the noise and anatomical complexity of MR images, patch-based representation using voxel intensities alone may not be able to effectively distinguish the voxels of different tissue in MRI. We propose to use a patch-based anatomical features are used as signature for each voxel to characterize the image appearance. Two types of images features – the Gabor wavelet feature and the local binary pattern (LBP) feature – are extracted from a small image patch centered at each voxel of each aligned training image. Gabor can provide complementary anatomical information to each other, and LBP can capture texture information from the input image. The 16 Gabor feature is used in this study. The LBP feature is extracted in three resolution levels and it has a dimension 30. Therefore, for each voxel, it is represented by a 46 dimensional feature signature.

2.3. Patch Preselection Based on Features

Patch preselection is applied here among all candidate patches, in order to reduce the computational burden and improve prediction accuracy by excluding irrelevant patches [26]. Based on the above features, we can obtain patch-based representation of each voxel. It should be noted that the patch-based anatomical signature may contain noisy and redundant features which will affect the performance in excluding the irrelevant patches. Therefore, feature selection should be performed to identify the most informative and salient features in the anatomical signature of each voxel [28]. The aim of feature selection is to select a small subset of most informative feature as anatomical signature, which can be well accomplished by enforcing the sparsity constraint during the regression process. Therefore, the feature selection problem can be formulated as a logistic sparse least-absolute- shrinkage-and-selection operator (LASSO) problem [29]. It is defined as,

$$J(\overrightarrow{\beta },b)=\sum _{c=1}^P\log (1+\exp (-{\overrightarrow{\beta }}^T\overrightarrow{f}({\overrightarrow{x}}_c)+b)))+\lambda {\Vert \overrightarrow{\beta }\Vert }_1$$ (1)

where $\overrightarrow{f}\left({\overrightarrow{x}}_c\right)$ denotes the original feature signature of voxel ${\overrightarrow{x}}_c$. $\overrightarrow{\beta }$ is the sparse coefficient vector, ${\Vert ·\Vert }_1$ is the l₁ norm, b is the intercept scalar, and λ is the regularization parameter. The first term is obtained by inputting the values of drawn samples and their original feature signatures to the function, and then takes the logarithm for maximum likelihood estimation. The second term is the l₁ norm which aims to enforce the sparsity constraint for LASSO. Through minimizing the sparse LASSO energy function the features with superior discriminant power are selected. Based on the selected features, we can directly measure their discriminant power to exclude the irrelevant patches, based on the Fisher’s score.

2.4. Random Forest Training and Pseudo CT Prediction

The random forest classifier is a learning-based method for classification and regression problems, and has been widely adopted in medical imaging applications [30]. A random forest comprises of multiple decision trees. At each internal node of a tree, a feature is chosen to split the incoming training samples to maximize the information gain. Specifically, let $u\in U\subset ℝ$ be an input feature vector, and $v\in U\subset ℝ$ be its corresponding target value for regression. For a given internal node j and a set of samples $S_j\subset U\times V$ , the information gain achieved by choosing the k^th feature to split the samples in the regression problem is computed by:

$$I_j^k=H\left(S_j\right)-\frac{\left|S_{j,L}^k\right|}{\left|S_j\right|}H\left(S_{j,L}^k\right)-\frac{\left|S_{j,R}^k\right|}{\left|S_j\right|}H\left(S_{j,R}^k\right)$$ (2)

$$H\left(S\right)=\frac{1}{\left|V\right|}\sum _v{\left(v-\overline{v}\right)}^2$$ (3)

$$\overline{v}=\frac{1}{\left|V\right|}\sum _{v}v$$ (4)

where L and R denote the left and right child nodes, $S_{j,L}^k=\left\{\left(u,v\right)\in S_j|u^k<{\theta }_j^k\right\}$, $S_{j,R}^k=S_j/S_{j,R}^k$, u^k is the k^th feature of feature vector u, ${\theta }_j^k$ is the splitting threshold chosen to maximize the information gain $I_j^k$ for the k^th feature u^k, and |•| is the cardinality of the set. H (S) denotes the variance of all target values in our regression problem. During the training stage, the splitting process is performed recursively until the information gain is not significant, or the number of training samples falling into one node is less than a pre-defined threshold. In the testing stage, to predict the CT image, the new given MR image follows the same sequence as learned during the training stage, to generate the final prediction.

3. EXPERIMENTS AND RESULTS

In order to test the proposed method, we applied our method to5 patients’ MR and CT data. All patients’ MR data were acquired using a Siemens MR scanner. Each 3D T1W MR data sets consisted of 256×256×147 voxels to cover the whole head and the voxel size was 1.0×1.0×1.0 mm³. All patients’ CT data were acquired using a Siemens CT scanner. Each 3D CT data sets consisted of 512×512×160 voxels to cover the whole head and the voxel size was 0.8×0.8×1.0 mm³. We performed leave-one-out cross-validation method to evaluate the proposed CT prediction algorithm. In other words, we used the 4 CT and MRI images as the training set and applied the proposed method to predict the remaining subject. Our pseudo CT were compared with the original CT images. In order to get a quantitative evaluation, we used peak signal-to-noise ratio (PSNR) [31] and feature similarity (FSIM) index [32]. Figure 1 shows an example of MR-based pseudo CT predicted by the proposed method. This pseudo CT is close to original CT images. Figure 2 shows the PSNR and FSIM between the pseudo and original CT for each patient. Overall the mean PSNR and FSIM were 27.27±1.00 and 0.77±0.03, which demonstrated the prediction accuracy of the proposed learning-based method.

Figure 1.

MR-based pseudo CT prediction results. Images from left to right are MRI, CT, pseudo CT and the difference between pseudo and original CT images.

Figure 2.

PSNR (a) and FSIM (b) between the pseudo and original CT for each patient.

4. CONCLUSION

In this paper, we propose a novel learning-based approach to predict a pseudo CT image from a MR image based on a random forest regression with a patch-based anatomical signature to effectively capture the relationship between the CT and MR images. We have demonstrated that the proposed method is capable of reliably predicting the CT image from the MR image. This pseudo CT prediction technique could be a useful tool for MRI-based radiation treatment planning and attenuation correction in PET/MRI scanner.