RANDOM FOREST FLAIR RECONSTRUCTION FROM T₁, T₂, AND P_D-WEIGHTED MRI

Abstract

Fluid Attenuated Inversion Recovery (FLAIR) is a commonly acquired pulse sequence for multiple sclerosis (MS) patients. MS white matter lesions appear hyperintense in FLAIR images and have excellent contrast with the surrounding tissue. Hence, FLAIR images are commonly used in automated lesion segmentation algorithms to easily and quickly delineate the lesions. This expedites the lesion load computation and correlation with disease progression. Unfortunately for numerous reasons the acquired FLAIR images can be of a poor quality and suffer from various artifacts. In the most extreme cases the data is absent, which poses a problem when consistently processing a large data set. We propose to fill in this gap by reconstructing a FLAIR image given the corresponding T₁-weighted, T₂-weighted, and P_D-weighted images of the same subject using random forest regression. We show that the images we produce are similar to true high quality FLAIR images and also provide a good surrogate for tissue segmentation.

1. INTRODUCTION

MS is a neurodegenerative disease characterized by focal lesions throughout the white matter (WM) of the brain, which can extend to the boundary of the WM and gray matter (GM). In both Alzheimer’s disease (AD) and MS, white matter lesions (WMLs) appear to be hypointense in T₁-weighted (T₁-w) images, similar to GM or cerebrospinal fluid (CSF). Typically other contrasts such as T₂-weighted (T₂-w), proton density (P_D-w) weighted, and FLuid Attenuated Inversion Recovery (FLAIR) are used to help distinguish lesions from GM and CSF. Lesion volumes are known to be correlated with the progression of many sub-types of MS [1]. The contrast of magnetic resonance (MR) images is of fundamental importance for WML tissue classification. We note that it is well established that FLAIR images provide the best contrast for detection and quantification, whereas conventional T₁-w, T₂-w, and P_D-w spin echo sequences offer diminished contrast for WML [2]. Most recent lesion segmentation algorithms therefore depend heavily on FLAIR for WML identification [3–5]. However, FLAIR images can suffer from artifacts for a variety of reasons [6]. Fig. 1 shows an artifact-ridden acquired FLAIR and the corresponding tissue segmentation result. We intend to overcome this problem by reconstructing a FLAIR image using information from the other contrasts. Our reconstructed FLAIR does not exhibit the artifacts, because each of the T₁-w, T₂-w, and P_D-w, images used are artifact free, and together they possess enough information to recreate the FLAIR tissue intensities.

Fig. 1.

The true FLAIR (T-FLAIR) shows visible artifacts which confound LesionTOADS and other lesion segmentation algorithms (not pictured), resulting in the misclassification of healthy WM as WMLs. Our reconstructed FLAIR (R-FLAIR) does not have the artifacts, as they are not present in the T₁-w, T₂-w, or P_D-w images.

Our work builds upon the ideas of image hallucination [7] which originally generated a high-resolution image from multiple low resolution acquisitions based on a textbook or atlas. The idea first gained traction in the medical imaging community with the work of Rousseau [8], and has since gained popularity and numerous applications from super-resolution [9, 10], contrast synthesis [9, 11–13], and inhomogeneity correction [14]. Our approach is to treat the unknown FLAIR image as dependent on other known images and model the reconstruction of the FLAIR as a nonlinear regression of image patches drawn simultaneously from the co-registered T₁-w, T₂-w, and P_D-w contrast images. The voxel intensity in MR is a function of many nuclear magnetic resonance (NMR) parameters like proton density (P_D), longitudinal and transverse relaxation times (T₁ and T₂) and others. We would like to frame the problem of predicting the FLAIR voxel intensities as a nonlinear function of the other contrast image intensities—which themselves are also a function of the underlying NMR parameters. The nonlinear function is learned as a nonlinear regression from data by a bagged ensemble of regression trees [15] and is significantly faster and better than the state-of-the-art.

In this paper, we build upon our previous FLAIR reconstruction work [16] which used only T₁-w and T₂-w data and a sparse priors model to resolve the intensity of the unknown contrast. We expand the work to include the third contrast of P_D-w to help improve the FLAIR reconstruction as well as reframing the problem in terms of a random forest based reconstruction [11]. We show that the reconstructed FLAIR (R-FLAIR) image matches very closely with the original high quality FLAIR image (T-FLAIR), while also providing an important surrogate image for use in a WML segmentation algorithm.

2. METHOD

We tackle the FLAIR reconstruction with an atlas based regression approach. Our atlas consists of ( , , , ), which have tissue contrasts of T₁-w, T₂-w, P_D-w, and FLAIR, respectively. The atlas images have the same voxel resolution and are sampled at the same coordinates in space. The subject images are assumed to consist of the triple of ( , , ) with the goal being to reconstruct from the learned nonlinear relationships in the atlas quadruple. For several of our experiments we will have the true available, to which we can compare the quality of the reconstruction. At the i^th voxel of the atlas images , , and , 3D patches of size p × q × r are rasterized to form a d × 1 vector (d = pqr, p = q = r = 3, chosen empirically) for each of the three images. This triple of vectors is concatenated to form the 3d × 1 vector x_i. x_i and the corresponding intensity, y_i, of the i^th voxel in form the independent and dependent variables, respectively, for our training data denoted as 〈x_i, y_i〉.

We solve the regression problem generated by the training data 〈x_i, y_i〉 using random forests (RF) [17]. A RF consists of an ensemble of regression trees [15], with each regression tree partitioning the space of x_i’s into regions based on a split at each node in the tree. During training, splits at a node are performed by randomly selecting one-third of the attributes and then determining the attribute in this subset that best minimizes a least squares criterion. We ensure that nodes with less than five data values are leaf nodes, thus avoiding over-fitting of the training data. The value of the dependent variable assigned at a leaf node is the mean of the y_i’s which have accumulated in that leaf during training. The bagged ensemble of regression trees allows for greater robustness in the predictive power of the RF. In our experiments, our RF consists of 60 trees each learned from bootstrapped training data. We used patches sampled from data acquired from five MS subjects, ensuring equal representation of CSF, GM, WM, and WMLs. The bootstrapping is done by choosing with replacement N times, where N is the size of the training data set (~ 10⁵ samples). To predict the FLAIR intensity for the subject image set, we construct the rasterized 3d × 1 vector, u_j, similarly to that used for the training data. The u_j’s are input into each of the regression trees in the trained RF. The output of each tree is the value of the leaf node the vector u_j is assigned. We aggregate these values by computing their mean to produce a final output intensity for the j^th voxel of the R-FLAIR image . Averaging the FLAIR intensity values at the leaf nodes of the RF lowers the peak values of the output FLAIR image. To avoid this, we linearly rescale the peak intensities of to match those of a typical FLAIR image. Training takes ten minutes and the prediction phase takes less than ten seconds, for images of size 256 × 256 × 173.

3. RESULTS

3.1. Image Similarity

Our goal in reconstructing the FLAIR from corresponding T₁- w (0:8³ mm³, MPRAGE), T₂-w, and P_D-w (Dual Spin Echo, 0:8 × 0:8 × 2:2 mm³) images is to provide an approximation to a T-FLAIR for defining lesions. Our data set consists of 49 MS subjects (44 test + 5 training), with all the data registered to the MNI space and resampled at a 1 mm³ isotropic resolution. We used the T₁-w, T₂-w, and P_D-w images for our R-FLAIR and compare it to the T-FLAIR image in the data set. Several inferior slices of the P_D-w, T₂-w, and FLAIR images—covering the cerebellum–are of considerably lower quality and were not used for evaluation.

We use the image similarity metrics of peak signal to noise ratio (PSNR), the universal quality index (UQI) [18], and the structural similarity index (SSIM) [19] for evaluation (see Table 1). UQI and SSIM have a range between 0 and 1, attaining the maximum if the T-FLAIR and the R-FLAIR image are perfectly equal. Example results are shown in Fig. 2. The R-FLAIR is smoother than the T-FLAIR due to its intensities being produced by the averaging of 60 trees in the RF.

Table 1.

Mean (Std. Dev.) of PSNR (in decibels), UQI, and SSIM values over 49 subjects.

PSNR	UQI	SSIM
25.58 (1.06)	0.88 (0.03)	0.87 (0.02)

Fig. 2.

Example slices of true (T-FLAIR) and reconstructed FLAIR (R-FLAIR) for a subject.

3.2. Image Segmentation

In this experiment we tested, using LesionTOADS [4], if the R-FLAIR can be used to provide reliable segmentation of brain tissues into CSF, GM, WM, and WMLs. LesionTOADS takes co-registered T₁-w and FLAIR images as input and generates a segmentation. We ran LesionTOADS with the original T₁-w image in conjunction with either the T-FLAIR or R-FLAIR. From the LesionTOADS segmentation, we computed the Dice coefficient to compare the stability of our R-FLAIR to T-FLAIR, the results are shown in Table 2. The Dice coefficient is low for WMLs because of algorithm errors caused by artifacts in the T-FLAIR images—see Fig. 1 for an example— which are overcome in our R-FLAIR images. For those cases where the T-FLAIR image is devoid of artifacts, the segmentation differences between the T-FLAIR and R-FLAIR are minimal (see Fig. 3).

Table 2.

Mean (Std. Dev.) of Dice coefficients based on LesionTOADS segmentation of the T-FLAIR and R-FLAIR.

WM	GM	CSF	WML
0.96 (0.01)	0.99 (0.0005)	0.95 (0.01)	0.46 (0.22)

Fig. 3.

LesionTOADS segmentations using true (T-FLAIR) and reconstructed FLAIRs (R-FLAIR).

For the 44 test MS subjects, we have manual segmentations of the WMLs, from which we computed the Dice coefficient for the WMLs between the truth and the segmentations generated from each of T-FLAIR and R-FLAIR. The manual and T-FLAIR had a mean Dice score of 0.42 with a standard deviation of 0.27. Whereas, the R-FLAIR had a score of 0.38 with a standard deviation of 0.21. The Dice coefficient as a metric can be misleading because of the small diffuse nature of lesions. Thus, we also looked at the absolute relative difference of the manual WML volumes and those calculated by LesionTOADS on either T-FLAIR or the R-FLAIR. The ratio for T-FLAIR is 8.81, while for R-FLAIR it is 1.37, which is considerably less. Fig. 4 illustrates the lesion volumes for different subjects sorted in order of increasing manual lesion load. It is clear that using a R-FLAIR provides a more accurate lesion load estimation than a T-FLAIR—as the T-FLAIR has real-world artifacts that can confound any algorithm, not just LesionTOADS.

Fig. 4.

Shown are WML volumes, ordered by increasing lesion load, for 44 MS subjects. The green is the manual segmentation, while the blue is the plot of LesionTOADS based on the T-FLAIR and the red is LesionTOADS using our R-FLAIR.

4. CONCLUSION

In this work, we demonstrated a method to reconstruct a FLAIR image from the corresponding T₁-w, T₂-w, and P_D-w images; the reconstruction is principally for image processing. Our previous work [16] could not handle WMLs, and our use of balanced training data and a third contrast has resolved this problem. The R-FLAIR is very close to the T-FLAIR in terms of image quality (see Table 1). We have shown that our R-FLAIR leads to more reasonable segmentation results with LesionTOADS than the T-FLAIR (see Fig. 4), in comparison to a human rater. In some cases, the artifacts in the T-FLAIRs produced dramatically bad segmentations (see Fig. 1), which is not the case with our R-FLAIRs.