Direct Patlak Reconstruction from Dynamic PET Data Using the Kernel Method with MRI Information Based on Structural Similarity

Abstract

Positron Emission Tomography (PET) is a functional imaging modality widely used in oncology, cardiology, and neuroscience. It is highly sensitive, but suffers from relatively poor spatial resolution, as compared with anatomical imaging modalities, such as magnetic resonance imaging (MRI). With the recent development of combined PET/MR systems, we can improve the PET image quality by incorporating MR information into image reconstruction. Previously kernel learning has been successfully embedded into static and dynamic PET image reconstruction using either PET temporal or MRI information. Here we combine both PET temporal and MRI information adaptively to improve the quality of direct Patlak reconstruction. We examined different approaches to combine the PET and MRI information in kernel learning to address the issue of potential mismatches between MRI and PET signals. Computer simulations and hybrid real patient data acquired on a simultaneous PET/MR scanner were used to evaluate the proposed methods. Results show that the method that combines PET temporal information and MRI spatial information adaptively based on the structure similarity index (SSIM) has the best performance in terms of noise reduction and resolution improvement.

I. Introduction

Positron Emission Tomography (PET) can produce three dimensional functional images of a living subject through the injection of a radioactive tracer. It has wide applications in oncology [1], cardiology [2] and neurology [3]. Though PET is highly sensitive, in the picomolar range, the image quality is still limited by various resolution degradation factors [4]. Compared with PET, x-ray Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) have better anatomical image resolution. With the wide availability of PET/CT and the new arrival of PET/MR systems, anatomical information from CT or MRI are readily available for use in the PET reconstruction. Research in this area has been going on for decades and different methods have been developed based on approaches such as mutual information or joint entropy [5]–[7], segmentation or boundary extraction [8]–[12], gradient information from a prior image [13], [14], intensity similarity in a prior image [15]–[19], wavelet [20], dictionary learning [21], [22], image sparse representation [23], convolutional neural network [24] and the kernel method [25]–[28]. The kernel method is a well-studied framework in machine learning that implicitly transfers the raw data points to a higher dimensional feature space using a kernel function. It has recently been applied to dynamic image reconstruction [25] and successfully extended to MRI aided PET reconstruction [26]–[28].

Here we combine the above two approaches to further improve the quality of dynamic PET reconstruction. Compared with static PET, dynamic PET provides additional temporal information of the tracer kinetics, which can be useful for tumor detection and treatment monitoring [29]–[31]. Conventionally dynamic PET images are reconstructed frame by frame, and time activity curves (TAC) from regions of interest (ROI) or individual pixels are fitted to a linear or nonlinear kinetic model. Direct reconstruction methods have been developed recently to estimate parametric images directly from the measured sinogram by combining the PET imaging model and tracer kinetics in an integrated framework [32]–[36]. It has been shown that direct reconstruction methods can reduce noise amplification compared with traditional indirect methods [37]. Anatomical information can be used to regularize the parametric images in direct reconstruction [28], [38], [39]. In this work, we focus on the Patlak model [40]–[44], which is a widely used graphical model. The slope of Patlak plot is a useful quantitative metric when the tracer uptake is irreversible. One challenge in using anatomical information in PET reconstruction is that there are often mismatches between MRI and PET signals in real data. For example, tumor regions showing up in a PET image may not appear in a co-registered MRI image [45]. It is thus crucial to account for this mismatch by exploiting both anatomical and PET information to construct the kernel matrix. In a previous work [27], spectral models were employed to model the PET temporal information and the kernel matrix were constructed using MRI information. The work here aims at combining the MRI based kernel matrix and dynamic PET based kernel matrix adaptively using a local similarity measure.

The main contributions of this paper include (1) combining the PET information and the MRI information adaptively based on local similarity and (2) applying this hybrid kernel to the direct Patlak reconstruction. This paper is organized as follows. Section 2 introduces the Patlak model and the kernel method. Section 3 describes the computer simulations and hybrid real data used in the evaluation. Experimental results are shown in section 4, followed by discussions in section 5. Finally conclusions are drawn in Section 6.

II. Method

A. Dynamic PET model

Dynamic PET data in the k^th time frame, y_k ∈ ℝ^{M × 1}, can be modeled as a collection of independent Poisson random variables with the mean ȳ_k ∈ ℝ^{M × 1} related to the unknown tracer distribution in the k^th time frame, x_k ∈ ℝ^{N × 1}, through an affine transform

$${\overline{\mathit{y}}}_k={\mathit{N}}_k\mathit{P}{\mathit{x}}_k+{\mathit{s}}_k+{\mathit{r}}_k$$ (1)

where N_k ∈ ℝ^{M × M} is the time dependent normalization matrix. P ∈ ℝ^{M × N} is the detection probability matrix with the (i, j)^th element p_ij being the probability of detecting a photon pair produced in voxel j by detector pair i. s_k ∈ ℝ^{M × 1} and r_k ∈ ℝ^{M × 1} denote the expectation of scattered and random coincidences, respectively, in the k^th time frame. M is the number of lines of response (LOR) and N is the number of image voxels. When concatenating T time frames together, we can get

$$\overline{\mathit{y}}=\mathit{N}({\mathit{I}}_T\otimes \mathit{P})\mathit{x}+\mathit{s}+\mathit{r},$$ (2)

where N = diag[N₁, N₂, …N_T], ⊗ denotes the Kronecker product, I_T is a T × T identity matrix, $\overline{\mathit{y}}=[{\overline{\mathit{y}}}_1^{\text{'}},{\overline{\mathit{y}}}_2^{\text{'}},\dots ,{\overline{\mathit{y}}}_T^{\text{'}}]\prime$, $\overline{\mathit{x}}=[{\overline{\mathit{x}}}_1^{\text{'}},{\overline{\mathit{x}}}_2^{\text{'}},\dots ,{\overline{\mathit{x}}}_T^{\text{'}}]\prime$, $\mathit{r}=[{\mathit{r}}_1^{\text{'}},{\mathit{r}}_2^{\text{'}},\dots ,{\mathit{r}}_T^{\text{'}}]\prime$, $\mathit{s}=[{\mathit{s}}_1^{\text{'}},{\mathit{s}}_2^{\text{'}},\dots ,{\mathit{s}}_T^{\text{'}}]\prime$, and “′” denotes matrix or vector transpose. Let $\mathit{y}=[{\mathit{y}}_1^{\text{'}},{\mathit{y}}_2^{\text{'}},\dots ,{\mathit{y}}_T^{\text{'}}]\prime$ denote the measured dynamic data. The log-likelihood function of y can be written as

$$L(\mathit{y}|\mathit{x})=\sum _{k=1}^T\sum _{i=1}^M{({\mathit{y}}_k)}_{i}log{({\overline{\mathit{y}}}_k)}_i-{({\overline{\mathit{y}}}_k)}_i-log{({\mathit{y}}_k)}_i!.$$ (3)

The maximum likelihood estimate of the unknown image x can be found by

$$\widehat{\mathit{x}}=arg\underset{\mathit{x}\ge 0}{max}L(\mathit{y}|\mathit{x}).$$ (4)

B. Patlak model

The Patlak graphical method [40] has been widely used for compartment models that contain at least one irreversible compartment. Under this model, the tracer concentration at time t, c(t) ∈ ℝ^{N × 1}, can be represented by a weighted sum of the blood input function C_p(t) and its integral after a sufficient length of time t^⋆ as

$$\mathit{c}(t)=\mathit{\kappa }{\int }_0^t{C}_p(\tau )d\tau +\mathit{b}{C}_p(t),t>t^{\star },$$ (5)

where κ ∈ ℝ^{N × 1} is the Patlak slope image and b ∈ ℝ^{N × 1} is the Patlak intercept image. The Patlak slope image κ can reflect the influx rate of the PET tracer and is a very important quantitative metric in practice. Considering the radioactive decay, the image intensity at the k^th frame, x_k, can be expressed as

$${\mathit{x}}_k={\int }_{t_{s,k}}^{t_{e,k}}\mathit{c}(\tau )e^{-\lambda \tau }d\tau ={\overline{S}}_p(k)\mathit{\kappa }+{\overline{C}}_p(k)\mathit{b}$$ (6)

where t_s,k and t_e,k are the start time and end time of frame k, λ is the decay constant of the PET tracer, and

$${\overline{S}}_p(k)={\int }_{t_{s,k}}^{t_{e,k}}{\int }_0^{\tau }{C}_p({\tau }_1)d{\tau }_1{e}^{-\lambda \tau }d\tau ,$$ (7)

$${\overline{C}}_p(k)={\int }_{t_{s,k}}^{t_{e,k}}{C}_p(\tau )e^{-\lambda \tau }d\tau .$$ (8)

Putting all time frames together, we can have

$$\mathit{x}=(\mathit{A}\otimes {\mathit{I}}_N)\mathit{\theta },$$ (9)

where θ = [κ′, b′]′, I_N is an N × N identity matrix, and

$$\mathit{A}=\left[\begin{array}{ll}{\overline{S}}_p(1)\hfill & {\overline{C}}_p(1)\hfill \\ {\overline{S}}_p(2)\hfill & {\overline{C}}_p(2)\hfill \\ ⋮\hfill & ⋮\hfill \\ {\overline{S}}_p(T)\hfill & {\overline{C}}_p(T)\hfill \end{array}\right].$$ (10)

Conventionally, images are reconstructed frame-by-frame according to (4) and then the Patlak parameters are estimated by fitting the time activity curves to the Patlak model (9). The least squares solution of the fitting is

$$\widehat{\mathit{\theta }}={((\mathit{A}\otimes {\mathit{I}}_N)\prime (\mathit{A}\otimes {\mathit{I}}_N))}^{-1}(\mathit{A}\otimes {\mathit{I}}_N)\prime \widehat{\mathit{x}}.$$ (11)

C. Direct reconstruction using the kernel method

Here we choose the direct reconstruction approach by substituting the Patlak graphical model (9) into (2) to get:

$$\overline{\mathit{y}}=\mathit{N}({\mathit{I}}_T\otimes \mathit{P})(\mathit{A}\otimes {\mathit{I}}_N)\mathit{\theta }+\mathit{s}+\mathit{r}.$$ (12)

Previously efforts have been made to add regularization in the parametric space by using a penalty function, which aims to enforce smoothness inside the region and allow sharp transition near the edges [38], [39]. Here we choose to regularize the direct reconstruction through the use of a kernel matrix [25]. The main idea is to represent the unknown parametric images by a linear combination of transformed features calculated from prior information. Using the kernel trick, we can express the Patlak slope and intercept images as

$$\mathit{\kappa }=\mathit{K}{\mathit{\alpha }}_{\kappa },$$ (13)

$$\mathit{b}=\mathit{K}{\mathit{\alpha }}_b,$$ (14)

where K ∈ ℝ^{N × N} is the kernel matrix, α_κ ∈ ℝ^{N × 1} and α_b ∈ ℝ^{N × 1} denote the kernel coefficient vectors of κ and b, respectively. Construction of the kernel matrix K is described in Section II.D. Here the same kernel matrix is used for the Patlak slope and intercept because we expect that they share the same regional boundaries, although they may have different intensities and contrasts. However, the proposed algorithm is amendable to use different kernel matrices for the Patlak slope and intercept, if necessary. In the implementation, we require the kernel coefficients to be non-negative, which guarantees non-negative Patlak slope and intercept. Defining $\mathit{\alpha }=[{\mathit{\alpha }}_{\kappa }^{\text{'}},{\mathit{\alpha }}_b^{\text{'}}]\prime$, θ can be expressed as

$$\mathit{\theta }=({\mathit{I}}_2\otimes \mathit{K})\mathit{\alpha }.$$ (15)

Substituting (15) into (12), we get

$$\overline{\mathit{y}}=\mathit{N}[\mathit{A}\otimes (\mathit{P}\mathit{K})]\mathit{\alpha }+\mathit{s}+\mathit{r},$$ (16)

Algorithm 1 Algorithm for direct Patlak reconstruction using the kernel method
Input: Maximum iteration number MaxIt, sub-iteration number SubIt, starting time frame FraSta and ending time frame FraEnd
1:	Initialize α0, SllbIt = 1N
2:	for n = 1 to MaxIt do
3:	xn (A ⊗ K)αn–I, SubIt
4:	αn,0 = αn–I, SubIt
5:	for k = FraSta to FraEnd do
6:	${\mathit{x}}_k^{n+1}=\frac{{\mathit{x}}_k^n}{({\mathit{N}}_k\mathit{P}\mathit{K})\prime {\mathbf{1}}_M}[({\mathit{N}}_k\mathit{P}\mathit{K})\prime \frac{\mathit{y}}{({\mathit{N}}_k\mathit{P}\mathit{K}){\mathit{x}}_k^n+\mathit{s}+\mathit{r}}]$
7:	end for
8:	for q = 1 to SubIt do
9:	${\mathit{\alpha }}^{n,q}=\frac{{\mathit{\alpha }}^{n,q-1}}{[\mathit{A}\otimes (\mathit{P}\mathit{K})]\prime \mathit{N}{\mathbf{1}}_{MT}}([\mathit{A}\otimes (\mathit{P}\mathit{K})]\prime \mathit{N}{\mathbf{1}}_{MT}\frac{{\mathit{x}}^{n+1}}{(\mathit{A}\otimes {\mathit{I}}_N){\mathit{\alpha }}^{n,q-1}})$, where 1MT is a MT × 1 all-one vector.
10:	end for
11:	end for
12:	return θ̂ = (I2 ⊗ K)α̂MaxIt,SubIt

where we use the following property of Kronecker product

$$({\mathit{I}}_T\otimes \mathit{P})(\mathit{A}\otimes {\mathit{I}}_N)({\mathit{I}}_2\otimes \mathit{K})=\mathit{A}\otimes (\mathit{P}\mathit{K}).$$ (17)

To estimate α, we use the nested EM algorithm [46] for faster convergence. The resulting algorithm flowchart is shown in Algorithm 1.

D. Kernel construction

Following the original paper [25], we use a radial Gaussian kernel. The (i, j)th element of the kernel matrix K is

$$K({\mathit{f}}_i,{\mathit{f}}_j)=exp(-\frac{{\Vert {\mathit{f}}_i-{\mathit{f}}_j\Vert }^2}{2N_f{\sigma }^2}),$$ (18)

where f_i ∈ ℝ^{N_f × 1} and f_j ∈ ℝ^{N_f × 1} represent the feature vectors of voxel i and voxel j, respectively, σ² is the variance of the image and N_f is the number of voxels in a feature vector. Two types of feature vectors have been proposed before. In [25], one-hour long dynamic PET data were re-binned into three twenty-minutes frames and the reconstructed images ${\widehat{\mathit{x}}}_m^{\text{reb}}$, m = 1, 2, 3, were used to form the feature vectors ${\mathit{f}}_i^P$,

$${\mathit{f}}_i^P=\{{({\widehat{\mathit{x}}}_1^{\text{reb}})}_i,{({\widehat{\mathit{x}}}_2^{\text{reb}})}_i,{({\widehat{\mathit{x}}}_3^{\text{reb}})}_i\}.$$ (19)

Hereinafter, this method is referred to as KPET. The second method extracts the feature vectors from a co-registered MR image [26], [27],

$${\mathit{f}}_i^M=\{{({\mathit{x}}^M)}_i\},$$ (20)

where (x^M)_i is a local patch centered at voxel i from the MR image. This method is referred to as KMRI. In this paper, a 3×3×3 local patch was extracted for each voxel to form the feature vectors. Before the feature extraction, the re-binned PET images and MR image were normalized so that the image standard variation inside the brain region is 1 and we set σ = 1 in (18).

As shown in previous publications [26], [27], KMRI can preserve sharp boundaries in PET images. However, it can also introduce bias when PET tracer uptake does not share the same boundary with the MRI image. A better approach is to combine the two types of kernels so that we can minimize the risk of smoothing out PET signals while exploiting anatomical information from KMRI. An intuitive choice is to weight MRI and PET features equally, which we refer to as K-PET-MRI-0.5. This method would unfortunately reduce the image quality in the regions where PET image and MRI image share same boundaries compared with that using the KMRI alone, because PET features are often noisier than MRI features. A better approach is to assign a spatially variant weight to PET and MRI features based on a similarity measure ρ. Ideally the similarity measure ρ should be high (close to 1) in regions where PET and MRI images share the same boundary and be low (close to 0) in regions where their boundaries do not match.

Given a good similarity measure, we can weight PET and MRI features based on the similarity. The hybrid kernel matrix is then formed as

$$K({\mathit{f}}_i,{\mathit{f}}_j)=exp(-\frac{D_{ij}}{2N_f{\sigma }^2}).$$ (21)

where

$$D_{ij}=\frac{(1-{\rho }_i)(1-{\rho }_j){\Vert {\mathit{f}}_i^P-{\mathit{f}}_j^P\Vert }^2+3{\rho }_i{\rho }_j{\Vert {\mathit{f}}_i^M-{\mathit{f}}_j^M\Vert }^2}{(1-{\rho }_i)(1-{\rho }_j)+3{\rho }_i{\rho }_j}.$$ (22)

The factor 3 is included because there are three PET images in (19) and only one MRI image in (20). When multiple MRI images are available (e.g., [47]), the factor should be adjusted accordingly. Equation (22) is constructed so that D_ij is symmetric.

A crucial task is to find an appropriate similarity measure ρ. We studied two approaches to calculating ρ. The first one was based on the correlation between the PET kernel and MRI kernel vectors [48] and the second method was based on the structural similarity [49] between the reconstructed images of the last frame data using KPET and KMRI. We found that the kernel-correlation based result is not always satisfactory as the correlation between matched flat regions is dominated by image noise. Therefore, we focus on the structural similarity based method in this paper. This method is referred to as K-PET-MRI-S.

The structural similarity (SSIM) index [49] is a widely used image quality index for comparing images. Before calculating the SSIM index map, the input images are scaled to have a mean value of one inside the brain regions. Given two local patches extracted from the input images, x and y, the general definition of SSIM is

$$\text{SSIM}(\mathbf{x},\mathbf{y})={[l(\mathbf{x},\mathbf{y})]}^{\alpha }\cdot {[c(\mathbf{x},\mathbf{y})]}^{\beta }\cdot {[s(\mathbf{x},\mathbf{y})]}^{\gamma },$$ (23)

where $l(\mathbf{x},\mathbf{y})=\frac{2{\mu }_x{\mu }_y+C_1}{{\mu }_x^2+{\mu }_y^2+C_1}$, $c(\mathbf{x},\mathbf{y})=\frac{2{\sigma }_x{\sigma }_y+C_2}{{\sigma }_x^2+{\sigma }_y^2+C_2}$ and $s(\mathbf{x},\mathbf{y})=\frac{{\sigma }_{xy}+C_3}{{\sigma }_x{\sigma }_y+C_3}$ are the luminance, contrast and similarity measures. Here μ_x, μ_y, ${\sigma }_x^2$, ${\sigma }_y^2$ are the means and variances of x and y, and σ_xy is the covariance between x and y. To avoid block artifacts, neighboring voxels are weighted by an isotropic Gaussian function. The standard deviation of the Gaussian function was set to 1 pixel in our experiment, because we found that the default value (1.5 pixels) suggested by [49] resulted in a blurry SSIM map for our application. C₁, C₂ and C₃ are small constants used to stabilize the results. We used the MATLAB implementation [49] with C₁ = (0.01L)², C₂ = (0.03L)² and C₃ = C₂/2, where L is a constant value. Based on our observations, the most influencing component of SSIM(x, y) for distinguishing an unmatched region from matched regions is the similarity measure s(x, y). Hence we fixed α and β to 1 and adjusted γ and L. We evaluated the mean SSIM values for a simulated unmatched tumor region and matched normal region for a wide range of γ and L values. We found that setting γ = 12 and L = 50 is a reasonable choice, which resulted in a low SSIM value for the unmatched regions while keeping the SSIM value for the matched regions to be greater than 0.95. More details of the calculation are given in the supplemental materials.

After obtaining SSIM index, we calculate ρ by scaling the SSIM index

$${\rho }_i=max(0,\frac{{\text{SSIM}}_i-C}{1-C}),$$ (24)

where C was set to 0.2. The reason for scaling is that the minimum value of the SSIM value is around 0.2 while the range of ρ_i should be between 0 and 1.

For efficient computation, all kernel matrices were constructed using a K-Nearest-Neighbor (KNN) search in a 7×7×7 search window with K = 50, i.e.,

$$K_{ij}=\{\begin{array}{ll}K({\mathit{f}}_i,{\mathit{f}}_j),\hfill & {\mathit{f}}_j\in K\text{NN of}{\mathit{f}}_i,\hfill \\ 0,\hfill & \text{otherwise}.\hfill \end{array}$$ (25)

Following [25], all kernel matrices were normalized by

$$\overline{\mathit{K}}={\text{diag}}^{-1}[\mathit{K}{\mathbf{1}}_N]\mathit{K}$$ (26)

and K̄ was then used in the image reconstruction.

III. Experiment evaluations

A. Computer simulation study

A 3D brain phantom from the Brainweb [50] was used in the simulation study. The scanner geometry models a Siemens mCT scanner [51]. The system matrix P was computed by using the multi-ray tracing method [52]. The image matrix size is 125 × 125 × 105 and the voxel size is 2 × 2 × 2 mm³. The time activity curves of the gray matter and white matter were generated mimicking an FDG scan using a two-tissue-compartment model with the kinetic parameters shown in Table I and an analytic blood input function [53] in Fig. 1. To simulate mismatches between the MR and PET images, twelve hot spheres of diameter 16 mm were inserted into the PET image as tumor regions, which are not visible in the MR image. The dynamic PET scan was divided into 24 time frames: 4×20 s, 4×40s, 4×60 s, 4×180 s, and 8×300 s. For the direct Patlak reconstruction, only the last 5 frames for a total duration of 25 minutes were used. Three orthogonal slices of the MRI prior image, PET attenuation map and PET activity image at the 24^th frame are shown in Fig. 2. Noise-free sinogram data were generated by forward-projecting the ground-truth images using the system matrix and the attenuation map. Poisson noise was then introduced to the noise-free data by setting the total count level to be equivalent to an 1-hour FDG scan with 5 mCi injection. Uniform random events were simulated and accounted for 30 percent of the noise free data in all time frames. Scatters were not simulated in this study. During image reconstruction, the blood input function and all the correction factors were assumed to be known exactly. Apart from the tumor study, we also performed an neural activation simulation study by simulating the activation regions according to [17]. Results are shown in the supplemental materials.

Table I. The simulated kinetic parameters of FDG.

Tissue	fυ	K1	k2	k3	k4
Gray matter	0.03	0.071	0.086	0.055	0.001
White matter	0.03	0.046	0.080	0.052	0.001
Tumor	0.05	0.082	0.055	0.085	0.002

Fig. 1.

The simulated time activity curves.

Fig. 2.

Three orthogonal slices of the MR prior image (first column), attenuation map (second column) and the 24th frame of the PET ground truth image (third column).

To quantify the noise properties across different realizations, multiple independent and identically distributed (i.i.d.) dynamic data sets were generated. The KPET kernel matrix and the hybrid kernels were calculated for each realization. The KMRI kernel matrix was kept the same in all realizations. When constructing the PET features for each realization, the first, second and last twenty-minutes frames were reconstructed using the ordered-subsets EM (OSEM) algorithm (10 iterations and 6 subsets), and smoothed by a Gaussian filter with a FWHM of 3 mm. We then used the kernels to reconstruct the last static frame as well as to perform the direct Patlak reconstruction using the last 5 frames. For the static reconstruction, 20 iterations with 6 subsets were run, while for the direct Patlak reconstruction, 100 iterations of the nested EM algorithm with N_s = 3 sub-iterations were run.

For quantitative comparison, contrast recovery coefficient (CRC) vs. the standard deviation (STD) curves were plotted. The CRC was computed between selected gray matter regions and white matter regions as

$$\text{CRC}=\frac{1}{R}\sum _{r=1}^R(\frac{{\overline{a}}_r}{{\overline{b}}_r}-1)/(\frac{a^{\text{true}}}{b^{\text{true}}}-1).$$ (27)

Here R is the number of realizations, which is equal to 50 in both the static reconstruction and the direct Patlak reconstruction. ${\overline{a}}_r=1/K_a{\sum }_{k=1}^{K_a}{a}_{r,k}$ is the average uptake of the gray matter over K_a ROIs in realization r. The ROIs were drawn in both matched gray matter regions and the unmatched tumor regions. The reason to quantify both the gray matter and tumor regions is to compare the performance of different methods under the situations with either matched or unmatched MR information. For the case of matched gray matter, we had K_a = 10, and for the unmatched tumor regions, we had K_a = 12. When choosing the matched gray matter regions, only those pixels inside the predefined 20-mm-diameter spheres and containing 80% of gray matter were included. ${\overline{b}}_r=1/K_b{\sum }_{k=1}^{K_b}{b}_{r,k}$ is the average value of the background ROIs in realization r, and K_b = 37 is the total number of background ROIs. The background ROIs were 12-mm spheres drawn in the white matter. The background STD was computed as

$$\text{STD}=\frac{1}{K_b}\sum _{k=1}^{K_b}\frac{\sqrt{\frac{1}{R-1}{\sum }_{r=1}^R{(b_{r,k}-{\overline{b}}_k)}^2}}{{\overline{b}}_k},$$ (28)

where ${\overline{b}}_k=1/R{\sum }_{r=1}^R{b}_{r,k}$ is the average of the kth background ROI over realizations, and K_b = 37 is the total number of background ROIs.

B. Hybrid real data

A 70-minutes dynamic FDG PET scan of a human subject was acquired on a Siemens Brain MR-PET scanner after a 5-mCi injection. The dynamic PET data were divided into 25 frames: 4×20 s, 4×40 s, 4×60 s, 4×180 s, 8×300 s and 1×600 s. The data were reconstructed into an image array of 256×256×153 voxels with a voxel size of 1.25 × 1.25 × 1.25 mm³ using different kernel matrices as those used in the simulation study. Correction factors for randoms, scatters were estimated using the standard software provided by the manufacturer and included in the forward model during reconstruction [54], [55]. The motion correction was performed in the LOR space based on the simultaneously acquired MR navigator signal [56]. Attenuation was derived from a T1-weighted MR image using the SPM based atlas method [57]. The first 20-minutes, middle 20-minutes and last 30-minutes PET data were reconstructed separately to form the PET features. The MR features were extracted from the T1-weighted MR image, which has the same voxel size as the PET images. To obtain the blood input function, blood regions were segmented from the MR image by thresholding inside a predefined mask as the blood vessel is bright in the T1-weighted MR image. The blood input function was then computed from the frame-by-frame OSEM reconstruction. A maximum intensity projection of the segmented blood region overlaid with the MRI image is shown in Fig. 3(a) and the extracted blood input function without decay correction is shown in Fig. 3(b). As the ground truth of the regional uptake is unknown, a hot sphere of diameter 12.5 mm, mimicking a tumor, was added to the PET data (invisible in the MRI image). It simulates the case where MRI and PET information does not match. The location of the tumor is shown in Fig. 7. The TAC of the hot sphere was set to the TAC of the gray matter, so the final TAC of the simulated tumor region is higher than that of the gray matter because of the superposition. The simulated dynamic tumor images were forward-projected to generate a set of noise-free sinograms, taking into account of detector normalization and patient attenuation. Randoms and scatters from the inserted tumor were not simulated because they would be negligible compared with the scattered and random events from the patient background. Poisson noise was then introduced and finally the tumor sinograms were added to the original patient sinograms to generate the hybrid real data sets. Apart from the inserted tumor, the left putamen region was also used in the quantification, considering only its absolute uptake as the ground truth is unknown. The Patlak plots of the left putamen and the inserted tumor along with the fitted lines are shown in Fig. 3(c,d), respectively, which indicate that the Patlak model is reasonable.

Fig. 3.

(a) The segmented MRI blood vessels; (b) The extracted blood input curves based on the MRI segmented vessels; (c) The Patlak plot for the left putamen TAC; and (d) the Patlak plot for the inserted tumor TAC.

Fig. 7.

The similarity maps used in K-PET-MRI-S for the hybrid real patient data. The tumor region is marked by the black circles and also indicated by the arrows.

We evaluated both the static reconstruction and direct Patlak reconstruction. To generate multiple realizations for the static reconstruction, the last 40 minutes PET data were pooled together and resampled with a 3/40 ratio to obtained 50 i.i.d. datasets that mimic 3-minutes frames. Specifically, for every list-mode event, we generated a random number uniformly distributed between 0 and 1 and the event was accepted only if the random number is less than 3/40 [58]. This process was repeated 50 times. The data were reconstructed using different methods to compare the bias versus standard deviation curves of the tumor uptake.

For the direct Patlak reconstruction, the last six dynamic frames were used. For each frame, 50 independent realizations were generated by randomly sampling the data with a 1/5 ratio. The resampled data mimic a set of dynamic data with a 1-mCi FDG injection. For tumor quantification, images with and without the inserted tumor were reconstructed and the difference was taken to obtain the tumor only image and compared with the ground truth. The tumor contrast recovery was calculated as

$$\text{Contrast Recovery}=\frac{1}{R}\sum _{r=1}^R{\overline{l}}_r/l_{\text{true}}$$ (29)

where l̄_r is the mean tumor intensity inside the tumor ROI, l_true is the ground truth of the tumor intensity, and R is the number of the realizations. For the background, 23 spherical ROIs with a diameter of 12.5 mm were drawn in the white matter and the standard deviation was calculated according to (28).

IV. Results

A. Simulation results

Fig. 4 shows the similarity map computed in K-PET-MRI-S. Most tumor regions are successfully identified by low values in the similarity map. The region in the coronal slice that shows low similarity but is outside the tumor region is caused by the blurring of a low-similarity tumor region in a neighboring coronal slice. Fig. 5 shows three orthogonal slices of the Patlak slope images reconstructed by different methods together with the ground truth images. Besides kernel based reconstructions, we also included the post-smoothed direct EM reconstruction (denoted by ‘EM+filter’). Note that the direct EM reconstruction was simply a special case of the kernel EM algorithm given in Algorithm 1 with K = I. A Gaussian filter was used to reduce the variance of the direct EM reconstruction to a comparable range to the kernel methods. By visual inspection, the Patlak slope image reconstructed using KMRI has a higher resolution than the KPET method. The fine details are also preserved by the hybrid kernels as pointed by the arrows.

Fig. 4.

The similarity maps used in K-PET-MRI-S for the simulated data set. Circles indicate the locations of the inserted tumors.

Fig. 5.

Three orthogonal slices of the reconstructed Patlak slopes using different methods for the simulation dataset.

Fig. 6 shows the CRC vs. STD curves for the matched gray-matter regions and tumor regions in the last static frame and the Patlak slope images. The error bars depict the standard deviation of the CRC values computed from 50 independent realizations (the gray-matter regions have larger error bars than the tumor regions because the tumor regions have more voxels and also higher intensity). We can see that for the matched gray-matter regions, KMRI method has the highest CRC value. The performance of K-PET-MRI-S is very close to that of KMRI. Both have much lower noise than either K-PET-MRI-0.5 and KPET, which are better than the EM + filter. For the tumor regions that are not visible in the MRI prior image, KMRI method has the lowest CRC, while KPET and K-PET-MRI-0.5 have higher CRC but also higher noise. By using the spatially adaptive weighting, K-PET-MRI-S achieves a slightly higher CRC than both KPET and KMRI while keeping the noise at the same level as that of KMRI. The noise in K-PET-MRI-0.5 method is very close to that of the KPET method because the MRI intensity in the white matter is almost uniform and does not have much influence in determining the kernel matrix when combined with PET features with equal weights. Overall, the K-PET-MRI-S method has the best performance.

Fig. 6.

CRC-STD curves at the gray matter region (left column) and the tumor region (right column) for the last frame reconstruction (top row) and the Patlak direct reconstruction (bottom row). For the static results, markers are plotted every two iterations with the lowest point corresponding to the 6^th iteration. For the Patlak results, markers are plotted every ten iterations with the lowest point corresponding to the 30^th iteration.

B. Hybrid real data results

Fig. 7 shows the similarity map in K-PET-MRI-S for the hybrid real data. The artificially inserted tumor was successfully identified by low similarity values. In other regions, we can see that the similarity values are lower than those of the simulation data. This could be caused by mismatch between the real MRI and PET data. Fig. 8 shows the reconstructed Patlak slope images of the patient data by different methods. Using the KMRI method, the cortices can be seen more clearly, but the shape of the tumor is distorted as the spherical tumor does not exist in the MRI image. The KPET method can preserve the spherical shape of the tumor, but the details of the cortex are lost. By using the hybrid kernels, the shapes of both the cortexand the tumor can be recovered.

Fig. 8.

Three orthogonal slices of the reconstructed Patlak slopes for the hybrid real patient data. The first column is the MR prior image.

Fig. 9 shows the tumor constrast recovery vs. STD curves, as well as the left putamen absolute uptake vs. STD curves, for the last frame static reconstruction and Patlak direct reconstruction. For the tumor region, we can see that KMRI method has the lowest noise, but also the lowest uptake ratio. KPET has higher contrast, but also higher noise. The performance of K-PET-MRI-0.5 is slightly better than that of KPET, but the noise is still higher than that of KMRI. Among all the methods, K-PET-MRI-S has the best performance with the high contrast close to that of KPET and low noise close to that of KMRI.

Fig. 9.

Quantification results of the hybrid real dataset for (a,c) last frame static reconstruction (markers are plotted for every iteration with the lowest point corresponding to the third iteration) and (b,d) Patlak direct reconstruction (markers are generated for every ten iterations with the lowest point corresponding to the 30^th iteration).

Comparing the results in Fig. 6 and Fig. 9, we can see that the noise reduction between the KMRI method and the KPET method is larger in the simulation than in the real data. We hypothesized that the reason for the difference was that the KPET kernel was recomputed for each realization in the simulation study while it was computed once using the full dynamic data in the real data experiment. This kernel variability in the simulation study introduces extra fluctuations in the reconstructed images of KPET. To test this hypothesis, we reprocessed the simulation data using the same procedure used in the real data experiment: (1) in the static reconstruction, 5 min data set were resampled from the last 40 min scan; (2) in the direct Patlak reconstruction, each of the last five frames was resampled to one-fifth of the events. The KPET kernel was computed once using the 60-minute data. Fig. 10 shows the noise comparison between KPET and KMRI methods using the resampling approach with a fixed KPET kernel. In static reconstruction, the noise reduction between KMRI and KPET is 16% (down from 25% in Fig. 6), and in the Patlak direct reconstruction, the noise reduction factor is 17% (down from 37% in Fig. 6). These values are consistent with the noise reduction factors that we saw in the real data experiment (Fig. 9).

Fig. 10.

Noise comparison between KPET and KMRI using the resampling method and a fixed KPET kernel. (a) Static image. (b) Patlak slope image. The red diamonds represent the noise reduction factor between KMRI and KPET.

V. Discussion

From the simulation data sets, we found that the noise reduction by KMRI over KPET is greater in the Patlak reconstruction than in the static reconstruction. In static reconstruction the noise reduction by KMRI over KPET is 25%, while in Patlak direct reconstruction the noise reduction by KMRI over KPET is 37%. This means including MRI information can have a greater impact on the Patlak direct reconstruction than on static reconstruction. One possible explanation for this is that in Patlak reconstruction, temporal information has been exploited through the Patlak model and the effect of temporal information derived from KPET is not as helpful as that in static reconstruction. In comparison, the noise reduction by KMRI over the direct EM reconstruction is less affected by the Patlak model than that of KPET because MRI information is separate from PET data.

In the simulation study, all reconstruction methods used the same system model that was used for the data generation, so the inconsistency in the data was solely due to Poisson noise. In the hybrid real data experiment, the reconstruction model was a ray-tracing model, since we do not know the true system model of the real scanner. Therefore, in addition to statistical noise, there also existed modeling error in the real data experiment. Despite the difference between the computer simulation and real data experiments, the proposed method worked reasonably well in both situations, demonstrating its robustness to the change of the system model used in reconstruction. While we did not use time-of-flight (TOF) information in our study, the proposed method can be directly applied to TOF reconstruction by adopting a TOF-based system model. There would be no change to the kernel matrix calculation because it is performed in the image space.

When combining MRI and PET information, assigning an equal weight to MRI and PET features (K-PET-MRI-0.5) can reduce the bias caused by missing information in the MRI prior image. However, for matched regions, K-PET-MRI-0.5 cannot fully take advantage of the MRI information. This is the reason that the K-PET-MRI-0.5 method does not improve the noise performance very much compared with the KPET method. KPET-MRI-S method uses SSIM as an index to compute the similarity between the last-frame reconstructions of KPET and KMRI methods. The SSIM method works well to identify the mismatched regions in both simulation and real data sets. As a result, K-PET-MRI-S achieved the best performance among all methods. In the real data experiment, we did observe low similarity regions outside the artificially inserted tumor. Currently, we cannot verify whether they were caused by real difference between MRI and PET information, or by the instrumentation resolution difference. More real data evaluation is needed to answer this question.

Finally, we note that for dynamic PET studies, motion correction may be necessary due to involuntary subject movements during long dynamic scans. In our study, the real PET data were corrected for motion using a simultaneously acquired MR navigator signal [56]. When we applied the kernel methods to patient data without MR-guided motion correction, we observed MR-induced artifacts in KMRI reconstructed images, possibly due to mismatches between MRI and PET data.

VI. Conclusion

We have proposed a hybrid kernel approach to direct Patlak reconstruction using both PET and MRI information. The results from simulation and real data experiments show that the proposed K-PET-MRI-S method provides a robust way to handle the potential mismatch between MRI and PET data while using matched MRI information to improve image quality. Future work will include more clinical evaluations.