PET Image Deblurring and Super-Resolution with an MR-Based Joint Entropy Prior

Abstract

The intrinsically limited spatial resolution of PET confounds image quantitation. This paper presents an image deblurring and super-resolution framework for PET using anatomical guidance provided by high-resolution MR images. The framework relies on image-domain post-processing of already-reconstructed PET images by means of spatially-variant deconvolution stabilized by an MR-based joint entropy penalty function. The method is validated through simulation studies based on the BrainWeb digital phantom, experimental studies based on the Hoffman phantom, and clinical neuroimaging studies pertaining to aging and Alzheimer’s disease. The developed technique was compared with direct deconvolution and deconvolution stabilized by a quadratic difference penalty, a total variation penalty, and a Bowsher penalty. The BrainWeb simulation study showed improved image quality and quantitative accuracy measured by contrast-to-noise ratio, structural similarity index, root-mean-square error, and peak signal-to-noise ratio generated by this technique. The Hoffman phantom study indicated noticeable improvement in the structural similarity index (relative to the MR image) and gray-to-white contrast-to-noise ratio. Finally, clinical amyloid and tau imaging studies for Alzheimer’s disease showed lowering of the coefficient of variation in several key brain regions associated with two target pathologies.

I. Introduction

POSITRON emission tomography (PET) is an in vivo medical imaging technique that enables visualization of the distribution of radiotracers that bind to specific molecular targets with functional or physiological significance [1]–[3]. The spatial resolution of PET is intrinsically very limited, thereby hindering its potential as a quantitative imaging tool, particularly for applications involving localized or constricted imaging targets. PET imaging systems capture coincidences of emitted pairs of photons traveling approximately collinearly in opposite directions following positron-electron annihilation events. A number of physical factors, some related to the imaging system hardware, limit PET resolution. These include the non-collinearity of the emitted photon pairs, intercrystal scatter, crystal penetration, and non-zero positron range of PET radionuclides [4]–[6]. Furthermore, it is fairly commonplace to use smoothing penalties during image reconstruction or to apply smoothing filters post-reconstruction for lowering the noise levels in the final images. This image-domain smoothing further compounds the resolution challenge [7]. Attempts to alleviate this challenge have led to the development of sophisticated edge-preserving penalties [8]–[10] or the incorporation of image-domain or sinogram-domain point spread functions (PSFs) in the PET image reconstruction framework [11]–[13]. Some techniques incorporate anatomical information from a segmented magnetic resonance (MR) image to guide the reconstruction process [14]–[19]. In contrast to these reconstruction methods, this paper aims to improve PET quantitation by means of a post-reconstruction processing technique that is applicable to existing images. Post-processing approaches have wider adoptability because they obviate the need for raw data. Furthermore, their relative simplicity resulting from the decoupling of the deblurring and reconstruction problems makes their adoption in a clinical setting easier than that for reconstruction-based tools.

In neurological contexts, quantitation based on PET usually entails the computation of a mean activity measure, such as standardized uptake value ratio (SUVR) or binding potential, over different regions-of-interest (ROIs). The combined effect of image blurring and spatial sampling for image digitization (the latter referred to as tissue fractioning) leads to the socalled partial volume effect, which manifests in PET images as spillover of estimated activity across different ROIs [20]. PET resolution recovery is key to the development of accurate image-based biomarkers based on ROI measures for Alzheimer’s disease and other neurological disorders [21]–[23]. From an image processing perspective, the combination of deblurring and upsampling to improve resolution and reduce tissue fractioning respectively is a super-resolution imaging problem [24]. In this paper, we present a super-resolution imaging framework for PET that incorporates anatomical information to assist and stabilize the deconvolution procedure required for deblurring.

While a range of iterative deconvolution techniques exist in literature, including those with additive or multiplicative updates like the Richardson-Lucy [25], [26] and Van Cittert [27] algorithms, standalone deconvolution is an ill-posed problem characterized by poor convergence and noise amplification. A variety of strategies have been employed to control noise, including early termination, edge-preserving penalty functions such as total variation [28], [29], and topology based regularization [30]. In [31], an MR-guided filtering strategy was demonstrated to be effective both as an edge-preserving post-filtering tool and as a deblurring tool. In [29], the PET image was modeled as a Markov random field with Gibbs interactions terms weighted by MR-derived edge information. Some correction techniques use wavelet or other transform domains for extracting anatomical information [32]. Additionally, a broad array of methods exist that attempt to correct for partial volume effects using a geometric transfer matrix (GTM) derived from anatomical parcellations [20], [33]–[37]. These methods assume (i) an approximate PSF that is stationary across the scanner bore and (ii) accurate segmentation of an anatomical image, which is often challenging. Here, we present an anatomically-guided PET image deblurring framework inspired by the success of information-theoretic priors in reconstruction [38], [39]. Unlike GTM-based methods, this technique does not require prior segmentation of the anatomical image. We offered a proof of concept for this idea in our earlier paper on this topic [40]. In this paper, we validate the idea using realistic simulation and phantom datasets and apply it to tau and amyloid imaging datasets from neuroimaging studies on Alzheimer’s disease. The details of estimation and optimization are presented in section II. Section III provides details on the simulation setup, experimental data acquisition, and validation. Section IV outlines the results obtained using this approach in comparison with other techniques. A discussion and a summary of this method are presented in sections V and VI respectively.

II. Theory

A. Blurring and Downsampling Model

PET images exhibit spatially-variant resolution depending on the radial and axial coordinates inside the cylindrical scanner bore. The PSF for PET can be modeled in either the sinogram (detector) domain or the image (voxel) domain. While modeling the PSF within the reconstruction framework, the choice between these two alternatives is guided by whether the primary contributor to blurring is the detector hardware (a sinogram-domain effect) or the positron range of the radioisotope (an image-domain effect). Post-reconstruction deblurring techniques, however, must exclusively rely on image-domain blur modeling. Here we model the blur using a series of image-based PSFs measured at multiple radial and axial locations in the scanner bore [12], [41], [42]. At all in-between locations, the blur model is determined as a linear combination of the PSF models at the nearest points of measurement, assuming radial and axial symmetry. As will be described in section III, to avoid an inverse crime, the simulation dataset for validation in this paper is generated using a sinogram-domain PSF while the correction is performed using an image-domain PSF model. Because the PSF depends on a number of factors, including the radioisotope choice and the scanner properties, we focused both simulation and experimental studies in this paper on a single radioisotope and scanner combination. The Siemens ECAT EXACT HR+ scanner was chosen due to its widespread use, and the ¹⁸F radioisotope was chosen as it is one of the preferred radionuclides for PET imaging studies.

Our framework utilizes a low-resolution 3D PET image which is scanner-generated and has n voxels and a high-resolution MR image with N voxels (N > n) to generate a super-resolution PET image with N voxels (same as the number of voxels in the scanner-generated high-resolution MR image). The PET image blurring and downsampling operation can be modeled using a system matrix $\mathit{A}\in {\mathbb{R}}^{n\times N}$, such that [43]:

$$\mathit{A}=\sum _{k=1}^K{\mathit{D}}_k{\mathit{H}}_k\mathit{S}.$$ (1)

Here ${\mathit{H}}_k\in {\mathbb{R}}^{n\times n}$ is a block Toeplitz matrix representing the kth PSF, $\mathit{S}\in {\mathbb{R}}^{n\times N}$ is a downsampling matrix, and K is the total number of PSFs. Linear combinations of the individual blur models are created using a series of diagonal matrices ${\mathit{D}}_k\in {\mathbb{R}}^{n\times n}$ containing interpolation weights, such that:

$$\sum _{k=1}^K{\mathit{D}}_k=\mathit{I},$$ (2)

where $\mathit{I}\in {\mathbb{R}}^{n\times n}$ is the identity matrix. For an unknown high-resolution PET image vectorized as $\mathit{x}\in {\mathbb{R}}^N$ and a scanner-reconstructed low-resolution PET image $\mathit{b}\in {\mathbb{R}}^n$, the least squares cost function for data fidelity is:

$${\Phi }_{\text{PSF}}(\mathit{x}\mid \mathit{b})=\frac{1}{2}(\mathit{A}\mathit{x}-\mathit{b}{)}^T(\mathit{A}\mathit{x}-\mathit{b}).$$ (3)

Multiplication by the forward model matrix (Ax) and its transpose (A^T b) can be efficiently computed as linear combinations of 3D fast Fourier transform (FFT) outputs.

B. Joint Entropy Penalty

Most clinical and preclinical PET scanners come equipped with anatomical imaging capabilities, in the form of computed tomography (CT) or MR imaging, to complement the functional information in PET with structural information. We denote the unknown PET image as a vector $\mathit{x}\in {\mathbb{R}}^N$, where N is the number of voxels in the image. Dividing the intensity range of the image into M equal-sized intensity bins, we create an intensity histogram vector $\mathit{u}\in {\mathbb{R}}^M$ from x, where u_i is the number of voxels in the ith intensity bin. We denote the MR image as a vector $\mathit{y}\in {\mathbb{R}}^N$ and the corresponding intensity histogram based on M equal-sized intensity bins as $\mathit{v}\in {\mathbb{R}}^M$. The joint entropy (JE) penalty function can be computed from the joint probability density function (pdf) in the intensity spaces of the two images as follows:

$${\Phi }_{\text{JE}}(\mathit{x}\mid \mathit{y})=-\sum _{i=1}^M\sum _{j=1}^M\delta u\delta vp(u_i,v_j)\log p(u_i,v_j).$$ (4)

Here δu and δv are the widths of the discrete intensity bins and p(u_i, V_j) is the ijth element of the joint pdf computed on the regularly spaced intensity grid. We approximately calculate the joint pdf using a Parzen window technique:

$$p(u_i,v_j)=\frac{1}{N}\sum _{k=1}^{N}G(u_i;x_k,{\sigma }_u)G(v_j;y_k,{\sigma }_v),$$ (5)

where $G(u;\mu ,\sigma )=1∕(\sqrt{2\pi }\sigma )\text{exp}[-(u-\mu {)}^2∕(2{\sigma }^2)]$ is a 1D Gaussian kernel. Assuming σ_u >> δu and σ_v >> δv, the pdf can be obtained through a 2D convolution operation, which can be efficiently computed using the FFT:

$$\begin{gathered} p(u_i,v_j)\approx \mathcal{F}\ast \mathcal{G}(u_i,v_j), \\ \mathcal{G}(u_i,v_j)\triangleq G(u_i;0,{\sigma }_u)G(v_j;0,{\sigma }_v), \\ \mathcal{F}(u_i,v_j)\triangleq \frac{1}{N}\sum _{k=1}^N\wedge (u_i-x_k;{\delta }_u)\wedge (v_j-y_k;{\delta }_v). \end{gathered}$$ (6)

Here $\mathcal{G}(u_i,v_j)$ represents the 2D convolution kernel, and $\mathcal{F}(u_i,v_j)$ is computed through bilinear interpolation of the joint histogram using the triangle function:

$$\wedge (u;\delta u)=\{\begin{array}{cc}1-\frac{\mid u\mid }{\delta u}\hfill & \text{if}\frac{\mid u\mid }{\delta u}<1,\hfill \\ 0\hfill & \text{otherwise}.\hfill \end{array}\phantom{\}}$$ (7)

C. Iterative Optimization

The target image is the solution to the following constrained optimization problem:

$$\widehat{\mathit{x}}=\text{arg}\underset{\mathit{x}>0}{\text{min}}{\Phi }_{\text{PSF}}(\mathit{x})+\beta {\Phi }_{\text{JE}}(\mathit{x}).$$ (8)

To determine the optimal image, we adopt a gradient-based optimization strategy based on an iterative update:

$${\mathit{x}}_{k+1}={\mathit{x}}_k+{\alpha }_k{\mathit{d}}_k,k=0,1,2,\dots ,$$ (9)

where d_k is the search direction at the kth iteration and α_k is a scalar step size, which is typically computed using a backtracking line search. A number of gradient-based approaches can be employed to determine the descent direction d_k. In Newton’s method, for example, the inverse of the Hessian is used as a preconditioner which modifies the negative gradient to compute the descent direction:

$${\mathit{d}}_k=-[\mathit{H}(x_k){]}^{-1}{\mathit{g}}_k.$$ (10)

Here H(x_k) is the Hessian matrix (comprised of second derivative terms) and g_k is the gradient computed at the kth iteration. The quasi-Newton family of methods avoids the expensive computation of the Hessian matrix and its inverse using first-order approximations. The descent direction is based on a preconditioned negative gradient derived from an inverse Hessian approximation (denoted L_k):

$${\mathit{d}}_k=-{\mathit{L}}_k{\mathit{g}}_k.$$ (11)

To efficiently solve the constrained optimization problem in (8), we use a quasi-Newton type preconditioner based on the Limited-Memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) method. A variant of this approach known as L-BFGS-B that combines L-BFGS updates with a gradient-projection strategy to impose box or bound constraints was utilized to ensure that the solution is restricted to the non-negative orthant. In the L-BFGS-B implementation, L_k is computed from first order-derivatives alone and is iteratively updated as follows:

$${\mathit{L}}_{k+1}=\left[\mathit{I}-\frac{\mathit{s}{\mathit{t}}^T}{{\mathit{s}}^T\mathit{t}}\right]{\mathit{L}}_k\left[\mathit{I}-\frac{\mathit{t}{\mathit{s}}^T}{{\mathit{s}}^T\mathit{t}}\right]+\frac{{\mathit{ss}}^T}{{\mathit{s}}^T\mathit{t}},$$ (12)

where $\mathit{I}\in {\mathbb{R}}^{N\times N}$, s = x_k+1 − x_k, and t = g_k+1 − g_k.

D. Gradient Computation

The gradient at the kth iteration is computed as a weighted sum of the cost and penalty gradients:

$${\mathit{g}}_k=\nabla {\Phi }_{\text{PSF}}+\beta \nabla {\Phi }_{\text{JE}}.$$ (13)

1). Data-Fidelity Cost:

The gradient for the data-fidelity cost function is given by:

$$\nabla {\Phi }_{\text{PSF}}={\mathit{A}}^T(\mathit{A}\mathit{x}-\mathit{b}).$$ (14)

2). JE Penalty:

The gradient for the JE penalty can be computed from partial derivatives of the joint probability density function as follows:

$$\frac{d{\Phi }_{\text{JE}}}{d{x}_k}=-\sum _{i=1}^M\sum _{j=1}^M\delta u\delta v(1+\log p(u_i,v_j))\frac{dp(u_i,v_j)}{d{x}_k},$$ (15)

where dp(u_i, v_j)/dx_k can be approximately but efficiently computed by 2D FFT-based convolution:

$$\begin{gathered} \frac{dp(u_i,v_j)}{d{x}_k}=\frac{1}{N}G(u_i;x_k,{\sigma }_u)\left(\frac{u_i-x_k}{{\sigma }_u^2}\right)G(v_j;y_k,{\sigma }_v) \\ \approx {\mathcal{F}}^k\ast {\mathcal{G}}^{\prime }(u_i,v_j), \\ {\mathcal{G}}^{\prime }(u_i,v_j)\triangleq G(u_i;0,{\sigma }_u)\left(\frac{u_i}{{\sigma }_u^2}\right)G(v_j;0,{\sigma }_v), \\ {\mathcal{F}}^k(u_i,v_j)\triangleq \frac{1}{N}\wedge (u_i-x_k;\delta u)\wedge (v_j-y_k;\delta v). \end{gathered}$$ (16)

III. Methods

A. Overview

In the following, we describe a simulation study using the BrainWeb digital phantom and the ¹⁸F-fluorodeoxyglucose (¹⁸F-FDG) radiotracer, an experimental phantom study using the Hoffman phantom and the ¹⁸F-FDG radiotracer, and two clinical neuroimaging studies for Alzheimer’s disease, one based on ¹⁸F-florbetapir (also known as ¹⁸F-AV-45), which targets amyloid plaques, and another based on ¹⁸F-flortaucipir (also known as ¹⁸F-AV-1451), which targets tau tangles. Amyloid plaques and tau tangles are two types of misfolded proteins that are pathological hallmarks of Alzheimer’s disease. All studies were based on the Siemens ECAT EXACT HR+ scanner. All low-resolution PET images have 2.54 mm × 2.54 mm × 2.43 mm voxels with a 128×128×63 grid size consistent with the HR+ output. All super-resolved PET images have 1 mm × 1 mm × 1 mm voxels with a 256×256×256 grid size consistent with the anatomical MR images used in this work. All PET images were reconstructed using the ordered subsets expectation maximization (OSEM) algorithm (6 iterations, 16 subsets) followed by 3D Gaussian filtering with 6 mm full width at half maximum (FWHM).

B. Simulation Setup

1). PSF Generation:

For ¹⁸F radiotracers, the positron range is negligible, and therefore the sinogram blurring model is reasonably accurate. A 1D radially-varying blur kernel has been demonstrated to be realistic via Monte Carlo simulations [12], [13]. All simulation data in the projection (sinogram) space were therefore subjected to a 1D radial blur. Axial variation in the blur kernel was not modeled in the simulation but has been incorporated into the PSF model for the phantom and clinical data subsequently described. Eight image-domain 3D PSFs corresponding to different radial locations were computed by reconstructing unit point sources at each location. These eight sampled PSFs, all fitted to Gaussian kernels, were used in the forward model in (1) for spatially-variant deblurring. The fitted PSFs at different radial locations (forming H_k) and the corresponding distribution of interpolation weights (forming D_k) in the transverse image plane are visualized in Fig. 1(a) and (b) respectively.

Fig. 1.

(a) Gaussian PSFs measured and fitted in the image domain from a simulated radially-varying sinogram-domain blur. The PSFs were sampled at eight radial locations starting at the center of the scanner bore and going outward. A dotted line has been included for reference. (b) Transverse slices showing the radially-varying linear interpolation weights (corresponding to the eight labeled PSFs). The weights at a given location are non-zero only for the two nearest PSFs.

2). Data Generation:

To validate the deblurring framework, we performed realistic simulations using the 3D BrainWeb digital phantom (http://brainweb.bic.mni.mcgill.ca/brainweb/). A T1-weighted MR image with 1 mm³ isotropic resolution was used from the BrainWeb database, and its corresponding segmented label volume with gray matter, white matter, blood pool, and cerebrospinal fluid ROIs was used to simulate a PET “ground truth” image. Dynamic PET time activity curves were simulated with ROI kinetic parameters as listed in Table I for gray matter (GM) and white matter (WM) tissue types. The dynamic frames were summed to generate a “ground-truth” static PET image arising from a 1 hour long ¹⁸F-FDG scan with a mean gray-to-white intensity ratio of 1.43. The geometric model of the HR+ scanner with sinogram-domain blurring was used to generate sinogram data. Noisy data were generated using Poisson noise realizations of the projected sinograms, a noise model widely accepted in the PET imaging community [44], The Poisson deviates were generated with a mean of 10⁸ counts for the full scan duration of 3640 s.

TABLE I.

Simulated kinetic parameters for different ROIs in the brain

ROI	Blood fraction	K1 (ml min−1 g−1)	k2 (min−1)	k3 (min−1)	k4 (min−1)
GM	0.0446	0.1104	0.1910	0.1024	0.0094
WM	0.0270	0.0622	0.1248	0.0700	0.0097

C. Experimental Setup

1). PSF Measurement:

To estimate the true PSF, we placed 0.5 mm diameter point sources filled with ¹⁸F-FDG at different radial and axial positions inside the scanner bore, 56.2 cm in diameter and 15.5 cm in length. PSFs were sampled at eight radial locations and two axial locations (total 16 PSFs). The reconstructed PSF images were fitted with Gaussian kernels. The FWHM of the fitted PSFs had ranges of 6.03-7.21 mm and 6.63-7.89 mm in the radial and axial directions respectively. Interpolation weights for the experimental datasets were determined by means of bilinear interpolation over an irregular grid consisting of the quadrilaterals formed by the nearest radial and axial PSF sampling locations from a given point.

2). Phantom Scan:

The Hoffman brain phantom filled with ¹⁸F-FDG was scanned on the HR+ scanner for 15 min when the phantom activity was 27 MBq. A T2-weighted MR image of the phantom was separately acquired and rigidly coregistered with the PET image using FSL (https://fsl.fmrib.ox.ac.uk/) [45], [46].

3). Amyloid Scan:

The amyloid imaging data used in this paper were derived from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) http://adni.loni.usc.edu, a public repository containing images and clinical data from 2000+ human datasets. An 81 year old male subject underwent a 10 mCi bolus injection of ¹⁸F-florbetapir followed by a 50 min scan on the HR+ in accordance with the ADNI-GO protocol. T1-weighted magnetization-prepared rapid gradientecho (MPRAGE) MR images were obtained and co-registered with the PET images using the FreeSurfer neuroimaging data processing suite (https://surfer.nmr.mgh.harvard.edu) [47], FreeSurfer was also used to parcellate the MR volume for ROI analysis.

4). Tau Scan:

The tau imaging data used in this paper were acquired at Massachusetts General Hospital (MGH). All procedures were performed after written informed consent and with approval from the MGH Institutional Review Board obtained satisfying appropriate protocol requirements. PET data were acquired for 90 min after a 10 mCi bolus injection of ¹⁸F-flortaucipir. MR image acquisition was performed on a 3T Tim Trio (Siemens). As with the amyloid study, the MR image was co-registered with the PET image using FreeSurfer. FreeSurfer was also used to parcellate the MR volume for ROI analysis.

D. Reference Approaches and Evaluation Metrics

We compare the results of deconvolution using a spatially-variant PSF with no penalty/prior (SVPSF), a quadratic penalty (QP) based on nearest neighbor differences, an anisotropic total variation (TV) penalty, and a Bowsher penalty (BP). These penalty functions are defined as follows:

$${\Phi }_{\text{QP}}(\mathit{x})=\frac{1}{2}(\Vert {\Delta }_1\mathit{x}{\Vert }_2^2+\Vert {\Delta }_2\mathit{x}{\Vert }_2^2+\Vert {\Delta }_3\mathit{x}{\Vert }_2^2),$$ (17)

$${\Phi }_{\text{TV}}(\mathit{x})=(\Vert {\Delta }_1\mathit{x}{\Vert }_1+\Vert {\Delta }_2\mathit{x}{\Vert }_1+\Vert {\Delta }_3\mathit{x}{\Vert }_1),$$ (18)

$${\Phi }_{\text{BP}}(\mathit{x})=\frac{1}{2}(\Vert {\mathit{w}}_1\circ {\Delta }_1\mathit{x}{\Vert }_2^2+\Vert {\mathit{w}}_2\circ {\Delta }_2\mathit{x}{\Vert }_2^2+\Vert {\mathit{w}}_3\circ {\Delta }_3\mathit{x}{\Vert }_2^2),$$ (19)

where Δ_k, (k = 1, 2, or 3) are finite difference operators along the three Cartesian coordinate directions, ‘o’ denotes the Hadamard product, and the weights, w_k, for the BP term are computed from the anatomical image y using a tuning parameter σ_b as follows [48]–[50]:

$${\mathit{w}}_k=\sqrt{\text{exp}[-({\Delta }_k\mathit{y}\circ {\Delta }_k\mathit{y})∕{\sigma }_b]},k\in \{1,2,3\}.$$ (20)

The evaluation metrics used here are defined below. We use the notation μ_x and σ_x for the mean and standard deviation respectively of an image x and ${\mu }_x^{\mathcal{R}}$ and ${\sigma }_x^{\mathcal{R}}$ for the same statistics computed over an ROI $\mathcal{R}$. The true and estimated images are denoted x and $\widehat{x}$ respectively.

1). Contrast-to-Noise Ratio (CNR):

The CNR for a given ROI $\mathcal{R}$ and a reference ROI ${\mathcal{R}}_{\text{ref}}$ is defined as:

$$\text{CNR}=\frac{\mid {\mu }_{\widehat{x}}^{\mathcal{R}}-{\mu }_{\widehat{x}}^{{\mathcal{R}}_{\text{ref}}}\mid }{\sqrt{\left[({\sigma }_{\widehat{x}}^{\mathcal{R}}{)}^2+({\sigma }_{\widehat{x}}^{{\mathcal{R}}_{\text{ref}}}{)}^2\right]}}.$$ (21)

2). Structural Similarity Index (SSIM):

The SSIM is a well-accepted measure of perceived image quality and is defined as:

$$\text{SSIM}(\widehat{\mathit{x}},\mathit{x})=\frac{(2{\mu }_x{\mu }_{\widehat{x}}+c_1)(2{\sigma }_x{\sigma }_{\widehat{x}}+c_2)}{({\mu }_x^2+{\mu }_{\widehat{x}}^2+c_1)({\sigma }_x^2+{\sigma }_{\widehat{x}}^2+c_2)}.$$ (22)

Here c₁ and c₂ are parameters stabilizing the division.

3). Root-Mean-Square Error (RMSE):

The RMSE is computed as:

$$\text{RMSE}(\widehat{\mathit{x}},\mathit{x})=\sqrt{\frac{1}{N}\sum _k({\widehat{x}}_k-x_k{)}^2}.$$ (23)

4). Peak Signal-to-Noise Ratio (PSNR):

The PSNR is the ratio of the maximum signal power to noise power and is defined as:

$$\text{PSNR}(\widehat{\mathit{x}},\mathit{x})=20{\log }_{10}\left(\frac{\max (\widehat{\mathit{x}})}{\text{RMSE}(\widehat{\mathit{x}},x)}\right).$$ (24)

5). Coefficient of Variation (CV):

The CV for a given ROI $\mathcal{R}$ is defined as:

$$\text{CV}=\frac{{\sigma }_{\widehat{x}}^{\mathcal{R}}}{{\mu }_{\widehat{x}}^{\mathcal{R}}}.$$ (25)

For the simulation study, where the ground truth is known, we compute all the aforementioned metrics. For the phantom experiment where the ground truth is unknown, we only compute the CNR and MR-based SSIM. For the clinical datasets, we compute means of specific ROIs relevant to each pathology. The ROI mean SUVRs are useful as image-based biomarkers in diagnostics and therapeutics, including clinical trials for drug discovery. For localized ROIs, the CV measures of biomarkers are critical, especially while using image processing pipelines that tend to amplify noise. Accordingly, CV values for important ROIs are used to compare different methods for the clinical data.

IV. Results

A. Simulation Study

1). Parameter Selection and Initialization:

To select regularization parameters (weights of the penalty functions in the combined cost) for the BrainWeb digital phantom study, the QP, TV, BP, and JE regularization parameters were swept over a range of values. As a rule of thumb, a typical test value of the regularization parameter was set based on the ratio of the norms of the data-fitting and penalty gradients to: ${\beta }_{\text{reg}}^0=\Vert \nabla {\Phi }_{\text{PSF}}(\tilde{\mathit{b}})\Vert ∕(2\Vert \nabla {\Phi }_{\text{reg}}(\tilde{\mathit{b}})\Vert )$, where $\tilde{\mathit{b}}$ is an upsampled version of the blurry image b generated via trilinear interpolation, p is swept over a range of values {…, 4, 2, 1, 1/2, 1/4, … } with ${\beta }_{\text{reg}}=p{\beta }_{\text{reg}}^0$. Fig. 2 shows plots of gray-to-white CNR, SSIM, RMSE, and PSNR for the QP, TV, BP, and JE cases for different regularization parameters. The regularization parameters were selected to ensure the highest CNR as indicated by the inverted red triangles in Fig. 2. We observed that JE typically exhibits a peak in CNR and a corresponding dip in RMSE when the regularization parameter is in the optimal range. Very large values of the regularization parameter can substantially increase the RMSE and lower the CNR.

Fig. 2.

Regularization parameter selection. Plots of CNR, SSIM, RMSE, and PSNR for the QP, TV, BP, and JE techniques for different values of the regularization parameters. The regularization parameters that led to the highest CNR for each case are indicated by the inverted red triangles.

SVPSF (no penalty) suffers from the well-known instabilities of the standalone deconvolution problem. Continued iteration leads to noise amplification. The iteration number was set to a value of 20 in way of ensuring early termination. All methods were initialized by setting x₀ to $\tilde{\mathit{b}}$. QP, TV, BP, and JE were iterated till convergence. For the JE penalty, the Parzen window widths were set to σ_u = max(b)/q and σ_v = max(y)/q with q = 50, and M was set to 385.

2). Simulation Results:

Fig. 3 shows a comparison of images obtained from the different methods. For SVPSF, the displayed image is the output of the 20th iteration. For QP, TV, BP, and JE, the displayed images correspond to the parameter values in Fig. 2 that led to maximum gray-to-white CNR for each case. Qualitatively, the JE penalty generated well-defined tissue boundaries well-conforming with the MR image. As expected, SVPSF led to noise amplification. With QP, the data-fitting cost function and smoothing penalty function are somewhat conflicted in terms of their ultimate goals but together lead to a stable solution unlike SVPSF. In comparison, TV and BP led to sharper images with better-defined edges. Table II shows the evaluation metrics for all displayed images. Among the methods compared here, JE produced the best values for all the metrics.

Fig. 3.

Simulation results. Transverse slices from the T1-MR image, true PET image, Gaussian-filtered OSEM image (GF), and deblurred images with the spatially-variant PSF but no penalty (SVPSF), quadratic penalty (QP), anisotropic total variation penalty (TV), Bowsher penalty (BP), and MR-based joint entropy penalty (JE).

TABLE II.

BrainWeb simulation study: Performance comparison

Metric	SVPSF	QP	TV	BP	JE
CNR	0.17	0.31	0.30	0.40	0.63
SSIM	0.90	0.91	0.91	0.92	0.94
RMSE	0.51	0.35	0.34	0.30	0.29
PSNR	17.22	18.92	18.99	20.23	20.59

3). Impact of the PSF Model:

To determine the impact of the spatially-variant blurring model, we performed a comparison of this model with a spatially uniform PSF (SUPSF) model. Fig. 4 shows the evaluation metrics CNR, SSIM, RMSE, and PSNR for JE+SUPSF and JE+SVPSF. For this study, the PSF corresponding to the center of the scanner was used to create the SUPSF model. Compared to JE+SUPSF, JE+SVPSF showed a modest improvement in the best value obtained for each metric. More importantly, however, these plots show that the variation of the metrics with the regularization parameter is noticeably less drastic for SVPSF than SUPSF, indicating improved robustness of this model.

Fig. 4.

Comparison of SUPSF and SVPSF. The plots show the evaluation metrics CNR, SSIM, RMSE, and PSNR for JE+SUPSF and JE+SVPSF. SVPSF shows improvement in each metric relative to SUPSF. Also, compared to SUPSF, SVPSF demonstrated slower variation of the metrics with varying regularization parameter indicating improved robustness.

B. Experimental Studies

1). Phantom Results:

The SVPSF, QP, TV, BP, and JE deblurring techniques were applied to the Hoffman phantom dataset. Initialization of each method and JE Parzen window width selection were performed as described for the simulation study. Fig. 5 shows a comparison of images obtained from the different methods. For SVPSF, the displayed image is the output of the 20th iteration. For QP, TV, BP, and JE, the regularization parameter was set with p = 1/2, 2, 1 and 1 respectively to maximize the gray-to-white CNR for each case. JE led to well-defined boundaries matching the MR for the Hoffman phantom. TV produced a sharp image with a piecewise-constant appearance, while BP successfully incorporated some boundary information from the MR. SSIM (using the MR as a reference) and CNR are reported in Table III. JE led to higher values for both metrics relative to all other reference approaches.

Fig. 5.

Phantom experiment results. Transverse slices from the T2-MR image, Gaussian-filtered OSEM image (GF), and deblurred images with the spatially-variant PSF but no penalty (SVPSF), quadratic penalty (QP), anisotropic total variation penalty (TV), Bowsher penalty (BP), and MR-based joint entropy penalty (JE).

TABLE III.

Hoffman phantom study: Performance comparison

Metric	SVPSF	QP	TV	BP	JE
CNR	1.2814	1.1904	1.2245	1.3452	1.7968
SSIM (MR)	0.85	0.83	0.82	0.85	0.89

2). Clinical Results:

The SVPSF, QP, TV, BP, and JE deblurring techniques were applied to the amyloid and tau imaging datasets. Initialization of each method and JE Parzen window width selection were performed as described for the simulation study. Figs. 6 and 7 show comparisons of images obtained from the different methods for the amyloid and tau PET scans respectively. For SVPSF, the displayed image is the output of the 20th iteration. The regularization parameter was set heuristically so as to yield the lowest CVs in several key ROIs. Qualitatively, TV, BP, and JE produced sharper images than SVPSF and QP. As with the previous studies, JE led to well-defined boundaries matching the MRs for the clinical datasets.

Fig. 6.

Clinical results. Amyloid imaging study: Transverse slices from the T1-MR image, Gaussian-filtered OSEM image (GF), and deblurred images with the spatially-variant PSF but no penalty (SVPSF), quadratic penalty (QP), anisotropic total variation penalty (TV), Bowsher penalty (BP), and MR-based joint entropy penalty (JE).

Fig. 7.

Clinical results. Tau imaging study: Transverse slices from the T1-MR image, Gaussian-filtered OSEM image (GF), and deblurred images with the spatially-variant PSF but no penalty (SVPSF), quadratic penalty (QP), anisotropic total variation penalty (TV), Bowsher penalty (BP), and MR-based joint entropy penalty (JE).

For the amyloid study, CV values were computed for several clinically relevant ROIs for image-derived amyloid measures. Of these, the frontal-lateral-retrosplenial (FLR) area is generally used for image-based amyloid measures [51], [52], Several major sub-ROIs within the FLR were also examined, including frontal, temporal, and parietal cortices. Lastly, the striatal region was also evaluated separately as recent studies have highlighted distinct roles for cortical (FLR) amyloid and striatal amyloid measures [53], [54], The CVs for these regions are reported in Table IV. JE led to consistently lower CV values in all the ROIs relative to SVPSF, QP, TV, and BP.

TABLE IV.

Amyloid imaging study: Performance comparison

Metric	SVPSF	QP	TV	BP	JE
CV: FLR	0.4259	0.3103	0.2783	0.3270	0.2506
CV: Frontal	0.4876	0.3517	0.3247	0.3772	0.2704
CV: Temporal	0.4275	0.3217	0.2889	0.3348	0.2565
CV: Parietal	0.4035	0.2960	0.2859	0.3032	0.1975
CV: Striatal	0.3962	0.3069	0.2549	0.3217	0.2286

For the tau study, CV values were computed for a different set of clinically relevant ROIs considered critical for image-based tau measures. These include the inferior temporal cortex (ITC), middle temporal cortex (MTC), superior temporal cortex (STC), fusiform gyrus (FG), parahippocampal gyrus (PHG), and hippocampus (HC) [55], [56]. The CVs for these regions are reported in Table V. JE led to consistently lower CV values in all but one of the ROIs relative to SVPSF, QP, TV, and BP. The only exception was the hippocampus where QP had the best performance.

TABLE V.

Tau imaging study: Performance comparison

Metric	SVPSF	QP	TV	BP	JE
CV: ITC	0.1549	0.1404	0.1353	0.1377	0.1251
CV: MTC	0.1734	0.1484	0.1423	0.1459	0.1227
CV: STC	0.1787	0.1460	0.1421	0.1440	0.1299
CV: FG	0.1324	0.1042	0.0982	0.1018	0.0915
CV: PHG	0.1758	0.1458	0.1382	0.1426	0.1315
CV: HC	0.3311	0.1919	0.2048	0.1937	0.2548

V. Discussion

Overall, our simulation results indicate that JE leads to noticeable improvements in CNR, SSIM, RMSE, and PSNR compared to SVPSF (no penalty), QP, TV, and BP techniques. JE’s performance, however, may be quite sensitive to the choice of the regularization parameter. Very large values of the regularization parameter can substantially increase the RMSE and lower the CNR and sometimes introduce visual artifacts. One concern while incorporating cross-modality information with JE is the potential for crosstalk introduced by structural discrepancies between the PET and its anatomical counterpart. For example, MR features could sometimes be incorrectly introduced into PET images, or PET features could incorrectly be suppressed due to not being represented in the MR images. A related concern about JE is the potential imperfect mapping of edges in certain scenarios. For example, edges in the MR image may be over- or underrepresented in the JE-based PET image. The JE penalty, as defined here, is based on the similarity of image intensity distributions and does not contain spatial neighborhood information for the PET and MR image voxels. Crosstalk and imperfect edge mapping are known issues with information theoretic penalty functions and have been addressed in existing literature by embedding multiscale spatial features to make the penalty more robust [39], [57]. Our future work will focus on further investigations to address/alleviate this concern.

It should be noted that any anatomically-guided correction requires accurate multimodality co-registration. For intrasubject cases, standard rigid registration tools with 6 degrees-of-freedom have been demonstrated to work well. All studies in this paper, therefore, used well-characterized rigid registration tools that are part of routine neuroimaging data processing pipelines [45]–[47]. It should also be noted that PET image resolution is not only spatially-variant but also object-dependent. The latter cannot be fully accounted for via an image-domain correction. However, to reduce overall variability, we used the same image reconstruction protocol for the simulation, phantom, and clinical studies. Finally, we would like to emphasize that the deblurring framework does not model the PET noise covariances in the data-fidelity term, because the computation of the (iteration-dependent) covariance matrices of OSEM-reconstructed PET images is a challenging task [5]. The performance of the method for low-count images may need to be separately evaluated. For dynamic PET datasets (which typically have low-count earlier image frames), a more attractive strategy than postreconstruction deblurring may be to implement a JE prior for direct reconstruction of kinetic parameters.

For deblurring of post-filtered images, the Gaussian filtering step is typically the leading contributor to the blur. In cases where the raw (unfiltered) OSEM-reconstructed image is available, an MR-guided edge-preserving filter is an attractive option over deblurring after Gaussian filtering. However, in such cases, parameter tuning for the filter will be critical because different protocols use different OSEM iteration numbers, leading to different final noise levels. It should be added that, compared to post-reconstruction processing, reconstruction-based priors tend to be more effective from a quantitative perspective. However, this requires access to raw data, correction factors, and specialized reconstruction tools and models. A key advantage of the post-processing method presented here is its direct applicability to existing datasets for which the raw data (sinograms) and unfiltered images are either unavailable or not readily available.

For the clinical amyloid and tau imaging studies, CV measures were computed for several clinically meaningful ROIs. Our results show consistent lowering of CV in all amyloid ROIs. For tau, the CV was lower in all ROIs except the hippocampus. However, despite the significant role of the hippocampus in memory consolidation, it should be noted that ¹⁸F-flortaucipir suffers from well-known off-target effects in the vicinity of this ROI making it a poor candidate for ROI-based tau quantitation with this tracer [58].

VI. Conclusion

Despite the versatility of quantitative PET imaging arising from the availability of a wide gamut of radiotracers targeting different physiologically meaningful biomolecules, its quantitative accuracy is often limited by its poor spatial resolution. We have designed and implemented an anatomically guided framework that recovers PET resolution by means of deconvolution based on spatially-variant blur kernels measured in the image domain. The ill-posed deconvolution problem is stabilized using a JE prior based on the high-resolution MR counterpart of the PET. We tested the performance of this method using a BrainWeb phantom simulation, a Hoffman phantom experiment, and clinical amyloid and tau imaging scans. The method was compared with standalone deconvolution and penalized deconvolution using QP, TV, and BP regularization and demonstrated to be promising in terms of a number of evaluation metrics, including CNR, SSIM, RMSE, PSNR, and CV. As future work, we plan to develop a modified JE penalty that incorporates multiscale spatial information and is more robust against parameter variations and potential crosstalk introduced by structural discrepancies between the two modalities. ROI-based tau quantitation is promising for new biomarker development for Alzheimer’s disease, which, in turn, is vital for enabling drug development efforts. We will, therefore, apply this technique to specific neuroimaging scenarios pertaining to Alzheimer’s disease and validate the method by deriving CV measures for a much larger cohort.