Quantitative Accuracy of Penalized-Likelihood Reconstruction for ROI Activity Estimation

Abstract

Estimation of the tracer uptake in a region of interest (ROI) is an important task in emission tomography. ROI quantification is essential for measuring clinical factors such as tumor activity, growth rate, and the efficacy of therapeutic interventions. Accuracy of ROI quantification is significantly affected by image reconstruction algorithms. In penalized maximum-likelihood (PML) algorithm, the regularization parameter controls the resolution and noise tradeoff and, hence, affects ROI quantification. To obtain the optimum performance of ROI quantification, it is desirable to use a moderate regularization parameter to effectively suppress noise without introducing excessive bias. However, due to the non-linear and spatial-variant nature of PML reconstruction, choosing a proper regularization parameter is not an easy task. Our previous theoretical study [1] has shown that the bias-variance characteristic for ROI quantification task depends on the size and activity distribution of the ROI. In this work, we design physical phantom experiments to validate these predictions in a realistic situation. We found that the phantom data results match well the theoretical predictions. The good agreement between the phantom results and theoretical predictions shows that the theoretical expressions can be used to predict the accuracy of ROI activity quantification.

I. Introduction

One important application of emission tomography is to quantify the absolute tracer uptake in regions of interest (ROIs). ROI quantification is essential for evaluating tumor activity, growth rate, and the efficacy of therapeutic interventions.

Quantitative accuracy in the emission image is significantly affected by the bias-variance characteristics of the tomographic reconstruction algorithm. To deal with the low signal-to-noise ratio of emission data, all practical emission reconstruction algorithms employ some type of regularization to reduce image noise, at the cost of increasing bias. To obtain the optimum quantitative result, it is desirable to use a moderate level of regularization that strikes a balance between bias and variance and, hence, minimizes the overall quantification error.

The bias-variance characteristics of linear reconstruction algorithms have been studied previously [2]. More recently, similar research has been extended to non-linear statistical reconstruction methods based on the penalized maximum likelihood (PML) or maximum a posteriori (MAP) principle. Similar to the filter cut-off frequency in the filtered backprojection method, the regularization parameter in PML reconstruction controls the resolution and noise tradeoff. However, the non-linear and spatial-variant nature of PML reconstruction makes it more difficult to choose a proper regularization parameter than linear methods.

Previously we have derived approximate theoretical expressions for the bias, variance, and mean square error (MSE) of PML method for the ROI quantification task [1]. The theoretical expressions have been validated using computer simulations and also revealed that the optimum regularization parameter of the PML algorithm depends on the size and activity distribution of the ROI. In this work, we design a physical phantom experiment to further examine and validate these theoretical results. The use of a physical phantom allows us to include various modeling errors in a real PET scanner as well as to obtain ground truth for quantitative evaluations.

This paper is organized as follows. In Section II we briefly review the PML algorithm, ROI quantification, and major theoretical results from the previous work. In Section III we describe our phantom experiment and data processing method for the evaluation of ROI quantification. The results are then presented in Section IV. Finally we discuss some practical issues in Section V and drawn conclusions in Section VI.

II. Theory

A. Penalized Maximum Likelihood (PML) Image Reconstruction

Emission data $\mathrm{y}∊{\mathbb{R}}^{M\times 1}$ are well-modeled as a collection of independent Poisson random variables with the expectation

$$\stackrel{‒}{\mathrm{y}}=\mathrm{Px}+\mathrm{r},$$ (1)

where $\mathrm{P}∊{\mathbb{R}}^{M\times N}$ is the system matrix; $\mathrm{x}∊{\mathbb{R}}^{N\times 1}$ is the unknown tracer distribution; $\mathrm{r}∊{\mathbb{R}}^{M\times 1}$ accounts for the presence of scattered and random events in the data. The log likelihood of the independent Poisson distribution is

$$L(\mathrm{y}\mid \mathrm{x})=\sum _i\left(y_i\log {\stackrel{‒}{y}}_i-{\stackrel{‒}{y}}_i-\log y_i!\right).$$ (2)

The maximum likelihood (ML) estimate can be found by maximizing the log likelihood function. A popular ML algorithm for PET reconstruction is the expectation-maximization (EM) algorithm. However, ML solutions are unstable (i.e., noisy) because the tomography problem is ill-conditioned. Thus, some form of regularization (or prior function) is needed to reconstruct a reasonable image.

PML methods regularize the image through the use of a roughness penalty function on the unknown image. Various penalty functions have been proposed for image reconstruction, such as the quadratic function [3], Huber function [4], and Geman-McClure function [5]. While nonquadratic functions can preserve sharp edges in reconstructed images, existing studies have not found significant benefits of using nonquadratic penalty functions for ROI quantification and lesion detections [6] [7]. Therefore, here we focus on the quadratic penalty function, which can be written as

$$U\left(\mathrm{x}\right)=\frac{1}{2}{\mathrm{x}}^T\mathrm{Rx},$$

where R is a positive semidefinite matrix. To avoid dc bias in reconstructed images, we require 1^T R1 = 0.

Combining the log-likelihood function and the penalty function, the PML reconstruction is found as

$$\widehat{\mathrm{x}}=\underset{\mathrm{x}\ge 0}{\arg \max }\left[L(\mathrm{y}\mid \mathrm{x})-\beta U\left(\mathrm{x}\right)\right]\cdot$$

The regularization parameter β controls the strength of the regularization. If β is too small, the reconstructed image approaches the ML estimate and becomes very noisy; if β is too large, the reconstructed image will be very smooth and useful information can be lost. To obtain the optimal detection or quantification performance, it is important to choose a proper β that strikes a balance between the bias and variance.

B. Region of Interest (ROI) Quantification

The total tracer uptake inside an ROI is computed as

$$\widehat{\eta }={\mathrm{f}}^T\widehat{\mathrm{x}},$$

where f is the indicator vector of the ROI, i.e. f_i is equal to one if voxel i is inside the ROI, and zero otherwise; $\widehat{\mathrm{x}}$ is the reconstructed image. f is usually obtained by either drawing the ROI on the reconstructed image or drawing the ROI on a co-registered anatomical image. The bias, variance, and mean squared error (MSE) of $\widehat{\eta }$ are defined as

$$\begin{array}{cc}\hfill \mathrm{bias}\left[\widehat{\eta }\right]& =E[\widehat{\eta }-{\eta }_{\mathrm{true}}],\hfill \\ \hfill \mathrm{var}\left[\widehat{\eta }\right]& =E\left[{\widehat{\eta }}^2\right]-E^2\left[\widehat{\eta }\right],\hfill \\ \hfill \mathrm{MSE}\left[\widehat{\eta }\right]& =E\left[\right(\widehat{\eta }-{\eta }_{\mathrm{true}}{)}^2]={\mathrm{bias}}^2[\widehat{\eta }]+\mathrm{var}[\widehat{\eta }],\hfill \end{array}$$

where η_true is the ground truth of the tracer uptake inside the ROI. In this work we use the MSE as the figure of merit to evaluate the performance of the ROI quantification task. The results are also applicable to other figures of merit that are functions of the bias and variance.

C. Theoretical Expressions for Errors in ROI Quantification

Built upon the bias and variance expressions of PML reconstruction at the fixed point of the objective function [8, 9], we have derived approximate expressions of the bias, variance, and MSE for ROI quantification [1]:

$$\begin{array}{cc}\hfill \mathrm{bias}\left[\eta \right]& \approx {\mathrm{f}}^T[\mathrm{F}+\beta \mathrm{R}{]}^{-1}{\mathrm{Fx}}_{ROI}-{\mathrm{f}}^T{\mathrm{x}}_{ROI},\hfill \\ \hfill \mathrm{var}\left[\eta \right]& \approx {\mathrm{f}}^T[\mathrm{F}+\beta \mathrm{R}{]}^{-1}\mathrm{F}[\mathrm{F}+\beta \mathrm{R}{]}^{-1}\mathrm{f},\hfill \\ \hfill \mathrm{MSE}\left[\eta \right]& ={\mathrm{bias}}^2+\mathrm{var}\approx {\mid {\mathrm{f}}^T[\mathrm{F}+\beta \mathrm{R}{]}^{-1}{\mathrm{Fx}}_{ROI}-{\mathrm{f}}^T{\mathrm{x}}_{ROI}\mid }^2+{\mathrm{f}}^T[\mathrm{F}+\beta \mathrm{R}{]}^{-1}\mathrm{F}[\mathrm{F}+\beta \mathrm{R}{]}^{-1}\mathrm{f},\hfill \end{array}$$ (3)

where x_ROI is the true activity distribution inside the ROI above the surrounding background; $\mathrm{F}={\mathrm{P}}^T\mathrm{diag}\left\{\frac{1}{\stackrel{‒}{y}}\right\}\mathrm{P}$ is the Fisher information matrix. These approximate expressions are valid for ROIs that are localized, low activity, and surrounded by a uniform background. The expression for MSE is very similar to the result that was derived for linear reconstruction methods [10].

Using a local stationary approximation and fast Fourier transform, the above theoretical formulae can be computed quickly and allow fast evaluation of image accuracy. The above expressions have been validated using computer simulations and have shown that the optimum smoothing parameter depends on the size and activity distribution of the ROI. In this work, we validate these results using physical phantom data.

III. Method

The experiment was designed to study the effect of three parameters, namely ROI size, ROI contrast, and overall noise level, on the PML algorithm for the task of ROI quantification.

A. Phantom and PET Acquisitions

The NEMA/IEC body phantom [11] and a set of four fillable spheres were used to simulate hot lesions in a uniform body background (Fig. 1). The outer dimension of the body phantom is 24.1×30.5×24.1 cm (height by width by length). The inner diameters of the four spheres are 13 mm, 17 mm, 22 mm, and 28 mm, respectively. We used a GE Discovery ST PET/CT scanner in 2D mode. The scanner has 420 detectors forming a ring with a diameter of 881 mm. The sinogram data have 210 angles of view, 220 lines of response in each view, and 47 transaxial planes.

Fig. 1.

The NEMA/IEC phantom used in the experiment.

The phantoms were filled with a solution of ¹⁸F-FDG. We performed two separate scans, one for the spheres and one for the body background. Performing two separate scans allows the spheres data and the background data to be superimposed with adjustable contrast. It also avoids the cold wall surrounding the sphere caused by its plastic shell when placed in a warm background. While the count-rate dependent factors (e.g. deadtime) can be different between the two scans, this was not an issue because the scanner was operated in the 2D mode and the total amount of the activity was low.

The four spheres were scanned in air for one hour with an initial total activity of 9.1 μCi (0.34 MBq). Then the body phantom was scanned for 8 hours with an initial activity of 2.59 mCi (95.8 MBq). The total number of detected coincident events, including random and scattered events, was 7.4 million for the spheres and 1.1 billion for the body background, respectively. Random events in the delayed coincidence window were saved separately from the prompt events.

CT scans were performed on the same scanner for both the sphere and the body phantom for attenuation correction. Fig. 2 shows the CT images depicting the relative position of the spheres and the body phantom.

Fig. 2.

The CT images. (a) Three orthogonal tomographic views of the body phantom. (b) Three orthogonal maximum intensity projections of the spheres. The scanner bed and other supporting materials have been removed from the coronal view in (b) for better visualization of the size and position of the spheres.

B. Sinogram Processing

The scanner manufacturer's software was used to obtain the attenuation correction factors, detector normalization factors, and scatter sinogram, which are incorporated into the forward model in (1). Since the spheres were scanned in the air, the sphere data were first corrected for attenuation using its own attenuation correction factors and then attenuated using attenuation factors from the body phantom. To reduce noise in the measured sinograms, multiple transaxial planes were summed together. For the body phantom data, all the 47 transaxial planes were summed into a single 2D sinogram. For the sphere data, the planes containing the two smaller spheres (13 mm and 17 mm) and the planes containing the two larger spheres (22 mm and 28 mm) were summed separately to avoid overlap between the spheres. Fig. 3(a) shows the summed sinogram of the body phantom and Fig. 3(b) shows the summed sinogram of the two larger spheres.

Fig. 3.

The 2D Sinograms. (a) The summed body phantom sinogram. (b) The summed sinogram of the two larger spheres. (c) The superimposed sinogram with the contrast=1.0. (d) A noisy realization of the sinogram in (c) with 200k events.

The sphere sinograms were scaled to a desired ROI-to-background contrast before they were superimposed onto the body phantom sinogram. The scaling factor was calculated as follows: We first reconstructed the sphere-only image and the background-only image from the sinograms without any scaling. Let I₁ denote the maximum pixel intensity in a hot region in the sphere-only image, and I₂ denote the pixel intensity at the same location in the background image. The scaling factor was then given by C · I₁/I₂, where C is the desired contrast. Three ROI-to-background contrasts, 0.5, 1.0 and 2.0, were simulated, which correspond to the ROI-to-background activity ratio of 1.5, 2.0, and 3.0, respectively. Fig. 3(c) shows the superimposed sinogram with the ROI-to-background contrast of 1.0.

The superimposed sinograms have high count-density and were treated as noise-free. To simulate the noise level in clinical emission data, computer generated Poisson noise was added to these high count data. The expected total numbers of events for the noisy realizations were 100k, 200k, and 400k. The count level of 200k events is roughly equivalent to a 3-min 2D single slice scan of a patient with a 10 mCi (370MBq) injection of activity. One hundred twenty-eight noisy realizations were generated for each configuration.

C. Reconstruction

Each noisy realization was reconstructed using a penalized-likelihood algorithm with quadratic pair-wise penalty function that penalizes differences between the 8 nearest neighboring pixels. A wide range of the regularization parameter, from 0 (no regularization) to 10⁻⁶ (over-smoothed image), was used. Reconstructed images were represented by 111×111 3-mm square pixels. The system matrix consists of a product of three matrices, i.e., a geometric projection matrix, attenuation matrix, and detector normalization matrix. The geometric projection matrix was analytically calculated with modeling of the crystal penetration [12]. The attenuation and normalization matrices are diagonal matrices formed using the attenuation correction and normalization factors obtained from the manufacturer's software. Scattered and random coincidences were included in the forward model (1) as the affine term. A preconditioned conjugate gradient algorithm was used to find the PML solution [13]. The algorithm was allowed to run for 30 iterations for all regularization parameters. The number of iterations was chosen to achieve a balance between the convergence and computation time, and is the typical setting for the algorithm when applied to real data. Each reconstruction takes about 10 seconds on a single AMD Opteron 2GHz processor.

D. ROI quantification

The indicator images of the four ROIs were obtained by thresholding the ML reconstruction (β = 0) of the high-count sphere-only PET data. The segmentation threshold was chosen so that the diameter of the ROIs are slightly (one or two pixels) larger than the physical inner diameter of the spheres. Doing so ensures that activity from the spheres will not be excluded from the ROI due to the finite size of the image pixels.

ROI uptake was calculated for each noisy realization. The ensemble bias, variance, and mean square error (MSE) of ROI quantification were calculated as

$$\begin{array}{cc}\hfill \mathrm{bias}\left[\widehat{\eta }\right]& =\frac{1}{N}\sum _{k=1}^N\left({\widehat{\eta }}_k-{\eta }_{\mathrm{true}}\right),\hfill \\ \hfill \mathrm{var}\left[\widehat{\eta }\right]& =\frac{1}{N}\left(\sum _{k=1}^N{\widehat{\eta }}_k^2\right)-\frac{1}{N}{\left(\sum _{k=1}^N{\widehat{\eta }}_k\right)}^2,\hfill \\ \hfill \mathrm{MSE}\left[\widehat{\eta }\right]& =\frac{1}{N}\sum _{k=1}^N({\widehat{\eta }}_k-{\eta }_{\mathrm{true}}{)}^2={\mathrm{bias}}^2[\widehat{\eta }]+\mathrm{var}[\widehat{\eta }],\hfill \end{array}$$

where ${\widehat{\eta }}_k={\mathrm{f}}^T{\widehat{\mathrm{x}}}_k$ is the ROI activity estimated from the kth noisy realization, and N = 128 is the number of noisy realizations.

For comparison we also calculated the bias, variance, and MSE using the theoretical expressions in [1]. The activity profile of the ROI, x_ROI, required by the theoretical calculations, was approximated using the ML reconstruction of the high-count sphere-only data.

IV. Results

Fig. 4 shows the ensemble mean and standard deviation images calculated from the noisy reconstructions with different regularization parameters. The two hot regions on the background are the 13-mm and 17-mm spheres. As we increase the regularization parameter, the reconstructed image becomes less noisy, but more blurry.

Fig. 4.

Mean and standard deviation images of the PML reconstruction with different regularization parameters.

Fig. 5 shows the bias and standard deviation of the ROI quantification obtained from the experimental data and the theoretical expressions. Each panel depicts the bias-variance characteristic for one particular ROI. The contrast of all the ROIs was fixed at 0.5. The total number of events is 200k. Good agreements are found between the experimental results and theoretical predictions. It is noticed that the bias obtained from the experimental data does not reduce to zero even for the ML reconstruction (β = 0). In addition to the intrinsic variation of Monte Carlo samples as indicated by the error bars, there are two other factors that could contribute to the bias. The first one is that we only ran the reconstruction algorithm for 30 iterations, which may not be sufficient for convergence with smaller regularization parameters. The second one is the non-negative constraint used in the reconstruction. With smaller regularization parameters, the reconstructed image is very noisy and the non-negative constraint is more active in the warm background than in the hot sphere regions, which tends to slightly underestimate the ROI activity. This effect is less pronounced for data containing less noise.

Fig. 5.

Normalized absolute bias and standard deviation of ROI quantification obtained from the experimental data and theoretical predictions. Each panel represents one particular ROI. The bias and standard deviation are normalized by the true tracer uptake in the corresponding ROI.

Fig. 6 plots the MSE of ROI quantification versus regularization parameter for different ROI configurations. The error bars of the real data represents plus and minus one standard deviation. The theoretical curves were compensated for the nonnegativity constraint in the reconstructed images using the method in [14]. In all configurations, the minimum MSE is achieved with a moderate regularization parameter. In particular, for ROIs of smaller size and lower contrast, and for data with higher noise levels (fewer events), stronger regularization is preferred. Overall, the theoretical results provide good predictions of the minimum MSE and the optimum smoothing parameter for the ROI quantification task.

Fig. 6.

Normalized MSE of ROI quantification for different ROI configurations. (“+”): real phantom measurements; (solid line): theoretical predictions. MSEs are normalized by the true activity in each ROI. The error bars of the real data represents plus and minus one standard deviation. The theoretical curves were compensated for the nonnegativity constraint in the reconstructed images using the method in [14].

It is worth noting that calculating the MSE curve for each ROI configuration using the theoretical expression takes only a few seconds using MATLAB on an single AMD Opteron 2GHz processor. Thus, the theoretical approach offers a fast alternative to the conventional Monte Carlo method.

V. Discussion

Fig. 6 shows that the optimum smoothing parameter depends on the size and activity distribution of the ROI. As a result the ROI has to be defined before selecting the optimum smoothing parameter. This can be done by either drawing the ROI on an initial reconstruction or drawing the ROI on a co-registered anatomical image provided by PET/CT and SPECT/CT scanners. The activity inside the ROI can be modeled using a prior distribution obtained from existing population data. The effect of the uncertainty in the prior distribution can be incorporated in the theoretical prediction as in [1]. Since the MSE of ROI quantification is a relatively smooth function of the regularization parameter, some error in the prior knowledge will not affect the optimum smoothing parameter and quantification results significantly.

The theoretical calculation also requires the Fisher information matrix $\mathrm{F}={\mathrm{P}}^T\mathrm{diag}\left\{\frac{1}{\stackrel{‒}{\mathrm{y}}}\right\}\mathrm{P}$, which in turn requires the noiseless projection data $\stackrel{‒}{\mathrm{y}}$. Since F is a low-pass filter in nature, one can simply plug-in a noisy data set in the place of the noiseless data $\stackrel{‒}{\mathrm{y}}$. More sophisticated methods have also been developed to obtain accurate estimation of the Fisher information matrix even with extremely noisy data [15] [16].

VI. Conclusion

We have performed physical phantom experiments to evaluate ROI quantification in PML reconstruction. The results show that the optimum regularization parameter that results in the minimum MSE is dependent on the size and activity of the ROI, and the noise level of the data. As expected, stronger regularization is preferred for ROIs of smaller size and lower contrast, as well as for noisier data. The results from the physical phantom data match the theoretical predictions reasonably well.