Estimation of the minimum detectable activity of preclinical PET imaging systems with an analytical method

Abstract

Purpose: The traditional figures of merit used in the evaluation of positron emission tomography (PET) systems, including system sensitivity and spatial resolution, do not directly reflect the minimum detectable activity (MDA) performance, despite the fact that it is one of the most important tasks for a PET system. MDA, as a combination of the more traditional PET system parameters, is directly related to lesion detection. However, MDA evaluation is task specific and cannot be done by a single measurement. Therefore, a simple method to evaluate system detectability needs to be developed.

Methods: In this work, an analytical method of MDA estimation was developed, taking into account system sensitivity, spatial resolution, source properties, and noise propagation in image reconstruction by using the Rose criterion and∕or the Curie equation as the detection standard. In the implementation, the source background, as well as the intrinsic activity background from the scintillation material of the system, was also taken into consideration. The accuracy of this method was evaluated in two commercially available preclinical PET systems, with phantom experiments that were designed to closely mimic in vivo tumor uptake without introducing finite boundaries between the source and the background.

Results: The lesion contrast-to-noise ratio calculated by the analytical evaluation showed good agreement with that obtained from the experiments. Visual assessment of the reconstructed images at the detection limit (based on analytical evaluation) also was in agreement with the Rose criterion. The MDA performance was quantitatively compared between the two preclinical PET systems and showed different detection limits under different imaging conditions, suggesting that the detection limit of a PET system strongly depends on the lesion properties and acquisition settings.

Conclusions: An analytical method of evaluating the PET system detectability was developed and validated by experiments. Overall, the analytical MDA calculation provides a simple way to evaluate the signal detectability of a PET system and can be used for comparing different systems. It also provides guidelines for designing new PET tomographs as well as optimizing data acquisition protocols.

INTRODUCTION

Preclinical positron emission tomography (PET) has become an important tool in biomedical research.^{1, 2, 3, 4} Novel molecular imaging applications, such as cell trafficking studies⁵ or gene expression imaging,^{6, 7} have brought the need to image small, weak sources on the order of nanocurie activity under low contrast conditions. These imaging tasks pose a significant challenge on system performance for reliable detection.

Preclinical PET systems are commercially available^{8, 9, 10, 11} and their performance has been evaluated either based on the NEMA-NU4 standard¹² or for the traditional figures of merit, including system sensitivity, spatial resolution, and noise equivalent count rate.^{13, 14, 15, 16, 17} These measures, however, do not directly reflect the detection limits of a PET system. The concept of minimum detectable activity (MDA) was introduced, as a combination of the more traditional PET system parameters, to evaluate the performance of a PET scanner at very low activity distributions.¹⁸ The MDA of a PET system is defined as the lowest activity level that can be detected in a reconstructed PET image for certain lesion properties under fixed acquisition conditions. MDA, as an additional parameter for PET performance, is directly related to detectability and thus is an important evaluation parameter of a tomographic system.

The MDA of a PET system is determined by several factors including object properties, injected radiopharmaceutical compound and isotope used for labeling, system performance, acquisition protocol, reconstruction algorithm, etc. All these factors interplay with each other and determine the system sensitivity, image resolution, background noise, lesion contrast, recovery coefficient, scatter acceptance, etc. These factors can be categorized into three groups as follows.

Imaged subject and radiopharmaceutical probe

Ideally, a lesion with large size, high uptake, and low nonspecific probe binding is easy to be detected. These properties increase the absolute signal and∕or the signal-to-background contrast. Since the subject, lesion properties, and probe selection are independent biological parameters, we will focus the investigation of MDA on the other two groups of factors that pertain to the hardware and software of an imaging system. However, for this work realistic tumor sizes, uptake activities, contrast levels, and scan durations were obtained from actual in vivo imaging studies with ¹⁸F based compounds that were relevant to the task.

PET imaging system

A system with high sensitivity, high spatial resolution, and low intrinsic background is always preferred for better performance in lesion detection. High sensitivity systems reduce the statistical noise in reconstructed images and thus increase the signal-to-noise ratio (SNR) for more reliable lesion detection. Higher spatial resolution reduces the partial volume effect (PVE) for a fixed object size and should therefore improve detectability. Scintillator crystals with intrinsic radioactivity generate nonzero system background, reducing lesion contrast and impairing lesion detectability.

Data acquisition protocol and image reconstruction algorithm

With a given subject and tomographic PET system, long acquisitions are always preferred to reduce statistical image noise for static studies, provided that the imaging agent does not become redistributed during the study. For a fixed acquisition time, the scan protocol (e.g., the energy window) and the reconstruction algorithm (e.g., analytical versus iterative reconstruction) need to be optimized for better MDA performance. The selection of the energy window, especially the choice of the lower level discriminator (LLD) determines the sensitivity of the system. The reconstruction algorithm determines the mapping of resolution and noise from the raw data to the reconstructed images.

To optimize the MDA performance, different improvements can be performed during the design of a system, the acquisition of data, and the reconstruction of images.

When designing a PET system, large solid angle coverage and deep scintillator crystals are preferred to increase system sensitivity. However, these approaches also result in increased photon penetration between crystals, lowering spatial resolution. While systems based on small cross section crystals provide higher intrinsic spatial resolution, this approach allows more intercrystal scatter, leads to mispositioning of coincidence events, and degrades the spatial resolution.¹⁹ Smaller crystals also increase manufacturing difficulties and reduce the detector packing fraction and overall system sensitivity. The scintillation crystal is preferred to have high stopping power, high atomic number Z, low intrinsic background, high light output, and fast decay constant. These features, though, cannot be achieved simultaneously. Therefore, the geometry of the system and the choice of detector material should be considered comprehensively and be application specific.

When setting up imaging protocols, while longer acquisition times are usually preferred, actual scan durations are limited by the throughput of the imaging center or by imaging probe pharmacokinetics. A lower LLD provides higher system sensitivity; however, it also increases the acceptance of object and crystal scatter as well as background events.

All these interrelated factors need to be systematically and collectively considered when evaluating the MDA. As a comprehensive parameter to evaluate the detection performance of a micro-PET system, MDA incorporates the traditional figures of merit, such as system sensitivity, spatial resolution, and contrast recovery, and is directly related to imaging tasks. Unlike traditional parameters used to evaluate the system performance, MDA evaluation is task specific and cannot be done by a single measurement. Therefore, it is important to develop a simple method to evaluate system detectability and to help optimize data acquisition protocols as well as image reconstruction algorithms.

MATERIALS AND METHODS

Evaluation criteria

Rose criterion

The detection limit of an imaging system depends on the lesion to background contrast and the noise (fractional standard deviation) in the background, so called contrast-to-noise ratio (CNR). The Rose criterion states that in order to be detectable, an object’s CNR must exceed 3–5 (Ref. ²⁰). The actual value depends on object shape, edge sharpness, viewing distance, observer experience, and so forth. For tomographic imaging systems, the lesion CNR can be expressed as²¹

$${\text{CNR}}_l\approx C_l\times \sqrt{n_l}\times {\text{SNR}}_{\text{pixel}},$$ (1)

where C_l is the lesion to background contrast, n_l is the number of pixels that the object occupies, and SNR_pixel is the signal-to-noise ratio for a single pixel in the background area with the assumption that the background pixels are not correlated. Since

$$C_l=\frac{S_l-S_B}{S_B}\text{and}{\text{SNR}}_{\text{pixel}}=\frac{S_B}{{\sigma }_B},$$ (2)

where S_l, S_B, and σ_B are the lesion signal, background signal, and background noise, respectively, Eq. 1 can be rewritten as

$${\text{CNR}}_l={\text{SNR}}_l\times \sqrt{n_l},$$ (3)

where SNR_l is the lesion signal-to-background noise ratio and equals to (S_l−S_B)∕σ_B.

Here, the S_l, S_B, and σ_B can be obtained directly from image analysis. It is also important to note that the noise in the CNR expression (1∕SNR_pixel) is the fractional standard deviation (unitless).

Curie equation

The Curie equation is also a method that evaluates lesion detectability.²² In order to meet the criterion that both the false-positive and false-negative probabilities for lesion detection are less than 5%, the minimal detectable level S_l must satisfy the following equation:

$${\text{S}}_{\text{net}}=S_l-S_B\approx 4.653{\sigma }_B,\text{or}{\text{SNR}}_l\approx \mathrm{4.653.}$$ (4)

Here, the noise in the SNR expression (σ_B) is the noise standard deviation and has the units of image pixel value.

The Curie equation 4 is most widely used for counting devices. For PET systems, it is an evaluation of the detectability for a single pixel lesion, or for a lesion with n_l=1. By substituting the Curie equation 4 into the Rose criterion [Eq. 3] and setting the lesion size n_l to 1 pixel, one can tell that the detection evaluation by the Curie equation is essentially equivalent to the Rose criterion, with the detection limit of CNR set to 4.653. The two detection evaluation criteria are indeed similar to each other for 1 pixel lesion: The Curie equation measures the SNR, while the Rose criterion measures the CNR. For images with lesions covering multiple pixels, a factor of $\sqrt{n_l}$ needs to be multiplied.

Analytical CNR calculation

The lowest lesion activity that can generate a CNR greater than 3–5 (4 is used in this work) is called the MDA of the PET tomograph. With the prior knowledge of the conventionally measured system performance and geometry as well as the source and background properties, a MDA evaluation method was developed based on the analytical estimation of the CNR.

When the source is smaller than or comparable to the system spatial resolution, the PVE needs to be considered. The actual value for the PVE can be either calculated by analytical equations for regular-shaped objects or be modeled by computer simulations for arbitrary-shaped subjects through convolution with a Gaussian filter of full width at half maximum (FWHM) at the system resolution. Assuming that the source and background concentrations are C_src and C_bkgd, the maximum value of the point source and the mean value of the background can be expressed as

$${\text{max}}_{\text{src}}=S_l=\left(C_{\text{src}}\times {\text{PVE}}_{\text{src}}+C_{\text{bkgd}}\right)\times k$$ (5)

and

$${\text{mean}}_{\text{bkgd}}=S_B=C_{\text{bkgd}}\times k,$$ (6)

where k is the imaging system calibration factor and PVE_src is the PVE of the point source in the tomographic system.

The noise (fractional standard deviation) in the background can be estimated from the noise propagation of the tomographic 2D filtered backprojection (2D FBP) image reconstruction with²³

$$\text{fractional}\text{stdev}=\frac{{\sigma }_B}{S_B}=\frac{1}{{\text{SNR}}_{\text{pixel}}}=\sqrt{\frac{{\pi }^2}{12}}\frac{{\left(D∕\Delta t\right)}^{3∕2}}{\sqrt{N_{\text{total}}}},$$ (7)

where D is the object diameter, Δt is the transverse sampling distance, and N_total can be estimated by equation

$$N_{\text{total}}=A\times T\times \eta \times \text{decay}\times \text{atten},$$ (8)

with A being the total activity of the background in Becquerels, T being the scan duration in seconds, and η being the system sensitivity. The decay in Eq. 8 is a correction factor for the radioisotope decay during the scan duration T. The atten in Eq. 8 is the averaged attenuation of the object and can be approximated by knowing the object dimensions and its attenuation coefficient for 511 keV gammas.²⁴ For water equivalent material, the theoretical attenuation coefficient is 0.095 cm⁻¹.

Thus, the CNR can be calculated by combining Eq. 1, 2, 5, 6, 7 as

$$\text{CNR}=\frac{C_{\text{src}}\times {\text{PVE}}_{\text{src}}}{C_{\text{bkgd}}}\times \sqrt{n_l}\times \sqrt{\frac{12}{{\pi }^2}}\frac{\sqrt{N_{\text{total}}}}{{\left(D∕\Delta t\right)}^{3∕2}}.$$ (9)

If the source size is smaller than or comparable to the system resolution, the source size in image space will not depend strongly on the actual source dimensions. In those cases, the $\sqrt{n_l}$ term in Eq. 9 can be set to 1.

Many preclinical PET scanners employ lutetium oxyorthosilicate (LSO) as scintillation crystal, which has high energy resolution and short decay constant. However, in the scintillation crystal Lu₂SiO₅, about 2.59% of the lutetium element is ¹⁷⁶Lu and has an intrinsic emission,^{25, 26, 27, 28} which results in the formation of “true” coincidences. These true coincidences generate a system background, possibly masking out low uptake regions. In order to evaluate the detection limit of a PET system, the LSO background activity needs to be taken into consideration depending on the energy window employed (see the Appendix0). In general, when the acquisition LLD of a LSO based system is lower than 350 keV, the LSO background contribution to the MDA performance cannot be ignored.

For general cases, the noise in the background comes from multiple sources (LSO, biological background, etc). If the system and biological background concentrations are comparable, one must consider both of them at the same time. The total background noise σ_T can be expressed by Eq. 10

$${\sigma }_T=\sqrt{\sigma _{\text{bkgd}}{}^2+\sigma _{\text{LSO}}{}^2},$$ (10)

where σ_bkgd and σ_LSO can be obtained from

$${\sigma }_{\text{bkgd}}=\sqrt{\frac{{\pi }^2}{12}}\frac{{\left(D∕\Delta t\right)}^{3∕2}}{\sqrt{N_{\text{bkgd}}}}\times \left(C_{\text{bkgd}}\times k\right)$$ (11)

and

$${\sigma }_{\text{LSO}}={\text{frac}̱\text{stdev}}_{\text{LSO}}\times \left(C_{\text{LSO}}\times k\right),$$ (12)

where _{frac̱stdevLSO} and C_LSO are the fractional standard deviation and concentration of the LSO background, respectively.

Therefore, the contrast-to-noise ratio when both LSO background and biological background are present can be expressed as

$$\text{CNR}=\frac{C_{\text{src}}\times \text{PVE}}{\sqrt{{\left(\sqrt{\frac{{\pi }^2}{12}}\frac{{\left(D∕\Delta t\right)}^{3∕2}}{\sqrt{N_{\text{bkgd}}}}\times C_{\text{bkgd}}\right)}^2+{\left(\mathrm{\%}{\text{stdev}}_{\text{LSO}}\times C_{\text{LSO}}\right)}^2}}\times \sqrt{n_l},$$ (13)

where the LSO concentration and LSO noise can be obtained from a real measurement or Monte Carlo simulations. If the LSO background can be corrected by a very long background scan which will not introduce additional noise, a less stringent condition needs to be met for the lesion to be detected.

The validation of Eq. 9 and the evaluation of LSO background effect on detection limit was carried out on two preclinical PET imaging systems available in our institute, the Inveon and the Focus220 (Siemens Preclinical Solutions, Inc. Knoxville, TN). The analytical evaluation was then compared to phantom experiments as well as visual assessment of the reconstructed images.

The Inveon and the Focus systems

The Inveon and the Focus220 systems represent subsequent generations of commercial tomographs and their performance has been thoroughly evaluated elsewhere.^{8, 11} The performance parameters related to the MDA evaluation, including sensitivity, resolution, and intrinsic background, are discussed below. Both the Inveon and the Focus220 employ LSO as scintillation crystal, with the Inveon having higher LSO concentration than the Focus220 (see the Appendix0).

Due to the large solid angle of the Inveon system, its absolute peak sensitivity is significantly higher than that of the Focus220 (6.7% versus 2.7% at a 350–650 keV energy window).

The transverse spatial resolutions of the two systems are comparable at the center of the field of view (FOV) since they both use crystals with the same cross section size. However, as the source moves to off-center positions, the resolution degrades faster on the Inveon system. That is due to the closer proximity of the detectors that causes increased crystal penetration, resulting in event mispositioning. For the same reason, the Inveon has lower axial spatial resolution than the Focus220 as well. For a point source located in the central transverse plane but at 10 mm radial offset, the axial resolution for the Inveon and the Focus220 are 2.4 and 2.0 mm, respectively. The lower Inveon resolution adversely affects the system detection performance in this comparison.

EXPERIMENTAL VALIDATION

Experimental design

Experimental evaluation of the MDA is not straightforward. This is because in conventional phantoms, it is difficult to avoid the finite wall thickness separating localized point or rod sources from the uniform background regions. Previous contrast phantoms were constructed either by chambers separated with a finite wall thickness or from two separate acquisitions, one representing the background and one representing a point source.^{29, 30} Some investigators achieved sources without borders, but they are difficult to control in the sizes required for preclinical imaging.³¹

In order to accurately evaluate the detection performance of a PET system for real studies, it is important to develop a method to create a well controlled source and background without introducing cold boundaries. In this work, a high resolution inkjet printer was modified to allow printing of various patterns using an ink and FDG solution mixture.³² The source and background could then be printed on glossy photo paper at different concentrations creating different contrast levels. With this inkjet printed source method, well controlled sources with arbitrary sizes can be created and no boundary is introduced between signal and background. This method provides a more realistic way to simulate real preclinical in vivo acquisitions.

Experimental setup

The Inveon system has higher system sensitivity, while it has worse spatial resolution at some locations in the FOV than the Focus220. At the same time, the two tomographs have different LSO background activity concentrations. Given this information, it is not immediately obvious to draw a conclusion that the Inveon tomograph performs better or worse than the Focus220 in terms of detectability, since the system detectability is directly related to all of the above performance parameters. Therefore, to achieve a comprehensive evaluation of the detectability performance for the Inveon and the Focus220, representative cases need to be carefully selected, including the effect of system sensitivity, LSO background, and spatial resolution under different circumstances.

Case 1: Detectability evaluation with biological background

Two ¹⁸F point sources of sizes 1.0×1.0 and 0.75×0.75 mm², respectively, and two ¹⁸F backgrounds of the same size (15.2×15.2 mm²) were printed separately using the inkjet printer method. The activity of the sources and backgrounds was calibrated in a γ-counter (Wallac 1480 Wizard 3 in., PerkinElmer, MA, Life Sciences). The size of the point sources and the backgrounds was precisely measured with a 4× loupe (Peak anastigmatic loupe, No. 1990-4) to ensure accurate calculation of the activity concentration per surface area. Each point source and uniform background was placed face-to-face, forming one 2D contrast set. The two pairs of printed contrasts (one with the 1.0×1.0 mm² point source and the other with the 0.75×0.75 mm² point source) were sandwiched by three pieces of 5 mm thick water equivalent material (Computerized Imaging Reference Systems Inc., Norfolk, VA) to ensure positron annihilation and mimic in vivo situations. The sources were positioned parallel to the transaxial planes and stacked axially [Fig. 1a].

Figure 1.

Schematic drawing of the experimental setup for (a) cases 1 and 2 and (b) case 3.

The data acquisition was performed for 12 h at an energy window of 350–650 keV on both systems to avoid the complications of LSO background in this experimental setup. Each list mode data was histogrammed into two series of 3D sinograms with 20 and 40 min frames, respectively. A span of 3 and the maximum ring difference of each system (47 for the Focus220 and 79 for the Inveon) were used. Random coincidences were subtracted with the delayed timing window method. The 3D sinograms were first Fourier rebinned into 2D sinograms and then reconstructed by 2D FBP with a ramp filter cutoff at the Nyquist frequency. A component based normalization was applied before reconstruction.³³ Attenuation and scatter correction was not applied since the contrast phantom was physically relatively small.

The activities of the point sources and the backgrounds at the start of the 12 h acquisition are summarized in Table 1. Since the point source was placed on top of the uniform background, the signal of the lesion comes from both the point source and the background activity. The contrast was obtained by taking the ratio of the lesion signal and the background signal. Therefore, each contrast set had a contrast around 2.0 because the point source and the background have the same activity concentration in this case. Since both the point sources and the backgrounds were of the same thickness and were stacked axially in the tomographs, they have the same blurring effect along the axial direction. The maximum value of the point source and the mean value of the background are both blurred to the same extent along the z direction and this PVE effect along the z axis (PVE_z) cancels out when calculating the CNR based on Eq. 9. Therefore, the axial resolution was not taken into consideration here and the PVE_src in Eq. 9 is essentially PVE_xy, which means the partial volume effect for the point source in the transverse (x-y) plane.

Table 1.

Properties of each contrast set at the start of the 12 h acquisition.

	Focus220		Inveon
Point source size (mm2)	1.0×1.0	0.75×0.75	1.0×1.0	0.75×0.75
Point source activity (Bq∕mm2)	311.9	311.8	320.8	306.5
Background activity (Bq∕mm2)	291.9	286.4	296.7	303.4
Contrast	2.07	2.09	2.08	2.01

Case 2: Detectability evaluation with LSO background

Four ¹⁸F square point sources with the same nominal concentrations but different sizes were printed with the inkjet printer method. The actual activity and size of each point source were measured with the γ-counter and the 4× loupe as described in case 1. The four point sources sandwiched in 5 mm thick water equivalent material were placed in the transaxial direction and stacked along the axial direction [Fig. 1a]. The acquisition was performed for 12 h at 250–650 and 350–650 keV energy windows on both the Focus220 and the Inveon systems. The same histogram (20 min frames), reconstruction, and correction algorithms were applied to the data set as in case 1.

Table 2 summarizes the source properties of the acquisitions at the two energy window settings. The actual sizes of the printed sources were close to the nominal sizes specified in the drawing software. The concentrations across point sources of different sizes were close to each other with a standard deviation below 8% in each set. These results indicate that the inkjet printer method can be well controlled.

Table 2.

Properties of each point source at the start of the 12 h acquisition.

Acquisition at 250–650 keV energy window
	Focus220
Nominal size (mm2)	1.25×1.25	1.00×1.00	0.75×0.75	0.50×0.50
Concentration (Bq∕mm2)	361.8	361.1	307.6	360.8
	Inveon
Nominal size (mm2)	1.25×1.25	1.00×1.00	0.75×0.75	0.50×0.50
Concentration (Bq∕mm2)	449.6	497.2	477.6	479.2
Acquisition at 350–650 keV energy window
	Focus220
Nominal size (mm2)	1.25×1.25	1.00×1.00	0.75×0.75	0.50×0.50
Concentration (Bq∕mm2)	1112.8	1134.8	1041.0	1164.1
	Inveon
Nominal size (mm2)	1.25×1.25	1.00×1.00	0.75×0.75	0.50×0.50
Concentration (Bq∕mm2)	894.5	957.9	928.8	969.1

Case 3: Detectability evaluation with consideration of axial spatial resolution

The axial resolution of the Inveon tomograph degrades when the point source is positioned at an off-center position in the central transverse plane. In the experimental design of case 1, where the sources and the backgrounds were stacked in the axial direction, the limited axial resolution did not have an effect on contrast calculation. Here, in this case, three sets of contrast were printed and sandwiched in four pieces of 5 mm water equivalent material. They were placed in the coronal planes and stacked vertically along the radial direction [Fig. 1b]. With this setup, the radial resolution effect of the point source and the background will cancel out. The MDA evaluation of this setup will include the tangential and axial resolution effects. The PVE_src in Eq. 9 should include the partial volume effect in the coronal (x-z) plane and can be represented as PVE_xz.

The point sources and backgrounds were printed with a nominal size of 0.70×0.70 and 15.2×15.2 mm², respectively. The six sets of contrast (three sets for the Inveon and three sets for the Focus) were controlled to be as similar as possible (the point source size and the contrast) and all had a contrast around 6. The percentage standard deviation for the activity concentration of the six backgrounds and six point sources are 1.3% and 4.6%, respectively. The sizes and activities are summarized in Table 3. The system parameters used in the analytical evaluation, including the system sensitivity, tangential, and axial resolutions at the three positions are different. Their exact values were obtained from measurements on the same tomographs.

Table 3.

Properties of each contrast set at the start of the 12 h acquisition.

System	Focus220			Inveon
Radial offset (mm)	0	5	10	0	5	10
Point source size (mm2)	0.70×0.70	0.70×0.70	0.71×0.71	0.73×0.73	0.72×0.72	0.71×0.71
Point source activity (Bq∕mm2)	361.7	354.9	358.9	369.4	364.7	361.9
Background activity (Bq∕mm2)	69.6	70.3	69.6	71.8	71.0	71.4
Contrast	6.2	6.0	6.2	6.1	6.1	6.1

The acquisition of this setup was performed on both tomographic systems for 12 h at 350–650 keV energy window to minimize the LSO background effect. The list mode file was histogrammed into 40 min frames, with random and deadtime corrections. Each sinogram was then corrected for normalization and reconstructed by FORE+2DFBP, without applying attenuation and scatter corrections.

RESULTS

Experimental validation

Case 1: Detectability evaluation with biological background

The analytical CNR was calculated from Eqs. 8, 9. For the image ROI based measurements, the CNR can be calculated by

$${\text{CNR}}_{\text{image}}=\frac{{\text{max}}_{\text{source}}-{\text{mean}}_{\text{bkgd}}}{{\text{mean}}_{\text{bkgd}}}∕{\text{frac\_stdev}}_{\text{bkgd}}=\frac{{\text{max}}_{\text{source}}-{\text{mean}}_{\text{bkgd}}}{{\text{stdev}}_{\text{bkgd}}},$$ (14)

where max_source is the maximum value in the source ROI, and mean_bkgd, frac_stdev_bkgd, and stdev_bkgd are the mean value, fractional standard deviation, and standard deviation in the background ROI, respectively.

The source ROI, positioned centered on the point source, was a cylinder with 4 mm diameter and occupied four axial planes. The maximum value inside the ROI was obtained as the max_source. However, for frames of short lengths and∕or point sources with decayed low activities, significant noise was present in the single maximum pixel value. Therefore, for the longest 12 h acquisition, where little noise was present, the ratio R of the maximum value over the mean value was calculated. This ratio R is activity and frame length independent and is fixed for each point source and predefined ROI. For shorter frames, the low noise ROI mean instead of the ROI maximum value was obtained from the image and was then used together with the ratio R to estimate the maximum value with reduced noise. Since a single high pixel value in the source ROI does not translate into improved detectability, this method is useful to represent the maximum value with low noise. The background ROI was defined in a single transverse plane where the max_source value was obtained. It had a rectangular shape of 10×3 mm² to ensure enough pixels were included, but at the same time was far enough from both the background edge and the point source.

Figure 2a shows the signal of the two point sources (max_src−mean_bkgd) (▲ for the 1.0 mm source and ▲ for the 0.75 mm source) and the background (mean_bkgd) for the Inveon system with 20 min acquisitions. The signal of the point sources corrected for the PVE (◆ for the 1.0 mm source and ◆ for the 0.75 mm source) is also shown. The PVE corrected value and the mean background value agree with each other well because the printed sources and backgrounds have the same concentration. The nice agreement shown in Fig. 2a proves the accuracy of the PVE estimation, based on the known source geometry.

Figure 2.

(a) Point source maximum value (▲ for 1.0 mm source and ▲ for 0.75 mm source), PVE corrected point source maximum value (◆ for 1.0 mm source and ◆ for 0.75 mm source), and background signal obtained from ROI analysis on the reconstructed images. (b) Background noise (fractional standard deviation) obtained from analytical estimation and ROI analysis. The data shown here were acquired with the Inveon system and is for 20 min frames.

Figure 2b shows the analytically calculated background noise and image ROI based noise of a series of 20 min frames measured on the Inveon system. The background noise was averaged for the two contrast sets since they had the same concentration. In general, the noise calculated by the two methods agrees with each other well, indicating the feasibility of using the analytical method to estimate image noise. The difference in calculated background noise between the analytical and ROI methods becomes larger toward later frames. A possible reason of the overall discrepancy is the correlation between pixels in the reconstructed image.

Figure 3 shows the analytical CNR versus the ROI based CNR for the Inveon and the Focus220 tomographs. Both the 20 and 40 min frame series are shown for the two contrast sets. A good agreement between the two CNR calculation methods can be observed from the scatter plots.

Figure 3.

CNR obtained from ROI analysis versus CNR obtained by analytical estimation for the Inveon and the Focus220 tomographic systems acquired with both 20 and 40 min frames.

Table 4 summarizes the comparison of CNR between the two systems and the two frame lengths investigated. The expected CNR ratio was obtained from the analytical calculation. The measured CNR ratio was from image ROI analysis and averaged over all frames. When the source was positioned at the center of the scanner and acquired at a 350 keV LLD, the CNR of the Inveon is about 1.5 times higher than the Focus, mainly due to its higher sensitivity. By doubling the acquisition from 20 to 40 min, the detectability increases about 1.4 times. Here, the CNR_Inveon∕CNR_Focus for the 0.75×0.75 mm² source is lower than that for the 1.0×1.0 mm² source, because the 0.75×0.75 mm² contrast set imaged in the Inveon had lower contrast than the one imaged in the Focus220 due to variability during source printing. For sources positioned at the center of the two systems, e.g., this case, the main difference in detectability comes from the system sensitivity, which is independent on the imaged source size.

Table 4.

Comparison between the detectability for the Inveon and the Focus220 systems and between the 40 and 20 min frame lengths. Both the theoretical and measured comparisons are presented.

	CNRInveon∕CNRFocus		CNR40 min∕CNR20 min
	1.0×1.0 mm2	0.75×0.75 mm2	Focus and Inveon
Expected	1.54	1.39	1.37
Measuredavg	1.57	1.35	1.36

Table 5 shows the last 20 or 40 min image above the detection limit (CNR>4) as the activity decays and the next image which is the first with a CNR<4 (based on analytical CNR calculation). For both the 20 and 40 min images acquired on the Focus220, the 0.75×0.75 mm² source cannot be detected even on the very first frame containing the most activity at the start of the experiment (the analytical CNR is less than 4 and the lesion is not distinguishable from the background for all frames); therefore, the images are not shown here. The results are the same for the 20 min images on the Inveon, but not for the Inveon 40 min images. By visual assessment, the detectability of the point sources agrees well with the calculated CNR value and a detection limit of CNR=4.

Table 5.

The images from the Focus220 and the Inveon systems used for visual comparison for the detectable and nondetectable images as defined by the Rose criterion. The point source activities and the analytically calculated CNRs are also listed. For the 0.75×0.75 mm² source imaged in the Focus220 (both 20 and 40 min frames) and the 0.75×0.75 mm² source imaged in the Inveon (20 min frame only), the highest CNRs of the lesion are smaller than the detection limit of 4.0. Therefore, the images for these cases are not shown.

Case 2: Detectability evaluation with LSO background

The acquisition with point sources was performed at two LLDs of 250 and 350 keV without introducing any biological backgrounds (printed backgrounds in this work). At a lower LLD, the system sensitivity is higher, which increases the system detectability. However, the LSO background is also increased at low LLDs, which adversely affects the MDA performance. The net result is a competition between the two effects. The relative MDA performance between (a) the Inveon and the Focus220 systems and (b) the two energy windows at 250 and 350 keV were compared with each other.

The LSO equivalent background concentration cannot be easily obtained from the amount of ¹⁷⁶Lu in the scintillation crystals analytically because the system sensitivity was measured for 511 keV gammas. This sensitivity is not the same or the emissions of ¹⁷⁶Lu, which emits β and γ at different energies and from inside of the scintillation crystal. Instead, this concentration can be more accurately measured directly from the scanners. If the LSO background of a system is measured in advance, the absolute detectability of a point source with known properties can be predicted.

For this case, the CNR of the system can be expressed as

$$\text{CNR}=\frac{C_{\text{src}}\times k\times \text{PVE}}{{\sigma }_{\text{LSO}}}=\frac{C_{\text{src}}\times \text{PVE}}{{\text{frac\_stdev}}_{\text{LSO}}\times C_{\text{LSO}}},$$ (15)

where the PVE is the partial volume effect in the x, y, and z directions. The frac_stdev_LSO is related to the background true coincidences and is proportional to $1∕\sqrt{\eta \times C_{\text{LSO}}}$; therefore, the CNR for a point source imaged in a system with only LSO background is proportional to $\sqrt{\eta ∕C_{\text{LSO}}}$ based on Eq. 15, where η is the absolute system sensitivity and C_LSO is the center equivalent activity concentration of the LSO background.

If point sources with the same concentration and size were imaged in the Focus220 and the Inveon, by assuming that the two systems have the same resolution where the point sources were imaged, which is true to the first approximation for point sources located at the system center, we have

$${\text{CNR}}_{\text{Inveon}}∕{\text{CNR}}_{\text{Focus}}={\text{MDA}}_{\text{Focus}}∕{\text{MDA}}_{\text{Inveon}}=\sqrt{\frac{{\eta }_{\text{Inveon}}∕C_{\text{LSO-Inveon}}}{{\eta }_{\text{Focus}}∕C_{\text{LSO-Focus}}}}.$$ (16)

If no LSO background is present, then the CNR is proportional to η and we have

$${\text{CNR}}_{\text{Inveon}}∕{\text{CNR}}_{\text{Focus}}={\text{MDA}}_{\text{Focus}}∕{\text{MDA}}_{\text{Inveon}}=\sqrt{\frac{{\eta }_{\text{Inveon}}}{{\eta }_{\text{Focus}}}}.$$ (17)

Similarly, to compare the MDA at different energy windows, CNR_{350 keV}∕CNR_{250 keV} can be obtained by $\sqrt{{\eta }_{350\text{keV}}∕C_{\text{LSO-}350\text{keV}}∕{\eta }_{250\text{keV}}∕C_{\text{LSO-}250\text{keV}}}$, where the LSO concentration at different LLDs are directly obtained from the reconstructed images.

Table 6 summarizes the comparison between the two systems and the two energy windows. At a 350 keV LLD, the detectability of the Inveon is 1.55 times better than the Focus, while if a 250 keV LLD is employed, the detectability ratio decreases to 1.13 times due to the higher LSO background of the Inveon (MDA_Focus∕MDA_Inveon). The detectability is expected to be 7.9 times higher when a 350 keV LLD is used instead of 250 keV for a point source imaged in the Inveon system. The improvement is 5.6 times for the Focus220 system at these two LLDs.

Table 6.

Detectability comparison between the two tomographic systems and two energy windows for point sources in LSO background. The predicted ratio and measured ratio based on ROI analysis are both presented.

	250 keV	350 keV	Inveon	Focus
Predicted ratio	MDAFocus∕MDAInveon		MDA250 keV∕MDA350 keV
	1.13	1.55	7.9	5.6
Measured ratio	MDAFocus∕MDAInveon		MDA250 keV∕MDA350 keV
	1.2	1.7	1.2	1.7

The measured ratio was calculated by dividing the lowest source concentrations for the two systems and two energy windows, with which the CNR is immediately above 4.0. The measured ratios obtained from images match reasonably well with the predicted ratio when comparing between the Inveon and the Focus220. While the expected trend is there, with a higher ratio for the Inveon versus the Focus, the actual measured ratio between different energy windows shows some discrepancy with the predicted ratio. The most possible reason for this is the large error in the estimation of the background concentration at 350 keV. The accepted LSO background at 350 keV LLD is very low and the mean value estimation can have significant uncertainty. But the measured ratios qualitatively show that when 350 keV LLD was employed, the detectability of the source substantially increases by at least three fold.

The CNR and point source concentration for the last 20 min frame, in which the point source was detected as the activity decayed, are listed in Table 7 for each source size based on visual assessment of the reconstructed images. The detection criterion of CNR>4 agrees well with the visual adjustment. For sources smaller in size, the activity concentration of the source at the detection limit becomes higher due to more significant PVE. At the 250 keV LLD energy window, the Inveon only performs slightly better than the Focus220 system due to its higher LSO background concentration. At 350 keV LLD energy window, the detectability improves more for the Inveon system than for the Focus220 system. The detectability of both systems increased more than three times by raising the LLD from 250 to 350 keV. The amount of improvement in detectability by using a higher LLD depends on the source concentration. For source concentrations close to the LSO background concentration, significant gains in detectability can be achieved.

Table 7.

The lowest concentration of the point sources that can be detected based on image visual assessment. The CNR at the visual detection limit for each point source is also listed.

	1.25×1.25	1.0×1.0	0.7×0.7	0.5×0.5
	Inveon-250 keV
CNR	4.5	4.8	4.2	4.6
Concentration (Bq∕mm2)	8.9	12.8	20.4	38.5
	Focus-250 keV
CNR	4.3	4.0	4.0	4.1
Concentration (Bq∕mm2)	10.4	15.5	27.8	47.7
	Inveon-350 keV
CNR	4.9	5.0	5.0	5.0
Concentration (Bq∕mm2)	1.9	2.9	5.9	8.9
	Focus-350 keV
CNR	4.0	4.2	4.1	4.6
Concentration (Bq∕mm2)	3.1	5.9	8.5	17.8

Case 3: Detectability evaluation with consideration of axial spatial resolution

In this case, sources and “biological” backgrounds were aligned in the coronal planes and no LSO background was considered by setting the LLD to 350 keV during the acquisitions. For each contrast set, the source ROI was a cylinder with a diameter of 4 mm and occupied eight coronal planes. The background ROI was defined in a single coronal plane and had a rectangular shape of 8×3 mm². In order to reduce noise, the maximum value of the source ROI for each frame was obtained from the mean value of that frame multiplied with the maximum over mean value ratio R, which was obtained from the image reconstructed from the entire 12 h acquisition.

The maximum value from each point source for the 16 consecutive 40 min frames of the Inveon and the Focus systems were obtained [Figs. 4a, 4b]. Although the three point sources had the same physical size and same concentration, their measured maximum value differs because they were positioned differently and have different PVE effect. Larger differences in the maximum value could be observed for point sources imaged in the Inveon than those imaged in the Focus220 due to the fact that resolution degradation is more significant on the Inveon as the source moves toward off-center positions. After applying the PVE correction, the maximum values for the three curves overlap with each other [Figs. 4c, 4d] as in case 1 above.

Figure 4.

The maximum value of the point source obtained from ROI analysis for (a) the Inveon and (b) the Focus220 systems and the maximum value after applying the PVE correction for (c) the Inveon and (d) the Focus220.

Figure 5 shows the ROI based CNR versus the analytical calculated CNR for both the Inveon and the Focus220 systems. Good agreement can be observed for each source position, indicating the feasibility of using the analytical method to predict the image ROI CNR and thus the source detectability in different situations.

Figure 5.

The ROI based CNR versus the analytical based CNR for (a) the Inveon and (b) the Focus220 systems. Sources located at the geometric center (◆), 5 mm off-center (◆), and 10 mm off-center (◆) are shown.

A comparison between the Inveon and the Focus220 at the three different source positions is listed in Table 8. The expected CNR obtained from analytical calculations matches well with the measured ones. The Inveon detectability degrades faster than the Focus220 as the source moved away from the center. At 10 mm radial offset, the Inveon MDA performance is worse than the Focus220 (CNR_Inveon∕CNR_Focus<1), although it has a much higher system sensitivity.

Table 8.

The CNR comparison between the Inveon and the Focus220 at three different radial positions.

CNRInveon∕CNRFocus
Distance from center (mm)	0	5	10
Expected	1.41	1.27	0.82
Measuredavg	1.41	1.28	0.82

Images right above and below the detection limit (CNR=4) are shown in Table 9. The point source activities and the CNRs obtained from ROI analysis are also listed. By visual assessment, the detectability agrees reasonably well with the theoretical CNR limit.

Table 9.

Images from the Focus220 and the Inveon systems and their calculated CNRs for visual comparison for the detectable and nondetectable images defined by the Rose criterion.

It is important to note that in this case, the MDA was investigated at different positions only for a fixed point source size: 0.7×0.7 mm². At the 10 mm off-center position, the CNR_Inveon∕CNR_Focus is about half compared to that at the center position. This means the detectability of the Inveon degrades faster than that of the Focus as the source is placed away from the center. For smaller sources, the CNR_Inveon∕CNR_Focus will degrade even faster as the source moves away from the center position along the radial direction.

DISCUSSION

The good agreement in the evaluation of PVE, background noise, and CNR between the analytical method and the image based ROI analysis demonstrates the feasibility of using this simple way to evaluate the absolute detectability of a system. By knowing the conventional system performance parameters, including the system sensitivity, spatial resolution. and sampling pitch, one can predict the detectability of a source with known properties, background conditions, and acquisition settings.

The MDA performance for the Inveon and the Focus220 tomographs was evaluated in this work. Due to the high system sensitivity of the Inveon, it performs better in small lesion detections when the lesion is located at the system center and if a 350 keV LLD is used. If the LLD is lowered to 250 keV, similar detectability for the two systems was achieved. In effect, the improved MDA from the high sensitivity of the Inveon was counteracted by the high intrinsic activity from LSO emissions. If the source was positioned off-center (e.g., 10 mm offset in radial direction), the MDA performance of the Inveon is actually worse than the Focus220 due to the degraded spatial resolution at that position. These comparisons demonstrate that the MDA is a task specific parameter. It is strongly related to the system performance, source properties, acquisition conditions, and data processing.

The spatial resolution used to estimate the PVE effect for the point sources located at different positions inside the FOV were obtained from measurements conforming to the NEMA NU-4 protocol,¹² i.e., with a ²²Na source embedded in an acrylic tube. These measurements were performed separately and the resolutions in FWHM along the radial, tangential, and axial directions were obtained following the procedures specified in the NEMA standards.

Other than comparing the system detectability as presented in this study and predict whether a source can be detected or not, the analytical evaluation method can also be used in other estimates. For example, the minimum scan time required for detecting a lesion can be estimated for static acquisitions. The acquisition protocol, e.g., the choice of LLD, can be optimized to maximize the detection probability by balancing between the system sensitivity and intrinsic background. This method can also be used in designing new systems, especially since traditional system sensitivity and spatial resolution performance do not directly reflect the MDA of a system.

In this manuscript, only analytical FBP reconstruction was investigated, while the noise propagation from iterative reconstruction algorithms can be quite different.³⁴ For a given reconstruction algorithm, there is always a tradeoff between the spatial resolution and background noise. The resolution and noise in the reconstructed images depend on the filters used in FBP reconstruction, the iteration numbers used in OSEM algorithms,^{35, 36, 37} and the smoothing parameter β in MAP reconstruction.^{29, 30} The FBP reconstruction algorithm is an analytical method and a ramp filter cutoff at the Nyquist frequency was chosen as the standard method for the detectability evaluation in this work, simplifying the analysis. In order to use this analytical method for statistical iterative reconstruction methods, an expression for the noise needs to be developed. That is beyond the scope of this manuscript, but certainly requires further investigation.

The background noise in the experiment was determined by drawing an ROI in the background region, which contained a certain amount of pixels. By doing this, we made the implicit assumption that the pixels in the background are fully uncorrelated. However, this might not be the case and this may account for some of the discrepancy we observed in Fig. 2b. Experiments with multiple realizations need to be performed to get pixelwise noise, perhaps using the bootstrap method.^{38, 39}

In this work, several approximations were introduced during the analytical estimation. Attenuation and scatter corrections were not applied for all the three cases evaluated since the phantoms used in the experiments are relatively small. Also, additional noise, which could be introduced by Fourier rebinning (FORE), and normalization corrections were not considered as well. But as a first approximation and a fast evaluation method, the feasibility of this method was demonstrated. In this work, only 2D cases were tested and evaluated. However, it is straightforward to extend this method from 2D to 3D.

Monte Carlo simulation can also be used to evaluate the system detectability. Simulations are easy to set up and arbitrary source∕background activities, contrasts, and shapes can be easily described. Simulations also work for both real and virtual systems, 2D and 3D cases. However, simulations may take a long time and reconstruction of the images is needed to evaluate the detection limit. Furthermore, real experiments are often required to validate the simulations. Experiments, on the other hand, are straightforward and can be a direct measurement for the detectability. The measurements, though, are only available on real systems and the activity and contrast levels are relatively difficult to control. The analytical method presented here is easy, fast, and straightforward and can be used for arbitrary source properties. While using this method, the approximations that were made should be taken into consideration and the accuracy and robustness of this method should be further tested.

CONCLUSION

An analytical method of evaluating the PET system detectability was developed and validated by experiments. The noise propagation of reconstruction, source size, LSO background, system sensitivity, and spatial resolution were taken into consideration. The detectability was quantitatively compared between two preclinical small animal PET systems: The Inveon and Focus220 under three different acquisition conditions. The Rose criterion and the Curie equation were used as the detection standard. The analytical result, image ROI analysis based on experiment data, and visual assessment was compared with each other and good agreement was obtained.

APPENDIX: THE LSO BACKGROUND OF THE INVEON AND FOCUS220

Both the Inveon and Focus220 tomographic systems use LSO as the scintillation material. Based on the percentage of radioactive ¹⁷⁶Lu and the volume of the LSO in the two systems, the intrinsic crystal activities contained in the Focus220 and the Inveon crystals were calculated to be 151 700 (4.1 μCi) and 162 800 Bq (4.4 μCi), respectively. In the evaluation for the MDA limit, the concentrated target activity levels reach the order of couple Becquerel/mL; therefore the LSO background cannot be ignored, despite the fact that it is distributed in the FOV.

The emitted β particles have an average energy of 420 keV, while the three simultaneous emitted γ rays have energies of 307, 202, and 88 keV, respectively. The coincidence events due to the LSO background can be reduced if a higher LLD is used to reject the γ rays. The LSO background for both the Inveon and the Focus220 at a fixed 650 keV upper level discriminator and different LLDs were acquired for 14 h on the tomographs and the intrinsic true coincidence count rates were calculated and shown in Fig. 6a. The two systems have similar volume of LSO, but the Inveon recorded more true coincidences due to its higher sensitivity. The equivalent activities, defined as the actual activity in the FOV that would produce the same number of true coincidence events as the LSO background when placed at the scanner center, were calculated based on the system sensitivity at the corresponding energy windows. The equivalent activity for the Inveon is lower than the Focus220 as indicated in Fig. 6b. The equivalent activity concentrations were then obtained by dividing the imaging volume of the system assuming a uniform LSO background distribution [Fig. 6b]. Due to the more compact geometry and thus much smaller imaging FOV, the Inveon has significantly higher equivalent activity concentration of LSO background.

Figure 6.

The intrinsic radioactivity of ¹⁷⁶Lu in the LSO scintillators for Focus220 and the Inveon systems at different LLDs. (a) Measured true coincidence count rate. (b) Equivalent total center activity based on the system sensitivity (solid line) and equivalent activity concentration based on the size of the active FOV (dotted line).

The higher LSO background will adversely affect the detectability for very low activity sources. However, if the LLD can be raised higher than 350 keV LLD, the intrinsic background can be minimized at the expense of system sensitivity, which drops from 9.3% to 6.7% when the LLD is raised from 250 to 350 keV for the Inveon and from 3.9% to 2.7% for the Focus220. Although the high energy β emissions from the LSO cannot be completely rejected by raising the LLD and it can still produce coincidences with the true activity during real acquisitions, these coincidences are random events and are corrected with the delayed timing window technique.

The 14 h background acquisitions on the actual scanners were histogrammed into different frame lengths with span 3 and maximum ring difference. The sinograms were then Fourier rebinned⁴⁰ (FORE) and reconstructed with 2D FBP with a ramp filter cutoff at the Nyquist frequency. A component based normalization³³ was applied before reconstruction. The background mean and noise were obtained from the reconstructed images and plotted versus the frame lengths.

An ROI of diameter 15 mm was drawn on the reconstructed background images of different acquisition lengths. The standard deviation of the background for different frame lengths followed Poisson statistics, confirmed by a curve fit with a power exponent close to the ideal value −0.50 (data not shown). The mean value of the ROI was very noisy for short frame lengths, but was relatively constant for frame lengths longer than 7 h. Therefore, the mean background value of short frame lengths could be obtained by scaling long acquisitions.