Benefit of Improved Performance with State-of-the Art Digital PET/CT for Lesion Detection in Oncology

Abstract

The latest digital whole-body PET scanners provide a combination of higher sensitivity and improved spatial and timing resolution. We performed a lesion detectability study on two generations of Biograph PET/CT scanners, the mCT Flow and the Vision, to study the impact of improved physical performance on clinical performance. Our hypothesis was that the improved performance of the Vision would result in improved lesion detectability, allowing shorter imaging times or, equivalently, a lower injected dose. Methods: Data were acquired with the Society of Nuclear Medicine and Molecular Imaging Clinical Trials Network torso phantom combined with a 20-cm-diameter cylindrical phantom. Spherical lesions were emulated by acquiring sphere-in-air data and combining them with the phantom data to generate combined datasets with embedded lesions of known contrast. Two sphere sizes and uptakes were used: 9.89-mm-diameter spheres with 6:1 (lung) and 3:1 (cylinder) local activity concentration uptakes and 4.95-mm-diameter spheres with 9.6:1 (lung) and 4.5:1 (cylinder) local activity concentration uptakes. Standard image reconstruction was performed: an ordinary Poisson ordered-subsets expectation maximization algorithm with point-spread function and time-of-flight modeling and postreconstruction smoothing with a 5-mm gaussian filter. The Vision images were also generated without any postreconstruction smoothing. Generalized scan statistics methodology was used to estimate the area under the localized receiver-operating-characteristic curve (ALROC). Results: The higher sensitivity and improved time-of-flight performance of the Vision leads to reduced contrast in the background noise nodule distribution. Measured lesion contrast is also higher on the Vision because of its improved spatial resolution. Hence, the ALROC is noticeably higher for the Vision than for the mCT Flow. Conclusion: Improved overall performance of the Vision provides a factor of 4–6 reduction in imaging time (or injected dose) over the mCT Flow when using the ALROC metric for lesions at least 9.89 mm in diameter. Smaller lesions are barely detected in the mCT Flow, leading to even higher ALROC gains with the Vision. The improved spatial resolution of the Vision also leads to a higher measured contrast that is closer to the real uptake, implying improved quantification. Postreconstruction smoothing, however, reduces this improvement in measured contrast, thereby reducing the ALROC for small, high-uptake lesions.

The first generation of whole-body time-of-flight PET/CT scanners, introduced in 2006, achieved a spatial resolution of 4–5 mm with a sensitivity of 5–10 kcps/MBq and system coincidence timing resolution (CTR) in the range of 450–600 ps. The sensitivity was largely determined by the axial field of view, ranging from 16 to 21 cm depending on the vendor. The design choice of a 4- to 5-mm spatial resolution was largely determined by the practical constraints of using photomultiplier tubes in the detector but was also linked to the sensitivity and acceptable scan times in the clinic. Improved spatial resolution would require a commensurate improvement in sensitivity (or count statistics) to maintain similar image noise levels (1). In another study (2), however, it was also shown that for high-uptake lesions, the increased contrast due to improved spatial resolution reduces the number of counts needed to produce images of the same visual quality. Neither of these studies, however, included the impact of time-of-flight imaging on the resultant images. Recently, we performed a detailed simulation study of a modern, fully 3-dimensional PET system with iterative image reconstruction using clinically relevant metrics for lesion detection and localization (3). We found that for a fixed number of counts and CTR, improved spatial resolution leads to improved contrast measurement and small-lesion detectability (3).

The last three years have seen the introduction of digital PET/CT systems with higher sensitivity (up to 20 kcps/MBq) because of a longer axial field of view and improved CTR (in the range of 210–400 ps) due to the transition from conventional photomultiplier tubes to silicon photomultiplier tubes (4–6). Some of these systems also have improved spatial resolution due to the use of crystals that are less than 4 × 4 mm in cross section (6). The overall improved physical performance is expected to translate into a more sensitive and efficient clinical use of these new systems.

When a molecular imaging expert examines a PET image, the goal is to determine whether the image contains abnormal uptake signifying disease. In this study, we used the localized receiver operating curve (7) to study the gain in observing small lesions produced by the overall performance improvement of the Siemens Biograph Vision compared with the Siemens Biograph mCT Flow. The Vision provides higher sensitivity due to a longer axial field of view, as well as better spatial resolution and CTR, than the mCT Flow (6,8). We performed measurements using a heterogeneous torso phantom representing varying attenuation and activity distributions, as occur in a real patient. Ideally, one would like to measure the ability of human observers to correctly identify lesions in a series of images, a task that is formidable given the number of images needed to produce an accurate measurement. Alternatively, automated numeric methods have been developed (7,9–11), and we chose one to serve as a proxy for a molecular imaging expert’s detection and localization task (9). The area under the localized receiver-operating-characteristic curve (ALROC), which represents the probability of correctly detecting and localizing a lesion, was calculated as a function of imaging time. The noise in the image can mimic small lesions, thus producing false-positives. In the case of no noise, the observer would be correct every time and the ALROC would be unity. As noise increases, the observer’s ability to correctly identify lesions decreases, thus producing a lower ALROC. By comparing results for varying imaging times, we were able to quantify the differences between the 2 scanners as a difference in sensitivity for this task.

MATERIALS AND METHODS

Scanner Description

All imaging was performed on the mCT Flow (8) and Vision (6) scanners. Table 1 summarizes the important physical characteristics and performance values for these 2 systems. Data were acquired in list mode before processing.

TABLE 1.

Physical Characteristics and Performance Metrics of the 2 PET/CT Scanners

Parameter	mCT Flow (8)	Vision (6)
Detector	PMT-based analog	Silicon PMT–based digital
Scintillator	4 × 4 × 20 mm LSO	3.2 × 3.2 × 20 mm LSO
Axial field of view (cm)	21.0	26.3
NEMA spatial resolution at 1 cm (mm)	4.2	3.6
CTR (ps)	538	215
NEMA sensitivity (kcps/MBq)	10.0	15.6

PMT = photomultiplier tube; NEMA = National Electrical Manufacturers Association; LSO = lutetium oxyorthosilicate.

Phantom

The Society of Nuclear Medicine and Molecular Imaging Clinical Trials Network (CTN) torso phantom was used (Fig. 1). This 30-cm-long phantom consists of a uniform background region and 2 lung regions filled with polystyrene foam beads that emulate the attenuation and activity of human lung (12). An additional 20-cm-diameter by 30-cm-long cylindrical phantom was placed axially below the CTN phantom to act as a uniform liver region. The CTN phantom plus the uniform cylinder act as good surrogates for lung and liver, with heterogeneous attenuation and activity uptake. Because the CTN phantom has a 5-cm (2-in) lip, there is a visible gap between the 2 phantoms in reconstructed images. The CTN phantom also has several physical lesions present in various regions that were not used for this study.

FIGURE 1.

Photograph of CTN torso phantom and 20-cm-diameter by 30-cm-long cylinder used for these measurements (left), along with example reconstructed coronal (middle) and maximum-intensity-projection (MIP) (right) images. Lip at bottom of phantom and top of cylinder shows as gap in reconstructed images. Spheres in these images are physical spheres that are part of phantom and were not used for detectability study.

Data Acquisition

The CTN phantom was filled according to the recommended instructions (12). The activity concentration in the CTN phantom and uniform cylinder was 5.2 and 5.3 kBq/mL on the Vision and mCT Flow scanners, respectively. Twenty datasets were acquired using continuous bed motion (CBM) at a bed speed of 3.0 mm/s on each scanner for a total scan time of 3 min per dataset, covering 54 cm of the phantom.

Data were also collected with 4.95- and 9.89-mm-diameter spheres in air filled with 37–74 MBq of activity placed in the central plane at many transverse locations in order to overlap with the phantom. These data were collected for a single bed position in step-and-shoot mode, and each acquisition was for t₀ = 30 s. Lesion-embedding methodology (13,14) was modified and used to add the sphere-in-air data to the CTN phantom data in order to generate lesion-present phantom datasets.

Data Generation and Image Reconstruction

The 20 phantom datasets were merged to produce 10 datasets that are equivalent to a total imaging time of 6 min per dataset (equivalent bed speed of 1.5 mm/s). To account for activity decay and changing randoms fraction, the first dataset was merged with the 20th dataset, the second with the 19th, and onward. Each of these 10 datasets was also parsed into smaller fractions to produce 6 imaging times: 6, 3, 2, 1.5, 1, and 0.5 min. These 60 datasets, labeled as CTN phantom background data, represent 6 imaging times with 10 statistical replicates per imaging time.

In each of the 60 background datasets, we volumetrically embedded 37 spherical lesions of a fixed size: 16 in the lung and 21 in the cylinder. Lesion embedding is more practical for producing statistical data replicates and can also be used for generating patient data with a known lesion uptake. Two different lesion sizes were used: 9.89-mm diameter with 6:1 (lung) and 3:1 (cylinder) local activity concentration ratios and 4.95-mm diameter with 9.6:1 (lung) and 4.5:1 (cylinder) local activity concentration ratios. In this way, 2 lesion datasets (1 for each of the 2 lesion sizes) were generated for each of the 60 background datasets.

Standard image reconstruction was performed using the ordinary Poisson ordered-subsets expectation maximization algorithm with point-spread function and time-of-flight modeling, along with a 5-mm postreconstruction gaussian filter. For mCT Flow data, we used 3 iterations with 21 subsets to generate images with 1.85 × 1.85 × 2.03 mm voxels. For Vision data, we used 3 iterations with 5 subsets to generate images with 1.65 × 1.65 × 1.65 mm voxels. The reconstruction algorithm and the number of iterations and subsets used are the default parameters recommended by the manufacturer and used clinically at the University of Pennsylvania. Standard mCT Flow reconstruction uses slightly larger 2.04 × 2.04 × 2.03 mm image voxels, but we chose this smaller value here in order to be closer to the Vision image voxel size. Also, the current choice of postreconstruction smoothing for the Vision images was made in our clinic to maintain consistency in qualitative image quality with the mCT Flow, since patients may be scheduled on either scanner in the clinic. To better leverage the improved performance of the Vision, we also reconstructed all Vision data without application of postreconstruction smoothing. However, this was not done for the mCT Flow images, since the goal was to adhere to the current clinical practice for this scanner.

Lesion Embedding

For lesion embedding, we chose the plane in the phantom CBM sinogram where we wanted to embed the lesion. A chunk of the phantom data equal in axial length to a single bed position, and centered over the chosen plane, was reconstructed. From this reconstruction, we obtained the CBM normalization data and the attenuation correction factors. The count survival probability for sphere data was calculated as [step-and-shoot mode normalization data/(CBM normalization data × t₀ × attenuation correction factors)]. The count survival probability was then used to correct the sphere sinogram. The corrected sphere sinogram was added to the phantom CBM sinogram centered over the chosen plane to generate a fused sinogram with the embedded lesion (15). To have a predefined sphere activity uptake relative to the background, we first reconstructed a fused sinogram using all the sphere data. This image together with the phantom-only image gives the scale factor between the reconstructed image and the original sphere-in-air dataset, which can be used to select the number of sphere-in-air counts needed to represent a predefined sphere local uptake ratio in a fused dataset.

Image Analysis

Lesion detectability was numerically estimated using a generalized scan statistics method (9). In this method, we used 2 sets of statistical replicate images, with one set comprising the CTN phantom background and the other set comprising the CTN phantom with embedded lesions. Local lesion contrast was calculated over the multiple embedded lesions in the lesion images as the ratio of mean counts in a spherical volume of interest (same as sphere diameter) to the mean counts in an annular shell around the spherical volume of interest (inner diameter twice the sphere diameter and shell thickness of 1 cm). The lesion contrast distribution was fitted to a gaussian function to estimate the lesion contrast (or signal) probability density function in the lung and cylinder. The local contrast distribution for noise nodules was calculated by scanning the lung and cylinder regions of the CTN phantom background images and calculating the contrast for each image voxel. The contrast for each image voxel was calculated in a manner analogous to the sphere contrast calculation, with volumes of interest centered over that voxel. The local contrast distribution for noise nodules was then used to estimate the maximum scan distribution of the noise nodule contrast (probability density function of highest-contrast noise nodules present in the background), separately for the lung and uniform cylinder as described previously (9). The 2 probability density functions (signal and noise) for each region were then used to calculate the localized receiver-operating-characteristic curve (7) from first principles. ALROC was calculated for each lesion size and lesion location, as well as for all 6 imaging times, for the mCT Flow and the Vision. The error in ALROC was determined as the SD of the results over 100 bootstrapped copies obtained from each of the 2 probability density functions. The local lesion contrast value averaged over all lesions and its relative SD were also used as measures of accuracy and precision for the lesion uptake measurement.

RESULTS

Figure 2 shows representative reconstructed images that provide a qualitative idea of the impact of scanner performance on lesion detectability as a function of the imaging time.

FIGURE 2.

Example coronal images as function of imaging time from mCT Flow and Vision scanners. Images show 9.89-mm-diameter (A) and 4.95-mm-diameter (B) spheres. Arrows show embedded spheres in lung; all 6 embedded spheres in cylinder at bottom are same size as those in lung. Physical spheres visible in Vision and mCT Flow images are not identical because of difference in coronal slice where embedded spheres were placed.

Figure 3 summarizes the maximum scan contrast distribution of background nodules for each sphere size and location. Figure 3A compares results from the Vision scanner with and without postreconstruction smoothing. As expected, the maximum scan distribution degrades (higher centroid) as the imaging time is reduced and is systematically worse for images without any postreconstruction smoothing, leading to an increase in false-positive results. The maximum scan distribution is also higher for the smaller spheres and for spheres located in the lung. The change in maximum scan centroid as a function of imaging time is larger in the cylinder. In the lung, the maximum scan centroid for the longest imaging time for images without postreconstruction smoothing is higher than the centroid for the shortest imaging time for images with postreconstruction smoothing. Figure 3B compares results from the Vision and mCT Flow scanners, both with postreconstruction smoothing. Because of lower sensitivity and worse CTR, the maximum scan centroids are systematically higher for the mCT Flow, but they are slightly lower than for the Vision without postreconstruction image smoothing (Fig. 3A).

FIGURE 3.

Maximum scan distribution of noise nodule contrast calculated in lung and cylinder regions for 9.89- and 4.95-mm-diameter spheres. (A) Plots showing data from Vision with (solid lines) and without (dashed lines) postreconstruction smoothing. (B) Plots showing data from Vision (solid lines) and mCT Flow (dashed lines), both with postreconstruction smoothing. Within each group of curves, scan time from left to right is 6 min (red), 2 min (magenta), 1 min (blue), and 0.5 min (green). VOI = volume of interest.

Figure 4 shows measured sphere contrast as a function of imaging time. With postreconstruction smoothing, both the mCT Flow and the Vision have very similar measured contrast (although Vision contrast is slightly higher). Without postreconstruction smoothing, the improved intrinsic spatial resolution of the Vision leads to higher measured sphere contrast. Despite the higher uptake of the 4.95-mm-diameter spheres, the measured contrast was lower than that for the 9.89-mm-diameter spheres because of an increased partial-volume effect. However, without postreconstruction smoothing, the partial-volume effect was reduced in the images from the Vision, with a larger relative effect on the smaller spheres.

FIGURE 4.

Measured contrast as function of imaging time for spheres in lung and cylinder regions for 9.89- and 4.95-mm-diameter spheres. True activity concentration ratios were 6:1 and 3:1 in lung and cylinder, respectively, for 9.89-mm-diameter spheres and 9.6:1 and 4.5:1 in lung and cylinder, respectively, for 4.95-mm-diameter spheres. Data are mean ± SD calculated over 160 lung spheres and 210 cylinder spheres. Red dotted and solid lines are for Vision scanner without and with postreconstruction smoothing, respectively, whereas blue dashed line is for mCT Flow with postreconstruction smoothing.

Figure 5 shows the ALROC calculated as a function of imaging time. In the cylinder and lung, the mCT Flow had a systematically lower ALROC due to a lower measured sphere contrast and a higher background maximum scan contrast distribution relative to the Vision with postreconstruction smoothing. In the cylinder, the ALROC values for the Vision with and without postreconstruction smoothing were very similar. However, in the lung, the Vision without postreconstruction smoothing produced a higher ALROC, especially for the 4.95-mm-diameter spheres.

FIGURE 5.

ALROC calculated as function of imaging time for 9.89- and 4.95-mm-diameter spheres in lung and cylinder regions. Red dotted and solid lines are for Vision scanner without and with postreconstruction smoothing, respectively, whereas blue dashed line is for mCT Flow with postreconstruction smoothing.

DISCUSSION

On the Vision, the ALROC reached more than 0.9 for 9.89-mm-diameter spheres in the cylinder after 1 to 1.5 min of imaging versus 6 min on the mCT Flow. The reduced contrast of the 4.95-mm-diameter spheres (despite a higher uptake) together with an increased maximum scan centroid for background nodules leads to a lower ALROC relative to the 9.89-mm-diameter spheres. However, a 6-min scan will lead to an ALROC of more than 0.8 on the Vision. A visual observation of images indicates that, qualitatively, images with an ALROC of 0.80 or higher would generally lead to a very high confidence in the lesion detection and localization task. This has been our experience when using this numeric observer in past work as well (16,17). In contrast, these spheres are not detectable on the mCT Flow even after 6 min of imaging time (ALROC, 0.08).

In the lung, the mCT Flow again underperformed relative to the Vision. The maximum ALROC achieved plateaus at less than 1.0 for the 2 sphere sizes and 2 locations, except for the smaller sphere in the Vision without postreconstruction smoothing, for which the ALROC continued to increase with imaging time. This finding was primarily due to the presence of nodular structures in the lung (polystyrene foam beads) that led to an additional nonstatistical feature in the maximum scan contrast distribution. In the Vision and mCT Flow images with postreconstruction smoothing, the ALROC for the 9.89-mm-diameter sphere was always higher than that for the 4.95-mm-diameter sphere because of increased measured contrast and lower maximum scan centroid. When comparing the Vision data with and without postreconstruction smoothing, we found that for the 9.89-mm-diameter sphere the ALROC values were similar, limited by the maximum scan centroid distribution due to the lung structure. However, for the 4.95-mm-diameter sphere, images without postreconstruction smoothing showed an increase in ALROC, driven primarily by the much higher contrast measured for these spheres. On the Vision, the ALROC reached approximately 0.75 for 9.89-mm-diameter spheres in the lung after 1 min as opposed to 6 min on the mCT Flow for an equivalent ALROC; this is consistent with the reduction in imaging time we observed for the same size of spheres in the cylinder. The 4.95-mm-diameter spheres were once again barely detectable in the mCT Flow, with an ALROC of 0.23 after 6 min, which is similar to or worse than the ALROC achieved on the Vision after 0.5 min.

The combination of improved spatial resolution, which provides higher measured contrast, and higher intrinsic sensitivity and improved CTR, which result in better noise characteristics (lower maximum scan centroid), leads to a significant increase in ALROC for the Vision relative to the mCT Flow. For the Vision, postreconstruction smoothing degrades spatial resolution and lowers measured contrast (similar to the mCT Flow) but improves noise properties relative to no postreconstruction smoothing. Generally, the trade-off results in no significant change in ALROC for the 9.89-mm-diameter spheres. However, the reduced partial-volume effect for smaller spheres on the Vision without any postreconstruction smoothing leads to a much higher measured contrast. In the case of high-uptake small spheres such as those modeled here in the lung, this leads to a distinct gain in ALROC over Vision images with postreconstruction smoothing.

The improved physical performance of the Vision leads to a factor of 4–6 reduction in overall scan time for comparable detectability when using ALROC as the metric for 9.89-mm-diameter spheres; the reduction is larger for 4.95-mm-diameter spheres. These results are consistent with past simulation work (3) in which we estimated a similar reduction in imaging times for a scanner with improved spatial resolution and CTR and accounting for the 50% higher sensitivity of the Vision relative to the mCT Flow. The improved imaging capability of the Vision can be used to obtain the same image performance with shorter imaging times (or a reduced injected dose). Alternatively, the noticeable gains in detection of smaller spheres, which are not visible in the mCT Flow, may lead to improved clinical diagnostic capabilities for small lesions not achievable on the mCT Flow.

The measured contrast in the Vision without postreconstruction smoothing was also higher and closer to the real uptake, indicating potential for improved quantification. Hence, these results indicate an advantage to not using any postreconstruction smoothing on Vision data. This could be extrapolated to conclude that it is likely that mCT Flow performance could also be improved without any postreconstruction smoothing, albeit for larger spheres. However, this possibility was not tested here since the goal was to adhere to the current clinical practice for this scanner.

Recently, we have seen the development of 2 long-axial-field-of-view scanners, the uExplorer (United Imaging) (18) (better spatial resolution and worse timing resolution than the Vision) and the PennPET Explorer (19) (closer in both spatial resolution and timing resolution to the Vision). The longer axial length of these systems leads to about a factor of 3 gain in sensitivity over the Vision for imaging single organs. This gain should lead to better detectability for similar scan times.

Our study used the CTN torso phantom with a uniform cylinder as a surrogate for patient data. Although this phantom does not include some of the real-world effects, such as respiratory motion, it does include heterogeneous activity and attenuation regions. Respiratory motion will add blurring in the lung lesions and reduce any advantage of the improved spatial resolution of the Vision. However, the higher sensitivity and CTR will still improve the ALROC performance. There are some additional limitations of this study arising from the phantom construction: the lung beads are not the same as real lung structures, and real liver is not completely uniform as is the cylinder. The methods used in our work are easily applicable to patient datasets with embedded spheres as recently used by us in a different study (17). In the future, we can envision expanding the current work in that direction. Finally, the embedded spheres had fixed sizes and local activity ratios, whereas lesions in patients can vary widely in size, shape, and intensity. However, our results show clear gains in ALROC achieved by the Vision scanner, beyond those one would predict from the increased sensitivity (×1.5) and improved CTR (×3.0) even for the 9.89-mm-diameter spheres. Hence, the improved spatial resolution leads to a higher ALROC, especially in smaller lesions.

Our study used a fixed number of iterations for image reconstruction as defined by the vendor. For iterative reconstruction algorithms, the image will change as a function of iteration number and hence can impact the ALROC results. However, for this work our aim was to study the impact of the scanner performance on ALROC results using commercially implemented reconstruction algorithms. In the future, we may consider a more in-depth study that investigates the impact of varying reconstruction algorithm parameters to better optimize the scanner performance.

CONCLUSION

The improved overall performance of the Vision will lead to factor of 4–6 reduction in imaging time (or injected dose) compared with the mCT Flow when using the ALROC metric for lesion detection and localization of spheres at least 9.89 mm in diameter. The relative gains for detecting smaller lesions are much larger than this, since the ALROC is very low on the mCT Flow even for the longest scan time considered here (6 min for imaging 54 cm of phantom length). The improved spatial resolution of the Vision also leads to a much higher measured contrast that is closer to the known uptake ratio, indicating improved quantification due to reduced partial-volume effect. Using postreconstruction smoothing with a 5-mm gaussian filter reduces the quantification accuracy in the Vision images and reduces the ALROC for small, high-uptake lesions.

DISCLOSURE

This work was supported by National Institutes of Health grants R01-CA113941 and R01-CA196528 and a Siemens research contract with the University of Pennsylvania. Maurizio Conti and Michael E. Casey are employees of Siemens Medical Solutions. No other potential conflict of interest relevant to this article was reported.

KEY POINTS

• QUESTION: How do the improved spatial and timing resolution, in addition to the higher sensitivity, of modern digital PET/CT scanners impact oncologic lesion detection?

• PERTINENT FINDINGS: Task-based evaluation of small-lesion detection and localization was performed in a heterogeneous CTN phantom. The results indicate up to a factor 6 gain in performance with the Vision scanner relative to the mCT Flow. In addition, 5-mm-diameter lesions have very low detectability in the mCT Flow for current clinical imaging times, but they are easily detected with the Vision.

• IMPLICATIONS FOR PATIENT CARE: The overall improved performance of the Vision will lead to improved diagnostic capability in patients for detecting small lesions with much shorter imaging times.