Small target repeatability of ⁶⁸Ga and ¹⁸F: effects of target concentration and imaging time on SUV measurements in clinically relevant phantoms

Abstract

Quantification of tumor uptake using PET imaging is important for the evaluation of therapy response. For ¹⁸F FDG PET scans, a change in uptake of 25% is commonly considered significant. For scans using novel radiopharmaceuticals, the threshold of significance is unclear. Factors including imaging time, tumor size, activity concentration, and radiopharmaceutical may affect the repeatablity of uptake metrics. This work evaluates the effect of these parameters on the repeatablity of maximum SUV (SUV_max) and mean SUV (SUV_mean) in phantoms using ¹⁸F and ⁶⁸Ga. An Esser PET phantom (Data Spectrum, Durham NC) was scanned on a Biograph Horizon PET/CT scanner (Siemens Medical Solutions, Malvern PA) using ¹⁸F and ⁶⁸Ga. Data were acquired for 5 minutes with reconstructions between 0.5-5 minutes. The background activity mimicked clinical scans with target-to-background (T/B) ratios from 1.7-19.8. The SUV_max and SUV_mean were measured for 5 slices. The mean, standard deviation, and coefficient of variation (COV) were calculated. The effects of radionuclide, imaging time, activity concentration, and target size on COV were evaluated using multivariate gamma regressions. COV for ⁶⁸Ga was 40% higher and 54% higher on average than for ¹⁸F for SUV_max and SUV_mean, respectively. Decreased lesion size, imaging time, and activity concentration were significantly associated with increased COV for both metrics (P < 0.001). COV was substantially reduced at high T/B for ⁶⁸Ga. At the highest T/B the COV for SUV_max and SUV_mean was within the typical range seen for ¹⁸F. COV is relatively high for small targets (8 mm) but is dramatically reduced with high radiotracer uptake.

Introduction

Early detection of small lesions by PET is important because therapeutic interventions may be altered based on the early detection of small metastatic lesions [1-4]. For example, ¹⁸F DCFPyl PET/CT identified early oligometastatic prostate lesions for targeted external beam radiation and demonstrated improved progression free survival and decreased the risk of new lesions in at 6 months [5]. Quantification of tumor uptake with ¹⁸F FDG PET imaging is also important to assess response to therapy and has numerous uses in drug development [6-8]. Changes in tumor uptake for FDG PET scans are not usually considered “significant” unless a greater than approximately 25% increase or decrease in Standardized Uptake Value (SUV) can be demonstrated in relatively large lesions, typically ≥ 2 cm in diameter [8].

The quantitative reproducibility of small lesions, less than 2 cm, has not been well characterized. Physical factors which may influence measurements include lesion size due to partial volume effect, target-to-background ratio (T/B), and imaging counts which are affected by scanner sensitivity, imaging time, and administered doses. Furthermore, quantitative repeatability has not been established for novel pharmaceuticals which use ⁶⁸Ga as the imaging radionuclide. The physical properties of ⁶⁸Ga compared to ¹⁸F include a shorter half-life, longer mean free path of the positron effecting partial volume, and the presence of high energy gamma emissions. Previous work has demonstrated that these properties may adversely affect signal-to-noise ratio, potentially impacting the ability to detect small lesions [9,10]. These properties, coupled with a typically lower administered activity for ⁶⁸Ga DOTATATE studies, and more rapid physical decay result in lower true coincidence events compared to ¹⁸F FDG, also potentially adversely affecting the quantitative reproducibility of uptake measurements.

Both physical and biological effects (radiopharmaceutical biodistribution properties, tumor biology, and individual tumor heterogeneity, and others) influence the repeatability of SUV measurements. The purpose of this study is to evaluate the physical effects of repeatability of SUV measurements for ⁶⁸Ga compared to ¹⁸F. Specifically for small targets, we evaluate the effects of target activity concentration, and imaging counts on the repeatability of maximum SUV (SUV_max) and mean SUV (SUV_mean) in clinically relevant phantom studies.

Methods

Data acquisition

Phantom scans were performed using an Esser PET phantom (Data Spectrum, Durham NC) with data acquired and reconstructed on a Biograph Horizon PET/CT scanner (Siemens Medical Solutions, Malvern PA). The phantom was dosed in accordance with the ACR testing instructions assuming an administered activity of 370 MBq (10 mCi) for the ¹⁸F phantom and 185 MBq (5 mCi) for ⁶⁸Ga phantom, resulting in a T/B of approximately 2.4 to 1 in both cases [11]. These activities where selected as they represent typical administered activities for common scans. A 60-minute decay period between phantom activity calibration and the scan was used to simulate decay during uptake. Data were acquired for 5 minutes in list mode using a single bed position centered over the hot cylinders. Following decay of the phantoms to background, these scans were repeated with the activity concentration in the cylinders adjusted to result in additional T/B ranging from 1.69 to 4.82 in ¹⁸F and 1.62 to 19.38 in ⁶⁸Ga. These T/B ratios were selected to result in similar activity concentrations as seen in ¹⁸F FDG and ⁶⁸Ga DOTATATE scans [12]. Table 1 lists the radionuclide, activity concentrations at scanning start time, nominal SUV in the background and cylinders, and T/B for each scan. The activity concentration in the background was chosen to yield a SUV_mean of 1 and is typical of 70 kg patient following an uptake time of one hour.

Table 1.

Phantom dosing information for each scan performed including radionuclide, activity concentration at scanning start time in both background and cylinders, nominal SUV for both background and cylinders, and target-to-background ratio

Scan	Radionuclide	Cylinder Concentration (kBq/ml)	Cylinder Nominal SUV	Background Concentration (kBq/ml)	Background Nominal SUV	Target to Background Ratio
1	18F	5.96	1.65	3.53	0.97	1.69
2	18F	8.41	2.32	3.46	0.96	2.43
3	18F	12.59	3.49	3.48	0.97	3.59
4	18F	16.82	4.64	3.49	0.96	4.82
5	68Ga	2.23	1.55	1.37	0.96	1.62
6	68Ga	3.21	2.24	1.36	0.95	2.37
7	68Ga	4.88	3.41	1.41	0.98	3.48
8	68Ga	6.56	4.57	1.37	0.96	4.78
9	68Ga	12.88	8.98	1.36	0.95	9.51
10	68Ga	25.93	18.08	1.34	0.93	19.38

Scans using ¹⁸F use activity concentrations determined assuming a standard administered activity of 370 MBq (10 mCi). Scans using ⁶⁸Ga use activity concentrations determined assuming a standard administered activity of 185 MBq (5 mCi). The background concentrations are selected to yield a nominal SUV of approximately 1. Consequently, cylinder nominal SUV is similar to the target-to-background ratio.

Image reconstruction

Images for all phantoms were reconstructed with a 180 × 180 image matrix and a voxel size of 4.1 mm × 4.1 mm × 4 mm using manufacturer (Siemens Medical Systems) point spread function correction and time-of-flight (TOF) using OSEM with 4 iterations and 10 subsets and a 5 mm gaussian post reconstruction filter. CT based attenuation correction was used with scatter modelled from the attenuation map. Each 5-minute data set was reconstructed utilizing variable imaging time with the data centered temporally in the 5-minute window as follows: 5 minutes, 4 minutes, 3 minutes, 2 minutes, 1 minute, and 0.5 minutes.

Measurement and analysis

To quantify uptake for each imaging time, target activity concentration, and lesion size, measurements were made on five adjacent slices centered in the axial extent of the hot cylinders. Circular regions of interest (ROI) were drawn on the CT images with a diameter set to match each cylinder (25 mm, 16 mm, 12 mm, and 8 mm). These ROIs were transferred to the PET images and their positioning was verified. The SUV_max and SUV_mean were recorded for each ROI. The mean, standard deviation, and coefficient of variation (COV) of the 5 slices were calculated for each SUV_max and SUV_mean.

Statistical analyses evaluated whether the following four design variables were associated with the COV: radionuclide type (¹⁸F and ⁶⁸Ga), imaging time, activity concentration, and lesion size. More specifically, COV in SUV was modeled separately for both SUV_max and SUV_mean using multivariate gamma regressions, with a random intercept for each cylinder to account for repeated measures under the different imaging times. For each SUV_max and SUV_mean, the interaction between radionuclide type and each of the other three parameters were investigated, which answers whether the radionuclide type had a modifying effect on the other parameters. If the interactions were not significant, then the effect of ¹⁸F vs. ⁶⁸Ga is interpreted independent of the other parameters. Moreover, the simplest model would include each of the four design variables (radionuclide type, time, size, and T/B) without any interactions, where each parameter’s effect on the COV is also accounting for the variance explained by the other parameters. For more statistical modeling details, the standard deviation was used as the outcome in a gamma regression (link = log) with the logarithm of the mean used as an offset, which avoids the concern of modeling a ratio as the outcome (COV = standard deviation/mean). This modeling approach does not change any objectives since the interpretation remains the same: in terms of the COV. The effects were estimated with empirical standard errors (or “robust sandwich estimators”). Ultimately, the interpretation of the results provides an estimated rate of change in the COV for each of the four design variables.

Additionally, two higher T/B were measured for ⁶⁸Ga only: 9.51:1 and 19.38:1 to mirror the T/B measured from clinical ⁶⁸Ga DOTATATE PET/CT studies [12]. Like the approach described above (using the same multivariate gamma regression format), the changes in COV for ⁶⁸Ga were modeled versus the parameters of imaging time, size, and T/B. However, two slightly different modeling strategies were used to accommodate the additional T/B. First, to assess the effect of T/B, the COV of SUV_max and SUV_mean from all six T/B were compared while accounting for imaging time and cylinder size. More specifically, both the 9.51:1 and 19.38:1 were each compared to the five other T/B using Dunnett’s adjustment for multiple comparisons (i.e., 9.5:1 versus all, and 19.4:1 versus all). Second, to assess the effect of imaging time and cylinder size at high T/B, only the data from the two additional T/B were used to evaluate the changes in COV.

Lastly, as an ad-hoc analysis, the effect of radionuclide type was assessed for the smallest cylinder size only (8 mm). Using the same multivariate regression approach, changes in COV were modeled versus the parameters of radionuclide type, imaging time, and T/B. Statistical modeling was conducted in SAS 9.4 software (SAS Institute Inc., Cary, NC, USA) and R. Using the Bonferroni correction for the two outcomes (SUV_max and SUV_mean), the threshold for determining statistical significance was set at 0.025.

Results

Effect of Radionuclide on COV

The COV for ¹⁸F and ⁶⁸Ga were found to be significantly different for both SUV_max and SUV_mean as an independent effect (separate from time, T/B, or target size). On average, the COV for ⁶⁸Ga scans is 40% higher for SUV_max and 54% higher for SUV_mean compared to ¹⁸F scans, after accounting for imaging time, cylinder size, and T/B (P < 0.01). While these values represent the average difference across all measurements, it should be noted that, when considering the 8 mm targets alone, there is not a significant difference in COV of SUV_max (P = 0.699) or SUV_mean (P = 0.858).

Figure 1 shows the five consecutive slices used for COV calculation for the 3-minute frame duration using ¹⁸F at a T/B of 2.4:1 (top row), ⁶⁸Ga at a T/B of 2.4:1 (middle row), ⁶⁸Ga at a T/B of 19.4:1 (bottom row). Image window width and level is adjusted to maintain consistent background intensity. As shown, ¹⁸F scans are observably less noisy than the ⁶⁸Ga scans. In addition, the 8 mm target is clearly observable in ¹⁸F scan at a T/B of 2.4:1 but not in the ⁶⁸Ga scan at 2.4:1. Increasing the T/B to 19.4:1 increases the visibility of all targets and renders the 8 mm target clearly observable when using ⁶⁸Ga.

Figure 1.

Five consecutive slices used for COV calculation for the 3-minute frame duration using ¹⁸F at a T/B of 2.4:1 (top row), ⁶⁸Ga at a T/B of 2.4:1 (middle row), ⁶⁸Ga at a T/B of 19.4:1 (bottom row). While ⁶⁸Ga scans are generally noisier that the ¹⁸F scans, high T/B may still result in clear visibility and repeatability of small targets.

Figure 2 illustrates the difference in COV calculated for both SUV_max and SUV_mean between ⁶⁸Ga and ¹⁸F. Here, data from the 30 second images has been omitted as the ⁶⁸Ga images were very noisy and visualization of both the 8 mm cylinder and the 12 mm cylinder was not possible at the standard concentration. As shown, COV is relatively high for both radionuclides for the smallest 8 mm targets.

Figure 2.

The COV for SUV_max (left column) and SUV_mean (right column) calculated for 5,4,3,2, and 1 minute for each target size is shown at the standard target-to-background ratio of 2.4:1 for ⁶⁸Ga and ¹⁸F. As shown, COV was greater for ⁶⁸Ga than for ¹⁸F at the 3 larger cylinder sizes. At the smallest target (8 mm), the difference in COV was not found to be significant for either SUV_max (P = 0.699) or SUV_mean (P = 0.858).

Effects of imaging time, T/B, and target size on COV

The COV for SUV_max ranged from 0.006 to 0.152 for ¹⁸F and from 0.012 to 0.206 for ⁶⁸Ga. The COV for SUV_mean ranged from 0.002 to 0.147 for ¹⁸F and from approximately 0.004 to 0.189 for ⁶⁸Ga. For both radionuclides, combinations of small lesion size, short imaging time, and low activity concentration in the targets tended to result in the highest COV. Table 2 shows the estimated rate of change in COV for both SUV_max and SUV_mean resulting from decreases in time, T/B, and target size for both ¹⁸F and ⁶⁸Ga. The radionuclide type did not significantly alter the effect from the imaging time, T/B, or size parameters on the COV (all P > 0.025). Consequently, radionuclide independent rates of change were calculated for each parameter and listed in Table 3. The changes listed for each parameter are associated with a significant increase in COV (all P < 0.001).

Table 2.

Regression estimated rates of change in COV for specific changes in time, target-to-background ratio, and target size for both SUV_max and SUV_mean are shown with their confidence intervals in parentheses

Parameter	Radionuclide	COV Rate of Change for SUVmax (C.I.)	P-Value	COV Rate of Change for SUVmean (C.I.)	P-Value
Time (1-minute decrease)	18F	32.4% (25.2, 40.1)	0.039	31.9% (24.6, 39.7)	0.108
	68Ga	23.1% (16.4, 30.2)		24.5% (17.6, 31.8)
T/B (1-unit decrease)	18F	25.1% (11.2, 40.7)	0.202	29.5% (13.4, 47.9)	0.875
	68Ga	13.8% (1.2, 27.9)		27.8% (12.0, 45.9)
Size (4 mm decrease)	18F	16.8% (6.8, 27.6)	0.143	38.2% (25.0, 52.8)	0.112
	68Ga	7.6% (-1.6, 17.5)		25.0% (13.1, 38.2)

It should be noted that these are 97.5% confidence intervals, as opposed to the common 95%, since our level of statistical significance was 0.025. These rates where not found to be statistically different between ¹⁸F and ⁶⁸Ga (all P > 0.025).

Table 3.

Radionuclide-independent estimated rates of change in COV for specific changes in time, target-to-background ratio, and target size for both SUV_max and SUV_mean are show with their confidence intervals in parentheses

Parameter	COV Rate of Change (SUVmax)	P-Value	COV Rate of Change (SUVmean)	P-Value
Time (1-minute decrease)	27.5% (22.5, 32.8)	< 0.001	28.2% (23.0, 33.5)	< 0.001
T/B (1-unit decrease)	19.1% (9.4, 29.8)	< 0.001	28.8% (17.2, 41.6)	< 0.001
Size (4 mm decrease)	12.0% (5.0, 19.4)	< 0.001	31.5% (22.4, 41.2)	< 0.001

It should be noted that these are 97.5% confidence intervals, as opposed to the common 95%, since our level of statistical significance was 0.025. P-values less than 0.025 indicate that the rate of change is significantly different than zero (zero = no change in COV).

Effect of high target-to-background ratios on COV for ⁶⁸Ga

As above, two additional T/B (9.51:1 and 19.38:1) were tested for ⁶⁸Ga. The rationale for these additional scans is based on ⁶⁸Ga DOTATATE lesions which are reported to result in higher uptake values than typically seen FDG ¹⁸F studies [12]. Overall, the addition of the two higher T/B did not significantly alter the results of this study. The results for T/B and imaging time are consistent with the primary findings described above. In general, greater T/B resulted in decreased COV across both SUV_max and SUV_mean. Figure 3 illustrates this effect across multiple imaging times and target sizes with the higher T/B, common of ⁶⁸Ga studies consistently resulting in lower COV. Figure 4 provides additional illustration of the advantage of high T/B, plotting COV for both SUV_max and SUV_mean for the small, 8 mm targets using a 3-minute imaging time at each T/B. As shown, COV is substantially reduced for small targets at the high T/B.

Figure 3.

For ⁶⁸Ga, the COV for SUV_max (left column) and SUV_mean (right column) calculated for 5, 4, 3, 2, and 1 minute for each target size is shown at both the standard target-to-background ratio of 2.4:1 as well as the highest target to background ratio of 19.4 to 1.

Figure 4.

For small, 8 mm targets and a 3-minute imaging duration, the mean COV predicted by regression for both SUV_max and SUV_mean is plotted at each target to background ratio using ⁶⁸Ga. Error bars represent the 95% confidence intervals. COV is substantially reduced at the highest target to background ratio, demonstrating high repeatability even for small targets. High target-to-background ratios are commonly seen in ⁶⁸Ga DOTATATE lesions.

Discussion

This study evaluated the COV of the SUV_max and SUV_mean while varying imaging time, cylinder size, and T/B for both ⁶⁸Ga and ¹⁸F. Although the phantoms used in this study are not subject to the biological variability of tumors in human patients, the ranges of the three design parameters were chosen to provide supporting evidence for SUV repeatability relevant to clinical practice. Using clinically relevant target sizes including the very small 8 mm targets, imaging times, and target activity concentrations, this study demonstrated that each of these variables have a significant, independent influence on the repeatability of uptake measurements as measured by COV.

While the COV for ⁶⁸Ga scans was generally higher than that of ¹⁸F scans, it is notable that at the high ⁶⁸Ga T/B, the COV for SUV_max was very low (~5%), even when tested at very short imaging times (30 seconds) and in very small targets (8 mm). This can be contrasted with the results from the standard concentration where the COV for the 8 mm object was ≥ 5% for both ¹⁸F and ⁶⁸Ga. A contributing factor to this relatively high COV in the 8 mm target may have been a combination of the partial volume effect and relatively modest T/B of 2.4:1. Also, as T/B decreases, smaller targets become less distinguishable from background and COV theoretically approaches that of background. This is illustrated in Figure 1, which shows the small 8 mm ⁶⁸Ga target at 2.4:1 is indistinguishable from background (second row).

These results have several important clinical implications. First, T/B may be the most important independent factor when considering variability of SUV measurements. New radiotracers may have relatively low COV if the tumor uptake is very high. Our data provides supporting evidence that small lesions (8 mm) may be reproducibly measured in clinical studies if counting statistics and tumor uptake are sufficiently high. This observation is particularly important when considered with the results of other studies evaluating the detectability of small objects at high concentrations. Previous work has shown that small objects (~0.6 cm³) that cannot be observed at relatively low activity concentrations may be consistently detected at higher T/B [9,10]. When considering our results in the context of prior studies, high T/B can not only result in small objects being readily detected, but uptake metrics for small objects (8 mm) can be quantified reproducibly.

Next, while images acquired using ⁶⁸Ga radiopharmaceuticals may have reduced counts due to several factors, the overall measured COV can be sufficiently low even with shorter acquisition times, due to high T/B. The physical disadvantages of ⁶⁸Ga compared to ¹⁸F may be overcome if ⁶⁸Ga radiotracer ligand properties provide high tumor uptake.

Finally, the magnitude of the effect of imaging time on COV may be mitigated at high concentrations. As expected, decreases in imaging time were associated with in increases in COV for both SUV_max and SUV_mean. However, at high concentrations, COV for both SUV_max and SUV_mean are still quite low, even at 1 minute of imaging time, as shown in Figure 3.

Repeatability in detection and quantification of small lesions is particularly relevant for novel ⁶⁸Ga radiotracers. For example, ⁶⁸Ga PSMA has shown very high sensitivity in detection of prostate cancer early biochemical recurrence [3,13]. Even though ¹⁸F radiotracers have superior physical properties compared to ⁶⁸Ga, ⁶⁸Ga PSMA has very high affinity, is trapped and retained intracellularly, and is rapidly washed out from normal background tissue. These in vivo biological properties of ⁶⁸Ga PSMA favor high target to background ratio and have ultimately proven to be superior to the ¹⁸F FACBC amino acid analogue in patient with biochemically recurrent prostate cancer [14,15]. Biologically important properties of high target-to-background ratio translate into clinical high lesion detectability which has a significant clinical impact by changing the treatment planning in radiation therapy [2,16]. It is also important for in identifying early bone metastases where very small newly appearing lesions may indicate disease progression, despite an overall decrease in quantified disease burden [17]. In clinical practice, despite the physical limitations of ⁶⁸Ga compared to ¹⁸F, the ⁶⁸Ga PSMA and ¹⁸F DCFPyL PET radiotracers which target PSMA show overall similar lesion detectability [18-20].

Limitations

This exploratory study has several limitations. First, these studies to evaluate the COV of SUV were performed to evaluate parameters relevant to clinical imaging. The ability to determine the effects of the small 8 mm lesions, however, may be limited by the voxel size (4 × 4 × 4 mm) which is typically 64 mm³. Error in measurements could be introduced by the relatively large voxel size compared to the 8 mm target, however, these are voxel sizes used in clinical practice. To better evaluate the effect of voxel size on 8 mm and smaller targets, phantom experiments and clinical PET studies should be performed to specifically address this question. Additionally, one factor not evaluated in this study was the effect of various reconstruction parameters in terms of iterations and subsets as well as different post-reconstruction filters. While all phantoms were reconstructed with the same parameters, it should be noted that increasing the magnitude of the post-reconstruction filter would like lead to reduced COV for both radionuclides due to further smoothing.

Second, the results reported here are specific to the experimental conditions used. The phantom data include clinically relevant target sizes ranging from 8 mm to 25 mm and activities selected to reflect clinical imaging, however clinical lesion activities and sizes may fall outside of these ranges. Additionally, these data were acquired on a single modern analog PET/CT system which uses conventional photomultiplier technology. Modern digital cameras with SiPM detectors have higher sensitivity (counts per unit activity), better TOF temporal resolution, and advanced correction techniques such as depth dependent resolution recovery, and improved image reconstruction algorithms. Overall, these improvements in both hardware and software are likely to have a favorable impact on COV compared to conventional PET/CT scanners.

Third, this study was designed to reflect clinical protocols, activity concentrations, and imaging times of clinical studies, substantial differences exist in clinical practice. Physical factors such as attenuation and scatter, are much different in clinical studies, and may contribute to overall variability. Additionally, biologic variability can also have a major impact, and may be larger and less predictable compared to the reproducibility of the instrumentation. Patient biologic variability includes radiotracer uptake properties, physical lesion uptake, background activity, patient motion, physiologic variability, and other factors which are beyond the scope of this study. This study, however, evaluates the physical aspects of scanner reproducibility, and characterizes the independent effects of lesion size, target activity, and scanning time on the variability of uptake measurements.

In conclusion, the physical properties of ⁶⁸Ga, when compared to ¹⁸F, are suboptimal and may result in higher variability as measured by COV. Target size, imaging time, and T/B are independent factors contributing to COV and decreasing these parameters significantly increases the COV. While the COV is relatively high for small targets (8 mm) for both radionuclides, this can be dramatically reduced if radiotracer target uptake is very high as is commonly found in clinical ⁶⁸Ga DOTATATE scans. This has clinical implications for repeatability of SUV measurements using other novel ⁶⁸Ga radiotracers with high tumor uptake.

Disclosure of conflict of interest

None.