Application of separable parameter space techniques to multi-tracer PET compartment modeling

Abstract

Multi-tracer positron emission tomography (PET) can image two or more tracers in a single scan, characterizing multiple aspects of biological functions to provide new insights into many diseases. The technique uses dynamic imaging, resulting in time-activity curves that contain contributions from each tracer present. The process of separating and recovering separate images and/or imaging measures for each tracer requires the application of kinetic constraints, which are most commonly applied by fitting parallel compartment models for all tracers. Such multi-tracer compartment modeling presents challenging nonlinear fits in multiple dimensions. This work extends separable parameter space kinetic modeling techniques, previously developed for fitting single-tracer compartment models, to fitting multi-tracer compartment models. The multi-tracer compartment model solution equations were reformulated to maximally separate the linear and nonlinear aspects of the fitting problem, and separable least-squares techniques were applied to effectively reduce the dimensionality of the nonlinear fit. The benefits of the approach are then explored through a number of illustrative examples, including characterization of separable parameter space multi-tracer objective functions and demonstration of exhaustive search fits which guarantee the true global minimum to within arbitrary search precision. Iterative gradient-descent algorithms using Levenberg–Marquardt were also tested, demonstrating improved fitting speed and robustness as compared to corresponding fits using conventional model formulations. The proposed technique overcomes many of the challenges in fitting simultaneous multi-tracer PET compartment models.

1. Introduction

Rapid multi-tracer PET is a technique where multiple PET tracers are imaged in a single scan. Unlike multi-isotope SPECT, where energy discrimination can be used to identify photons from each tracer, all PET tracers give rise to indistinguishable 511 keV photon pairs. It has been previously shown that, when using dynamic scanning with staggered injections, the imaging signals from 2–3 PET tracers can be reliably recovered through application of appropriate kinetic constraints for each tracer (Koeppe et al 1998, Koeppe et al 2001, Converse et al 2004, Koeppe et al 2004, Kadrmas and Rust 2005, Kudomi et al 2005, Rust and Kadrmas 2006, Black et al 2008, 2009, Gao et al 2009, Joshi et al 2009, Kadrmas et al 2010, 2013, Kadrmas and Hoffman 2013). Perhaps the most robust multi-tracer PET signal-separation algorithms rely upon parallel compartment modeling of all tracers present in order to apply the kinetic constraints and recover imaging estimates from each individual tracer.

The conventional compartment model is comprised of a series of homogenous compartments driven by an input function, and where temporal exchange between compartments is governed by rate parameters and simple linear differential equations. The solutions to these equations are nonlinear and present a complex fitting environment which becomes further compounded in the presence of multiple tracers. Figure 1 presents several generic serial compartment models in order of increasing complexity, along with a shorthand nomenclature that will be used in this paper to quickly reference each generic model. The input function, b(t), drives the system; it is assumed in this work that the input function is known a priori from direct measurement or some other estimation technique. The imaging signal, R(t), typically cannot measure each compartment individually; rather, the imaging signal comprises the sum over all compartments, C(t; b(t), {k_i}), often with the addition of a vascular term due to imaging signal from whole-blood, B(t):

Figure 1.

Generic serial compartment models, each consisting of an input driving 1–3 tissue compartments in series that exchange according to the labeled rate parameters. We use a shorthand nomenclature to quickly reference each generic model as shown. For example, the ‘3K’ model refers to the model with input plus two additional compartments and three rate parameters.

$$R(t)=f_{B}B(t)+(1-f_B)C(t;b(t),\{k_i\}),$$ (1)

where f_B is the fractional contribution of B(t) to the imaging measurement. The rate parameters {k_i}, along with f_B, comprise the unknown parameters of the model to be estimated. While the differential equations are linear in these unknowns, the solution equations are nonlinear, containing weighted sums of exponentials convolved with the input function. As such, fitting compartment models to measured datasets involves a multidimensional nonlinear minimization problem.

We previously described separable parameter space techniques for fitting compartment models which reformulate the model equations to explicitly and maximally separate the linear and nonlinear aspects of the fitting problem (Kadrmas and Oktay, 2013). Separable nonlinear least-squares techniques (Lawton and Sylvestre 1971, Kaufman and Pereyra 1978, Golub and Pereyra 2003) are then used to rapidly fit the linear and nonlinear sub-problems, providing very fast and robust fits. Related approaches, such as basis pursuit or basis function methods, have previously been studied for less generalized cases (Gunn et al 1997, Reutter et al 1998, Gunn et al 2002, Boellaard et al 2005, Watabe et al 2005, Hong and Fryer 2010). The separable parameter space technique begins with a generalized reformulation of the compartment model solution equations:

$$R(t)={\kappa }_{B}B(t)+{\kappa }_b\stackrel{⌣}{b}(t)+{\kappa }_1{S}_1(t;{\upsilon }_1;b(t))+{\kappa }_2{S}_2(t;{\upsilon }_1,{\upsilon }_2;b(t)),$$ (2)

where:

$$\begin{array}{l}\stackrel{⌣}{b}(t)\equiv {\int }_0^{t}b(\tau )\mathrm{d}\tau ,\hfill \\ S_1(t;{\upsilon }_1;b(t))\equiv {\int }_0^t{\mathrm{e}}^{-{\upsilon }_1(t-\tau )}b(\tau )\mathrm{d}\tau ,\text{and}\hfill \\ S_2(t;{\upsilon }_1,{\upsilon }_2;b(t))\equiv {\int }_0^t({\mathrm{e}}^{-{\upsilon }_2(t-\tau )}-{\mathrm{e}}^{-{\upsilon }_1(t-\tau )})b(\tau )\mathrm{d}\tau .\hfill \end{array}$$

Here, reformulated parameters {κ_B, κ_b, κ₁, κ₂} and {υ₁, υ₂} have replaced the rate parameters {k_i} from the conventional compartment model formulations. Using appropriate definitions of these new parameters (described in (Kadrmas and Oktay 2013); see also table 1), equation (2) represents all of the serial compartment models shown in figure 1. Of note, this formulation is explicitly linear in {κ_B, κ_b, κ₁, κ₂} and nonlinear in {υ₁, υ₂}. These terms and parameters will be further discussed in the Theory section. The reformulation maximally separates the linear and nonlinear parameters of the models and facilitates application of the separable nonlinear least squares technique to effectively constrain the solution space to include only solutions that are least-squares in the linear sense. This approach effectively reduces the dimensionality of the nonlinear fitting problem to the smallest mathematically identical nonlinear sub-problem, thereby greatly simplifying the fit. This method was shown to provide fast and robust fits for single-tracer compartment models.

Table 1.

Comparison of dimensionality and parameters for 1K-5K compartment models.

Model	Conventional formulation	Separable parameter space reformulation
1K	2D nonlinear fit	0D Nonlinear fit (no free parameters)
	fB	2D linear sub-problem
	K1	κB = fB κb = (1 − fB)K1
2K	3D nonlinear fit	1D nonlinear fit
	fB	υ1 = k2
	K1	2D linear sub-problem
	k2	κB = fB κ1 = (1 − fB)K1
3K	4D nonlinear fit	1D nonlinear fit
	fB	υ1 = k2 + k3
	K1	3D linear sub-problem
	k2	κB = fB
	k3	κb = (1 − fB)K1k3/(k2 + k3) κ1 = (1 − fB)K1k2/(k2 + k3)
4K	5D nonlinear fit	2D nonlinear fit
	fB	${\upsilon }_1=\frac{1}{2}[k_2+k_3+k_4+\sqrt{{(k_2+k_3+k_4)}^2-4k_2{k}_4}]$
	K1	${\upsilon }_2=\frac{1}{2}[k_2+k_3+k_4-\sqrt{{(k_2+k_3+k_4)}^2-4k_2{k}_4}]$
	k2	3D linear sub-problem
	k3	κB = fB
	k4	κ1 = (1 − fB)K1 κ2 = (1 − fB)K1(υ2 − k3 − k4)/(υ2 − υ1)
5K	6D nonlinear fit	2D nonlinear fit
	fB	${\upsilon }_1=\frac{1}{2}[k_2+k_3+k_4+k_5+\sqrt{{(k_2+k_3+k_4+k_5)}^2-4(k_2{k}_4+k_2{k}_5+k_3{k}_5)}]$
	K1	${\upsilon }_2=\frac{1}{2}[k_2+k_3+k_4+k_5-\sqrt{{(k_2+k_3+k_4+k_5)}^2-4(k_2{k}_4+k_2{k}_5+k_3{k}_5)}]$
	k2	4D linear sub-problem
	k3	κB = fB
	k4	κb = (1 − fB)K1k3k/(k2k4 + k2k5 + k3k5)
	k5	κ1 = (1 − fB)K1k2(k4 + k5)/(k2k4 + k2k5 + k3k5) κ2 = (1 − fB)K1[k3 + k4 + k5 − υ2 − (k3k5/υ2)]/(υ1 − υ2)

Note: Parameters that are not listed are zero.

The current work extends the generalized separable parameter space technique to fitting multi-tracer PET compartment models. The benefits are of even greater value for multi-tracer compartment model fitting than for single-tracer, where the reduced dimensionality brings even greater returns. For example, conventional dual-tracer 3K + 3K modeling involves a 7D nonlinear fitting problem (f_B plus K₁ − k₃ for each tracer)—very difficult to fit robustly in the presence of high statistical noise. However, this is reduced to only 2 nonlinear dimensions through application of the separable parameter space reformulation, making the fit both simple and robust by enabling exhaustive searches to be quickly performed. This paper first extends the theory of the separable parameter space approach to fitting multi-tracer PET compartment models. The properties and benefits of this approach are then explored using a number of example dual-tracer PET applications where compartment models were fit using both exhaustive search and iterative Levenberg–Marquardt algorithms. Fitting performance using the separable parameter space approach is compared and contrasted with that obtained from conventional compartment model formulations with the same fitting algorithms.

2. Theory

The theory for the generalized separable parameter space technique is presented in the context of kinetic modeling in dynamic nuclear medicine imaging, e.g. positron emission tomography (PET). As such, quantities are described as tracer radioactivity concentrations. The concepts and mathematics, however, are generalizable to other dynamic imaging modalities and non-imaging applications of kinetic modeling. A glossary of symbols is provided to be used as a reference for the terms used in the model equations.

Glossary of symbols
b(t)	Input function
B(t)	Tracer concentration in whole-blood
R(t)	Modeled time-activity curve
fB	Fractional contribution of B(t) to the imaging measurement
K1, k2 − k5	Rate constants for 1K-5K serial compartment models
κB, κb, κ1, κ2	Linear parameters of the separable parameter space formulations
υ1, υ2	Nonlinear parameters of the separable parameter space formulations
b̆(t)	Temporal term (integral of b(t)), present in 1K, 3K, and 5K models
S1(t; υ1; b(t))	Temporal term present in 2K–5K compartment models
S2(t; υ1, υ2; b(t))	Temporal term present in 4K–5K compartment models

2.1. Review of single-tracer separable parameter space reformulation

The generalized separable parameter space reformulation of equation (2) has 6 degrees-of-freedom (κ_B, κ_b, κ₁, κ₂, υ₁, υ₂), and it is written to be explicitly linear in κ_B, κ_b, κ₁, and κ₂, and nonlinear in υ₁ and υ₂ (which appear as exponents in the convolutions of temporal terms S₁ and S₂). Defning κ_B, κ_b, κ₁, κ₂, υ₁ and υ₂ as listed in table 1, it can be shown that this general formulation encompasses the solution equations for well known 1K-4K serial compartment models, as well as the lesser-used 5K serial compartment model (Bertoldo et al 2001). Inspection of the 2K and 3K models (see table 1) reveals that there is inherently one convolution integral containing a single free parameter in the exponent (υ₁), resulting in one nonlinear degree-of-freedom; the remaining degrees-of-freedom are linear. Likewise, inspection of the 4K and 5K models reveals 2 inherent convolution integrals with differing exponents. The free parameters in the exponents, υ₁ and υ₂, are independent parameters: υ₁ cannot be written in terms of (κ_B, κ_b, κ₁, κ₂, υ₂), and likewise υ₂ cannot be written in terms of (κ_B, κ_b, κ₁, κ₂, υ₁). As such, two separate free parameters are required to represent these convolution integrals, and these models inherently have two nonlinear degrees-of-freedom. Since the generalized separable parameter space reformulation of equation (2) is written explicitly to have the minimum number of nonlinear free parameters (1 for 2K-3K models, 2 for 4K-5K models), and likewise the maximum number of linear free parameters (the κs), this formulation can be considered to maximally separate the linear and nonlinear parameters. The linear sub-problem can then be solved analytically, which effectively reduces the dimensionality of the nonlinear fitting problem as compared to the conventional approach (shown in table 1).

After completing the fit, the kinetic rate parameters K₁, k₂ − k₅ are easily calculated from the best-fit rate parameters κ_B, κ_b, κ₁, κ₂, υ₁, and υ₂ (table 2). The generalized separable parameter space technique was shown to provide fast and robust compartment model fits. It was also found to provide identical time activity curves as the conventional compartment model formulation. The generalized separable parameter space reformulation of (Kadrmas and Oktay 2013) was originally developed for single-tracer compartment modeling; however, it is not limited to single-tracers. In this work, we extend the approach to parallel compartment modeling of multiple tracers.

Table 2.

Recovery of kinetic rate parameters.

Model	fB	K1	k2	k3	k4	k5
1K	κB	$\frac{{\kappa }_1}{(1-f_B)}$	0	0	0	0
2K	κB	$\frac{{\kappa }_1}{(1-f_B)}$	υ1	0	0	0
3K	κB	$\frac{{\kappa }_b+{\kappa }_1}{(1-f_B)}$	$\frac{{\kappa }_1{\upsilon }_1}{{\kappa }_b+{\kappa }_1}$	$\frac{{\kappa }_b{\upsilon }_1}{{\kappa }_b+{\kappa }_1}$	0	0
4K	κB	$\frac{{\kappa }_1}{(1-f_B)}$	$\frac{({\kappa }_1-{\kappa }_2){\upsilon }_1+{\kappa }_2{\upsilon }_2}{{\kappa }_1}$	$\frac{{\kappa }_2({\kappa }_1-{\kappa }_2){({\upsilon }_2-{\upsilon }_1)}^2}{{\kappa }_1[({\kappa }_1-{\kappa }_2){\upsilon }_1+{\kappa }_2{\upsilon }_2]}$	$\frac{{\kappa }_1{\upsilon }_1{\upsilon }_2}{({\kappa }_1-{\kappa }_2){\upsilon }_1+{\kappa }_2{\upsilon }_2}$	0
5K	κB	$\frac{{\kappa }_b+{\kappa }_1}{(1-f_B)}$	$\frac{({\kappa }_1-{\kappa }_2){\upsilon }_1+{\kappa }_2{\upsilon }_2}{{\kappa }_b+{\kappa }_1}$	$\{\frac{{\kappa }_2({\upsilon }_1-{\upsilon }_2)+{\kappa }_b{\upsilon }_1}{({\kappa }_b+{\kappa }_1)}+\frac{{\kappa }_2{\upsilon }_2({\upsilon }_2-{\upsilon }_1)}{({\kappa }_1-{\kappa }_2){\upsilon }_1+{\kappa }_2{\upsilon }_2}\}$	$\{\frac{{\kappa }_1{\upsilon }_1{\upsilon }_2}{({\kappa }_1-{\kappa }_2){\upsilon }_1+{\kappa }_2{\upsilon }_2}-\frac{{\kappa }_b{\upsilon }_1{\upsilon }_2}{k_3({\kappa }_b+{\kappa }_1)}\}$	$\frac{{\kappa }_b{\upsilon }_1{\upsilon }_2}{k_3({\kappa }_b+{\kappa }_1)}$

2.2. Generalized multi-tracer separable parameter space reformulation

Parallel multi-tracer compartment modeling involves fitting a model similar to that shown in equation (1) with the form:

$$R^{(\text{Multi})}(t)=f_B{B}^{(\text{Multi})}(t)+(1-f_B)\sum _{n=1}^N{C}^{(n)}(t;b^{(n)}(t),\{k_i^{(n)}\}),$$ (3)

where B^(Multi)(t) is the total tracer concentration in whole-blood (including all tracers), b⁽ⁿ⁾(t) is the input function for tracer n, $\{k_i^{(n)}\}$ are the rate parameters for tracer n, and N is the number of tracers present. As written above, $C^{(n)}(t;b^{(n)}(t),\{k_i^{(n)}\})$ is the modeled activity for tracer n in the extravascular tissue compartments. We first reformulate the multi-tracer equation to maximally separate the linear and nonlinear parameters of the models. Here, the reformulated linear and nonlinear parameters for tracer n are denoted by ${\kappa }_i^{(n)}\text{and}{\upsilon }_i^{(n)}$, respectively. This gives rise to the generalized reformulated multi-tracer modeling equation:

$$R{(t)}^{(\text{Multi})}={\kappa }_{B}B{(t)}^{(\text{Multi})}+\sum _{n=1}^N[{{\kappa }_b}^{(n)}\stackrel{⌣}{b}{(t)}^{(n)}+{\kappa }_1^{(n)}{S}_1^{(n)}(t;{\upsilon }_1^{(n)};b{(t)}^{(n)})+{\kappa }_2^{(n)}{S}_2^{(n)}(t;{\upsilon }_1^{(n)},{\upsilon }_2^{(n)};b{(t)}^{(n)})],$$ (4)

where $S_1^{(n)}(t;{\upsilon }_1^{(n)};b^{(n)})\text{and}{S}_2^{(n)}(t;{\upsilon }_1^{(n)},{\upsilon }_2^{(n)};b^{(n)})$ are nonlinear temporal terms for each tracer n, analogous to nonlinear terms shown in equation (2). With appropriate definitions of the κs and υs (table 1), the generalized formulation becomes a precise mathematical representation of multi-tracer compartment modeling.

The model fitting problem amounts to finding the value of these parameters which minimize some objective function. We consider the weighted sum square error (WSSE) objective function in this work. Since measured data are generally discrete samples in time for dynamic imaging, the WSSE can be written in the form:

$$\text{WSSE}=\sum _{j=1}^T{w}_j{[{\kappa }_B{B}_j^{(\text{Multi})}+\sum _{n=1}^N[{{\kappa }_b}^{(n)}{\stackrel{⌣}{b}}_j^{(n)}+{\kappa }_1^{(n)}{S}_{1,j}^{(n)}({\upsilon }_1^{(n)};b^{(n)})+{\kappa }_2^{(n)}{S}_{2,j}^{(n)}({\upsilon }_1^{(n)},{\upsilon }_2^{(n)};b^{(n)})]-{\stackrel{\sim }{R}}_j]}^2,$$ (5)

where T is the number of discrete samples in time, w_j are the weights for each time sample j, and R̃_j is the measured activity at time j. The separable nonlinear least-squares technique is then applied—effectively solving for the linear parameters (κs) and writing them in terms of the nonlinear parameters (υs) and known quantities. The kinetic rate parameters for each individual tracer can then be calculated for the best-fit solution as listed in table 2.

2.3. Theoretical benefits of multi-tracer separable parameter space reformulation

The separable parameter space model formulation applied to multi-tracer compartment modeling has various benefits as compared to both the single-tracer separable parameter space and the conventional model formulations. Notably, the proposed reformulation includes 1−2 nonlinear free parameters $({\upsilon }_i^{(n)})$ and 2–4 linear parameters $({\kappa }_i^{(n)})$ for each tracer n. The linear parameters can be solved directly for each iteration of the nonlinear fit. As a result, the dimensionality of the nonlinear multi-tracer fitting problem is greatly reduced (see table 3). For example, fitting a dual-tracer 3K + 3K model requires a 7D fit in the conventional formulation (f_B plus K₁ − k₃ for each tracer), but reduces to only a 2D nonlinear fit using separable parameter space techniques.

Table 3.

Comparison of nonlinear fit dimensionality.

Multi-tracer model	Conventional formulation	Separable parameter space reformulation
2K + 2K	5D	2D
3K + 3K	7D	2D
3K + 4K	8D	3D
4K + 4K	9D	4D
5K + 5K	11D	4D

2.4. Fitting algorithms for the nonlinear sub-problem

One of the major advantages of the separable parameter space reformulation is that the nonlinear fitting problem can be solved using any appropriate minimization algorithm. After completing the fit, the kinetic rate parameters K₁, k₂ − k₅ for each tracer n are easily calculated (table 2) from the best-fit reformulated parameters κ_B, κ_b, κ₁, κ₂, υ₁ and υ₂ for the individual tracers. In this work, we consider two fitting algorithms for the nonlinear sub-problem. The first, exhaustive search, samples the entire nonlinear solution space (within appropriate parameter ranges) to arbitrary precision. This brute-force approach guarantees identifcation of the global minimum within the selected search precision and parameter ranges. Since the dimensionality of the separable parameter space nonlinear fit is reduced compared to the conventional approach, exhaustive search becomes computationally feasible. The second algorithm, Levenberg−Marquardt, is a ‘fast’ iterative fitting algorithm based on local gradients. This algorithm, like all such gradient-descent fitting algorithms, has the potential of being trapped by local minima as well as converging or diverging outside of the boundary conditions. Fits using Levenberg−Marquardt are sensitive to initial conditions and can vary in the number of iterations required (Press et al 1988). The degree and extent of these confounding factors for iterative nonlinear minimization are compared and contrasted for both the conventional and separable parameter space formulations for multi-tracer compartment modeling in this work.

3. Methods

3.1. Test datasets

The benefits and limitations of the separable parameter space approach for multi-tracer model fitting were explored through a series of illustrative examples. Here, three sets of representative multi-tracer PET data were retrospectively selected from ongoing investigator-initiated trials at the University of Utah performed with Informed Consent under protocols approved by the university Institutional Review Board. These trials were designed to evaluate recovery of single-tracer measures from combined multi-tracer data; as such, it was important to have knowledge of the actual individual-tracer data components. This was done by acquiring separate, single-tracer scans with each tracer. The PET imaging data were then combined, with appropriate shifts in time, in order to ‘emulate’ single-scan dual-tracer measurements. This approach has been widely used for multi-tracer PET research, e.g. (Kadrmas et al 2013), and provides dual-tracer images with exactly paired single-tracer components that can be used as gold standard for evaluating dual-tracer processing algorithms.

The first test dataset used in this work represented single-scan rest-stress ¹³N-ammonia myocardial perfusion imaging for one of the patients described in (Rust et al 2006). The data correspond to a 20 min duration dynamic PET scan with 20 mCi ¹³N-ammonia administered at rest at time 0 min, with stress induced via adenosine infusion from 7 to 13 min, and with a second tracer administration of 20 mCi was performed near peak stress at 10 min. The input function was obtained from a region-of-interest (ROI) drawn in the left ventricle, including correction for circulating labeled metabolites (Rosenspire et al 1990, Bormans et al 1995). The left ventricle myocardium was segmented into 17 ROIs, and time-activity curves were generated for each region. A dual-tracer 3K + 3K compartment model was applied and fit to the time-activity curve data. Here, the rest and stress tracer administrations were treated as separate ‘tracers’, where the kinetic parameters for rest and stress correspond to each ‘tracer’ administration. This is an approximation of the underlying physiologic conditions, where in actuality the kinetics change over time in response to the stress agent; however, the approximate is accurate for tracers such as ammonia which experience high first pass extraction and strong tracer retention from the rest administration before stress takes effect.

The second dataset represented dual-tracer ¹⁸F-fluorothymidine (FLT) + ¹⁸F-fluorodeo xyglucose (FDG) imaging of a patient with non-small cell lung cancer. Here, the dynamic dual-tracer PET data spanned a duration of 92 min, with 5 mCi FLT administered at time 0 min. and 7.5 mCi FDG administered at 32 min. Whole-blood and plasma input functions were obtained from ‘arterialized’ heated-hand venous blood sampling (Copeland et al 1992), and the FLT input function was metabolite corrected by the method of (Shields et al 2005). The images showed three malignant tumors in the lungs, and ROIs were drawn over each tumor to obtain time-activity curve data. A dual-tracer 3K + 3K compartment model was fit to these data.

The third dataset represented dual-tracer FLT + ¹¹C-acetate (ACE) imaging of a patient with a primary brain tumor (glioblastoma). Here, the dynamic PET data covered 64 min., with 5 mCi FLT administered at time 0 min. and 15 mCi ACE administered at 16 min. Input functions were obtained as just described for the lung cancer dataset. Five abnormal regions of tracer uptake were noted, and ROIs were drawn over each and used to obtain time-activity curve data. A dual-tracer 3K + 4K compartment model was used for the dual-tracer fits. Overall, the data from these three subjects provide a variety of patient data, tracer combinations, injection times, and kinetic models that is representative of the variable and complex challenges found in kinetic modeling of multi-tracer PET images.

3.2. Implementation of fitting algorithms

All fitting algorithms were implemented in C using custom software written by the authors. Here a single software package was used for both the conventional and separable parameter space model formulations, including both the exhaustive search and iterative Levenberg– Marquardt algorithms. Pre-compiler directives were used to select code specific to each individual algorithm, and all operations shared between algorithms used the same lines of code. For example, all convolution integrals and sum-squared error calculations common to all algorithms used the same code. This practice ensures that any differences in fits arose from differences in the algorithms themselves, not from differences in implementation. The code was validated through comparisons with both RFIT (Lawrence Berkeley National Laboratory) and PMOD (PMOD Technologies, Zurich Switzerland). The software included user-selectable options for constraining parameter ranges, and the linear sub-problem for the separable parameter space methods was implemented with the non-negativity constraint described in the appendix of Kadrmas and Oktay (2013).

3.3. Exhaustive search fits and characterization of objective functions

For each case, exhaustive search fits were first performed using the separable parameter space formulation. Here the search sampled υ₁ from 0.0 to 4.0 min⁻¹ for both rest and stress in the cardiac rest/stress ammonia example, and for the other examples υ₁ was sampled from 0.0 to 1.0 min⁻¹ for both tracers. All cases used 1000 evenly spaced samples for the exhaustive search. These fits demonstrate how the reduced-dimensionality of the new approach makes such exhaustive searches computationally feasible, and they also provide a means for identifying the true global-minimum fit for each test case. CPU times for each exhaustive search fit were computed, as well as corresponding projected times for exhaustive search fits using the conventional model formulations. Since the exhaustive search algorithm completely samples the solution space to arbitrary precision, it also provides a convenient means of characterizing the WSSE objective function by computing and storing the value at each sampled point in the parameter space. For the 3K + 3K test cases, where the separable parameter space WSSE nonlinear objective function is 2D, the objective functions were plotted and analyzed in terms of fitting topology, structure, and presence/absence of local minima. Objective functions for the 3K + 4K test case and all conventional model formulations are not shown in this paper due to complexities in visualizing fitting topology in more than two dimensions.

3.4. Iterative fits

Iterative Levenberg–Marquardt fits were also performed for each test case using both the conventional and reformulated model equations. In order to test dependency upon initial conditions, 2 sets of fits were run repeatedly for each test case with different initial rate parameter values. In the first set, each fit was run repeatedly with 25 sets of initial rate parameter values. Here, the initial conditions were selected so that the corresponding υ values for each tracer were evenly sampled across the parameter spaces, i.e. forming a 5 × 5 grid spanning the nonlinear solution space (see, for example, figures 4 and 5). Here, the values of the individual rate parameters were randomly selected within the constraints imposed upon the corresponding υ values just described. This provided sets of initial conditions which stochastically and broadly encompassed the parameter spaces. In the second set of fits, each fit was repeated 1000 times with initial values randomly distributed across the parameter spaces. These fits provided a thorough sampling of different initial conditions and how they affected iterative fit performance.

Figure 4.

Example Levenberg–Marquardt fits using both the conventional (top-left) and separable parameter space (bottom-left) model formulations for 3K + 3K cardiac rest-stress ammonia. Each curve was fit using 25 different random initial conditions. The separable parameter space WSSE objective function, plotted next to both plots, illustrate the initial conditions (blue) and minimum found (green: global minimum; red: local minima). The conventional Levenberg–Marquardt fits converged to the global minimum 17 times out 25, whereas the separable parameter space Levenberg– Marquardt converged to the global minimum given any of the 25 initial conditions.

Figure 5.

Example Levenberg–Marquardt fits using both the conventional (top-left) and separable parameter space (bottom-left) model formulations for 3K + 3K FLT + FDG multi-tracer PET. Each curve was fit using 25 different random initial conditions. The separable parameter space WSSE objective function, plotted next to both plots, illustrate the initial conditions (blue) and minimum found (green: global minimum; red: local minima). The conventional Levenberg–Marquardt fits converged to the global minimum 14 times out 25, whereas the separable parameter space Levenberg– Marquardt converged to the global minimum given any of the 25 initial conditions.

Each iterative fit was run to convergence, where the stopping criterion was defined by a reduction in WSSE of less than 0.01% on 2 successive iterations, with a minimum of 5 iterations and maximum of 100,000 iterations (never reached for these data). The true global minimum fit was determined from the exhaustive search fits (guaranteed theoretically to provide the global minimum fit to within the search precision, and verified empirically in (Kadrmas and Oktay 2013)). For each test case, the number of fits that converged to the global minimum were counted, as well as the number of fits that were trapped in local minima or diverged out of the parameter ranges. The number of iterations and CPU times (single thread fitting time on 2.8 GHz Intel Xeon X5660 processor, no parallelization) for each fit were also recorded.

4. Results

4.1. Exhaustive search separable parameter space fits

Figure 2 shows representative dual-tracer time-activity curves and compartment model fits for each of the three test datasets. The kinetic rate parameters for each model were recovered from the best-fit separable parameter space solutions and are listed in the figure, as well as the net influx parameter (K_i) for FDG. In order to provide a brief comparison of dual-tracer versus single-tracer fit results, the corresponding rate parameters from single-tracer fits to the constituent individual-tracer component time-activity curves are also shown. The exhaustive search fit results closely match the measured data, and provide the global minimum fits for each of these test cases. The CPU times for each fit are provided in section 4.4, along with projected times for exhaustive search fits using the conventional model formulations. These data demonstrate that the separable parameter space reformulation makes exhaustive search computationally feasible due to the reduced dimensionality of the nonlinear fitting problem, providing a robust means of obtaining the true global optimum compartment model fit for multi-tracer datasets.

Figure 2.

Example dual-tracer time activity curves showing separable parameter space exhaustive search fits for three different models: 3K + 3K cardiac rest/stress ¹³N-ammonia (top), 3K + 3K lung tumor FLT + FDG (middle), and 3K + 4K brain tumor FLT + ACE (bottom). Best-fit kinetic rate parameters for each case are also shown, in units of min⁻¹, along with corresponding best-fit single-tracer rate parameters from the constituent single-tracer time-activity curves.

4.2. Characterization of separable parameter space wsse objective functions

The values of the WSSE objective functions were recorded for each step of the separable parameter space exhaustive search fits, thereby characterizing the nonlinear fitting spaces for each case. Figure 3 shows representative 3K + 3K separable parameter space WSSE objective functions for the cardiac and lung tumor datasets. The objective functions were generally well-behaved, showing a well-defined global minimum and few if any local minima. There is some evidence that dual-tracer objective functions may be more complex than corresponding single-tracer objective functions—for example, the vertical shelf in the cardiac case, which we have not observed for single-tracer data. These characterizations suggest that WSSE objective functions for the separable parameter space formulation for multi-tracer PET are considerably less complex than the corresponding (higher-dimensional) conventional model formulations, providing a simpler and potentially more robust environment for fast iterative nonlinear fitting.

Figure 3.

Contour plots of the separable parameter space WSSE objective functions for cardiac ammonia rest/stress imaging with 3K + 3K compartment model (left) and lung tumor FLT + FDG imaging with 3K + 3K compartment model (right). The grayscale represents WSSE values, and the contour lines are drawn at regular intervals. The objective functions are generally well-behaved, contain a well-defined global minimum, and do not show complex topological features such as local minima which could confound gradient-descent iterative fitting algorithms.

4.3. Iterative nonlinear fits with Levenberg–Marquardt

Iterative fits were also performed for each of the test cases using the Levenberg–Marquardt algorithm with both the conventional and separable parameter space model formulations. Here, each iterative fit was first repeated 25 times over a grid of initial conditions as described in the methods, and then each fit was repeated 1000 times with random initial conditions spanning the parameter space. The result of each fit was compared against the exhaustive search fit results in order to determine how many fits correctly found the global minimum. Results for representative ROIs for each test case are summarized in figures 4–6.

Figure 6.

Example Levenberg–Marquardt fits using both the conventional (top) and separable parameter space (bottom) model formulations for the 3K + 4K dual-tracer compartment model example. Each curve was fit using 25 different random initial conditions. The conventional Levenberg–Marquardt fits converged to the global minimum for only 1 of the 25 sets of initial conditions, whereas the separable parameter space Levenberg–Marquardt fits converged to the global minimum for all sets of initial conditions tested.

Representative results for rest-stress cardiac PET case with 3K + 3K model are shown in figure 4. The iterative fits with conventional model formulation correctly found the global minimum for 17 of the gridded initial conditions in this example, but either diverged or got trapped in local minima for 8 of the initial conditions. The separable parameter space fits, however, correctly found the global minimum for all 25 initial conditions. Considering the full population of fits over 17 myocardial segments with 1000 initial conditions each, the conventional Levenberg–Marquardt fits successfully reached the global minimum in 82.8 ± 19.4 (mean ± SD over ROIs) percent of tries, whereas the success rate for the separable parameter space fits was 98.9 ± 2.9%.

The dual-tracer FLT + FDG lung cancer 3K + 3K example provided a somewhat more challenging fitting problem, as shown in figure 5. Here, 14 of 25 fits with gridded initial conditions correctly found the global minimum with the conventional model formulation, whereas all 25 of the corresponding separable parameter space fits converged to the true global minimum. The fits for all three ROIs run with 1000 random initial conditions resulted in a success rate of 66.5 ± 23.9% for the conventional fits, whereas the separable parameter space fits converged to the global minimum in 100% of cases with no failures.

The brain tumor FLT + ACE dataset provided the most challenging fitting problem studied, as the 3K + 4K compartment model had more degrees of freedom than the previous 3K + 3K examples. In this case, only 1 of the 25 gridded initial conditions for the conventional model formulation reached the true global minimum, and all others were trapped in local minima or diverged. However, the separate parameter space fits again correctly found the global minimum for all 25 sets of initial conditions. These results are summarized in figure 6 (WSSE objective function not shown for this case, since it is 3D for the 3K + 4K model and difficult to visualize). Considering the fits with 1000 random initial conditions for all 5 ROIs, the conventional model formulation reached the global minimum in 46.8 ± 8.9% of tries, as compared to 99.1 ± 1.3% for the separable parameter space iterative fits.

These data show that Levenberg–Marquardt multi-tracer fitting with the separable parameter space reformulation was robust, correctly finding the global minimum regardless of the initial conditions in the large majority test cases studied. In contrast, the fits with the conventional model formulation often failed by converging to local minima or diverging to parameter boundaries, and the performance varied widely for the three cases. It should also be noted that multi-tracer PET fits depend upon tracer combinations, injection order and timing, and noise properties of the images; the results found here are representative of typical imaging cases, but cannot be guaranteed to apply for all multi-tracer PET imaging scenarios.

4.4. Number of iterations and fitting times

Table 4 shows the number of iterations and CPU fitting times (mean ± SD) for each of the test datasets for both the conventional and separable parameter space model formulations. The separable parameter space approach enabled exhaustive search fits to be performed in approximately 3 s for the 3K + 3K models, and in approximately 30 min. for the 3K + 4K model. These times were measured using a single-CPU with no parallelization, and marked improvements would be expected using standard multi-threading or GPU processing techniques. In contrast, exhaustive search fitting times for the conventional model formulation would require many years of CPU time for each fit, and are not computationally feasible for routine use. As such, the separable parameter space technique makes exhaustive search feasible for routine use—a highly significant result considering that no previous technique could provide robust, high confidence multi-tracer PET compartment model fits without excessive computational cost.

Table 4.

Number of iterations and fitting times.

	3K + 3K Ammonia		3K + 3K FLT + FDG		3K + 4K FLT + ACE
Model	Number of iterations	CPU time (ms)	Number of iterations	CPU time (ms)	Number of iterations	CPU time (ms)
Exhaustive search
Projected conventional formulation	1021	∼1018	1021	∼1018	1024	∼1021
Separable parameter space	106	3114	106	2987	109	1749206
Levenberg–Marquardt
Conventional formulation	519 ± 321	557 ± 1984	5246 ± 11424	2299 ± 4804	1551 ± 8008	565 ± 2588
Separable parameter space	27 ± 8	7.68 ± 2.13	18 ± 4	18.96 ± 3.39	62 ± 12	48.91 ± 9.32

Significant improvements in iterative fitting times were also observed when using separable parameter space Levenberg–Marquardt as compared to conventional Levenberg–Marquardt. Here, the number of iterations and fitting times were reduced by 1–2 orders of magnitude when using the separable parameter space approach. This is a direct result of the reduced dimensionality of the nonlinear fitting problem, coupled with the less-complex fitting topology as discussed in the previous sections of this paper. Overall, the new technique provided robust iterative fits within about 8–19 ms and 49 ms for 3K + 3K and 3K + 4K models, respectively. Considering the corresponding improvements in fitting robustness discussed above, the new technique provides substantial improvements in both robustness and speed as compared to conventional approaches.

4.5. Feasibility of voxelwise fits for parametric imaging

The fitting times attained with the separable parameter space technique are fast enough, at least for 3K + 3K models, to make fitting of dual-tracer compartment models to individual voxels for parametric imaging computationally feasible. Such fits also provide a more demanding fitting application than fitting ROI time-activity curves as studied throughout the rest of this paper—noise levels are much higher for individual voxels than for sizeable ROIs, background tissues often have much lower tracer uptake than target tissues, and different tissues may experience varying tracer kinetics which may not be consistent with using the same compartment model throughout the image. These issues were briefly explored by performing voxelwise fits to the dynamic rest-stress ¹³N-ammonia test dataset. The fits were performed on reoriented short-axis slices, comprising a 96 × 96 × 48 image array with cubic 1.64 mm voxels. Two sets of voxelwise fits were performed: (i) fitting all voxels in the left ventricle myocardium, where a binary mask was used to select only image voxels in the myocardium; and (ii) fitting all voxels with significant tracer uptake in the reoriented short-axis images, including e.g. neighboring soft tissue background and liver voxels.

Fits were performed for each case using both the exhaustive search and Levenberg–Marquardt (LM) algorithms, where the exhaustive search fits were used to determine whether or not the LM fits reached the global minimum versus converging to local minima. The exhaustive search fits were coded to re-use integrals for each sample point when possible, markedly reducing the computational requirements for performing a large number of fits with the same input function. The initial conditions for each LM fit were arbitrarily set to values typical for myocardial blood flows $(K_1^{\text{Re}\mathit{st}}=1.0\text{ml}{min}^{-1}{\mathrm{g}}^{-1},k_2^{\text{Re}\mathit{st}}=1.0{min}^{-1},k_3^{\text{Re}\mathit{st}}=0.05{min}^{-1},K_1^{\mathit{\text{Stress}}}=3.5\text{ml}{min}^{-1}{\mathrm{g}}^{-1},k_2^{\text{Stress}}=3.5{min}^{-1},\text{and}{k}_3^{\text{Stress}}=0.05{min}^{-1})$, providing initial values appropriate for the voxels in the myocardium but far from the expected values in the background tissues. All other aspects of the fits were the same as previously described for the ROI-based fits.

A total of 31 106 voxels were fit in the myocardium, requiring 772 s and 204 s for exhaustive search and Levenberg–Marquardt, respectively (single-thread CPU time). The LM fits required substantially more iterations (71 ± 197 iterations) than for the ROI fits (27 ± 8 iterations, table 4), and 96.5% of the iterative LM fits reached the global minimum. These results indicate a high degree of fitting success, but also indicate that local minima were present. The presence of these local minima and increased fitting times reflect the more challenging fitting application due, in large part, to the high noise levels in the time-activity curves for individual voxels. The full-image test fit a total of 168 338 voxels, requiring 4,664s and 2322s for exhaustive search and Levenberg–Marquardt, respectively. Here, 83.9% of the LM fits reached the global minimum, suggesting that the differing kinetics and lower tracer uptake in non-myocardial background voxels also increased the complexity of the fits. The fitting times measured here were for single-threaded code. Since fitting multiple voxels is highly receptive to parallelization (each fit is independent and can be run in parallel), total fitting times for parametric imaging could likely be reduced to a few minutes or less on modern multi-core workstations.

5. Summary and conclusions

In this work we've extended the theory of the separable parameter space technique to simultaneous fitting of 1K-5K serial compartment models to temporally overlapping multi-tracer PET data, and explored the properties and benefits of the approach using several representative multi-tracer PET fitting examples. As for single-tracer fitting, the technique effectively reduces the dimensionality of the nonlinear fitting space, providing much faster and more robust fits than corresponding fits using conventional model formulations. These benefits are of even greater importance for multi-tracer fitting, where the very large and complex fitting topology of the conventional model formulations present a very challenging fitting environment which is substantially simplified through application of the separable parameter space technique. The reduced dimensionality of the nonlinear fitting space makes exhaustive search for multi-tracer compartment models computationally feasible—guaranteeing identification of the true global minimum to within the selected search precision. Nonlinear fitting topology for the separable parameter space formulation were also found to be well behaved for the example cases studied, having few local minima or confounding structures within typical parameter ranges. These properties were confirmed through tests of iterative gradient-descent fits using the Levenberg–Marquardt algorithm, which provided both fast and robust fits to dual-tracer time-activity curves when using the separable parameter space approach. The feasibility of performing dual-tracer fits to individual voxels for parametric imaging was also demonstrated, though the incidence of local minima was higher for low-count individual voxel data as compared to ROI data. In conclusion, the separable parameter space technique has been extended to fitting multi-tracer PET compartment models, providing a promising method for obtaining fast and accurate fits for this challenging problem.