Generalized separable parameter space techniques for fitting 1K-5K serial compartment models

Abstract

Purpose: Kinetic modeling is widely used to analyze dynamic imaging data, estimating kinetic parameters that quantify functional or physiologic processes in vivo. Typical kinetic models give rise to nonlinear solution equations in multiple dimensions, presenting a complex fitting environment. This work generalizes previously described separable nonlinear least-squares techniques for fitting serial compartment models with up to three tissue compartments and five rate parameters.

Methods: The approach maximally separates the linear and nonlinear aspects of the modeling equations, using a formulation modified from previous basis function methods to avoid a potential mathematical degeneracy. A fast and robust algorithm for solving the linear subproblem with full user-defined constraints is also presented. The generalized separable parameter space technique effectively reduces the dimensionality of the nonlinear fitting problem to one dimension for 2K-3K compartment models, and to two dimensions for 4K-5K models.

Results: Exhaustive search fits, which guarantee identification of the true global minimum fit, required approximately 10 ms for 2K-3K and 1.1 s for 4K-5K models, respectively. The technique is also amenable to fast gradient-descent iterative fitting algorithms, where the reduced dimensionality offers improved convergence properties. The objective function for the separable parameter space nonlinear subproblem was characterized and found to be generally well-behaved with a well-defined global minimum. Separable parameter space fits with the Levenberg-Marquardt algorithm required fewer iterations than comparable fits for conventional model formulations, averaging 1 and 7 ms for 2K-3K and 4K-5K models, respectively. Sensitivity to initial conditions was likewise reduced.

Conclusions: The separable parameter space techniques described herein generalize previously described techniques to encompass 1K-5K compartment models, enable robust solution of the linear subproblem with full user-defined constraints, and are amenable to rapid and robust fitting using iterative gradient-descent type algorithms.

INTRODUCTION

Dynamic imaging techniques can measure and characterize temporal changes in imaging signals for many modalities, providing more indepth functional information than static images that provide a snapshot or time-average over the acquisition period. Analysis of dynamic images usually involves applying a kinetic model that describes the temporal behavior of image voxels or regions in terms of a set of unknown scalar parameters. Many types of kinetic models exist, most involving a combination of scalar coefficients that weight temporal terms. These temporal terms are functions of time and may also depend on additional scalar parameters. The archetypal example kinetic model is the compartment model,^{1, 2, 3, 4} which comprises a series of homogenous compartments driven by an input function, and where temporal exchange between compartments is governed by rate parameters and simple linear differential equations. The solutions to such equations, however, are nonlinear and present a complex fitting environment. Figure 1 presents several generic serial compartment models in order of increasing complexity, along with a shorthand nomenclature that will be used in this paper to quickly reference each generic model. This paper is focused on the mathematical problem of fitting compartment model equations to dynamic imaging data in order to estimate best-fit kinetic rate parameters for a predetermined fitting criterion (e.g., weighted least-squares). Other related issues, such as measurement of the input function or selection of fitting weights, fall outside the scope of this work and are not discussed.

Figure 1.

Generic serial compartment models, each consisting of an input driving 1–3 tissue compartments in series that exchange according to the labeled rate parameters. We use a shorthand nomenclature to quickly reference each generic model as shown. For example, the “3K” model refers to the model with input plus two additional compartments and three rate parameters.

Exchange of tracer between connected compartments is governed by linear differential equations of the form

$$\frac{d{C}_i}{dt}=\sum k_{\mathrm{in}}{C}_{\mathrm{in}}-\left(\sum k_{\mathrm{out}}\right)C_i,$$ (1)

where the subscripts in and out refer to rate parameters that describe exchange into (from compartment C_in) or out of compartment C_i, respectively. The input function, b(t), drives the system; it is assumed in this work that the input function is known a priori from direct measurement or some other estimation technique. We also assume that all tracer activity concentrations are decay corrected, although the effects of radioactive decay could also easily be included in the modeling equations with a few simple modifications. The imaging measurement R(t) typically cannot measure each compartment individually; rather, the imaging signal comprises the sum over all compartments, C(t; b(t), {k_i}), often with the addition of a vascular term due to imaging signal from whole-blood, B(t),

$$R\left(t\right)=f_{B}B\left(t\right)+\left(1-f_B\right)C\left(t;b\left(t\right),\left\{k_i\right\}\right),$$ (2)

where f_B is the fractional contribution of B(t) to the imaging measurement. The rate parameters {k_i}, along with f_B, comprise the unknown parameters of the model to be estimated. While the differential equations are linear in these unknowns, the solution equations are nonlinear, containing weighted sums of exponentials convolved with the input function. As such, fitting compartment models to measured datasets involves a nonlinear minimization problem.

A large body of work has been performed in addressing the compartment model fitting problem. Perhaps the most robust—but also most computationally demanding—approaches for estimating individual rate parameters for multicompartment models involve nonlinear least-squares estimation (NLLS) in a variety of forms. Numerous nonlinear curve-fitting algorithms have been investigated, including gradient-based algorithms such as Levenberg-Marquardt,^{5, 6} ridge-regression,^{7, 8, 9, 10, 11} and simulated annealing methods.^{12, 13, 14, 15} When using conventional compartment model formulations, the multidimensional fitting space can be large and complex, and it often contains local minima and/or broad shallow valleys that slow or confound iterative minimization routines. The result is sensitive to initial conditions, and one can never be certain that the true global optimum has been reached.

One of the most promising approaches to improving robustness and fitting speed is to reformulate the compartment model solution equations into separable linear and nonlinear components prior to fitting. Separable nonlinear least-square techniques^{16, 17, 18, 19} can then be used, either directly or indirectly, to simplify and accelerate the fitting problem. This approach has been used for years in fitting 2K compartment models,^{9, 20, 21, 22, 23, 24} and more recently has been extended to 3K and 4K models.^{25, 26, 27, 28} The nonlinear components of the reformulated equations have largely been interpreted as forming basis functions for the kinetic model, and the fitting algorithm searches among these bases to identify (weighted sums of) curves that best-match the measured data. The best-fit basis functions and linear weighting parameters are then used to compute corresponding kinetic rate parameters (and/or macroparameters) of the compartment model. We collectively refer to these approaches as basis function methods throughout this paper.

Basis function methods bring the advantage of simplifying the nonlinear compartment model fitting problem by effectively separating the nonlinear aspects of the fit (i.e., identification of the best-fit basis functions) from the linear aspects (i.e., finding best-fit scalar weights for a given set of basis functions). However, current basis function methods have a few limitations. The basis function formulations for 4K models exhibit a potential degeneracy in the nonlinear parameters of the fit (see Sec. 2A below), which can lead to indeterminate results. Similarly, full implementation of boundary conditions and user-definable constraints on kinetic rate parameter estimates has not yet been described. Moreover, the algorithms employed for identifying the best-fit basis functions have largely been based on exhaustive search or simplex strategies, and fast iterative gradient-descent algorithms have not been broadly applied in this context.

In this work, we describe an alternate mathematical formulation of basis function methods that generalizes separable parameter space techniques for serial compartment models with up to three tissue compartments and five rate parameters. The alternate formulation avoids a potential degeneracy that may arise for previously described basis function formulations. A theoretical perspective of interpreting the resulting equations as effectively reducing the dimensionality of the solution space of the fitting problem is then discussed, followed by a description of a rigorous method for solving the linear subproblem with full user-defined constraints. The reformulated modeling equations are validated against conventional model formulations, and we demonstrate that both formulations provide identical time-activity curves and have the same least-squares global minimum. The application of fast iterative gradient descent algorithms for solving the nonlinear subproblem is then explored by characterizing separable parameter space objective functions. Finally, the performance of both exhaustive search and the iterative Levenberg-Marquardt algorithms are studied for separable parameter space fits and compared to fits using the conventional model formulations.

THEORY

Without loss of generality, the theory for the generalized separable parameter space modeling formulation is presented in the context of kinetic modeling in dynamic nuclear medicine imaging, e.g., positron emission tomography (PET) and single-photon emission computed tomography (SPECT). As such, quantities are described as tracer radioactivity concentrations. The concepts and mathematics, however, are generalizable to other dynamic imaging modalities and nonimaging applications of kinetic modeling.

Generalized kinetic modeling formulation

Let κ_B, κ_b, κ₁, κ₂, υ₁, and υ₂ be scalar parameters that will be further described below, and let B(t) be the whole-blood activity concentration. Further, define the following temporal terms that depend on the input function, b(t), as well as scalar parameters υ₁ and υ₂,

$$\stackrel{⌣}{b}\left(t\right)\equiv {\int }_0^{t}b\left(\tau \right)d\tau ,$$ (3)

$$S_1\left(t;{\upsilon }_1;b\left(t\right)\right)\equiv {\int }_0^t{e}^{-{\upsilon }_1\left(t-\tau \right)}b\left(\tau \right)d\tau ,$$ (4)

$$\begin{array}{c}\hfill S_2\left(t;{\upsilon }_1,{\upsilon }_2;b\left(t\right)\right)\equiv {\int }_0^t\left(e^{-{\upsilon }_2\left(t-\tau \right)}-e^{-{\upsilon }_1\left(t-\tau \right)}\right)b\left(\tau \right)d\tau .\end{array}$$ (5)

Note that the temporal terms S₁(t; υ₁; b(t)) and S₂(t; υ₁, υ₂; b(t)) are convolutions of exponentials with the input function. We can now write a generalized modeling equation

$$\begin{array}{ccc}\hfill R\left(t\right)& =& {\kappa }_{B}B\left(t\right)+{\kappa }_b\stackrel{⌣}{b}\left(t\right)+{\kappa }_1{S}_1\left(t;{\upsilon }_1;b\left(t\right)\right)\hfill \\ & & +{\kappa }_2{S}_2\left(t;{\upsilon }_1,{\upsilon }_2;b\left(t\right)\right),\hfill \end{array}$$ (6)

which is linear in {κ_B, κ_b, κ₁, κ₂} and nonlinear in {υ₁, υ₂}. Defining κ_B, κ_b, κ₁, κ₂, υ₁, and υ₂ as listed in Tables 1, 2, it can easily be shown that this general formulation encompasses the solution equations for well known 1K-4K serial compartment models,^{1, 2, 29, 30} as well as the lesser-used 5K serial compartment model.³¹

Table 1.

Nonlinear parameter definitions for 1K-5K compartment models.

Model	υ1	υ2	Constraint
1K	0	0	n/a
2K	k2	0	0 < υ1
3K	k2 + k3	0	0 < υ1
4K	$\frac{1}{2}\left\{\left(k_2+k_3+k_4\right)+\sqrt{{\left(k_2+k_3+k_4\right)}^2-4k_2{k}_4}\right\}$	$\frac{1}{2}\left\{\left(k_2+k_3+k_4\right)-\sqrt{{\left(k_2+k_3+k_4\right)}^2-4k_2{k}_4}\right\}$	0 < υ2 < υ1
5K	$\frac{1}{2}\left\{\begin{array}{c}\left(k_2+k_3+k_4+k_5\right)+\hfill \\ \sqrt{{\left(k_2+k_3+k_4+k_5\right)}^2-4\left(k_2{k}_4+k_2{k}_5+k_3{k}_5\right)}\hfill \end{array}\right\}$	$\frac{1}{2}\left\{\begin{array}{c}\left(k_2+k_3+k_4+k_5\right)-\hfill \\ \sqrt{{\left(k_2+k_3+k_4+k_5\right)}^2-4\left(k_2{k}_4+k_2{k}_5+k_3{k}_5\right)}\hfill \end{array}\right\}$	0 < υ2 < υ1

Table 2.

Linear parameter definitions for 1K-5K compartment models.

Model	κB	κb	κ1	κ2	Constraints
1K	fB	(1 − fB)K1	0	0	0 ⩽ κB ⩽ 1; 0 < κb
2K	fB	0	(1 − fB)K1	0	0 ⩽ κB ⩽ 1; 0 < κ1
3K	fB	$\frac{\left(1-f_B\right)K_1{k}_3}{k_2+k_3}$	$\frac{\left(1-f_B\right)K_1{k}_2}{k_2+k_3}$	0	0 ⩽ κB ⩽ 1; 0 < κb; 0 < κ1
4K	fB	0	(1 − fB)K1	$\frac{\left(1-f_B\right)K_1}{\left({\upsilon }_2-{\upsilon }_1\right)}\left({\upsilon }_2-k_3-k_4\right)$	0 ⩽ κB ⩽ 1; 0 < κ2 < κ1
5K	fB	$\frac{\left(1-f_B\right)K_1{k}_3{k}_5}{k_2{k}_4+k_2{k}_5+k_3{k}_5}$	$\frac{\left(1-f_B\right)K_1{k}_2\left(k_4+k_5\right)}{k_2{k}_4+k_2{k}_5+k_3{k}_5}$	$\frac{\left(1-f_B\right)K_1}{\left({\upsilon }_2-{\upsilon }_2\right)}\left[\left(k_3+k_4+k_5-{\upsilon }_2\right)-\frac{k_3{k}_5}{{\upsilon }_2}\right]$	0 ⩽ κB ⩽ 1; 0 < κb; 0 < κ2 < κ1

We refer to Eq. 6 as the “generalized separable parameter space formulation.” For 2K models, the formulation presented above is equivalent to that previously used with the application of separable nonlinear least-squares for these models.²⁰ Similarly, for 3K models it is identical to the basis function for plasma input compartment (BAFPIC) method of Hong et al.²⁸ The proposed generalized formulation extends the approach to include the 5K compartment model, and for ≥4K models, there is a subtle difference as compared to previously described basis function methods.²⁷ The difference lies in our definition of the S₂ term, which in previous approaches was defined as $S_2\left(t;{\upsilon }_2;b\left(t\right)\right)\equiv {\int }_0^t{e}^{-{\upsilon }_2\left(t-\tau \right)}b\left(\tau \right)d\tau$. Subtracting $e^{-{\upsilon }_{1}t}$ in the S₂ term as in Eq. 5 ensures that S₁ and S₂ are determinant and distinguishable (i.e., S₁ ≠ S₂ for all nonzero values of υ_{1, 2}), which avoids the potential degeneracy in the previous approach that could arise for υ₁ ≈ υ₂. Further, our formulation of S₂ immediately leads to the constraint 0 < υ₂ < υ₁ in order to ensure positivity for this temporal term, which is required in order to limit Eq. 6 to represent valid compartment model solutions. Similarly, the special case of υ₁ = υ₂ in our formulation implies that both k₄ and k₅ must be zero in order to represent a valid compartment model, which corresponds to simplification to a 3K model (see below). Analogous constraints and boundary conditions could be applied when using previously described basis function formulations,²⁷ for example, by implementing a constraint to ensure that υ₁ and υ₂ differ by some minimum value. However, we find that the description and interpretation of these cases may be more straightforward when using the formulation described in Eq. 5.

The constraints listed in Tables 1, 2 limit the generalized formulation to valid compartment model solutions with positivity constraints on the conventional kinetic rate parameters K₁, k₂ − k₅. For 4K and 5K models, the constraint 0 < υ₂ < υ₁ is equivalent to requiring k₂ to be positive and k₃ and k₄ to be non-negative, and 0 < κ₂ < κ₁ ensures positivity for k₂ − k₄. Of note, we have used positivity constraints here (except for the vascular term κ_B) rather than non-negativity constraints, thus the constraints preclude solutions with rate parameters equal to zero. Solutions where a rate parameter is zero are encompassed via model simplification, e.g., a 4K model with k₄ = 0 simplifies to a 3K model. This approach also ensures that the modeling formulation does not attempt to compute inestimable rate parameters. For example, if k₃ = 0, then no information is available regarding k₄ and k₅ (as there is no tracer exchange with the unobservable compartment), and the estimable model reduces to a 2K model.

Construction of the WSSE objective function

For dynamic imaging, the measured data are generally discrete samples in time. Depending on the modality and other factors, two discretization schemes are commonly encountered—instantaneous samples, and samples averaged over time-frame durations. For each case we define the following discretizations, where j = 1…M represents the time (time-frame) index:

Instantaneous samples:

$$\begin{array}{c}\hfill \begin{array}{c}{R}_j\equiv R\left(t_j\right)B_j\equiv B\left(t_j\right){\stackrel{⌣}{b}}_j\equiv \stackrel{⌣}{b}\left(t_j\right)\hfill \\ S_{1,j}\left({\upsilon }_1;b\right)\equiv S_1\left(t_j;{\upsilon }_1;b\right)S_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right)\equiv S_2\left(t_j;{\upsilon }_1,{\upsilon }_2;b\right).\hfill \end{array}\end{array}$$ (7)

Averaged over time-frame durations:

$$\begin{array}{ccc}& & R_j\equiv \frac{1}{\left(t_{j2}-t_{j1}\right)}{\int }_{t_{j1}}^{t_{j2}}R\left(\tau \right)d\tau \hfill \\ & & B_j\equiv \frac{1}{\left(t_{j2}-t_{j1}\right)}{\int }_{t_{j1}}^{t_{j2}}B\left(\tau \right)d\tau \hfill \\ & & {\stackrel{⌣}{b}}_j\equiv \frac{1}{\left(t_{j2}-t_{j1}\right)}{\int }_{t_{j1}}^{t_{j2}}\stackrel{⌣}{b}\left(\tau \right)d\tau \hfill \\ & & S_{1,j}\left({\upsilon }_1;b\right)\equiv \frac{1}{\left(t_{j2}-t_{j1}\right)}{\int }_{t_{j1}}^{t_{j2}}{S}_1\left(\tau ;{\upsilon }_1;b\right)d\tau \hfill \\ & & S_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right)\equiv \frac{1}{\left(t_{j2}-t_{j1}\right)}{\int }_{t_{j1}}^{t_{j2}}{S}_2\left(\tau ;{\upsilon }_1,{\upsilon }_2;b\right)d\tau .\hfill \end{array}$$ (8)

In addition, we distinguish between modeled values ${\widehat{R}}_j$ and noisy measurements ${\tilde{R}}_j$ using a carat and tilde, respectively.

Using either discretization, the generalized kinetic model at each time sample j is written

$$\begin{array}{c}\hfill {\widehat{R}}_j={\kappa }_B{B}_j+{\kappa }_b{\stackrel{⌣}{b}}_j+{\kappa }_1{S}_{1,j}\left({\upsilon }_1;b\right)+{\kappa }_2{S}_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right).\end{array}$$ (9)

The model fitting problem amounts to finding the values of the parameters κ_B, κ_b, κ₁, κ₂, υ₁, and υ₂ that minimize some objective function. We consider the weighted least-squares (WLS) criterion, and write the weighted sum-squared error objective function as

$$\begin{array}{ccc}\hfill WSSE& =& \sum _{j=1}^T{w}_j{\left({\widehat{R}}_j-{\tilde{R}}_j\right)}^2\hfill \\ & =& \sum _{j=1}^T{w}_j[{\kappa }_B{B}_j+{\kappa }_b{\stackrel{⌣}{b}}_j+{\kappa }_1{S}_{1,j}\left({\upsilon }_1;b\right)\hfill \\ & & +{\kappa }_2{S}_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right)-{{\tilde{R}}_j]}^2,\hfill \end{array}$$ (10)

where w_j are the weights for each time sample j.

Application of separable nonlinear least-squares

The generalized formulation of Eq. 6 has 6 degrees-of-freedom (κ_B, κ_b, κ₁, κ₂, υ₁, υ₂), as does the conventional model formulation for the 5K model (f_B, K₁, k₂ − k₅); however, the generalized formulation is written to be explicitly linear in κ_B, κ_b, κ₁, and κ₂, and nonlinear in υ₁ and υ₂ (which appear as exponents in the convolutions of temporal terms S₁ and S₂). Inspection of the 2K and 3K models (see, e.g., Tables 1, 2) reveals that there is inherently one convolution integral containing a single free parameter in the exponent (υ₁), resulting in one nonlinear degree-of-freedom; the remaining degrees-of-freedom are linear. Likewise, inspection of the 4K and 5K models reveals two inherent convolution integrals with differing exponents. The free parameters in the exponents, υ₁ and υ₂, are independent parameters: υ₁ cannot be written in terms of (κ_B, κ_b, κ₁, κ₂, υ₂), and likewise υ₂ cannot be written in terms of (κ_B, κ_b, κ₁, κ₂, υ₁). As such, two separate free parameters are required to represent these convolution integrals, and these models inherently have two nonlinear degrees-of-freedom. Since the generalized formulation of Eq. 6 is written explicitly to have the minimum number of nonlinear free parameters (one for 2K-3K models, two for 4K-5K models), and likewise the maximum number of linear free parameters (the κs), this formulation can be considered to maximally separate the linear and nonlinear parameters.

Separating the linear and nonlinear parameters in this manner provides for the full benefit of separable nonlinear least-squares techniques.^{16, 17, 18, 19} The fitting problem is essentially separated into two problems: a nonlinear fit in (υ₁, υ₂), coupled simultaneously with a linear fit in (κ_B, κ_b, κ₁, κ₂). Previous basis function methods have largely used exhaustive search or downhill simplex algorithms to search among the (υ₁, υ₂) “basis functions,” analytically solving the linear subproblem to find the best-fit (κ_B, κ_b, κ₁, κ₂) for each point (υ₁, υ₂) encountered along the way. In this section, we show that the linear subproblem can also be directly incorporated into the nonlinear fit, providing a “Reduced Parameter Space Reformulation” with interesting theoretical perspectives that illustrate the value of separable parameter space fitting techniques.

Consider the linear subproblem that arises at each iteration/step of the nonlinear fit: finding (κ_B, κ_b, κ₁, κ₂) that minimizes WSSE given the current value of (υ₁, υ₂). For the moment, we ignore constraints on the κs and consider the unconstrained linear subproblem. Taking the derivative of WSSE [Eq. 10] with respect to each linear parameter and setting the result to zero, we obtain the linear subproblem

$$\mathbf{Ax}=\mathbf{b},$$

where

$$\begin{array}{ccc}\hfill \mathbf{A}& =& \left(\begin{array}{cccc}\hfill \sum _{j=1}^T{w}_j{B}_j^2& \hfill \sum _{j=1}^T{w}_j{B}_j{\stackrel{⌣}{b}}_j& \hfill \sum _{j=1}^T{w}_j{S}_{1,j}\left({\upsilon }_1;b\right)B_j& \sum _{j=1}^T{w}_j{S}_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right)B_j\\ \hfill .& \hfill \sum _{j=1}^T{w}_j{\stackrel{⌣}{b}}_j^2& \hfill \sum _{j=1}^T{w}_j{S}_{1,j}\left({\upsilon }_1;b\right){\stackrel{⌣}{b}}_j& \sum _{j=1}^T{w}_j{S}_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right){\stackrel{⌣}{b}}_j\\ \hfill .& \hfill .& \hfill \sum _{j=1}^T{w}_j{S}_{1,j}^2\left({\upsilon }_1;b\right)& \sum _{j=1}^T{w}_j{S}_{1,j}\left({\upsilon }_1;b\right)S_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right)\\ \hfill .& \hfill .& \hfill .& \sum _{j=1}^T{w}_j{S}_{2,j}^2\left({\upsilon }_1,{\upsilon }_2;b\right)\end{array}\right)\hfill \\ \hfill \mathbf{x}& =& \left(\begin{array}{c}{\kappa }_B\\ {\kappa }_b\\ {\kappa }_1\\ {\kappa }_2\end{array}\right),\mathbf{b}=\left(\begin{array}{c}\sum _{j=1}^T{w}_j{\tilde{R}}_j{B}_j\\ \sum _{j=1}^T{w}_j{\tilde{R}}_j{\stackrel{⌣}{b}}_j\\ \sum _{j=1}^T{w}_j{\tilde{R}}_j{S}_{1,j}\left({\upsilon }_1;b\right)\\ \sum _{j=1}^T{w}_j{\tilde{R}}_j{S}_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right)\end{array}\right),\hfill \end{array}$$ (11)

and “.” denotes symmetric matrix elements.

The conventional approach, as used with previously described basis function methods, is to analytically invert the linear subproblem ($\widehat{\mathbf{x}}={\mathbf{A}}^{-1}\mathbf{b}$) to directly compute the best-fit κs that minimize WSSE given the current values of υ₁ and υ₂. The estimated κs are then fed back to the nonlinear fitting algorithm, and the fit in (υ₁, υ₂) progresses accordingly.

Consider an alternative approach, however, where the inverted linear subproblem (which computes best-fit κs as functions of υ₁, υ₂, and measured/known quantities) is substituted back into the original modeling Eq. 6. The result is a reformulated modeling equation, which we call the Unconstrained Generalized Reduced Parameter Space Reformulation, which effectively removes the linear parameters from the model:

Unconstrained generalized reduced parameter space reformulation:

$$\begin{array}{ccc}\hfill {\widehat{R}}_j& =& {\left({\left(\begin{array}{cccc}\sum _{j=1}^T{w}_j{B}_j^2& \sum _{j=1}^T{w}_j{B}_j{\stackrel{⌣}{b}}_j& \sum _{j=1}^T{w}_j{S}_{1,j}\left({\upsilon }_1;b\right)B_j& \sum _{j=1}^T{w}_j{S}_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right)B_j\\ .& \sum _{j=1}^T{w}_j{\stackrel{⌣}{b}}_j^2& \sum _{j=1}^T{w}_j{S}_{1,j}\left({\upsilon }_1;b\right){\stackrel{⌣}{b}}_j& \sum _{j=1}^T{w}_j{S}_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right){\stackrel{⌣}{b}}_j\\ .& .& \sum _{j=1}^T{w}_j{S}_{1,j}^2\left({\upsilon }_1;b\right)& \sum _{j=1}^T{w}_j{S}_{1,j}\left({\upsilon }_1;b\right)S_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right)\\ .& .& .& \sum _{j=1}^T{w}_j{S}_{2,j}^2\left({\upsilon }_1,{\upsilon }_2;b\right)\end{array}\right)}^{-1}\left(\begin{array}{c}\sum _{j=1}^T{w}_j{\tilde{R}}_j{B}_j\\ \sum _{j=1}^T{w}_j{\tilde{R}}_j{\stackrel{⌣}{b}}_j\\ \sum _{j=1}^T{w}_j{\tilde{R}}_j{S}_{1,j}\left({\upsilon }_1;b\right)\\ \sum _{j=1}^T{w}_j{\tilde{R}}_j{S}_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right)\end{array}\right)\right)}^T\hfill \\ & & \times \left(\begin{array}{c}{B}_j\\ {\stackrel{⌣}{b}}_j\\ S_{1,j}\left({\upsilon }_1;b\right)\\ S_{2,j}\left({\upsilon }_1,{\upsilon }_2;b\right)\end{array}\right).\hfill \end{array}$$ (12)

In essence, the reformulated model equation has been constrained to only permit solutions that minimize the WSSE objective function in the linear sense. Inspection of each term in Eq. 12 reveals that the reformulated model depends on υ₁, υ₂, and the measurements; however, no linear free parameters (κs) remain. One interpretation of this is that the application of separable nonlinear least-squares can be considered to constrain the solution space to only include solutions that minimize the objective function in the linear sense, effectively reducing the dimensionality of the fit to only the nonlinear free parameters. More specifically, the dimensionality of the “reduced” fitting space for 1K-5K serial compartment models becomes:

Model	Conventional formulation	Reduced parameter space reformulation
1K	2D (fB, K1)	0D (no free parameters)
2K	3D (fB, K1, k2)	1D (υ1)
3K	4D (fB, K1, k2 − k3)	1D (υ1)
4K	5D (fB, K1, k2 − k4)	2D (υ1, υ2)
5K	6D (fB, K1, k2 − k5)	2D (υ1, υ2)

While of limited practical interest (due to difficulty in constraining the κs with this approach), this theoretical perspective illustrates the value of separable parameter space fitting techniques (including basis function methods). The effective dimensionality of the nonlinear fitting problem is greatly reduced, simplifying and speeding the fit. Of note, any iterative minimization procedure employed to solve the nonlinear subproblem would act effectively the same regardless of whether it uses Eq. 11 or 12, except for approaches which implement constraints on the κs.

Constraining fitted parameters in the linear subproblem

The constraints listed in Tables 1, 2 ensure that the generalized separable parameter space formulation [Eq. 6] represents valid compartment models with positivity constraints on all kinetic parameters (plus 0 ⩽ f_B ⩽ 1). In practice, one may well wish to impose additional constraints on parameter estimates for the particular application at hand. In either case, constraints on the nonlinear parameters (υ₁, υ₂) can be imposed in the standard fashion for whichever nonlinear fitting algorithm is employed. Implementing constraints on the linear subproblem, however, is less straightforward. Direct inversion of the linear subproblem of Eq. 11 does not account for any constraints on the κs, and the direct analytical solution to the linear subproblem is unconstrained. Ad hoc approaches to implementing a certain level of constraint for the linear subproblem have been proposed;³² however, these approaches do not guarantee that the resulting fitted parameters minimize the objective function within the parameter ranges allowed.

The Appendix describes a pseudo-code algorithm for solving the linear subproblem with full user-defined constraints on the estimated parameter values (κs). The algorithm first computes the direct inversion solution. If any parameter is out of range, then an exhaustive search over that parameter is performed, analytically computing the direct inverse for the remaining parameters at each step. When more than one parameter goes out of range, nested loops are used to exhaustively search among those parameters whose analytical solution is out of range while computing analytical solutions for those parameters remaining in range. While this algorithm can require a large number of steps (i.e., when exhaustive searches are required for more than one κ), each step is extremely fast as no new convolution integrals need be performed and many of the sums can be reused for each step. The resulting computational cost per step is orders-of-magnitude lower than the cost per step (or iteration) of the nonlinear subproblem [where changing υ₁ and/or υ₂ requires recalculation of convolution integral(s)].

One may also desire to apply physiologic equality constraints to certain parameters, e.g., setting K₁/k₂ to some fixed value known a priori. A number of different such scenarios could arise for different tracers and modeling applications, and discussion of each falls outside the scope of this work. However, separable parameter space fitting approaches could still be used by applying the equality constraint to the original modeling equations and rederiving the separable parameter space formulation. Depending on the equality constraint(s) applied, the resultant formulations may have fewer linear and/or nonlinear free parameters than shown in this work for the generalized formulations.

Fitting algorithms for the nonlinear subproblem

Two fitting algorithms for solving the nonlinear subproblem were used in this work. The first algorithm, exhaustive search, is equivalent to the basis function search algorithms used with much of the prior work on basis function modeling methods (with some slight differences in the details of implementation). In some sense, this algorithm can be considered as searching among the temporal terms (“basis functions”) that correspond to each nonlinear model parameter (υ₁, υ₂). Taking a somewhat different perspective, one can also treat the nonlinear parameters themselves as parameters of the fit and apply iterative gradient-descent fitting approaches. In this work, we study the Levenberg-Marquardt algorithm for this purpose.

Exhaustive search algorithm

The dimensionality of the separable parameter space nonlinear subproblem—only one or two free parameters for 2K-3K and 4K-5K compartment models, respectively—is small enough that the entire solution space (within appropriate parameter ranges) can be exhaustively searched to an arbitrary precision with a reasonable amount of computational effort. Such brute-force exhaustive search guarantees identification of the global minimum to within the selected search precision and parameter ranges. As such, exhaustive search provides the most robust fit for the given data, weights, and input functions. In this work, we implemented exhaustive search routines in C for solving the nonlinear subproblem (finding best-fit υ₁ and υ₂). The routine loops over υ₁ from 0.0 to 2.0 min⁻¹ in 1000 steps, sampling υ₁ at a precision of 0.002 min⁻¹. Here, we chose uniformly spaced sampling in order to simplify discussion of the precision of the algorithm; for practical application, logarithmic spacing would likely be more appropriate as used in most previous implementations of these algorithms. For the 4K and 5K models, a nested loop over υ₂ was likewise included for each value of υ₁. Recalling that υ₂ should be constrained less than υ₁ in order to represent a valid compartment model, this inner loop ranged from 0.0 up to the current value of υ₁ in steps of 0.002 min⁻¹. This resulted in 1000 total steps for the 2K-3K models, and ∼500 000 steps for the 4K-5K models. For comparison, consider that exhaustive search with conventional compartment model formulations would require 10⁹, 10¹², and 10¹⁵ steps for 2K, 3K, and 4K models, respectively—becoming computationally intractable.

Levenberg-Marquardt algorithm

Nonlinear minimization with the Levenberg-Marquardt algorithm was also implemented in C for both the generalized separable parameter space formulation and for conventional 1K-5K compartment model formulations. Single-threaded coding was used with no parallelization. Close attention was paid to code optimization for both model formulations, and all concurrent routines applicable to both algorithms were shared in order to match the implementations as closely as possible and to ensure valid comparisons between the two implementations could be performed. For the conventional implementation, the vascular fraction (f_B) and all applicable rate parameters (K₁, k₂ − k₅) were estimated by the fitting algorithm, whereas only the nonlinear parameters υ₁, υ₂ were fit for the separable parameter space implementation. The algorithm was implemented as described in Press et al.,³³ where the second derivative term of the Hessian matrix was ignored. Iterations were stopped after a successful update when the fractional change in WSSE reached 0.0001 or less, indicating convergence to a (possibly local) minimum. The magnitude of the update vector was bound to fall within the range [0.01,1.00] in order to avoid wild steps as well as to reduce the number of steps required to traverse a shallow valley of complicated topology. Various initial conditions were used for the evaluation studies as described in Sec. 4.

Recovery of individual rate parameters K₁-k₅

After completing the fit, the kinetic rate parameters K₁, k₂ − k₅, as well as f_B, are easily calculated from the best-fit parameters κ_B, κ_b, κ₁, κ₂, υ₁, and υ₂. The calculation does merit some discussion of parameter estimability, however. Table 3 provides conversions for the 1K-5K serial compartment models. In all cases f_B = κ_B. When macro parameters such as the volume of distribution or net influx are desired, they can either be computed using the individual rate parameters, or more directly in some cases (e.g., the net influx parameter for the 3K model is more directly calculated as K_i = κ_b/(1 − f_B)).

Table 3.

Recovery of kinetic rate parameters.

Model	fB	K1	k2	k3	k4	k5
1K	κB	$\frac{{\kappa }_1}{\left(1-f_B\right)}$	0	0	0	0
2K	κB	$\frac{{\kappa }_1}{\left(1-f_B\right)}$	υ1	0	0	0
3K	κB	$\frac{{\kappa }_b+{\kappa }_1}{\left(1-f_B\right)}$	$\frac{{\kappa }_1{\upsilon }_1}{{\kappa }_b+{\kappa }_1}$	$\frac{{\kappa }_b{\upsilon }_1}{{\kappa }_b+{\kappa }_1}$	0	0
4K	κB	$\frac{{\kappa }_1}{\left(1-f_B\right)}$	$\frac{\left({\kappa }_1-{\kappa }_2\right){\upsilon }_1+{\kappa }_2{\upsilon }_2}{{\kappa }_1}$	$\frac{{\kappa }_2\left({\kappa }_1-{\kappa }_2\right){\left({\upsilon }_2-{\upsilon }_1\right)}^2}{{\kappa }_1\left[\left({\kappa }_1-{\kappa }_2\right){\upsilon }_1+{\kappa }_2{\upsilon }_2\right]}$	$\frac{{\kappa }_1{\upsilon }_1{\upsilon }_2}{\left({\kappa }_1-{\kappa }_2\right){\upsilon }_1+{\kappa }_2{\upsilon }_2}$	0
5K	κB	$\frac{{\kappa }_b+{\kappa }_1}{\left(1-f_B\right)}$	$\frac{\left({\kappa }_1-{\kappa }_2\right){\upsilon }_1+{\kappa }_2{\upsilon }_2}{{\kappa }_b+{\kappa }_1}$	$\left\{\begin{array}{c}\frac{{\kappa }_2\left({\upsilon }_1-{\upsilon }_2\right)+{\kappa }_b{\upsilon }_1}{\left({\kappa }_b+{\kappa }_1\right)}+\hfill \\ \frac{{\kappa }_2{\upsilon }_2\left({\upsilon }_2-{\upsilon }_1\right)}{\left({\kappa }_1-{\kappa }_2\right){\upsilon }_1+{\kappa }_2{\upsilon }_2}\hfill \end{array}\right\}$	$\left\{\begin{array}{c}\frac{{\kappa }_1{\upsilon }_1{\upsilon }_2}{\left({\kappa }_1-{\kappa }_2\right){\upsilon }_1+{\kappa }_2{\upsilon }_2}-\\ \frac{{\kappa }_b{\upsilon }_1{\upsilon }_2}{k_3\left({\kappa }_b+{\kappa }_1\right)}\end{array}\right\}$	$\frac{{\kappa }_b{\upsilon }_1{\upsilon }_2}{k_3\left({\kappa }_b+{\kappa }_1\right)}$

The above equations provide one-to-one correspondence between any set of separable parameter space variables and a set of conventional rate parameters, provided that divide-by-zero singularities are not encountered. The majority of such potential singularities are avoided through application of the constraints listed in Tables 1, 2, which concurrently ensure that the generalized modeling equations represent valid 1K-5K compartment models. The remaining potential singularities are avoided through model simplification (as described in Sec. 2A). For example, consider the potential singularity of f_B = 1.0, affecting all K₁ calculations. If the best-fit value of f_B is 1.0, this means the best-fit time-activity curve is pure whole-blood and there is no extravascular tissue uptake. In that case, K₁ is zero and all higher order rate parameters (k₂ − k₅) are undefined and not meaningful (one could consider this a “0K” model).

EVALUATION METHODS

Separable parameter space modeling techniques, which are generalized in this work, have already been thoroughly evaluated in the research literature for both 2K models^{9, 20, 21, 22, 23, 24} and more complex models with a basis function methods.^{25, 26, 27, 28} As such, the evaluation provided in this work is designed to first broadly characterize separable parameter space model formulations vs conventional compartment model formulations, and then to evaluate the applicability and receptiveness of the generalized separable parameter space formulations to fast gradient-descent fitting algorithms. Sections 3A, 3B, 3C, 3D describe the evaluations performed in this work. The emphasis of this evaluation is focused on 2K, 3K, and 4K serial compartment models, as these represent the most commonly used compartment models in dynamic PET imaging and cover both 1D (3K) and 2D (4K) separable parameter space solution equations.

Simulation of time-activity curves

A set of 24 typical input functions and whole-blood curves were selected from dynamic PET exams using ¹⁵O-water (2K model), ¹⁸F-fluorodeoxyglucose (FDG; 3K model), and ¹¹C-acetate (4K model) in patients with malignant solid tumors at our institution. Populations of kinetic parameters were then generated with a pseudorandom number generator to be randomly distributed within the following ranges: 0.001 ⩽ f_B ⩽ 0.2, 0.02 ⩽ K₁ ⩽ 0.5, 0.02 ⩽ k₂ ⩽ 0.7, 0.002 ⩽ k₃ ⩽ 0.4, and 0.002 ⩽ k₄ ⩽ 0.4. Here, each parameter was independently and uniformly distributed over the given range except for k₂, which was set according to a uniform distribution of 0.7 ⩽ K₁/k₂ ⩽ 1.1. These distributions were not designed to represent any particular tracer or target organ, but rather were set to cover a broad range of values that might be encountered in a variety of applications. The input functions and rate parameter ranges were used to simulate populations of 200–1000 time-activity curves each for the 2K, 3K, and 4K models using the conventional model formulations. The number of curves in each population was heuristically selected to include enough datapoints such that “outlier” points (representing best-fit differences between the separable parameter space and conventional model fits—see, e.g., results Figs. 67) presented themselves for study. Progressive temporal sampling schemes were used, for example, 18 × 10, 6 × 20, 4 × 30, 5 × 60, 5 × 120, 4 × 300, 3 × 600 s for a total duration of 72 m for the ¹⁸F 3K model simulations.

Figure 6.

Scatter plots of the 3K model WSSE objective function, macroparameters, and individual rate parameters comparing conventional versus separable parameter space fit results for noisy populations of time-activity curves. In each case, results for three different numbers of iterations are shown for the conventional fits with simulated annealing, demonstrating convergence effects as progressively higher numbers of iterations produced successively better fits (lower WSSE) for some curves. Note that in all cases the separable parameter space fits with exhaustive search produced equal or lower WSSE than the conventional model fits to within the numerical precision of the search (approximately 1 part in 10⁻³). As a population, the conventional model fits approached the separable parameter space fits as the number of iterations increased.

Figure 7.

Scatter plots of the 4K model WSSE objective function, macroparameters, and individual rate parameters (f_B not shown) comparing conventional vs separable parameter space fit results for noisy populations of time-activity curves. The conventional model results are shown for 10⁵, 10⁶, and 10⁷ iterations of simulated annealing, showing convergence effects similar to those shown in Fig. 6 for the 3K model (but at 10× more iterations due to the increased complexity of the 4K model). Again, the separable parameter space exhaustive search fits provided the lowest WSSE for all cases to within the precision of the search, and the conventional model fits progressive convergence toward these global minima across the full population of curves with increasing iteration.

Noise-free curves were first simulated for each case. Statistical noise was then added using a noise model that approximates the noise characteristics of iteratively reconstructed PET images. Here, Gaussian deviates were simulated for each datapoint with variance inversely proportional to the time-frame durations, and scaled to five different noise levels. When a noisy sample went negative, however, the absolute value was used—providing an asymmetric non-negative distribution roughly approximating the statistical distribution of iteratively reconstructed PET values. The resulting noisy curves had Gaussian-like non-negative statistics that varied with time-frame duration, providing more realistic test cases than simulating curves that were consistent with either Gaussian or Poisson noise. The noise model did not account for other factors, such as differences due to radioactive decay, deadtime, randoms rates, or reconstruction algorithm, that would be encountered in practice. Note that this noise model is inconsistent with the L2-norm criterion typically used in model fitting (which essentially assumes a Gaussian noise model); however, such inconsistency is a real and present challenge for actual dynamic PET data. Five different noise levels were used, covering the approximate range encountered in the dynamic PET datasets—ranging from high noise in individual voxels to lower noise curves representative of larger tumor ROIs.

Characterization of separable parameter space WSSE objective functions

Given the reduction in the number of free parameters offered by the separable parameter space formulations, the objective functions reside within 1D (2K-3K models) or 2D (4K-5K models) solution spaces. Thus, they can be plotted and analyzed much more easily than conventional objective functions in three to six dimensions. Using the exhaustive search routines just described, the WSSE objective functions were characterized for a number of noisy 3K-5K time-activity curves by storing and plotting the WSSE at each value of υ₁ (3K) or υ₁, υ₂ (4K-5K). The resulting objective functions were analyzed for the presence/absence of local minima or other confounding structures that could affect the performance of curve fits using the separable parameter space formulations with various minimization algorithms.

Characterization of global minimums for both conventional and separable parameter space formulations

In order for the separable parameter space formulations to be a viable alternative for kinetic modeling, the reformulated modeling equations must provide the same global minimum and best-fit rate parameters as the conventional model equations. In order to perform this study, fits to both model formulations need to correctly reach the true global minimums. Identification of the true global minimum is not straightforward, however, as commonly used fitting algorithms may be sensitive to initial conditions and converge to local minima (see Sec. 4D, for example, data regarding convergence for the Levenberg-Marquardt algorithm for conventional model fits). To overcome these shortcomings, the simulated annealing algorithm^{13, 14, 15, 33} was selected for the conventional model fits. As implemented, the simulated annealing algorithm was independent of starting conditions (aside from the set parameter ranges used for all fits in this paper) and included means for escaping local minima and progressing toward the true global minimum provided that enough iterations and a slow enough relaxation schedule were used. We progressively applied simulated annealing with 10⁴–10⁸ iterations, increasing the number of iterations by 10× each step and observing the number of cases in each population where the best-fit solution did not change versus those that continued moving toward a better fit. While this approach does not guarantee that the true global minimum was reached for all fits (only exhaustive search could so guarantee), it did allow visualization of the progression of the population of fits toward the true global minima as will be shown in Figs. 67 below. As such, the simulated annealing algorithm was used with the conventional model formulations in order to identify the true global minimum fits for the simulated population studies. The exhaustive search algorithm was used to identify the global minimum fits for the separable parameter space formulations, where the reduced dimensionality of the separable parameter space nonlinear subproblem makes such exhaustive search computationally feasible (guaranteeing identification of the true global minimum to within the search range and precision for all cases).

Performance of Levenberg-Marquardt algorithm

The performance of the Levenberg-Marquardt algorithm for solving the nonlinear subproblem of the separable parameter space formulations was studied and compared to fitting performance of the same algorithm for conventional compartment model formulations. Iterative fitting performance was evaluated in terms of convergence to the true global minimum (as opposed to local minima), sensitivity to initial values, and number of iterations and fitting time required to reach convergence. Here the populations of noisy time-activity curves for 2K-4K models described above were fit again using the Levenberg-Marquardt algorithm with both model formulations. Three sets of fits were performed with differing initial conditions. First, each curve was fit repeatedly using 100 randomly generated sets of rate parameters as initial estimates. These fits provide a measure of sensitivity of the iterative fits to the initial parameters. In practice, the initial parameter estimates for conventional model Levenberg-Marquardt fits would be selected manually by the user to be a reasonably close match to the measured curve. In addition to introducing a level of subjectivity into the final result, the manual intervention adds significantly to the real-world fitting time. Rather than replicating this subjective process, randomly selected initial parameters were studied in order to characterize the sensitivity of the fits to these initial conditions. In the second set of fits, the initial parameter values were each set to the minimum value (e.g., 0.001), which results in the Levenberg-Marquardt algorithm finding the (possibly local) minimum closest to the minimum parameter values. Finally, in the third set of fits, the initial parameter values were set to midrange values, defined as being 1/4th the way between the minimum and maximum value for each parameter (e.g., for a parameter ranging from 0.0 to 2.0, the initial value was set at 0.5). The number of iterations and CPU times were recorded for each fit, and the best-fit results were analyzed relative to the true global minimum fits for each case (determined as described in Sec. 3C above).

RESULTS

Verification that the conventional and reformulated equations provide the same time-activity curves

Figure 2 shows example time-activity curves for 2K (based on ¹⁵O-water), 3K (based on ¹⁸F-FDG), and 4K (based on ¹¹C-acetate) compartment models. For each model, three representative sets of rate parameters were selected and curves were simulated using both conventional compartment model formulations and the separable parameter space formulation of Eq. 6. For the latter case, the values of κ_B, κ_b, κ₁, κ₂, υ₁, and υ₂ were computed from the conventional rate parameters according to the conversions described in Sec. 2A. The curves for each case were identical up to the numerical precision of the computer, confirming that the separable parameter space formulations correctly represent valid compartment models for these data.

Figure 2.

Example time-activity curves for 2K-4K serial compartment models simulated using the conventional and separable parameter space model formulations. Three example curves are shown for each model. The curves for the conventional and reformulated models were identical up to the numerical precision of the computer, and overlap exactly in the plots.

Characterization of objective functions

The separable parameter space reformulations reduce the dimensionality of the nonlinear fitting problem to only one free parameter for 2K-3K compartment models, and to only two free parameters for 4K-5K compartment models. The resultant 1D and 2D objective functions can be plotted and analyzed more easily than conventional objective functions in three to six dimensions. Figure 3 shows a representative WSSE objective function for a 3K compartment model fit to a noisy time-activity curve modeled on dynamic FDG PET tumor imaging. This typical objective function is well behaved, having a single well-defined minimum and no other confounding structures such as local minima or shelves. The plot also shows how the values of κ_B, κ_b, and κ₁ vary as functions of υ₁ to keep WSSE minimized according to the linear subproblem [Eq. 11]; the corresponding values of f_B and the conventional rate parameters are also shown in the lower panel for comparison.

Figure 3.

Plots of the separable parameter space WSSE objective function for an example noisy time-activity curve with 3K compartment model. The left plot shows how the values of κ_B, κ_b, and κ₁ that minimize WSSE change as a function of υ₁ (recall υ₁ = k₂ + k₃ for the 3K model). Similarly, the plot at right shows the same objective function with the corresponding values of f_B, K₁, k₂, and k₃ at each point. The objective function is well behaved, having a single clearly defined minimum with no other local minima, shelves, or confounding structures.

Figure 4 shows example separable parameter space WSSE objective functions for another representative 3K model curve at five different noise levels. The minimum is deepest for the lowest noise curve, and becomes shallower with increasing noise—consistent with the observation that the best-fit parameters will have greater statistical variability for noisier data. However, the objective function remained well behaved for all noise levels studied. Upon reviewing hundreds of 3K model objective functions for the separable parameter space formulation, the majority of functions are well behaved with only a single minimum and little or no complicated topology; however, shallow local minima were present in a minority of cases that could affect descent-type iterative fitting algorithms. In each of these cases, the local minimum occurred at relatively large values of υ₁—often larger than the physiologically meaningful range—and furthermore the local minima were shallow relative to the global minimum. The potential presence of such local minima has implications for descent-type fitting algorithms, but do not affect best-fit results when the exhaustive search algorithm is used. One potential solution for handling such local minima would be to provide an upper bound constraint on the value of υ₁ during the fitting procedure.

Figure 4.

Example separable parameter space WSSE objective functions for fitting a 3K compartment model to curves at five different noise levels. The minimum is deeper and better-defined for lower noise data, and becomes shallower as noise increases. However, for all noise levels the reformulated objective function remained well-behaved, and no complex topological features appeared with increasing noise levels.

Representative objective functions in two dimensions for 4K and 5K compartment models using the separable parameter space formulation are provided in Fig. 5. Here the reformulation has two nonlinear unknowns, υ₁ and υ₂, both of which are constrained to be non-negative, and further the solution space was constrained to values υ₂ < υ₁ in order to limit the solution space to regions that represent valid compartment models. The majority of these 2D objective functions were again generally well-behaved, with only a single minimum, although some complexity was present for lower values of υ₁ that may be considered a “side valley.” These curvatures in the objective function arise from imaginary “poles” in the excluded regions υ₂ > υ₁ and υ₂ < 0 (corresponding to values where the reformulated equations do not represent value compartment models—giving rise to rate parameters with either negative or imaginary values). The potential for local minima again cannot be ruled out, although empirical evidence suggests that the prevalence of local minima is low, and also that such minima are shallow and occur at relatively large values of υ₁ or υ₂. Overall, least-squares objective functions for the separable parameter space formulations were generally well-behaved with topology considerably less complex than that for the conventional model formulations, providing a simpler and potentially more robust environment for fast iterative nonlinear fitting.

Figure 5.

Contour plots of two examples separable parameter space WSSE objective functions for 4K (left) and 5K (right) compartment models. The grayscale represents WSSE values, and contour lines are drawn at regular intervals. Recall that the separable parameter space formulation for 4K-5K models has two nonlinear unknowns (υ₁, υ₂), and also that υ₂ is constrained to be less than υ₁ (hence the plots are only defined for regions υ₂ < υ₁). The objective functions are generally well behaved and do not show complex topological features. However, some complex structure exists in the lower-left corner near υ₁ = υ₂ line and near υ₂ = 0 which could slow convergence of gradient-descent type algorithms in these regions. These structures arise from “attraction” due to imaginary poles in the excluded υ₂ > υ₁ and υ₂ < 0 regions of the solution space.

Verification that the conventional and reformulated model equations have the same global minimum

Populations of 200–1000 noisy time-activity curves representative of dynamic PET tumor imaging with ¹⁵O-water (2K model), ¹⁸F-FDG (3K model), and ¹¹C-acetate (4K model) were simulated using methods described in Sec. 3A. Each noisy curve was fit to the separable parameter space formulation using the exhaustive search algorithm with the constraints listed in Tables 1, 2. The curves were also fit to the conventional model formulations using simulated annealing, where the fits were repeated using 10⁴–10⁸ iterations progressively in 10× increments in order to progressively approach the true global minimum fits across the population of curves as the number of iterations increased.

Figure 6 shows scatter plots of the best-fit WSSE and rate parameters for the 3K model comparing conventional versus separable parameter space fit results. Analogous results for the 4K model are provided in Fig. 7. These figures also include results for macroparameters K_i and V_D for each model. In all cases, the separable parameter space fits with exhaustive search produced equal or lower WSSE than the conventional fits with simulated annealing to within the numerical precision of the exhaustive search algorithm (approximately 1 part in 10⁻³). This result is consistent with the prediction that exhaustive search identifies the true global minimum to within the selected precision for all fits. In addition, the conventional model fits with simulated annealing progressively approached the separable parameter space exhaustive search results, further confirming that both model formulations provide the same best-fit global minimums.

Linear regression analysis between the conventional (maximum no. iterations) and separable parameter space fits revealed slopes of 1.000 ± 0.001 and Pearson correlation coefficients greater than 0.999 for all individual rate parameters and macroparameters. These results confirm that both the conventional and reformulated model equations provided identical best-fit results across these populations of curves to within the precision of the search. Small differences on the order of 10⁻⁴ were observed in the final rate parameters, consistent with a precision on the order of 10⁻³ for the exhaustive search algorithm [note that this precision is applicable to the nonlinear parameters υ₁ and υ₂; calculation of the linear (κ) parameters and then conversion to kinetic rate parameters K₁ − k₄ were computed at full floating point double- precision].

Performance of Levenberg-Marquardt algorithm

Table 4 summarizes the performance of the Levenberg-Marquardt algorithm for fitting both the conventional and separable parameter space model formulations. The number of iterations required to attain the stopping criterion provide a measure of iterative convergence properties. Here, the separable parameter space fits required significantly fewer iterations to converge than did the conventional model fits, suggesting that the reduced dimensionality of the nonlinear fitting space simplified the iterative fit. In order to determine whether or not each fit converged to the global minimum, as opposed to a local minimum, the true global minimum fit for each case was identified from the simulated annealing and exhaustive search results from Sec. 4C. For each Levenberg-Marquardt fit, the difference in best-fit WSSE between the Levenberg-Marquardt fit and the global minimum was computed. The median value and range are shown in the table. Similarly, the best-fit kinetic parameters (f_B, K₁ − k₄) for the Levenberg-Marquardt fits were evaluated versus those for the global minimum fits by computing the magnitude of the error vector (root sum-square error in parameter estimates). Again, the median value and range of the error vector magnitude are listed in the table.

Table 4.

Levenberg-Marquardt fit results.

	Conventional model fits				Separable parameter space fits
Model	Initial condition	Number of iterations (mean ± SD)	WSSE difference (median [range])	Magnitude error vector (median [range])	Number of iterations (mean ± SD)	WSSE difference (median [range])	Magnitude error vector (median [range])
2K	Random	314 ± 390	314 [0–76705]	5.300 [0–6.97]	19 ± 16	0.00 [0–76706]	0.001 [0.00–4.97]
	Minimum	205 ± 349	205 [0–814]	1.900 [0–2.88]	20 ± 24	0.00 [0–18.9]	0.001 [0.00–1.99]
	Midrange	123 ± 250	167 [0–1045]	1.100 [0–1.32]	20 ± 6	0.00 [0–0]	0.000 [0.00–0.11]
3K	Random	117 ± 260	3214 [0–136933]	2.990 [0.0–6.05]	15 ± 14	26 [0–695]	0.084 [0.00–7.14]
	Minimum	906 ± 234	107 [0–1393]	1.909 [0.062–4.82]	15 ± 18	32 [0–285]	2.881 [0.00–4.38]
	Midrange	117 ± 178	155 [10–2566]	1.008 [0.065–3.56]	28 ± 18	0.00 [0.0–0.0]	0.000 [0.00–0.002]
4K	Random	287 ± 389	1593 [0–56568]	2.357 [0.01–5.39]	130 ± 177	282 [0–16545]	1.550 [0.00–3.23]
	Minimum	723 ± 422	460 [0–54330]	1.894 [0.02–2.94]	62 ± 171	0.00 [0–796]	0.004 [0.00–2.79]
	Midrange	483 ± 460	16 [0–245]	1.478 [0.00–2.01]	80 ± 124	0.00 [0.0–0.0]	0.000 [0.00–0.01]

The fit results were sensitive to the initial conditions for both the conventional and separable parameter space model formulations, indicating that local minima may be present (at least some of the time) for both cases. However, the separable parameter space fits were less sensitive to initial conditions than were the conventional model fits. In particular, initializing the fit with midrange parameter values provided excellent results. When using these initial values, the separable parameter space fits attained the true global minimum fit for all parameters (defined as magnitude of the best-fit error vector < 0.01) in 96% of cases for the 2K model, 98% of cases for the 3K model, and 94% of cases for the 4K model. In contrast, the conventional model fits attained the true global minimum fit for all parameters in 53% of cases for the 2K model, in 4.8% of cases for the 3K model, and in 1.5% of cases for the 4K model. These results demonstrate the importance of manually selecting good initial conditions for the Levenberg-Marquardt algorithm with the conventional model formulation. Conversely, the Levenberg-Marquardt fits for the separable parameter space formulation were much less sensitive to initial conditions, which can be seeded within midrange initial values and converge to the true global minimum in the majority of cases.

Fitting times

Table 5 shows the CPU times for fitting noisy time-activity curves with 2K, 3K, and 4K models using both the conventional and separable parameter space model formulations. Times were computed for single-threaded code run on a 2.80 GHz Intel Xeon workstation, and no acceleration via multithreading was implemented. Each algorithm was implemented in C, sharing routines when possible, and with identical attention to optimizing the code for computational efficiency. The CPU time per iteration was similar for both model formulations, on the order of 0.04–0.07 ms for 2K-3K models and 0.09–0.10 ms for 4K models. Overall, the separable parameter space fits required approximately 10 ms for 2K-3K models and 1.1 s for 4K models; these times were reduced to about 1 and 7 ms, respectively, using the Levenberg-Marquardt algorithm.

Table 5.

CPU times for fits.

	2K		3K		4K
Model	Number of iterations	CPU time (ms)	Number of iterations	CPU time (ms)	Number of iterations	CPU time (ms)
Conventional model formulations
Simulated annealing	104	326 ± 22	105	2227 ± 163	106	24056 ± 4508
Levenberg-Marquardt	123 ± 250	6.87 ± 13.97	117 ± 178	8.79 ± 13.39	483 ± 460	48.80 ± 46.19
Separable parameter space reformulations
Exhaustive search	103	10.2 ± 1.5	103	10.1 ± 0.9	106	1109 ± 172
Levenberg-Marquardt	20 ± 6	0.90 ± 0.27	28 ± 18	1.32 ± 0.85	80 ± 124	7.34 ± 11.38

DISCUSSION AND CONCLUSIONS

The separable parameter space techniques discussed in this paper generalize previously proposed methods for applying separable nonlinear least-squares to the compartment model fitting problem. The formulation described herein encompasses 1K-5K serial compartment models, including a modified formulation for higher order models that avoids a potential degeneracy present in previously proposed reformulations for these models. As a theoretical exercise, the linear subproblem of the reformulated equations was solved analytically and substituted back into the remaining nonlinear fitting equations. The resultant system can be interpreted as effectively reducing the dimensionality of the fitting space to only one dimension for 2K-3K compartment models, and to two dimensions for 4K-5K models—providing a nonlinear fitting problem that is both simpler and easier to solve than that of the original model formulation. This illustrates the value of the separable parameter space technique, but is not amenable to imposing constraints on the linear parameters. When keeping the linear subproblem separate, a fast and robust algorithm was presented for solving the linear subproblem with complete user-defined constraints on the fitted parameters. The generalized technique also treats the nonlinear unknowns as fittable parameters, rather than treating their corresponding temporal terms as basis functions. The receptiveness of the technique to fast gradient-descent type nonlinear fitting algorithms was explored by characterizing the objective functions, and by studying the performance of the Levenberg-Marquardt algorithm for solving the nonlinear subproblem.

The work focused specifically on the topic of performing the mathematical fit according to a preestablished fitting criterion—the WLS criterion. In order to retain focus upon the mathematical fitting approach, we did not explore other issues such as consideration of other fitting criteria, selection of the fitting weights, or identification of the input function. While these issues affect the bias and noise properties of the fitted parameters, they are distinct from issues regarding the mathematical fit—the fitting algorithm is a mathematical procedure for attaining the fitting solution that minimizes the fitting criterion, whereas the criterion, weights, and input function determine the properties of the fit at the global minimum.

When using the Levenberg-Marquardt algorithm, it is a common practice to estimate the covariance matrix of the standard errors in the fitted parameters by inverting the Hessian matrix used by the fitting algorithm³³ (typically with second derivatives ignored). When using separable parameter space techniques, the Hessian matrix used in the nonlinear subproblem corresponds to only the nonlinear free parameters [as well as being subject to the constraint of minimizing the linear subproblem, as per Eq. 12]. In order to estimate the covariance matrix of the fitted kinetic rate parameters, one should compute and invert the Hessian matrix of the conventional full-parameter space model formulation using the best-fit rate parameters obtained from the separable parameter space fit. This approach is viable since both the conventional and separable parameter space formulations have the same global minimum, and requires a small degree of additional computation after completing the fit.

The generalized separable parameter space technique was found to provide identical time-activity curves as conventional compartment model formulations, and both the conventional and reformulated equations give rise to the same global minima. As such, fitting the reformulated equations (with appropriate constraints) and calculating conventional rate parameters from the best-fit reformulated parameters effectively finds the best-fit to the conventional model equations. However, the reduced dimensionality of the reformulated nonlinear fitting subproblem greatly simplifies separable parameter space fits. This reduced dimensionality makes exhaustive search algorithms feasible (and indeed computationally rapid), providing fits that guarantee the true global minimum to within the search precision in as little as 10 ms for 2K-3K models and 1.1 s for 4K models. For applications in which more rapid fitting is desired, e.g., voxelwise parametric imaging, fast iterative fitting algorithms, such as Levenberg-Marquardt, can be used to reduced the fitting times to approximately 1 ms for 2K-3K models and 7 ms for 4K models. Such fits were found to be less sensitive to initial conditions than comparable iterative fits with the conventional model formulations, and to converge in fewer iterations. In conclusion, the separable parameter space techniques described in this work generalize previously described techniques to encompass 1K-5K compartment models, enable robust solution of the linear subproblem with full user-defined constraints, and are amenable to rapid and robust fitting using iterative gradient-descent type algorithms.

NOMENCLATURE

b(t)

Input function

$\stackrel{⌣}{b}\left(t\right)$

Temporal term (integral of b(t)); used in 1K, 3K, and 5K models

B(t)

Tracer concentration in whole-blood

f_B

Fractional contribution of B(t) to the imaging measurement

K_i

Net influx macroparameter, K_i = K₁k₃/ (k₂ + k₃)

K₁, k₂ − k₅

Rate constants for 1K-5K serial compartment models

κ_B, κ_b, κ₁, κ₂

Linear parameters of the separable parameter space formulations

R(t)

Modeled time-activity curve

S₁(t; υ₁; b(t))

Temporal term present in 2K-5K compartment models

S₂(t; υ₁, υ₂; b(t))

Temporal term present in 4K-5K compartment models

υ₁, υ₂

Nonlinear parameters of the separable parameter space formulations

V_D

Volume of distribution

APPENDIX: PSEUDOCODE ALGORITHM FOR COMPUTING CONSTRAINED LINEAR SUBPROBLEM

Let ${\kappa }_i^{\min }$ and ${\kappa }_i^{\max }$ be arbitrary user-defined minimum and maximum values for each linear parameter κ_i.

The linear subproblem described in Sec. 2C can be rapidly solved subject to these constraints using the following pseudocode algorithm:

(1) Analytically invert the linear subproblem directly to obtain unconstrained best-fit values of all κs.

(2) If any κi is out of range, where i ∈ {B, 2, 1, b}, then:

Loop over ${\kappa }_i^{\min }\le {\kappa }_i\le {\kappa }_i^{\max }$ with step size Δi: (i.e., exhaustively search over κi; typical Δi = 0.001 min−1)

Fixing κi, analytically invert the remaining linear subproblem for the remaining free κs.

If any κj is out of range, where j ∈ {B, 2, 1, b} and j ≠ i, then:

Loop over ${\kappa }_j^{\min }\le {\kappa }_j\le {\kappa }_j^{\max }$ with step size Δj: (i.e., exhaustively search over κj)

Fixing κj, analytically invert the remaining linear subproblem for the remaining free κs.

If any κk is out of range, where k ∈ {B, 2, 1, b} and k ≠ i and k ≠ j, then:

Loop over ${\kappa }_k^{\min }\le {\kappa }_k\le {\kappa }_k^{\max }$ with step size Δk: (i.e., exhaustively search over κk)

Analytically compute κl, where l ∈ {B, 2, 1, b} and l ≠ i, l ≠ j, and l ≠ k.

If κl is out of range, then:

Loop over ${\kappa }_l^{\min }\le {\kappa }_l\le {\kappa }_l^{\max }$ with step size Δl: (i.e., exhaustively search over κl)

Find κl that maximizes WSSE for the current values of κi, κj, and κk.

End loop.

End if.

Find κk that maximizes WSSE for the current values of κi, κj, and κl.

End loop.

End if.

Find κj that maximizes WSSE for the current values of κi, κk, and κl.

End loop.

End if.

Find κi that maximizes WSSE for the current values of κj, κk, and κl.

End loop.

End if.