An Efficient Deconvolution Algorithm for Estimating Oxygen Consumption During Muscle Activities

Abstract

The reconstruction of an unknown input function from noisy measurements in a biological system is an ill-posed inverse problem. Any computational algorithm for its solution must use some kind of regularization technique to neutralize the disastrous effects of amplified noise components on the computed solution. In this paper, following a hierarchical Bayesian statistical inversion approach, we seek estimates for the input function and regularization parameter (hyperparameter) that maximize the posterior probability density function. We solve the maximization problem simultaneously for all unknowns, hyperparameter included, by a suitably chosen quasi-Newton method. The optimization approach is compared to the sampling-based Bayesian approach. We demonstrate the efficiency and robustness of the deconvolution algorithm by applying it to reconstructing the time courses of mitochondrial oxygen consumption during muscle state transitions (e.g., from resting state to contraction and recovery), from the simulated noisy output of oxygen concentration dynamics on the muscle surface. The model of oxygen transport and metabolism in skeletal muscle assumes an in vitro cylindrical structure of the muscle in which the oxygen from the surrounding oxygenated solution diffuses into the muscle and is then consumed by the muscle mitochondria. The algorithm can be applied to other deconvolution problems by suitably replacing the forward model of the system.

1 Introduction

Consider a dynamical system in which an observable output is related to the not directly measurable input through a transfer function, which in the linear case is the system’s response to a unit impulse function. The problem of reconstructing an input signal from the measured output in such a dynamical system is called deconvolution. In addition, the transfer function may depend on unknown parameters, and it is often of interest to estimate these model parameters simultaneously with the linear input. When the transfer function is unknown or not fully determined, the problem is usually called blind deconvolution. Deconvolution problems, and blind deconvolution problems in particular, are typically ill-posed inverse problems, i.e., small errors in the data propagate, strongly amplified, to the estimated inputs unless some sort of regularization is used.

In the quantitative studies of complex physiological and pharmacokinetic systems, deconvolution allows reconstruction of important non-accessible input signals; see, e.g., [2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]. The particular application that we consider in this paper is the estimation of the time course of mitochondrial oxygen consumption (nonmeasurable) in muscle tissue during muscle state transitions from rest to contraction and recovery, from the samples of its causally-related measurable effects, such as the dynamics of oxygen concentration in the surrounding medium [2, 3]. Quantifying the time course of mitochondrial oxygen consumption during muscle activities is of great importance in the understanding of the dynamic regulation of oxidative phosphorylation and muscle energetics [3, 4, 5, 13, 14, 15, 16, 17, 18, 19].

In a recent paper, Dash et al. [3] developed a computational model of oxygen transport and metabolism in a cylindrically-shaped muscle, immersed in vitro in an oxygenated chamber, and used a recently developed hierarchical Bayesian statistics-based parametric deconvolution algorithm [20] to obtain the estimate of the time course of mitochondrial oxygen consumption from the polarographic measurements of decayed oxygen concentration on the muscle surface (decayed oxygen partial pressure in the chamber) measured before, during and after an isometric twitch contraction of the muscle [19]. Although their analysis facilitated a better formal approach for estimating mitochondrial oxygen consumption in comparison to the previous analyses [5, 18, 19], their estimates were inaccurate and oscillatory at higher noise levels in the measured data. Furthermore, the algorithm was computationally expensive due to the empirical Bayes approach [6, 9, 10, 20, 21] of estimating the unknown input function and regularization parameter (hyperparameter).

In this paper, we propose a hybrid deconvolution algorithm for determining the mitochondrial oxygen consumption time course during muscle activities, which is found to be very efficient and robust. The algorithm is based on a Bayesian statistical framework and computes simultaneously the maximum a posteriori (MAP) estimate of the unknown input function parameters and the hyperparameter. The algorithm employs a sequential optimization (quasi-Newton) scheme that updates alternatively the model parameters and the hyperparameter to maximize the posterior probability density function. The algorithm requires solving a normal equation at each iteration of the optimization, which is computationally simpler than solving the ill-conditioned linear systems [22, 23]. The performance of the algorithm is compared and validated with a Markov Chain Monte Carlo sampling-based analysis of the posterior probability density. Similar ideas can be found in the literature, see, e.g., [6, 9, 10, 20, 21]. We remark that the applicability of the proposed algorithm is not limited to the particular problem considered here. It can be applied to other deconvolution problems by suitably replacing the forward model of the system.

2 Mathematical Model of Oxygen Transport and Metabolism

In this section, we briefly review the forward mathematical model simulating the oxygen uptake, transport and metabolism in an isolated skeletal muscle during muscle activities, i.e, during the resting, contraction and recovery periods. This model is presented in detail, including the experimental background, in Ref. [3].

The experimental protocol consists of an isolated skeletal muscle mounted in a glass chamber filled with a highly concentrated oxygenated solution. The muscle is electrically stimulated at a certain frequency for a time interval from the resting state to give a twitch contraction and then allowed to return to the resting state. The decay of oxygen in the chamber is measured continuously through a polarographic electrode [3, 19, 5]. The physiological problem of estimating the time course of mitochondrial oxygen consumption and muscle oxygen uptake from the time course of oxygen decay in the chamber leads to a deconvolution problem considered in Ref. [3].

The mathematical model is based on the assumption of cylindrical geometry of the muscle, with circular radius R_mu much smaller than the length L_mu. The oxygen concentration C_ch = C_ch(t) in the chamber decays due to the consumption by the muscle mitochondria in addition to the consumption by the polarographic electrode and leakage from the chamber, which is referred to as the apparatus baseline. The oxygen concentration C_mu = C_mu(r, t) in the muscle varies due to the radial diffusion and mitochondrial oxygen consumption. The mitochondrial oxygen consumption is highly complex and depends, in general, on the concentrations of available oxygen, substrate dehydrogenase (NADH) and phosphates [14, 15, 16].

All the dependencies are lumped here into a simplified time-dependent flux expression. For numerical computations, we impose a positivity constraint on concentrations to avoid non-physiological solutions arising due to the simplified flux expression for mitochondrial oxygen consumption. The governing equations are given by

$$d{C}_{\text{ch}}=\left\{{-\frac{F_{\text{tot}}}{V_{\text{ch}}}-D\frac{2\pi R_{\text{mu}}{L}_{\text{mu}}}{V_{\text{ch}}}\left(\frac{\partial C_{\text{mu}}}{\partial r}\right)|}_{r=R_{\text{mu}}}\right\}dt,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{C}_{\text{ch}}\ge 0,$$ (1)

$$d{C}_{\text{mu}}=\left\{D\left(\frac{{\partial }^2{C}_{\text{mu}}}{\partial r^2}+\frac{1}{r}\frac{\partial C_{\text{mu}}}{\partial r}\right)-\frac{F_{\text{mr}}+F_{\text{ms}}(t)}{\pi R_{\text{mu}}^2{L}_{\text{mu}}}\right\}dt,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{C}_{\text{mu}}\ge 0.$$ (2)

In Eq. (1), F_tot denotes the total rate (nmoles/sec) of oxygen decay in the chamber due to the apparatus baseline, V_ch denotes the chamber volume, and D is the diffusion coefficient, all of which are assumed to be known [3]. The second term in the right-hand side of Eq. (1) represents the muscle oxygen uptake through the muscle surface. In Eq. (2), F_mr and F_ms(t) denote the fluxes (nmoles/sec) of mitochondrial oxygen consumption at resting state and during muscle stimulation above the resting state, respectively. The initial–boundary conditions for the system of diffusion-consumption equations (1) – (2) are given by

$$C_{\text{ch}}(t=0)=C_{\text{ch},0},$$ (3)

$$C_{\text{mu}}(r,t=0)=C_{\text{mr}}(r),$$ (4)

$$C_{\text{mu}}(r=R_{\text{mu}},t)=C_{\text{ch}}(t),$$ (5)

$$\frac{\partial C_{\text{mu}}}{\partial r}(r=0,t)=0.$$ (6)

The concentration C_mr(r), which is obtained from the steady state version of the differential equation (2) and boundary conditions (5) – (6), has an explicit form:

$$C_{\text{mr}}(r)=max\left\{0,C_{\text{ch},0}-\frac{F_{\text{mr}}}{4V_{\text{mu}}D}(R_{\text{mu}}^2-r^2)\right\},$$ (7)

Where $V_{\text{mu}}=\pi R_{\text{mu}}^2{L}_{\text{mu}}$ is the volume of the cylindrical muscle. The numerical parameter values are as in Ref. [3].

The inverse problem that is addressed in this article can be stated as follows: Based on the diffusion-consumption model (1) – (2) and the initial-boundary conditions (3) – (6), estimate {F_mr, F_ms(t) | 0 ≤ t ≤ T} from the noisy observation of{C_ch(t) | 0 ≤ t ≤ T}, where T is duration of the experiment.

The numerical solutions of the forward model is obtained using a stable explicit finite difference scheme (Euler-type) as presented in Ref. [3]. Briefly, we use a standard central difference in radial direction and forward difference in temporal direction to discretize the system. For the stability of the marching scheme, the spatial and temporal discretization steps Δr and Δt are chosen to satisfy the stability criterion Δt ≤ Δr²/2D. The positivity constraint for concentrations is implemented by a non-negativity projection at each time step. Observe that this projection renders is what makes this problem non-linear.

3 Estimation of the Input

In this section, we describe the deconvolution algorithm which is used in estimating the unknown parameter F_mr and the unknown input function F_ms(t) in the diffusion-consumption equations (1) – (2) from the noisy output of decaying oxygen concentration C_ch(t) in the chamber or on the muscle surface C_mu(R_mu, t). This algorithm is based on a Bayesian statistical framework [24].

3.1 Posterior Probability Density

Consider a parameter-dependent mapping of a given input function, f(t) ↦ Ψ(θ, f(t), t), θ = [θ₁, … θ_K] ∈ ℝ^K, modeling an idealized noiseless output of a dynamical system. In our application, f(t) = F_ms(t), and θ = F_mr ∈ ℝ, i.e., the model depends of only one parameter. The idealized output is oxygen concentration in the chamber, Ψ(θ, f(t), t) = C_ch(F_mr, F_ms(t), t) which is governed by the diffusion-consumption equations (1) – (2) and initial-boundary conditions (3) – (6). We denote by y_i the output data point at time t_i which is assumed to be corrupted by additive Gaussian noise e_i with mean zero and variance ${\sigma }_i^2$. Then the noisy output y_i can be written as

$$y_i=\psi (\theta ,f(t_i),t_i)+e_i,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{e}_i\sim N(0,{\sigma }_i^2),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots M.$$ (8)

The unknown input function f(t) is assumed to satisfy the conditions f(0) = 0 and f(T) = 0, since at the beginning and at the end the muscle is at rest. We approximate f(t) by a truncated Fourier sine series over the interval [0,T]:

$$f(t)\approx f(\alpha ,t)=\sum _{j=1}^N{\alpha }_j{\phi }_j,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\phi }_j=\sqrt{\frac{2}{T}}sin({\beta }_{j}t),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{\beta }_j=\frac{\pi j}{T},$$ (9)

where α = [α₁, …, α_N ] ∈ ℝ^N is the vector of amplitude coefficients and φ(t) = [φ₁(t), …, φ_N (t)] ∈ ℝ^N is the vector of orthonormal basis functions. We denote the resulting approximation of the model by Ψ(θ, α, t).

Let y = [y₁, …, y_M ]^T ∈ ℝ^M be the vector of noisy data, e = [e₁, …, e_M ]^T ∈ ℝ^M be the vector of Gaussian noise, Ψ(θ, α) = [Ψ(θ, α, t₁), …, Ψ(θ, α, t_M)]^T ∈ ℝ^M be the vector of model predictions, and S = diag[1/σ₁, …, 1/σ_M ] ∈ ℝ^M×M be a diagonal matrix. If we assume that the noise at time t_i is independent of the noise at time t_j, j ≠ i, then the likelihood probability density of the data y conditioned on the parameter vectors θ and α is given by [24]

$$\pi (y|\theta ,\alpha )\propto exp\left(-\frac{1}{2}{\Vert S[y-\psi (\theta ,\alpha )]\Vert }^2\right).$$ (10)

Here, “∝” means “equal up to a multiplicative constant”. Furthermore, if we assume a first order smoothness prior on the parameter vector α with variance λ > 0, then the probability density of α conditioned on λ is given by

$${\pi }_{\text{pr}}(\alpha |\lambda )\propto \frac{1}{{(2\pi \lambda )}^{N/2}}exp\left(-\frac{1}{2\lambda }{\Vert f^{\prime }(\alpha ,·)\Vert }_{L^2}^2\right)=exp\left(-\frac{1}{2\lambda }\sum _{j=1}^N{\alpha }_j^2{\beta }_j^2-\frac{N}{2}log(2\pi \lambda )\right),$$ (11)

where fʹ (α, t) denotes the derivative of f(α, t) with respect to t. Such prior indicates an a priori belief that the signal does not contain significant high frequency components that would result into large amplitude fast oscillations. From Eqs. (10) and (11), the joint probability density of y and α conditioned on θ and λ is then given by

$$\begin{array}{l}\pi (y,\alpha |\theta ,\lambda )=\pi (y|\theta ,\alpha ){\pi }_{\text{pr}}(\alpha |\lambda )\\ \propto \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}exp\left(-\frac{1}{2}{\Vert S[y-\psi (\theta ,\alpha )]\Vert }^2-\frac{1}{2\lambda }\sum _{j=1}^N{\alpha }_j^2{\beta }_j^2-\frac{N}{2}log(2\pi \lambda )\right).\end{array}$$ (12)

In general, the values of θ and λ are not known a priori. However, since λ and θ must be a positive constant, we can use the flat hyperprior:

$${\pi }_{\text{h}}(\theta ,\lambda )\propto {\pi }_{+}(\theta ,\lambda )=\{\begin{array}{ll}1\hfill & \text{if\hspace{0.17em}}\lambda >0\hspace{0.17em}\text{and\hspace{0.17em}}\theta >0,\hfill \\ 0\hfill & \text{otherwise}.\hfill \end{array}$$ (13)

It follows from Bayes’ formula [24] and Eqs. (12) and (13) that the joint probability density of parameters θ, α and λ conditioned on the data y is given by

$$\begin{array}{l}\pi (\theta ,\alpha ,\lambda |y)\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\propto \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\pi (y,\theta ,\alpha ,\lambda )=\pi (y,\alpha |\theta ,\lambda ){\pi }_{\text{h}}(\theta ,\lambda )\\ \propto \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}exp\left(-\frac{1}{2}{\Vert S[y-\psi (\theta ,\alpha )]\Vert }^2-\frac{1}{2\lambda }\sum _{j=1}^N{\alpha }_j^2{\beta }_j^2-\frac{N}{2}log(2\pi \lambda )\right),\end{array}$$ (14)

subject to the constraints θ, λ > 0. The conditional density in equation (14) is also known as the posterior probability density, and in statistical inverse problems can be considered as the solution of the inverse problem. In the following sections, we discuss various methods to explore the posterior density and how to calculate estimates from it.

3.2 Exploring the Posterior Density

It is possible to calculate various estimates of the unknown based on the posterior density, or explore the density by Monte Carlo methods. We discuss here some of the estimation techniques which can be found in the literature and propose a numerically efficient optimization method for the parameter estimation.

3.2.1 Empirical Bayes Approach

A common practice when dealing with hierarchical Bayesian models is to marginalize the posterior density with respect to the parameters of primary interest, and then use the marginal density to estimate the hyperparameters [6, 24, 21, 9, 8]. In the present setting, this amounts to calculating the marginal density

$$\pi (\lambda |y)={\int }_{{ℝ}^N}{\int }_{ℝ}\pi (\theta ,\alpha ,\lambda |y)d\theta d\alpha .$$ (15)

In general, there is no analytic formula for this integral and one has to resort to Monte Carlo integration techniques.

We write a partially linerized approximation α, i.e.,

$$\psi (\theta ,\alpha )\approx {\psi }_0(\theta )+A_{\theta }\alpha ,$$ (16)

where the mappings θ ↦ Ψ ₀(θ) ∈ ℝ^M and θ ↦ A_θ ∈ ℝ^M×N are non-linear. Observe that the forward differential equation model without positivity constraints defines (1)–(6) a linear model, and the nonlinearities are due to to the positivity projection. The above approximation is found reasonably accurate by numerical tests. Using this model, we may first integrate out analytically the variable α, using the Gaussian form of the posterior density. Notice that the dependency of A_θ of the parameter θ is due to the projection step in the forward solver. By writing

$${\Vert S\left[y-\psi (\theta ,\alpha )\right]\Vert }^2+\frac{1}{\lambda }\sum _{j=1}^N{\alpha }_j^2{\beta }_j^2={\Vert \left[\begin{array}{c}S{A}_{\theta }\\ {\lambda }^{-1/2}L\end{array}\right]\hspace{0.17em}\alpha -\left[\begin{array}{c}S(y-{\psi }_0(\theta ))\\ 0\end{array}\right]\Vert }^2={\Vert D\alpha -b\Vert }^2,$$ (17)

where L = diag(β) ∈ ℝ^N×N, we obtain, after some algebraic manipulations, the formula

$$\pi (\theta ,\lambda |y)={\int }_{{ℝ}^N}\pi (\theta ,\alpha ,\lambda |y)d\alpha ={\left(det(D^{\text{T}}D)\right)}^{-1/2}exp\left(-\frac{1}{2}{b}^{\text{T}}Pb-\frac{N}{2}log(2\pi \lambda )\right),$$ (18)

where P ∈ ℝ ^(M+N)×(M+N) is the projection matrix

$$P=I-D{(D^{\text{T}}D)}^{-1}{D}^{\text{T}}.$$ (19)

Here the matrices D, P and the vector b depend on both parameters θ and λ. The integration with respect to θ has to be performed numerically, e.g., by MCMC techniques. The procedure of integrating out part of the variables analytically is sometimes referred to as Rao–Blackwellization (see, e.g., [25]), and it is known to produce estimates with smaller variance than a full MCMC integration.

In general, the MCMC runs that update the parameter θ are computationally intensive because for each new value of θ, the matrix A_θ and the vector b need to be recomputed. The computational cost decreases dramatically if we have a good estimate for the baseline parameter. Indeed, with θ fixed and A_θ precomputed, a sample can be generated by a block form of the Gibbs sampler via the following updating steps:

1. Given the current pair (λ^ℓ, α^ℓ), draw a new value a^ℓ+1 from the distribution

   $$\pi (\alpha |\theta ,{\lambda }^{\ell },y)\propto exp\left(-\frac{1}{2}{\Vert D\alpha -b\Vert }^2\right),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}D=D({\lambda }^{\ell }),$$ (20)

   by drawing w_j ~ $N$ (0, 1), 1 ≤ j ≤ N + M, and solving the system Dα = b + w in the least squares sense;

2. Draw a new value λ^ℓ+1 from the one-dimensional density

   $$\pi (\lambda |{\alpha }^{\ell +1},\theta ,y)\propto exp\left(-\frac{1}{2\lambda }\sum _{j=1}^N{\beta }_j^2{({\alpha }_j^{\ell +1})}^2-\frac{N}{2}log(2\pi \lambda )\right).$$ (21)

This updating is fast because it does not require the solution of the forward model.

3.2.2 Maximum A Posteriori Estimator

The Maximum A Posteriori (MAP) estimate maximizes the posterior probability density function π(θ, α, λ | y), or, equivalently, minimizes the negative of its logarithm. Thus,

$$({\theta }_{\text{MAP}},{\alpha }_{\text{MAP}},{\lambda }_{\text{MAP}})=\text{argmax\hspace{0.17em}}\pi (\theta ,\alpha ,\lambda |y)=\text{argmin\hspace{0.17em}}\mathrm{\Phi }(\theta ,\alpha ,\lambda ),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\lambda >0,$$ (22)

where

$$\mathrm{\Phi }(\theta ,\alpha ,\lambda )=-log[\pi (\theta ,\alpha ,\lambda |y)]=\frac{1}{2}{\Vert S[y-\psi (\theta ,\alpha )]\Vert }^2+\frac{1}{2\lambda }\sum _{j=1}^N{\alpha }_j^2{\beta }_j^2+\frac{N}{2}log(2\pi \lambda ).$$ (23)

The minimization of the objective function Φ is performed with a sequential algorithm, where one minimizes it with respect to the model parameters (θ, α) and with respect the prior parameter λ alternatingly. The algorithm can be described as follows.

1. Initialize k = 0, (θ, α, λ) = (θ₀, α₀, λ₀).

2. Keeping λ = λ_k fixed, update the model parameters θ and α as

   $$({\theta }_{k+1},{\alpha }_{k+1})=argmin(F_k(\theta ,\alpha )),\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}{F}_k(\theta ,\alpha )=\frac{1}{2}{\Vert S[y-\psi (\theta ,\alpha )]\Vert }^2+\frac{1}{2{\lambda }_k}\sum _{j=1}^N{\alpha }_j^2{\beta }_j^2.$$ (24)

3. Keeping the model parameters θ = θ_k+1, α = α_k+1 fixed, update λ by

   $${\lambda }_{k+1}=\frac{1}{N}\sum _{j=1}^N{\alpha }_{k+1,j}^2{\beta }_j^2.$$ (25)

4. Increment k and repeat from step 2 until convergence.

The updating formula for λ follows from the condition ∂Φ(θ_k+1, α_k+1, λ)/∂λ = 0. This updating is fast because it does not require the solution of the forward model. The second step, updating the model parameters, is calculated with a quasi-Newton algorithm. Writing ξ_k = [θ_k, α_k]^T, and δξ = [δθ, δα]^T, we write the quadratic approximation

$$F_k({\xi }_k+\delta \xi )\approx F_k({\xi }_{\text{k}})+G_k({\xi }_k)\delta \xi +\frac{1}{2}\delta {\xi }^T{H}_k({\xi }_k)\delta \xi ,$$ (26)

where G_k(ξ) and H_k(ξ) are the gradient and Hessian of F_k(ξ), respectively.

Approximating the gradient of F_k by the gradient of the quadratic approximation (26), and setting it equal to zero leads to the linear system

$$H_k({\xi }_k)\delta \xi =-G_k({\xi }_k),$$ (27)

whose solution we use to compute the update, ξ_k+1 = ξ_k + tδξ, where t is chosen as

$$t=arg\underset{0<s\le 1}{\min }{F}_k({\xi }_k+s\delta \xi ).$$ (28)

For details on the selection of a suitable t, known in the optimization literature as backtracking algorithm, we refer to [22], Ch. 5. Details on the calculation of the gradient and the Hessian, and the solution of the associated linear system, can be found in the Appendix.

4 Computed Examples

In this section we test the algorithms on two different input functions. The first is a continuous function, starting at the resting value F_mr = 0.005 moles/min, increasing linearly to five-fold value, and, after staying there for a while, decreasing linearly back to the original resting value. We refer to this function as ramp input. The second input is similar, except that the peak value is reached discontinuously. This input is referred to as step input. Observe that the step input is in conflict with what we expect to see in light of the prior density, so the results may be of inferior quality than those obtained with the ramp input. Indeed, the Fourier coefficients of a discontinuous input behave as α_j ~ 1/j. We include this test just to demonstrate the robustness of the approach with respect to prior modeling.

In the first test, we generate the data using the forward model with using the ramp input and we add Gaussian noise with σ_i = 0.15. In particular, we want to compare the empirical Bayes reconstructions and the MAP estimate. The conditional mean for θ, calculated from the two-dimensional marginal distribution π(θ, λ | y), given by (18) and shown in Figure 1 (upper left), corresponds well to the true resting value θ = 0.005 nmoles/min. After fixing the resting value θ, we perform a block Gibbs MCMC run to produce a sample S = {(α^ℓ, λ;^ℓ)) | 1 ≤ l ≤L} of size L = 2000, that is distributed according to the conditional distribution π(α, λ | θ, y). The chain is started at λ = 1, α_j = 0, and a short burn-in sequence (of length 200) is removed from the beginning of the chain. It is our experience that the chain stabilizes very quickly, after a few iterations. Figure 1 (upper right) shows the sampling history of λ, indicating that the mixing appears to be efficient. The histogram and the conditional mean computed from this sample are shown in the same figure (lower left). We then calculated all the inputs corresponding to the sample S and form the envelopes containing 50% and 90% of the sample curves, respectively. The envelopes are also displayed in Figure 1 (lower right).

Figure 1.

Results of the empirical Bayes approach with the ramp input. Top left: marginal distribution π(θ, λ | y). The CM values are marked by the hair-cross. Top right: MCMC scatter plot of the hyperparameter drawn from the distribution π(α, λ | θ, y). Bottom left: histogram of the hyperparameter based on the same sample. Bottom right: 50% and 90% confidence envelopes based on the MCMC run. The yellow curve represents the estimated conditional mean. In all figures, the simulated data was corrupted with additive Gaussian white noise with standard deviation σ = 0.15.

The corresponding results using the step input are plotted in Figure 2. The noise level was the same as for ramp input. Due to the fact that for discontinuous functions, the convergence of the Fourier expansion is slower, the modeling discrepancy between the data and the forward model used in the likelihood is larger than in the previous case. The effect is that the marginal variances are larger and the conditional mean deviates more from the true input.

Figure 2.

The corresponding results as in the previous figure with a step input. The noise level is the same as before.

We then applied our proposed algorithm to the same data, performing the optimization to find an approximation for the MAP estimate of the input as described in Section 3.2.2. The results are shown in Figure 3. As expected, the quality of reconstruction of the ramp input is better than for the step input. The optimal value of the prior parameter λ in this case is λ_MAP = 9.7078 × 10⁻⁸ for the ramp input and λ_MAP = 1.0015 × 10⁻⁷ for the step input. This should be compared to the conditional mean value λ_CM = 3.0070 × 10⁻⁷ for the ramp input and λ_CM = 3.4526 × 10⁻⁷ for the step input, found with the empirical Bayes approach. The parameters values found by the two different approaches are in the same range. Observe that larger λ corresponds to more oscillatory reconstructions. An effective estimation algorithm could be designed based on the maximization of the analytically computed marginal density π(θ, λ | y), but this relies on partial linearization of the model. Since the densities of λ are skewed to the right, the solution obtained with the parameter value found via maximization gives less oscillatory estimates. Estimating the conditional mean of all the parameters from an MCMC run with an exact model is significantly more expensive.

Figure 3.

Estimated flux of mitochondrial oxygen consumption and corresponded model fit for ramp and step input for different number of basis functions and noise level ε = 0.15.

The main purpose of this comparison is to verify that the MAP solution computed by our algorithm falls well within the support of the probability density. In fact, since the MAP solution only returns one estimate it not convey any information about stability. This is a well know shortcoming of the MAP approach. On the other hand, the estimation of the model parameters and the hyperparameter using the proposed sequential MAP estimate algorithm is very efficient and robust. The iterations converge to the optimal parameter values extremely fast irrespective of the choice of the number of basis function, the tolerance levels for the linear system solver and quasi-Newton iterations, and the initial guess for the parameter values. The computation time using MATLAB is less than 5 minutes for a typical data set with standard setting of computational parameter values. The passage from the linear system (27) to the normal equation (see Appendix) means that we use for its solution standard iterative linear system solver, like the Conjugate Gradient Least Squares (CGLS) or MINimum RESidual (MINRES) methods, [22, 23]. The use of an iterative solver in our case is not dictated by the dimensionality of the problem, but rather by the need of enforcing the nonnegativity of some of the parameters; see [1] for details. The overall scheme is quite fast since the updating of hyperparameter does not require the solution of the forward model. While it is also possible to estimating all the parameters at once, without resorting to an alternating procedure, the convergence rate of the resulting quasi-Newton method in that case appears to be quite sensitive to organization of the computation, thus making the algorithm more difficult to implement. Therefore, the proposed sequential optimization algorithm has the computational advantage of being both quite fast and rather straightforward to implement over other optimization algorithms proposed in the literature for the computation of the MAP estimate [2, 3, 6, 9, 8, 10].

A comment on the selection of the parameter N in the basis function representation of the signal is now in order. Clearly, if N is too small, the basis functions may not be able to adequately represent the input signals, and one might believe that choosing N large could make the problem unstable, in view of its inherent ill-conditioning. While a systematic statistical study of an optimal choice of N is beyond the scope of this article, to test the robustness of the proposed MAP estimator algorithm with respect to N, we solved the problem repeatedly for various values of N. The results, shown in Figure 3, indicate that the solution remains quite stable as N increases, thus suggesting that the truncation index should be always chosen large enough to well represent the input signal. On the other hand, since choosing N too large would only add to the computational load without improving the results, the selection of the truncation index should take into consideration both the computational complexity and representation power of the basis functions.

Finally, we applied the algorithm to the analysis of experimental data published in [3]. The estimated mitochondrial oxygen consumption during muscle state transitions and the corresponding output of the forward model versus the data are shown in Figure 4. The estimates are in good agreement with the expected behavior of the true mitochondrial oxygen consumption. The actual resting oxygen consumption was estimated at 0.005 nmoles/min from the data, and increased to a five-fold value during the peak muscle stimulation. For details of the significance of this result, we refer the interested readers to [3]. We remark that since in the real data, the initial oxygen concentration in the chamber as well as the actual noise level in the data were not precisely known, the estimate of the actual initial oxygen concentration in the chamber was the average of the readings in the first 15 seconds of the experiment. The value of the noise level in the data was also based on the standard deviation the readings in the first 15 seconds.

Figure 4.

Estimated flux of mitochondrial oxygen consumption and corresponded model fit to real data in Ref. [3].

5 Discussion and Conclusion

The article proposes an optimization algorithm for solving deconvolution problems with the impulse response depending on unknown parameters. The approach is therefore applicable to blind deconvolution problems as well. The optimization step estimates also the hyperparameters of the prior density. A comparison of the computed result with the solution obtained by using sampling-based approaches which are more time-consuming are also discussed. Although the number of unknowns to be estimated is not very high, MCMC based algorithms become easily prohibitively time-consuming since each evaluation of the forward map is tantamount to solving a partial differential equation describing the oxygen diffusion. In our comparison we use a block form of the Gibbs sampler, and to keep the computation time for the simulation reasonable, we used a partial linearization of the problems which could be used also as an effective proposal move for a Metropolis-Hastings sampling algorithm. Since the focus in this work is in the optimization approach, this approach is not discussed further here. We remark that since the proposed optimization algorithm does not require the derivatives in analytic form, it does not not require any such approximation. The optimization algorithm is applied to estimating the mitochondrial oxygen consumption in a numerically simulated in vitro setup.

Our algorithm is able to produce less oscillatory solutions than approaches described earlier in the literature and at considerably smaller computational cost, see, e.g., [3] and references therein.

The present algorithm parametrizes the solution via a truncated Fourier series. However, the Fourier representation is not necessary for the algorithm and other basis functions can be chosen instead. The algorithm can be applied to other deconvolution problems simply by replacing the forward model of the system and providing the derivatives of the model with respect to the parameters of interest needed to calculate the Hessian, as explained in the Appendix. Extensions of this work should include statistical modeling of the truncation error in the construction of the likelihood, which would improve the performance of the approach and diminish the dependency of the model on the truncation parameter N, e.g., see [24, 26].

Appendix: Gradient and Hessian

This appendix gives the necessary details for calculating the gradient and Hessian approximation that are used in the quasi-Newton algorithm for finding the MAP estimate. For the sake of completeness, we assume that the parameter vector θ may have dimension K ≥ 1. To simplify the notation, we suppress the subindex k from F_k and its derivatives.

The first K components of G(ξ) are the partial derivatives of F(ξ) with respect to [ξ₁, …, ξ_K] = [θ₁, …, θ_K]. Since only the first term in the right hand side of Eq. (23) is a function of θ₁, …, θ_K, we have

$$G_i(\xi )=\frac{\partial }{\partial {\xi }_i}F(\xi )=-{\left[\frac{\partial }{\partial {\theta }_i}\psi (\theta ,\alpha )\right]}^T{S}^2[y-\psi (\theta ,\alpha )],\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,K.$$ (29)

The partial derivatives of F(ξ) with respect to [ξ_K+1, …, ξ_K+N ] = [α₁, …, α_N ] are

$$G_{K+i}(\xi )=\frac{\partial }{\partial {\xi }_{K+i}}F(\xi )=-{\left[\frac{\partial }{\partial {\alpha }_i}\psi (\theta ,\alpha )\right]}^T{S}^2[y-\psi (\theta ,\alpha )]+\frac{1}{{\lambda }_k}{\alpha }_i{\beta }_i^2,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}i=1,\dots ,N.$$ (30)

Similarly, the Hessian H ∈ ℝ ^(K+N)×(K+N) is

$$\begin{array}{l}{H}_{i,j}(\xi )=\frac{{\partial }^{2}F}{\partial {\xi }_i\partial {\xi }_j}(\xi )={\left[\frac{\partial \psi }{\partial {\xi }_i}(\xi )\right]}^T{S}^2\left[\frac{\partial \psi }{\partial {\xi }_j}(\xi )\right]-\frac{1}{2}{\left[\frac{{\partial }^2\psi }{\partial {\xi }_i\partial {\xi }_j}(\xi )\right]}^T{S}^2[y-\psi (\xi )]\\ +\frac{{\partial }^2}{\partial {\xi }_i\partial {\xi }_j}\left(\frac{1}{2{\lambda }_k}\sum _{i=1}^N{\alpha }_i^2{\beta }_i^2\right).\end{array}$$ (31)

The computation of the second order partial derivatives of the model Ψ(ξ) with respect to the parameters ξ can be expensive. In view of the fact that, when a good fit of the model to the data can be achieved, the residual y – Ψ(ξ) becomes small, instead of the full Hessian, we use an approximation obtained by ignoring the second term in Eq. (31). Hence, we write

$$H(\xi )\approx \left[\begin{array}{ll}{R}_{11}\hfill & R_{12}\hfill \\ R_{21}\hfill & R_{22}+(1/{\lambda }_k)B^2\hfill \end{array}\right],$$ (32)

where the blocks R_ij are given as

$$\begin{array}{l}{R}_{11,ij}={\left[\frac{\partial \psi }{\partial {\theta }_i}\right]}^T{S}^2\left[\frac{\partial \psi }{\partial {\theta }_j}\right],\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}1\le i,j\le K,\\ R_{12,ij}=R_{21,ji}={\left[\frac{\partial \psi }{\partial {\theta }_i}\right]}^T{S}^2\left[\frac{\partial \psi }{\partial {\alpha }_j}\right],\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}1\le i\le K,\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}1\le j\le N,\\ R_{22,ij}={\left[\frac{\partial \psi }{\partial {\alpha }_i}\right]}^T{S}^2\left[\frac{\partial \psi }{\partial {\alpha }_j}\right],\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}1\le i,j\le N,\end{array}$$

and B = diag(β). We remark that since the matrices R₁₁ and R₂₂ are symmetric and $R_{21}=R_{12}^T$ the matrix R is symmetric.

It is easy to show that the gradient vector G and Hessian matrix H can be factorized into

$$G=J^{T}V\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{and\hspace{0.17em}}\hspace{0.17em}\hspace{0.17em}H=J^{T}J,$$ (33)

where

$$J=-\left[\begin{array}{cc}S\left[\frac{\partial \psi }{\partial \theta }\right]& S\left[\frac{\partial \psi }{\partial \alpha }\right]\\ 0& \frac{B}{\sqrt{\lambda }}\end{array}\right]\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{and\hspace{0.17em}}\hspace{0.17em}\hspace{0.17em}V=\left[\begin{array}{c}S[y-\psi ]\\ \frac{B\alpha }{\sqrt{\lambda }}\end{array}\right].$$ (34)

Thus the Hessian matrix H is symmetric and positive definite. Also the associated linear system (27) for the quasi-Newton step δξ_c forms a normal equation which can be solved very fast and efficiently using the Conjugate Gradient Least Square (CGLS) or MINimum RESidual (MINRES) algorithm [22, 23].