A COMPUTATIONAL MEASURE THEORETIC APPROACH TO INVERSE SENSITIVITY PROBLEMS II: A POSTERIORI ERROR ANALYSIS

Abstract

In part one of this paper [T. Butler and D. Estep, SIAM J. Numer. Anal., to appear], we develop and analyze a numerical method to solve a probabilistic inverse sensitivity analysis problem for a smooth deterministic map assuming that the map can be evaluated exactly. In this paper, we treat the situation in which the output of the map is determined implicitly and is difficult and/or expensive to evaluate, e.g., requiring the solution of a differential equation, and hence the output of the map is approximated numerically. The main goal is an a posteriori error estimate that can be used to evaluate the accuracy of the computed distribution solving the inverse problem, taking into account all sources of statistical and numerical deterministic errors. We present a general analysis for the method and then apply the analysis to the case of a map determined by the solution of an initial value problem.

1. Introduction

In part one of this paper [4], we develop and analyze a numerical method to solve an inverse stochastic sensitivity analysis problem for a smooth deterministic map. Namely, given a (probability) measure on the output of the map, compute the (probability) measure on the input space (comprising data and/or parameters) that produce the output measure. This is the stochastic version of the deterministic inverse problem for the map, and it is also the direct inversion of the forward stochastic sensitivity analysis problem for the map. As such, it deals directly with the inverse of the map in question, rather than, say, a statistical model of the output of the map.

In [4], we formulate this inverse problem using the law of total probability and then analyze an approximate solution method assuming that the map in question can be evaluated exactly. The solution method borrows heavily from techniques used inmeasure theory. The computed solution provides a systematic method for approximating the probability of any specified event in the input space.

However, our interest lies in situations in which the output of the map is determined implicitly and is difficult and/or expensive to evaluate, e.g., requiring the solution of a differential equation. In addition, we wish to consider the situation in which the measure on the output of the map is only described approximately using a finite number of samples. These practical discretization choices introduce additional numerical errors that affect the computed inverse distribution.

In this paper, we carry out an analysis of the effects of these two numerical discretizations on the computed inverse distribution. As a consequence of the analysis, we prove that the numerical error of the approximate parameter density computed from the algorithm for solving the inverse problem converges to zero as the discretization (both statistical and deterministic) converge to zero. But our main goal is the derivation of an a posteriori error estimate that can be used to evaluate the accuracy of the computed distribution solving the inverse problem, taking into account all sources of statistical and numerical deterministic errors.

While our particular interest is a numerical analysis of the method constructed for the inverse problem in [4], aspects of the analysis we present hold general interest. We present a general error analysis for a computed probability distribution accounting for the effects of finite sampling and errors in each sample resulting from evaluation of a numerically approximated map. Because of the application to inverse problems, we also require error estimates for gradients of quantities of interest computed from numerical solutions of ordinary differential equations, which again is of interest in other contexts, e.g., applications to optimization.

1.1. The inverse problem

As mentioned, the problem we study in [4] is a direct inversion of the forward stochastic sensitivity problem for a deterministic model. We consider an operator q(λ) that maps values in a parameter (and data) space Λ to an output space D. We assume there is a parameter volume measure μ_Λ on Λ that determines the volume of sets in Λ. The volume measure depends on the units of measure used for the parameters and also reflects the structural dependency among the parameters, e.g., depending on whether μ_Λ is a product measure. The volume measure is specified as part of the model that defines the map q(λ) since the parameters must be explicitly defined in the physical model that determines q. We assume that μ_Λ is absolutely continuous with respect to the Lebesgue measure and the volume V of Λ is finite.

The deterministic model can be expressed in terms of a likelihood function L(q|λ) of the output q values given the input parameter values λ, where L(q|λ) = δ(q–q(λ)) is the unit mass distribution at q = q(λ). If we specify a density σ_Λ(λ) on the parameter space Λ, then the law of total probability implies

$${\rho }_{\mathcal{D}}\left(q\right)={\int }_{\Lambda }L(q\mid \lambda ){\sigma }_{\Lambda }\left(\lambda \right)d{\mu }_{\Lambda }\left(\lambda \right).$$ (1.1)

The stochastic inverse sensitivity analysis problem that we study is the inversion of the integral equation (1.1). Namely, we assume that an observed probability density ${\rho }_{\mathcal{D}}\left(q\left(\lambda \right)\right)$ is given on the output value q(λ) and we seek to compute the corresponding parameter density σ_Λ(λ) that yields ${\rho }_{\mathcal{D}}\left(q\left(\lambda \right)\right)$ via (1.1).

1.2. The solution method

In [4], we present a computational measure theoretic algorithm to approximate the solution of the inverse problem (1.1) by a simplefunction σ_Λ,M’(λ) with respect to a partition ${\left\{b_i\right\}}_{i=1}^{M^{\prime }}$ of Λ. Paraphrasing the main result from [4], we have the following theorem.

Theorem 1.1. Given a measurable set A ⊂ Λ, we can approximate P(A) using the simple function

$${\sigma }_{\Lambda ,M^{\prime }}\left(\lambda \right)=\sum _{k=1}^{M^{\prime }}\frac{P\left(b_i\right)}{{\mu }_{\Lambda }\left(b_i\right)}{1}_{b_i}\left(\lambda \right).$$ (1.2)

The constructive proof yields a computational algorithm that generates a probability P(b_i) for each cell b_i, using only calculations of volumes in Λ. The main steps of the algorithm are based on the following observations:

• The probability of an interval of output data [q_m, q_M) $\subset \mathcal{D}$ is equal to the probability of the region generalized contours defined by A = q⁻¹([q_m, q_M)).

• If ${\rho }_{\mathcal{D}}\left(q\right)$ is constant on [q_m, q_M), then the probability of b∩A for any event b ⊂ Λ is equal to the probability of A times the ratio of volumes μ_Λ(b∩A)/μ_Λ(A).

Thus, the algorithm proceeds by first approximating ${\rho }_{\mathcal{D}}\left(q\right)$ with a simple function, which induces regions of contours with probabilities defined by the approximate output density. Then, the ratios of volumes for each cell in the partition ${{\{}_{b_i}\}}_{i=1}^{M^{\prime }}$ are computed with respect to all the induced regions of contours. From this, we obtain P(b_i) for each cell and obtain (1.2).

The main focus of the analysis in [4] is on the convergence of the approximate representation to the true representation on a given partition ${{\{}_{b_i}\}}_{i=1}^{M^{\prime }}$ assuming that map the map is evaluated exactly.

1.3. Sources of error

In this paper, we analyze and estimate errors affecting the values {P(b_i)} in the representation (1.2) for a fixed partition ${{\{}_{b_i}\}}_{i=1}^{M^{\prime }}$. Since we fix the partition ${{\{}_{b_i}\}}_{i=1}^{M^{\prime }}$, we simplify the notation in [4] by dropping the hat on the piecewise-linear representation q̂(λ) of q(λ).

In particular, we consider two sources of error that affect the approximation of the representation σ_Λ,M’(λ). The first is “statistical error” that arises if the observed probability density ${\rho }_{\mathcal{D}}\left(q\left(\lambda \right)\right)$ is known only through a finite collection of random samples. This type of error affects the left-hand side of (1.1). For example, finite sampling of the distribution of random variable q(λ) is used when the observed distribution is complicated to evaluate or when it is determined by experimental observations. Given an analytic, easy-to-evaluate distribution function for q(λ), we need not perform any sampling.

The second source of error arises when we use numerical approximations in the evaluation of the map q, e.g., as happens if q involves solving a differential equation. This means that we use approximate values of q and its gradient to form the approximate representation q̂(λ) ≈ q̂(λ). This source of error affects the evaluation of the likelihood function in (1.1).

In this paper, we present two kinds of error analysis. We give an a priori convergence analysis that shows that the error tends to zero as the discretization is refined. This analysis uses error bounds that are robust in the sense of holding under general conditions but which are generally orders of magnitude too large for particular computed solutions. Our main purpose in this paper is to give an a posteriori error analysis that provides the means to compute an relatively accurate error estimate on any particular computed solution. The latter result is important for the purposes of uncertainty quantification and for distributing computational resources in order to achieve a desired accuracy with efficiency.

We let F(t) denote the probability distribution on Λ that represents (1.2), where $t\in {\mathbb{R}}^d$, and

$$F\left(t\right)=P\left(\{\lambda \mid \lambda \le t\}\right)=P(\lambda \ge t).$$ (1.3)

Here the inequality, λ ≤ t, is considered componentwise. We use F_q(t) to denote the probability distribution function of q(λ). To simplify the presentation, we assume b_i is contained in a region of contours A_i induced by the simple function approximation to ${\rho }_{\mathcal{D}}\left(q\right)$. If no sampling is used to evaluate ${\rho }_{\mathcal{D}}\left(q\right)$ or F_q(t), then the algorithm yields

$$P\left(b_i\right)=F_q\left(q\left(b_i\right)\right)\frac{{\int }_{b_i}d{\mu }_{\Lambda }\left(\lambda \right)}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)},1\le i\le M^{\prime },$$ (1.4)

where q(b_i) = {q(λ), λ ∈ b_i}. (If b_i ⊄ A_i, then we alter (1.4) to sum over the regions of induced contours A_j such that b_i ∩ A_j ≠ ∅.) Using (1.4) in (1.3) gives

$$F\left(t\right)=\sum _{i=1}^{M^{\prime }}{F}_q\left(q\left(b_i\right)\right)\frac{{\int }_{b_i\cap \{\lambda \le t\}}d{\mu }_{\Lambda }\left(\lambda \right)}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)}.$$ (1.5)

For the first source of error, we let F_q(t) denote a sample distribution function computed from a finite collection of error-free sample values {Q₁, ∆ , $Q_{\mathcal{N}}$},

$$F_{q,\mathcal{N}}\left(t\right)=\frac{1}{\mathcal{N}}\sum _{n=1}^{\mathcal{N}}1(Q_n\le t).$$

This leads to an approximation of $F_{\mathcal{N}}\left(t\right)\approx F\left(t\right)$ defined

$$F_{\mathcal{N}}\left(t\right)=\sum _{i=1}^{M^{\prime }}{F}_{q,\mathcal{N}}\left(q\left(b_i\right)\right)\frac{{\int }_{b_i\cap \{\lambda \le t\}}d{\mu }_{\Lambda }\left(\lambda \right)}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)}.$$

Next we consider the use of an approximation q̂(λ) ≈ q(λ), which leads to an error in computation of $F_{q,\mathcal{N}}\left(t\right)$. We define the approximate sample distribution function ${\tilde{F}}_{\mathcal{N}}\left(t\right)$ as

$${\tilde{F}}_{\mathcal{N}}\left(t\right)=\sum _{i=1}^{M^{\prime }}{F}_{q,\mathcal{N}}\left(\tilde{q}\left(b_i\right)\right)\frac{{\int }_{b_i\cap \{\lambda \le t\}}d{\mu }_{\Lambda }\left(\lambda \right)}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)}.$$ (1.6)

We calculate probabilities using (1.6) and seek to determine the error $F\left(t\right)-{\tilde{F}}_{\mathcal{N}}\left(t\right)$. We decompose the error to get

(1.7)

2. General error analysis for a computed probability distribution

We begin by bounding ${\mathcal{E}}_S$. We conduct an a posteriori analysis similar to that used for nonparametric density estimation for elliptic problems with randomly perturbed diffusion coefficients in [13]. The error in the distribution is bounded by

$${\mathcal{E}}_S\left(t\right)\le \sum _{i=1}^{M^{\prime }}\mid F_q\left(q\left(b_i\right)\right)-F_{q\mathcal{N}}\left(q\left(b_i\right)\right)\mid \frac{{\int }_{b_i\cap \{\lambda \le t\}}d{\mu }_{\Lambda }\left(\lambda \right)}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)}.$$ (2.1)

Using standard statistical arguments [13], for any ∊ > 0,

$$\underset{t\in \mathbb{R}}{\text{sup}}\mid F_q\left(t\right)-F_{q,\mathcal{N}}\left(t\right)\mid \le 2{\left(\frac{\log \left({∊}^{-1}\right)}{2\mathcal{N}}\right)}^{1∕2}$$ (2.2)

with probability greater than 1 – ∊. It is possible to prove other forms of this bound [13]. Using (2.2) in (2.1) yields for any ∊ > 0,

$${\mathcal{E}}_S\left(t\right)\le 2{\left(\frac{\log \left({∊}^{-1}\right)}{2\mathcal{N}}\right)}^{1∕2}\sum _{i=1}^{M^{\prime }}\frac{{\int }_{b_i\cap \{\lambda \le t\}}d{\mu }_{\Lambda }}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)}\le 2{\left(\frac{\log \left({∊}^{-1}\right)}{2\mathcal{N}}\right)}^{1∕2}$$ (2.3)

with probability greater than 1 – ∊.

For ${\mathcal{E}}_D$, we assume a bound or estimate E_i for the error in q̂(b_i) on each cell b_i. More precisely, the piecewise linear function q is defined on the partition {B_i} of Λ, where q(λ) = q(μ_i) + ∇q(μ_i)(λ – μ_i) on B_i and μ_i is a chosen value in B_i, and q̂(λ) = q̂(μ_i) + ∇q̂(μ_i)(λ – μ_i) on B_i. Hence the error has the form

$$q\left(\lambda \right)-\tilde{q}\left(\lambda \right)=(q\left({\mu }_i\right)-\tilde{q}\left({\mu }_i\right))+(\nabla q\left({\mu }_i\right)-\nabla \tilde{q}\left({\mu }_i\right))(\lambda -{\mu }_i)\text{on}{B}_i.$$ (2.4)

Hence, we require estimates or bounds for the errors in both q̂(μ_i) and ∇q̂(μ_i), respectively. The derivation of the a priori bound or a posteriori error estimate is specific to a particular map q. In section 3, we derive the necessary estimates for nonlinear ordinary differential equations. Similar results hold for elliptic problems [5].

For convenience, we choose the fine partition {b_i} so that for each 1 ≤ i ≤ M’, b_i ⊂ B_j for some 1 ≤ j ≤ M. Thus, for all cells b_i ⊂ B_j for a fixed j, there is the same deterministic error term associated with q̂(b_i). We let E_j, 1 ≤ j ≤ M, denote the deterministic error associated with each q̂(b_i) for all b_i ⊂ B_j. Using an analogous argument as in [13],

$$\begin{array}{cc}\hfill {\mathcal{E}}_D& \le \sum _{i=1}^{M^{\prime }}\mid F_{q,\mathcal{N}}(M_i+E)-F_{q,\mathcal{N}}(m_i-E)\mid \frac{{\int }_{b_i\cap \{\lambda \le t\}}d{\mu }_{\Lambda }\left(\lambda \right)}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)}\hfill \\ \hfill & \le \sum _{i=1}^{M^{\prime }}\mid F_q(M_i+E)-F_{q,\mathcal{N}}(m_i+E)\mid \frac{{\int }_{b_i\cap \{\lambda \le t\}}d{\mu }_{\Lambda }\left(\lambda \right)}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)}\hfill \\ \hfill & +\sum _{i=1}^{M^{\prime }}\mid F_q(m_i-E)-F_{q,\mathcal{N}}(m_i-E)\mid \frac{{\int }_{b_i\cap \{\lambda \le t\}}d{\mu }_{\Lambda }\left(\lambda \right)}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)}\hfill \\ \hfill & +\sum _{i=1}^{M^{\prime }}\mid F_q(m_i+E)-F_q(m_i-E)\mid \frac{{\int }_{b_i\cap \{\lambda \le t\}}d{\mu }_{\Lambda }\left(\lambda \right)}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)},\hfill \end{array}$$

where E = max_j |E_j|, M_i = max q(b_i), and m_i = min q(b_i).

Using (2.2) for the first two terms on the right-hand side of the inequality we have that for any ∊ > 0

$${\mathcal{E}}_D\le 2{\left(\frac{\log \left({∊}^{-1}\right)}{2\mathcal{N}}\right)}^{1∕2}+\sum _{i=1}^{M^{\prime }}\mid F_q(M_i+E)-F_q(m_i-E)\mid \frac{{\int }_{b_i\cap \{\lambda \le t\}}d{\mu }_{\Lambda }\left(\lambda \right)}{{\int }_{A_i}d{\mu }_{\Lambda }\left(\lambda \right)}$$

with probability greater than 1 – ∊. Assuming Lipschitz continuity of the distribution F_q with constant L, for any ∊ > 0,

$${\mathcal{E}}_D\le 2{\left(\frac{\log \left({∊}^{-1}\right)}{2\mathcal{N}}\right)}^{1∕2}+L\underset{1\le i\le M^{\prime }}{\text{max}}(M_i-m_i)\times E$$

with probability greater than 1 – ∊.

Putting together the bounds yields the next theorem.

Theorem 2.1. For any ∊ > 0,

$$\mid F\left(t\right)-{\tilde{F}}_{\mathcal{N}}\left(t\right)\mid \le 2{\left(\frac{\log \left({∊}^{-1}\right)}{2\mathcal{N}}\right)}^{1∕2}+L\underset{1\le i\le M^{\prime }}{\text{max}}(\text{max}q\left(b_i\right)-\text{min}q\left(b_i\right))\times E$$ (2.5)

with probability greater than 1 – ∊. If no sampling is used to evaluate ${\rho }_{\mathcal{D}}\left(q\right)$ or F_q(t), then

$$\mid F\left(t\right)-\tilde{F}\left(t\right)\mid \le L\underset{1\le i\le M^{\prime }}{\text{max}}(\text{max}q\left(b_i\right)-\text{min}q\left(b_i\right))\times E.$$ (2.6)

Note that F̂(t) is the distribution calculated using exact values of ${\rho }_{\mathcal{D}}\left(q\right)$ but approximate values of q.

3. Application to nonlinear ordinary differential equations

We apply the general error analysis to a finite dimensional map q defined implicitly by the solution to a differential equation that depends on a finite number of parameters in the model. We consider the initial value problem

$$\{\begin{array}{cc}\stackrel{.}{y}=f(y;{\lambda }_1),\hfill & t>0,\hfill \\ y\left(0\right)={\lambda }_0,\hfill & \hfill \end{array}\phantom{\}}$$ (3.1)

where $y\in {\mathbb{R}}^n$, $f:{\mathbb{R}}^{n+p}\to {\mathbb{R}}^n$ is smooth, and $\lambda ={({\lambda }_1^{⊺},{\lambda }_0^{⊺})}^{⊺}\in \Lambda \subset {\mathbb{R}}^d(d=p+n)$ are the parameters. We solve (3.1) to calculate a linear functional of the solution, or a quantity of interest,

$$q\left(y\right)={\int }_0^T\langle y,\psi \rangle dt.$$ (3.2)

We assume that the solution y of (3.1) depends (implicitly) on parameters λ in a smooth way and denote solutions of (3.1) as y_λ and the quantity of interest as q(λ) to emphasize the implicit dependence of the quantity of interest on the parameters. The smooth dependence of solutions to (3.1) on parameters λ implies the dependence of the quantity of interest on λ is also smooth.

3.1. Construction of the piecewise-linear representation

Computing the gradient information is problematic for a differential equation. We use an adjoint equation and variational analysis to do this implicitly. We solve the initial value problem at a reference parameter value $\mu ={({\mu }_1^{⊺},{\mu }_0^{⊺})}^{⊺}$,

$$\{\begin{array}{cc}{\stackrel{.}{y}}_{\mu }=f(y_{\mu };{\mu }_1),\hfill & t>0,\hfill \\ y\left(0\right)={\mu }_0,\hfill & \hfill \end{array}\phantom{\}}$$ (3.3)

where (y_μ, μ) is a reference point. We define the exact adjoint problem,

$$\{\begin{array}{cc}-\stackrel{.}{\varphi }-D_{y}f{(y_{\mu };{\mu }_1)}^T\varphi =\psi ,\hfill & T>t\ge 0,\hfill \\ \varphi \left(T\right)=0.\hfill & \hfill \end{array}\phantom{\}}$$ (3.4)

The following theorem [25] relates the value of q(λ) to q(μ) for λ near μ.

Theorem 3.1. If f(y; λ) is twice continuously differentiable with respect to both y and λ and Lipschitz continuous in both y and λ, then the quantity of interest is Fréchet differentiable at (y_μ, μ) with derivative $\nabla q\left(\mu \right):{\mathbb{R}}^d\to \mathbb{R}$ given by

$$\nabla q\left(\mu \right)\left[\lambda \right]=\langle ({\lambda }_0-{\mu }_0),\varphi \left(0\right)\rangle +{\int }_0^T\langle D_{{\lambda }_1}f(y_{\mu };\mu )({\lambda }_1-{\mu }_1),\varphi \rangle dt.$$ (3.5)

Additionally,

$$q\left(\lambda \right)\approx q\left(\mu \right)+\nabla q\left(\mu \right)\left[\lambda \right].$$ (3.6)

In the absence of numerical error,

$$q\left(\lambda \right)\approx {\int }_0^T\langle y_{\mu },\psi \rangle dt+\langle ({\lambda }_0-{\mu }_0),\varphi \left(0\right)\rangle +{\int }_0^T\langle D_{{\lambda }_1}f(y_{\mu };{\mu }_1)({\lambda }_1-{\mu }_1),\varphi \rangle dt$$ (3.7)

for λ close to μ.

The global piecewise-linear approximation on the partition ${\left\{B_i\right\}}_{i=1}^M$ of Λ is constructed by using the local linearization on each cell B_i to obtain

$$\widehat{q}\left(\lambda \right)≔\sum _{i=1}^M(q\left({\mu }_i\right)+\langle {\nabla }_q\left({\mu }_i\right),(\lambda -{\mu }_i)\rangle )1_{B_i}\left(\lambda \right),$$ (3.8)

where μ_i is the reference parameter value chosen in cell B_i.

3.2. Discretization

The a posteriori error estimate uses a variational analysis after introducing an adjoint problem. The variational analysis makes it natural to write the discretization method in the finite element framework. This is not restrictive as most common finite difference schemes can be written as a finite element method with a particular choice of quadrature for evaluating integrals.

A finite element method is based on the variational formulation of the differential equation. For the differential equation,

$$\{\begin{array}{cc}\stackrel{.}{x}=g(x,t),\hfill & 0<t\le T,\hfill \\ x\left(0\right)=x_0,\hfill & \hfill \end{array}\phantom{\}}$$ (3.9)

the problem is to find $x\in {\mathcal{C}}^1\left([0,1]\right)$ such that

$$\{\begin{array}{cc}{\int }_0^t(\stackrel{.}{x},v)ds={\int }_0^t(g(x,t),v)ds,\hfill & t>0,\hfill \\ x\left(0\right)=x_0\hfill & \hfill \end{array}\phantom{\}}$$ (3.10)

for all $v\in {\mathcal{C}}^1\left([0,1]\right)$. (We use g instead of f because there are several problems that have to be solved below.)

We compute a solution on the interval [0, T], and we discretize the interval 0 = t₀ < t₁ < … < t_N = T with time intervals I_j = (t_j–1, t_j) and time steps k_j = t_j–t_j–1. The finite element method produces a piecewise polynomial approximation. We use ${\mathcal{P}}^q\left(I_j\right)$ to denote the space of polynomials of degree q and less on time interval I_j and define the space of piecewise polynomials,

$${\mathcal{V}}^{\left(q\right)}=\{U:U\mid I_j\in {\mathcal{P}}^{\left(q\right)}\left(I_j\right),j=1,2,3,\dots ,\}.$$

We consider the discontinuous Galerkin (dGq) finite element method that produces an approximate solution $X\in {\mathcal{V}}^{\left(q\right)}$ [18]. Since X may be discontinuous at time nodes, we define $X_{j=1}^{+}={\text{lim}}_{t\downarrow t_{j-1}}X\left(t\right),X_{j-1}^{-}={\text{lim}}_{t\uparrow t_{j-1}}X\left(t\right)$, and ${\left[X\right]}_{j-1}=X_{j-1}^{+}-X_{j-1}^{-}$. The approximation is computed interval by interval. We set X₀ = x₀. Then we compute $X\in {\mathcal{P}}^q(t_{j-1},t_j)$ successively for j = 1, 2, … , N, satisfying

$${\int }_{t_{j-1}}^{t_j}(\stackrel{.}{X},v)ds+({\left[X\right]}_{j-1},v_{j-1}^{+})={\int }_{t_{j-1}}^{t_j}(g(X,t),v)ds,\text{for}\text{all}v\in {\mathcal{P}}^q(t_{j-1},t_j).$$ (3.11)

Remark 3.1. If g(x, t) ≡ g(x) and q = 0, the dG0 approximation matches the backward Euler approximation at the time nodes. In general, we may obtain various difference schemes, e.g., the subdiagonal Pade schemes, by employing quadrature to evaluate the integrals in (3.11) [17, 27]. There is also a continuous Galerkin (cG) approximation that produces yet other classes of approximations [9]. We carry out the analysis below for the dG scheme assuming the integrals in (3.11) are computed exactly. The extension of the a posteriori analysis to handle quadrature and the cG method is straightforward [12] and we do not discuss this further.

3.3. The effect of using an approximate solution on the piecewise-linear representation

The main interest is in treating the effects of using a numerical approximation Y_μ in the linearization of the forward problem used to construct an adjoint. We define the approximate adjoint using (3.4) with “perturbed” operator D_yf(Y_μ; μ₁),

$$\{\begin{array}{cc}-\stackrel{.}{\Phi }-D_{y}f{(Y_{\mu };{\mu }_1)}^T\Phi =\psi ,\hfill & T>t\ge 0,\hfill \\ \Phi \left(T\right)=0.\hfill & \hfill \end{array}\phantom{\}}$$ (3.12)

We assume f(y; λ) is twice continuously differentiable with respect to both y and λ, so that standard convergence results for Y_μ imply that over some (short) time interval [0, T],

$${\parallel D_{y}f(y_{\mu };{\mu }_1)-D_{y}f(Y_{\mu };{\mu }_1)\parallel }_V\le K{\parallel y_{\mu }-Y_{\mu }\parallel }_U,$$ (3.13)

where ∥·∥_V and ∥·∥_U are the L²([0, T]) norm of some appropriate matrix and vector norms of the arguments, respectively.

Let q̂(λ) denote the approximate quantity of interest calculated using (3.6) with Y_μ and Φ in place of y_μ and φ,

$$\tilde{q}\left(\lambda \right)={\int }_0^T\langle Y_{\mu },\psi \rangle dt+\langle ({\lambda }_0-{\mu }_0),\Phi \left(0\right)\rangle +{\int }_0^T\langle D_{{\lambda }_1}f(Y_{\mu };{\mu }_1)({\lambda }_1-{\mu }_1),\Phi \rangle dt$$ (3.14)

with error q(λ) – q̂(λ). Taking the difference of (3.7) and (3.14) gives

(3.15)

Term I is a linear functional of the error y_μ – Y_μ and it can be estimated using standard a posteriori analysis techniques as described below. Term II measures theeffect of using Y_μ and Φ on the sensitivity of q(λ) to changes in the initial conditions. Term III measures the effect of using Y_μ and Φ on the sensitivity of q(λ) to changes in model parameters.

The terms II and III depend linearly on the vector λ – μ. The analysis below produces estimates that also depend on this vector linearly so that the error estimates for these terms are also linear functions of this vector. Thus, following the analysis described below for p linearly independent vectors λ – μ, we obtain a set of error estimates such that the error defined by II and III for any vector λ – μ can be written as a linear combination from this set of error estimates.

3.4. Convergence and order of accuracy

We can use straightforward a priori error analysis on (3.15) to show that |q(λ) – q̂(λ) converges at the same order as the dGq method over a short time period under the assumption that f is twice continuously differentiable.

3.5. Estimate of the error in a quantity of interest

We compute an a posteriori error estimate using variational analysis and adjoint problems [9, 18, 7, 8, 12, 27, 6]. We begin by recalling the a posteriori estimate of error in a quantity of interest. Let X denote the dGq approximation to (3.9) and let e = x – X, where x solves (3.9) exactly. We linearize around X in the sense of perturbing the operator to arrive at the adjoint problem

$$\{\begin{array}{cc}-\stackrel{.}{v}={\overline{g^{\prime }(x,X)}}^T\varphi +{\psi }_1\left(t\right),\hfill & T>t\ge 0,\hfill \\ v\left(T\right)={\psi }_2,\hfill & \hfill \end{array}\phantom{\}}$$ (3.16)

where $\overline{g^{\prime }(x,X)}={\int }_0^1{g}^{\prime }(sx+(1-s)X,t)$. For simplicity, we use g^’ for $\overline{g^{\prime }(x,X)}$ below. If ψ₁(t) ≡ 0, then the quantity of interest is (e(T), ψ₂). If ψ₂ = 0, then the quantity of interest is ${\int }_0^T(e\left(t\right),{\psi }_i\left(t\right))$ dt.

Assume ψ₁(t) ≡ 0 in (3.16). Take the inner product of the adjoint problem with e and integrate from 0 to T to obtain

$$-{\int }_0^T(\stackrel{.}{v},e)dt-{\int }_0^T({\left(g^{\prime }\right)}^{T}v,e)dt=0.$$ (3.17)

We decompose (3.17) into a sum of integral equations over each time interval in the discretization and integrate by parts over each interval to get

$$-\sum _{n=1}^N(e,v){\mid }_{t_{n-1}}^{t_n}+\sum _{n=1}^N{\int }_{t_{n-1}}^{t_n}(v,\stackrel{.}{e})dt-\sum _{n=1}^N{\int }_{t_{n-1}}^{t_n}(v,g^{\prime }e)dt=0.$$ (3.18)

Since e = x – X might be discontinuous at the boundaries of each interval, we expand the first term on the right-hand side of (3.18) to

$$-\sum _{i=1}^N(e,v){\mid }_{t_{n-1}}^{t_n}=(e\left(0\right),v\left(0\right))+\sum _{n=2}^N({\left[X\right]}_{n-1},v_{n-1})-(e\left(T\right),v\left(T\right))$$ (3.19)

with  _n–1 = ∂(t_n–1). Substitution of (3.19) into (3.18) and rearranging the terms yields

$$(e\left(T\right),v\left(T\right))=(e\left(0\right),v\left(0\right))+\sum _{n=1}^N({\left[X\right]}_{n-1},v_{n-1})+\sum _{n=1}^N{\int }_{t_{n-1}}^{t_n}(v,\stackrel{.}{e})dt-{\int }_{t_{n-1}}^{t_n}(v,g^{\prime }e)dt.$$

Substituting e = x – X and using ẋ – g(x) = 0 gives

$$(e\left(T\right),v\left(T\right))=(e\left(0\right),v\left(0\right))+\sum _{n=2}^N({\left(X\right)}_{n-1},v_{n-1})-\sum _{n=1}^N{\int }_{t_{n-1}}^{t_n}(\stackrel{.}{X}-g\left(X\right),v)dt.$$ (3.20)

Similarly, if ψ₂ = 0 and ψ₁(t) is nonzero for some t ∈ (0, T), we obtain

$${\int }_0^T(e,{\psi }_1)dt=(e\left(0\right),v\left(0\right))+\sum _{n=2}^N({\left[X\right]}_{n-1},v_{n-1})-\sum _{n=1}^N{\int }_{t_{n-1}}^{t_n}(\stackrel{.}{X}-g\left(X\right),v)dt.$$ (3.21)

We summarize as the following theorem.

Theorem 3.2. A computable estimate of the error in a quantity of interest of (3.9) is obtained by solving (3.16) and computing either (3.20) or (3.21).

Implementation details

Using the a posteriori estimate involves several important practical considerations. We discuss two.

Often “Galerkin orthogonality” is used to introduce a projection of the adjoint solution into the approximation space for the forward solution. This makes the estimate easier to compute and has the effect of “localizing” the error contributions from each time step.

The estimate (3.20) is computable provided that we can compute the adjoint solution ∂. This raises several issues. The first is that we cannot use $\overline{g^{\prime }(x,X)}$ in practice since this requires the unknown solution x. Typically, we use $\overline{g^{\prime }(x,X)}\approx \overline{g^{\prime }(X,X)}=g^{\prime }\left(X\right)$. The effect of this approximation on the computation of ∂ can be analyzed, e.g., [10]. The error depends on the accuracy of X, so typical short time error bounds can be proved. The second issue is that in practice we solve the adjoint problem using a numerical method, typically using a higher-order method than used for the forward solution.

The consequence is that in practice we use an approximate adjoint solution. We can alter the analysis below to take into the account the effect on the estimate, but this significantly complicates the presentation of the results while it is generally not significant.

3.6. Estimating term II in (3.15)

We first observe that term II is a linear functional of the error arising from solving the exact adjoint with the approximate adjoint. We adapt the a posteriori analysis to estimate the error of this approximation. We define the adjoint to the approximate adjoint as

$$\{\begin{array}{cc}\stackrel{.}{w}=D_{y}f(Y_{\mu };{\mu }_1)w,\hfill & 0<t\le T,\hfill \\ w\left(0\right)=({\lambda }_0-{\mu }_0).\hfill & \hfill \end{array}\phantom{\}}$$

Since ẇ – D_y f(Yμ; μ) = 0, we have

$$\begin{array}{cc}\hfill 0& ={\int }_0^T\langle \stackrel{.}{w}-D_{y}f(Y_{\mu };{\mu }_1)w,(\varphi -\Phi )\rangle dt\hfill \\ \hfill & =\left[\langle w\left(T\right),(\varphi \left(T\right)-\Phi \left(T\right))\rangle -\langle (w\left(0\right),(\varphi \left(0\right)-\Phi \left(0\right)))\rangle -{\int }_0^T\langle w,(\stackrel{.}{\varphi }-\stackrel{.}{\Phi })\langle dt\right]-{\int }_0^T\langle w,D_{y}f{(Y_{\mu };{\mu }_1)}^T(\varphi -\Phi )\rangle dt\hfill \\ \hfill & =-\langle ({\lambda }_0-{\mu }_0),(\varphi \left(0\right)-\Phi \left(0\right))\rangle +{\int }_0^T\langle w,\left[-\stackrel{.}{\varphi }-D_{y}f{(Y_{\mu };{\mu }_1)}^T\varphi +\stackrel{.}{\Phi }+D_{y}f{(Y_{\mu };{\mu }_1)}^T\Phi \right]\rangle dt.\hfill \end{array}$$

This gives

$$\text{II}={\int }_0^T\langle w,\left[-\stackrel{.}{\varphi }-D_{y}f{(Y_{\mu };{\mu }_1)}^T\varphi +\stackrel{.}{\Phi }+D_{y}f{(Y_{\mu };{\mu }_1)}^T\Phi \right]\rangle dt.$$ (3.22)

By adding and subtracting D_yf(Y_μ; μ₁)^⊺ ϕ to the differential equation in (3.4) for the exact adjoint, we have

$$-\stackrel{.}{\varphi }-D_{y}f{(Y_{\mu };{\mu }_1)}^T\varphi ={[D_{y}f(y_{\mu };{\mu }_1)-D_{y}f(Y_{\mu };{\mu }_1)]}^T\varphi +\psi .$$ (3.23)

Substituting (3.23) into (3.22) and using (3.12), we have

$$\text{II}={\int }_0^T\langle w,{[D_{y}f(y_{\mu };{\mu }_1)-D_{y}f(Y_{\mu };{\mu }_1)]}^T\Phi \rangle dt$$ (3.24)

$$+{\int }_0^T\langle w,{[D_{y}f(y_{\mu };{\mu }_1)-D_{y}f(Y_{\mu };{\mu }_1)]}^T(\varphi -\Phi )\rangle dt.$$ (3.25)

We show that the second term on the right-hand side of the last equation is higherorder and estimate the first term on the right-hand side. If f(y; λ) is three times continuously differentiable, then we use Taylor’s theorem to get

where J denotes the n × n identity matrix and the vector operator denoted vec is a map from ${\mathbb{R}}^{l\times m}\to {\mathbb{R}}^{lm}$ defined by stacking the columns (in order) of a matrix to form a column vector. We let

$${\psi }_{II}={\left[D_y\left(\text{vec}\left(D_{y}f{(Y_{\mu };{\mu }_1)}^T\right)\right)\right]}^T{[{\Phi }^T\otimes J]}^{T}w.$$

The first term on the right-hand side is a linear functional of the error y_μ – Y_μ and can be estimated by Theorem 3.2.

We now show that the second term is higher-order. Let η = ϕ – Φ then

$$\{\begin{array}{cc}-\stackrel{.}{\eta }=D_{y}f{(y_{\mu };{\mu }_1)}^T\varphi -D_{y}f{(Y_{\mu };{\mu }_1)}^T\Phi ,\hfill & T>t\ge 0,\hfill \\ \eta \left(T\right)=0.\hfill & \hfill \end{array}\phantom{\}}$$ (3.26)

If Y_μ is sufficiently close to y_μ over [0, T], then

$$D_{y}f{(Y_{\mu };{\mu }_1)}^T=D_{y}f{(y_{\mu };{\mu }_1)}^T+∊\left(t\right),t\in [0,T],$$ (3.27)

where ∊(t) is a perturbation matrix satisfying ∥∊(t)∥ ≤ C ∥y_μ – Y_μ∥ for some C > 0 and all t ∈ [0, T]. Substituting (3.27) into (3.26) gives

$$\{\begin{array}{cc}-\stackrel{.}{\eta }=D_{y}f{(y_{\mu };{\mu }_1)}^T\eta +∊\left(t\right)\Phi \left(t\right),\hfill & T>t\ge 0,\hfill \\ \eta \left(T\right)=0.\hfill & \hfill \end{array}\phantom{\}}$$ (3.28)

Let Σ(t) denote the fundamental matrix of (3.29); then

$$\eta \left(t\right)=-\Sigma \left(t\right){\int }_T^t{\left[\Sigma \left(s\right)\right]}^{-1}∊\left(s\right)\Phi \left(s\right)ds.$$

This implies that

$$\parallel \eta \left(t\right)\parallel \ge \parallel \Sigma \left(t\right)\parallel {\int }_0^T\parallel \Sigma {\left(s\right)}^{-1}\parallel \parallel ∊\left(s\right)\parallel \parallel \Phi \left(s\right)\parallel ds\le C{\parallel y_{\mu }-Y_{\mu }\parallel }_U.$$ (3.29)

Here, ∥·∥_U is interpreted as before to mean the L²([0, T]) norm of a given vector norm of the argument, and C > 0 is some constant that bounds the product of sup_t∈[0,T] ∥Σ(t)∥, sup_t∈[0,T] ∥Σ(t)⁻¹∥, and sup_t∈[0,T] ∥Φ(t)∥. Thus, by Lipschitz continuity of the first derivatives of f(y; λ) and (3.29),

$$\mid {\int }_0^T\langle w,{[D_{y}f(y_{\mu };{\mu }_1)-D_{y}f(y_{\mu };{\mu }_1)]}^T(\varphi -\Phi )\rangle dt\mid \le C{\parallel y_{\mu }-Y_{\mu }\parallel }_U^2.$$

3.7. Estimating term III in (3.15)

Add and subtract 〈D_λ f(Y_μ; μ₁)(λ₁ – μ₁),ϕ〉 and write III = IIIa + IIIb, where

$$\begin{array}{cc}\hfill \text{IIIa}& ={\int }_0^T\langle \left(D_{\lambda }f(y_{\mu };{\mu }_1)\right)-D_{\lambda }f(Y_{\mu };{\mu }_1)({\lambda }_1-{\mu }_1),\varphi \rangle dt,\hfill \\ \hfill \text{IIb}& ={\int }_0^T\langle D_{\lambda }f(Y_{\mu };{\mu }_1)({\lambda }_1-{\mu }_1),(\varphi -\Phi )\rangle dt,\hfill \end{array}$$

and estimate IIIa and IIIb.

Estimating term IIIa. Add and subtract 〈(D_λf(y_μ; μ₁) – D_μf(Yμ; μ₁))(μ₁ – μ₁), Φ〉 and write IIIa = IIIaa + IIIab, where

$$\begin{array}{cc}\hfill \text{IIIaa}& ={\int }_0^T\langle (D_{\lambda }f(y_{\mu };{\mu }_1)-D_{\lambda }f(Y_{\mu };{\mu }_1))({\lambda }_1-{\mu }_1),(\varphi -\Phi )\rangle dt,\hfill \\ \hfill \text{IIIab}& ={\int }_0^T\langle (D_{\lambda }f(y_{\mu };{\mu }_1)-D_{\lambda }f(Y_{\mu };{\mu }_1))({\lambda }_1-{\mu }_1),\Phi \rangle dt.\hfill \end{array}$$

We show that IIIaa is higher-order. We know that ∥ϕ – Φ∥ ≤ C ∥y_μ – Y_μ∥_U for some constant C > 0; therefore

$$\mid \text{IIIaa}\mid \le C{\parallel y_{\mu }-Y_{\mu }\parallel }_U^2$$

for some constant C > 0.

Again assuming that f(y; λ) is three times continuously differentiable,

$$(D_{\lambda }f(y_{\mu };{\mu }_1)-D_{\lambda }f(y_{\mu };{\mu }_1))({\lambda }_1-{\mu }_1)\approx [{({\lambda }_1-{\mu }_1)}^T\otimes J]\left[D_y\left(\text{vec}\left(D_{\lambda }f(Y_{\mu };{\mu }_1)\right)\right)\right](y_{\mu }-Y_{\mu }).$$

We substitute this estimate into IIIab so that

$$\begin{array}{cc}\hfill \text{IIIab}& \approx {\int }_0^T\langle [{({\lambda }_1-{\mu }_1)}^T\otimes J]\left[D_y\left(\text{vec}\left(D_{\lambda }f(Y_{\mu };{\mu }_1)\right)\right)\right](y_{\mu }-Y_{\mu }),\Phi \rangle dt\hfill \\ \hfill & ={\int }_0^T\langle (y_{\mu }-Y_{\mu }),{\left[D_y\left(\text{vec}\left(D_{\lambda }f(Y_{\mu };{\mu }_1)\right)\right)\right]}^T{[{({\lambda }_1-{\mu }_1)}^T\otimes J]}^T\Phi \rangle dt.\hfill \end{array}$$

We let

$${\psi }_{IIIab}={\left[D_y\left(vec\left(D_{\lambda }f(Y_{\mu };{\mu }_1)\right)\right)\right]}^T{[{({\lambda }_1-{\mu }_1)}^T\otimes J]}^T\Phi .$$

Thus, we have represented IIIab as a linear functional of the error in y_μ – Y_μ, which can be estimated by Theorem 3.2.

Estimating term IIIb. We let ψ_IIIb = D_λf(Y_μ, μ₁)(λ₁ – μ₁) so that

$$\text{IIb}={\int }_0^T\langle {\psi }_{IIb},(\varphi -\Phi )\rangle dt.$$

Thus, IIIb is a linear functional of the error in the adjoint solutions ϕ – Φ. We apply Theorem 3.2. We again define an adjoint to the approximate adjoint as

$$\{\begin{array}{cc}\stackrel{.}{z}-D_{y}f(Y_{\mu };{\mu }_1)z={\psi }_{IIb},\hfill & 0<t\le T,\hfill \\ z\left(0\right)=0.\hfill & \hfill \end{array}\phantom{\}}$$

We perform a standard variational argument to obtain

Using (3.26)–(3.28) in the right-hand side above, we have

$$\text{IIIb}={\int }_0^T\langle z,{[D_{y}f(y_{\mu };{\mu }_1)-D_{y}f(Y_{\mu };{\mu }_1)]}^T\Phi \rangle dt-{\int }_0^T\langle z,{[D_{y}f(y_{\mu };{\mu }_1)-D_{y}f(Y_{\mu };{\mu }_1)]}^T(\varphi -\Phi )\rangle dt.$$

The two terms on the right-hand side are analagous to (3.24) and (3.25). The second term on the right-hand side has already been proved to be higher-order. Therefore, the second term is neglected in the estimate. The first term is estimated similarly to how (3.24) was estimated. We define

$${\tilde{\psi }}_{IIIb}={\left[D_y\left(\text{vec}\left(D_{y}f{(Y_{\mu };{\mu }_1)}^T\right)\right)\right]}^T{({\Phi }^T\otimes J)}^{T}z,$$

and the first term is approximated by

$${\int }_0^T\langle {\tilde{\psi }}_{IIIb},(Y_{\mu }-Y_{\mu })\rangle dt,$$

which is a linear functional of the error of y_μ – Y_μ and is estimable by Theorem 3.2. This completes the proof of Theorem 3.3.

Theorem 3.3.

• Let Y_μ and Φ denote the numerical solutions to the initial value problem (3.3) and the approximate adjoint problem (3.12), respectively.

• Apply Theorem 3.2 to estimate termI.

• Let p_m be the number of model parameters and pi the number of initial conditions (p_m + p_i = p)

• For I = 1, … , p do

• if i ≤ p_m then

Let z denote the solutions to the adjoint to the approximate adjoint problem

$$\{\begin{array}{cc}\stackrel{.}{x}-D_{y}f(Y_{\mu };{\mu }_1)x=D_{\lambda }f(Y_{\mu },{\mu }_1){\delta }_i,\hfill & 0<t\le T,\hfill \\ x\left(0\right)=0,\hfill & \hfill \end{array}\phantom{\}}$$

where δ_i denotes the ith standard basis vector in ${\mathbb{R}}^{pm}$

Set

$$\begin{array}{cc}\hfill {\psi }_{IIIab}& ={\left[D_y\left(\text{vec}\left(D_{\lambda }f(Y_{\mu };{\mu }_1)\right)\right)\right]}^T{[{\delta }_i^T\otimes J]}^T\Phi ,\hfill \\ \hfill {\tilde{\psi }}_{IIIb}& ={\left[D_y\left(\text{vec}\left(D_{y}f{(Y_{\mu };{\mu }_1)}^T\right)\right)\right]}^T{[{\Phi }^T\otimes J]}^{T}z.\hfill \end{array}$$

Solve (3.12) with data given by the above vectors and use Theorem 3.2 to compute the standard error representations given by

$$e_1^i≔{\int }_0^T\langle (y_{\mu }-Y_{\mu }),{\psi }_{IIab}\rangle dt,e_2^i≔{\int }_0^T\langle {\tilde{\psi }}_{IIb},(y_{\mu }-Y_{\mu })\rangle dt.$$

else

Let w denote the solutions to the adjoint to the approximate adjoint problem

$$\{\begin{array}{cc}\stackrel{.}{x}-D_{y}f(Y_{\mu };{\mu }_1)x-0,\hfill & 0<t\le T,\hfill \\ x\left(0\right)={\delta }_i,\hfill & \hfill \end{array}\phantom{\}}$$

where δ_i denotes the ith standard basis vector in ${\mathbb{R}}^{pi}$

Set

$${\psi }_{II}={\left[D_y\left(\text{vec}\left(D_{y}f{(Y_{\mu };{\mu }_1)}^T\right)\right)\right]}^T{[{\Phi }^T\otimes J]}^{T}w$$

Solve (3.12) with data given by the above vectors and use Theorem 3.2 to compute the standard error representations given by

$$e_3^i≔{\int }_0^T\langle {\psi }_I,(y_{\mu }-Y_{\mu })\rangle dt$$

end if end for Fix λ and set u := λ – μ, where

$$u={({\nu }_1\cdots ,{\nu }_{pm}{\rho }_1\cdots {\rho }_{pi})}^T.$$

The error q(λ) – q̂(λ) is given by

$$q\left(\lambda \right)-\tilde{q}\left(\lambda \right)=e_u+h.o.t,$$

where e_u is the computable error estimate given by

$$e_u=e_0+\sum _{i=1}^{pm}{\nu }_i\sum _{j=1}^2{e}_j^i+\sum _{i=1}^{p_i}{p}_i{e}_3^i.$$

This theorem provides a means of computing the E_i required in Theorem 2.1. Set E_i = max_λ∈Bi e_u. For convex polygonal cells B_i, computation of E_i is straightforward since e_u is a linear function of λ, so the maximum occurs on the boundary.

4. Examples

We present examples that illustrate the properties of the computable a posteriori error estimate. In the case of deterministic computations, it is standard to test the accuracy of the estimate by direct comparison to the error on problems for which the actual error is known or can be approximated using an extremely accurate reference solution. However, it is more complicated to test accuracy for the estimates we have derived for stochastic computations because of the nature of the a posteriori bound we use for the stochastic component of the error.

We have explored the accuracy of the a posteriori bound on the effects of finite sampling in [13, 14]. Likewise, the accuracy of the a posterior error estimate for deterministic problems is well recorded; e.g., see [9, 18, 7, 8, 12] and many research papers. We do not repeat tests on these aspects here. Rather, in Examples 1 and 2 we explore the accuracy of the a posteriori error estimates on errors in computed derivatives that are needed for the estimate on the solution of the inverse problem. These estimates are new so their properties have not been explored in the literature. Finally, in Example 3 we present an example in which we check the a posteriori estimate against a direct approximation of the error.

In all the examples, the numerical solution and error estimates are computed using GAASP.¹ We use a first-order discontinuous Galerkin (dG1) method for the forward solve and a second-order continuous Galerkin (cG2) method for all of the adjoint solves. We use the adaptive time step capability in GAASP to control the numerical integration error. In the first two examples, we terminate time step refinement once the error estimate corrects the estimated gradient by less than 10%.

4.1. Example 1

The first example is a coupled linear system with four parameters:

$$\{\begin{array}{cc}{\stackrel{.}{x}}_1=\frac{x_1}{{\lambda }_1(1+t)}-{\lambda }_{2}t{x}_2,\hfill & 0<t\le T,\hfill \\ {\stackrel{.}{x}}_2=\frac{x_2}{{\lambda }_3(1+t)}+{\lambda }_{4}t{x}_1,\hfill & 0<t\le T,\hfill \\ x_1\left(0\right)=x_1,0,\hfill & \hfill \\ x_2\left(0\right)=x_2,0.\hfill & \hfill \end{array}\phantom{\}}$$ (4.1)

The adjoint problem is

$$\{\begin{array}{cc}-{\stackrel{.}{\varphi }}_1-\frac{1}{{\lambda }_1(1+t)}{\varphi }_1+{\lambda }_{4}t{\varphi }_2={\psi }_1,\hfill & T>t\ge 0,\hfill \\ -{\stackrel{.}{\varphi }}_2+{\lambda }_{2}t{\varphi }_1-\frac{1}{{\lambda }_3(1+t)}{\varphi }_2={\psi }_2,\hfill & T>t\ge 0,\hfill \\ {\varphi }_1\left(T\right)={\varphi }_1,T,\hfill & \hfill \\ {\varphi }_2\left(T\right)={\varphi }_{2}T.\hfill & \hfill \end{array}\phantom{\}}$$ (4.2)

Computing the true errors requires knowledge of the exact ϕ. To this end, we choose ψ(t) = (ψ₁, ψ₂)^⊺ and ϕ(T) = (ϕ_1,T, ϕ_2,T)^⊺ so that ϕ(t) = (t, 1)^⊺.

In this linear example, II = 0. We report the error estimates for term III. We take μ = (2, 2, 2, 2)^⊺, so $y_{\mu }={(\sqrt{1+t}\text{cos}\left(t^2\right),\sqrt{1+t}\text{sin}\left(t^2\right))}^{⊺}$.

We consider both T = 3 and T = 10. We plot the forward solutions y_μ and Y_μ for T = 3 with two different time steps in Figure 4.1. Table 4.1 shows the error estimate results for T = 3. Since the computed error estimates tend to be accurate, we can often compute a corrected gradient by adding the error estimate to the computed (estimated) gradient. We see improvement in the corrected gradient by comparing the fourth and last columns of Table 4.1. At T = 3, Y_μ is a good approximation of y_μ at the coarse time step of 0.2 as seen in Figure 4.1, so the second derivative calculations involving Y_μ used in the error estimates produce accurate error estimates beginning at this time step.

Fig. 4.1.

Solutions to (4.1) for T = 3. Left two plots: Y_μ,1 and Y_μ,2 with a time step of 0.2. Right two plots: Y_μ,1 and Y_μ,2 with a time step of 0.1. The dotted lines indicate the corresponding exact solutions y_μ,1 and y_μ,2 evaluated on the same time mesh as the dashed-lined numerical approximations Y_μ,1 and Y_μ,2.

Table 4.1. Example 1 results for the three partial derivatives of the solution at T = 3.

Δ t	true ∂1q	est. ∂1q	Error	IIIab	IIb	Estimate	Error Estimate	est. ∂1q+Est. true ∂1q
0.2	1.71E – 01	1.62E – 01	8.97E – 03	9.43E – 03	−7.53E – 07	9.43E – 03	1.052	1.00
0.1	1.71E – 01	1.69E – 01	1.58E – 03	1.47E – 03	−7.32E – 07	1.47E – 03	0.931	0.999
0.05	1.71E – 01	1.70E – 01	2.21E – 04	1.97E – 04	2.58E – 06	1.99E – 04	0.904	1.00
Δ t	true ∂2q	est. ∂2q	Error	IIIab	IIb	Estimate	Error Estimate	est. ∂2q+Est. true ∂2q
0.2	−3.17E + 00	−2.95E + 00	−2.27E – 01	−1.76E – 01	1.07E – 06	−1.76E – 01	0.776	0.984
0.1	−3.17E + 00	−3.14E + 00	−2.91E – 02	−2.29E – 02	−7.29E – 07	−2.29E – 02	0.788	0.998
0.05	−3.17E + 00	−3.17E + 00	−3.54E – 03	−2.81E – 03	−9.58E – 06	−2.82E – 03	0.796	1.00
Δ t	true ∂3q	est. ∂3q	Error	IIIab	IIb	Estimate	Error Estimate	est. ∂3q+Est. true ∂3q
0.2	5.36E – 01	5.30E – 01	5.82E – 03	4.40E – 03	−4.19E – 07	4.40E – 03	0.757	0.997
0.1	5.36E – 01	5.35E – 01	7.24E – 04	5.59E – 04	−6.41E – 07	5.58E – 04	0.771	1.00
0.05	5.36E – 01	5.36E – 01	8.69E – 05	6.77E – 05	9.55E – 08	6.78E – 05	0.780	1.00
Δ t	true ∂4q	est. ∂4q	Error	IIIab	IIb	Estimate	Error Estimate	est. ∂4q+Est. true ∂4q
0.2	2.78E – 01	2.45E – 01	3.27E – 02	3.53E – 02	7.92E – 07	3.53E – 02	1.08	1.01
0.1	2.78E – 01	2.72E – 01	5.92E – 03	5.58E – 03	−1.09E – 06	5.57E – 03	0.941	0.999
0.05	2.78E – 01	2.77E – 01	8.36E – 04	7.50E – 04	−8.04E – 06	7.41E – 04	0.887	1.00

We plot the forward solutions y_μ and Y_μ for T = 10 with four different time steps in Figures 4.2 and 4.3. The oscillations of y_μ increase in magnitude and with higher frequency as time increases. As seen in Table 4.2, when the error estimate is of the same order of magnitude as the estimated gradient, the estimate cannot be used to correct the gradient.

Fig. 4.2.

Solutions to (4.1) for T = 10. Left two plots: Y_μ,1 and Y_μ,2 with a time step of 0.2. Right two plots: Y_μ,1 and Y_μ,2 with a time step of 0.1. The dotted lines indicate the corresponding exact solutions y_μ,1 and y_μ,2 evaluated on the same time mesh as the dashed-lined numerical approximations Y_μ,1 and Y_μ,2.

Fig. 4.3.

Solutions to (4.1) for T = 10. Left two plots: Y_μ,1 and Y_μ,2 with a time step of 0.05. Right two plots: Y_μ,1 and Y_μ,2 with a time step of 0.025. The dotted lines indicate the corresponding exact solutions y_μ,1 and y_μ,2 evaluated on the same time mesh as the dashed-lined numerical approximations Y_μ,1 and Y_μ,2.

Table 4.2. Example 1 results for the three partial derivatives of the solution at T = 10.

Δ t	true ∂1q	est. ∂1q	Error	IIIab	IIb	Estimate	Error Estimate	est. ∂1q+Est. true ∂1q
0.2	–1.35E – 02	6.30E – 02	–7.66E – 02	9.59E – 03	2.20E – 05	9.61E – 03	–0.126	–5.36
0.1	–1.35E – 02	5.75E – 02	–7.10E – 02	–1.07E – 01	–5.60E – 06	–1.07E – 01	1.51	3.66
0.05	–1.35E – 02	9.58E – 03	–2.31E – 02	–2.36E – 02	3.40E – 06	–2.36E – 02	1.02	1.04
0.025	–1.35E – 02	–9.20E – 03	–4.39E – 03	–3.91E – 03	–9.80E – 06	–3.92E – 03	0.893	0.965
Δ t	true ∂2q	est. ∂2q	Error	IIIab	IIb	Estimate	Error Estimate	est. ∂2q+Est. true ∂2q
0.2	1.40E + 01	–3.39E – 01	1.44E + 01	1.26E + 01	2.07E – 04	1.26E + 01	0.876	0.873
0.1	1.40E + 01	–6.37E – 01	1.47E + 01	5.82E + 00	–3.24E – 04	5.82E + 00	0.397	0.370
0.05	1.40E + 01	7.68E + 00	6.34E + 00	4.85E + 00	–2.87E – 04	4.85E + 00	0.765	0.894
0.0250	1.40E + 01	1.30E + 01	9.91E – 01	7.88E – 01	–2.53E – 04	7.87E – 01	0.794	0.985
Δ t	true ∂3q	est. ∂3q	Error	IIIab	IIb	Estimate	Error Estimate	est. ∂3q+Est. true ∂3q
0.2	4.51E – 01	4.63E – 01	–1.29E – 02	–1.14E – 02	2.18E – 05	–1.14E – 02	0.882	1.02
0.1	4.51E – 01	4.64E – 01	–1.32E – 02	–5.13E – 03	–9.29E – 06	–5.14E – 03	0.388	1.02
0.05	4.51E – 01	4.56E – 01	–5.73E – 03	–4.37E – 03	–2.20E – 06	–4.37E – 03	0.763	1.00
0.025	4.51E – 01	4.51E – 01	–8.98E – 04	–7.10E – 04	–8.74E – 06	–7.19E – 04	0.801	1.00
Δ t	true ∂4q	est. ∂4q	Error	IIIab	IIb	Estimate	Error Estimate	est. ∂4q+Est. true ∂4q
0.2	–9.52E – 01	–1.16E – 01	–8.37E – 01	1.11E – 01	2.18E – 04	1.11E – 01	–0.133	1.36
0.1	–9.52E – 01	–1.78E – 01	–7.75E – 01	–1.17E + 00	–3.28E – 04	–1.18E + 00	1.52	0.458
0.05	–9.52E – 01	–7.01E – 01	–2.51E – 01	–2.58E – 01	–2.83E – 04	–2.58E – 01	1.03	1.01
0.025	–9.52E – 01	–9.04E – 01	–4.81E – 02	–4.27E – 02	–2.52E – 04	–4.29E – 02	0.892	0.995

4.2. Example 2

The second example is a nonlinear problem with two parameters:

$$\{\begin{array}{cc}\stackrel{.}{x}=-({\lambda }_1+\sin \left({\lambda }_{2}t\right))x^2,\hfill & 0<t\le T,\hfill \\ x\left(0\right)=x_0.\hfill & \hfill \end{array}\phantom{\}}$$ (4.3)

We set μ = (−0.1, 20)^⊺ so y_μ(t) = 20/ (20 + 1 + (−0.1)20t – cos(20t)). The quantity of interest is y(T ). The adjoint problem is

$$\{\begin{array}{cc}-\stackrel{.}{\varphi }+2y_{\mu }(-0.1+\sin \left(20t\right))\varphi =0,\hfill & T>t\ge 0,\hfill \\ \varphi \left(T\right)=1.\hfill & \hfill \end{array}\phantom{\}}$$ (4.4)

The solution to (4.4) is ϕ(t) = C(20 + 1 + 20(−0.1)t – cos(20t))², where C is chosen so ϕ(T ) = 1. Since (4.3) is nonlinear, we report the error estimates for both terms II and III.

We show results for both T = 3.9 and T = 10. We plot the forward solutions y_μ and Y_μ and adjoint solutions ϕ and Φ for T = 3.9 and T = 10 with three different time steps in Figures 4.4 and 4.5, respectively. Tables 4.3 and 4.4 show the error estimate results for T = 3.9 and T = 10, respectively.

Fig. 4.4.

Solutionsto (4.3) for T = 3.9 with a time step of 0.3 (left), 0.15 (middle), and 0.075 (right). Top plots: Y_μ and y_μ. Bottom plots: Φ and φ. The dotted lines indicate the corresponding exact solutions y_μ and φ evaluated on the same time mesh theY as dashed-lined numerical approximations μ and Φ.

Fig. 4.5.

Solutions to (4.3) for T = 10 with a time step of 0.04 (left), 0.02 (middle), and 0.01 (right). Top plots: Y_μ and y_μ for time interval [9, 10]. Bottom plots: Φ and ϕ. The dotted lines indicate the corresponding exact solutions y_μ and ϕ evaluated on the same time mesh as the dashed-lined numerical approximations Y_μ and Φ.

Table 4.3. Example 2 results for the two partial derivatives of the solution (upper) and the adjoint solution (lower) at T = 3.9.

Δ t	true ∂1q	est. ∂1q	Error	IIIab	IIb	Estimate	Error Estimate	est. ∂1q+Est. true ∂1q
0.3	−7.89E + 00	−4.42E + 00	−3.47E + 00	−4.06E + 00	−1.49E + 00	−5.55E + 00	1.60	1.26
0.15	−7.89E + 00	−9.32E + 00	1.43E + 00	5.88E – 01	8.18E – 01	1.41E + 00	0.986	1.00
0.075	−7.89E + 00	−8.11E + 00	2.12E – 01	8.05E – 02	1.01E – 01	1.82E – 01	0.858	1.00
Δ t	true ∂2q	est. ∂2q	Error	IIIab	IIb	Estimate	Error Estimate	est. ∂2q+Est. true ∂2q
0.3	−1.94E – 01	2.26E – 01	−4.20E – 01	9.44E – 01	1.09E + 00	2.03E + 00	−4.83	−11.7
0.15	−1.93E – 01	5.62E – 02	−2.50E – 01	−1.91E – 01	−5.68E – 02	−2.48E – 01	0.992	0.990
0.075	−1.93E – 01	−1.78E – 01	−1.55E – 02	−4.55E – 03	−1.44E – 02	−1.90E – 02	1.22	1.02

Δ t	φ(0)	Φ(0)	Error	II	Error Estimate	Φ(0)+ Est. φ(0)
0.3	2.02E + 00	2.49E + 00	−4.64E – 01	1.19E + 00	−2.57	1.82
0.15	2.02E + 00	2.35E + 00	−3.31E – 01	−3.54E – 01	1.07	0.988
0.075	2.02E + 00	2.07E + 00	−5.03E – 02	−4.93E – 02	0.981	1.00

Table 4.4. Example 2 results for the two partial derivatives of the solution (upper) and the adjoint solution (lower) at T = 10.

Δ t	∂1q (true)	∂1q (est)	∂1q (est) ∂1q (true)	True Error	Term IIIab	Term IIb	Error Estimate	Error Ratio	∂1q (est)+Error Estimate ∂1q (true)
0.04	−1.52E + 04	−1.91E + 04	1.25290	3.85E + 03	5.24E + 02	7.08E + 03	7.60E + 03	1.97601	0.75317
0.02	−1.52E + 04	−1.56E + 04	1.02488	3.78E + 02	4.93E + 01	2.60E + 02	3.10E + 02	0.81829	1.00452
0.01	−1.52E + 04	−1.53E + 04	1.00369	5.61E + 01	5.98E + 00	3.61E + 01	4.21E + 01	0.75009	1.00092
Δ t	∂2q (true)	∂2q (est)	∂2q (est) ∂2q (true)	True Error	Term IIIab	Term IIb	Error Estimate	Error Ratio	∂2q (est)+Error Estimate ∂2q (true)
0.04	6.66E + 02	7.03E + 02	1.05505	−3.67E + 01	1.17E + 03	−1.12E + 03	4.73E + 01	−1.29028	1.12609
0.02	6.66E + 02	6.79E + 02	1.01928	−1.28E + 01	8.71E + 01	−9.40E + 01	−6.94E + 00	0.54057	1.00886
0.01	6.66E + 02	6.68E + 02	1.00328	−2.19E + 00	1.02E + 01	−1.17E + 01	−1.48E + 00	0.67836	1.00106

Δ t	φ(0)	Φ(0)	Φ(0) φ(0)	True Error	Term II	Error Ratio	Φ(0)+Error Estimate φ(0)
0.04	1.52E + 03	1.90E + 03	1.25043	−3.81E + 02	−7.63E + 02	2.00188	0.74909
0.02	1.52E + 03	1.56E + 03	1.02465	−3.75E + 01	−3.11E + 01	0.83007	1.00419
0.01	1.52E + 03	1.53E + 03	1.00366	−5.57E + 00	−4.23E + 00	0.75982	1.00088

4.3. Example 3

We consider the nonlinear example first presented in [4]:

$$\{\begin{array}{cc}\stackrel{.}{x}={\lambda }_1\sin \left({\lambda }_{2}x\right),\hfill & 0<t\le T,\hfill \\ x\left(0\right)=1.\hfill & \hfill \end{array}\phantom{\}}$$

The quantity of interest is the average value of x(t) over the time interval [0, 2]. Thus, we set ψ(t) = 1_[0,2](t)/2 in the adjoint problem. We use a time step of 0.25 to solve at each point of a 20 20 grid of uniformly spaced parameter values in Λ = [.8, 1.2] × [.1, π × – .1] and compute the simple function approximation of the σ_Λ shown in Figure 4.6, where we use 10,000 samples of the quantity of interest to approximate the output density. We denote the associated distribution function by F̃⁽¹⁾(t).

Fig. 4.6.

Left: The global piecewise-linear approximation to q(λ) using a coarse 20 × 20 set of cells. The circles in each cell indicate the reference parameter used to linearize q(λ) in that cell. Right: A contour plot of the computed probability distribution on a grid of 60 × 60 cells corresponding to a normal distribution on q(λ).

We have max_1≤i≤M’ (max q(b_i) – min q(b_i)) ≤ 2.69 × 10⁻¹ and the corresponding estimate is E ≤ 7.53 × 10⁻⁰⁴. The normal distribution imposed on the quantity of interest has a small variance (approximately 6.72 10⁻⁰³) so the Lipschitz constant of the distribution is bounded by 5. Thus, using (2.5) with ∊ = 0.05, we have that

$$\mid F\left(t\right)-{\tilde{F}}^1\left(t\right)\mid \le 2.55\times {10}^{-22}$$

with probability 95%.

In order to compare the computed a posteriori error estimate to the true error, we directly approximate the error in a computed solution by using another more accurate solution. We use a time step of 1.0 × 10⁻⁰² to compute solutions to the forward problem at each parameter in the 20 × 20 grid, and we invert using 10⁸ samples of the output data and use the same resolution in Λ of 60 × 60 small cells to obtain another approximate distribution function that we denote F̃⁽²⁾(t). We compare F̃⁽¹⁾(t) to F̃⁽²⁾(t) and compare to the error bound above. We evaluate the difference in these distributions at the upper-right corner of each b_i and plot the absolute value of the difference in Figure 4.7. The maximum computed absolute value of error at these points is less than 6.70 × 10⁻⁰³, which is within the error bound above.

Fig. 4.7.

Plot of absolute values of approximate errors in probabilities over the 60 × 60 grid of cells used to approximate the solution to the inverse problem. The errors are approximated using a more accurate approximation F̃⁽²⁾(t) computed using a refined numerical solution with a time step of 10⁻², resulting in E < 10⁻⁷, and 10⁸ samples of the output density to make statistical errors small. The maximum in this plot is approximately 6.70 × 10⁻⁰³, which is less than the computed bound 2.55 × 10⁻⁰².