Comparison of finite difference based methods to obtain sensitivities of stochastic chemical kinetic models

Abstract

Sensitivity analysis is a powerful tool in determining parameters to which the system output is most responsive, in assessing robustness of the system to extreme circumstances or unusual environmental conditions, in identifying rate limiting pathways as a candidate for drug delivery, and in parameter estimation for calculating the Hessian of the objective function. Anderson [SIAM J. Numer. Anal. 50, 2237 (2012)]10.1137/110849079 shows the advantages of the newly developed coupled finite difference (CFD) estimator over the common reaction path (CRP) [M. Rathinam, P. W. Sheppard, and M. Khammash, J. Chem. Phys. 132, 034103 (2010)]10.1063/1.3280166 estimator. In this paper, we demonstrate the superiority of the CFD estimator over the common random number (CRN) estimator in a number of scenarios not considered previously in the literature, including the sensitivity of a negative log likelihood function for parameter estimation, the sensitivity of being in a rare state, and a sensitivity with fast fluctuating species. In all examples considered, the superiority of CFD over CRN is demonstrated. We also provide an example in which the CRN method is superior to the CRP method, something not previously observed in the literature. These examples, along with Anderson's results, lead to the conclusion that CFD is currently the best estimator in the class of finite difference estimators of stochastic chemical kinetic models.

INTRODUCTION

Recent years have seen an increasing popularity of stochastic chemical kinetic models due to their role in describing and explaining critical biological phenomena.^{3, 4, 5, 6, 7, 8, 9} One useful tool for understanding these models is the chemical master equation, which describes the evolution of the probability density of the system. The solution of the master equation is computationally tractable only for simple systems. Rather, approximation techniques such as finite state projection,¹⁰ that operates on a reduced state space, or the stochastic simulation algorithm (SSA),^{11, 12} that generates exact sample paths, are employed to reconstruct a system's probability distribution and statistics (usually the mean and variance). Applying these techniques to solve models of biological processes leads to significant improvements in our understanding of intrinsic noise and its effect on cellular behavior.

These stochastic chemical kinetic models depend on parameters whose values are often unknown and can change due to changes in the environment. Sensitivities quantify the dependence of the system's output to changes in the model parameters. Sensitivity analysis is useful in determining parameters to which the system output is most responsive, in assessing robustness of the system to extreme circumstances or unusual environmental conditions, and in identifying rate limiting pathways as a candidate for drug delivery. However, one of the most important applications of sensitivities is in parameter estimation. Sensitivities provide a way to approximate the Hessian of the objective function through the Gauss-Newton approximation (Ref. ¹³, p. 535).

A popular unbiased method of sensitivity estimation is the likelihood ratio gradient method.^{14, 15, 16} The unbiasedness of the likelihood ratio gradient method comes at the cost of a high variance of the estimator if there are several reaction events in the estimation of the output of interest. The convergence rate, which is a measure of the rate at which the mean squared error of the estimator converges to zero, of this estimator is O(N^−1/2), in which N is the number of estimator simulations. Komorowski et al.¹⁷ use a linear noise approximation of stochastic chemical kinetic models for sensitivity analysis. However, use of linear noise approximation limits their analysis to only stochastic differential equation models that incorporate Brownian motions. Gunawan et al.¹⁸ compare the sensitivity of the mean with the sensitivity of the entire distribution. They explain why the sensitivity of the mean can be inadequate in determining the sensitivity of stochastic chemical kinetic models.

Despite being easier to implement and intuitive to understand, finite difference based methods produce biased sensitivity estimates. However, implemented with consideration of the trade-off between the statistical error of the estimator and its bias, finite difference based methods can have a convergence rate close to the best possible convergence rate of O(N^−1/2).¹⁹ McGill et al.²⁰ compare the applicability of likelihood ratio gradient and finite difference based methods. They discuss situations where one method performs better than the other. Drew et al.²¹ demonstrate usefulness of sensitivity analysis on Monte Carlo simulations of copper electrodeposition.

Several different estimators using finite difference have been proposed.^{1, 2, 19} Anderson¹ proposed a new estimator, coupled finite difference (CFD), using a single Markov chain for the nominal and perturbed processes. The CFD estimator incorporates a tight coupling between the nominal and perturbed processes, thereby producing a significant reduction in estimator variance.¹

In this paper, we show the superiority of CFD over CRN in the estimation of sensitivities. We do not discuss the independent random number² estimator, also known as the Crude Monte Carlo¹ estimator, because either estimator, CRN or CFD, usually has several orders of magnitude smaller variance than this estimator. We calculate sensitivity estimates of five different quantities of interest. In example one, the quantity of interest is the expected value of a species. Example two looks at the likelihood of experimental data. Example three looks at the probability of a rare state. Example four looks at the expected value of a fast fluctuating species. Example five looks at the expected value of a gene product in a model of a genetic toggle switch. In this example, the CRN method is shown to be superior to the CRP method, something not previously observed in the literature.

This paper is arranged as follows. Section 2 defines the estimators that are used in the subsequent examples. Section 3 shows the results we obtain from the five examples. Finally, Sec. 4 discusses the conclusions of this paper and summarizes the contributions.

THE ESTIMATORS

Common random number (CRN; Refs. ² and ¹⁹): A single simulation of the CRN estimator uses two coupled SSA simulations: the first coupled SSA simulation uses the rate parameter k and randomness ω and the second one uses the perturbed rate parameter k + ε and the same randomness ω. By the same randomness ω, we mean that both first and second coupled simulations use the same seed of the pseudo-random number generator in an implementation of Gillespie's direct method.¹¹

Coupled Finite difference (CFD; Ref. 1): A single simulation of the CFD estimator simulates a Markov chain with an enlarged state space. The marginal processes of this Markov chain yield the realizations of the coupled processes with rate parameters k and k + ε. The new Markov chain is constructed in such a way that there is a tight coupling between the marginal processes, yielding a low variance for the estimator. See Anderson¹ for the complete description.

Common Reaction Path (CRP; Ref. 2): A single simulation of the CRP estimator uses two SSA simulations coupled through random time change representation. Thus, it is CRN for the next reaction method.

Finite difference approximations of the sensitivity can be obtained from any of the above methods by sample averaging appropriate differences of the realizations.

EXAMPLES

Sensitivity of an expected value of a population of a species

Consider the following simple reaction network consisting of two reactions

$$A\stackrel{k_1}{\to }B,$$ (R1)

$$B\stackrel{k_2}{\to }C.$$ (R2)

Figure 1 shows a typical SSA simulation of the network involving reactions R1, R2.

Figure 1.

A typical simulation of the network involving reactions R1, R2.

We wish to estimate the sensitivity of the expected value of B with respect to the rate constant k₁,

$$\begin{array}{cc}\hfill s(t;k_1)& =\frac{d\mathbb{E}B(t;k_1)}{d{k}_1},\hfill \end{array}$$ (1)

where B(t; k₁) represents the number of B molecules at time t with a choice of rate constant of k₁. The forward finite difference approximation to Eq. 1 is

$$\begin{array}{cc}\hfill s(t;k_1)& \approx \frac{\mathbb{E}B(t;k_1+\epsilon )-\mathbb{E}B(t;k_1)}{\epsilon },\hfill \end{array}$$ (2)

which has a bias of O(ε). That is,

$$\begin{array}{cc}\hfill s(t;k_1)& =\frac{\mathbb{E}B(t;k_1+\epsilon )-\mathbb{E}B(t;k_1)}{\epsilon }+O\left(\epsilon \right).\hfill \end{array}$$

Centered differences produce a bias of O(ε²). Throughout the paper we use the forward finite difference to approximate the sensitivity. We denote an estimator of the right hand side of 2 using either CRN or CFD as ${\widehat{s}}_{\text{est}}$ in which est ∈ {CRN, CFD}. Let $B_i^{\text{est}}(t;k_1)$ and $B_i^{\text{est}}(t;k_1+\epsilon )$ denote the population of B obtained through the ith simulation of estimator est. Then the estimator ${\widehat{s}}_{\text{est}}$ for s(t; k₁) of 1 is defined as

$$\begin{array}{cc}\hfill {\widehat{s}}_{\text{est}}& =\frac{1}{N}\sum _{i=1}^N\frac{B_i^{\text{est}}(t;k_1+\epsilon )-B_i^{\text{est}}(t;k_1)}{\epsilon }.\hfill \end{array}$$ (3)

The sample standard deviations ($\widehat{\sigma }\left[{\widehat{s}}_{\text{est}}\right]$) of the estimator of Eq. 3 is given by

$$\begin{array}{ccc}\hfill \widehat{\sigma }\left[{\widehat{s}}_{\text{est}}\right]& =& \left(\frac{1}{N(N-1){\epsilon }^2}\sum _{i=1}^N[\left\{B_i^{\text{est}}(t;k_1+\epsilon )-B_i^{\text{est}}(t;k_1)\right\}\right.\hfill \\ & & {\left.{-\Delta {\widehat{B}}^{\text{est}}]}^2\right)}^{1/2}\hfill \end{array}$$ (4)

in which

$$\Delta {\widehat{B}}^{\text{est}}=\frac{1}{N}\sum _{i=1}^N{B}_i^{\text{est}}(t;k_1+\epsilon )-B_i^{\text{est}}(t;k_1).$$

Because the model is linear, the exact expected value of B and the exact sensitivity of the expected value of B can be calculated and are given by

$$\begin{array}{cc}\hfill \mathbb{E}B(t;k_1)& =n_{{\text{B}}_0}{e}^{-k_{2}t}+n_{{\text{A}}_0}\frac{k_1}{k_2-k_1}(e^{-k_{1}t}-e^{-k_{2}t}),\hfill \end{array}$$ (5)

$$\begin{array}{ccc}\hfill s_{\text{ex}}(t;k_1)=\frac{d\mathbb{E}B}{d{k}_1}& =& \frac{n_{{\text{A}}_0}}{{(k_1-k_2)}^2}\left[k_2{e}^{-k_{1}t}+k_1(k_1-k_2)t{e}^{-k_{1}t}\right.\hfill \\ & & \left.-k_2{e}^{-k_{2}t}\right],\hfill \end{array}$$ (6)

in which $n_{{\text{A}}_0}$, $n_{{\text{B}}_0}$ are the initial values of A and B, respectively.

Figure 2 compares the performance of the CRN and CFD estimators, and Table 1 lists the parameters used to generate Figure 2. Figure 2a shows a comparison of the sensitivity estimates obtained from the CRN and CFD estimators. We define root mean squared error of the estimator as

$$\begin{array}{cc}\hfill e_{\text{est}}& ={\left[\frac{1}{n_d}\sum _{i=1}^{n_d}{\left({\widehat{s}}_{\text{est}}(t_i,k_1)-s_{\text{ex}}(t_i,k_1)\right)}^2\right]}^{1/2}\hfill \end{array}$$ (7)

in which n_d = 41 is the total number of time points at which we calculate the sensitivity ${\widehat{s}}_{\text{est}}$, 0 ⩽ t_i ⩽ 2.0, and t_i+1 − t_i = 0.05. Root mean squared errors calculated from the data of Figure 2a give $\frac{e_{\text{crn}}}{e_{\text{cfd}}}=4$. A value greater than one for this ratio demonstrates that on average across all the time points considered the CFD estimator tracks the exact sensitivity better than the CRN estimator. Figure 2b quantifies the efficiency of the two estimators, by comparing their standard deviations. We can see that starting from t = 0.3, the CFD estimator has half the standard deviation of the CRN estimator. Lower standard deviation of the CFD estimator compared to the CRN estimator points to its higher efficiency.

Figure 2.

Comparison of CRN and CFD estimators for the model R1, R2: (a) Estimated and analytical sensitivities. (b) Sample standard deviation of the two estimators.

Table 1.

Parameter values for Sec. 3A.

Parameter	$n_{{\text{A}}_0}$	$n_{{\text{B}}_0}$	$n_{{\text{C}}_0}$	k1	k2	ε	N
Value	100	0	0	2.	1.	0.1	100

Sensitivity of negative log likelihood function

Consider reactions R1, R2 again. Experimental data $\mathbf{y}={(B_{t_1},B_{t_2},...,B_{t_n})}^T$ from a single experiment, shown in Figure 3, are given as a time series of B.

Figure 3.

Experimental data for Sec. 3B generated from the model with parameter choices given in Table 2.

We assume that the rate constant k₂ is known. It can be shown²² that an estimate of the likelihood of the experimental data y under certain reasonable assumptions is given by

$$\begin{array}{cc}\hfill L(k_1,N)& =\frac{1}{N{\left(2\pi \right)}^{n_d/2}{\left|\mathbf{R}\right|}^{1/2}}\sum _{i=1}^N{e}^{-(1/2){(\mathbf{y}-{\mathbf{x}}_i\left(k_1\right))}^{\prime }{\mathbf{R}}^{-1}(\mathbf{y}-{\mathbf{x}}_i\left(k_1\right))}\hfill \end{array}$$ (8)

in which R is a known positive definite matrix, n_d is the number of elements in the experimental data vector y, and ${\mathbf{x}}_i\left(k_1\right)={(B_{t_1},B_{t_2},...,B_{t_n})}_i^T$ is the time series of the population of B, obtained by the ith SSA simulation using rate constant value k₁ for reaction R1. Note that as the number of samples N goes to infinity, the likelihood estimate L(k₁, N) from 8 approaches the true likelihood of the experimental data. The estimate of the negative log likelihood is defined as

$$\begin{array}{ccc}\hfill \varphi (k_1,N)& =& -\log L(k_1,N)=-\log \left[\frac{1}{N{\left(2\pi \right)}^{n_d/2}{\left|\mathbf{R}\right|}^{1/2}}\right.\hfill \\ & & \left.\times \sum _{i=1}^N{e}^{-(1/2){(\mathbf{y}-{\mathbf{x}}_i\left(k_1\right))}^{\prime }{\mathbf{R}}^{-1}(\mathbf{y}-{\mathbf{x}}_i\left(k_1\right))}\right].\hfill \end{array}$$ (9)

To find the parameters that describe the experimental data, we need to minimize the estimate of the negative log likelihood function given in Eq. 9. Sensitivities can be used to obtain gradients required in any gradient based optimization algorithm. Here, we are interested in the sensitivity

$$\begin{array}{cc}\hfill s(k_1,N)& =\frac{d\varphi (k_1,N)}{d{k}_1}\hfill \end{array}$$ (10)

of the estimated negative log likelihood function and the convergence of this sensitivity with the number of samples, N. The forward finite difference approximation of 10 is given by

$$\begin{array}{cc}\hfill s(k_1,N)& \approx \frac{\varphi (k_1+\epsilon ,N)-\varphi (k_1,N)}{\epsilon }.\hfill \end{array}$$ (11)

We write estimator est ∈ {CRN, CFD} of the sensitivity s(k₁, N) of Eq. 10 as

$$\begin{array}{cc}\hfill {\widehat{s}}_{\text{est}}(k_1,N)& =\frac{{\varphi }_{\text{est}}(k_1+\epsilon ,N)-{\varphi }_{\text{est}}(k_1,N)}{\epsilon }\hfill \end{array}$$ (12)

in which ϕ_est(k₁, N) is the estimate of the negative log likelihood obtained from Eq. 9 using the estimator est ∈ {CRN, CFD}.

Figure 4 shows the steps in obtaining the sensitivity of the negative log likelihood function using the CRN estimator. Table 2 contains the parameters used in this example. Figure 4a shows the variation of the quadratic form (y − x_i(k₁))′R⁻¹(y − x_i(k₁)) as a function of individual SSA simulation number i. Figure 4a reveals the wide variation in the value of the quadratic form for different individual SSA simulations. Figure 4b is a plot of $e^{-(1/2){(\mathbf{y}-{\mathbf{x}}_i\left(k_1\right))}^{\prime }{\mathbf{R}}^{-1}(\mathbf{y}-{\mathbf{x}}_i\left(k_1\right))}$ as a function of the individual SSA simulation number i. The wide variation in (y − x_i(k₁))′R⁻¹(y − x_i(k₁)) of Figure 4a leads to even wider variation in the exponential, as depicted in Figure 4b. Figure 4c depicts convergence of L(k₁, N) with N. L(k₁, N) in Figure 4c is given by Eq. 8. Large variation in $e^{-(1/2){(\mathbf{y}-{\mathbf{x}}_i\left(k_1\right))}^{\prime }{\mathbf{R}}^{-1}(\mathbf{y}-{\mathbf{x}}_i\left(k_1\right))}$ as depicted in Figure 4b explains the sharp jumps in L(k₁, N) which occur whenever the last SSA simulation i = n dominates all the previous 1 ⩽ i ⩽ n − 1 simulations. Figure 4d shows the convergence of the nominal and perturbed negative log likelihoods: ϕ_crn(k₁, N) and ϕ_crn(k₁ + ε, N). The sharp jumps in ϕ_crn(k₁, N) occur at the same n values as the jumps in L(k₁, N) in Figure 4c. Finally, Figure 4e shows the convergence of the sensitivity of the negative log likelihood using the CRN estimator, ${\widehat{s}}_{\text{crn}}(k_1,N)$, as a function of N.

Figure 4.

Convergence of sensitivity estimate for Sec. 3B from the CRN estimator: (a) Plot of the quadratic form as a function of simulation number for the unperturbed process. (b) Plot of exponential of quadratic form as a function of simulation number for the unperturbed process. (c) Likelihood as a function of total number of simulations for the unperturbed process. (d) Negative log likelihood as a function of total number of simulations. (e) Estimated sensitivity from the CRN estimator as a function of total number of simulations.

Table 2.

Parameter value for Sec. 3B.

Parameter	$n_{{\text{A}}_0}$	$n_{{\text{B}}_0}$	$n_{{\text{C}}_0}$	k1	k2	ε	N	nd	R
Value	100	0	0	1.	1.	0.1	4000	21	$I_{n_d\times n_d}$

Next, we change focus from the CRN estimator to the CFD estimator. In Figures 5a, 5b, 5c, 5d, 5e, we plot the analogous results to Figures 4a, 4b, 4c, 4d, 4e. Finally, Figure 6 compares convergence of CRN estimator, ${\widehat{s}}_{\text{crn}}(k_1,N)$, and CFD estimator, ${\widehat{s}}_{\text{cfd}}(k_1,N)$. We see that the convergence of the CFD estimator to the estimated true value is faster than the CRN estimator. An estimated true value is obtained by performing 50 000 simulations of the CFD estimator. The quicker convergence property makes the CFD estimator a natural choice for the estimator of sensitivity of negative log likelihood. One point to note is that, as N → ∞, the final converged value for both CFD and CRN estimators is going to be the same, because both the estimators have the same bias; the CRN estimator shown in Figure 6 simply has not converged by N = 4000.

Figure 5.

Convergence of sensitivity estimate for Sec. 3B from the CFD estimator: (a) Plot of quadratic form as a function of simulation number for the unperturbed process. (b) Plot of exponential of quadratic form as a function of simulation number for the unperturbed process. (c) Likelihood as a function of total number of simulations for the unperturbed process. (d) Negative log likelihood as a function of total number of simulations. (e) Estimated sensitivity from the CFD estimator as a function of total number of simulations.

Figure 6.

Convergence of the estimated sensitivities of 11 from CRN and CFD estimators. Compared to the CRN estimator the CFD estimator shows quicker convergence to the estimated true value.

Sensitivity of a rare state probability

Consider the pap operon regulation^{10, 23} which has a biologically important rare state. In Srivastava et al.,²³ we obtained a significantly better estimate of the rare state probability using stochastic quasi-steady-steady perturbation analysis (sQSPA), compared to both the full model and importance sampling based methods.^{24, 25} Figure 7 shows the schematic of the pap operon regulation.

Figure 7.

Schematic diagram of the pap regulatory network. There are four possible states of the pap operon depending on the LRP-DNA binding.

State g₁ is the rare state. The master equation for the system is

$$\begin{array}{cc}\hfill \frac{d{P}_1}{dt}& =-(r_1+r_3)P_1+r_2{P}_2+r_4{P}_3,\hfill \end{array}$$ (13)

$$\begin{array}{cc}\hfill \frac{d{P}_2}{dt}& =-(r_2+r_5)P_2+r_1{P}_1+r_6{P}_4,\hfill \end{array}$$ (14)

$$\begin{array}{cc}\hfill \frac{d{P}_3}{dt}& =-(r_4+r_7)P_3+r_3{P}_1+r_8{P}_4,\hfill \end{array}$$ (15)

$$\begin{array}{cc}\hfill \frac{d{P}_4}{dt}& =-(r_6+r_8)P_4+r_5{P}_2+r_7{P}_3,\hfill \end{array}$$ (16)

in which P_i(t; r₂): i = 1, 2, 3, 4 is the probability of state g_i and r_j: j = 1, 2, …, 8 are the rates of transition defined in Table 3. Define

$$\begin{array}{cc}\hfill S_i(t;r_2)& =\frac{\partial P_i}{\partial r_2}i=1,2,3,4.\hfill \end{array}$$ (17)

The governing equations for the sensitivities S_i: i=1,2,3,4 are obtained by differentiating 13, 14, 15, 16 with respect to r₂,

$$\begin{array}{cc}\hfill \frac{d{S}_1}{dt}& =-(r_1+r_3)S_1+P_2+r_2{S}_2+r_4{S}_3,\hfill \end{array}$$ (18)

$$\begin{array}{cc}\hfill \frac{d{S}_2}{dt}& =-P_2-(r_2+r_5)S_2+r_1{S}_1+r_6{S}_4,\hfill \end{array}$$ (19)

$$\begin{array}{cc}\hfill \frac{d{S}_3}{dt}& =-(r_4+r_7)S_3+r_3{S}_1+r_8{S}_4,\hfill \end{array}$$ (20)

$$\begin{array}{cc}\hfill \frac{d{S}_4}{dt}& =-(r_6+r_8)S_4+r_5{S}_2+r_7{S}_3.\hfill \end{array}$$ (21)

Table 3.

Reaction stoichiometry and reaction rates for Sec. 3C.

Number	Reaction stoichiometry	Reaction rate (ri)
1	g1 → g2	100.0
2	g2 → g1	0.625
3	g1 → g3	100.0
4	g3 → g1	1.033
5	g2 → g4	0.99
6	g4 → g2	1.033
7	g3 → g4	0.99
8	g4 → g3	0.625

In Srivastava et al.,²³ we showed that the sQSPA model reduction leads to the reaction network shown in Figure 8.

Figure 8.

Reduced system of the pap regulatory network of Sec. 3C in the slow time scale regime.

In the sQSPA model reduction, we write probabilities P_i: i = 1, 2, 3, 4 in a power series expression given by

$$P_i=W_{i0}+{\epsilon }_{\text{sq}}{W}_{i1}+{\epsilon }_{\text{sq}}^2{W}_{i2}+O\left({\epsilon }_{\text{sq}}^3\right),$$

in which ${\epsilon }_{sq}=\frac{1}{r_1+r_3}$. By comparing $O\left({\epsilon }_{sq}^0\right)$ terms, the sQSPA model reduction finds the expression for W_i0, i = 1, 2, 3, 4. The master equation for the sQSPA reduced model is

$$\begin{array}{cc}\hfill W_{10}& =0,\hfill \end{array}$$ (22)

$$\begin{array}{cc}\hfill \frac{d{W}_{20}}{dt}& =-[\widetilde{r_1}+r_5]W_{20}+\widetilde{r_2}{W}_{30}+r_6{W}_{40},\hfill \end{array}$$ (23)

$$\begin{array}{cc}\hfill \frac{d{W}_{30}}{dt}& =-[\widetilde{r_2}+r_7]W_{30}+\widetilde{r_1}{W}_{20}+r_8{W}_{40},\hfill \end{array}$$ (24)

$$\begin{array}{cc}\hfill \frac{d{W}_{40}}{dt}& =-[r_6+r_8]W_{40}+r_5{W}_{20}+r_7{W}_{30},\hfill \end{array}$$ (25)

in which W_ij are the jth-order probabilities of state g_i: i = 1, 2, 3, 4, ${\tilde{r}}_1=r_2{r}_3/(r_1+r_3)$, and ${\tilde{r}}_2=r_1{r}_4/(r_1+r_3)$. By comparing $O\left({\epsilon }_{sq}^1\right)$ terms, one crucial equation²³ that comes out is the approximation of the probability ($P_{{\text{sq}}_1}$) of the rare state in terms of the probabilities of the other states satisfies

$$\begin{array}{cc}\hfill P_{{\text{sq}}_1}& ={\epsilon }_{\text{sq}}(r_2{W}_{20}+r_4{W}_{30})\hfill \end{array}$$ (26)

in which ε_sq = 1/(r₁ + r₃).

We are interested in the sensitivity of probability of the rare state g₁, with respect to r₂,

$$\begin{array}{cc}\hfill s(t;r_2)& =S_1=\frac{\partial P_1(t;r_2)}{\partial r_2}.\hfill \end{array}$$ (27)

To estimate s(t; r₂) of Eq. 27, we use three different estimators: CRN, CFD, and sQSPA with common random numbers (SRN). The CRN estimator is given as

$$\begin{array}{ccc}\hfill {\widehat{s}}_{\text{crn}}(t;r_2)& =& \frac{1}{N\epsilon }\sum _{i=1}^N[\mathbb{1}\left(p{a}_i^{\text{crn}}(t,r_2+\epsilon )=g_1\right)\hfill \\ & & -\mathbb{1}\left(p{a}_i^{\text{crn}}(t,r_2)=g_1\right)]\hfill \end{array}$$ (28)

in which $p{a}_i^{\text{crn}}(t;r_2)$ is the state of the pap operon at time t with rate parameter r₂ obtained through the ith CRN simulation, and N is the number of CRN simulations. A point to note is that the ith CRN simulation uses one SSA simulation with rate parameter r₂, i.e., $p{a}_i^{\text{crn}}(t;r_2)$, and one SSA simulation with rate parameter r₂ + ε, i.e., $p{a}_i^{\text{crn}}(t;r_2+\epsilon )$. The indicator random variable $\mathbb{1}\left(A\right)$ evaluates to 1 whenever the event A happens and 0 otherwise. In an analogous fashion, we have the CFD estimator for Eq. 27 as

$$\begin{array}{ccc}\hfill {\widehat{s}}_{\text{cfd}}(t;r_2)& =& \frac{1}{N\epsilon }\sum _{i=1}^N[\mathbb{1}\left(p{a}_i^{\text{cfd}}(t,r_2+\epsilon )=g_1\right)\hfill \\ & & -\mathbb{1}\left(p{a}_i^{\text{cfd}}(t,r_2)=g_1\right)].\hfill \end{array}$$ (29)

The SRN estimator estimates the sensitivity of $P_{{\text{sq}}_1}$ of Eq. 26 with respect to r₂. The SRN estimator is given by

$$\begin{array}{ccc}\hfill {\widehat{s}}_{\text{srn}}& =& \frac{{\epsilon }_{\text{sq}}}{\epsilon }[(r_2+\epsilon ){\widehat{W}}_{20}(t;r_2+\epsilon )+r_4{\widehat{W}}_{30}(t;r_2+\epsilon )\hfill \\ & & -r_2{\widehat{W}}_{20}(t;r_2)-r_4{\widehat{W}}_{30}(t;r_2)]\hfill \end{array}$$ (30)

in which ${\widehat{W}}_{20}(t;r_2)$ and ${\widehat{W}}_{20}(t;r_2+\epsilon )$ are obtained by simulating the reduced system shown in Figure 8 and governed by the master equation 22, 23, 24, 25 using common random numbers and SSA simulations. Figure 9 shows a comparison of the CRN, CFD, and SRN estimators. The ratio of root mean squared errors of CRN and CFD estimators is $\frac{e_{\text{crn}}}{e_{\text{cfd}}}=4.75$. As the number of reaction events is small in the pap operon, we also apply the likelihood gradient method.¹⁵ We performed 500 simulations for each of the four estimators. The likelihood method performs better than both CRN and CFD for this example, but it performs worse than SRN. The SRN estimator tracks the true sensitivity closely except for a small initial time. This example reveals the distinct advantage of analytical insight and model reduction, e.g., the sQSPA analysis, over the several proposed estimators that do not use the reduced model.

Figure 9.

Estimated sensitivity from the CRN, CFD, and SRN estimators for Sec. 3C. The SRN estimator tracks the true sensitivity closely except for a small initial time. The CFD estimator performs better than the CRN estimator. On the right, we see that the likelihood method performs better than both the CRN and CFD methods but it performs worse than SRN.

In this example, the number of reactions fired within the time interval of consideration was small, which is precisely when the likelihood method can produce a low variance estimator. In fact, in this particular application the likelihood method even outperformed the CFD method. For models with even a moderate number of reaction events, the likelihood method will not outperform CFD. We also note that in this example, the SRN method outperforms all other methods. However, SRN requires the ability to perform an analytic model reduction using sQSPA, which was possible in this example, thought not in general.

Sensitivity of a fast fluctuating species

In a stochastic simulation of the infection cycle of vesicular stomatitis virus (VSV), there is a fast fluctuation in a protein at low copy number along with a rapid increase in the population of the viral genome. Such a system is expensive to simulate because the frequency of the fluctuation increases as the simulation progresses leading to small time steps in the SSA simulation.²⁶ To illustrate the phenomenon, consider the following simple 3-species, 3-reaction system

$$A+G\stackrel{k_1}{\to }C+G,$$ (R3)

$$C+G\stackrel{k_2}{\to }2G+A,$$ (R4)

$$2G\stackrel{k_3}{\to }G$$ (R5)

with k₂ ∼ k₁ ≫ k₃. This reaction system describes the interaction of three species in a simplified VSV replication process – two forms of viral polymerase, A and C, and viral genome G. The two forms of the polymerase arise because VSV has two different complexes that serve as viral transcriptase and replicase.²⁷ The viral transcriptase form A is a complex of constituent VSV proteins L and P. The replicase form C is a complex of L, N, and P proteins. The species A is involved in the transcription reaction R3 to produce messenger RNA. The transcription reaction leads to the conversion of transcriptase A into replicase C. We further assume that produced mRNA from reaction R3 is short lived and hence we do not include it in the model. Species C and G are involved in replication reaction R4 to produce an additional viral genome G. The replication reaction R4 leads to the conversion of replicase C into transcriptase A. Finally, there is a second-order degradation reaction R5 of the viral genome. The model R3, R4, R5 is insufficient to predict the full viral infection cycle, but it is instructive in understanding the simulation challenges of the full infection cycle model used by Hensel et al.²⁶ The reaction rate constants k₁, k₂, k₃ denote macroscopic reaction rate constants with units μm³/(mol s). We express microscopic reaction rates in terms of macroscopic rate constants (k₁, k₂, k₃) and the system size Ω,

$$r_1=\frac{1}{\Omega }{k}_{1}ag,r_2=\frac{1}{\Omega }{k}_{2}cg,r_3=\frac{1}{\Omega }{k}_{3}g(g-1)$$

in which a, c, g represent the number of molecules of A, C, and G, respectively, and the system size appears because the reactions are second order. For the purposes of this example, we take Ω = 10⁵μm³. A stochastic simulation with the parameter values given in Table 4 is shown in Figure 10. Species G increases continuously and this increase forces species C to fluctuate with increasing frequency as shown in Figure 10.

Table 4.

Parameter values for Sec. 3D.

Parameter	$n_{{\text{A}}_0}$	$n_{{\text{C}}_0}$	$n_{{\text{G}}_0}$	k1	k2	k3	ε	N
Value	3	0	1	2 × 105	3 × 105	1	0.1	100

Figure 10.

A typical SSA simulation of the network of R3, R4, R5. (a) Counts of species C vs. time. (b) Counts of species G vs. time.

We want to investigate the sensitivity

$$\begin{array}{cc}\hfill s(t;k_3)& =\frac{d\mathbb{E}X(t,k_3)}{d{k}_3}.\hfill \end{array}$$ (31)

In which X ∈ {C, G}. An estimator of the sensitivities of interest 31 is

$$\begin{array}{cc}\hfill {\widehat{s}}_{\text{est}}& =\frac{1}{N}\sum _{i=1}^N\frac{X_i^{\text{est}}(t;k_3+\epsilon )-X_i^{\text{est}}(t;k_3)}{\epsilon }\hfill \end{array}$$ (32)

in which est ∈ {CRN, CFD}. Figure 11 shows a comparison of the standard deviations of the two estimators, CRN and CFD, obtained using Eq. 4. The parameters used to generate Figure 11 are shown in Table 4. Figure 11a shows that for the abundant species G, CFD, and CRN estimators have similar standard deviations, which demonstrates that both CRN and CFD are capable of obtaining good sensitivity estimates for the abundant species G. Figure 11b shows that for fast fluctuating species C, the CFD estimator has less than one third the standard deviation of the CRN estimator, which demonstrates that the CFD estimator captures the sensitivity of the fast fluctuating species C better than the CRN estimator. This example again demonstrates the superiority of the CFD estimator over the CRN estimator.

Figure 11.

Comparison of standard deviations of CFD and CRN estimators for Sec. 3D: (a) Species G and (b) Species C.

Sensitivity of genetic toggle switch

We conclude with a model of a genetic toggle switch^{1, 2, 28}

$$\varnothing \underset{1}{\stackrel{{\lambda }_1}{\rightleftarrows }}A,\varnothing \underset{1}{\stackrel{{\lambda }_2}{\rightleftarrows }}B,$$ (33)

where the respective propensities are

$${\lambda }_1\left(t\right)=\frac{b}{1+X_B{\left(t\right)}^{\beta }}\text{and}{\lambda }_2\left(t\right)=\frac{a}{1+X_A{\left(t\right)}^{\alpha }},$$

and where X_A(t) and X_B(t) will denote the number of gene products from the two interacting genes. As in Refs. ^{1, 2, 28}, we take parameter values of b = 50, β = 2.5, a = 16, and α = 1. We consider the derivative of the expectation of X_A with respect to α at the value one, with [X_A(0), X_B(0)] = [0, 0] as our choice of initial condition. We will consider the behavior of three finite difference methods on this example: CFD, CRN, and CRP.² For each of the three finite difference methods employed, we use a perturbation of ε = 1/50.

Figure 12 shows a comparison of the standard deviations of the three estimators. We again see the superiority of the CFD estimator over both the CRN and CRP estimators. Further, starting at time t = 5s, the standard deviation of the CRN estimator is less than that of the CRP estimator. This result demonstrates that it is not always the case that CRP is superior to CRN, something not previously observed in the literature.

Figure 12.

Comparison of standard deviations of CFD, CRN, and CRP estimators on the model 33.

CONCLUSIONS

In this paper, we compared the performance of several finite difference sensitivity estimators on a number of examples not previously considered in the literature. In all the examples and sensitivities of interest, we found that the newly developed CFD estimator performs significantly better than the CRN estimator, which is currently the most commonly used method. Further, in Sec. 3E we provided a case in which the CRN estimator is superior to the CRP estimator.

In previous work, it had been shown that the CFD estimator performs better than the CRP estimator.¹ The comparisons made in this paper, along with Anderson's previous results, lead to the conclusion that CFD is currently the best available estimator in the class of finite difference estimators of stochastic chemical kinetic models.

Estimating sensitivities of stochastic chemical kinetic models accurately and efficiently remains an important problem. With variance reduction ideas incorporated in the CFD estimator through a tight coupling of the nominal and perturbed systems, we believe this estimator has significant advantages over previous estimators without such a coupling. The CFD estimator has significant potential for application in parameter estimation where it can provide accurate estimates of the Hessian of the objective function. Recently, Wolf and Anderson²⁸ have also proposed a CFD2 estimator to directly estimate the Hessian of the objective function. Evaluating the effectiveness of CFD and CFD2 estimators in parameter estimation of stochastic chemical kinetic models is a topic of ongoing research.