Time-Ordered Product Expansions for Computational Stochastic Systems Biology

Abstract

The time-ordered product framework of quantum field theory can also be used to understand salient phenomena in stochastic biochemical networks. It is used here to derive Gillespie’s Stochastic Simulation Algorithm (SSA) for chemical reaction networks; consequently, the SSA can be interpreted in terms of Feynman diagrams. It is also used here to derive other, more general simulation and parameter-learning algorithms including simulation algorithms for networks of stochastic reaction-like processes operating on parameterized objects, and also hybrid stochastic reaction/differential equation models in which systems of ordinary differ-ential equations evolve the parameters of objects that can also undergo stochastic reactions. Thus, the time-ordered product expansion (TOPE) can be used systematically to derive simulation and parameter-fitting algorithms for stochastic systems.

1 Introduction

The master equation for a continuous-time stochastic dynamical system may be expressed as dp/dt = W · p where W is the time-evolution operator, often an infinite-dimensional matrix. Particular choices for W lead to the special case of the “chemical master equation” for stochastic chemical kinetics, often useful in bioligical applications; we will see this and other applications below. The general master equation has the formal solution p(t) = exp(tW) · p(0). If W can be decomposed as a sum W₀ + W₁, then there is a perturbation theory for exp(tW) in terms of exp(tW₀) and its perturbations by W₁. The time-ordered product expansion (which we refer to by the acronym TOPE) gives a formula for the solution of a master equation [1–3] which can be expressed as follows [4]:

$$\begin{array}{l}exp(tW)·p_0=exp(t(W_0+W_1))·p_0\\ =\sum _{k=0}^{\infty }[{\int }_0^t{dt}_k{\int }_0^{t_k}{dt}_{k-1}\cdots {\int }_0^{t_2}{dt}_{1}exp((t-t_k)W_0)·W_1\\ ·exp((t_k-t_{k-1})W_0)\cdots ·W_1·exp(t_1{W}_0)·p_0]\end{array}$$ (1)

This expression can be derived (as in [4]) by expanding in powers of W₁, each expanded to all orders in W₀, and using the normalization formula for the Dirichlet distribution to subdivide the time interval [0, t]) into k subintervals.

Since W₀ and W₁ do not generally commute, this expression involves alternation from right to left of W₀ and W₁ related operations. Using the “time-ordered exponential” of operators [5], this result can be compactly reexpressed as:

$$exp(t(W_0+W_1))=exp({tW}_0){\left(exp({\int }_0^t{W}_1(\tau )d\tau )\right)}_{+}$$ (2)

where

$$W_1(\tau )\equiv exp(-\tau W_0)W_{1}exp(\tau W_0).$$ (3)

Here ${(exp({\int }_0^{t}G(\tau )d\tau ))}_{+}$ is obtained term by term from the Taylor series for the operator exponential, by reordering all monomials containing terms evaluated at different times so that they are indexed by ordered sequences of times (τ_k, …, τ₁) that increase right to left (details reviewed in Section 3.5.1 below). In field theory it is standard to prefer Equation 2 over Equation 1 for theoretical calculations, but for algorithmic concreteness this paper will favor the more explicit expression Equation 1 where possible. Indeed, each summand of Equation 1 already looks like a Markov chain in which the matrix or operator product operation “·”. which sum over states, is supplemented by integration over an extra time variable. This observation will be made precise in Section 3. In general there is a risk that the infinite sum over terms could diverge. However, the master equation must conserve total probability and this constrains W to have zero column-sums and also constrains the spectrum of W to have nonpositive real parts. In this setting some decompositions W = W₀ + W₁ converge well enough, as we will see by example below.

One particular specialization of TOPE lets us derive Gillespie’s Stochastic Simulation Algorithm (SSA): take W₀ = −D = the diagonal part of W, and W₁ = Ŵ = the off-diagonal part of W. Then for chemical reaction networks TOPE generates Feynman-like diagrams. An example is illustrated below for the simple reaction network with just two reactions, the forwards and backwards parts of the generic trivalent reaction A + B ⇌ C, to which others can be reduced.

The TOPE (Equation 1 or Equation 2) can be applied recursively, since it reduces one operator exponential exp(tW) to another one exp(tW₀). This fact will be exploited in Section 3 below. But eventually one must get to an operator exponential that is tractable by other means. One way to do this is to let W₀ = D = the diagonal part of W, as in the SSA algorithm derivation below.

2 Methods

2.1 Creation/annihilation operator notation

We will use operator notation for molecule (or other reactant) creation and annihilation state changes [1–4]. Here we just review the notation as used in [4]. The elementary operators a and â act (respectively) to destroy and create identical particles of a given type. In the particle-number basis their elements have the entirely off-diagonal expressions

$$a_{ij}=j{\delta }_{ij-1}\text{and}{\widehat{a}}_{ij}={\delta }_{ij+1}\text{forall}i,j\in \{0,1,2,\dots \}.$$ (4)

Here δ_ij is the Kronecker delta function. The creation and annihilation operators satisfy the Heisenberg algebra [a, â] = I but are different from those of quantum mechanics because they are not conjugates or transposes of one another. (This is the reason we do not denote the creation operator a^†, as it is in quantum mechanics, or a^*.) Instead of being conjugate to â, the annihilator a encodes the chemical law of mass action since its nonzero entries are equal to the number of particles available to react or decay. The diagonal “number operator” is N ≡ âa.

The creation and annihilation operators may be represented in terms of their action on probability generating functions $g(z)={\sum }_{n=0}^{\infty }{p}_n{z}^n$, where p_n is the probability that there exist n particles of a given type. In this case:

$$a={\partial }_z\cdots \mathit{and}\widehat{a}=z\times \cdots$$ (5)

In the presence of different types of particles (eg. molecules or other objects) the creation/annihilation operator notation is generalized, eg to a_α and â_β for molecule types A_α, in which all operators for unequal types commute:

$$[a_{\alpha },{\widehat{a}}_{\beta }]\equiv a_{\alpha }{\widehat{a}}_{\beta }-{\widehat{a}}_{\beta }{a}_{\alpha }={\delta }_{\alpha \beta }$$

Operating on an empty “vacuum” state |0〉 with no objects, the monomials in the creation operators â_β span a Fock space. Molecule or object types indexed by α may even be taken to include arbitrary discrete-valued molecular attributes (or attributes of other objects) such as phosphorylation state or integer-valued parameters. Continuous-valued parameters such as position (in quantum field theory it would more naturally be the conserved momentum, unlike the typical viscous-medium dynamics in biology) may be encoded into a real-valued vector argument x which requires a Dirac delta function instead of a Kronecker delta function, so for example:

$$[a_{\alpha }(x),{\widehat{a}}_{\beta }(y)]={\delta }_{\alpha \beta }\delta (x-y)$$ (6)

A non-molecular example of such parameterized objects would be: cells of a given real-valued volume and/or lengthscale as in Section 3.5.5 below.

However for some attributes such as real-valued object positions one may wish to limit the state space to between zero or n_max,α molecules (or other objects) at each unique real value. The resulting commutator is still diagonal as described in [4]. The particular case n_max,α = 1 is not a stochastic version of fermions because particles with different types or values of the attributes still commute rather than anticommuting.

The basic rule for translating chemical reactions into creation/annihilation operator notiation is: first, annihilate all objects on the incoming or left hand side of a reaction; then create all the objects on the outgoing or right hand side of a reaction. Thus, the off-diagonal part of the operator for a reaction

$$\begin{array}{l}{\{A_{\alpha (p)}(x_p)\mid 1\le p\le p_{max}\}}_{\ast }\to {\{A_{\beta (q)}(y_q)\mid 1\le q\le q_{max}\}}_{\ast }\\ \mathbf{with}\text{reaction}\text{rate}{\rho }_r({[x_p]}_1^{p_{max}},{[y_q]}_1^{q_{max}})\end{array}$$

that converts an incoming multiset {···}_* of numerically parameterized reactants {A_α(p)(x_p)|p ∈ lhs(r)}_* each with parameter vector x_p (reactants can appear multiple times in a multiset) into an outgoing multiset {A_β(q)(y_q)|q ∈ rhs(r)}_* each with parameter vector y_q, is:

$${\widehat{O}}_r={\rho }_r([x_p],[y_q])\left[\prod _{q\in \text{rhs}(r)}{\widehat{a}}_{\beta (q)}(y_q)\right]\left[\prod _{p\in \text{lhs}(r)}{a}_{\alpha (p)}(x_q)\right].$$ (7)

There is one such operator for every possible set of values for the numerical parameters. Since time-evolution operators for different processes just add, a generic operator for all parameter values must sum and/or integrate the operator of Equation 18 over all the parameters, in the Cartesian product of measure spaces in which they take values:

$${\widehat{O}}_r=\int \int d\{x\}d\{y\}{\rho }_r([x_p],[y_q])\left[\prod _{q\in \text{rhs}(r)}{\widehat{a}}_{\beta (q)}(y_q)\right]\left[\prod _{p\in \text{lhs}(r)}{a}_{\alpha (p)}(x_q)\right]$$ (8)

The generalization is conceptually straightforward because we have simply used a function ρ_r([x_a], [y_b]) to express the possibly infinite number of different reaction rates that pertain to objects that differ only in their attributes. Because of the algebra of noncommuting basic creation and annihilation operators, reaction operators Ô_r and Ô_r′ for reactions r ≠ r′ that produce and consume a shared reactant A_α(x) (or A_α for reactants with type α but no other parameters) generally also have nonzero commutators.

Equation 18 or Equation 8 add probability to the new state of the system, but do not take it away from the old state of the system before a reaction. That job requires a negative diagonal matrix as shown in Equation 10 below. In the case of Equation 18, the corresponding diagonal operator is $D_r={\rho }_r[{\displaystyle \prod _{a\in \text{lhs}(r)}}{N}_{\alpha (p)}(x_p)]$. Examples are provided in [4] and below.

2.2 Solvable example: An exact solution for SSA behavior

For a few very simple examples, we can not only solve analytically for the behavior of the biochemical system, but we can even add in the behavior of the SSA simulation algorithm and solve for that exactly as well. For example consider the minimal bidirectional reaction A ↔ Ø. This case is analytically solvable, including the complete statistics of its SSA algorithm simulation. It has forward synthesis and backwards decay reactions. The operator expression is therefore:

$$W=k_s(\alpha \widehat{a}-I)+k_d(\alpha a-N)$$ (9)

Here α = 1 is the generating function variable for the total number of reactions, corresponding to off-diagonal matrix elements of W. Power series in α will decompose total probability according to this number.

Translating the master equation for Equation 9 into a PDE in the two variables t and z using representation Equation 5, and solving analytically, this model has the exact solution

$$\begin{array}{l}{g}_m(z,t\mid \alpha )={(\alpha +(z-\alpha )e^{-k_{d}t})}^{m}exp[-\frac{k_s}{k_d}((1-\alpha )k_{d}t+(z\alpha -{\alpha }^2)(e^{-k_{d}t}-1))]\\ ={(\alpha +(z-\alpha )e^{-k_{d}t})}^{m}exp[\frac{k_s}{k_d}(z\alpha -1)(1-e^{-k_{d}t}))]exp[\frac{k_s}{k_d}({\alpha }^2-1)(k_{d}t+e^{-k_{d}t}-1))]\\ =\text{Binomialinitialconditionwithdecay}\ast \text{Poissononforwardreactions}\\ \ast \text{Poissononforward}/\text{backdwardreactionpairs}.\end{array}$$

As usual z is the generating function variable whose exponent is the total number n_A of A molecules or particles, m is the initial number of molecules, and t is continuous time. The * operation is a convolution of probability distributions. A product of generating functions with the same variable is a convolution of distributions [7]. Note the interpretation in terms of Binomials and Poissons with time-varying parameters. The third factor represents a linearly increasing number of canceling forward/backward reaction pairs as a function of time - a kind of random walk.

The full derivation below will generalize this solvable example, again separating the diagonal from the off-diagonal terms in W.

2.3 Notation for SSA rederivation from TOPE

One specialization of TOPE lets us derive SSA for biochemical reaction networks, as follows. First decompose W into nonegative off-diagonal and non-positive diagonal parts, as must be possible by the conservation and nonnegativity of probability. For example conservation of probability implies ∀p 0 = d(1 · p)/dt = (1 · W) · p ⇒ 1 · W = 0. Then

$$\begin{array}{l}W=\widehat{W}-D,\text{where}D\triangleq \text{diag}(\mathbf{1}·\widehat{W}),\text{i}.\text{e}.\\ {\widehat{W}}_{IJ}\triangleq (1-{\delta }_{IJ})W_{IJ}\text{and}{D}_{IJ}\triangleq {\delta }_{IJ}\sum _K{\widehat{W}}_{KJ}\triangleq {\delta }_{IJ}{D}_I\end{array}$$ (10)

where I and J index the possible states of the system. To prevent negative probabilities from evolving under the master equation, all entries of Ŵ and therefore D must be nonnegative. In this circumstance the TOPE becomes:

$$\begin{array}{l}exp\left(t\left(\widehat{W}-D\right)\right)=\sum _{k=0}^{\infty }[{\int }_0^t\cdots {\int }_0^t({\prod }_{q=0}^{k}d{\tau }_q)\delta \left(t-\sum _{q=0}^k{\tau }_q\right)\times exp(-{\tau }_{k}D)\widehat{W}\cdots exp(-{\tau }_{1}D)\widehat{W}exp(-{\tau }_{0}D)]\\ =\sum _{k=0}^{\infty }{\int }_0^t\cdots {\int }_0^t({\prod }_{q=0}^{k}d{\tau }_q)\delta \left(t-\sum _{q=0}^k{\tau }_q\right)exp(-{\tau }_{k}D)\left[\prod _{q=k-1\downarrow }^0\widehat{W}exp(-{\tau }_{q}D)\right]\end{array}$$

Since the summands over k and integrants over ${[{\tau }_q]}_0^k$ are mutually exclusive, exhaustive and nonnegative, we define the conditional probability distribution on k and ${[{\tau }_q]}_0^k$ by these summand/integrands (where ${[{\tau }_q]}_0^k\triangleq [{\tau }_0,\dots {\tau }_k]$ denotes an ordered contiguous sequence of time intervals):

$$Pr(I,{[{\tau }_q]}_0^k,k\mid J,t)\triangleq {\left\{exp(-{\tau }_{k}D)\left[\prod _{q=k-1\downarrow }^0\widehat{W}exp(-{\tau }_{q}D)\right]\delta \left(t-\sum _{q=0}^k{\tau }_q\right)\right\}}_{I,J}$$ (11)

(where a product over zero terms such as ${\displaystyle \prod _{q=-1\downarrow }^0}$ is interpreted as the identity matrix, and products over negative numbers of terms such as ${\displaystyle \prod _{q=-2\downarrow }^0}$ should not occur). For D_II ≠ 0,

$$Pr(I,{[{\tau }_q]}_0^k,k\mid J,t)={\left\{exp(-{\tau }_{k}D)\left[\prod _{q=k-1\downarrow }^0\left(\begin{array}{cc}\widehat{W}& D^{-1}\end{array}\right)(Dexp(-{\tau }_{q}D))\right]\delta \left(t-\sum _{q=0}^k{\tau }_q\right)\right\}}_{I,J}$$

Either way,

$$Pr(I\mid J,t)=\sum _{k=0}^{\infty }{\int }_0^t\cdots {\int }_0^t({\prod }_{q=0}^{k}d{\tau }_q)Pr(I,{[{\tau }_q]}_0^k,k\mid J,t)={\left[exp(t\left(\widehat{W}-D\right))\right]}_{I,J}.$$

The bracket notation ${[X_q]}_{min}^{max}\triangleq [{\tau }_{min},\dots {\tau }_{max}]$ for an ordered set of components indexed by q will also be used for state variables ${[I_q]}_{min}^{max}$. The notation “≜” means “equal by definition”. In what follows, the notation “Θ(Pred)” where Pred is a predicate is the Heaviside step function or indicator function taking the value 1 if the predicate is true and 0 if it is false.

2.4 Semigroup property

Suppose t = t₁ + t₂, all nonnegative. Then for any time-evolution equation we must have the semigroup property:

$$Pr(I\mid J,t)=\sum _{K}Pr(I\mid K,t_2)Pr(K\mid J,t_1).$$

Is there a k-event version of this rule, for k = k₁ + k₂? In other words, can we add (nonnegative) numbers of reaction events rather than time intervals? We observe (where again ${[{\tau }_q]}_0^k\equiv [{\tau }_0,\dots {\tau }_k]$)

$$Pr(I,k\mid J,t)={\int }_0^t\cdots {\int }_0^t({\prod }_{q=0}^{k}d{\tau }_q)Pr(I,{[{\tau }_q]}_0^k,k\mid J,t).$$

Then, according to a derivation given in Appendix I, if k = k₁ + k₂ and for any ${\tau }_{k_1}^{\prime }\in [0,{\tau }_{k_1}]$, there is a semigroup law:

$$Pr(I,{[{\tau }_q]}_0^k,k\mid J,t)=\sum _K{\int }_0^{t}d\tau Pr(I,[{\tau }_{k_1}^{\prime },{\tau }_{k_1+1},\dots {\tau }_k],k_2\mid K,\tau )\times Pr(K,\left[{\tau }_0,\dots {\tau }_{k_1-1},{\tau }_{k_1}-{\tau }_{k_1}^{\prime }\right],k_1\mid J,t-\tau ).$$ (12)

In this result there is an arbitrary choice of ${\tau }_{k_1}^{\prime }$ from the interval [0, τ_k1]. However this form does not yet pertain to conditional probabilities of the form Pr(I, t|k, J), as needed to obtain a computable Markov process algorithm.

3 Results and discussion

Given the foregoing notation, we undertake the derivation of a Markov chain representing the SSA algorithm. We then consider extensions of this result, including parameterized reactants, but focussing mainly on hybrid stochastic event/ordinary differential equation dynamical systems.

3.1 Derivation of a Markov chain

3.1.1 Bayesian recurrence

In Appendix I we argue that the correct Bayesian strategy for moving from Pr(I, k|t, J) to Pr(I, t|k, J), as needed to obtain a simulatable Markov chain, is to consider large stopping times T ≫ t which are overwhelmingly likely to have large reaction numbers n ≫ k; then to marginalize the probability distribution Pr([I], [τ], n|J, T) over all event numbers n > k and to conditionalize it over all event numbers q < k. By that means in Appendix I we derive the recurrence relation

$$\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)=\stackrel{\sim }{Pr}(I_k,{\tau }_{k-1}\mid 1,I_{k-1})\stackrel{\sim }{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k-1,J)$$ (13)

where

$$\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)\triangleq \underset{T\to \infty }{lim}\sum _{n=k+1}^{\infty }\sum _{\{{[I_q]}_{k+1}^n\}}{\int }_0^T\cdots {\int }_0^T{[d{\tau }_q]}_k^{n}Pr({[I_q]}_1^n,{[{\tau }_q]}_0^n,n\mid J,T)$$ (14)

and in particular

$$\begin{array}{l}\stackrel{\sim }{Pr}(I_k,{\tau }_{k-1}\mid 1,I_{k-1})=\frac{\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)}{\stackrel{\sim }{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k-1,J)}\\ ={\widehat{W}}_{I_k{I}_{k-1}}exp(-{\tau }_{k-1}{D}_{I_{k-1}})\mathrm{\Theta }({\tau }_{k-1}\ge 0).\end{array}$$ (15)

Note that $\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k.J)$ marginalizes over τ_k, the time elapsed since the last event k, as well as all later times and events. It is a distribution on histories up to and including the “just-fired” k’th reaction event, within a much longer history.

3.1.2 Markov chain - Summary

From the forgoing Bayesian recurrence equation, and the definition

$$\stackrel{\sim }{Pr}(I,t_k\mid k,J)\triangleq \sum _{\{{[I_q]}_1^{k-1}\}}{\int }_0^{\infty }\cdots {\int }_0^{\infty }{[d{\tau }_q]}_0^{k-1}\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)\delta \left(t_k-\sum _{q=0}^{k-1}{\tau }_q\right),$$

Appendix I shows $\stackrel{\sim }{Pr}(I,t_k\mid J,k)$ is a probability density function and proves the following Markov-like property:

$$\stackrel{\sim }{Pr}(I,t\mid k,J)=\sum _K{\int }_0^{t}d\tau \stackrel{\sim }{Pr}(I,\tau \mid 1,K)\stackrel{\sim }{Pr}(K,t-\tau \mid k-1,J).$$

We may reexpress this result as

$$\stackrel{\sim }{Pr}(I,t\mid k,J)\approx \sum _K{\int }_0^{t}d\tau \mathcal{W}(I,t\mid K,t-\tau )\stackrel{\sim }{Pr}(K,\tau \mid k-1,J)$$

where we define the Markov chain kernal

$$\mathcal{W}(I,t\mid K,t-\tau )\triangleq \stackrel{\sim }{Pr}(I,\tau \mid 1,K)={\widehat{W}}_{IK}exp(-\tau D_K)\mathrm{\Theta }(\tau \ge 0)$$ (16)

In vector/operator notation, for k ≥ 2,

$$\stackrel{\sim }{Pr}(.,.\mid k,J)\equiv \mathcal{W}\circ \stackrel{\sim }{Pr}(.,.\mid k-1,J)$$

and finally the algorithmic Markov chain expression for SSA including now an initial distribution $\stackrel{\sim }{Pr}(J\mid k=0)$ over J at time t = 0, we have for all k ≥ 0:

$$\stackrel{\sim }{Pr}(.\mid k)={\mathcal{W}}^k\circ \stackrel{\sim }{Pr}(.\mid k=0)$$ (17)

which expresses the iteration of a Markov chain of the SSA algorithm. Of course the factor [D_K exp(−τD_K)Θ(τ ≥ 0)] in Equation 16 is just the SSA exponential distribution of non-negative waiting times τ between reaction events, and Ŵ_IK/D_K is just the branching probability for immediately thereafter chosing a reaction that leads to state I.

The foregoing derivation was outlined in far less detail in [8]. A similar equation for SSA was reached by very different methods in [9], Theorem 10.1. To our knowledge this is the first complete derivation of SSA from field theory methods such as TOPE.

This Markov chain expression has also been used as the starting point for the derivation of exact accelerated stochastic simulation algorithms [10,11] that execute many reactions per step (i.e. they “leap” forward) and thus go much faster than SSA, while also sampling from the exact probability distribution given by the just-fired probabilities above. These derivations proceed by algebraic rearrangement of terms to express computationally efficient versions of rejection sampling. The algorithm of [11] has been parallelized, which is often difficult for discrete-event simulations.

3.2 Algorithm: SSA

The SSA algorithm represented by the Markov chain in Equation 16 and Equation 17 above may be written out in pseudocode as follows:

repeat {
   compute propensities k(r)
   compute $k^{(\text{total})}={\displaystyle \sum _r}{k}^{(r)}$
   draw waiting time Δt from k(total) exp(−Δtk(total))
     t: = t + Δt; // advance the clock by Δt
   draw reaction r from distribution k(r)/k(total)and execute reaction r
} until t ≥ tmax

3.3 Extension: Parameterized rule and graph grammar SSA-like algorithm

For biological modeling, including spatial and mechanical modeling of biological systems, it is important to generalize from pure particles to particles with both discrete and continuous attributes. The complication is that reaction or process rates can then depend on the attributes both of the incoming and outgoing objects. A non-molecular example of such parameterized objects would be cells of a given size, whose propensity to divide may actually depend on their real-valued size parameter (as in Section 3.5.5). More generally, this capability enables agent-based modeling and simulation since it allows interacting objects to have dynamic internal state and even (as explained in Section 3.3.2 below) dynamic relationships.

The time-evolution operator of Equation 18 requires that each attribute or parameter vector consist of constants or variables, each variable appearing just once, and any relationships between variables (such as x_2,1 = y_1,2, x_2,1 = x_1,2, and/or y_2,1 = y_1,2) enforced by the reaction rate ρ_r([x_p], [y_q]). Alternatively we can allow repeated appearances of symbolic variables X_c upon which the attributes may depend, through the identity function or otherwise. This is a useful improvement in reaction notation which however may require special-purpose symbolic variable-binding algorithms to support efficiently. Generalizing from Equation 18 and Equation 8, as in [4], we include all possible instantiations of parameters x_p[X] and y_q[X], allowing for repeated occurences of some or all of the variables X_c in [X], with the integrated off-diagonal process representation operator

$${\widehat{O}}_r=\int \cdots \int d{\mu }_c(X_c){\rho }_r([x_p[X]],[y_q[X]])\times \left[\prod _{q\in \text{rhs}(r)}{\widehat{a}}_{\beta (q)}(y_q[X])\right]\left[\prod _{p\in \text{Ihs}(r)}{a}_{\alpha (p)}(x_p[X])\right].$$ (18)

As before, α(p) and β(q) represent the type of a parameterized object i.e. an object with attributes. Now the symbolic variables X_c each have a type c which has an integration measure μ_c. Again (as in Equation 18 or 8), summation over all discrete-valued parameters and integration over all continuous-valued parameters generalizes the operator to handle all possible sets of parameter values.

3.3.1 Algorithm: SSA with parametrized reactant objects

The resulting variant of the SSA algorithm for parameterized reactions can be expressed in pseudocode as follows (outlined briefly in [8]):

forall reactions r factor ρ(r)(xin, xout) = k(r)(xin)p(r)(xout|xin);
repeat {
   compute SSA propensities as k(r)(xin);
   compute $k^{(\text{total})}={\displaystyle \sum _r}{k}^{(r)}(x_{\text{in}})$;
   draw waiting time Δt from k(total) exp(−Δtk(total));
     t: = t + Δt; // advance the clock by Δt
   draw reaction r from distribution k(r)(xin)/k(total);
   draw xout from p(r)(xout|xin) and execute reaction r;
} until t ≥ tmax

3.3.2 Structural matching

The functions ρ(x, y) appearing in Equation 18 may impose constraints including equality of variables; equivalently we may allow some variables to appear multiple times in object parameter lists. Either way there follows a mechanism to encode structural relations - graphs and labelled graphs - in the input and output variable lists. Object attributes can include Object ID codes which other objects can also include in their parameter lists. (Of course, the numeric values of Object IDs can be globally permuted without changing the structural relationships among extant objects.) In this way, the integrated version of the parameterized reaction operator above encodes structural pattern matching, including variable-binding in logical formulae, among the preconditions that can be enforced before such a generalized reaction or “rule” can fire.

From this point of view, syntactic variable-binding has the semantics of multiple integration [4]. In this way we can entrain pattern-matching systems such as the computer algebra system Mathematica, or logic-based programming languages, to the job of simulating complex process rules. As in rule-based expert systems, when multiple rules might fire the Rete algorithm [12] can be used to speed up the computations required to maintain knowledge of their relative rates.

The resulting systems have the power to model and simulate dynamic labelled graphs including growing multicellular tissues with dynamical cell-neighbor relationships [4] and molecular complexes with dynamical binding structure [13–15]. Thus, the TOPE operator algebra approach also explains why and how structural (graph-) matching computations arise naturally in biochemical and multicellular biological simulation.

3.4 Hybrid SSA/ODE setup

As will be shown in Section 3.4.1 below, the operator formulation for a system of ordinary differential equations is [4]:

$${\widehat{O}}_{\text{drift}}=-\int \int d\{x\}d\{y\}\widehat{a}(\{y\})a(\{x\})\left[\sum _i{\nabla }_{y_i}\left(v_i(\{y\})\prod _k\delta (y_k-x_k)\right)\right]$$ (19)

Here and in the calculations that follow, the Dirac delta function can be considered as a Gaussian with very small variance, which participates in a limiting process by which, at the end of each calculation, the limit of zero variance is taken.

In [4] this operator expression is generalized from ordinary differential equations to stochastic differential equations, for example those pertaining to the diffusion of particles, as equivalently represented by the Fokker-Planck equation.

3.4.1 Computation of matrix elements

From the commutator

$$[a(y),\widehat{a}(x)]=\delta (y-x)(I+Q(N(x)\mid n_{max})N(x)),$$

we may calculate matrix elements of Ô_drift in Equation 18 such as:

$$\begin{array}{l}\langle w\left|{\widehat{O}}_{\text{drift}}\right|z\rangle =-\langle \{w\}|\int d\{x\}\int d\{y\}\left(\sum _i{\nabla }_{y_i}{v}_i(\{y\})\delta (\{y\}-\{x\})\right)\times \widehat{a}(\{y\})a(\{x\})\widehat{a}(\{z\})|0\rangle \\ =-\langle \{w\}|\int d\{x\}\int d\{y\}\left(\sum _i{\nabla }_{y_i}{v}_i(\{y\})\delta (\{y\}-\{x\})\right)\times \widehat{a}(\{y\})\delta (\{x\}-\{z\})[I+Q(N(x))N(\{x\})]|0\rangle \\ =-\int d\{x\}\int d\{y\}\left(\sum _i{\nabla }_{y_i}{v}_i(\{y\})\delta (\{y\}-\{x\})\right)\delta (\{x\}-\{z\})\langle \{w\}\mid \{y\}\rangle \\ =-\int d\{y\}\left(\sum _i{\nabla }_{y_i}{v}_i(\{y\})\delta (\{y\}-\{z\})\right)\delta (\{w\}-\{y\})\\ =+\int d\{y\}\delta (\{y\}-\{z\})\left(\sum _i{v}_i(\{y\}){\nabla }_{y_i}\delta (\{w\}-\{y\})\right)\\ -{\int }_{\partial }d\{y\}\left(\sum _i{v}_i(\{y\})\delta (\{y\}-\{z\})\delta (\{y\}-\{w\})\right)\\ =\sum _i{v}_i(\{z\})\nabla z_i\delta (\{w\}-\{z\})+\text{boundaryterm}(\to 0\text{here})\end{array}$$

The easiest treatment for the boundary terms is to add the assumptions that boundaries are at infinity in the space of parameters x, y and z, and that initial conditions place zero probability there, and that finite velocities v(x) ensure the probability remains zero at infinity at finite times. In that case boundary terms can be neglected. Alternatively, we can define O_drift = Ô_drift − diag(1·Ô_drift) which in this case subtracts off the boundary term. Then

$$O_{\text{drift}}=-\int \int d\{x\}d\{y\}(\widehat{a}(\{y\})a(\{x\})-\widehat{a}(\{x\})a(\{x\}))\times \left[\sum _i{\nabla }_{y_i}\left(v_i(\{y\})\prod _k\delta (y_k-x_k)\right)\right]$$ (20)

If we define x(t) as a time-varying version of z, satisfying

$$\frac{\partial x_i}{\partial t}=v_i(\{x_k\}),$$

then

$$\sum _i{v}_i(\{z\}){\nabla }_{x_i}\delta (\{w\}-\{x(t)\})=\sum _i\left(\frac{\partial x_i}{\partial t}\right)\left(\frac{\partial }{\partial x_i}\right)\delta (\{w\}-\{x(t)\})=\left(\frac{d}{dt}\right)\delta (\{w\}-\{x(t)\})$$

Next we calculate 〈w|exp(τO_drift)|z〉. To this end, Taylor’s theorem may be written

$${\text{Shift}}_{\tau }\circ f(t)=f(t+\tau )\simeq \sum _{n=0}^{\infty }\frac{{\tau }^n}{n!}{\left(\frac{d}{dt}\right)}^{n}f(t)\simeq e^{(\tau D_t)}f(t)$$

if τ is a constant. For small τ we have

$$\begin{array}{l}\langle w\mid exp(\tau O_{\text{drift}})\mid x\rangle =\langle w\mid x\rangle +\tau \langle w\mid O_{\text{drift}}\mid x\rangle +O({\tau }^2)\\ =\left(1+\tau (\frac{d}{dt})\right)\delta (\{w\}-\{x(t)\})+O({\tau }^2)\\ ={\text{Shift}}_{\tau }\delta (\{w\}-\{x(t)\})\equiv \delta (\{w\}-\{x(t+\tau )\})+O({\tau }^2)\end{array}$$

For larger τ we have

$$\begin{array}{l}\langle w\mid exp(\tau O_{\text{drift}})\mid x\rangle ={lim}_{n\to \infty }\langle w\left|\prod _{i=1}^{n}exp(\frac{\tau }{n}{O}_{\text{drift}})\right|x\rangle \\ =\int \cdots \int {dx}_{n-1}\cdots {dx}_1[{\text{Shift}}_{\tau /n}\delta (\{w\}-\{x_{n-1}\})]\cdots [{\text{Shift}}_{\tau /n}\delta (\{x_1\}-\{x\})]\\ ={lim}_{n\to \infty }\left(\prod _{i=1}^n{\text{Shift}}_{\tau /n}\right)\delta (\{w\}-\{x(t)\})\\ ={\text{Shift}}_{\tau }\delta (\{w\}-\{x(t)\})\equiv \delta (\{w\}-\{x(t+\tau )\})\\ =\delta (\{w\}-\left(z(t=0)+{\int }_0^t{v}_i(z(t))dt\right))\end{array}$$

Thus (where “IC” means initial condition)

$$\begin{array}{l}\langle w\mid exp({tO}_{\{\text{DE}\}})\mid z\rangle =exp(t\sum _i{v}_i(\{z\}){\nabla }_{z_i})\delta (\{w\}-\{z\})\\ =\delta (\{w\}-\left(\{z(0)=z\}+{\int }_0^t{v}_i(z(t^{\prime }))dt\right))\\ =\delta (\{w\}-\left(\text{Solutionof}\frac{\partial x_i}{\partial t}=v_i(\{x_k\})\text{withIC}z(0)=z\right))\end{array}$$ (21)

QED.

As far as we know this detailed derivation has not appeared previously, though our previous work [4] outlined a simplified version. As a corrollary, using Equation 21 we may multiply by f(w) and integrate over w to calculate

$$\begin{array}{l}exp(tv(\{z\})·{\nabla }_z)\delta (w-z)=\delta (w-\left(z(0)+{\int }_0^{t}v(z(t^{\prime })){dt}^{\prime }\right))\\ \Rightarrow exp(tv(\{z\})·{\nabla }_z)f(z)=f(z(0)+{\int }_0^{t}v(z(t^{\prime })){dt}^{\prime }).\end{array}$$ (22)

3.5 Hybrid SSA/ODE: Operator algebra derivation

We now derive a new SSA-like simulation algorithm for hybrid combinations of discrete events and ODE dynamics, using operator algebra. The main idea is to replace the exponential distribution factor exp(−tD)with a time integral [15]:

$$exp(-tD)\to exp(-{\int }_0^{t}D(t^{\prime }){dt}^{\prime }),$$ (23)

and to add an extra ODE to the system of ODEs in order to keep track of the integral. We will now use the more compact formulation of TOPE in Equation 2 to derive this method.

3.5.1 Heisenberg picture

Let the operators, rather than the states, evolve in time according to W₀ according to Equation 3. This is traditionally called the “Heisenberg picture” in distinction to the “Schroedinger picture” in quantum mechanics. Recall Equation 2 and Equation 3, where (···)₊ is the time-ordering super-operator:

$${(O(t_i)O(t_j))}_{+}=\{\begin{array}{ccc}O(t_i)O(t_j)& \text{if}& t_i\ge t_j\\ O(t_j)O(t_i)& \text{if}& t_i\le t_j\end{array}$$

(and likewise for higher order products). Note that if O(t_i) and O(t_j) commute for all pairs of times t_i and t_j, then (O(t_i)O(t_j))₊ = O(t_i)O(t_j), and the time-ordering operator (···)₊ can be dropped. Often the notation (O(t_i)O(t_j)) is used in place of (O(t_i)O(t_j))₊ to denote the super-operator that time-orders operator products.

3.5.2 Application to ODE + decay clock

The hybrid system consisting of chemical reactions (possibly parameterized) together with ordinary differential equations has the combined operator W = (Ô_react − D_react) + O_DE, which we can regroup as

$$W=(O_{\text{DE}}-D_{\text{react}})+{\widehat{O}}_{\text{react}}$$

and then apply TOPE to O_DE − D_react first with W₀₀ = O_DE and W₀₁(t_k) = − D_react(t_k), and then again to (O_DE − D_react)+ Ô_react with W₀ = W₀₀ + W₀₁ = O_DE − D_react and W₁ = Ô_react.

In the first application of TOPE to O_DE − D_react with W₀₀ = O_DE, the opererators W₀₁(t_k) = −D_react(t_k) defined at different times are all diagonal in the same (particle number basis and therefore commute:

$$[D_{\text{react}}(t_i),D_{\text{react}}(t_j)]=0.$$

In this circumstance, we can simply drop the time-ordering super-operator (···)₊ in Equation 2 and write

$$exp(t(O_{\text{DE}}-D_{\text{react}}))=exp({tO}_{\text{DE}})exp(-{\int }_0^t{dt}^{\prime }{D}_{\text{react}}(t^{\prime }))$$ (24)

where, as in Equation 3, D_react(t′) = exp(−t′O_DE)D_react exp(t′O_DE). In our case, Equation 24 specializes to:

$$\begin{array}{l}\langle w\mid exp(t(O_{\{\text{DE}\}}-D_{\text{react}}))\mid z\rangle =exp(t\sum _i{v}_i(\{z\}){\nabla }_{z_i})exp(-{\int }_0^t{dt}^{\prime }{D}_{\text{react}}(t^{\prime }))\delta (\{w\}-\{z\})\\ =exp\left(-{\int }_0^t{dt}^{\prime }{D}_{\text{react}}\left(z(0)+{\int }_0^{t^{\prime }}v(\{z\})d{t}^{″}\right)\right)\delta (w-\left(z(0)+{\int }_0^{t}v(z(t^{\prime })){dt}^{\prime }\right))\end{array}$$ (25)

This result looks very similar to Equation 22 applied to

$$f(z)=exp\left(-{\int }_0^t{dt}^{\prime }{D}_{\text{react}}(t^{\prime }))\delta (\{w\}-\{z\})\right),$$

and we now aim to understand and exploit this similarity.

3.5.3 Equivalent ODE

Consider the dynamics expressed in Equation 25. Can we obtain the first factor from ODE’s alone? Yes, if we introduce a new state variable τ involved in every ODE-related rule. Set τ(0) = 0 as the new variable’s initial condition, and augment the ODE operators as follows

$$\begin{array}{l}Z=(z,\tau )\\ V(z)=(v\{z\}),-D(z))\\ {\nabla }_Z=({\nabla }_z,{\partial }_{\tau })\\ {\stackrel{\sim }{O}}_{\{\text{DE}\}}=V(Z){\nabla }_Z=v(\{z\})·{\nabla }_z+D(z){\partial }_{\tau }\end{array}$$ (26)

In other words, we have added a differential equation for τ to the ODE system

$$\frac{\partial x_i}{\partial t}=v_i(\{x_k\})\text{and}\frac{d\tau }{dt}=D(z).$$ (27)

This equation is solvable in terms of a “warped time” coordinate

$$\tau (t)={\int }_0^t{D}_{\text{react}}(z(t^{\prime })){dt}^{\prime }.$$ (28)

(Cf. Equation 23.) There are degenerate cases D_react = 0 only if there are terminal states in the reaction network.

To see that this is the correct procedure, calculate from Equation 21:

$$\begin{array}{l}\langle \left(\begin{array}{c}w\\ {\tau }_{max}\end{array}\right)|exp(t{\stackrel{\sim }{O}}_{\{\text{DE}\}})exp(-{\tau }_{max})|\left(\begin{array}{c}z(0)\\ \tau (0)=0\end{array}\right)\rangle \\ =exp\left(t\left(\sum _i{v}_i(\{z\}){\nabla }_{z_i}+D(z){\partial }_{\tau }\right)\right)\delta (\{w\}-\{z\})\delta ({\tau }_{max}-\tau )exp(-{\tau }_{max})\\ =\delta \left(\{w\}-\left(z(0)+{\int }_0^t{v}_i(z(t^{\prime }))dt\right)\right)\delta \left({\tau }_{max}-{\int }_0^t{D}_{\text{react}}(z(t^{\prime })){dt}^{\prime }\right)exp(-{\tau }_{max})\\ =\delta (\{w\}-(\text{SolutionofEquation}27,\text{withIC}z(0),\tau (0)=0))\times exp(-{\tau }_{max})\end{array}$$ (29)

This expression agrees with Equation 25, as required. But how do we insure the IC on τ? That can be done as follows:

$$\begin{array}{l}\langle \left(\begin{array}{c}w\\ {\tau }_{max}\end{array}\right)|exp(t{\stackrel{\sim }{O}}_{\{\text{DE}\}})exp(-{\tau }_{max})|\left(\begin{array}{c}z\\ 0\end{array}\right)\rangle \\ \langle \left(\begin{array}{c}w\\ {\tau }_{max}\end{array}\right)|exp(t{\stackrel{\sim }{O}}_{\{\text{DE}\}})exp(-{\tau }_{max})|\left(\begin{array}{c}z\\ 0\end{array}\right)\rangle \times \left(1=\int d{\tau }^{\prime }{dz}^{\prime }\langle \left(\begin{array}{c}{z}^{\prime }\\ {\tau }^{\prime }\end{array}\right)|\left(\begin{array}{c}z\\ \tau \end{array}\right)\rangle \right)\\ =\langle \left(\begin{array}{c}w\\ {\tau }_{max}\end{array}\right)|exp(t{\stackrel{\sim }{O}}_{\{\text{DE}\}})exp(-{\tau }_{max})\left(\int d{\tau }^{\prime }{dz}^{\prime }|\left(\begin{array}{c}{z}^{\prime }\\ 0\end{array}\right)\rangle \langle \left(\begin{array}{c}{z}^{\prime }\\ {\tau }^{\prime }\end{array}\right)|\right)\mid \left(\begin{array}{c}z\\ \tau \end{array}\right)\rangle \\ =\langle \left(\begin{array}{c}w\\ {\tau }_{max}\end{array}\right)|exp(t{\stackrel{\sim }{O}}_{\{\text{DE}\}})exp(-{\tau }_{max})P_{\tau :=0}|\left(\begin{array}{c}z\\ \tau \end{array}\right)\rangle \\ P_{\tau :=0}\equiv \int d{\tau }^{\prime }{dz}^{\prime }|\left(\begin{array}{c}{z}^{\prime }\\ 0\end{array}\right)\rangle \langle \left(\begin{array}{c}{z}^{\prime }\\ {\tau }^{\prime }\end{array}\right)|\end{array}$$ (30)

is a projection operator (i.e. one that satisfies P · P = P) that resets the variable τ to zero after each use. In summary,

$$\begin{array}{l}\langle \left(\begin{array}{c}w\\ {\tau }_{max}\end{array}\right)|exp(t{\stackrel{\sim }{O}}_{\{\text{DE}\}})exp(-{\tau }_{max})P_{\tau :=0}|\left(\begin{array}{c}z\\ \tau \end{array}\right)\rangle \\ =\delta (\{w\}-(\text{SolutionofEquation}27,\text{withIC}z(0),\tau (0)=0))\times exp(-{\tau }_{max})\end{array}$$ (31)

Clearly this result is equivalent to Equation 25 and is in the correct form for a Markov chain that can represent a computation. Of course, the matrix element calculated is only relevant if τ_max as drawn from the exponential is constrained to be equal to the final value of τ in the final state $\langle \left(\begin{array}{c}w\\ {\tau }_{max}\end{array}\right)|$ as solved by the ODE system Õ_{DE}. We can implement this constraint with a factor of δ(τ, − τ_max) in the Markov chain over states and times. Thus a step in the Markov chain in between reactions can be written as:

$$\begin{array}{c}{\mathcal{W}}_{\text{betweenreactions}}=\delta (\tau -{\tau }_{max})exp(\begin{array}{cc}t& {\stackrel{\sim }{O}}_{\{\text{DE}\}}\end{array})P_{\tau :=0}exp(-{\tau }_{max})\mathrm{\Theta }({\tau }_{max}\ge 0)\\ \mathcal{W}={\widehat{O}}_{\text{react}}·{\mathcal{W}}_{\text{betweenreactions}}\end{array}$$

As in the SSA derivation, the reaction step is given by factors of Ô_react which need to be normalized by D_react. Using δ(t − t_max(τ_max))dt = δ(τ − τ_max)dτ and dτ/dt = D_react(t), we find

$$\begin{array}{c}{M}_{01}=exp(-{\tau }_{max})\mathrm{\Theta }({\tau }_{max}\ge 0)\\ M_{00}=\delta (t-t_{max}({\tau }_{max})exp(\begin{array}{cc}t& {\stackrel{\sim }{O}}_{\{\text{DE}\}}\end{array})·P_{\tau :=0}\\ M_1={\widehat{O}}_{\text{react}}/D_{\text{react}}\\ \mathcal{W}=M_1·M_{00}·M_{01}\end{array}$$ (32)

where represents the Markov chain corresponding to the simulation algorithm.

In implementations so far [4,15] we have used instead the equivalent differential operator

$${\stackrel{\sim }{O}}_{\{\text{DE}\}}=V(Z){\nabla }_Z=v(\{z\})·{\nabla }_z-D(z)p{\partial }_p$$

with p = exp(−τ), initialized at p₀ = 1, and a uniform distribution on p_final ∈ [0, 1]. This variant of the ODE was reported independently in [16], though the derivation there did not proceed by general field theory techniques.

3.5.4 Algorithm: Hybrid SSA/ODE solver

By Equation 32 above, a Markov chain algorithm for simulating the hybrid system can be represented in the following SSA-like pseudocode:

factor ρ(r)(xin, xout) = k(r)(xin)p(r)(xout|xin);
repeat {
   initialize SSA propensities as k(r)(xin);
   initialize $k^{(\text{total})}:={\displaystyle \sum _r}{k}^{(r)}(x_{\text{in}})$;
   initialize τ: = 0;
   draw effective waiting time τmax from exp(−τmax)
   solve ODE system, including an extra ODE updating$\tau :\frac{d\tau }{dt}=k^{(\text{total})}(t)$
     until τ = τmax
   draw reaction r from distribution k(r)(xin)/k(total);
   draw xout from p(r)(xout|xin) and execute reaction r;
} until t ≥ tmax

3.5.5 Application: Cell division

As a simplified model of stochastic cell division, we may consider constant growth of a linear dimension l of each cell, dl/dt = v, coupled with a stochastic cell division rule whose propensity depends on the ratio of l to a threshold length l₀ for likely division:

$$\text{cell}(l)\to \text{cell}(l/2),\text{cell}(l/2)\mathbf{with}{\rho }_{\text{division}}\sigma (\beta (l/l_0-1))$$

with a sigmoidal function such as σ(x) = 1/(1 + exp(−x)). In this model the parameter β varies the sharpness of the threshold, and ρ_division is the maximal propensity for division. Experimental evidence for stochastic dependence of division events on cell size in plant cells is reviewed in [17].

The differential equation for length can also be put in the form of a reaction rule that includes an ODE:

$$\text{cell}(l)\to \text{cell}(l+dl)\mathbf{solving}dl/dt=v(l)$$

as described in [15]. Clearly this model could be augmented with other parameters such as growth signals with their own dynamics. This was done in models of biological stem cell niches in mouse olfactory epithelium and plant root growth, using the foregoing cell division rules. These systems were studied and simulated using the hybrid SSA/ODE algorithm above, in [15,18].

3.5.6 Application: Time-varying propensity for complete polymerization

Consider the n-step polymerization reaction

$$\begin{array}{c}\{A\to X_1\mathbf{with}{k}_1,X_1\to X_2\mathbf{with}{k}_2,\dots ,X_{n-1}\to B\mathbf{with}{k}_n\}\\ {\tau }_i=\tau /n\text{and}{k}_i=nk.\end{array}$$

There is an n^(max). Then

$$\begin{array}{c}\widehat{W}=\lambda {\widehat{a}}_{n+1}\\ W=\lambda (({\widehat{a}}_{n+1}+c_{n+1})-I_{n+1})=\lambda ({\widehat{b}}_{n+1}-I_{n+1})\end{array}$$

where c_n+1 is all zeros except for a “1” entry in the lower right corner. Since b̂ and I are matrices that commute, exp(tW) = exp(tλb̂_n+1) exp(−tλI_n+1) and we easily compute

$$P(t\mid \tau ,n)={[exp(tW)]}_{1,n+1}=\frac{{\lambda }^n{t}^{n-1}{e}^{-\lambda t}}{(n-1)!}$$

This is the distribution on polymer completion times. It is an Erlang distribution (a Gamma distribution with integral values of n). If τ is held fixed and n tends towards infinity, this distribution approaches a delta function δ(t − τ), which can lead to differential-delay equation models for reaction networks involving polymerization processes such as transcription [19]. This probability distribution for termination times also corresponds to the time-varying propensity function

$$\rho (t\mid \tau ,n)=P(t\mid \tau ,n)/\left[1-{\int }_0^{t}P(t\mid \tau ,n)dt\right]\frac{{\lambda }^n{t}^{n-1}{e}^{-\lambda t}}{\mathrm{\Gamma }(n,t\lambda )},$$ (33)

which increases monotonically in time.

As in Equation 32, the resulting time-varying propensity still fits within the framework of a Markov chain (I, t′|J, t) that advances the time variable by an increment that is a random variable. The method of the previous section can be used to implement an SSA-like algorithm, with differential equations that govern propensities replaced by algebraic equations (Equation 33) or, if differential equations are also present, by differential-algebraic equations.

3.5.7 Extended Application: Tissue-level model of Arabidopsis root growth

A full tissue-level model of a hybrid SSA/ODE system has been presented in [18], which details a mathematical model of auxin growth hormone patterning along the developing root of the plant Arabidopsis thaliana, including the pattern formation system in the root apical meristem (RAM). The model was first formulated using a fixed 1D geometry of cells along the central “stele” of the root, including both passive diffusion of auxin originating in the above ground part of the plant, and more importantly autoregulated active transport of auxin. This much of the model is formulated using ordinary differential equations and spatial discretization at the scale of one cell.

However the real root involves cell growth, division and possibly biomechanics in an essential way, so the model was reimplemented in the “Plenum” implementation of the “Dynamical Grammars” modeling language. Dynamical Grammars support parameterized rules such as those of Section 3.5 above at multiple scales (eg cellular and/or molecular scales), and the Plenum implementation [15] uses the foregoing hybrid SSA/ODE algorithm as an essential part of its simulation engine. It also uses a data structure of pattern-matched objects (somewhat akin to that of the Rete algorithm [12]) for efficient handling of the variable-binding involved when there are many rules in a grammar, some of which include repeated variable names. The resulting root growth and patterning model includes rules for cell growth, cell division, mechanical forces between neighboring cells in 1D, cell death at the tip of the root, auxin influx from the shoot, production of a hypothetical second morphogen “Y” possibly playing a role similar to cytokinin, autoregulated active transport of auxin between neighboring cells, passive transport of auxin and Y between neighboring cells, degradation of auxin and Y, and dilution of auxin and Y due to cell growth. There are a total of 13 grammar rules that specify the foregoing mechanisms, with one or two rules per listed mechanism. As in the previous cell division example, each rule is either of “solving” keyword ODE type or of “with” keyword discrete event type. We now present the first four rules of this model.

In the root model there is just one type of parameterized object, a cell. Each cell carries its own internal state information in the form of the values of an ordered list of parameters, each of which is constrained to be of some type (often integers or real values) associated with a measure that can be summed or integrated over. In the plant root model, the parameter types of a cell object are as follows:

cell[currID : ℕ mode : ℕ, l : ℝ, r : ℝ, A : ℝ, Y : ℝ, prevID : ℕ, nextID : ℕ]

Here currID is the integer-valued (or “integer-typed”, currID: ℕ) unique identification number (ID) of the current cell, prevID is the integer-valued ID of the previous cell in the 1D line, nextID is the integer-valued ID of the next cell, mode is an integer-valued label specifying the cell’s internal growth state, l is the real-valued (l: ℝ) current cell length, r is its real-valued size or “radius”, A is its real-valued concentration of auxin, Y is its real-valued concentration of hypothetical substance Y. Here A and Y could alternatively be typed as nonnegative integers in a stochastic molecular simulation, but that was not the modeling choice in this investigation. A slightly simplified version of the rules for cell growth, cell division, biomechanics, and passive diffusion of chemical species between neighboring cells is:

grammar root {
   /* cell growth: */
   cell[curr, mode, x, r, A, Y, prev, next] → cell[curr, mode, x, r + dr, A, Y, prev, next]
   solving {dr/dt = 1/τcycle}
   /* cell biomechanics (point masses, dissipation dominated) */
     C1 = cell[curr, mode, x, r, A, Y, prev, next],
     C2 = cell[next, mode′, x′, r′, A′, Y′, curr, nextnext]
     → C1 = cell[curr, mode, x + dx, r, A, Y, prev, next], C2
            solving {dx/dt = −∂xVspring(x − x′, r, r′)}
     /*plus similar rule exchanging next and prev; dx/dt just adds up over rules */
   /* switch from growth mode (mode=1) to division-waiting mode (=2): */
     cell[curr, 1, x, r, A, Y, prev, next] → cell[curr, 2, x, r, A, Y, prev, next]
         with ρstop/(1 + exp(− (r − rlim))/Tdiv)
   /* cell replication, preserving 1D structure: */
     cell[curr, 2, x, r, A, Y, prev, next], cell[prev, mode′, x′, r′, A′, Y′, prevprev, curr]
        cell[next, mode, x″, r″, A″, Y″, curr, nextnext]
     → cell[new1, 1, x − r + 2rα + r(1 − α), r(1 − α), A, Y, prev, new2],
        cell[new2, 1, x − r + rα, rα, A, Y, new1, next],
        cell[prev, mode′, x′, r′, A′, Y′, prevprev, new1],
        cell[next, mode, x″, r″, A″, Y″, new2, nextnext]
     with $[\text{base}+\text{ampl}(Y/Y_0)/\left(1+{(Y/Y_0)}^5\right]\times \mathrm{\Theta }(\frac{1}{2}+\mathrm{\Delta }\le \alpha \le \frac{1}{2}+\mathrm{\Delta })$
   /* auxin/Y passive transport between two neighboring cells: */
     C1 = cell[curr, mode, x, r, A, Y, prev, next],
     C2 = cell[next, mode′, x′, r′, A′, Y′, curr, nextnext]
     → C1 = cell[curr, mode, x, r, A + dA, Y + dY, prev, next],
     C2 = cell[next, mode′, x′, r′, A′ + dA′, Y ′ + dY′, curr, nextnext]
            solving {dA/dt = DA(A′ − A), dA′/dt = DA(A − A′),
                    dY/dt = DY (Y′ − Y), dY′/dt = DY (Y − Y′) }
}

The actual code for these rules is given in Appendix III. It is written using the Plenum implementation [15] of the Dynamical Grammars framework [4]. Plenum is embedded in the Mathematica computer algebra problem-solving environment. Thus, ordinary and partial derivatives as used above are actually a part of the language. The full model file is available as Supplementary Data to this paper. Repeated variables on the left hand side (LHS) must have identical values for the rule to apply. This situation occurs in the cell biomechanics, cell replication and passive transport rules above, where left-right cell neighbor pairs point to one another by sharing ID parameter values like “curr”, “prev” and “next” in the first and last two parameter positions. By contrast, there is no repetition of variables on the LHS of the autonomous cell growth or cell mode-switching rules above, since they have only one object on the LHS of each rule. Algorithmically such repeated variable matching is achieved by symbolic pattern matching or variable-binding; mathematically it is expressed by operator integrals such as Equation 18. The coordinate system used in this example may seem “backwards” since it is a minor convention that roots grow from left to right, but that the quiescent center near the right tip is the origin of coordinates.

The Plenum implementation also performs several symbolic computations including variable-binding for efficient implementation of rules with repeated variables (Equation 18 above and the present extended example), and aggregating the ODE dτ/dt = k^(tota)(t) of Section 3.5.4 by adding up the symbolic expressions from the individual rules.

Selected pattern formation snapshots are shown in Figure 3. The phenomenology of the resulting simulations, and of the actual root observations with which they largely agree, is discussed in [18]. Root apical meristem is an example of a stem cell niche in plants. A somewhat more complex stem cell niche model for mouse olfactory epithelium, in two dimensions using Plenum, is given in [15].

Figure 3.

Snapshots of the root growth model, showing cell positions along the horizontal axis (root tip to the right), and concentrations of auxin (solid red curve with one or two peaks) and hypothetical substance Y (dashed blue curve with one or two peaks) with increasing time. Cell state (1=idling in preparation for cell division, vs. 0=growth) is shown in green dotted curve. Parameters are: ρ_stop = 1, base = .005, ampl = 100, Y₀ = 5, r_lim = 1, T_div = .01, Δ= .2, D_A = 0.08, D_Y = 0.16. Some parameter sets including this one develop extra auxin peaks to the left of the Quiescent Center (~rightmost blue peak), which may specify the location for a new lateral root. Full interpretation of this model is given in [18].

4 Conclusion and outlook

We have shown that the time-ordered product expansion (TOPE) can be used systematically to derive computational simulation and parameter-fitting algorithms for stochastic systems, connecting two seemingly distant areas of research. In doing so we have developed the means to translate formally between field theory language and the language of computable Markov chains in which randomized algorithms can be expressed and derived. By this means we hope to open the door to the use of TOPE and related methods from quantum and statistical field theory in the computational simulation of stochastic biochemical kinetics, with broad applicability in physically based biology. The particular hybrid stochastic process/ordinary differential equation simulation algorithm derived here is very different from interleaving and operator splitting algorithms which are intrinsically approximate; instead, this algorithm is exact in the same sense that SSA is (that is, it draws from the same distribution of just-fired reactions), except for any errors introduced by the ODE solver and in the solver’s detection of the ODE stopping criterion, which is that an auxiliary variable reaches a threshold value. A future prospect for the field theory approach is application to reaction-diffusion systems in which the propagator for particles between reactions is the heat kernal Green’s function for the diffusion equation. The result may be an alternative avenue for derivation of novel particle-based, off-grid stochastic numerical solvers for reaction-diffusion problems as treated in [2], which, like the algorithms shown here, are also amenable to generalizations to exact “leaping” acceleration and to hybrid stochastic/differential equation solution algorithms.

Figure 1.

A time history of the reaction A + B ⇌ C. Time flows left to right. Open circles represent reaction events, with probability factor ×W₁. In between reaction events are unimolecular particle propagators exp((t_k −t_k−1)W₀), labelled by arrows and particle names (repeated for clarity). This is a non-spatial version of the Lee model in quantum field theory (cf. for example [6]).

Figure 2.

Erlang-derived time-dependent propensities for completion of a multistage process τ = 1, n ∈ {1, …, 10}. Horizontal axis: time, t. Vertical axis: propensity, ρ(t|τ, n). Plots for varying n are superimposed. For larger n there is a “maturation” phenomenon whereby completion at small times is very unlikely, and when a process is “overdue” for completion then its propensity becomes very high. By comparison, propensities for very small n increase rapidly at first and are then relatively flat.

Abbreviations list

IC

Initial Condition

ODE

Ordinary Differential Equation

MC

Markov Chain

SSA

(Gillespie) Stochastic Simulation Algorithm

TOPE

Time-Ordered Product Expansion

LHS

Left Hand Side

RHS

Right Hand Side

C Appendix I: Bayesian inference derivation

C.1 Semigroup property

Here we provide the omitted details for Section 2.4. For nonnegative times t = t₁ + t₂, any time-evolution equation must obey the semigroup property:

$$Pr(I\mid J,t)=\sum _{K}Pr(I\mid K,t_2)Pr(K\mid J,t_1)$$

i.e.

$${\left[exp(t\left(\widehat{W}-D\right))\right]}_{I,J}=\sum _K{\left[exp(t_2\left(\widehat{W}-D\right))\right]}_{I,K}{[exp(t_1\left(\widehat{W}-D\right))]}_{K,J}.$$

Is there a k-event version of this rule, for k = k₁ + k₂? We observe (where again ${[{\tau }_q]}_0^k\equiv [{\tau }_0,\dots {\tau }_k]$)

$$Pr(I,k\mid J,t)={\int }_0^t\cdots {\int }_0^t({\prod }_{q=0}^{k}d{\tau }_q)Pr(I,{[{\tau }_q]}_0^k,k\mid J,t)$$

and we calculate for 0 ≤ k₁ ≤ k:

$$\begin{array}{l}\sum _K{\int }_0^{t}d\tau Pr(I,[{\tau }_{k_1}^{\prime },{\tau }_{k_1+1},\dots {\tau }_k],k_2\mid K,\tau )Pr(K,{[{\tau }_q]}_0^{k_1},k_1\mid J,t-\tau )\\ ={\int }_0^{t}d\tau \{\left[\prod _{q=k\downarrow }^{k_1+1}exp(-{\tau }_{q}D)\widehat{W}\right]exp(-{\tau }_{k_1}^{\prime }D)\\ {\times \delta \left(\tau -\left(\sum _{q=k_1+1}^k{\tau }_q+{\tau }_{k_1}^{\prime }\right)\right)exp(-{\tau }_{k_1}D)\left[\prod _{q=k_1-1\downarrow }^0\widehat{W}exp(-{\tau }_{q}D)\right]\delta (t-\tau -\sum _{q=0}^{k_1-1}{\tau }_q)\}}_{I,J}\\ =\{\left[\prod _{q=k\downarrow }^{k_1+1}exp(-{\tau }_{q}D)\widehat{W}\right]exp(-({\tau }_{k_1}^{\prime }+{\tau }_{k_1})D)\\ {\times \left[\prod _{q=k_1-1\downarrow }^0\widehat{W}exp(-{\tau }_{q}D)\right]\delta \left(t-\left(\sum _{q=k_1+1}^k{\tau }_q+{\tau }_{k_1}^{\prime }+\sum _{q=0}^{k_1-1}{\tau }_q\right)\right)\}}_{I,J}\\ =Pr(I,\left[{[{\tau }_q]}_0^{k_1-1},{\tau }_{k_1}^{\prime }+{\tau }_{k_1},{[{\tau }_q]}_{k_1+1}^k\right],k\mid J,t).\end{array}$$

Thus, if k = k₁ + k₂ and for any ${\tau }_{k_1}^{\prime }\in [0,{\tau }_{k_1}]$, there is a semigroup law:

$$Pr(I,{[{\tau }_q]}_0^k,k\mid J,t)\sum _K{\int }_0^{t}d\tau Pr(I,\left[{\tau }_{k_1}^{\prime },{\tau }_{k_1+1},\dots {\tau }_k\right],k_2\mid K,\tau )\times Pr(K,\left[{\tau }_0,\dots {\tau }_{k_1-1},{\tau }_{k_1}-{\tau }_{k_1}^{\prime }\right],k_1\mid J,t-\tau ).$$

In this derivation there is an arbitrary choice of ${\tau }_{k_1}^{\prime }$ from the interval [0, τ_k1].

C.2 Bayesian recurrence relation

Here we provide the omitted details for Section 3.1.1. We seek a version of the semigroup law that pertains to Pr(I, t|k, J) rather than to Pr(I, k|J, t). This is achieved by a somewhat involved application of Bayes’ rule.

We observe (where again by definition ${[{\tau }_q]}_0^k\triangleq [{\tau }_0,\dots {\tau }_k]$)

$$Pr(I,{[{\tau }_q]}_0^k,k\mid J,t)={\left[exp(-{\tau }_{k}D)\prod _{q=k-1\downarrow }^0\widehat{W}exp(-{\tau }_{q}D)\mathrm{\Theta }({\tau }_q\ge 0)\right]}_{I,J}\delta \left(t-\sum _{q=0}^k{\tau }_q\right)$$

so we may define (where J = I₀ and I = I_k and D_Iq ≜ D_IqIq

$$Pr({[I_q]}_1^k,{[{\tau }_q]}_0^k,k\mid J,t)\triangleq exp(-{\tau }_k{D}_{I_k})\left[\prod _{q=0}^{k-1}{\widehat{W}}_{I_{q+1}{I}_q}exp(-{\tau }_q{D}_{I_q})\mathrm{\Theta }({\tau }_q\ge 0)\right]\delta \left(t-\sum _{q=0}^k{\tau }_q\right)$$

We seek a simple expression for Pr(I, t|k, J), and claim that with suitable caveats it will be determined recursively by

$$Pr(I,t\mid k=1,J)={\widehat{W}}_{IJ}exp(-{tD}_J),$$

the two factors of which have inverse cancelling normalizations. The obstacle to overcome is that, from the Bayesian point of view, simultaneous knowledge of the simulation end time and final event number can trickle backwards and influence the distribution of likely event firing times at earlier times and event numbers - a completely nonphysical artifact. To avoid this effect we must be careful to ask the right questions for Bayesian inference to answer. To begin with we consider simulation ending times T much longer than event times t that we wish to sample. All events after event k at time t are assumed to be of no interest, so we integrate them out. All earlier events are assumed to be known already, so we conditionalize over them. This is the correct Bayesian way to introduce time asymmetry into the global distribution $Pr({[I_q]}_1^k,{[{\tau }_q]}_0^k,k\mid \dots )$ entire trajectories ( ${[I_q]}_1^k,{[{\tau }_q]}_0^k$) above.

The strategy then is to consider large times T ≫ t which are overwhelmingly likely to have large reaction numbers n ≫ k; then to marginalize the probability distribution Pr([I], [τ], n|J, T) over all event numbers n > k and to conditionalize it over all event numbers q < k.

C.2.1 Marginalizing

Define

$$\widehat{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J,T)\triangleq \sum _{n=k+1}^{\infty }\sum _{\{{[I_q]}_{k+1}^n\}}{\int }_0^T\cdots {\int }_0^T{[d{\tau }_q]}_k^{n}Pr({[I_q]}_1^n,{[{\tau }_q]}_0^n,n\mid J,T)$$ (34)

This is a “just-fired” probability, in which any wait times τ and events after the kth event are integrated out (marginalized).

In the limit T → ∞ only summands with n ≫ k will contribute (assuming terminal states have been formally eg. by adding an extra, isolated, slow, reversible reaction). First, is this object really a probability distribution? Clearly every value is nonnegative. They also add up to one in the limit T → ∞:

$$\begin{array}{l}\sum _{\{{[I_q]}_1^k\}}{\int }_0^T{[d{\tau }_q]}_0^{k-1}\widehat{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J,T)\\ =\sum _{n=k+1}^{\infty }\sum _{\{{[I_q]}_1^n\}}{\int }_0^T{[d{\tau }_q]}_0^{n}Pr({[I_q]}_1^n,{[{\tau }_q]}_0^n,n\mid J,T)\\ \underset{T\to \infty }{\to }\sum _{n=0}^{\infty }\sum _{\{{[I_q]}_1^n\}}{\int }_0^T{[d{\tau }_q]}_0^{n}Pr({[I_q]}_1^n,{[{\tau }_q]}_0^n,n\mid J,T)=1\end{array}$$ (35)

due to the normalization of $Pr({[I_q]}_1^n,{[{\tau }_q]}_0^n,n\mid J,T)$. So, $\stackrel{\sim }{Pr}(\cdots \mid k,J)=\underset{T\to \infty }{lim}\widehat{Pr}(\cdots \mid k,J,T)$ is also a probability density function.

Next we compute $\widehat{Pr}(\cdots \mid J,T)$ using TOPE:

$$\begin{array}{l}\widehat{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J,T)=\sum _{n=k+1}^{\infty }\sum _{\{{[I_q]}_{k+1}^n\}}{\int }_0^T\cdots {\int }_0^T{[d{\tau }_q]}_k^n\delta \left(T-\sum _{q=0}^n{\tau }_q\right)\\ \times exp(-{\tau }_n{D}_{I_n})\left[\prod _{q=0}^{n-1}{\widehat{W}}_{I_{q+1}{I}_q}exp(-{\tau }_q{D}_{I_q})\mathrm{\Theta }({\tau }_q\ge 0)\right]\\ =\sum _{n=k+1}^{\infty }\sum _{\{{[I_q]}_{k+1}^n\}}{\int }_0^T{\int }_0^T{[d{\tau }_q]}_k^n\delta \left(T-\sum _{q=0}^{k-1}{\tau }_q-\sum _{q=k}^n{\tau }_q\right)exp(-{\tau }_n{D}_{I_n})\\ \times \left[\prod _{q=0}^{k-1}{\widehat{W}}_{I_{q+1}{I}_q}exp(-{\tau }_q{D}_{I_q})\mathrm{\Theta }({\tau }_q\ge 0)\right][\prod _{q=k}^{n-1}{\widehat{W}}_{I_{q+1}{I}_q}exp(-{\tau }_q{D}_{I_q})]\end{array}$$

But now the product [ ${\displaystyle \prod _{q=0}^{k-1}}\cdots$] is a common factor and can be moved out of all the integrals and sums. Thus

$$\widehat{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J,T)=\left[\prod _{q=0}^{k-1}{\widehat{W}}_{I_{q+1}{I}_q}exp(-{\tau }_q{D}_{I_q})\mathrm{\Theta }({\tau }_q\ge 0)\right]\mathcal{I}\left(I_k,k,J,T-\sum _{q=0}^{k-1}{\tau }_q\right),$$

(the first factor of which is independent of T), where

$$\mathcal{I}(I_k,k,J,T^{\prime })\triangleq \sum _{n=k+1}^{\infty }\sum _{\{{[I_q]}_{k+1}^n\}}{\int }_0^T\cdots {\int }_0^T{[d{\tau }_q]}_k^n\delta \left(T^{\prime }-\sum _{q=k}^n{\tau }_q\right)\times exp(-{\tau }_n{D}_{I_n})\left[\prod _{q=k}^{n-1}{\widehat{W}}_{I_{q+1}{I}_q}exp(-{\tau }_q{D}_{I_q})\right].$$

If we define new dummy variables I′_q ≜ I_q+k, τ′_q ≜ τ_q+k, and n′ ≜ n − k, then τ′_n′ = τ_n, I′_n′ = I_n, and

$$\begin{array}{l}\mathcal{I}(I_k,k,J,T^{\prime })=\sum _{n^{\prime }=1}^{\infty }\sum _{\{{[{I^{\prime }}_q]}_1^{n^{\prime }}\}}{\int }_0^T\cdots {\int }_0^T{[d{{\tau }^{\prime }}_q]}_k^{n^{\prime }}\delta \left(T^{\prime }-\sum _{q=k}^{n^{\prime }}{{\tau }^{\prime }}_q\right)\\ \times exp(-{{\tau }^{\prime }}_n{D}_{{I^{\prime }}_n})\left[\prod _{q=0}^{n^{\prime }-1}{\widehat{W}}_{{I^{\prime }}_{q+1}{I^{\prime }}_q}exp(-{{\tau }^{\prime }}_q{D}_{{I^{\prime }}_q})\right]\\ =\mathbf{1}·\left(e^{T^{\prime }(\widehat{W}-D)}-e^{-T^{\prime }D}\right)·\delta ({I^{\prime }}_0-J)\\ =1-\mathbf{1}·e^{-T^{\prime }D}·\delta ({I^{\prime }}_0-I_k),\end{array}$$

by adding and subtracting the missing n′= 0 summand and using TOPE again. Now we can take limits:

$$\underset{T\to \infty }{lim}\mathcal{I}(I_k,k,J,T^{\prime })=1,$$

and

$$\begin{array}{l}\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)\triangleq \underset{T\to \infty }{lim}\widehat{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J,T)\\ \prod _{q=0}^{k-1}\left({\widehat{W}}_{I_{q+1}{I}_q}exp(-{\tau }_q{D}_{I_q})\mathrm{\Theta }({\tau }_q\ge 0)\right).\end{array}$$ (36)

As a special case, for k = 1 we find $\stackrel{\sim }{Pr}([I_1],[{\tau }_0]\mid 1,J)={\widehat{W}}_{I_1{I}_0}exp(-{\tau }_0{D}_{I_0})\mathrm{\Theta }({\tau }_0\ge 0)$.

C.2.2 Conditionalizing

If 2 ≤ k < n, Bayes’ Rule implies:

$$\widehat{Pr}(I_k,{\tau }_{k-1}\mid {[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2},k,J,T)=\frac{\widehat{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J,T)}{\widehat{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k,J,T)}$$ (37)

The denominator is a new quantity (since the k’s don’t match up the way they do in the numerator) and it is the integral of the numerator that normalizes the left hand side. It can be evaluated in the limit of large T:

$$\begin{array}{l}\widehat{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k,J,T)=\sum _{I_k}{\int }_0^{T}d{\tau }_{k-1}\widehat{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J,T)\\ =\sum _{n=k+1}^{\infty }\sum _{\{{[I_q]}_k^n\}}{\int }_0^T{[d{\tau }_q]}_{k-1}^{n}Pr({[I_q]}_1^n,{[{\tau }_q]}_0^n,n\mid J,T)\\ \underset{T\to \infty }{\to }\underset{T\to \infty }{lim}\sum _{n=k}^{\infty }\sum _{\{{[I_q]}_k^n\}}{\int }_0^T{[d{\tau }_q]}_{k-1}^{n}Pr({[I_q]}_1^n,{[{\tau }_q]}_0^n,n\mid J,T)\\ =\stackrel{\sim }{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k-1,J)\end{array}$$

from Equation 34 and Equation 36. As before, the second step is justified by the fact that n ≤ k has probability that approaches zero as T → ∞. Defining

$$\check{Pr}(I_k,{\tau }_{k-1}\mid {[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2},k,J)\triangleq \underset{T\to \infty }{lim}\widehat{Pr}(I_k,{\tau }_{k-1}\mid {[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2},k,J,T)$$

we find (from Equation 37)

$$\begin{array}{l}\check{Pr}(I_k,{\tau }_{k-1}\mid {[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2},k,J)=\underset{T\to \infty }{lim}\frac{\widehat{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J,T)}{\widehat{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k,J,T)}\\ =\frac{\underset{T\to \infty }{lim}\widehat{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)}{\underset{T\to \infty }{lim}\widehat{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k,J)}\end{array}$$

since for valid nonnegative τs the ratio of limits exists and is finite, as we will shortly see. Thus

$$\begin{array}{l}\check{Pr}(I_k,{\tau }_{k-1}\mid {[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2},k,J)=\frac{\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)}{\stackrel{\sim }{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k-1,J)}\\ =\frac{{\displaystyle \prod _{q=0}^{k-1}}\left({\widehat{W}}_{I_{q+1}{I}_q}exp(-{\tau }_q{D}_{I_q})\mathrm{\Theta }({\tau }_q\ge 0)\right)}{{\displaystyle \prod _{q=0}^{k-2}}\left({\widehat{W}}_{I_{q+1}{I}_q}exp(-{\tau }_q{D}_{I_q})\mathrm{\Theta }({\tau }_q\ge 0)\right)}\\ ={\widehat{W}}_{I_k{I}_{k-1}}exp(-{\tau }_{k-1}{D}_{I_{k-1}})\mathrm{\Theta }(\tau \ge 0).\end{array}$$

The last line is actually independent (in the functional rather than probabilistic sense of “independent”) of the quantitites ${[I_q]}_1^{k-2},{[{\tau }_q]}_0^{k-2}$, and k, so we can drop all these arguments from $\stackrel{\sim }{Pr}(\cdots )$. Restating,

$$\begin{array}{l}\check{Pr}(I_k,{\tau }_{k-1}\mid I_{k-1})={\widehat{W}}_{I_k{I}_{k-1}}exp(-{\tau }_{k-1}{D}_{I_{k-1}})\mathrm{\Theta }({\tau }_{k-1}\ge 0)\\ \text{or}\\ \check{Pr}(I,\tau \mid K)={\widehat{W}}_{IK}exp(-\tau D_K)\mathrm{\Theta }(\tau \ge 0).\end{array}$$

Importantly, this expression is equal to $\stackrel{\sim }{Pr}(I,\tau \mid k=1,K)$ as calculated at the end of the last section. Also the recursive statement of the Bayesian recurrence property Equation 37 becomes:

$$\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)=\check{Pr}(I_k,{\tau }_{k-1}\mid I_{k-1})\stackrel{\sim }{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k-1,J).$$

Since $\check{Pr}(I,\tau \mid J)=\stackrel{\sim }{Pr}(I,\tau \mid k=1,J)$, we find for all k ≥ 2 the Bayesian recurrence relation in terms of $\stackrel{\sim }{Pr}(\cdots )$ alone:

$$\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)=\stackrel{\sim }{Pr}(I_k,{\tau }_{k-1}\mid 1,I_{k-1})\stackrel{\sim }{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k-1,J).$$ (38)

C.3 Markov Chain Derivation

Here we provide the omitted details for Section 3.1.2. Continuing from the foregoing Bayesian recurrence property (Equation 38), we now sum over all I_q except I_k = I and I₀ = J, and integrate over all τ_q, the following equation:

$$\begin{array}{l}\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)\delta \left(t_k-\sum _{q=0}^{k-1}{\tau }_q\right)=\stackrel{\sim }{Pr}(I_k,{\tau }_{k-1}\mid 1,I_{k-1})\\ \times \stackrel{\sim }{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k-1,J)\delta \left(t_k-\sum _{q=0}^{k-1}{\tau }_q\right).\end{array}$$

We define and calculate

$$\begin{array}{l}\stackrel{\sim }{Pr}(I,t_k\mid k,J)\triangleq \sum _{\{{[I_q]}_1^{k-1}\}}{\int }_0^{\infty }\cdots {\int }_0^{\infty }{[d{\tau }_q]}_0^{k-1}\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)\delta \left(t_k-\sum _{q=0}^{k-1}{\tau }_q\right)\\ =\sum _{\{{[I_q]}_1^{k-1}\}}{\int }_0^{\infty }\cdots {\int }_0^{\infty }{[d{\tau }_q]}_0^{k-1}\stackrel{\sim }{Pr}(I_k,{\tau }_{k-1}\mid 1,I_{k-1})\stackrel{\sim }{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k-1,J)\delta \left(t_k-\sum _{q=0}^{k-1}{\tau }_q\right)\\ =\sum _{I_{k-1}}{\int }_0^{\infty }d{\tau }_{k-1}\stackrel{\sim }{Pr}(I_k,{\tau }_{k-1}\mid 1,I_{k-1})\\ \times \sum _{\{{[I_q]}_1^{k-2}\}}{\int }_0^{\infty }\cdots {\int }_0^{\infty }{[d{\tau }_q]}_0^{k-2}\stackrel{\sim }{Pr}({[I_q]}_1^{k-1},{[{\tau }_q]}_0^{k-2}\mid k-1,J)\delta \left(t_k-{\tau }_{k-1}-\sum _{q=0}^{k-1}{\tau }_q\right)\\ =\sum _{I_{k-1}}{\int }_0^{\infty }d{\tau }_{k-1}\stackrel{\sim }{Pr}(I_k,{\tau }_{k-1}\mid 1,I_{k-1})\stackrel{\sim }{Pr}(I_{k-1},t_k-{\tau }_{k-1}\mid k-1,J)\end{array}$$

This $\stackrel{\sim }{Pr}(I,t_k\mid J)$ is also probability density:

$$\sum _{I_k}{\int }_0^{\infty }{dt}_k\stackrel{\sim }{Pr}(I_k,t_k\mid k,J)=\sum _{\{{[I_q]}_1^k\}}{\int }_0^{\infty }\cdots {\int }_0^{\infty }{[d{\tau }_q]}_0^{k-1}\stackrel{\sim }{Pr}({[I_q]}_1^k,{[{\tau }_q]}_0^{k-1}\mid k,J)=1$$

as shown in Equation 35 above, using the definition of Equation 36. Summarizing its Markov property:

$$\stackrel{\sim }{Pr}(I,t\mid k,J)=\sum _K{\int }_0^{t}d\tau \stackrel{\sim }{Pr}(I,\tau \mid 1,K)\stackrel{\sim }{Pr}(K,t-\tau \mid k-1,J).$$ (39)

D Appendix II: Maximum likelihood parameter inference

Application of the TOPE to maximum-likelihood parameter learning in stochastic reaction networks has previously been presented [20]. Here, for completeness of presentation for a different audience, we just show the essential gradient calculation step.

Suppose we have observations of the state of a chemical reaction network at times {t_s}, and wish to improve the probability P (Data|Model) of a reaction network model for the flow of probability at intermediate times. We will use the TOPE for each time interval in between observation times t_s:

$$\left[e^{(t_{s+1}-t_s)\stackrel{\sim }{W}}\right](x(t_{s+1}),x(t_s))=\sum _{k=0}^{\infty }{\int }_{t_s}^{t_{s+1}}\cdots {\int }_{t_s}^{t_{s+1}}d{[\tau ]}_0^n\delta \left((t_{s+1}-t_s)-\sum _{p=0}^n{\tau }_p\right)\times \left[e^{{\tau }_n\stackrel{\sim }{D}}\widehat{W}\dots e^{{\tau }_1\stackrel{\sim }{D}}\widehat{W}{e}^{{\tau }_0\stackrel{\sim }{D}}\right](x(t_{s+1}),x(t_s))$$ (40)

We will need to compute the derivatives of this probability with respect to reaction rates:

$$\begin{array}{l}{\rho }_r\frac{\partial }{\partial {\rho }_r}[e^{(t_{s+1}-t_s)W}](x(t_{s+1}),x(t_s))=\sum _{k=0}^{\infty }{\int }_{t_s}^{t_{s+1}}\cdots {\int }_{t_s}^{t_{s+1}}d{[\tau ]}_0^n\delta \left((t_{s+1}-t_s)-\sum _{p=0}^n{\tau }_p\right)\\ \times \sum _{p=0}^n\left[e^{-{\tau }_{n}D}\widehat{W}\dots e^{-{\tau }_{p+1}D}({\rho }_r{\widehat{W}}_r)e^{-{\tau }_{p}D}\dots e^{-{\tau }_{1}D}\widehat{W}{e}^{-{\tau }_{0}D}\right](x(t_{s+1}),x(t_s))\\ -\sum _{k=0}^{\infty }{\int }_{t_s}^{t_{s+1}}\cdots {\int }_{t_s}^{t_{s+1}}d{[\tau ]}_0^n\delta \left((t_{s+1}-t_s)-\sum _{p=0}^n{\tau }_p\right)\\ \times \sum _{p=0}^n\left[e^{-{\tau }_{n}D}\widehat{W}\dots \widehat{W}({\rho }_r{\tau }_p{D}_r)e^{-{\tau }_{p}D}\widehat{W}\dots e^{-{\tau }_{1}D}\widehat{W}{e}^{-{\tau }_{0}D}\right](x(t_{s+1}),x(t_s))\\ {\rho }_r{[{\widehat{W}}_r]}_{IJ}=\left(\frac{{\rho }_r{[{\widehat{W}}_r]}_{IJ}}{\sum _r{\rho }_r{[{\widehat{W}}_r]}_{IJ}}\right)\left({[\widehat{W}]}_{IJ}\right)=b_{\mathit{rIJ}}{[\widehat{W}]}_{IJ}\end{array}$$

where we defined the “branching ratio”

$$b_{\mathit{rIJ}}\equiv \left(\frac{{\rho }_r{[{\widehat{W}}_r]}_{IJ}}{\sum _r{\rho }_r{[{\widehat{W}}_r]}_{IJ}}\right)={\langle {\delta }_{r,R(I,J)}\rangle }_{p(I\mid J)}$$

for reaction r in state J, assuming each reaction r results in just one output state I per input state J. Here R(I, J) is the random variable denoting the actual reaction chosen in transitioning from state J to state I. Then

$$\begin{array}{l}{\rho }_r\frac{\partial }{\partial {\rho }_r}[e^{(t_{s+1}-t_s)\stackrel{\sim }{W}}](x(t_{s+1}),x(t_s))=\sum _{k=0}^{\infty }{\int }_{t_s}^{t_{s+1}}\cdots {\int }_{t_s}^{t_{s+1}}d{[\tau ]}_0^n\delta \left((t_{s+1}-t_s)-\sum _{p=0}^n{\tau }_p\right)\\ \times \sum _{p=0}^n\left[e^{{\tau }_{n}D}\widehat{W}\dots e^{{\tau }_{p+1}D}(b_r\widehat{W}-\widehat{W}{\rho }_r{\tau }_p{D}_r)e^{{\tau }_{p}D}\dots e^{{\tau }_{1}D}\widehat{W}{e}^{{\tau }_{0}D}\right](x(t_{s+1}),x(t_s))\\ \text{or}\\ {\rho }_r\frac{\partial }{\partial {\rho }_r}[e^{(t_{s+1}-t_s)\stackrel{\sim }{W}}](x(t_{s+1}),x(t_s))\\ =\sum _{k=0}^{\infty }\sum _{p=0}^n{\langle b_r(\text{reaction}\text{event}p\text{out}\text{of}n)\rangle }_{\widehat{W},x(t_{s+1}),x(t_s)}\\ -{\rho }_r\sum _{k=0}^{\infty }\sum _{p=0}^n{\langle {\tau }_p{D}_r\rangle }_{\widehat{W},x(t_{s+1}),x(t_s)}\end{array}$$

This finally is a quantity that is easy to compute as a running average during a simulation of the network with incorrect values of the parameters, thereby contributing to the calculation of an improved set of parameter values in a stochastic gradient descent algorithm. This is the key update equation in a learning algorithm for reaction rates in stochastic biochemical networks (extensible to other process networks). Algorithmic details can be found in [20], noting particularly Equation 2.4 therein. A related stochastic learning algorithm is proposed in [8].

E Appendix III: Dynamical grammar for root growth

Given the following function simplified definitions among others:

gGrowthModelMult = 1;
growthConst = 1/gCellCyleTime;
yEffectOnDivisionFunc[y_] := Module[{δ, h1, h2},
$0.005+100\ast \frac{{(\frac{\text{y}}{\text{q}1})}^{{\text{p}}_{\text{v}1}}}{1+{(\frac{\text{y}}{{\text{q}}^2})}^{{\text{p}}_{\text{v}2}}}/·\{\text{q}1\to 5,\text{q}1\to 1,{\text{p}}_{\text{v}1}\to 1,{\text{p}}_{\text{v}2}\to 5\}$
]
cellGrowthLocFunc[rad_] := Module[{},
gGrowthModelMult * growthConst
];
springXFunc[curPosx_, curRad_, nbrPosx_, nbrRad_] :=
-∂curPosx springPotential[curPosx, curRad, nbrPosx, nbrRad]

The actual grammar for selected rules is shown here:

gRootGrowth := Grammar[rules→
{
(* continuous change in cell c1 radius *)
{c1 Equal cell[cellID1, 1(* growth mode *), loc1, rad1, auxin1, y1, cellIDP, cellIDN]}→ c1,
solving[rad1′EqualcellGrowthLocFunc[rad1]],
(* continuous change in cell c1 location *)
{c1 Equal cell[cellID, cMode, loc, rad, auxin, y, cellIDPrev, cellIDNext],
c2 Equal cell[cellIDNext, cModeN, locN, radN, auxinN, yN, cellID, cellIDNN]}→ {c1, c2},
solving[loc′EqualgGrowthModelMult * springXFunc[loc, rad, locN, radN]],
(* continuous change in cell c1 location *)
{c1 Equal cell[cellID, cMode, loc, rad, auxin, y, cellIDPrev, cellIDNext],
c2 Equal cell[cellIDPrev, cModeP, locP, radP, auxinP, yP, cellIDPP, cellID]}→ {c1, c2},
solving[loc′EqualgGrowthModelMult * springXFunc[loc, rad, locP, radP]],
(* change cell mode from growth to wait, when over a radius threshold *)
cell[cellID, 1, loc, rad, auxin, y, cellIDPrev, cellIDNext]→
cell[cellID, 2, loc, rad, auxin, y, cellIDPrev, cellIDNext],
with[gGrowthModelMult * stopGrowthConst * grammarSigmoid[rad - gLimitCellRad, gDivideTemp]],
(* divide a cell when its in wait mode *)
cell[cellID, 2, loc, rad, auxin, y, cellIDPrev, cellIDNext]→ {
cell[cellIDPrev, cModeP, locP, radP, auxinP, yP, cellIDPP, cellID]→
cell[cellIDPrev, cModeP, locP, radP, auxinP, yP, cellIDPP, grammarCreateObjectID[1]],
cell[cellIDNext, cModeN, locN, radN, auxinN, yN, cellID, cellIDNN]→
cell[cellIDNext, cModeN, locN, radN, auxinN, yN, grammarCreateObjectID[2], cellIDNN],
cell[grammarCreateObjectID[1], 1, loc - rad + 2rad * cellpart + rad * (1 - cellpart),
rad * (1 - cellpart), auxin, y, cellIDPrev, grammarCreateObjectID[2]],
cell[grammarCreateObjectID[2], 1, loc - rad + rad * cellpart,
rad * cellpart, auxin, y, grammarCreateObjectID[1], cellIDNext]},
with[gGrowthModelMult * yEffectOnDivisionFunc[y] *
grammarPDF[UniformDistribution[{0.5 - gRangeParam, 0.5 + gRangeParam}], cellpart]],
(* … more rules … *)
(* auxin/y passive transport between two neighboring cells *)
{c0 Equal cell[cellID0, cMode0, loc0, rad0, auxin0, y0, cellIDP0, cellID1],
c1 Equal cell[cellID1, cMode1, loc1, rad1, auxin1, y1, cellID0, cellIDNext]}→ {c0, c1},
solving[auxin1′Equal pt(auxin0 - auxin1), auxin0′Equal pt(auxin1 - auxin0),
y1′Equal pty(y0 - y1), y0′Equal pty(y1 - y0)],
(* … more rules … *)
}];

Note that for efficiency, the symbolic partial derivative is taken out of the grammar (rule 3, biomechanics) and precomputed. Also, the cell division rule above actually has the form of a compound rule, whose right hand side comprises two further rules. This point was simplified out of the notation in the main text. It is an efficiency measure that allows a rule firing to be a multistep process (similar to the subgrammars or macros of [4]) without slowing down the computational identification of cells likely to divide specified by the with clause of the rule. However, its use here relies on the dynamically invariant, domain-specific fact that each cell is the nth neighbor (in this case n=1 or 2) of at most one other cell.