A master equation and moment approach for biochemical systems with creation-time-dependent bimolecular rate functions

Abstract

Noise and stochasticity are fundamental to biology and derive from the very nature of biochemical reactions where thermal motion of molecules translates into randomness in the sequence and timing of reactions. This randomness leads to cell-to-cell variability even in clonal populations. Stochastic biochemical networks have been traditionally modeled as continuous-time discrete-state Markov processes whose probability density functions evolve according to a chemical master equation (CME). In diffusion reaction systems on membranes, the Markov formalism, which assumes constant reaction propensities is not directly appropriate. This is because the instantaneous propensity for a diffusion reaction to occur depends on the creation times of the molecules involved. In this work, we develop a chemical master equation for systems of this type. While this new CME is computationally intractable, we make rational dimensional reductions to form an approximate equation, whose moments are also derived and are shown to yield efficient, accurate results. This new framework forms a more general approach than the Markov CME and expands upon the realm of possible stochastic biochemical systems that can be efficiently modeled.

I. INTRODUCTION

Individual cells in isogenic populations exhibit significant variability under the same environmental conditions.^1–4 The variability observed in these cases is believed to take root in the stochastic nature of biochemical reactions. For example, fluctuations in gene expression, a major contributor of cell-to-cell variability, can arise from the stochastic steps involved in transcription and translation.^3,4 Further sources of fluctuations include diffusion-reactions and dissociations, allosteric changes, and degradation of biological molecules. In general, for a given biochemical network, the occurrence of the underlying biochemical reactions depends on the channels of possible interactions (topological structure of the interaction network), the interaction affinity (reaction rates), as well as the number of molecules available for such interactions.⁵ This inevitably results in fluctuations in the levels of the various molecular species, and translates into cell-to-cell phenotypic differences in cellular behaviors.

In some cases, cells may exploit fluctuations in expression levels productively to allow populations to “hedge their bets” with respect to future environmental shifts. This is thought to explain the ability of subpopulations of bacteria to resist antibiotics and promote latency of infection by viruses.^1,2 The impact of stochastic fluctuations have even been identified in simple synthetic gene circuits, inducing substantial variability in the period of oscillators and stochastic transitions in synthetic toggle switches.⁶ As a consequence, the thorough investigation of the principles underlying such non-genetic individuality is essential for engineering genetic circuits as well as for understanding the differential susceptibility of cells and organisms to diseases, drugs, and pathogens. These behaviors can only be captured and rigorously dissected in the context of stochastic mathematical and computational models which are essential to understand how cells may exploit or filter stochastic fluctuations to achieve function.

A commonly used stochastic representation of biological systems adopts the formalism of the chemical master equation (CME),^7–9 a differential-difference equation that describes the time-dependent joint probability distribution of the biochemical species in a system, or its accompanying numerical Stochastic Simulation Algorithm (SSA).¹⁰ The SSA is a Monte Carlo procedure that generates stochastic trajectories of memoryless processes governed by the CME. The CME/SSA approach has found great success in modeling cellular networks, including stochasticity in gene expression³ and larger transcriptional pathways that generate interesting stochastic behaviors such as excitability.¹¹ However, there are numerous cases where the Markov approach is not directly applicable. One biologically prominent case is that of diffusion-limited reactions such as for proteins in the cytosol¹² and for receptors on membranes.¹³ In the latter case, many important biological processes rely on sensing external signals by modulating the activity of receptors that diffuse on the surface of cells or cellular organelles. These receptors often dimerize or oligomerize, a feature that is important for their activity. We have recently established that the precise modeling of the stochastic diffusion reactions of these molecules is needed to account for their dimerization dynamics, and illustrated that the CME/SSA framework does not constitute an appropriate framework to do so.¹³ This is because the instantaneous propensity for a diffusion reaction to occur depends on the creation times of the molecules involved. Here, creation times could be, for example, the times at which the given molecules were exocytosed onto the membrane to allow for the diffusion reaction to occur. Our work has defined new creation-time dependent (CTD) propensities, specifically bimolecular diffusion-reaction propensities. We derived CTD propensities that model the diffusion reaction of spatially uncorrelated molecules (random initial starting location for each molecule) as well as the rebinding of recently dissociated molecules (spatially correlated molecules).¹³ We have developed an exact algorithm¹³ (NMSSA) to simulate stochastic realizations of systems with these types of CTD propensities. We will refer to the NMSSA as the CTD-SSA to keep in theme with the creation-time-dependent nomenclature. In this work, we develop a CTD-CME for general systems of this type. We then derive an approximate representation of this equation that leads to moment equations which can be efficiently solved. We illustrate our results by deriving the CME and corresponding moment equations for a number of biologically motivated examples where the CME/SSA framework is not directly applicable. Overall, our CTD-CME framework constitutes a general formulation in which the Markov CME is a limiting special case.

II. TRADITIONAL MARKOV APPROACHES TO MODEL STOCHASTIC BIOCHEMICAL SYSTEMS

The CME/SSA framework assumes a system of well-stirred chemical reactions with V molecular species. We use the state X(t) to denote the vector whose integer elements X_m(t) are the number of molecules of the mth species at time t. If there are W elementary chemical reactions that can occur among these V species, then we associate with each reaction r_k (k = 1, …, W) a nonnegative propensity function defined such that a_k(X(t)) τ + o(τ) is the probability that reaction r_k will happen in the next small time interval (t, t + τ], as τ → 0. Under certain assumptions such as a well-stirred reaction volume, a_k(x) assumes a polynomial form.¹⁰ The occurrence of a reaction r_k leads to a change of ν_k ∈ Z^V (the set of nonnegative integers) in the state X. ν_k is therefore a stoichiometric vector that reflects the integer change in reactant species due to a reaction r_k.

This set of well-stirred chemical reactions can be represented by the time-dependent joint probability distribution p(x, t) which describes the probability of the system being in state X(t) = x at time t. The evolution of p(x, t) is given by

$$\begin{array}{ccc}\hfill \frac{\partial p(x,t)}{\partial t}& =& \sum _{k=1}^W[a_k(x-{\nu }_k)p(x-{\nu }_k,t)\hfill \\ & & -a_k\left(x\right)p(x,t)].\hfill \end{array}$$ (1)

Equation (1) is the so-called CME.^7–9 The CME is the limit of the Chapman-Kolmogorov equation, an identity that must be obeyed by the transition probabilities of any Markov process.

The CME/SSA approach has found great success in modeling stochastic cellular networks, however, as discussed above, there are numerous cases where the Markov approach is not applicable, for example, in many diffusion-limited reactions of receptors on membranes.¹² For these cases, a more appropriate stochastic simulation framework (CTD-SSA) was developed.¹³ The basic premise for the CTD-SSA is the concept of CTD propensities which we describe below. The main contribution of this work is the development of a chemical master equation for these types of systems.

III. CREATION-TIME-DEPENDENT BIMOLECULAR PROPENSITIES

We have recently addressed cases that occur in diffusion reaction systems on membranes where the Markov formalism, which assumes constant reaction propensities (mean reaction times), is not applicable.¹³ This is due to the diffusion-reaction propensities being CTD. Let us assume that the kth reaction type in the system is a hetero-bimolecular reaction resulting from the diffusion reaction between species m and n. For the Markov case, the propensity takes the form a_k(x) = c_mnx_mx_n where c_mn is the rate constant and x_mx_n is the number of different reacting pairs. For the CTD case, let the creation time variables ${\tau }_i^m$ and ${\tau }_j^n$ denote the creation times of the ith molecule of species m and the jth molecule of species n. The rate function for the hetero-bimolecular reaction between the two molecules takes the form $k_{mn}(t-\max ({\tau }_i^m,{\tau }_j^n))$. The starting time of the pairwise rate function is determined by the “max” function where an interaction cannot begin until the last of the two molecules is created. We define τ^m as the creation time vector of the species m and has the form ${\tau }^m=[{\tau }_1^m{\tau }_2^m...{\tau }_{x_m}^m]$ where the number of species m molecules in the system at time t is x_m (likewise for the creation time vector τⁿ). The propensity, a_k(x, t, τ^m, τⁿ), of the reaction evaluated at time t, given all pairwise rate functions between the two species m and n would be

$$\begin{array}{c}\hfill a_k(x,t,{\tau }^m,{\tau }^n)=\sum _{i=1}^{x_m}\sum _{j=1}^{x_n}{k}_{mn}\left(t-\max \left({\tau }_i^m,{\tau }_j^n\right)\right).\end{array}$$ (2)

Similarly, if the kth reaction was a homo-bimolecular reaction of species m, the Markov propensity would be a_k(x) = c_mmx_m(x_m − 1)/2!. For the CTD case, the rate function is $k_{mm}(t-\max ({\tau }_i^m,{\tau }_j^m))$ with a propensity of $a_k(x,t,{\tau }^m,{\tau }^n)={\sum }_{i=1}^{x_m}{\sum }_{j=i+1}^{x_m}{k}_{mm}(t-\max ({\tau }_i^m,{\tau }_j^m))$. To keep notation to a minimum, the propensity form a_k(x, t, τ^m, τⁿ) will also be used to denote memoryless propensities, a_k(x). In a_k(x, t, τ^m, τⁿ), we will also assume the species indices, m and n, to be implicit functions of k.

IV. A CREATION-TIME-DEPENDENT CHEMICAL MASTER EQUATION

For our reacting system, we assume that the first U species are each involved in at least one reaction whose propensity depends on that species’ creation times. We will specify these species as CTD species. For the rest of the paper, we define the state x as the species state x, and the creation time vectors τ¹ through τ^U as the creation time state [τ¹ …  τ^U]. With the state of our system fully defined, we can write down the joint distribution as

$$\begin{array}{ccc}\hfill p(x,t,{\tau }^1,...,{\tau }^U)& =& p(x,t)p({\tau }^1,...,{\tau }^U|x,t),\hfill \end{array}$$ (3)

where the left side of (3) is simply the typical joint distribution representation and the right side is the chain rule representation. The chain rule representation in (3) quantitatively represents the probability of being in species state x at time t multiplied by the probability density function of the creation time state [τ¹ …  τ^U] given x and t. We impose a causality constraint that there can only be non-zero probability in (3) for ${\tau }_e^h\le t$ for 1 ⩽ h ⩽ U and 1 ⩽ e ⩽ x_h. That is, any molecule affecting the system at time t must have been created before or at time t. Given the stoichiometry of the reactions, the form of propensities and the joint distribution, the chemical master equation can be constructed.^7–10 The chemical master equation that evolves (3) over time can be written in its most general form as

$$\begin{array}{ccc}& & \frac{\partial p(x,t)p({\tau }^1,...,{\tau }^U|x,t)}{\partial t}\hfill \\ & & =\sum _{k=1}^W[O_k^{in}(p(x-{\nu }_k,t)p({\tilde{\tau }}^1,...,{\tilde{\tau }}^U|x-{\nu }_k,t)\hfill \\ & & \times a_k(x-{\nu }_k,t,{\tilde{\tau }}^m,{\tilde{\tau }}^n),{\tau }^1,...,{\tau }^U)\hfill \\ & & -p(x,t)p({\tau }^1,...,{\tau }^U|x,t)a_k(x,t,{\tau }^m,{\tau }^n)].\hfill \end{array}$$ (4)

We will refer to this equation as the CTD-CME. The second flux term on the right side of (4), the out-flux, describes how probability leaves species state x, characterized by a given creation time state [τ¹ …  τ^U], through the propensity of the kth reaction, a_k(x, t, τ^m, τⁿ). The first flux term on the right side of (4) defines the in-flux operator $O_k^{in}$ which dictates how the probability that leaves species state x − ν_k and creation time state $[{\tilde{\tau }}^1...{\tilde{\tau }}^U]$ through reaction k enters species state x and creation time state [τ¹ …  τ^U]. Notice in state x − ν_k the creation time vector for any species h (for 1 ⩽ h ⩽ U) is represented as ${\tilde{\tau }}^h$ and for species state x it is τ^h. This is because ${\tilde{\tau }}^h$ and τ^h are each from different creation time distributions, $p({\tilde{\tau }}^1,...,{\tilde{\tau }}^U|x-{\nu }_k,t)$ vs. p(τ¹, …, τ^U|x, t). Thus, ${\tilde{\tau }}^h$ and τ^h can have different dimensions, for example, if species h either gains or loses a molecule through reaction k. Hence, how the in-flux maps into p(x, t)p(τ¹, …, τ^U|x, t) will explicitly depend on the type of reaction that occurs. To determine this mapping, next we develop the form of $O_k^{in}$.

A. Determining the form of $O_k^{in}$

Although the approach we present is applicable to general CTD systems, we develop the CTD-CME for the following simple absorbing system for illustrative purposes:

$$\begin{array}{c}\hfill \varnothing \stackrel{{\beta }_1}{\to }{X}_1,\end{array}$$ (5a)

$$\begin{array}{c}\hfill \varnothing \stackrel{{\beta }_2}{\to }{X}_2,\end{array}$$ (5b)

$$\begin{array}{c}\hfill X_1+X_2\stackrel{k_{12}\left(t\right)}{\to }\varnothing +X_2,\end{array}$$ (5c)

$$\begin{array}{c}\hfill X_2\stackrel{{\gamma }_2}{\to }\varnothing ,\end{array}$$ (5d)

where reaction equations (5a)–(5d) represent the four reactions in the CTD-CME, respectively. In this system, the X₁ molecules are created at a rate β₁ (mol s⁻¹). The X₂ molecules are created at a rate β₂ (mol s⁻¹) and decay at a rate γ₂ (s⁻¹). The bimolecular absorbing reaction between X₁ with X₂ results in the destruction of X₁ while X₂ itself remains unaffected. For this system, the bimolecular absorbing reaction contains the only CTD propensity, and requires the knowledge of both the X₁ creation time and the X₂ creation time. Thus, our joint probability function will take the form p(x, t)p(τ¹, τ²|x, t) where x = [x₁ x₂].

In constructing $O_k^{in}$, we will first define a non-symmetric in-flux operator $F_k^{in}$ which produces a CTD-CME but whose creation time distribution lacks certain symmetric properties. Through symmetry operations on $F_k^{in}$ we will then obtain a unique symmetrized in-flux operator $O_k^{in}$ whose CTD-CME evolves a creation time distribution with corresponding symmetry properties. The resulting symmetric form of the CTD-CME allows for dimensionality reductions enabling the derivation of moment equations for the CTD-CME and tractable numerical solutions.

1. Application of the sifting property to construct the in-flux operator $F_k^{in}$

For the kth reaction, we construct how the probability flux enters p(x, t)p(τ¹, τ²|x, t) from $p(x-{\nu }_k,t)p({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_k,t)a_k(x-{\nu }_k,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2)$. To achieve this we take advantage of the sifting property¹⁴ of products of delta functions

$$\begin{array}{ccc}& & \int \cdots \int f(x_1,...,x_n)\delta (x_1-y_1)\cdots \delta (x_n-y_n)d{x}_1\cdots d{x}_n\hfill \\ & & =f(y_1,...,y_n).\hfill \end{array}$$ (6)

For ease of presentation, we will refer to a given product of delta functions as a sifting function. The utility in this case will be to assign the creation time state [τ¹ τ²] to particular creation time state space coordinates in the flux term $p(x-{\nu }_k,t)p({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_k,t)a_k(x-{\nu }_k,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2)$ due to the kth reaction. For the CTD-CME of this absorbing system, the abstract form of the sifting function is $S_k^h(x,{\tilde{\tau }}^h,{\tau }^h)$. It describes how the creation times in τ^h of species h are assigned to particular creation time state space coordinates in the flux term $p(x-{\nu }_k,t)p({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_k,t)a_k(x-{\nu }_k,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2)$.

For the k = 1 reaction, the creation reaction for X₁, the species 1 population is increased by one with creation propensity $a_1(x-{\nu }_1,t,{\tilde{\tau }}^m,{\tilde{\tau }}^n)={\beta }_1$. For the set of molecules of species 1 in state x₁ − 1 with creation times $[{\widetilde{{\tau }_1}}^1...{\tilde{\tau }}_{x_1-1}^1]$, a molecule created at time t is added. A particular sifting function for species 1 that achieves this is

$$\begin{array}{c}\hfill S_1^1(x,t,{\tilde{\tau }}^1,{\tau }^1)=\left[\prod _{e=1}^{x_1-1}\delta \left({\widetilde{{\tau }_e}}^1-{\tau }_e^1\right)\right]\times \delta \left(t-{\tau }_{x_1}^1\right),\end{array}$$ (7)

where the delta functions will allow us to “assign” each ${\widetilde{{\tau }_e}}^1$ to ${\tau }_e^1$ for e ⩽ x₁ − 1 as well as assign the newly created molecule at time t to ${\tau }_{x_1}^1$. Since the creation times in species 2 are not affected by reaction 1, its sifting function is a one-to-one mapping of the form

$$\begin{array}{c}\hfill S_1^2(x,t,{\tilde{\tau }}^2,{\tau }^2)=\prod _{e=1}^{x_2}\delta \left({\widetilde{{\tau }_e}}^2-{\tau }_e^2\right).\end{array}$$ (8)

We define the in-flux operator $F_1^{in}$ as $p(x-{\nu }_1,t)p({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_1,t)a_1(x-{\nu }_1,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2)$ multiplied by the sifting functions integrated over all ${\tilde{\tau }}_e^h$ variables, i.e., h = 1: 2 and e = 1: x_h − ν_1h. The in-flux operator $F_1^{in}$ would be

$$\begin{array}{ccc}& & F_1^{in}(p(x-{\nu }_1,t)p({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_1,t)\hfill \\ & & \times a_1(x-{\nu }_1,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2),{\tau }^1,{\tau }^2)\hfill \\ & & ={\int }_0^t{\int }_0^{t}p(x-{\nu }_1,t)p\left({\tilde{\tau }}_1^1,...,{\tilde{\tau }}_{x_1-1}^1,{\tilde{\tau }}^2|x-{\nu }_1,t\right){\beta }_1\hfill \\ & & \times \prod _{h=1}^2{S}_1^h(x,t,{\tilde{\tau }}^h,{\tau }^h)d{\tilde{\tau }}_1^1\cdots d{\tilde{\tau }}_{x_2}^2\hfill \\ & & =p(x-{\nu }_1,t)p\left({\tau }_1^1,...,{\tau }_{x_1-1}^1,{\tau }^2|x-{\nu }_1,t\right)\delta (t-{\tau }_{x_1}^1){\beta }_1.\hfill \end{array}$$ (9)

Notice that the final form of the in-flux in (9) is a function of the creation time variables from species state x, i.e., τ¹ and τ². For the k = 2 reaction, the creation reaction of species 2, one can easily see that the sifting functions would have the same form with, but with the species switched. The in-flux operator $F_2^{in}$ would be

$$\begin{array}{ccc}& & F_2^{in}(p(x-{\nu }_2,t)p({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_2,t)\hfill \\ & & \times a_2(x-{\nu }_2,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2),{\tau }^1,{\tau }^2)\hfill \\ & & ={\int }_0^t{\int }_0^{t}p(x-{\nu }_2,t)p\left({\tilde{\tau }}^1,{\tilde{\tau }}_1^2,...,{\tilde{\tau }}_{x_2-1}^2|x-{\nu }_2,t\right){\beta }_2\hfill \\ & & \times \prod _{h=1}^2{S}_1^h(x,t,{\tilde{\tau }}^h,{\tau }^h)d{\tilde{\tau }}_1^1\cdots d{\tilde{\tau }}_{x_2-1}^2\hfill \\ & & =p(x-{\nu }_2,t)p\left({\tau }^1,{\tau }_1^2,...,{\tau }_{x_2-1}^2|x-{\nu }_2,t\right)\delta \left(t-{\tau }_{x_2}^2\right){\beta }_2.\hfill \end{array}$$ (10)

The k = 3 reaction, the absorbing reaction, is a hetero-bimolecular reaction and, hence, has a propensity of the form $a_3(x,t,{\tau }^1,{\tau }^2)={\sum }_{i=1}^{x_1}{\sum }_{j=1}^{x_2}{k}_{12}(t-\max ({\tau }_i^1,{\tau }_j^2))$. The sifting functions for the absorbing reaction for species 1 where the ith x₁ molecule is absorbed would be

$$\begin{array}{c}\hfill S_3^1(x,t,{\tilde{\tau }}^1,{\tau }^1)=\prod _{e=1}^{i-1}\delta \left({\tilde{\tau }}_e^1-{\tau }_e^1\right)\prod _{e=i+1}^{x_1+1}\delta \left({\tilde{\tau }}_e^1-{\tau }_{e-1}^1\right),\end{array}$$ (11)

where the creation time variable ${\tilde{\tau }}_i^1$ is removed since the ith x₁ molecule from state x − ν₃ has been absorbed. The sifting function for species 2, which is unaffected would be

$$\begin{array}{c}\hfill S_3^2(x,t,{\tilde{\tau }}^2,{\tau }^2)=\prod _{e=1}^{x_2}\delta \left({\tilde{\tau }}_e^2-{\tau }_e^2\right).\end{array}$$ (12)

The in-flux operator $F_3^{in}$ would be

$$\begin{array}{ccc}& & F_3^{in}(p(x-{\nu }_3,t)p({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_3,t)\hfill \\ & & \times a_3(x-{\nu }_3,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2),{\tau }^1,{\tau }^2)\hfill \\ & & =\sum _{i=1}^{x_1+1}\sum _{j=1}^{x_2}{\int }_0^t\cdots {\int }_0^t{k}_{12}\left(t-\max \left({\tilde{\tau }}_i^1,{\tilde{\tau }}_j^2\right)\right)p(x-{\nu }_3,t)\hfill \\ & & \times p({\tilde{\tau }}_1^1,...,{\tilde{\tau }}_{x_1+1}^1,{\tilde{\tau }}^2|x-{\nu }_3,t)\hfill \\ & & \times \prod _{h=1}^2{S}_3^h(x,t,{\tilde{\tau }}^h,{\tau }^h)d{\tilde{\tau }}_1^1\cdots d{\tilde{\tau }}_{x_2}^2\hfill \\ & & =\sum _{i=1}^{x_1+1}\sum _{j=1}^{x_2}{\int }_0^t{k}_{12}\left(t-\max \left({\tilde{\tau }}_i^1,{\tau }_j^2\right)\right)p(x-{\nu }_3,t)\hfill \\ & & \times p\left({\tau }_1^1,...,{\tau }_{i-1}^1,{\tilde{\tau }}_i^1,{\tau }_i^1,...,{\tau }_{x_1}^1,{\tau }^2|x-{\nu }_3,t\right)d{\tilde{\tau }}_i^1,\hfill \end{array}$$ (13)

where the creation time variable ${\tilde{\tau }}_i^1$ is integrated out, over all possible values, since the ith x₁ molecule from state x − ν₃ has been absorbed. This ensures that the in-flux has the proper creation time variable dimensions. The decay reaction of X₂, the k = 4 reaction, would use the same sifting functions as the absorbing reactions with the species switched. Here, the propensity would be $a_4(x,t,{\tau }^1,{\tau }^2)={\sum }_{j=1}^{x_2}{\gamma }_2$. The in-flux operator $F_4^{in}$ would be

$$\begin{array}{ccc}& & F_4^{in}(p(x-{\nu }_4,t)p({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_4,t)\hfill \\ & & \times a_4(x-{\nu }_4,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2),{\tau }^1,{\tau }^2)\hfill \\ & & =\sum _{j=1}^{x_2+1}{\int }_0^t\cdots {\int }_0^{t}p(x-{\nu }_4,t)p\left({\tilde{\tau }}^1,{\tilde{\tau }}_1^2,...,{\tilde{\tau }}_{x_2+1}^2|x-{\nu }_4,t\right)\hfill \\ & & \times \prod _{h=1}^2{S}_4^h(x,t,{\tilde{\tau }}^h,{\tau }^h)d{\tilde{\tau }}_1^1\cdots d{\tilde{\tau }}_{x_2+1}^2{\gamma }_2\hfill \\ & & =\sum _{j=1}^{x_2+1}{\int }_0^{t}p(x-{\nu }_4,t)\hfill \\ & & \times p\left({\tau }^1,{\tau }_1^2,...,{\tau }_{j-1}^2,{\tilde{\tau }}_j^2,{\tau }_j^2,...,{\tau }_{x_2}^2|x-{\nu }_4,t\right)d{\tilde{\tau }}_j^2{\gamma }_2.\hfill \end{array}$$ (14)

2. A natural order CTD-CME

We can constrain p(τ¹, τ²|x, t) such that non-zero probability density exists only in the creation time state-space where ${\tau }_1^h\le {\tau }_2^h\le ...\le {\tau }_{x_h}^h$ for the h = 1, 2 given the species state x. We call this the natural-order constraint simply because the creation time values from a given sample ascend in the order of the indexes of the creation time variables. To visualize the natural-order constraint in 2D, Figure 1(a) plots the simple 2D distribution ${\tilde{p}}_0\left({\tau }^1\right|x,t)={\tilde{p}}_0({\tau }_1^1,{\tau }_2^1|x,t)$ which has a single CTD species with two creation time variables and adheres to the natural order constraint. For the simple absorbing system, we denote the creation time distribution p(τ¹, τ²|x, t) that adheres to the natural order constraint as p₀(τ¹, τ²|x, t) where the zero in the subscript indicates the natural order constraint. Examining the sifting operations, $F_1^{in}$ through $F_4^{in}$, one can see that for this particular system, they preserve the natural order and causality constraints. We can therefore construct a CTD-CME of the form

$$\begin{array}{ccc}& & \frac{\partial p(x,t)p_0({\tau }^1,{\tau }^2|x,t)}{\partial t}\hfill \\ & & =\sum _{k=1}^4\left[F_k^{in}\right(p(x-{\nu }_k,t)p_0({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_k,t)\hfill \\ & & \times a_k(x-{\nu }_k,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2),{\tau }^1,{\tau }^2)\hfill \\ & & -p(x,t)p_0({\tau }^1,{\tau }^2|x,t)a_k(x,t,{\tau }^1,{\tau }^2)]\hfill \end{array}$$ (15)

that will maintain the natural order and causality constraints in p₀(τ¹, τ²|x, t) for t > 0. However, p₀(τ¹, τ²|x, t) lacks symmetry. Next, we develop symmetrized versions of both p₀(τ¹, τ²|x, t), $F_k^{in}$, and (15). The symmetrized versions enable approximations that lead to computationally tractable equations.

FIG. 1.

Simple 2D case of natural-ordered, permuted, and S-symmetric distributions. (a) The natural-ordered distribution ${\tilde{p}}_0({\tau }_1^1,{\tau }_2^1|x,t)$, nonzero for ${\tau }_1^1\le {\tau }_2^1$. (b) The permuted distribution ${\tilde{p}}_1({\tau }_1^1,{\tau }_2^1|x,t)$, nonzero for ${\tau }_2^1\le {\tau }_1^1$. (c) The S-symmetric distribution $\max ({\tilde{p}}_0({\tau }_1^1,{\tau }_2^1|x,t),{\tilde{p}}_1({\tau }_1^1,{\tau }_2^1|x,t))$.

3. A symmetrized CTD-CME

To begin, for the hth species consider the creation time vector ${\kappa }^h=[{\kappa }_1^h,...,{\kappa }_{x_h}^h]$ for h = 1, 2 and where ${\kappa }_1^h\le {\kappa }_2^h\le ...\le {\kappa }_{x_h}^h$. Here, κ^h satisfies the natural-order constraint and therefore p₀(τ¹ = κ¹, τ² = κ²|x, t) can be nonzero. There are x_h! vectors that could be created by permuting the order of ${\kappa }_1^h$ through ${\kappa }_{x_h}^h$ in the vector κ^h. Therefore, applying this to each of the vectors, κ¹ and κ², there would be x₁!x₂! unique ways to permute κ¹ and κ². Let $P_g^h\left({\kappa }^h\right)$ be the vector due to the gth permutation where $P_g^h$ represents the permuting operator of the hth species creation time vector for the gth permutation. We can generate a class of probability distributions, p_g(τ¹, τ²|x, t), that map as

$$\begin{array}{ccc}& & p_g({\tau }^1=P_g^1\left({\kappa }^1\right),{\tau }^2=P_g^2\left({\kappa }^2\right)|x,t)\hfill \\ & & =p_0({\tau }^1={\kappa }^1,{\tau }^2={\kappa }^2|x,t),\hfill \end{array}$$ (16)

where ${\tau }^1=P_g^1\left({\kappa }^1\right)$ and ${\tau }^2=P_g^2\left({\kappa }^2\right)$ are equivalent permuted state space coordinates, that is the values within κ¹ and κ² are unchanged. From this point of view, all p_g(τ¹, τ²|x, t) are equivalent, but the non-zero probability density resides in an equivalent permuted state space region for each p_g(τ¹, τ²|x, t). To visualize this idea in 2D, consider again the simple 2D distribution ${\tilde{p}}_0({\tau }_1^1,{\tau }_2^1|x,t)$ in Figure 1(a). Figure 1(b) shows the permuted but equivalent distribution ${\tilde{p}}_1({\tau }_1^1,{\tau }_2^1|x,t)$.

We can then create a symmetrized probability distribution of the form

$$\begin{array}{ccc}& & p_s({\tau }^1,{\tau }^2|x,t)\hfill \\ & & =\frac{1}{x_1!x_2!}\max (p_0({\tau }^1,{\tau }^2|x,t),...,p_{x_1!x_2!}({\tau }^1,{\tau }^2|x,t)),\hfill \end{array}$$ (17)

which naturally fuses and normalizes all p_g(τ¹, τ²|x, t) together and guarantees continuity at the interfaces of the various p_g(τ¹, τ²|x, t). To visualize this, consider again the simple 2D distribution ${\tilde{p}}_0({\tau }_1^1,{\tau }_2^1|x,t)$ in Figure 1(a). Figure 1(c) plots the symmetrized distribution $\max ({\tilde{p}}_0({\tau }_1^1,{\tau }_2^1|x,t),{\tilde{p}}_1({\tau }_1^1,{\tau }_2^1|x,t))$. The distribution from (17) has the following property:

$$\begin{array}{ccc}& & p_s({\tau }^1=P_g^1\left({\kappa }^1\right),{\tau }^2=P_g^2\left({\kappa }^2\right)|x,t)\hfill \\ & & =p_s({\tau }^1=P_e^1\left({\kappa }^1\right),{\tau }^2=P_e^2\left({\kappa }^2\right)|x,t)\hfill \end{array}$$ (18)

for all g, e ⩽ x₁!x₂!, i.e., it is a symmetric function with respect to permutations of the creation times within a given species. We call this property S-symmetry. One can think of p_s(τ¹, τ²|x, t) as taking p₀(τ¹, τ²|x, t) and equally redistributing the probability density at τ¹, τ² over all x₁!x₂! equivalent, permuted creation time state-space coordinates. Applying the results from (15) to (18), we can construct a master equation for p_g(τ¹, τ²|x, t) with the in-flux operator applied to $p_s({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_k,t)$ yielding

$$\begin{array}{ccc}& & \frac{\partial p(x,t)p_g({\tau }^1,{\tau }^2|x,t)}{\partial t}\hfill \\ & & =\left\{\begin{array}{c}\sum _{k=1}^4\left[\sum _{\ell =1}^{x_1!x_2!}{F}_k^{in}\right(p(x-{\nu }_k,t)\hfill \\ \\ \times p_s({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_k,t)\hfill \\ \times a_k(x-{\nu }_k,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2),P_{\ell }^1\left({\tau }^1\right),P_{\ell }^2\left({\tau }^2\right))\hfill \\ -p(x,t)p_g({\tau }^1,{\tau }^2|x,t)a_k(x,t,{\tau }^1,{\tau }^2)]\hfill \\ \text{for}{\tau }^1=P_g^1\left({\kappa }^1\right)\text{,}{\tau }^2=P_g^2\left({\kappa }^2\right)\hfill \\ \\ \\ 0\text{for}{\tau }^1\ne P_g^1\left({\kappa }^1\right)\text{,}{\tau }^2\ne P_g^2\left({\kappa }^2\right)\hfill \end{array}\right.\hfill \end{array}$$ (19)

for all g ⩽ x₁!x₂!. For the in-flux from the S-symmetric distribution, $p_s({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_k,t)$, we have equally weighted all possible permutations to the sift operation of the kth reaction. To be clear, recall that the sift operation in $F_k^{in}(p(x-{\nu }_k,t)p_s({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_k,t)a_k(x-{\nu }_k,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2),{\tau }^1,{\tau }^2)$ assigns the creation time values of ${\tau }_1^1$ to ${\tau }_{x_2}^2$ to a particular creation time state space coordinate in $p_s({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_j,t)$. The permuting operators in $F_k^{in}(p(x-{\nu }_k,t)p_s({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_k,t)a_k(x-{\nu }_k,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2),P_{\ell }^1\left({\tau }^1\right),P_{\ell }^2\left({\tau }^2\right))$ from (19) simply assign ${\tau }_1^1$ to ${\tau }_{x_2}^2$ to equivalent, but permuted coordinates. We are simply recombining the in-flux from all possible equivalent, permuted coordinates, each of which have the same probably density (since p_s(τ¹, τ²|x, t) is S-symmetric).

Next, we apply the same approach from (17), i.e., the $\frac{1}{x_1!x_2!}\max (p_g,g\le x_1!x_2!)$ operator, to (19) which fuses all x₁!x₂! CDT-CME equations together and results in a CTD-CME for the S-symmetric p_s(τ¹, τ²|x, t) of the form

$$\begin{array}{ccc}& & \frac{\partial p(x,t)p_s({\tau }^1,{\tau }^2|x,t)}{\partial t}\hfill \\ & & =\sum _{k=1}^4[\frac{1}{x_1!x_2!}\sum _{\ell =1}^{x_1!x_2!}{F}_k^{in}(p(x-{\nu }_k,t)p_s({\tilde{\tau }}^1,{\tilde{\tau }}^2|x-{\nu }_k,t)\hfill \\ & & \times a_k(x-{\nu }_k,t,{\tilde{\tau }}^1,{\tilde{\tau }}^2),P_{\ell }^1\left({\tau }^1\right),P_{\ell }^2\left({\tau }^2\right))\hfill \\ & & -p(x,t)p_s({\tau }^1,{\tau }^2|x,t)a_k(x,t,{\tau }^1,{\tau }^2)]\hfill \end{array}$$ (20)

which preserves S-symmetry in p_s(τ¹, τ²|x, t) throughout time for all τ¹, τ². The only difference between (15) and (20) is that (20) maximally utilizes the creation time state space, but their respective solutions yield identical creation time sampling statistics and species-state distribution p(x, t). In addition, as we will later show, (20) is more amenable to approximate dimensionality reductions to make the equation computationally tractable to solve.

Without any loss of generality, the S-symmetric preserving approach can be applied to the general system in (4). Given that the general distribution p(τ¹, …, τ^U|x, t) is S-symmetric at time t = 0, we define the symmetrized version of the in-flux operator $O_k^{in}$ from (4) to be

$$\begin{array}{ccc}& & O_k^{in}(p(x-{\nu }_k,t)p({\tilde{\tau }}^1,...,{\tilde{\tau }}^U|x-{\nu }_k,t)\hfill \\ & & \times a_k(x-{\nu }_k,t,{\tilde{\tau }}^m,{\tilde{\tau }}^n),{\tau }^1,...,{\tau }^U)\hfill \\ & & =\frac{1}{x_1!\cdots x_U!}\sum _{\ell =1}^{x_1!··x_U!}{F}_k^{in}(p(x-{\nu }_k,t)\hfill \\ & & \times p({\tilde{\tau }}^1,...,{\tilde{\tau }}^U|x-{\nu }_k,t)\hfill \\ & & \times a_k(x-{\nu }_k,t,{\tilde{\tau }}^m,{\tilde{\tau }}^n),P_{\ell }^1\left({\tau }^1\right),...,P_{\ell }^U\left({\tau }^U\right))\hfill \\ & & =\frac{1}{x_1!\cdots x_U!}\sum _{\ell =1}^{x_1!··x_U!}{\int }_0^t\cdots {\int }_0^{t}p(x-{\nu }_k,t)\hfill \\ & & \times p({\tilde{\tau }}^1,...,{\tilde{\tau }}^U|x-{\nu }_k,t)a_k(x-{\nu }_k,t,{\tilde{\tau }}^m,{\tilde{\tau }}^n)\hfill \\ & & \times \prod _{h=1}^U{S}_k^h\left(x,t,{\tilde{\tau }}^h,P_{\ell }^h\left({\tau }^h\right),{\tilde{\tau }}^m,{\tilde{\tau }}^n\right)\hfill \\ & & \times d{\tilde{\tau }}_1^{1}d{\tilde{\tau }}_2^1\cdots d{\tilde{\tau }}_{x_U-1}^{U}d{\tilde{\tau }}_{x_U}^U.\hfill \end{array}$$ (21)

Note that the sifting function $S_k^h(x,t,{\tilde{\tau }}^h,P_{\ell }^h\left({\tau }^h\right),{\tilde{\tau }}^m,{\tilde{\tau }}^n)$ in (21) is written to include ${\tilde{\tau }}^m$ and ${\tilde{\tau }}^n$. This was done to include more general reaction types where molecules from species m or n that appear in the propensity $a_k(x-{\nu }_k,t,{\tilde{\tau }}^m,{\tilde{\tau }}^n)$ can map to species h through reaction k. Reactions of this type occur in spatio-temporally correlated systems, an example of which, we model in the numerical section below. In addition to the ones defined for the absorbing system above, all other sifting functions for the reacting systems in this paper are presented in the supplementary material.¹⁵ Because of the S-symmetry, the expression for $O_k^{in}$ can be greatly simplified. However, the simplification is reaction specific, e.g., creation reaction vs. bimolecular reaction. In the supplementary material,¹⁵ we present simplified forms for each reaction type used in the paper. A requirement for (21) to preserve S-symmetry is that the propensity a_k(x, t, τ^m, τⁿ) is S-symmetric. In all our cases, this is true. For example, it is easy to see that under any of the permuting operators $a_k(x,t,{\tau }^m,{\tau }^m)={\sum }_{i=1}^{x_m}{\sum }_{j=i+1}^{x_m}{k}_{mm}(t-\max ({\tau }_i^m,{\tau }_j^m))$ (homodimer reaction) and $a_k(x,t,{\tau }^m,{\tau }^n)={\sum }_{i=1}^{x_m}{\sum }_{j=1}^{x_n}{k}_{mn}(t-\max ({\tau }_i^m,{\tau }_j^n))$ (heterodimer reaction) are all S-symmetric. This guarantees from (4) that p(τ¹, …, τ^U|x, t), if S-symmetric at t = 0 will stay S-symmetric for t > 0.

We can apply (21) to (4) and integrate out all creation times, resulting in the equation

$$\begin{array}{ccc}\hfill \frac{\partial p(x,t)}{\partial t}& =& \sum _{k=1}^W[p(x-{\nu }_k,t)\mathrm{E}\left[a_k(x-{\nu }_k,t,{\tilde{\tau }}^m,{\tilde{\tau }}^n)\right]\hfill \\ & & -p(x,t)\mathrm{E}\left[a_k(x,t,{\tau }^m,{\tau }^n]\right]\hfill \end{array}$$ (22)

which has the same form as (1), but with expected values for the propensities with respect to the creation time distributions $p({\tilde{\tau }}^1,...,{\tilde{\tau }}^U|x-{\nu }_k,t)$ and p(τ¹, …, τ^U|x, t). Indeed, if the propensities have no creation time dependencies, then as one would expect, (22) reduces to the Traditional Markov CME (Eq. (1)).

B. Approximations for enabling tractable solutions of the CTD-CME and its moment equations

Our motivation for enforcing that p(τ¹, …, τ^U|x, t) be an S-symmetric function is for dimensionality reduction purposes to enable a computationally tractable approximate solution. Arguably, the simplest S-symmetric function that the CTD-CME distribution might be approximated by would be one where the creation times within a given species are independent and identically distributed (iid) random variables. Let $p_d\left({\tau }_i^d\right|x,t)$ be the iid distribution for the ith molecule of the dth species. We approximate the creation time distribution with

$$\begin{array}{c}\hfill p({\tau }^1,...,{\tau }^U|x,t)\approx \prod _{d=1}^U\prod _{i=1}^{x_d}{p}_d\left({\tau }_i^d|x,t\right).\end{array}$$ (23)

This approximation necessitates only one creation time variable to be updated for each CTD species. The second approximation we make for numerically solving the systems in this paper is that $p_d\left({\tau }_i^d\right|x,t)\approx \frac{1}{t-{\overline{\tau }}_i^d(x,t)}\exp (-\frac{t-{\tau }_i^d}{t-{\overline{\tau }}_i^d(x,t)})$ where ${\overline{\tau }}_i^d(x,t)$ is the single parameter necessary to uniquely describe the distribution at time t. In the supplementary material,¹⁵ we show that if the variance in $p_d\left({\tau }_i^d\right|x,t)$ is dominated by creation reactions terms, then the exponential distribution is the natural solution at steady state. This is a testable assumption once the numerical solution is obtained (Appendix B). For this paper, our numerical examples will focus on systems at steady state. Fully capturing the time dynamics of $p_d\left({\tau }_i^d\right|x,t)$ may require more sophisticated approaches such as Discontinuous Galerkin (DG) numerical methods.^16,17 However, in many systems the exponential assumption may be sufficient when the distributions associated with the CTD propensities are in quasi-steady state (fast) as compared with the dominant timescales of the system (slow).

In Sec. V, we solve the CTD-CME using the assumptions above for the first numerical example. However, most of our examples will be analyzed with moment equations derived from the CTD-CME. The general approach for constructing the CTD-moment equations are discussed in Appendix A. In the supplementary material,¹⁵ we present the mathematical expressions as they appear in the CTD-CME and CTD-moment equations for every reaction type used in this paper.

V. NUMERICAL EXAMPLES

A. CTD-CME results for the simple absorber system when the number of absorbers remains constant over time

For the absorber system (5a)–(5d), we first consider a CTD-CME when the number of X₂ molecules remains constant over time, i.e., β₂ = γ₂ = 0. The X₁ molecules are created at a rate β₁ (mol s⁻¹). The absorber diffusion reaction of X₁ with X₂ results in the destruction of X₁ while X₂ itself remains unaffected. The rate function $k_{12}(t-\max ({\tau }_i^1,{\tau }_j^2))$ for this reaction is derived from the time-dependent mathematical solution of the probability that the diffusion reaction between the ith X₁ and the jth X₂ molecules has occurred by time t, given that their initial starting locations are uncorrelated with each other (spatially uncorrelated).¹³ Figure 2(a) plots the particular rate function used for this example which was derived for a particular absorber system discussed in Chevalier and El-Samad¹³ whose CTD-SSA results were shown to agree well with brownian dynamics simulations. In a biological context, absorber-like reactions model the binding of receptors to clathrin-coated-pits, resulting in their endocytosis.¹⁸ Because the number of X₂ molecules is constant over time, i.e., all X₂ molecules were created at t = −∞, the absorber diffusion-reaction propensity is only dependent on the X₁ creations times. This is because the bimolecular rate function term $\max ({\tau }_i^1,{\tau }_j^2)=\max ({\tau }_i^1,-\infty )={\tau }_i^1$. The instantaneous absorbing rate for the ith X₁ molecule in state x₁ is thus $k_{12}(t-{\tau }_i^1)N_{x_2}$, where $N_{x_2}$ is the constant number of X₂ molecules.

FIG. 2.

Results for simple absorbing system: (a) Creation-Time-Dependent (CTD) rate function used for the absorbing reaction. (b) SSA and CTD-SSA results (constant absorber case). (c) CTD-CME and CTD-moments results (constant absorber case). (d) CTD-SSA results (variable absorber case). (e) CTD-moments results (variable absorber case).

We first run the CTD-SSA for 3 cases accounting for a range of parameters. For case 1: β₁ = β = 5 × 10⁴ and $N_{x_2}=450$, case 2: ${\beta }_1=2\beta ,N_{x_2}=900,$ and case 3: ${\beta }_1=\frac{\beta }{9},N_{x_2}=50$. Incidentally, these cases are chosen to differentiate non-memoryless systems from memoryless ones. If the system was memoryless, then in the steady state one can show that ${\overline{x}}_1=\frac{{\beta }_1}{k_{on}{N}_{x_2}}$, where k_on is a fundamental rate constant that can be fit from one of the cases. For the three cases, $\frac{{\beta }_1}{N_{x_2}}$ is a constant and therefore, the means of the distributions should be identical. One can see in Figure 2(b) that this is not the case where the CTD-SSA simulations show a lower mean for case 2 than for case 1 and a higher mean for case 3 (Figure 2(b)). This example clearly demonstrates that for such systems, a fundamental rate constant k_on in the mass-action sense does not exist, and that it is crucial to account for creation-time-dependent propensities in order to accurately capture the statistics of molecular reactions.

It is easy to see in the CTD-CME framework that only the X₁ species will have a creation time distribution. The joint distribution will be of the form p(x₁, t)p(τ¹|x₁, t) where x₁ is the number of X₁ molecules and ${\tau }^1=[{\tau }_1^1,...,{\tau }_x^1]$ is its vector of creation times. Here, p(τ¹|x₁, t) starts and evolves as a symmetric function. Applying the iid approximation (23) to p(x₁, t)p(τ¹|x₁, t), we project the CTD-CME onto the ${\tau }_i^1$ creation time variable, i.e., integrate out all other creation time variables. For simpler notation, we set ${\tau }_i^1={\tau }_1$ to get the equation

$$\begin{array}{ccc}& & \frac{\partial p(x_1,t)p_1({\tau }_1,t|x_1)}{\partial t}\hfill \\ & & =\left[p(x_1-1,t)\right[\frac{x_1-1}{x_1}{p}_1({\tau }_1,t|x_1-1)\hfill \\ & & +\frac{1}{x_1}\delta (t-{\tau }_1)]\beta -p(x_1,t)p_1({\tau }_1,t|x_1)\beta ]\hfill \\ & & +[p(x_1+1,t)p_1({\tau }_1,t|x_1+1)\hfill \\ & & \times \mathrm{E}\left[k_{12}(t-{\tau }_1)\right|x_1+1](x_1+1)N_{x_2}\hfill \\ & & -p(x_1,t)p_1({\tau }_1,t|x_1)[\frac{x_1-1}{x_1}\mathrm{E}\left[k_{12}(t-{\tau }_1)\right|x_1]\hfill \\ & & +\frac{1}{x_1}{k}_{12}(t-{\tau }_1)\left]x_1{N}_{x_2}\right].\hfill \end{array}$$ (24)

Here, we assume that p₁(τ₁|x₁, t) is an exponential distribution. In practice, (24) is transformed into two time dependent equations, one for p(x₁, t) and one for p₁(τ₁, t|x₁). For p₁(τ₁, t|x₁), since we assume an exponential distribution, we only need to solve for the time-dependent mean ${\overline{\tau }}_1(x_1,t)$. We directly solve for the time-dependent CTD-CME by applying the Finite-State-Projection¹⁹ approach and then numerically integrating the resulting coupled ODE equations with ODE45 in MATLAB. The direct CTD-CME solution results are in satisfactory agreement with the CTD-SSA results (Figures 2(b) and 2(c)).

If the system was memoryless, the moment equations²⁰ derived from the Markov CME can be shown to be

$$\begin{array}{c}\hfill \frac{\partial {\overline{x}}_1}{\partial t}=\beta -k_{on}{N}_{x_2}{\overline{x}}_1,\end{array}$$ (25a)

$$\begin{array}{c}\hfill \frac{\partial C_{x_1{x}_1}}{\partial t}=\beta +k_{on}{N}_{x_2}{\overline{x}}_1-2k_{on}{N}_{x_2}{C}_{x_1{x}_1}.\end{array}$$ (25b)

As with the Markov CME, the CTD-CME describes the time evolution of a probability function for the systems biochemical species and therefore can be used to derive moment equations for the first two moments²⁰ (Appendix A and the supplementary material¹⁵). However, in the context of the CTD-CME, the mean equation also need to be derived for the random variable τ₁. The system of moment equations for the CTD-CME are

$$\begin{array}{c}\hfill \frac{\partial {\overline{x}}_1}{\partial t}=\beta -\mathrm{E}\left[k_{12}(t-{\tau }_1)\right]N_{x_2}{\overline{x}}_1,\end{array}$$ (26a)

$$\begin{array}{ccc}\hfill \frac{\partial C_{x_1{x}_1}}{\partial t}& =& \beta +\mathrm{E}\left[k_{12}(t-{\tau }_1)\right]N_{x_2}{\overline{x}}_1\hfill \\ & & -2\mathrm{E}\left[k_{12}(t-{\tau }_1)\right]N_{x_2}{C}_{x_1{x}_1},\hfill \end{array}$$ (26b)

$$\begin{array}{ccc}\hfill \frac{\partial {\overline{\tau }}_1}{\partial t}& =& \frac{1}{{\overline{x}}_1}(t-{\overline{\tau }}_1)\beta +[\mathrm{E}\left[k_{12}(t-{\tau }_1)\right]{\overline{\tau }}_1\hfill \\ & & -\mathrm{E}\left[{\tau }_1{k}_{12}(t-{\tau }_1)\right]]N_{x_2}.\hfill \end{array}$$ (26c)

Here, the notation E[f(τ₁)] is used to represent the expected value of a function f(τ₁). With the exponential distribution assumption, explicit formulas can be generated for E[f(τ₁)].¹⁵ Notice here that the creation time distributions are coupled to the species X moment equations through E[k₁₂(t − τ₁)]. When k₁₂(t − τ₁) is a constant, the creation time distributions uncouple from the species distributions and the moment equations for the chemical species reduce to those derived for the Markov CME. Results for the CTD-CME moment equations used as parameters in a Gaussian agree extremely well with the direct CTD-CME results for all the cases (Figure 2(c)).

B. CTD-CME results for the absorber system when the number of absorbers varies over time

Next, we allowed fluctuations in the number of X₂ molecules, i.e., β₂ ≠ 0 and γ₂ ≠ 0 and with β₂ = 1 × 10⁵. In this case, the steady-state average of X₂ molecules is ${\overline{x}}_2={\overline{N}}_{x_2}={\beta }_1/{\gamma }_2$. To be consistent with the constant absorber system cases, the mean value was set to be ${\overline{N}}_{x_2}=450$ (case 1, γ₂ = 2.22 × 10²), ${\overline{N}}_{x_2}=900$ (case 2,γ₂ = 1.11 × 10²), and ${\overline{N}}_{x_2}=50$ (case 3,γ₂ = 2 × 10³). As above, we keep ${\beta }_1/{\overline{N}}_{x_2}$ constant to test deviation from the memoryless (exponential) assumption as these parameters change. The bimolecular rate function from Figure 2(a) is used for the absorber diffusion reaction. Because both X₁ and X₂ molecules are being created over time, the bimolecular rate function takes the standard form $k_{12}(t-\max ({\tau }_i^1,{\tau }_j^2))$. For the CTD-SSA simulations (Figure 2(d)), case 1 and case 2 for the variable absorber system are quite similar to that for the constant absorber system. In contrast, for case 3 in the variable absorber system, the mean of the distribution shifts below the means in cases 1 and 2. We will explain this peculiar shift from the CTD-moment results below. For the CTD-CME, we model this system with the iid approximation and project the creation time distribution onto the ${\tau }_i^1$ and ${\tau }_j^2$ creation time variables and for simpler notation set ${\tau }_i^1={\tau }_1$ and ${\tau }_j^2={\tau }_2$. The joint distribution then has the form p(x₁, x₂, t)p₁(τ₁|x₁, x₂, t)p₂(τ₂|x₁, x₂, t). We derived both the CTD-CME and the CTD-moments for this system (supplementary material¹⁵). Because p(x₁, x₂, t) is 2D with respect to species, its direct solution is computationally inefficient. However, we efficiently solve for the CTD-moment equations (Figure 2(e)) and use them as parameters in a Gaussian which agree extremely well with the CTD-SSA results.

To understand the shift in case 3 between the constant and variable absorber case, we analyze the steady state creation time distributions for the cases 2 and 3. Figure 3(a) (constant absorber system) only plots p₁(τ, t) since the absorbers are constant, while Figure 3(b) (variable absorber system) plots p₁(τ, t) and p₂(τ, t). Here, τ is the dummy variable for the plots in Figure 3. For case 2, for both the constant and variable absorber system, p₁(τ, t), are very similar. The mean of p₁(τ, t), $t-{\overline{\tau }}_1$, for each system are therefore quantitatively similar as well. It is easy to see that for the variable absorber system, in case 2, the mean for p₂(τ, t), $t-{\overline{\tau }}_2$, is larger than $t-{\overline{\tau }}_1$ which implies that ${\overline{\tau }}_1>{\overline{\tau }}_2$. Therefore, the average rate function will approximately be E[k₁₂(t − max (τ₁, τ₂)) ≈ E[k₁₂(t − τ₁)], dominated by the creation times of the X₁ molecules. Hence, for case 2, there is agreement between the two systems. Applying the same arguments for case 3, in the variable absorber system one can see that ${\overline{\tau }}_2\gg {\overline{\tau }}_1$. Therefore, the average rate function will approximately be E[k₁₂(t − max (τ₁, τ₂))] ≈ E[k₁₂(t − τ₂)], dominated by the creation times of the X₂ absorbing molecules. Therefore, the two systems for case 3 will behave differently, predicting a substantially lower mean for the variable absorber case as observed.

FIG. 3.

(a) Creation time distributions (scaled to the plot) overlaid onto the rate function curve for Cases 2 and 3 (constant absorber system) where green curves represent creation time distribution for species X₁. (b) Creation time distributions (scaled to the plot) overlaid onto the rate function curve for Cases 2 and 3 (variable absorber system) where green and black curves represent creation time distributions for species X₁ and X₂, respectively.

C. Application of CTD-CME to model systems with spatio-temporal correlations

We model a homodimerizing system with spatial-temporal correlated rebinding whose basic chemical equations are

$$\begin{array}{c}\hfill X_1+X_1\stackrel{k_{11}\left(t\right)}{\to }{X}_3,\end{array}$$ (27a)

$$\begin{array}{c}\hfill X_3\stackrel{{\gamma }_d}{\to }{X}_2,\end{array}$$ (27b)

$$\begin{array}{c}\hfill X_2\stackrel{k_2\left(t\right)}{\to }{X}_3,\end{array}$$ (27c)

$$\begin{array}{c}\hfill X_1+X_2\stackrel{k_{11}\left(t\right)}{\to }{X}_3+X_1^{*},\end{array}$$ (27d)

$$\begin{array}{c}\hfill X_2+X_2\stackrel{k_{11}\left(t\right)}{\to }{X}_3+X_1^{*}+X_1^{*}.\end{array}$$ (27e)

Here, X₃ represents a dimer complex. X₂ represents a dissociated spatially correlated monomer pair that can rebind. X₁ represents a spatially uncorrelated monomer. And $X_1^{*}$ represents a newly spatially uncorrelated monomer whose spatially correlated partner associated with another monomer. Furthermore, the $X_1^{*}$ creation time is preserved, i.e., the time when it first dissociated from the dimer complex. In essence, there is a spatially uncorrelated population and a spatially correlated population of monomers, each of which can dimerize with monomers in the other population as well as with monomers from their own population. The dissociation rate constant of the dimer complex is γ_d. The rate functions, k₁₁(t) and k₂(t) (Figures 4(a) and 4(b)), represent the spatially uncorrelated bimolecular rate function and spatially correlated rebinding rate function, respectively. These functions were derived for a particular homodimerization system discussed in Chevalier and El-Samad¹³ where results from the CTD-SSA were shown to agree well with brownian dynamics simulations. From Figure 4(b) one can see that upon dissociation, the dissociated molecules have a high rebinding rate (the two molecules are in close proximity at early times) and then rapidly decreases with time as the tendency to diffuse away from each other increases. Simulations for this system were run for the CTD-SSA. Figures 4(c) and 4(d) illustrate the difference when one accounts for rebinding using the spatially correlated rate function, k₂(t), as opposed to the well-mixed assumption where the rebinding rate function is modeled with the spatially uncorrelated rate function, k₁₁(t). The use of the spatially correlated rate function yields the correct distributions of the molecules while not accounting for rebinding (uncorrelated assumption) results in pronounced errors.

FIG. 4.

(a) Uncorrelated bimolecular rate function k₁₁(t − τ). (b) Correlated rebinding rate function k₂(t − τ). (c)–(f) Distribution at steady state of dimer numbers for a set number of total receptors N_tot and a given value of γ_d. (c) Dashed (CTD-SSA/correlated re-association), dotted (CTD-SSA/uncorrelated case) for γ_d = 1000. (d) Dashed (CTD-SSA/correlated re-association), dotted (CTD-SSA/uncorrelated case) for γ_d = 100. (e) Dashed (CTD-moments/correlated re-association), dotted (CTD-moments/uncorrelated case) γ_d = 1000. (f) Dashed (CTD-moments/correlated re-association), dotted (CTD-moments/uncorrelated case) γ_d = 100.

We then generated CTD-moment equations for the system. Figures 4(e) and 4(f) show Gaussians based on the steady state moments of the CTD-CME for the same parameters used in the CTD-SSA simulations. There is good agreement, but with a small error incurred by the CTD-moment solutions for the spatially correlated system. For the N_tot = 1500, γ_d = 1000 case, the error is most easily noticeable, and we suspect it is due to assuming all creation time distributions to be exponential, an assumption that might not be accurate for the correlated CTD species. Future work will investigate different representations of creation time distributions that can accommodate for the subtleties of spatial-temporal correlations.

VI. SUMMARY AND FUTURE WORK

In this work, we have extended the CTD-SSA approach to include the CTD-CME. The new CTD-CME/CTD-SSA represents a general framework for systems with CTD propensities. This new framework forms a more general approach than the Markov CME and expands upon the realm of possible stochastic biochemical systems that can be modeled, including systems with spatial-temporal correlations.

To derive moment equations, we approximated the creation time distribution to assume an iid, exponential form. Our calculation of the normalized residual error (NRE) (Appendix B) suggests the steady solution is very close to iid for the systems modeled. In the future, to capture more accurate realizations of the CTD-CME and the CTD-moments, we need to better approximate the creation time distributions, especially when the system undergoes time-dynamic changes. One potential avenue to pursue is DG methods which constitutes a finite, discontinuous basis function approach for solving hyperbolic and parabolic partial differential equations.¹⁶ A successful approach will also allow the iid approximation to be tested during dynamical changes in the system.

It is possible to map time-dependent propensities as Markov chains into Markov systems.^21,22 It would be interesting to map our systems to a Markov chain and generate time-dynamic moment equations to compare with the time-dynamic CTD-moment equations. This is a topic of future research.

Our original work on the CTD-SSA provided a new means for simulating reactions that have CTD propensities, such as diffusion reactions on membranes. As with the SSA, when the population of a CTD species becomes large, the algorithm spends the majority of its time on CTD-dependent reactions, such as CTD diffusion reactions, thereby increasing the computational cost. To circumvent this, future work will exploit the CTD-moment equations derived to develop faster approximate Monte Carlo methods that speed up the CTD-SSA incorporating ideas from techniques like tau-leaping/Langevin^23,24 and other hybrid approaches.²⁵ In summary, a wide-ranging battery of methods, similar to that developed for the Markov CME/SSA framework, is necessary for the application of the CTD-CME/CTD-SSA formalism to efficiently model and analyze stochastic biochemical systems with CTD propensities.

APPENDIX A: GENERAL METHOD FOR CONSTRUCTING CTD-MOMENT EQUATIONS

In the master equation with expected propensity values (22), we transform $\mathrm{E}\left[a_k(x-{\nu }_k,t,{\tilde{\tau }}^m,{\tilde{\tau }}^n)\right]$ and E[a_k(x, t, τ^m, τⁿ)] into equivalent, but simpler forms that are amenable to moment equation generation. For instance, take the expected propensity value for the hetero-bimolecular reaction, $E\left[a_k(x,t,{\tau }^m,{\tau }^n)\right]={\sum }_{i=1}^{x_m}{\sum }_{j=1}^{x_n}E\left[k_{mn}(t-\max ({\tau }_i^m,{\tau }_j^n))\right|x]$. Here, $E\left[k_{mn}(t-\max ({\tau }_i^m,{\tau }_j^n))\right|x]$ is the expected value of the bimolecular rate function for the ith species m molecule and jth species n molecule evaluated at species state x and at time t. Because of S-symmetry, it is easy to see that $E\left[k_{mn}(t-\max ({\tau }_i^m,{\tau }_j^n))\right|x]$ is the same for all i and j. We can therefore express the propensity as $E\left[a_k(x,t,{\tau }^m,{\tau }^n)\right]=E\left[k_{mn}(t-\max ({\tau }_1^m,{\tau }_2^n))\right|x]x_m{x}_n$. The same logic holds for the homo-bimolecular reaction as well. More generally, we can express E[a_k(x, t, τ^m, τⁿ)] as $E\left[k_k(t,{\tau }_1^m,{\tau }_2^n)\right|x]f_k\left(x\right)$ where $E\left[k_k(t,{\tau }_1^m,{\tau }_2^n)\right|x]$ is the expected value of the rate function evaluated at state x and at time t, and f_k(x) is the state-dependent function, for example, x_mx_n in the hetero-bimolecular case. The master equation with expected propensity values (22) becomes

$$\begin{array}{ccc}& & \frac{\partial p(x,t)}{\partial t}\hfill \\ & & =\sum _{k=1}^W[p(x-{\nu }_k,t)E\left[k_k\left(t,{\tilde{\tau }}_1^m,{\tilde{\tau }}_2^n\right)|x-{\nu }_k\right]f_k(x-{\nu }_k)\hfill \\ & & -p(x,t)E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|x\right]f_k\left(x\right)].\hfill \end{array}$$ (A1)

We will generate moment equations assuming local linearity in the propensities. To do so we first represent $E\left[k_k(t,{\tau }_1^m,{\tau }_2^n)\right|x]$ and f_k(x) as linear Taylor series about the mean z(t) of the distribution p(x, t). This yields the linearized propensities

$$\begin{array}{ccc}& & E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|x\right]f_k\left(x\right)\hfill \\ & & \approx [E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|z\left(t\right)\right]\hfill \\ & & +\sum _{e^{\prime }=1}^N\frac{\partial E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|x\right]}{\partial x_{e^{\prime }}}{|}_{x=z\left(t\right)}[x_{e^{\prime }}-z_{e^{\prime }}\left(t\right)]]\hfill \\ & & \times \left[f_k\left(z\left(t\right)\right)+\sum _{e=1}^N\frac{\partial f_k\left(x\right)}{\partial x_e}{|}_{x=z\left(t\right)}[x_e-z_e\left(t\right)]\right]\hfill \\ & & \approx E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|z\left(t\right)\right]f_k\left(z\left(t\right)\right)\hfill \\ & & +E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|z\left(t\right)\right]\sum _{e=1}^N\frac{\partial f_k\left(x\right)}{\partial x_e}{|}_{x=z\left(t\right)}[x_e-z_e\left(t\right)]\hfill \\ & & +f_k\left(z\left(t\right)\right)\sum _{e^{\prime }=1}^N\frac{\partial E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|x\right]}{\partial x_{e^{\prime }}}{|}_{x=z\left(t\right)}[x_{e^{\prime }}-z_{e^{\prime }}\left(t\right)]].\hfill \end{array}$$ (A2)

The final expression in (A2) keeps up to linear terms only which assumes that higher order terms are negligible in the volume of x where the distribution p(x, t) spans. Given this constraint, standard moment generation approaches derived for the markov/memoryless Master Equation²⁰ can be applied. The time-dependent mean equation can be expressed as

$$\begin{array}{c}\hfill \frac{\partial z_h\left(t\right)}{\partial t}=\sum _{k=1}^K{\nu }_{kh}E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|z\left(t\right)\right]f_k\left(z\left(t\right)\right),\end{array}$$ (A3)

where notice that all linear terms integrate out. We can also generate time-dependent covariance equations. To directly apply the work of Engblom,²⁰ we must make a further assumption in (A2), which is $E\left[k_k(t,{\tau }_1^m,{\tau }_2^n)\right|z\left(t\right)]\frac{\partial f_k\left(x\right)}{\partial x_e}{{|}_{x=z\left(t\right)}\gg f_k\left(z\left(t\right)\right)\frac{\partial E\left[k_k(t,{\tau }_1^m,{\tau }_2^n)\right|x]}{\partial x_{e^{\prime }}}|}_{x=z\left(t\right)}$ for e = e^′. This assumption removes the last term in (A2) and allows the time-dependent covariance equation to be expressed as

$$\begin{array}{ccc}& & \frac{\partial C_{h{h}^{\prime }}\left(t\right)}{\partial t}\hfill \\ & & =\sum _{k=1}^K\left({\nu }_{kh}\sum _e^{N}E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|z\left(t\right)\right]\frac{\partial f_k\left(x\right)}{\partial x_e}{|}_{x=z\left(t\right)}{C}_{e{h}^{\prime }}\right.\hfill \\ & & \left.+{\nu }_{k{h}^{\prime }}\sum _{e^{\prime }}^{N}E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|z\left(t\right)\right]\frac{\partial f_k\left(x\right)}{\partial x_{e^{\prime }}}{|}_{x=z\left(t\right)}{C}_{h{e}^{\prime }}\right)\hfill \\ & & +\sum _{k=1}^K{\nu }_{kh}{\nu }_{k{h}^{\prime }}E\left[k_k\left(t,{\tau }_1^m,{\tau }_2^n\right)|z\left(t\right)\right]f_k\left(z\left(t\right)\right).\hfill \end{array}$$ (A4)

The time-dependent mean and covariance equations are necessarily coupled to equation(s) that evolve the creation time distribution which enables the calculation of $E\left[k_k(t,{\tau }_1^m,{\tau }_2^n)\right|z\left(t\right)]$. Just as the moment equations are species state averaged equations, a species state averaged equation(s) for the creation time distribution is required in order to continuously calculate $E\left[k_k(t,{\tau }_1^m,{\tau }_2^n)\right|z\left(t\right)]$. The linear Taylor series approximation of $E\left[k_k(t,{\tau }_1^m,{\tau }_2^n)\right|x]$ that was applied to the linear propensity approximation in (A2) makes the assumption that the creation time distribution takes the form

$$\begin{array}{ccc}\hfill p({\tau }^1,...,{\tau }^U|x,t)& \approx & p({\tau }^1,...,{\tau }^U|z\left(t\right),t)\hfill \\ & & +\sum _e^N{f}_{x_e}({\tau }^1,...,{\tau }^U|z\left(t\right),t)\frac{x_e-z_e\left(t\right)}{w_e\left(t\right)}\hfill \end{array}$$ (A5)

which is simply the creation time distribution at the mean z(t) plus a sum of species dependent basis functions, $f_{x_e}$, whose weight varies linearly away from the mean. Similar to the linear propensity approximation, we will keep up to first order terms only in the general equation(s) evolving the creation time distribution and then average over species state to yield the species state averaged equation(s). Because we are assuming that p(x, t) is approximately gaussian, an even-function about the mean, only the p(τ¹, …, τ^U|z(t), t) term will remain in a given time-dependent creation-time-distribution equation, i.e., all the $f_{x_e}$ basis function terms will integrate out. All the terms, due to specific reactions from the systems in this paper, which appear in the species state averaged equation(s) evolving the time-dependent creation time distribution are presented in the supplementary material.¹⁵

APPENDIX B: NORMALIZED RESIDUAL ERROR IN THE CREATION TIME DISTRIBUTION VARIANCE AND COVARIANCE EQUATIONS

In the supplementary material,¹⁵ we derive the terms for the mean, variance, and covariance equations for the creation time distributions due to all reactions applied to the systems in this paper. Given our iid, exponential distribution assumptions to solve the systems, we can take the resulting solution and plug it into the exact variance and covariance equations to look at NRE a dimensionless quantity. To get the dimensionless NRE for the variance equations, the equations are normalized by ${(t-{\tau }_i^n\left(t\right))}^2{\sum }_{k}E\left[a_k(x,t,{\tau }^m,{\tau }^n)\right]/z_n\left(t\right)$, where ${(t-{\tau }_i^n\left(t\right))}^2$ is variance of the exponential distribution. This gives a NRE relative to the exponential assumption. Similarly for the covariance equations, the equations are normalized by $[t-{\tau }_i^m\left(t\right)][t-{\tau }_j^n\left(t\right)]{\sum }^{k}E\left[a_k(x,t,{\tau }^m,{\tau }^n)\right]/\sqrt{(}{z}_m\left(t\right)z_n\left(t\right))$ where $[t-{\tau }_i^m\left(t\right)][t-{\tau }_j^n\left(t\right)]$ is the multiplied standard deviations assuming exponential distributions for CTD species m and n. This yields a dimensionless NRE for the covariance equations relative to the exponential assumption and can be interpreted as a pearson-like correlation coefficient. If the solution is truly iid and exponential, then the NRE should be zero for both the variance and covariance equations.

We ran the solutions for all systems and cases within the system. For the constant absorber case, given that the rate function is $k_{12}(t-{\tau }_i^1)$, one can show that the covariance equations result in zero normalized residual error. Hence, the system is always iid. The maximum NRE for the variance equations was 0.036 (Case 2), thus the exponential assumption is very reasonable.

For the variable absorber case, given the bimolecular rate function $k_{12}(t-\max ({\tau }_i^1,{\tau }_j^2))$, one can show that this causes the system to be non-iid, i.e., non-zero NRE. For the covariance equations, the max NRE is 1.1 × 10⁻⁵, suggesting that it is very close to iid for all cases. And max variance NRE is 0.0276 (Case 2). Again, the exponential distribution assumption is reasonable.

Finally, for the correlated homodimer systems in all cases the covariance equations yielded a maximum NRE of 1.9 × 10⁻⁴, very close to iid. However, for the correlated homodimer system the maximum variance NRE was 0.141 (the N_tot = 1500, γ_d = 1000 case), at least four times higher than the max NRE of the variable and constant absorber systems.

For the steady state systems presented, the NRE results provide support for the iid assumption. For the exponential creation distribution assumption, the correlated homo-dimer system (the N_tot = 1500, γ_d = 1000 case) has the most significant level of NRE in the variance equations which might explain the error between the distributions calculated by the CTD-SSA and the CTD-moments for this case.