Dichotomous noise models of gene switches

Abstract

Molecular noise in gene regulatory networks has two intrinsic components, one part being due to fluctuations caused by the birth and death of protein or mRNA molecules which are often present in small numbers and the other part arising from gene state switching, a single molecule event. Stochastic dynamics of gene regulatory circuits appears to be largely responsible for bifurcations into a set of multi-attractor states that encode different cell phenotypes. The interplay of dichotomous single molecule gene noise with the nonlinear architecture of genetic networks generates rich and complex phenomena. In this paper, we elaborate on an approximate framework that leads to simple hybrid multi-scale schemes well suited for the quantitative exploration of the steady state properties of large-scale cellular genetic circuits. Through a path sum based analysis of trajectory statistics, we elucidate the connection of these hybrid schemes to the underlying master equation and provide a rigorous justification for using dichotomous noise based models to study genetic networks. Numerical simulations of circuit models reveal that the contribution of the genetic noise of single molecule origin to the total noise is significant for a wide range of kinetic regimes.

I. INTRODUCTION

Molecular noise functions as a driving force for phenotypic heterogeneity in cell populations.^1–4 With the growth of sophisticated imaging methods and biochemical techniques that probe gene expression at a single cell level, the omnipresence of molecular noise in cellular processes has become widely appreciated.^1,5 Noise in biology therefore should no longer be viewed as a nuisance with which cells must cope but rather it must be viewed as a natural facilitator for adaptation, growth, and development.^2,6–8 Molecular noise in genetic networks has at least two sources:^9,10 one source is the probabilistic nature of molecular associations such as the binding/unbinding of promoters to operator sites on the DNA while the other obvious source is randomness in the sequential reactive events such as the stochastic synthesis and degradation of mRNA, protein molecules, which generate the whole population of molecules. The dichotomous Markov noise (DMN) that arises from a single copy of a gene stochastically switching between its ON and OFF states, whether through promoter binding or through architectural changes of chromatin, is of a qualitatively different mathematical character from the other sources of molecular noise which may be treated as controllably small perturbations when the molecular populations of mRNA and proteins are sufficiently large.¹¹

Dichotomous gene noise leads to bursty intermittent production of mRNAs. The intermittency arises because mRNA molecules are rapidly synthesized in large groups when the gene becomes activated but these bursts are then followed by intervals of silence when the gene is turned off. “Poissonian” stochastic kinetics arises when the mRNAs are synthesized independently from each other at random intervals with uniform probability. In contrast, single cell experiments provide ample evidence that the statistics of gene expression is often non-Poissonian, with bursty behavior having been observed in both prokaryotic^12–15 and eukaryotic^16–20 cells. As a rule, the bursty stochastic kinetics leads to higher overall levels of noise. The detailed microscopic origins of dichotomous gene noise and the observed transcriptional and translational bursts still remain under debate.¹³ Possible mechanisms include local chromatin remodeling, thermally induced nucleosome turnover, covalent modifications, and genome wide chromatin fiber dynamics.²¹

Dichotomous noise generally breaks detailed balance in the circuits and thus creates non-equilibrium steady states which cannot always be described by quasi-equilibrium fluctuation statistics. Dichotomous noise driven phenomena include robust phase synchronization,^22,23 stochastic hypersensitivity,^24,25 enhanced stochastic resonance,²⁶ hysteresis,²⁷ and patterning.^28,29

Brute force simulation of the full master equation for genetic networks has already yielded many insights.^30,31 Nevertheless, in practice the sheer number of cellular components often renders such an explicit approach overwhelming. Most conventional approximations for treating stochastic chemical reactions, such as size expansion methods,^32,33 are not applicable when there is sufficient dichotomous noise since these approximations usually rely on the uniform scaling with size of the entire population of species to the near deterministic limit.³⁴ Time averaging can play a role, however, in reducing the influence of the single molecule dichotomous noise. In the limit of infinitely fast switching of the gene, one can again obtain the strictly deterministic result if the molecular populations of all the other components are sufficiently large. This fast gene switching regime has come to be known as the adiabatic limit.³⁵ In the adiabatic limit, there is a very large time scale separation between gene switching at the single molecule level and the biochemical synthesis and degradation reactions in the rest of the network which can then again be treated as nearly deterministic in the large system size limit. Thus, it is most important to develop approximations appropriate to the non-adiabatic case.

Recently, several schemes for modeling gene networks have been proposed which offer various ways of treating the stochastic aspects of gene switching.^36–43 Most of these schemes still, however, rely on there being a sufficient degree of time scale separation between gene and protein degrees of freedom, which means that switching dynamics must usually be assumed to be very fast. While such an assumption could be a sound one for many gene circuits, the intermediate regime where there is no such time scale separation is also of interest especially for understanding the behavior of mammalian gene circuits. In this contribution, we present a framework which lifts the requirement for the genes to operate near the adiabatic limit. This framework leads to a hybrid multi-scale scheme for numerically exploring potentially large scale genetic networks. We demonstrate the usefulness of our scheme by analyzing some well known models of simple genetic circuits.

II. RESULTS

A. Schematic discussion of trajectory statistics in gene networks

In this section, we discuss the notion of trajectory probabilities for stochastic gene circuits with dichotomous elements. This notion motivates hybrid simulation schemes based on piecewise deterministic systems of equations. We illustrate the ideas using a simple one component model of a self-regulating gene as the prototype for a gene network. Due to its simplicity, this idealized problem has been the subject of numerous theoretical studies.^11,44–49 The state of the circuit is defined by two dynamical variables: an operator state variable $s=\left\{0,1\right\}$ describing whether the gene is active s = 1 (ON) or inactive s = 0 (OFF) and the number of proteins $n=\left\{0,1,2,\dots ,\infty \right\}$. Since the gene circuit is stochastic, one is interested among other things in knowing the probability p_s(n, t) of having n proteins and the gene being in state s at time t. At this level, the most satisfying description of such a circuit (if one assumes a well stirred mixture of molecules) is given by a discrete space master equation,

$${\displaystyle \frac{\partial P(n,t)}{\partial t}=\mathbb{G}(P(n-1,t)-P(n,t))+\mathbb{K}((n+1)P(n+1,t)-nP(n,t))+{\mathbb{W}}_{n}P(n,t),}$$ (1)

where P(n, t) = |p₀(n, t), p₁(n, t)〉 is the state vector specifying the probabilities for each gene/protein state. The diagonal birth/death matrices 𝔾 = diag(g₀, g₁) and 𝕂 = diag(k, k) contain the rates of synthesis g and degradation k which are assumed to be Poisson processes, while 𝕎_n is the rate matrix that describes gene state switching probabilities as a function of protein numbers. In the case, when the populations of the species are not too small (n ≫ 1), such a discrete representation defined with integer population number accuracy (Eq. (1)) may no longer be absolutely necessary. When the populations of species in the genetic network are sufficiently large, one may instead treat n as a continuous variable and use diffusive dynamics for the evolution of protein population. This approximation is well justified by size expansion arguments. Numerical simulations⁵⁰ show that Fokker-Planck approximations can be quantitatively adequate even when protein numbers are as small as 100. In this limit, one may reduce the fully discrete master equation description to a master equation in mixed continuous and discrete variables $\frac{\partial P(n,t)}{\partial t}=-\nabla J+{\mathbb{W}}_{n}P(n,t)$, where the birth/death terms are replaced with $J=-A\left(n\right)-\frac{1}{2}{\partial }_{n}D\left(n\right)$ corresponding to an effective flux of continuous number of molecules. To visualize trajectories generated by such dynamics, consider a particular realization of the gene circuit’s history that goes through a particular sequence of gene state flips: σ = {s₀ → s₁ → s₂ → ⋯ → s_N}. These are governed by the master equation for the state variable conditioned at any time by the species population ${\dot{p}}_{s^{\prime }}(n,t^{\prime })=\mathbb{W}{p}_s(n,t)$. For the present model, this conditional master equation reads explicitly

$${\displaystyle \left(\begin{array}{c}\hfill {\dot{p}}_0(n,t)\hfill \\ \hfill {\dot{p}}_1(n,t)\hfill \end{array}\right)=\left(\begin{array}{cc}\hfill -k_{\mathit{off}}\hfill & \hfill k_{\mathit{on}}\left(n\right)\hfill \\ \hfill k_{\mathit{off}}\hfill & \hfill -k_{\mathit{on}}\left(n\right)\hfill \end{array}\right)\left(\begin{array}{c}\hfill p_0(n,t)\hfill \\ \hfill p_1(n,t)\hfill \end{array}\right).}$$ (2)

In between gene flipping events, the protein population will evolve diffusively ν = {n₀ → n₁ → n₂ → ⋯ → n_N} according to one of the following Fokker-Planck equations:

$${\displaystyle \begin{array}{rcr}\hfill {\partial }_t{p}_0(n,t)& \hfill =\hfill & \hfill [-{\partial }_n{A}_0\left(n\right)+\frac{1}{2}{\partial }_n^2{D}_0\left(n\right)]p_0(n,t),\\ \hfill {\partial }_t{p}_1(n,t)& \hfill =\hfill & \hfill [-{\partial }_n{A}_1\left(n\right)+\frac{1}{2}{\partial }_n^2{D}_1\left(n\right)]p_1(n,t),\end{array}}$$ (3)

where the A_s(n) and D_s(n) stand for the drift and the diffusion coefficients that correspond to the gene state s at any time. For the self-regulated gene model, these terms are A_s = g_s − kn and D_s(n) = g_s + kn. In this example, g_s is the protein synthesis rate when the gene is in state s while k is the rate coefficient corresponding to a first order model of protein degradation.^11,46 Equations (1) and (2) completely define the dynamics of the simplest gene circuit. System of equations (1) and (2) is known in mathematics as a coupled or switched diffusion process.^51–53 It features prominently in the problems of molecular motors, ratchets, and gene networks. The coupled diffusion process is analytically intractable for all but the simplest systems.

Now let us consider a particular realization of the stochastic trajectory of the gene state and then ask what is the probability given this gene state trajectory of observing a certain sequence of corresponding protein numbers (n₁, n₂, …n_N) at times (t₁, t₂, …, t_N). Most experimental studies measure time dependence of protein concentrations rather than follow the gene state changes explicitly. First let us consider a system with a fixed gene state (e.g., let us say s = 1 at all times). Using a convenient bracket notation, the conditional probability that protein state n_i+1 follows n_i at an earlier time can be expressed as $p(n_{i+1},t_{i+1}|n_i,t_i)=〈n_{i+1}\left|\hat{U}\left(\mathrm{\Delta }t\right)\right|n_i〉$ where $\hat{U}\left(\mathrm{\Delta }t\right)$ is the evolution operator of the Fokker-Planck equation corresponding to the particular fixed gene state. If we then allow the gene state to change in between gene switching events, the diffusion of protein numbers $n_0\stackrel{s_0}{⟶}{n}_1\stackrel{s_1}{⟶}\cdots$ will be governed by one of the Fokker Planck equations (Eq. (3)). The evolution operator ${\hat{U}}_s\left(\mathrm{\Delta }t\right)$ will therefore depend on the gene states s at each time t as indicated by its subscript. Thus, for any particular sequence of gene states σ, we have propagators $p_{s_i}\left(n_{i+1}{t}_{i+1}\right|n_i,t_i)=〈n_{i+1}\left|{\hat{U}}_{s_i}\right|n_i〉$ defined on the intervals between gene switching events. Similarly, for a particular realization of protein numbers ν at times t_i, we can write the probabilities of the gene switching, p_ni(s_i+1t_i+1|s_i, t_i) = 〈s_i+1|G_ni|s_i〉. Again this is defined on the intervals between protein number diffusion events. Suppose we are given a set of protein numbers observed at regular time intervals Δt, ν = {n₀, n₁, n₂, …, n_N} which could be obtained by recording a stochastic trajectory from a numerical simulation on a computer or by observing the gene expression profile in a single cell in the laboratory. Using the assumed Markovian property of the underlying molecular stochastic processes, we can write the probability of observing a trajectory ν in terms of the protein states irrespective of the gene trajectory as the product of transition probabilities between protein and gene states, which is then summed over all possible realizations of the gene state trajectory σ = {s₀, s₁, s₂, …, s_N},

$${\displaystyle p(n,n_{i-1},\dots ,n_0)=\sum _{\sigma }\prod _{i=0}^{N-1}〈s_{i+1}\left|G_{n_i}\right|s_i〉〈n_{i+1}\left|U_{s_i}\right|n_i〉p(s_0,n_0).}$$ (4)

The evolution operators for gene and protein states are G_ni = e^{−𝕎_niΔt_i} and $U_{s_i}=e^{-{\hat{L}}_{s_i}\mathrm{\Delta }{t}_i}$ with 𝕎_ni being the rate matrix for gene state transitions at fixed protein state n_i while ${\hat{L}}_{s_i}$ is the Fokker-Planck operator or protein diffusion corresponding to a fixed gene state s_i for times t_i. Explicitly, the short time protein state propagator can be approximated by an Onsager-Machlup like Lagrangian function,

$${\displaystyle 〈n_{i+1}\left|e^{-{\hat{L}}_{s_i}\mathrm{\Delta }{t}_i}\right|n_i〉=\mathcal{N}\left(\mathrm{\Delta }{t}_i\right)\exp [-\frac{1}{2D_{s_i}\left(n_i\right)}{(\frac{n_{i+1}-n_i}{\mathrm{\Delta }{t}_i}-A_{s_i}\left(n_i\right))}^2\mathrm{\Delta }{t}_i]d{n}_i.}$$ (5)

The term $\mathcal{N}\left(\mathrm{\Delta }{t}_i\right)$ takes care of normalization and we have omitted the Jacobian and gauge terms in the Lagrangian that come from the nonlinearity of variable transformations and the multiplicative nature of noise. These corrections are vanishingly small in the small noise limit which we are going to take later.^42,54 The terms A_si and D_si stand for drift and diffusion coefficients corresponding to gene state s_i at time t_i. We may formally take the limit of Δt_i → 0 and write the transition probabilities between protein numbers in a path integral form,

(6)

We consider the full trajectory probabilities rather than two time probabilities because after averaging over the gene states in Eq. (4) the reduced probability p(n, n_i−1, …, n₀) no longer obeys Markovian rules when the dichotomous noise has a finite correlation time. The eigenvalues λ_m(n_i) of the gene state change rate matrix 𝕎_ni are λ₀ = 0 which corresponds to a steady state and λ₁ = − (k_on + k_off), the next largest eigenvalue which sets the time scale ${\tau }_d={\lambda }_1^{-1}$ for gene state equilibration. The limit of τ_d → 0 corresponds to the adiabatic regime while τ_d → ∞ corresponds to the non-adiabatic regime of gene expression. To investigate how the time scale τ_d of the dichotomous noise affects the trajectory probabilities, we can perform a cumulant expansion of the exponential average (6),

(7)

On the left side of Eq. (7), the first term inside the exponent is the gene equilibrated action. The additional C_k terms are the higher order (k > 1) cumulants that account for the fluctuations from the gene averaged action. We can see that in the adiabatic limit of fast gene switching (τ_d → 0), the higher order cumulants will vanish because the state of the gene is instantaneously equilibrated at each time. In this regime, the trajectory probability p(n, n_i−1, …, n₀) can be approximated by the first cumulant term which corresponds to gene state averaged stochastic dynamics of protein numbers. Put another way, the gene states cease to be correlated at subsequent time intervals Δt, hence the probability p(n, n_i−1, …, n₀) acquires a Markovian property and is therefore reduced to an effective Fokker-Planck description (Fig. 1). The correlation function of the dichotomous gene variable⁵⁵ is given by $C_{\mathit{DMN}}\left(\mathrm{\Delta }t\right)=\frac{D}{{\tau }_d}\exp (-\frac{\mathrm{\Delta }t}{{\tau }_d})$, where ${\tau }_d=\frac{1}{k_{\mathit{on}}+k_{ff}}$ is the gene switching time scale and $D=k_{\mathit{on}}{k}_{\mathit{off}}{\tau }_d^3{(g_{\mathit{on}}-g_{\mathit{off}})}^2$ is the strength of the dichotomous noise. The explicit expression for D in the context of stochastic gene expression was earlier obtained by Ref. 46 where the term “churning noise” was coined, emphasizing that cycles in the space of gene state and protein number variables generated by alternating seemingly futile processes of binding/unbinding and protein production/degradation but that nevertheless these cycles lead to dispersion in the probability space. In the case of very slow gene switching (τ_d → ∞), the memory of a gene state is preserved for a long time, thus the trajectory probability can be expressed by a sum of two terms $〈e^{{\int }_{n_0\left(t_0\right)}^{n\left(t\right)}dt{\mathcal{L}}_s(\dot{n},n,t)}{〉}_{\sigma }\approx p_0{e}^{{\int }_{n_0\left(t_0\right)}^{n\left(t\right)}dt{\mathcal{L}}_0}+p_1{e}^{{\int }_{n_0\left(t_0\right)}^{n\left(t\right)}dt{\mathcal{L}}_1}$ corresponding to diffusive motion of protein numbers with the initial state of the gene being either OFF or ON with probabilities p₀ and p₁, respectively. In this extreme non-adiabatic regime, one can simply model the dynamics of the gene circuit via Fokker Planck equations for each gene state independently.

FIG. 1.

(A) Pictorial view of how dichotomous noise generates churning cycles in the regime with finite gene switching τ_d. (B) Top figure shows the dynamics with full noise (FN) generated by a master equation which accounts for diffusive motion of protein population and discrete jump transitions of the gene. In the bottom figure (DMN), the diffusive dynamics is replaced by deterministic equations of motion, while gene transitions are generated from the master equation while coupled to its deterministic counterpart.

In the intermediate regimes (with finite τ_d), however, a significant component of a gene circuit’s stochasticity comes from cyclic motion in the protein number space and one needs to consider contributions from trajectories that perform such driven cyclic drift (Fig. 1). The contribution of cyclic motion is contained in the higher order cumulants. We now outline a particularly simple recipe for numerically sampling stochastic trajectories in the intermediate regime of gene switching which accounts for the cyclic motion in the protein number space.

So far, other than the diffusive assumption for species represented by continuous population, no major approximations have been invoked in deriving the expression for the short time propagator for the coupled white-dichotomous noise dynamics of genetic circuit. We can further approximate gene path sum by taking the limit of small birth/death noise D(n) → 0, which removes the protein fluctuations while preserving only the cyclic component of motion (Fig. 1). In this limit, the path integrals for each realization of gene state trajectory σ in Eq. (6) are replaced with deterministic trajectories and the whole path sum is dominated by the stochastic trajectories which follow piecewise jump dynamics in the space of protein numbers,

$${\displaystyle \frac{dn}{dt}=A(s,n),}$$ (8a)

$${\displaystyle \frac{d{p}_n(s,t)}{dt}={\mathbb{W}}_n{p}_n(s,t).}$$ (8b)

Eq. (8a) which governs protein dynamics completely ignores fluctuations in protein number that arise from birth/death processes. One may account for birth/death noise by adding a fluctuating linear term correction as noise B_sη(t) that stems from the size expansion approximation³² to each deterministic mode A_s,

$${\displaystyle \frac{dn}{dt}=A(s,n)+B_s\eta \left(t\right),}$$ (9a)

$${\displaystyle \frac{d{p}_n(s,t)}{dt}={\mathbb{W}}_n{p}_n(s,t),}$$ (9b)

$${\displaystyle 〈\eta \left(t\right)\eta \left(t^{\prime }\right)〉=\delta (t-t^{\prime }).}$$ (9c)

System of equations (9) is one of the key results of this work focusing on gene networks. We note that piecewise deterministic equations and other hybrid equations similar to Eqs. (8) and (9) have already been used in the study of stochastic ion channels, neuronal dynamics, chemical reactions, and finance.^56,57 An interesting application of a hybrid scheme to model stochastic bursting effects in toggle switches can be found in the work by Bokes et al.⁵⁸ Particularly attractive hybrid schemes for studying general reaction systems can be found in the works of Salis-Kaznessis⁵⁹ and Haseltine-Rawlings.⁶⁰ In those schemes, the master equation of the entire system is partitioned into fast and slow reaction blocks which are respectively modeled with a jump and continuous state Markov processes. Moreover, the partitioning of the reactions is dynamically adjusted over the course of simulations in order to achieve high quantitative accuracy. These and many other hybrid schemes while differing in the algorithmic details have the same underlying theme of exploiting time scale separation present in reactions for constructing physically and numerically sound approximations. The distinguishing features of the dichotomous noise based models proposed in this work are the emphasis on conceptual simplicity and the gene centric treatment of reactions where dichotomous noise is used exclusively for reactions involving gene states regardless of the rates of other reactions and white noise is used to model fluctuations in the molecular populations of all other species. Thus, the proposed scheme is specifically aimed at genetic networks as a relatively coarse-grained (compared to kinetic Monte Carlo and other stochastic schemes) yet fairly simple explorative tool for studying steady state attractors and the associated intrinsic noise caused by the discrete dynamics of the promoter architecture (see the section below for the numerical algorithm). In the context of genetic networks, we would like to single out the related works by Ge et al.⁴³ who recently derived a fluctuation model for the self-regulating gene model to study the kinetics of barrier crossing in the intermediate regime of gene switching. Their fluctuation model is a particular application of Eq. (8). It is also worth mentioning the works by Karmakar-Bose⁶¹ and Zeiser et al.⁶² who derived closed analytic expressions for moments and steady state distributions for hybrid models of self-regulating gene.

In Secs. III A and III B, we will be referring to the sets of equations (8) and (9) as the DMN and the dichotomous + linear noise (DMN+LNA) approximations, respectively. Simulating DMN or DMN+LNA schemes can offer a computationally efficient alternative to direct kinetic Monte Carlo simulations of the full master equation. Finally, the noise obtained via either the DMN or the DMN+LNA scheme can be used in order to decompose the total noise into genetic and non-genetic contributions via ${\sigma }_{\mathit{tot}}^2={\sigma }_{\mathit{DMN}}^2+{\sigma }_{\mathit{BD}}^2$, where ${\sigma }_{\mathit{DMN}}^2$ is the variance that can be generated by the DMN scheme, ${\sigma }_{\mathit{tot}}^2$ is the variance computed at the level of master equation, and ${\sigma }_{\mathit{BD}}^2$ is the non-genetic noise coming from birth-death type of reactive events. One may also find it useful to further decompose the birth-death noise ${\sigma }_{\mathit{BD}}^2={\sigma }_{\mathit{DMN}+\mathit{LNA}}^2+{\sigma }_{\mathit{corr}}^2$ into a Gaussian uncorrelated part ${\sigma }_{\mathit{DMN}+\mathit{LNA}}^2$ and higher orders of noise ${\sigma }_{\mathit{corr}}^2$.

III. HYBRID MULTI-SCALE STOCHASTIC SIMULATION OF GENETIC CIRCUITS

The set of equations (8) and (9) offer a multi-scale scheme for simulating genetic networks, where one accounts for the dichotomous gene noise at the master equation level, while generating the birth-death noise either via deterministic dynamics in between stochastic gene state changes (DMN) or via a linear noise level of approximation (DMN+LNA). There are many ways of simulating a set of piece-wise deterministic equations and here we use a method similar to the one proposed in the work of Alfonsi et al.⁶³ The state of a gene circuit (n, s) is represented by vectors of continuous (n) and discrete species (s). Since the deterministic protein evolution is coupled to stochastic gene switching, the rate coefficients ($k_{ij}^s\left(n\left(t\right)\right)$) of stochastic switching (s_i → s_j) become time dependent and one cannot use a conventional Gillespie type Monte Carlo algorithm. The numerical procedure for simulating hybrid stochastic schemes (DMN/DMN+LNA) outlined in Section II consists of the following steps. First, one specifies the initial conditions for protein and gene states (n₀, s₀) for t = 0. Then, the dynamics is evolved by solving either an ODE (Eq. (8a)) or SDE (Eq. (9a)) set with a reasonably small time step Δt which needs to be comparable or smaller than the typical time for gene switching τ_d = (k_on + k_off)⁻¹. All the simulations in the paper are carried out with Δt = 0.1τ_d. Given the initial conditions and the specified time step, the trajectories are generated by numerically integrating the ODEs or SDEs for each time t_i = i∗Δt_i with i = [0, 1, 2, …]. The numerical integration continues until the condition for the stochastic gene switching (s_i → s_j) is reached. The switching condition is obtained by noting that the waiting time probability distribution for stochastic switching is given via $P\left(\tau \right)=\exp (-{\int }_t^{t+\tau }\alpha (n,t)dt)$ where $\alpha =\sum _{j\left(\ne i\right)}{k}_{ij}^s$ is the escape rate from the state (n_i, s_i) and τ is the elapsed time after which stochastic event occurs. One can sample the stochastic transition events by generating a uniform random number r₁ on the interval [0, 1] and setting the condition for the stochastic switching as $r_1=\exp (-{\int }_t^{t+\tau }\alpha (n,t)dt)$. If one has multiple genes in the system, the criteria for choosing which one happens at the switching time is determined by drawing a second random uniform number r₂ and testing for the condition $\sum _{j=1}^i\underset{ij}{\overset{s}{k}}\le \alpha \cdot r_2<\sum _{j=1}^{i+1}\underset{ij}{\overset{s}{k}}$ which determines the s_i → s_j gene switching event.

A. Self-regulating gene

In this section, we study the simplest model of the self-regulating gene in order to explore and understand the ways white and dichotomous noise sources contribute to the overall noise level and also to test the quality of DMN and DMN+LNA approximations. The self-regulating gene circuit involves the following reactions:

$${\displaystyle \mathit{ON}\stackrel{k_{\mathit{off}}}{⟶}\mathit{OFF}+P,}$$ (10a)

$${\displaystyle \mathit{OFF}+P\stackrel{k_{\mathit{on}}}{⟶}\mathit{ON},}$$ (10b)

$${\displaystyle P\stackrel{k}{⟶}0̸,}$$ (10c)

$${\displaystyle 0̸\stackrel{g_{\mathit{on}}}{⟶}P,}$$ (10d)

$${\displaystyle 0̸\stackrel{g_{\mathit{off}}}{⟶}P.}$$ (10e)

The first two reactions represent the gene switching processes which are responsible for the dichotomous noise, while the other reactions correspond to birth and death events. The deterministic model for birth/death dynamics is $\dot{n}=g\left(s\right)-kn$, where s stands for the gene state. The deterministic steady state attractors lead to differing protein numbers $n=\frac{g\left(s\right)}{k}$ for each gene state s. The linear noise approximation around each attractor is $dn=f(s,n)dt+\frac{g\left(s\right)}{k}dW$ where dW stands for the Wiener process. Varying the off rate k_off while maintaining the relationship k_on = 0.1k_off changes the adiabaticity with τ_d of the circuit smoothly going from the adiabatic to the non-adiabatic regime while not affecting the “equilibrium” properties of the binding process. To quantify the protein noise level, we compute the Fano factor σ²/μ. The Fano factor quantifies the deviation from Poissonian stochastic kinetics which corresponds to having a steady state distribution of protein numbers with σ²/μ = 1. The computed Fano factors shown in Fig. 2 indicate that both the DMN and the DMN+LNA schemes do quite well in capturing the order of magnitude of the noise in both the adiabatic and non-adiabatic limits. This is quite remarkable, since the noise in DMN comes entirely from the local steady nonfluctuating states. This level of approximation does not account for the smallness of protein populations. Nevertheless, the approximation provides a good estimation of the observable Fano factors. The noise decomposition scheme outlined at the end of Section II shows how noise from protein fluctuations caused by pure birth/death events compares to the total noise produced by the gene circuit as a whole: ${\sigma }_{\mathit{BD}}^2/{\sigma }_{\mathit{tot}}^2\sim 16\%$ in the non-adiabatic regime (k_off = 0.1 s⁻¹) and only ${\sigma }_{\mathit{BD}}^2/{\sigma }_{\mathit{tot}}^2\sim 70\%$ even in the adiabatic regime (k_off = 10 s⁻¹) for the present model circuit. Birth/death fluctuations become more important as the circuit approaches the adiabatic regime. Nevertheless, we see that the contribution of dichotomous noise remains significant throughout this wide range of kinetic regimes.

FIG. 2.

Dependence of Fano factor on the unbinding rate of promoter (k_off) computed with dichotomous noise (DMN), DMN with linear noise (DMN+LNA) simulations, and full noise simulations (FN) obtained by solving master equation. Inset shows the schematic topology of the model of self-regulated gene. Sample trajectories are shown on the left side. The values of the remaining rate coefficients are g_on = 25, g_off = 60, and k = 1.

Inspection of the detailed features of steady state probability distributions on the other hand reveals that pure dichotomous noise captures the skeleton of the distribution (Fig. 3(a)), properly predicting whether the system’s distribution is bimodal or unimodal along with even the relative weight of attractor states. Nevertheless, birth/death induced fluctuations in each steady state attractor are needed to give the distribution more reasonable widths. The peak heights are overestimated in the DMN approximation since the system does not fluctuate much in protein number around the steady state attractors (g₀/k and g₁/k). The DMN+LNA scheme on the other hand does a much better job at reproducing the distribution with higher accuracy. In the adiabatic limit both the DMN and the DMN+LNA schemes do quite well, although pure dichotomous noise alone naturally leads to more narrowly peaked distributions. A more stringent comparison between probability distributions is provided by the Kullback-Leibler (KL) divergence $D_{\mathit{KL}}\left(p\right|q)=\sum _{x}p\left(x\right)\log \frac{p\left(x\right)}{q\left(x\right)}$, also known as the relative entropy in information theory.⁶⁴ D_KL is a measure of statistical distinguishability between two probability distributions p(x) and q(x). It is a non-symmetric and positive function which vanishes only when the distributions are identical p(x) = q(x). When p(x) is the exact distribution directly obtained from the master equation and q(x) is the approximate estimation via DMN+LNA or DMN schemes, D_KL(p|q) quantifies the amount of information in bits (Fig. 4) which is irrevocably lost due to approximations. From the comparison (Fig. 4), we see that DMN+LNA is a better approximation in the intermediate range of gene switching and becomes worse in the very extreme limits. Thus, the intermediate regime of gene switching is where such schemes are expected to shine. Many biochemical genetic circuits are expected to operate in this regime.

FIG. 3.

Comparison of probability distributions obtained with full master equation (red lines) with those of (A) DMN and (B) DMN+LNA schemes for different values of adiabaticity parameter k_off.

FIG. 4.

The Kullback-Leibler divergence D_KL(p|q) between exact distribution p and approximate estimations q for steady state probabilities obtained via (A) DMN and (B) DMN+LNA simulations plotted against the adiabaticity parameter k_off.

Overall, these results are encouraging in that they show that one can obtain the steady state landscapes of large scale gene networks by only including dichotomous gene noise while modeling birth death events with the simple additive white noise. Simulating every reaction at a single molecule level is not only computationally prohibitive but probably also unnecessary for most cases. Purely deterministic models by themselves however are too crude for modeling gene networks. For instance, the macroscopic model of a self-regulating gene with monomeric binding does not predict the possibility of bistable behavior in the non-adiabatic regime,^45,65 hence including gene states explicitly is crucial for exploring steady states of genetic circuits.

B. Extended toggle switch model and the importance of extinction states

In this section, we consider a more complex and realistic circuit, the toggle switch. Toggle switches are ubiquitous bistable control modules. They have been implicated in regulating major cell fate decisions such as differentiation, cell death, and development. Some well known examples of genetic toggle switches include λ phage’s lysis/lysogeny switch, cell-cycle control circuits, signal transduction, and stem cell differentiation pathways. Here, we will look closely at an extended toggle switch circuit model proposed by Strasser,^66,67 which displays interesting multi-attractor dynamics. The circuit of the extended toggle switch consists of two genes A and B. It is based on a two state gene expression model with explicit transcription (producing mRNAs M_A and M_B) and translation (producing P_A and P_B) steps. The proteins P_A and P_B are mutually inhibiting which gives rise to a bistable behavior. The reaction scheme of the extended toggle switch circuit contains the following elementary steps:

$${\displaystyle {\mathit{ON}}_A\stackrel{k_{{\mathit{off}}_A}}{⟶}{\mathit{OFF}}_A+P_B,}$$ (11a)

$${\displaystyle {\mathit{OFF}}_A+P_B\stackrel{{\mathit{kon}}_A}{⟶}{\mathit{ON}}_A,}$$ (11b)

$${\displaystyle {\mathit{ON}}_B\stackrel{k_{{\mathit{off}}_B}}{⟶}{\mathit{OFF}}_B+P_A,}$$ (11c)

$${\displaystyle {\mathit{OFF}}_B+P_A\stackrel{{\mathit{kon}}_B}{⟶}{\mathit{ON}}_B,}$$ (11d)

$${\displaystyle {\mathit{OFF}}_A\stackrel{{\beta }_A}{⟶}{\mathit{OFF}}_A+{\mathit{mRNA}}_A,}$$ (11e)

$${\displaystyle {\mathit{OFF}}_B\stackrel{{\beta }_B}{⟶}{\mathit{OFF}}_B+{\mathit{mRNA}}_B,}$$ (11f)

$${\displaystyle {\mathit{mRNA}}_A\stackrel{g_A}{⟶}{\mathit{mRNA}}_A+P_A,}$$ (11g)

$${\displaystyle {\mathit{mRNA}}_B\stackrel{g_B}{⟶}{\mathit{mRNA}}_B+P_B,}$$ (11h)

$${\displaystyle {\mathit{mRNA}}_A\stackrel{{\gamma }_A}{⟶}0̸,}$$ (11i)

$${\displaystyle {\mathit{mRNA}}_B\stackrel{{\gamma }_B}{⟶}0̸,}$$ (11j)

$${\displaystyle P\stackrel{{\delta }_A}{⟶}0̸,}$$ (11k)

$${\displaystyle P\stackrel{{\delta }_B}{⟶}0̸.}$$ (11l)

Reactions (11(a)-11(d)) are gene switching events, ((11e) and (11f)) correspond to transcription process, ((11g) and (11h)) to translation process, and the remaining reactions describe the degradation of mRNA and protein molecules. The deterministic equations describing the birth-death processes are

$${\displaystyle {\dot{M}}_A=f_1(s_A,M_A)=s_A{\beta }_A-{\gamma }_A{M}_A,}$$ (12a)

$${\displaystyle {\dot{M}}_B=f_2(s_B,M_B)=s_B{\beta }_B-{\gamma }_B{M}_B,}$$ (12b)

$${\displaystyle {\dot{P}}_A=f_3(s_B,M_A,P_A)=g_A{M}_A-{\delta }_A{P}_A+k_{{\mathit{off}}_B}{s}_B-k_{{\mathit{on}}_B}(1-s_B)P_A,}$$ (12c)

$${\displaystyle {\dot{P}}_B=f_4(s_A,M_B,P_B)=g_B{M}_B-{\delta }_B{P}_B+k_{{\mathit{off}}_A}{s}_A-k_{{\mathit{on}}_A}(1-s_A)P_B.}$$ (12d)

We study the case of equal rate coefficients for genes A and B, making the toggle switch symmetric,

$${\displaystyle M^{ss}\left(s\right)=\frac{\beta \cdot s}{\gamma },}$$ (13a)

$${\displaystyle P^{ss}\left(s\right)=\frac{g{M}^{ss}\left(s^{\prime }\right)+k_{\mathit{off}}(1-s)}{\delta +k_{\mathit{on}}\cdot s}.}$$ (13b)

The variable s stands for the gene state of the first gene (0 for OFF and 1 for ON) and s′ denotes the state of the second gene. The DMN+LNA equations are

$${\displaystyle d{M}_A\left(t\right)=f_1(s_A,M_A)dt+s_A\sqrt{{\beta }_A}d{W}_1-\sqrt{{\gamma }_A\cdot M^{ss}\left(s_A\right)}d{W}_2,}$$ (14a)

$${\displaystyle d{M}_B\left(t\right)=f_2(s_B,M_B)dt+s_B\sqrt{{\beta }_B}d{W}_3-\sqrt{{\gamma }_B\cdot M^{ss}\left(s_B\right)}d{W}_4,}$$ (14b)

$${\displaystyle d{P}_A\left(t\right)=f_3(s_B,M_A,P_A)dt+\sqrt{g_A{M}^{ss}\left(s_A\right)}d{W}_5-\sqrt{{\delta }_A\cdot P^{ss}\left(s_B\right)}d{W}_6,}$$ (14c)

$${\displaystyle d{P}_B\left(t\right)=f_4(s_A,M_B,P_B)dt+\sqrt{g_B{M}^{ss}\left(s_B\right)}d{W}_7-\sqrt{{\delta }_B\cdot P^{ss}\left(s_A\right)}d{W}_8.}$$ (14d)

The stochastic transitions of the gene state s are again governed by a master equation. In the present model, the basal rate of protein production in the repressed bound state has been set to zero (g_off = 0), which creates strongly bistable behavior between extinction and steady production. The extinction attractors correspond to small or virtually non-existent mRNA and protein populations. Such extinction states are interesting to study with our methods both for understanding how different types of noise sources impact the stability of bistable switches and also for finding the true limitations of our approximation schemes. Both the DMN and DMN+LNA equations are based on deterministic evolution of molecular populations and they either completely neglect the birth/death fluctuations or treat them approximately. It is interesting to see how such schemes would behave in the regimes where the birth/death fluctuations are significant. By varying the mRNA degradation rate γ, we vary the average lifetime of mRNAs which controls the burst size of proteins. Thus, we go from the regime of high birth/death noise (small γ and ∼10 molecules) to low birth/death noise (large γ and ∼10² molecules). The dependence of the Fano factor on γ shows (Fig. 5) that indeed both the DMN and DMN+LNA schemes capture the total noise levels of the circuit quite well with the agreement improving steadily with the decreasing contribution of the birth/death noise. Detailed inspection of the probability distributions however reveals that the DMN+LNA scheme fails to capture metastable near-extinction states. To better visualize all of the steady state attractors, we have projected the 2D probability distribution on the order parameters q_A = M_A − M_B and q_B = P_A − P_B which emphasize the importance of states with M_A ≈ M_B ∼ 10 and P_A ≈ P_B ∼ 10, where both proteins and mRNAs are present in equally low quantities. From the probability distribution at high birth/death noise limit obtained from the simulation with full noise (see the top figure in Fig. 6(a)), we see that the toggle switch displays multi-attractor behavior with two main attractors corresponding to two configurations of the toggle switch with gene states s_A = ON(OFF) and s_B = OFF(ON) and the metastable attractor corresponding to both genes being simultaneously repressed (s_A = s_B = ON) which gives rise to a narrow peak in the center of 2D probability distribution.

FIG. 5.

Dependence of Fano factor on the mRNA degradation rate γ computed with full noise (FN) dichotomous noise (DMN) and DMN with linear noise correction (DMN+LNA). Inset shows the schematic topology of the model of extended toggle switch. Sample trajectories are shown on the left side. The values of the remaining rate coefficients are g_on = 0.05, g_off = 0, δ = 0.005, k_on = 5, k_off = 0.1, and β = 0.05 for both A and B genes.

FIG. 6.

Steady state probability distribution projected in the space of q_A = N(mRNA_A) − N(mRNA_B) and q_B = N(P_A) − N(P_B) order parameters reveals the multi-attractor nature of extended toggle switch. Comparing (A) full Noise with (B) DMN+LNA approximations for the different values of mRNA lifetime γ.

The steady state probability distributions obtained with the DMN+LNA scheme capture the main attractors in the system but do not reveal the presence of metastable states. This is because in the linear noise approximation of the birth death noise, the fluctuations of discrete species are treated as uncorrelated Brownian noise terms. Thus, the correlated fluctuations that are obtained by treating the noise at the full master equation level are essential to the existence of the extinction state attractors in this system. Such metastable attractors while conceptually interesting have a negligible contribution to the total noise level. They are only observed when the proteins and mRNAs are present in extremely low quantities on the order of ∼10. Once the numbers of molecules reach levels of ∼100, the DMN+LNA scheme provides an excellent approximation to the full master equation as can be seen from the comparison of distributions on the lower panel of Fig. 6. If one is interested in studying such metastable states with hybrid stochastic techniques, one possibility would be to treat the reactions involving extremely small number molecules into the space of discrete variables and treating them at the same master equation level as the gene states.

Finally, we study how the molecular population size affects the residence times of a gene being in the ON state which provides a measure of stability of an attractor. We vary the population size by changing the translation rate (g = g_A = g_B) (Fig. 7) for both genes. In the case of the full noise, the residence times grow linearly with the population size (number of proteins and mRNAs) which is in agreement with the previous studies on the extended toggle switch circuit.^66,67 Overall, due to the white noise contribution, the noise at the DMN+LNA level allows for more fluctuations in the gene state which in turn results in more transition events. This aspect allows faster exploration of gene circuit distribution, a feature that may be useful for quick scanning for the steady state attractors in large genetic networks. The residence times computed with the DMN scheme are obviously insensitive to the changes of protein numbers. This insensitivity is due to the deterministic treatment of these reactions involving species other than genes which do not affect the stochastic gene switching events.

FIG. 7.

Residence times in the ON state of a gene vs the protein population size of an extended toggle switch model.

IV. CONCLUSION

In this work, we have explored hybrid stochastic schemes based on treating the genetic noise of DNA binding explicitly while making linear noise approximations for the birth and death reactions of other species. The theoretical motivation for such schemes along with the key assumptions and connections of hybrid schemes with the underlying master equation is elucidated in a path sum based analysis of trajectory statistics. We have explored the steady state attractors of two classic toy models of gene circuits, the self-regulating gene, and an extended version of a genetic toggle switch. Our findings indicate that gene state change noise accounts for a large fraction of the total noise for a wide range of time scales. The noise decomposition obtained by comparing the total noise estimated by the different schemes for the extended model of toggle switch circuit revealed that the noise-induced metastable attractor originates from correlations of discrete molecular populations. Our study suggests that stochastic hybrid schemes will be useful tools for exploring large scale genetic networks, which display complex multi-attractor dynamics caused by discrete fluctuations in the promoter architecture and should find their place alongside more traditional full scale kinetic Monte Carlo simulations. For future work, it would be interesting to use such hybrid schemes to model the effects of gene co-localization,^68–70 heterogeneous distribution of transcription factors in nuclear/cytoplasmic milieu,⁷¹ and 1D/3D diffusional motions^72,73 by adding additional deterministic steps for each molecular component and enlarging the number of internal discrete promoter states. The latest studies have shown that in eukaryotic cells gene expression is an intricate function of chromatin configuration as well as spatial distribution of decoy binding sites on the genome.^71,74 Thus, it would be exciting to use hybrid schemes accounting for internal gene states with a high level of detail via a discrete web of Markov states to get a good understanding of some of these complex facets in eukaryotic gene regulation. At last, a possible extension to hybrid schemes with coupled diffusion introduced for selected species could also be an attractive way of treating in a coarse grained manner the cases where the well stirred assumption does not hold.