Modeling stochasticity and variability in gene regulatory networks

Abstract

Modeling stochasticity in gene regulatory networks is an important and complex problem in molecular systems biology. To elucidate intrinsic noise, several modeling strategies such as the Gillespie algorithm have been used successfully. This article contributes an approach as an alternative to these classical settings. Within the discrete paradigm, where genes, proteins, and other molecular components of gene regulatory networks are modeled as discrete variables and are assigned as logical rules describing their regulation through interactions with other components. Stochasticity is modeled at the biological function level under the assumption that even if the expression levels of the input nodes of an update rule guarantee activation or degradation there is a probability that the process will not occur due to stochastic effects. This approach allows a finer analysis of discrete models and provides a natural setup for cell population simulations to study cell-to-cell variability. We applied our methods to two of the most studied regulatory networks, the outcome of lambda phage infection of bacteria and the p53-mdm2 complex.

1 Introduction

Variability at the molecular level, defined as the phenotypic differences within a genetically identical population of cells exposed to the same environmental conditions, has been observed experimentally [1-4]. Understanding mechanisms that drive variability in molecular networks is an important goal of molecular systems biology, for which mathematical modeling can be very helpful. Different modeling strategies have been used for this purpose and, depending on the level of abstraction of the mathematical models, there are several ways to introduce stochasticity. Dynamic mathematical models can be broadly divided into two classes: continuous, such as systems of differential equations (and their stochastic variants) and discrete, such as Boolean networks and their generalizations (and their stochastic variants). This article will focus on stochasticity and discrete models.

Discrete models do not require detailed information about kinetic rate constants and they tend to be more intuitive. In turn, they only provide qualitative information about the system. The most general setting is as follows. Network nodes represent genes, proteins, and other molecular components of gene regulation, while network edges describe biological interactions among network nodes that are given as logical rules representing their interactions. Time in this framework is implicit and progresses in discrete steps. More formally, let x₁, ..., x_n be variables, which can take values in finite sets X₁, ..., X_n, respectively. Let X = X₁ × ⋯ × X_n be the Cartesian product. A discrete dynamical system (DDS) in the variables x₁, ..., x_n is a function

$$f=\left(f_1,\dots ,f_n\right):X\to X$$

where each coordinate function f_i: X → X_i is a function in a subset of {x₁, ..., x_n}. Dynamics is generated by iteration of f, and different update schemes can be used for this purpose. As an example, if X_i = {0, 1} for all i, then each f_i is a Boolean rule and f is a Boolean network where all the variables are updated simultaneously. We will assume that each X_i comes with a natural total ordering of its elements (corresponding to the concentration levels of the associated molecular species). Examples of this type of dynamical system representation are Boolean networks, logical models and Petri nets [5-7].

To account for stochasticity in this setting several methods have been considered. Probabilistic Boolean networks (PBNs) [8,9] introduce stochasticity in the update functions, allowing a different update function to be used at each iteration, chosen from a probability space of such functions for each network node. For other approaches, see [10-12]. These models will be discussed in more detail in the next section. In this article we present a model type related to PBNs, with additional features. We show that this model type is natural and a useful way to simulate gene regulation as a stochastic process, and is very useful to simulate experiments with cell populations.

1.1 Modeling stochasticity in gene regulatory networks

Gene regulation processes are inherently stochastic. Accurately modeling this stochasticity is a complex and important goal in molecular system biology. Depending on the level of knowledge of the biological system and the availability of data for it one could follow different approaches. For instance, viewing a gene regulatory network as a biochemical reaction network, the Gillespie algorithm can be applied to simulate each biochemical reaction separately generating a random walk corresponding to a solution of the chemical master equation of the system [13,14]. At an even more detailed level one could introduce time delays into the Gillespie simulations to account for realistic time delays in activation or degradation such as in circadian rhythms [15-17]. At a higher level of abstraction, stochastic differential equations [18] contain a deterministic approximation of the system and an additional random white noise term. However, all these schemes require that all the kinetic rate constants to be known which could represent a strong constraint due to the difficulty of measuring kinetic parameters, limiting these approaches to small systems.

As mentioned in the introduction, discrete models are an alternative to continuous models, which do not depend on rate constants. In this setting, several approaches to introduce stochasticity have been proposed. Specially for Boolean networks, stochasticity has been introduced by flipping node states from 0 to 1 or vice versa with some flip probability [12,19-21]. However, it has been argued that this way of introducing stochasticity into the system usually leads to over-representation of noise [11]. The main criticism of this approach is that it does not take into consideration the correlation between the expression values of input nodes and the probability of flipping the expression of a node due to noise. In fact, this approach models the stochasticity at a node regardless of the susceptibility to noise of the underlying biological function [11].

Probabilistic Boolean networks [8,9,22] is another stochastic method proposed within the discrete strategy. PBNs model the choice among alternate biological functions during the iteration process, rather than modeling the stochasticity of the function failure itself. We have adopted a special case of this setting, in which every node has associated to it two functions: the function that governs its evolution over time and the identity function. If the first is chosen, then the node is updated based on its logical rule. When the identity function is chosen, then the state of the node is not updated. The key difference to a PBN is the assignment of probabilities that govern which update is chosen. In our setting, each function gets assigned two probabilities. Precisely, let x_i be a variable. We assign to it a probability $p_i^{\uparrow }$, which determines the likelihood that x_i will be updated based on its logical rule, if this update leads to an increase/activation of the variable. Likewise, a probability $p_i^{\downarrow }$ determines this probability in case the variable is decreased/inhibited. The necessity for considering two different probabilities is that activation and degradation represent different biochemical processes and even if these two are encoded by the same function, their propensities in general are different. This is very similar to what is considered in differential equations modeling, where, for instance, the kinetic rate parameters for activation and for degradation/decay are, in principle, different.

Note that all these approaches only take account of intrinsic noise which is generated from small fluctuations in concentration levels, small number of reactant molecules, and fast and slow reactions. Another source of stochasticity is related to extrinsic noise such as a noisy cellular environment and temperature. For more about intrinsic vs extrinsic noise see [3,23].

2 Method

Our aim is to model stochasticity at the biological function level under the main assumption that even if the expression levels of the input nodes of an update function guarantee activation or degradation there is a probability that the process will not occur due to stochasticity, for instance, if some of the chemical reactions encoded by the update function may fail to occur. This is similar to models based on the chemical master equation. This model type introduces activation and degradation propensities. More formally, let x₁, ..., x_n be variables which can take values in finite sets X₁, ..., X_n, respectively. Let X = X₁ × ⋯ × X_n be the Cartesian product. Thus, the formal definition of a stochastic discrete dynamical system (SDDS) in the variables x₁, ..., x_n is a collection of n triplets

$$F={\left\{f_i,p_i^{\uparrow },p_i^{\downarrow }\right\}}_{i=1}^n$$

where

• f_i: X → X_i is the update function for x_i, for all i = 1, ..., n.

• $p_i^{\uparrow }$ is the activation propensity.

• $p_i^{\downarrow }$ is the degradation propensity.

• $p_i^{\uparrow }$, $p_i^{\downarrow }\in \left[0,1\right]$.

We now proceed to study the dynamics of such systems and two specific models as illustration.

2.1 Dynamics of SDDS

Let $F={\left\{f_i,p_i^{\uparrow },p_i^{\downarrow }\right\}}_{i=1}^n$ be a SDDS and consider x ∈ X. For all i we define π_{i, x}(x_i → f_i(x)) and π_{i, x}(x_i → x_i) by

$${\pi }_{i,x}\left(x_i\to f_i\left(x\right)\right)=\left\{\begin{array}{cc}{p}_i^{\uparrow },\hfill & \text{if}{x}_i<f_i\left(x\right),\hfill \\ p_i^{\downarrow },\hfill & \text{if}{x}_i>f_i\left(x\right),\hfill \\ 1,\hfill & \text{if}{x}_i=f_i\left(x\right).\hfill \\ \hfill \end{array}\right)$$

$${\pi }_{i,x}\left(x_i\to x_i\right)=\left\{\begin{array}{cc}1-p_i^{\uparrow },\hfill & \text{if}{x}_i<f_i\left(x\right),\hfill \\ 1-p_i^{\downarrow },\hfill & \text{if}{x}_i>f_i\left(x\right),\hfill \\ 1,\hfill & \text{if}{x}_i=f_i\left(x\right).\hfill \\ \hfill \end{array}\right)$$

That is, if the possible future value of the i-th coordinate is larger (smaller, resp.) than the current value, then the activation (degradation) propensity determines the probability that the i-th coordinate will increase (decrease) its current value. If the i-th coordinate and its possible future value are the same, then the i-th coordinate of the system will maintain its current value with probability 1. Notice that π_{i, x}(x_i → y_i) = 0 for all y_i ∉ {x_i, f_i(x)}.

The dynamics of F is given by the weighted graph X which has an edge from x ∈ X to y ∈ X if and only if y_i ∈ {x_i, f_i(x)} for all i. The weight of an edge x → y is equal to the product

$$w_{x\to y}=\prod _{i=1}^n{\pi }_{i,x}\left(x_i\to y_i\right)$$

By convention we omit edges with weight zero. See Additional file 1 for pseudocodes of algorithms to compute dynamics of SDDS. Software to test examples is available at http://dvd.vbi.vt.edu/adam.html[24] as a web tool (choose SDDS in the model type).

Given $F={\left\{f_i,p_i^{\uparrow },p_i^{\downarrow }\right\}}_{i=1}^n$ a SDDS, it is straightforward to verify that F has the same steady states (fixed points) as the deterministic system $G={\left\{f_i\right\}}_{i=1}^n$ (see Additional file 1). It is also important to note that the dynamics of F includes the different trajectories that can be generated from G using other common update mechanisms such as the synchronous and asynchronous schemes (see Additional file 1).

2.1.1 Example

Let n = 2, X = {0, 1} × {0, 1}, F = (f₁, f₂): X → X, where Table 4 represents the regulatory rules for x1 and x2 and Table 5 represents their propensity parameters

and

Pr(01 → 10) = (.1)(.9) = .09, Pr(01 → 00) = (1 - .1)(.9) = .81

Pr(01 → 01) = (1 - .1)(1 - .9) = .09, Pr(01 → 11) = (.1)(1 - .9) = .01

Pr(10 → 10) = (1 - .2)(1 - .5) = .4, Pr(10 → 01) = (.2)(.5) = .1

Pr(10 → 00) = (.2)(1 - .5) = .1, Pr(10 → 11) = (1 - .2)(.5) = .4

Pr(11 → 11) = (1)(1 - .9) = .1, Pr(11 → 10) = (1)(.9) = .9

Pr(00 → 00) = (1)(1) = 1.

Figure 1 shows that there is a 9% chance that the system will transition from 01 to 10. Similarly, there is an 81% chance that the system will transition from 01 to 00. The latter was expected because there is a high degradation propensity for f₂. Note that 00 is a fixed point, i.e., there is 100% chance of staying at this state.

Figure 1.

State Space Diagram. This diagram depicts all trajectories to follow from any given initial state of the network. The numbers next to the edges specify the transition probabilities. Note that dynamics here is not deterministic, most of the states have different trajectories to follow.

3 Applications

We illustrate the advantages of this model type by applying it to two widely studied biological systems, the regulation of the p53-mdm2 network and the control of the outcome of phage lambda infection of bacteria. These regulatory networks were selected because stochasticity plays a key role in their dynamics.

3.1 Regulation in the p53-Mdm2 network

The p53-Mdm2 network is one of the most widely studied gene regulatory networks. Abou-Jaude et al. [25] proposed a logical four-variable model to describe the dynamics of the tumor suppressor protein p53 and its negative regulator Mdm2 when DNA damage occurs. The wiring diagram of this model is represented in Figure 2, where P denotes cytoplasmic p53, nucleic p53, and the gene p53. Mc and Mn stand for cytoplasmic Mdm2 and nuclear Mdm2, respectively. DNA damage caused by ionic irradiation decreases the level of nucleic Mdm2 which enables p53 to accumulate and to remain active, playing a key role in reducing the effect of the damage. There is a negative feedback loop involving three components: p53 increases the level of cytoplasmic Mdm2 which, in turn, increases the level of nuclear Mdm2. Nucleic Mdm2 reduces p53 activity. This model also contains a positive feedback loop involving two components where p53 inhibits its negative regulator nucleic Mdm2. Note the dual role of P, as it positively regulates nucleic Mdm2 through cytoplasmic Mdm2. On the other hand, P negatively regulates nucleic Mdm2 by inhibiting Mdm2 nuclear translocation [25]. For more about the p53-Mdm2 system (see [4,25,26]).

Figure 2.

Four-variable model for the p53-Mdm2 regulatory network. P, Mc, and Mn stand for protein p53, cytoplasmic Mdm2, and nuclear Mdm2, respectively.

The dynamic behavior of the system is represented in a network of transitions called its state space (see Figure 3). This specifies the different paths to follow and the probabilities of following a specific trajectory from a given state. Dynamics here is not deterministic, i.e., most of the state vectors have different trajectories they can follow. The propensity parameters in Table 1 determine the likelihood of following certain paths. The state 0010 is a steady state, which is differentiated from the others by its oval shape.

Figure 3.

State space diagram for parameters described in Table 1. The numbers next to the edges encode the transition probabilities. The order of the variables in each vector state is P, Mc, Mn, DNA damage. Self-loops are not depicted. States with darker background comprise the cycle with DNA damage. A second cycle with a lighter shaded background corresponds to the cycle with no DNA damage. The oval shaped state is a steady state.

Table 1.

Propensity probabilities for the p53-Mdm2 regulatory network

	P	Mc	Mn	Dam
Activation	.9	.9	.9	1
Degradation	.9	.9	.9	.05

Note that there is a low degradation propensity for DNA damage.

The state space for this model is specified by [0, 2] × [0, 1] × [0, 1] × [0, 1], that is, except for the first variable P which has three levels {0, 1, 2}, all other variables are Boolean. The update functions for this model are provided in Additional file 1 and also in the model repository of our web tool at http://dvd.vbi.vt.edu/adam.html.

Individual cell simulations render plots similar to the ones shown in Figure 4. Each subfigure shows oscillations as long as the damage is present with a variability in the timing of damage repair. On the other hand, cell population simulations, Figure 5, exhibit damped oscillations of the expression level of p53 as the degradation propensities of the damage increases. This is correlated with the fact that, if the intensity of the damage is increased, more cells exhibit oscillations in the level of p53 which was experimentally observed in [4]. The initial state for all simulations was 0011 which represents the state when DNA damage is introduced (0010 is the steady state without perturbation).

Figure 4.

Individual cell simulations for parameters described in Table 1. Each subfigure shows oscillations as long as the damage is present. This figure shows variability in the timing of damage repair and in the period of the oscilations. Each frame was generated from a single simulation with sixty time steps. The x-axis represents discrete time steps and the y-axis the expression level. The initial state for all simulations is 0011.

Figure 5.

Cell population simulations. Each subfigure was generated from 100 simulations, each representing a single cell with sixty time steps. Starting from the top left frame to the right bottom frame the degradation propensity for DNA damage was increased by 5%, i.e. $p_{\text{dam}}^{\downarrow }=.05$ (top left), $p_{\text{dam}}^{\downarrow }=.10$ (top right), $p_{\text{dam}}^{\downarrow }=.15$ (bottom left), and $p_{\text{dam}}^{\downarrow }=.2$ (bottom right). The x-axis represents discrete time steps and the y-axis the average expression level. The initial state for all simulations was 0011. This figure shows that, if the intensity of the damage is increased more cells exhibit oscillations in the level of p53, in agreement with experimental observations [4].

Table 2.

Propensity parameters for Figure 7 (top frame)

	CI	CRO	CII	N
Activation	.8	.2	.9	.9
Degradation	.2	.8	.9	.9

There is a high activation propensity for CI while a low activation propensity for CRO.

Table 3.

Propensity parameters for Figure 7 (bottom frame)

	CI	CRO	CII	N
Activation	.3	.7	.9	.9
Degradation	.7	.3	.9	.9

There is a high activation propensity for CRO while a low activation propensity for CI.

To highlight the features of our approach we compare our model with the one presented in [25] in which variability has been analyzed. The main difference between these two models is in the way the simulations are performed. In [25], the transition from one state to the next is determined by parameters called "on" and "off" time delays. For instance, to transition from 2001 to 2101 it is required that $t_{Mc}<t_{\overline{\text{dam}}}$ which means that the "on" delay for Mc (time for activating) is less than the "off" delay (time for degrading) of the damage. Otherwise, if $t_{Mc}>t_{\overline{\text{dam}}}$ the system will transition from 2001 to 2000. In this article, transitions from one state to others are given as probabilities which are determined from the propensity probabilities. Therefore, the complexity of the model presented here is at the level of the wiring diagram (i.e. the number of variables) while the complexity of the model in [25] is at the level of the state space (i.e. number of possible states) which is exponential in the number of variables. Another key difference is the way DNA damage repair is modeled. In [25], a delay parameter $t_{\overline{\text{dam}}}$ is associated with the disappearance of the damage, and this is decreased by a certain amount τ at each iteration so that $t_{\overline{\text{dam}}}^{\left(n\right)}=t_{\overline{\text{dam}}}^{\left(0\right)}-n\tau \ge 0$ where n is the number of iterations. In order to simulate DNA damage with this approach it is required to estimate τ, n, and $t_{\overline{\text{dam}}}^{\left(0\right)}$. Within our model framework a single parameter, the degradation propensity, is used to model the damage repair which is a more natural setup.

3.2 Phage lambda infection of bacteria

Control of the outcome of phage lambda infection is one of the best understood regulatory systems [3,27,28]. Figure 6 depicts its core regulatory network that was first modeled by Thieffry and Thomas [28] using a logical approach. This model encompasses the roles of the regulatory genes CI, CRO, CII, and N. From experimental reports [3,28-30] it is known that, if the gene CI is fully expressed, all other genes are off. In the absence of CRO protein, CI is fully expressed (even in the absence of N and CII). CI is fully repressed provided that CRO is active and CII is absent.

Figure 6.

Wiring diagram for phage lambda infection model.

The dynamics of this network is a bistable switch between lysis and lysogeny, Figure 7. Lysis is the state where the phage will be replicated, killing the host. Otherwise, the network will transition to a state called lysogeny where the phage will incorporate its DNA into the bacterium and become dormant. It has been suggested [28,31] that these cell fate differences are due to spontaneous changes in the timing of individual biochemical reaction events.

Figure 7.

State space for phage lambda model. The order of variables in each vector state is CI, CRO, CII, N. The steady state 2000 represents lysogeny where CI is fully expressed while other genes are off. The cycle between 0200 and 0300 represents lysis where CRO is active and other genes are repressed. Self-loops are not depicted.

The state space for this model is specified by [0, 2] × [0, 3] × [0, 1] × [0, 1], that is, the first variable, CI, has three levels 0, 1, 2, the second variable, CRO, has four levels {0, 1, 2, 3}, and the third and fourth variables, CII and N, are Boolean. Update functions for this model are available in our supporting material, Additional file 1. This model has a steady state, 2000, and a 2-cycle involving 0200 and 0300. The steady state 2000 represents lysogeny where CI is fully expressed while the other genes are off. The cycle between 0200 and 0300 represents lysis where CRO is active and other genes are repressed.

Cell population simulations were performed to measure the cell-to-cell variability. Figure 8 was generated using the probabilities given in Tables 2 (top frame) and 3 (bottom frame). The x-axis in both subfigures represents discrete time steps while the y-axis captures the average expression level. The initial state for all simulations was 0000 which represents the state of the bacterium at the moment of phage infection. Figure 8 shows variability in developmental outcome, some of the networks transition to lysis while others transition to lysogeny. To measure how sensitive the dynamics of the network is to changes in the propensity probabilities, we have plotted the outcome of lysis-lysogeny percentages for different choices of these parameters. Figure 9 shows the variation in developmental outcome as a function of the propensity parameters of CI and CRO. Star points indicate the percentage of networks that transition to lysogeny and circle shaped points indicate the percentage of networks that end up in lysis. The bottom x-axis contains activation propensities for CI and degradation propensities for CRO while the top x-axis contains activation propensities for CRO and degradation propensities for CI. The activation and degradation propensities for CII and N were all set equal to .9. Although the probability distributions for CI and CRO are very symmetric in Figure 9, it gives a good idea of how the variability in developmental outcome will change as the propensity parameters change.

Figure 8.

Cell population simulations. Both figures were generated from 100 simulations, each representing a single cell iteration of ten time steps. Top frame for parameters in Table 2 shows 93% lysis and 7% lysogeny while bottom frame for parameters in Table 3 shows 4% lysis and 96% lysogeny. The x-axis represents discrete time steps while the y-axis shows the average expression level. The initial state for all the simulations is 0000. Solid (circle) points correspond to the average of CI (CRO), and dashed lines represent standard deviations.

Figure 9.

Variation in developmental outcome as a function of the propensity parameters. Star points indicate the percentage of networks that transition to lysogeny and circle shaped points indicate the percentage of networks that end up in lysis. Bottom axis represents the activation (and degradation) propensities for CI (CRO) in increasing order. Likewise, the top axis represents the activation (and degradation) propensities for CRO (CI) in decreasing order.

4 Conclusions

Using a discrete modeling strategy, this article introduces a framework to simulate stochasticity in gene regulatory networks at the function level, based on the general concept of PBNs. It accounts for intrinsic noise due to spontaneous differences in timing, small fluctuations in concentration levels, small numbers of reactant molecules, and fast and slow reactions. This framework was tested using two widely studied regulatory networks, the regulation of the p53-Mdm2 network and the control of phage lambda infection of bacteria. It is shown that in both of these examples the use of propensity probabilities for activation and degradation of network nodes provides a natural setup for cell population simulations to study cell-to-cell variability. The new features of this framework are the introduction of activation and degradation propensities that determine how fast or slow the discrete variables are being updated. This provides the ability to generate more realistic simulations of both single cell and cell population dynamics. In the example of the p53-Mdm2 system, one can see that individual simulations show sustained oscillations when DNA damage is present, while at the cell population level these individual oscillations average to a damped oscillation. This agrees with experimental observations [4]. In the second example, λ-phage infection of bacteria, it is observed that differences in developmental outcome due to intrinsic noise can be captured with this framework. Due to the lack of experimental data we are unable to calibrate the model so that it reproduces the correct difference in percentages due to intrinsic noise. So instead we present a plot of the difference in developmental outcome as a function of the propensity parameters.

It is worth noting that this article addresses only intrinsic noise generated from small fluctuations in concentration levels, small numbers of reactant molecules, and fast and slow reactions. Extrinsic noise is another source of stochasticity in gene regulation [3,23], and it would be interesting to see if this framework or a similar setup can be adapted to account for extrinsic stochasticity under the discrete approach. This framework also lends itself to the study of intrinsic noise and it is useful for the study of developmental robustness. For instance, one could ask what the effect of this type of noise is on the dynamics of networks controlled by biologically inspired functions.

Relating the propensity parameters to biologically meaningful information or having a systematic way for estimating them is very important. A preliminary analysis shows that it is possible to relate the propensity parameters in this framework with the propensity functions in the Gillespie algorithm under some conditions (see Additional file 1 where for a simple degradation model, the degradation propensity is correlated by a linear equation with the decay rate of the species being degraded). More precisely, in the Gillespie algorithm [13,14], if one discretizes the number of molecules of a chemical species into discrete expression levels such that within these levels the propensity functions for this species do not change significantly, then one obtains the setup of the framework presented here as a discrete model. That is, simulation within the framework presented here can be viewed as a further discretization of the Gillespie algorithm, in a setting that does not require exact knowledge of model parameters. For a similar approach see [10].

Competing interests

The authors declare that they have no competing interests.

Acknowledgements

DM and RL were partially supported by NSF grant CMMI-0908201. RL and DM thank Ilya Shmulevich for helpful suggestions. The authors thank the anonymous reviewers for many suggestions that improved the article.

Supplementary Material

Additional file 1

Additional File 1contains Supporting Material.