Analytical distributions for stochastic gene expression

Abstract

Gene expression is significantly stochastic making modeling of genetic networks challenging. We present an approximation that allows the calculation of not only the mean and variance, but also the distribution of protein numbers. We assume that proteins decay substantially more slowly than their mRNA and confirm that many genes satisfy this relation by using high-throughput data from budding yeast. For a two-stage model of gene expression, with transcription and translation as first-order reactions, we calculate the protein distribution for all times greater than several mRNA lifetimes and thus qualitatively predict the distribution of times for protein levels to first cross an arbitrary threshold. If in addition the fluctuates between inactive and active states, we can find the steady-state protein distribution, which can be bimodal if fluctuations of the promoter are slow. We show that our assumptions imply that protein synthesis occurs in geometrically distributed bursts and allows mRNA to be eliminated from a master equation description. In general, we find that protein distributions are asymmetric and may be poorly characterized by their mean and variance. Through maximum likelihood methods, our expressions should therefore allow more quantitative comparisons with experimental data. More generally, we introduce a technique to derive a simpler, effective dynamics for a stochastic system by eliminating a fast variable.

Gene expression in both prokaryotes and eukaryotes is inherently stochastic (1–4). This stochasticity is both controlled and exploited by cells and, as such, must be included in models of genetic networks (5,6). Here we will focus on describing intrinsic fluctuations, those generated by the random timing of individual chemical reactions, but extrinsic fluctuations are equally important and arise from the interactions of the system of interest with other stochastic systems in the cell or its environment (7,8). Typically, experimental data are compared with predictions of mean behaviors and sometimes with the predicted standard deviation around this mean, because protein distributions are often difficult to derive analytically, even for models with only intrinsic fluctuations.

We will propose a general, although approximate, method for solving the master equation for models of gene expression. Our approach exploits the difference in lifetimes of mRNA and protein and is valid when the protein lifetime is greater than the mRNA lifetime. Typically, proteins exist for at least several mRNA lifetimes, and protein fluctuations are determined by only time-averaged properties of mRNA fluctuations. Following others (7,9–11), we will use this time-averaging to simplify the mathematical description of stochastic gene expression.

For many organisms, single-cell experiments have shown that gene expression can be described by a three-stage model (3,4,12–14). The promoter of the gene of interest can transition between two states (10,15–17), one active and one inactive. Such transitions could be from changes in chromatin structure, from binding and unbinding of proteins involved in transcription (3,4,12), or from pausing by RNA polymerase (18). Transcription can only occur if the promoter region is active. Both transcription and translation, as well as the degradation of mRNAs and proteins, are usually modeled as first-order chemical reactions (5).

By taking the limit of a large ratio of protein to mRNA lifetimes, we will study the three-stage model and a simpler two-stage version where the promoter is always active. For this two-stage model, we will derive the protein distribution as a function of time. We will derive the steady-state protein distribution for the full, three-stage model. We also include expressions for the corresponding mRNA distributions (14,16) in the supporting information (SI) Appendix.

A Two-Stage Model of Gene Expression

We will first consider the model of gene expression in Fig. 1A (9). This model assumes that the promoter is always active and so has two stochastic variables: the number of mRNAs and the number of proteins. The probability of having m mRNAs and n proteins at time t satisfies a master equation:

with v₀ being the probability per unit time of transcription, v₁ being the probability per unit time of translation, d₀ being the probability per unit time of degradation of an mRNA, and d₁ being the probability per unit time of degradation of a protein. By defining the generating function, F(z′,z), by F(z′,z) = ∑_m,nz′^mzⁿP_m,n, we can convert Eq. 1 into a first-order partial differential equation:

where we have rescaled (19), with a = v₀/d₁, b = v₁/d₀, γ = d₀/d₁, and τ = d₁t, and where u = z′ − 1 and v = z − 1.

Fig. 1.

Predictions and simulations for a two-stage model of gene expression. (A) Both transcription and translation are modeled as first-order processes: transcription occurs with a probability v₀ per unit time and translation with a probability of v₁ per unit time. Degradation of mRNA and protein are also both first-order processes: mRNA degrades with a probability d₀ per unit time and protein degrades with a probability d₁ per unit time. (B and C) A comparison of Eq. 9, shown as the distribution in green, and stochastic simulations for large and small γ. Protein distributions can be either peaked or have a maximum at n = 0 (19). The mean number of mRNAs, a/γ, is either 2 or 20 in B and either 0.05 or 0.5 in C. D The accuracy of Eq. 9 improves with larger γ. The Kullback–Leibler divergence between the analytical and simulated protein distributions is plotted as a function of γ. For γ ≫ 1, the distributions become almost indistinguishable.

If the protein lifetime is much greater than the mRNA lifetime and γ ≫ 1, Eq. 2 can be solved by using the method of characteristics. Let r measure the distance along a characteristic which starts at τ = 0 with u = u₀ and v = v₀ for some constant u₀ and v₀, then Eq. 2 is equivalent to (20)

Consequently direct integration implies r = v = v₀e^τ. For γ ≫ 1, u(v) obeys (SI Appendix)

or

as v = v₀e^τ > v₀ for τ > 0. When γ ≫ 1, u rapidly tends to a fixed function of v: for most of a protein's lifetime, the dynamics of mRNA is at steady state. The generating function then obeys

Intuitively, Eq. 6 arises from Eq. 2 because large γ causes the term in square brackets in Eq. 2 to tend to zero to keep F(u,v) finite and well defined. Eq. 6 describes only the distribution for protein numbers: F(u,v) is just a function of v. Terms of higher order in γ⁻¹ will depend on u. Large γ implies that most of the mass of the joint probability distribution of mRNA and protein is peaked at m = 0: P_m,n ≃ P_0,n.

We can find the probability distribution for protein numbers as a function of time by integrating Eq. 6. Integration gives

assuming that no proteins exist at τ = 0. From the definition of a generating function, expanding F(z) in z gives (SI Appendix)

where P_n(τ) = P_0,n(τ). Here, ₂F₁(a,b,c; z) is a hypergeometric function and Γ denotes the gamma function (21). Eq. 8 is valid when γ ≫ 1, τ ≫ γ⁻¹ to allow the mRNA distribution to reach steady state, and a and b are finite. The mean, 〈n〉 = ab(1 − e^−τ), and the variance, 〈n²〉 − 〈n〉² = 〈n〉 (1 + b + be^−τ), of Eq. 8 agree with earlier results (9). At steady state τ ≫ 1 and

which is a negative binomial distribution. We verified Eq. 8 and Eq. 9 with stochastic simulations by using the Gibson–Bruck version (22) of the Gillespie algorithm (23) and the Facile network complier and stochastic simulator (24). If γ ≫ 1, Eq. 9 accurately predicts the distribution described by Eq. 1 (Fig. 1B and C), but it fails as expected for smaller γ. This effect can be quantified by calculating the Kullback–Leibler divergence between the predicted and simulated distributions for different γ (Fig. 1D). Eq. 8 is illustrated in Fig. 2A and B. As well as γ ≫ 1, times with τ > γ⁻¹ are necessary for negligible Kullback–Leibler divergences (Fig. 2C).

Fig. 2.

Predictions for the time-dependent solution of the two-stage model of gene expression. (A and B) The distribution of protein numbers at different times with time increasing in the direction of the arrow. Parameters in A correspond to Fig. 1B. There are zero proteins initially. Parameters in B correspond to Fig. 1C. There are 50 proteins initially. C The Kullback–Leibler divergence for the distributions of A and B. The divergence decreases as τ = td₁ grows above γ⁻¹. It is small for small times because both the simulations and the calculations start from the same initial distribution. (D) Noise in protein numbers as a function of time. Initially, proteins have a negative binomial distribution chosen with a particular magnitude of noise. The noise at steady state is shown by a dashed line. (E) The calculated distributions for the first-time protein levels reach a given threshold, N, if initially there are zero proteins. These distributions are qualitative with the probability typically underestimated for small td₁. They obey a renewal equation (26), which we solve numerically.

Eq. 8 allows complete characterization of the Markov process underlying the two-stage model. The “propagator” probability, P_n|k(τ), which is the probability of having n proteins at time τ given k proteins initially, satisfies (SI Appendix)

where P_n(τ) = 0 if n < 0. With Eqs. 8 and 10, two-stage gene expression is in principle completely characterized for γ ≫ 1 and τ ≫ γ⁻¹. For example, we can calculate how the noise in protein numbers, η (their standard deviation divided by their mean), changes with time. If protein numbers initially have a distribution P_k⁽⁰⁾, then at a time τ their distribution will be ∑_kP_n|k(τ)P_k⁽⁰⁾. The noise of this distribution can either increase, decrease, or behave nonmonotonically as time increases (Fig. 2D). We can also calculate non-steady-state autocorrelation functions and first-passage time distributions for protein levels to first cross a threshold, N (with some standard numerics). In general, such distributions are only qualitative because contributions from times with τ < γ⁻¹ are always relevant. Accuracy can be improved by having γ ≫ 10 and a sufficiently high threshold (Fig. 2E and SI Appendix).

We can derive Eq. 9 more intuitively. An mRNA undergoes a competition between translation and degradation because ribosomes and degradosomes bind to it mutually exclusively (25). For each competition, the probability of a ribosome binding to the mRNA is $\frac{v_1}{v_1+d_0}=\frac{b}{1+b}$. If we assume that proteins have longer lifetimes than mRNAs (γ ≫ 1), then each protein synthesized from a given mRNA will not on average be degraded before the mRNA is degraded. On protein timescales, all the proteins synthesized from an mRNA will appear to be synthesized simultaneously (SI Appendix Fig. S1). Consequently, the probability of r new proteins being produced by the synthesis and degradation of one mRNA is equal to the probability of an mRNA being translated r times. This probability is (25)

which is a geometric, or “burst,” distribution. Alternatively, we can consider the lifetime t′ of each mRNA. This lifetime is stochastic and satisfies P(t′) = d₀e^−d₀t′, the distribution expected for any first-order decay process (26). Protein synthesis is also first order, and the number of proteins, r, synthesized by an mRNA during its lifetime satisfies a Poisson process: $\frac{{(v_{1}t\text{'})}^r}{r!}{\text{e}}^{-v_{1}t\text{'}}$ (26). Consequently, the probable number of proteins synthesized from a particular mRNA is given by

which integrates to Eq. 11. Eq. 11 is equivalent to an exponential distribution with a parameter λ, where $\lambda =-log\left(1-\frac{b}{1+b}\right)$ (27). If b < 1, then λ ≃ b. Exponential bursts of protein synthesis have been characterized experimentally (28,29). We note that Eq. 11 has a generating function f(z) = (1 + b − bz)⁻¹.

Given that the synthesis and degradation of one mRNA generates a burst of r proteins, then the number of proteins at steady state is given by the typical number of mRNAs synthesized during a protein lifetime, $\frac{v_0}{d_1}=a$, and the r_i for each mRNA. The number of proteins n will be sum of these r_i. If we assume that there are sufficient ribosomes and charged tRNAs, then translation from each mRNA is independent. The generating function of a sum of independent variables is the product of their individual generating functions (26). Consequently, the generating function for P_n, F(z), satisfies

which is Eq. 7 when τ ≫ 1, and so derives Eq. 9.

By assuming explicitly that protein synthesis occurs in bursts, we can derive an effective master equation for gene expression that considers only proteins, but implicitly includes mRNA fluctuations (19,30). We will show that this master equation has Eq. 8 as its solution and so is equivalent to the large γ approximation to Eq. 1, the master equation for both mRNA and protein. If we assume that each mRNA synthesized leaves behind a burst of r proteins then

where the size of each burst has been determined by Eq. 11 (30). Eq. 14 has Eq. 7 as its generating function (SI Appendix). By introducing bursts of protein synthesis, mRNA fluctuations can be absorbed into a one-variable master equation provided γ ≫ 1. Friedman et al. used a continuous version of this approach with an exponential burst distribution inspired by their experimental results (28,29). They derived a gamma distribution for steady-state protein numbers (19). Eq. 9 tends to this distribution

for large n (SI Appendix). Friedman et al. (19) also demonstrated that the burst approximation remains valid when negative or positive feedback is included.

In summary, we have shown that exploiting the difference between protein and mRNA lifetimes through a large value of γ, but finite a and b, allows powerful mathematical simplifications. Large γ implies that mRNA is at steady state for most of the lifetime of a protein and that the probability mass of the joint distribution of protein and mRNA is peaked at zero mRNAs, although the mean number of mRNAs need not be zero (Fig. 1). The number of proteins translated from an mRNA obeys a geometric distribution in both the two-stage and three-stage models (25), but large γ implies that the proteins translated from an mRNA all appear, on protein timescales, simultaneously so that the synthesis and degradation of an mRNA leaves behind a geometric burst of proteins. If γ < 1, then proteins synthesized from a particular mRNA will be degraded as further proteins are synthesized, and the distribution describing the number of proteins remaining once the mRNA is degraded will no longer be geometric. Explicitly including geometric bursts accurately describes the effects of mRNA fluctuations on the distribution of protein numbers when γ ≫ 1. It allows the model of Fig. 1A to be described by a one-variable master equation: Eq. 14.

A Three-Stage Model of Gene Expression

We next consider the full three-stage model of gene expression (Fig. 3A). We find the protein distribution for this system by taking the large γ limit of the master equation. Let P_m,n⁽⁰⁾ be the probability of having m mRNAs and n proteins when the DNA is inactive and P_m,n⁽¹⁾ be the probability of having m mRNAs and n proteins when the DNA is active. We then have two coupled equations:

where κ₀ = k₀/d₁ and κ₁ = k₁/d₁.

Fig. 3.

Predictions and simulations for a three-stage model of gene expression. (A) The region of the DNA containing the promoter region transitions between inactive and active forms with probabilities per unit time of k₀ and k₁. As an example, we show the TATA-box binding protein driving the transition. (B–D) A comparison of Eq. 18, shown as the distribution in green, and stochastic simulations for large and small γ. The mean number of mRNAs, $\frac{a{k}_0}{\gamma (k_0+k_1)}$, is either 3 or 30 in B, 0.075 or 0.75 in C, and either 0.3 or 3 in D.

We solve Eqs. 16 and 17 at steady state by taking the large γ limit of the equivalent equations for their generating functions (a generating function is defined for each state of the promoter). Our approach is a natural extension of the method used to solve the two-stage model (SI Appendix). We find that

where

and ϕ² = (a + κ₀ + κ₁)² − 4aκ₀. Eq. 18 is valid when γ ≫ 1 and a and b are finite. The mean of this distribution is 〈n〉 = $\frac{ab{k}_0}{k_0+k_1}$ and the protein noise, η, satisfies

where 〈m〉 is the mean number of mRNAs, and is inversely proportional to γ, and η_D is the noise in the active state of DNA: η_D² = k₁/k₀ (4). As well as a Poisson-like term expected for any birth-and-death process, protein noise has time-averaged contributions from fluctuations in the number of mRNAs and fluctuations in the state of DNA. We verify Eq. 18 by simulation in Fig. 3.

The protein distribution for the three-stage model can have similar behavior to the two-stage model of Fig. 1A, but it can also generate a bimodal distribution with a peak both at zero and nonzero numbers of molecules (Fig. 3D). This bimodality is not a reflection of an underlying bistability, but arises from slow transitions driving the DNA between active and inactive states (5,17,31,32).

As expected, Eq. 18 recovers the negative binomial distribution under certain conditions. It tends to Eq. 9 when κ₁ → 0: the DNA is then always active at steady state. When κ₁ = 0, Eqs. 19 and 20 imply that α = a and β = κ₀, and recall that ₂F₁(a,0,c;z) = 1 for all a, c, and z. Similarly, when κ₀ and κ₁ are both large, but κ₀/κ₁ is fixed, then α → κ₀ + κ₁ and $\beta \to \frac{{\kappa }_{0}a}{{\kappa }_0+{\kappa }_1}$. Consequently,

because ₂F₁(a,b,a;z) = (1 − z)^−b. With fast switching of the DNA between active and inactive states, Eq. 18 becomes Eq. 9, but with a replaced by $\frac{{\kappa }_{0}a}{{\kappa }_0+{\kappa }_1}$.

Discussion

We have shown we can calculate distributions for protein numbers by assuming that protein lifetimes are longer than mRNA lifetimes with a, the number of mRNAs transcribed during a protein remaining, and b, the number of proteins translated during a mRNA lifetime, remaining finite. Fig. 4A shows the ratio γ measured for almost 2,000 genes in budding yeast. Approximately 80% of the genes have γ > 1 and the median value is ≈3 (we include the dataset in SI Appendix). We therefore expect our predicted distributions to be widely applicable in budding yeast. In bacteria, too, γ is expected to be above 1 because mRNA lifetimes are usually minutes (they are typically tens of minutes in yeast) and protein lifetimes are often determined by the length of the cell cycle (typically 30 or more minutes) (33).

Fig. 4.

The ratio of the protein to mRNA lifetime, γ, for 1,962 genes in budding yeast. (A) Most proteins have γ > 1. Protein lifetimes are from Belle et al. (41) and mRNA lifetimes are from Grigull et al. (circles) (42) or from Wang et al. (squares) (43). The median of γ is ≃ 3 (shown by a dashed line), whereas its mean is >10 (although this value is probably erroneously high because of outliers). Overall, we found little correlation between mRNA and protein lifetimes. (B) The median value of γ for genes in different gene ontology classes. We plot the mean of the medians for the two datasets. Errors in the medians are ≈25% (using 1,000 bootstrap samples for each gene ontology class). Gene annotations are from the Saccharomyces cerevisiae genome database (www.yeastgenome.org).

Values of γ > 1 reduce protein fluctuations by allowing more averaging of the underlying mRNA fluctuations (Eq. 21). We indeed observe a small, but statistically significant, negative correlation between total noise and γ by using the data of Newman et al. (34) (a rank correlation of ≃ −0.2 with a P value of 10⁻⁶). In Fig. 4B, we have calculated the median γ for yeast genes in different gene ontology classes. All classes have a median γ > 1. Proteins involved in transferring nucleotidyl groups, which include RNA and DNA polymerases, have high median γ > 5, presumably because high stochasticity in these proteins can undermine many cellular processes. Similarly, proteins that contribute to the structural integrity of protein complexes have a median γ > 5. Large fluctuations can vastly reduce the efficiency of complex assembly by preventing complete complexes forming because of a shortage of one or more components (11,35). Perhaps surprisingly transcription factors have a low median γ > 1. Although low γ does increase stochasticity, it can allow quick response times if the protein degradation rate is high. A high protein degradation rate may also keep numbers of transcription factors low to reduce deleterious nonspecific chromosomal binding.

We show that protein synthesis occurs in bursts in both the two- and the three-stage model when γ ≫ 1. Such bursts of gene expression have been measured in bacteria and eukaryotes (12–14,28,29). They allow mRNA to be replaced in the master equation by a geometric distribution for protein synthesis for all times greater than several mRNA lifetimes if their source is translation and the protein lifetime is substantially longer than the mRNA lifetime. Such an approach has already been proposed (19), but without determining its validity. Similarly, if mRNA fluctuations are negligible, the master equation reduces to one variable (protein), and describing the protein distribution becomes substantially easier (31,36).

An important problem in systems biology is to determine which properties of biochemical networks and the intracellular environment must be modeled to make accurate, quantitative predictions. Besides obscuring the process driving the observed phenotype, models more complex than needed are harder to correctly parameterize and to simulate to generate predictions. Our results show that complexity, here two states of the promoter, can be modeled by effective parameters that under certain conditions will give accurate predictions of the entire distribution of protein numbers: Eq. 22. Alternatively, they show that not only the mean and variance (37), but also the protein distribution may not have enough information to determine the biochemical mechanism generating gene expression from measurements of protein levels: Eqs. 9 and 22. Such effects are likely to be compounded by non-steady-state dynamics (Fig. 2) and extrinsic fluctuations. Collecting data on the corresponding mRNA distribution may disfavor the two-stage over the three-stage model because mRNA distributions in the three-stage model can have two peaks even though the protein distribution has only one (6). In general, though, time series measurements, preferably with and without perturbations, may provide the most discriminative power (37).

Experimental measurements are best compared with the predicted distribution rather than its mean, standard deviation, or mode. Both the protein and mRNA distributions are typically not symmetric and may not be unimodal. Consequently, the mean and the mode can be significantly different, and the standard deviation can be a poor measure of the width of the distribution at half-maximum (38). Such distributions are poorly characterized by the commonly used coefficient of variation because they are not locally Gaussian around their mean (Figs. 1–3). In addition, fitting moments to find model parameters can be challenging. Moments, more so than distributions, are functions of combinations of parameters and can also be badly estimated without large amounts of data, particularly for asymmetric distributions. We therefore believe a Bayesian or maximum likelihood approach is most suitable where the experimental protocol is replicated by the fitting procedure and explicitly accounts for the shape of the distribution and the number of measurements. For example, irrespective of how many measurements are available, the likelihood of the data for a particular set of parameters can always be determined from the assumed distribution of protein numbers. Our analytical expressions will greatly speed up such approaches by avoiding large numbers of simulations and by aiding in deconvolving extrinsic fluctuations that can substantially change the shape of protein distributions (8).

Our results should also allow more general fluctuation analyzes of gene expression data. Such analyzes convert fluorescence measurements into absolute units (numbers of molecules) by exploiting that the magnitude of fluctuations is determined by the number of molecules independently of how those numbers are measured (39). Converting into absolute units is essential if information from different experiments is to be combined into a larger, predictive framework, a goal of systems biology.

More generally, our approach is an example of a technique to simplify the dynamics of a stochastic system by exploiting differences in timescales. We remove a fast stochastic variable through replacing a constant parameter (the parameter a) by a time-dependent parameter (the burst distribution) whose variation captures the effects of fluctuations in the fast variable on the dynamics of the slow one (40).