Analytic Approaches to Stochastic Gene Expression in Multicellular Systems

Abstract

Deterministic thermodynamic models of the complex systems, which control gene expression in metazoa, are helping researchers identify fundamental themes in the regulation of transcription. However, quantitative single cell studies are increasingly identifying regulatory mechanisms that control variability in expression. Such behaviors cannot be captured by deterministic models and are poorly suited to contemporary stochastic approaches that rely on continuum approximations, such as Langevin methods. Fortunately, theoretical advances in the modeling of transcription have assembled some general results that can be readily applied to systems being explored only through a deterministic approach. Here, I review some of the recent experimental evidence for the importance of genetically regulating stochastic effects during embryonic development and discuss key results from Markov theory that can be used to model this regulation. I then discuss several pairs of regulatory mechanisms recently investigated through a Markov approach. In each case, a deterministic treatment predicts no difference between the mechanisms, but the statistical treatment reveals the potential for substantially different distributions of transcriptional activity. In this light, features of gene regulation that seemed needlessly complex evolutionary baggage may be appreciated for their key contributions to reliability and precision of gene expression.

Introduction

Thanks to the concerted efforts of many labs and large consortia like the Encyclopedia of DNA Elements (ENCODE, National Human Genome Research Institute, https://www.genome.gov/10005107) and the Model Organism Encyclopedia of DNA Elements (modENCODE, National Human Genome Research Institute, https://www.genome.gov/26524507), we are beginning to understand the noncoding “dark matter” of the genome, and we are discovering that much of it plays a critical role in the regulation of gene expression (1). Indeed many of the differences in complexity among yeast, flies, and humans likely reflect the dramatic increase in noncoding regulatory DNA between these organisms, rather than the slight difference in the number of genes. Like Lego bricks (The Lego Group, Billund, Denmark), a common set of components (genes) can be combined and reused to create a complex variety of structures (cell types). The list of components is insufficient to determine the structure; assembly directions for how they go together are also critical. For many organisms, we now have a good grasp on the list of components, the coding genes. The next major challenge is to learn to read the assembly directions, and understand how noncoding DNAs control precise expression levels and combination of these genes.

In simpler organisms with much smaller genomes, mathematical models have played an important role in developing a predictive and mechanistic understanding of how regulatory sequences effect gene transcription. Models of classical systems such as the Lac operon from Escherichia coli (2, 3) and cI/cro regulation from phage-λ (4) facilitate an intuitive understanding of how transcription depends quantitatively on transcription factor (TF) concentration, binding site organization, and binding site affinities. Stochastic models of this regulation have provided further insight into biological behaviors of these systems, such as the spontaneous switching of individual cells between Lac-expressing and Lac-silent states at intermediate inducer concentrations (5).

However, in higher multicellular organisms, the transcriptional state of a gene is controlled not by a few individual proteins, but by large macromolecular assemblies of transcription factors. The assembly of these factors is mediated by numerous distinct binding sites in the regulatory DNA sequence, as well as substantial protein-protein interactions between the factors (6, 7). Chromatin and DNA packaging play major roles in shaping expression (8–10). A standard mathematical framework with which to handle this substantial array of chemical states and the associated range of kinetic transitions has yet to be established.

A good start has been made with the application of thermodynamic models (also called site-occupancy models) to understand binding site interactions in developmental regulatory sequences (called enhancers) (11–20). Thanks to careful work, numerous regulatory sequences have been experimentally and computationally analyzed to identify the number, type, and organization of the binding sites they contain for a variety of transcription factors (for review, see Spitz and Furlong (6) and Levine (7)). The thermodynamic models of regulatory properties of these sequences combine estimates of the binding site strengths and the TF concentrations to predict the expected fractional occupancy of regulatory TFs at their target sites (11–20). In most cases, a heuristic expression is then used to relate fractional occupancy to an expected transcription rate (12–19). Predicted differences in transcription rate under different concentrations of TFs and different organizations of binding sites can be compared to experimental data on relative expression levels (14,17,19,20). While providing an excellent first step into understanding how a noncoding sequence affects properties of gene expression, these models have a few important limitations. The use of ad hoc, heuristic relations between transcription factor site occupancy and transcription rates may hide some important biophysical properties of regulation that could be detected with more biophysically grounded models. Most notably, though, the deterministic treatment excludes any exploration of how the regulatory organization (determined by noncoding sequence) effects variability in gene expression.

Evidence is accumulating that the regulation of stochastic effects may be just as important for understanding the qualitative behavior of transcription control in metazoa as in prokaryotes and viruses. In the first part of this review, I substantiate the need for stochastic models of gene regulation in metazoa with a few recent examples from developmental biology where regulation of expression noise has been shown to be essential for proper development. In the second part, We I will review an emerging mathematical framework general enough to explore stochastic properties of complex regulatory pathways and illustrate its utility through three examples of mechanisms that could control the degree of variability in gene expression.

Stochastic Expression in Metazoa

Single cell imaging experiments have clearly demonstrated that the response of multiple genetically identical cells to the same stimulus may be highly variable. Pioneering work by Ko et al. (21) showed in 1990 that the response of the glucocorticoid-inducible transgene in cell culture exhibited a stochastic all-or-none response on the level of individual cells, giving rise to the previously described smooth dose-response phenomenon only in aggregate. Ferrell and Machleder (22) similarly demonstrated that the endogenous MAPK response in Xenopus oocytes is also heterogeneous on the level of individual cells. Throughout the early 90s, Jiang et al. (23,24), Ip et al. (25), and Jiang and Levine (26) reported increased cell variability in transcription of reporter genes driven by partially disrupted enhancers for snail, twist, or rho. These experiments, among others, demonstrated that metazoan gene expression occurs in a regime where stochastic effects cannot be readily ignored.

However, it was not until more-recent technological improvements developed such as fluorescence-activated cell sorting (27), counting of nascent transcription foci (28–34), and counting of individual mRNA molecules, (35–43) that stochastic cellular variation could be readily studied quantitatively and the effects linked to particular sequences of regulatory DNA. Using these approaches, recent studies have identified endogenous regulatory sequences that appear to improve survival fitness because of how they interact with the intrinsically stochastic nature of gene expression. Before such measurements, it was largely believed that genetic differences between individuals’ DNA and stochastic differences in their environment accounted for all phenotypic variation during embryonic development. These quantitative experiments raise the possibility that plain luck can also strongly shape developmental outcomes. Moreover, they suggest that a substantial part of the genome may be dedicated to altering probabilities of rare events (weighting the dice) to reduce the chance that rare molecular events during development lead to a permanent decrease in fitness. If this is true, then a deterministic view of gene regulation will never explain the function provided by much of the genome sequence information. Here I briefly review some recent experiments linking endogenous noncoding sequences to a role in controlling the reliability of embryonic development.

In 2009, it was reported that developmentally important genes, which were regulated during the early stages of transcriptional elongation, were expressed in a more synchronous fashion than those using a more familiar mechanism of regulating polymerase binding (29). (See Fig. 1, A–D, for an explanation of how cell-cell variability in the onset of transcription can be measured.) This synchronous versus stochastic response is largely determined by sequences near the transcription start site, which determine whether polymerase binding or polymerase elongation is regulated by nearby enhancers. Subsequent work by Lagha et al. (44) in 2013 showed that interchanging these sequences could change just how synchronous the activation of the gene is without changing the expression pattern. They also showed that promoter exchanges which increase variation in expression timing for the gene snail lead to defects in downstream cellular movements essential for development—illustrating the importance of controlling expression variability for normal growth.

Figure 1.

Regulation of transcriptional noise in multicellular animals. (A) Fly embryo. Transcriptionally active cells expressing tup (green) and pnr (red) in the dorsal ectoderm of the embryo. (B) Pol II ChIP from dorsal ectoderm and mesoderm tissue showing that the gene tup (top trace) regulates Pol II elongation whereas the gene pnr regulates Pol II binding or initiation. Note only elongation regulated genes show initiated Pol II at the promoter in tissues where the gene is not expressed. (C) Fraction of transcriptionally active nuclei present over time for tup, exhibiting more synchronous induction and pnr, exhibiting more variable induction (data adapted from Boettiger and Levine (29)). (D) Images of nuclei at metaphase when transcription of all genes is aborted. All cell cycles are synchronized. As cells progress into interphase, sites of nascent transcription (green dots) for some genes appear in a synchronous fashion and for others in a more stochastic one. (E) Worm embryo, labeled for let-2 (red) (image reprinted by permission from Macmillan Publishers Ltd: Nature, Raj et al. (38), copyright (2010)). (F) Schematic of partially redundant activation of end-1 in wild-type and skn-1 mutant cells. (G) Schematic of frequency for different total levels of end-1 mRNA per embryo for wild-type and skn-1 mutants. Embryos with high levels of end-1 (from either genotype) successfully activate let-2. Data adapted from Raj et al. (38). (H) Fly larvae indicating trichrome bristles (image reprinted by permission from Macmillan Publishers Ltd: Nature, Frankel et al. (46), copyright (2010)). (I) Schematic of the svb regulatory region. Three proximal enhancers together ensure svb expression throughout the trichrome zone. A largely redundant pattern is also produced by the combined action of two further upstream shadow enhancers. (J) Bristle number from embryos with and without the shadow enhancers at different temperatures (data adapted from Frankel et al. (46)). To see this figure in color, go online.

Similar evidence for the importance of controlling expression variability came from experiments by Raj et al. (38) in gut patterning of the developing Caenorhabditis elegans embryo (Fig. 1 E). In wild-type embryos, the gene end-1 is an important activator of let-2, a gene critical for intestinal precursor cell development (Fig. 1 F). Embryos that are mutant for one of several genes activating end-1 can still transcribe it, but with much more variation in expression levels between embryos (Fig. 1 G). Those lucky enough to be in the upper quintile of expression will activate let-2, but the rest fail to do so and cannot pattern the gut properly. By contrast, wild-type embryos, with their redundant mechanisms of activating end-1, maintain a more narrow distribution of end-1 expression that is essentially always sufficient to properly specify gut tissues (Fig. 1 G) (38).

Regulatory sequences in fact routinely exploit redundancy to reduce expression variability. The rapid pace for discovering new cis-regulatory sequences (enhancers) has identified many such sequences with apparently redundant activities (6,7,45). For instance, the early mesodermal expression pattern of the gene snail in Drosophila can be independently produced with a reporter gene driven by either of two upstream noncoding sequences. Recent experiments by Perry et al. (30) have shown that either sequence is sufficient to activate endogenous snail expression, but variability in the number of actively transcribing cells increases under heat stress or genetic stress if either enhancer is removed. In these conditions, the fraction of embryos with abnormal mesodermal cell invagination (normal mesodermal cell invagination being required for proper muscle development) also increases. In examining the control of bristle patterning in the early fly larvae (Fig. 1 H) Frankel et al. (46) demonstrated a similar increased variability of developmental phenotypes under thermal or genetic stress upon removing the apparently redundant enhancers of the shavenbaby (Fig. 1, I and J).

These results collectively suggest that there are genomic features which may have evolved and been conserved specifically to mitigate variability (noise) in gene expression. It is conceivable that such features even dominate the total amount of sequence dedicated to gene regulation. While each individual mechanism only buffers against the rare chance of a particular defect, a genome lacking in all such variability-controlling mechanisms may have unacceptably low odds of producing a fully fit organism.

While to a large degree stochastic effects in gene regulation may consequently have lead to the evolution of mechanisms that minimize variation (allowing for the coordinated and cooperative cell behavior characteristic of multicellularity), some metazoan gene networks exploit and amplify variability to simplify the regulatory problem of patterning different cell types. For example, in the Drosophila compound eye, 30% of ommatidia express short-wave-length responsive (blue) Rh3 and Rh5 photoreceptors whereas the remaining 70% express receptors responsive to longer wavelengths (yellow). This ratio, and the spatial distribution of different color-sensitive cells, is determined not by differential control of a complex gene network measuring spatial position to determine the correct fate, but rather by an essentially uniform population of cells with a 30% stochastic probability of expressing stable levels of the gene spineless (47). Similar mechanisms are believed to determine the ratio of different color-responsive cell populations in the mammalian eye (48) and the fate specification required in patterning some 1000 different types of olfactory cells in the mouse (49,50).

Mathematical Approaches to Stochastic Gene Expression

Given the important role stochastic effects of gene expression have in development, it is essential to have a framework in which to understand their origin and how they change in the context of different mechanisms of regulation.

As of this writing, detailed models exploring how cis-regulatory mechanisms particular to higher metazoa affect stochasticity in gene expression, are largely lacking in the literature. However, recent theoretical work on gene expression exploiting mathematical results from the theory of finite Markov processes has started to construct a sufficiently general framework with which the stochastic properties of these regulatory architectures can be explored.

Mathematical approaches to biochemical reactions

The fundamental chemistry (or statistical physics) of gene regulation consists of molecular interactions that occur because of random thermal collisions, mediated by the binding energies of the associated molecules. Mathematically, the system is discrete (it has integer components that vary in their number and chemical composition) and stochastic (components combine, are born, and die as random processes). The behavior of such a system is described by its chemical master equation (CME), which simply enumerates all the different molecular states that might exist (e.g., the number and type of all chemical species), and the probabilities that one state will convert to any another (e.g., two molecules bind to each other, or one of the molecules degrades). The transition probabilities depend only on the identity of the prevailing state and not the history of the process (e.g., a dimer has a fixed probability of disassociating into monomers, which is independent of how many times it may have disassociated and rebound in the past). In mathematics, such a system is called a Markov process. I recommend Durrett (51) for a more thorough introduction to Markov processes and the mathematical tools for analyzing them.

For arbitrary chemical reactions, the CME has an infinite number of complex states, and so additional assumptions are introduced to make it more analytically tractable. If the number of molecules of each chemical species is large, each may be approximated by a continuous variable, and the system approximated by a set of differential equations. As the number of molecules of each chemical species decreases, the stochastic nature of the system can no longer be ignored. In this case, alternative approximations to the CME which still exploit the continuum limit but do allow treatment of stochastic variation have been used, such as Langevin methods (52–55), and/or linear noise approximations (van Kampen expansion) (56,57). Although such approaches have been fruitfully applied to the study of gene regulation, they must be used with care. As the number of molecules gets especially small, some continuum approximations start to exhibit qualitatively different behavior than the corresponding CMEs (58,59). For further details of these and other analytical master-equation approaches to stochastic regulatory networks, I recommend Walczak et al. (60).

An emerging framework for gene-regulatory models

Here, I focus on an alternative approach to the challenges of working with an infinite state CME that does not require molecules to be approximated with continuous variables (as is done in Langevin methods or the van Kampen expansion). This is particularly important for accurate models of gene regulation, because most of the key chemical species involved occur in very small copy numbers (often 1–4 in metazoa), and thus are poor candidates for continuum approximations. For example, a diploid cell in G1 phase may have at most two copies of the chemical species gene A with a polymerase bound at the promoter.

The insight of this alternative approach is to exploit the fact that copy numbers are so small to apply some powerful tools for handling finite Markov processes. For transcription regulatory reactions, the infinite part of the chemical master equation has a simple structure, based on the birth and death of mRNA molecules (and proteins). The complex protein-protein and protein-DNA interactions in the system are mediated on the DNA template. Because there are only a few templates per cell, the number of unique chemical states for this complex part of the system is manageably finite. With clever tricks to separate the simple infinite aspects from the complex finite aspects of the problem, analytic Markov methods for finite systems can be directly applied to study the probability distributions of the stochastic properties of interest.

In the next section, I present key results from recent work that have generalized the finite Markov approach to describe as wide a range of regulatory structures as are currently studied with deterministic, site-occupancy models. For proof and derivation of the results, I direct the reader to the original works. The elegance of this formulation is that it provides a prescription to go from a cartoon description of the biochemical states to analytic expressions that describe a rich set of stochastic properties of transcription—all by evaluating a few finite matrix expressions (easily done on a computer algebra system). To practice applying this prescription, we will walk through three examples about how the appropriate choice of regulatory mechanism can substantially alter variability in gene regulation without changing average properties.

Model Formalism

We begin with a biochemical cartoon detailing the molecular events that lead to a transcriptionally competent promoter and assign rates to each of these events (see Fig. 2 A, Fig. 3, A and D–F). Transcription factor (i) binds at a rate determined by the product of its site-specific binding affinity $\left(k_{a_i}\right)$ and its concentration, and leaves at a rate $\left(k_{d_i}\right)$ determined by its stability in the complex. The resulting cartoon can be summarized as a Markov chain. For example, the shaded box in Fig. 2 A indicates two chemical states in the promoter, which are represented as separate states in the Markov chain (Fig. 2 B, shaded box). All state changes in the promoter composition are described by the associated transition matrix M. The (i, j) elements of M specify the probability of the system to hop from state i to j for i ≠ j. The elements (i, i) are chosen so that all of the rows sum to zero. We write the probability that the system is in state j at time t, given that it started in state i at time 0, as the element P_{i, j}(t) of the matrix P. From the theory of Markov processes, the evolution of P is given by the forward Kolmogorov equation,

$$\frac{d}{dt}P\left(t\right)=P\left(t\right)M.$$ (1)

Figure 2.

From cartoons to Markov models. (A) A simple promoter that is transcriptionally active when the activator A is bound. When active, mRNA is produced at rate k_f and degraded at rate δ. (B) Complete Markov model corresponding to the cartoon in panel A, arranged to show the orthogonal nature of promoter states and mRNA counts. (Shaded box) Core promoter state Markov chain. (C) A simple promoter in which polymerase binds and then initiates transcription. (D) The corresponding Markov model. Each transition from state 2 to 3 produces a new mRNA molecule.

Figure 3.

Example of regulatory mechanisms explored through Markov models. (A) Cartoon model of the two-state promoter. (B) Corresponding Markov chain. State 2 is the transcriptionally active state. (C) Simulation results illustrating differences in expression variability over time for strong and weak binding cases (reproduced from Sanchez et al. (63), Copyright 2011 CC-BY). (D) A linear transcription cycle with a single active transcription state, of the sort considered in Pedraza and Paulsson (66). (E) Modified transcription cycle where mRNA is produced during one of the transitions in the firing cycle, rather than as an exponential process while the chain is in an active state. (F) Minimal models of separate enhancer and promoter binding events. Two potential regulated transitions are indicated by the gate symbols over the transition arrows. (G) Markov chains for the combined promoter-enhancer states shown in panel F for the case where k₁₂ is regulated (initiation-regulation gated, left diagram) or where k₃₄ is regulated (elongation-regulation gated, right diagram). (H) (Top panel) Histogram of the ratio of the average response time between the initiation- and elongation-regulation schemes for a uniform sampling through all parameter space. (Lower panels) As in top panel, for the variance in response time, and COV in mRNA produced (reproduced from Boettiger et al. (64), Copyright 2011 CC-BY).

So far, all the information about the chemical state of the promoter, all the complex protein-DNA, and protein-protein binding events that may precede transcription, are now described by the finite transition matrix M. The other dimension we will be interested in is the number of mRNA molecules for this gene in the cell, N (Fig. 2 B, x axis). This mRNA counting dimension is the feature that makes the chemical master equation infinite and intractable. The clever insight of Peccoud and Ycart (61) in treating two-state promoters (generalized by Sánchez and Kondev (62) to arbitrary promoter matrices) is that for many descriptions of regulatory systems, the promoter state and the counting state never change at the same time. This is the case for all systems that can be approximated as having some transcriptionally active state or states, in which repeated initiation events occur at some characteristic firing rate (e.g., Fig. 2 A and Fig. 3, A and D). In this case, the state of the promoter does not change when an mRNA is produced, or when it decays, and elegant solutions can be derived for the distribution of N (which I present next).

Steady-state behavior

The distribution of mRNA levels N may reach steady state rapidly when the timescale of macromolecular assembly required to activate transcription is fast relative to the duration of expression. In this case, the average rate of mRNA production is determined by the fraction of the time the system spends in the active state (p_i), times the rate of mRNA synthesis in that state. The average number of mRNA in the cell, μ_N, is this production rate over the degradation rate. If multiple states i can transcribe mRNA at different rates r_i, the synthesis rate can be written as a dot product (63):

$${\mu }_N=\frac{1}{\delta }\overrightarrow{r}·\overrightarrow{p}.$$ (2)

Now we need only find the vector $\overrightarrow{p}$, which gives probability the system is in state i. This probability is given by the normalized nullspace of the transpose of the transition rate matrix,

$$0=M^T\overrightarrow{p}.$$ (3)

The variance for the number of mRNA molecules at steady state for this class of models, which assume mRNA is produced as an exponential process while the system is in some active state, can also be computed from the state transition matrix M. This is

$${\sigma }_N^2=\frac{1}{\delta }\overrightarrow{r}·\overrightarrow{m}+{\mu }_N-{\mu }_N^2,$$ (4)

where $\overrightarrow{m}$ is the steady-state mean number of mRNA molecules produced by each promoter state, which can be obtained from the solution to the matrix expression

$$0=\left(M^T-\delta \mathbb{I}\right)\overrightarrow{m}+R\overrightarrow{p},$$ (5)

and where R is a diagonal matrix with $R_{ii}={\overrightarrow{r}}_i$. For details of this derivation, see Sánchez et al. (63).

Kinetic properties of transcription

In addition to computing the probability that the system is in a given macromolecular state and the moments of the steady-state distribution of mRNA in the population, we may also be interested in the kinetics of the process. For example, how long does it take to reach the assembled state from the completely unassembled state on average? How much variation is there in the time of complex assembly? And how do these properties depend on the fundamental architecture of the underlying macromolecular machinery?

These questions can be answered with an alternative formulation of the system that does not require the assumption that the promoter state not change when a new mRNA is initiated. This alternative formulation will prove useful to study how the details of polymerase firing and reloading affect stochastic properties of transcription output. Rather than assume exponential synthesis rates while an active state is occupied, we can specify explicitly which chemical transitions lead to mRNA production (compare the cartoon description in Fig. 2, A and B). In its contemporary form in the literature, this extra detail comes with a tradeoff, which is to ignore degradation and model only total mRNA produced by time t (N_p(t)), not steady-state distributions of total mRNA, N. The distinction is shown in the comparison of Fig. 2, B and D. In some experiments N_p(t) rather than N(t) is the only measurable variable because only birth events are detected or because mRNA decay has been substantially hindered by the detection approach.

The first step is to compute the time it takes a system that starts in an initial state i to assemble into some final state f (called the first-passage time from i to f). Because the intermediate jumps are all stochastic events, this time is also a random variable, τ, and has a probability distribution p(τ). It can be shown that the Laplace transform of the probability density function for τ is given by (64)

$$\varphi \left(\lambda \right)=\underset{0}{\overset{\infty }{\int }}p\left(\tau \right)e^{-\lambda \tau }d\tau =\lambda {\left(\lambda \mathbb{I}-\tilde{M}\right)}_{sf}^{-1}.$$ (6)

Here I is the identity matrix, λ is the Laplace variable, and $\tilde{M}$ is a modified form of the transition matrix M, with the row corresponding to the final state set to zero. A useful property of the Laplace transforms of probability distributions is that they may be differentiated to find all the moments,

$$\underset{0}{\overset{\infty }{\int }}{\tau }^{n}p\left(\tau \right)d\tau ={\left(-1\right)}^n\frac{d^n}{d{\lambda }^n}\varphi \left(\lambda \right){|}_{\lambda =0}.$$ (7)

The average transcription rate can be computed as the inverse of the first passage time from the state which is entered after the polymerase has fired (s), to the state which is entered during polymerase firing (f) (64). Therefore the average number of mRNA molecules produced by time t is

$${\mu }_{N_p\left(t\right)}=\frac{t}{{\mu }_{\tau }}=t{\left(-\frac{d}{d\lambda }\lambda {\left(\lambda \mathbb{I}-\tilde{M}\right)}_{sf}^{-1}{|}_{\lambda =0}\right)}^{-1}.$$ (8)

The variance in the number of visits through the mRNA-producing transition per unit time gives the variance the total mRNA produced, ${\sigma }_{N_p\left(t\right)}^2$. This can be approximated from renewal theory (64) from the moments for the first passage time, computed with Eqs. 6 and 7, as

$${\sigma }_{N_p\left(t\right)}^2\approx \frac{t{\sigma }_{\tau }^2}{{\mu }_{\tau }^3},$$ (9)

from which it can be seen the coefficient of variation (COV) for the number of transcripts produced by time t is

$${\eta }_{N_p\left(t\right)}=\frac{{\sigma }_N}{{\mu }_N}\approx \frac{{\sigma }_{\tau }}{\sqrt{{\mu }_{\tau }t}}.$$ (10)

We next explore the utility of these two similar frameworks by applying their results to develop an intuitive understanding of stochastic behaviors of a few regulatory mechanisms recently examined in the literature.

Examples

Example 1: when weak is better than strong

Our first example comes from work by Sánchez et al. (63), which asks how does the strength of transcription factor-DNA interactions affect expression variability? Much intuition for this question can be derived from a simple toy model of a single transcriptional activator (see Fig. 3 A). When activator A is bound (occurring at rate k_a[A]), the promoter is in the active state and polymerase molecules may bind and start transcribing RNA (at rate k_f). When the activator dissociates (occurring at rate k_d), the promoter is silent. We would like to understand quantitatively how the noise differs as a function of binding site strength. For a fair comparison between weak and strong sites, Sánchez and colleagues required that the average expression level achieved by each promoter is the same, and assume that the cell regulates this expression level by tuning the concentration of the transcriptional activator.

We begin by writing down the corresponding transition rate matrix M for the schematic version of this system shown in Fig. 3 B. This requires one new matrix row for each state i in the model. Recall the element (i, j) records the rate at which state i transitions to j, so in this case element (1, 2) is k_a[A] and (2, 1) is k_d. The elements (i, i), are chosen so the rows sum to zero, thus the matrix we will need to work with is

$$M=\left[\begin{array}{cc}-k_a\left[A\right]& k_a\left[A\right]\\ k_d& -k_d\end{array}\right].$$ (11)

If we assume reinitiation events are dominated by a single rate-limiting step, then we can use Eqs. 2–5 to compute how the COV depends on the binding strength.

The average amount of RNA at steady state is given by solving Eq. 2:

$${\mu }_N=\frac{k_f}{\delta \left(k_a\frac{k_d}{\left[A\right]}+1\right)}.$$ (12)

Observe now that the average expression depends only on the ratio of binding strength to activator concentration (k_d/[A]), not on their individual values. So we introduce the following change of variables which allow us to keep the mean expression level constant and still explore effects of binding site strength: We let k²_c = k_d/[A] and k²_b = k_d/[A], substitute these variables into the transition matrix, and use Eqs. 4 and 5 to compute the COV, η,

$$\eta =\frac{\sqrt{\delta k_a{k}_f\left(k_c\left(\delta k_a{k}_c^2+\delta +k_f\right)+k_b{\left(k_a{k}_c^2+1\right)}^2\right)}}{k_a{k}_c{k}_f\sqrt{\delta k_c+k_a{k}_b{k}_c^2+k_b}}.$$ (13)

This may look a little messy given the simplicity of the model, which may be why Sánchez and colleagues do not present the results in variable form. However, with a little more thought we can derive some very useful intuition from this expression.

Our fair comparison requires that k_c is fixed, so that mean expression remains the same. The relative binding strength in this case is then captured by $k_b=\sqrt{k_d\left[A\right]}$. This quantity is large when binding is weak (concentration is high but dissociation is fast), and small when binding is strong. As k_b gets big (weak binding regime), η approaches

$$\underset{k_b\to \infty }{\text{lim}}\eta =\frac{1}{k_c}\sqrt{\frac{\delta k_a{k}_c^2+\delta }{k_a{k}_f}},$$ (14)

and as k_b gets small, binding strength is high, and η approaches

$$\underset{k_b\to 0}{\text{lim}}\eta =\frac{1}{k_c}\sqrt{\frac{\delta k_a{k}_c^2+\delta +k_f}{k_a{k}_f}},$$ (15)

from which it is apparent that the COV is largest when binding strength is strong, regardless of the other parameters chosen. Moreover, we can see from the difference of Eqs. 14 and 15 that the effect of strong binding on enhancing variability is greatest when the firing rate k_f is much greater than the degradation rate δ. An analogous treatment of the case of a repressor-controlled, two-state promoter shows that system is also more variable (noisy) the stronger the binding of the repressor (the solution I leave as an exercise to the reader).

At first pass, one may have expected the promoters with strong, stable binding of regulatory factors to be more precise, more controllable, and less noisy. Reality turns out to be the opposite, for reasons that become intuitive upon closer examination. Strong binding means the number of on-off binding cycles in a given window is less than for weaker binding, and, as such, strong sites have less opportunity to properly sample the concentration of protein factor around them. Rare events, such as the binding of an individual activator when the overall activator concentration is low and in the average cell the site is unbound, have large effects, because the activator will remain there for substantial time. This effect is clearly seen in the simulations of Sánchez et al. (63) for the single repressor case, reproduced in Fig. 3 C, using rate constants estimated from measurements of different operators from the Lac repressor.

Several additional principles can be derived from these analytic results, such as the typical noise differences for activator versus repressor-based regulation (63). But the true strength of the framework we have just practiced is in dealing with multistate systems, as we will begin to see with the next examples.

Example 2: effects of multistep reactions

Transcriptional initiation in eukaryotic cells requires the assembly of a substantial macromolecular preinitiation complex at the core promoter (65). In higher multicellular organisms, many genes also require a substantial array of transcription factors to be bound to associated regulatory elements (also known as enhancers) before the onset of productive transcription (see Spitz and Furlong (6) and Levine (7) for recent reviews). The two-state promoter model just considered provides a reasonable approximation for such systems only when one of these many binding events is much slower than all the others.

The alternative extreme, when there are many events all with near equal-transitions rates, has also been recently examined in the literature through a Markov approach. In 2008, Pedraza and Paulsson (66) argued that the stepwise progression through a series of inactive promoter states before reaching the final transcriptionally competent state could be a mechanism employed by metazoa to reduce transcription noise (see Fig. 3 D). Using a Markovian description of promoter assembly (and a different approach to solving this system than discussed above), they first demonstrate that the COV in mRNA copy number approximately scales with the sum of the variation in transcription cycle completion time and the variation in the number of mRNA produced per cycle:

$${\eta }_{\text{mRNA}}^2\approx \frac{b}{2\mu }\left({\eta }_{\tau }^2\right)+\frac{b}{2\mu }\left(b{\eta }_b^2+1\right).$$ (16)

Here b is the average and η²_b the squared COV for the number of mRNA molecules produced per cycle. The value μ is the mean number of mRNA molecules present in the cell at steady state. The derivation of this relation assumes that the bursts are essentially instantaneous and the size distribution of bursts is independent of the waiting time between bursts. They then explore how the COV for mRNA scales with the number of transitions N, in a linear cascade of binding events with identical forward rates (k) and zero backward rates (also known as an Erlang process). The cycle completion time (τ) for such a system follows the well-known γ-distribution (66–68)

$$f\left(\tau ,N,k\right)=\frac{k^N{\tau }^{N-1}{e}^{-k\tau }}{\text{Γ}\left(N\right)},$$ (17)

where Γ(N) is a gamma-function and in this case with integer N, becomes simply (N−1)!. This system has a mean waiting time N/k and variance N/k². The contribution of η_τ to the COV therefore decreases like $1/\sqrt{N}$ as the number of states is increased. Pedraza and Paulsson (66) termed this the “gestation approach” for reducing variability in gene expression, and postulated that the multistep assembly might be one mechanism evolved by eukaryotic cells for minimizing transcription noise.

However, most of the binding events that lead to the assembly of the eukaryotic preinitiation complex are reversible. Indeed, live imaging measurements with fluorescently tagged polymerase by Darzacq et al. (69) suggested that the backward reactions are faster than the forward assembly at several steps. Bel et al. (67) in 2010 used some basic tools from Markov theory to show in the case of reversible binding (and equivalent states), that the COV for the waiting time between transcription events (η²_τ) depends on the ratio of the forward and backward rates, K = k_b/k_f, as

$${\eta }_{\tau }^2=\frac{N-4K-\left(N+1\right)K^2+4\left(N-NK+1\right)K^{N+1}+K^{2N+2}}{{\left(N-NK+K\left(K^N-1\right)\right)}^2},$$ (18)

which goes like 1/N for k_f ≥ k_b, and if k_b ≫ k_f approaches 1. In the intermediate regime where the rates are balanced, k_b = k_f,

$${\eta }_{\tau }^2=\frac{2\left(1+N+N^2\right)}{3N\left(N+1\right)},$$ (19)

which is 1 for N = 1 and approaches 2/3 as N → ∞.

The corresponding mean transition time-scales as N,

$${\mu }_{\tau }=\frac{1}{k_f}\frac{N-\left(N+1\right)K+K^{N+1}}{{\left(1-K\right)}^2}.$$ (20)

Pedraza and Paulsson (66) and Bel et al. (67) arrive at these results through a long de novo treatment of the case of linear assembly (with added assumptions such as instantaneous bursting in the former case). A direct application of the equation for completion time for arbitrary assembly pathways from Eqs. 6 and 7 allows these results to be recovered in only a few lines of algebra (and without the need for additional approximation).

The observed reduction in cycle-completion time variation can be understood intuitively as the effect of averaging out the variation in the individual dwell times. The total passage time is the sum of all the independent (random) passage times, T = τ₁ + τ₂ + … + τ_n, which by the law of large numbers converges to the nonrandom time T = nE[τ] as n gets large. Adding backward reactions intuitively increases variation, by allowing some assembly paths to make repeated loops through the same step before reaching the final state. Similarly, having unequal forward transition rates will result in the slower rates dominating, damping the reduction in COV. Indeed, Aldous and Shepp (70) presented a proof in 1987 that any modification (adding loops, back-reactions, etc.) of the basic Erlang process can only increase the COV in first passage times. A corollary to this proof is the observation that the squared COV for first passage time from state A to state B through any Markov chain must have at least N = 1/η²_τ states between A and B (70). Moffitt et al. (71–73) termed this quantity n_min = 1/η²_τ in the interpretation of waiting time distributions for enzymatic processes, to highlight its intuitive association with this aspect of the underlying chemical state-space.

Although increasing the number of transitions decreases cycle completion time variation, it does not (yet) decrease the COV for mRNA counts. We can see this from Eq. 16, where the second term scales like 1/μ_mRNA, which from Bel et al. (67) we know decreases as 1/N even for the minimally variable Erlang architecture. Consequently, the COV in mRNA grows with N, even though the return time variation vanishes.

This result may seem to contradict the initial findings of Pedraza and Paulsson (66). We can invoke that a fair comparison requires the total average expression levels (μ_mRNA) to be equal and assume that this is achieved by accelerating the transition rates for the multistep reaction so the mean return times are equal (the approach taken by Pedraza and Paulsson in their simulations). The resulting gains in terms of mRNA variation are actually quite modest, which can be shown by evaluating Eq. 4 for the system shown in Fig. 3 D. That fact should also be intuitive, because at the minimal N = 1, the return time from the off-state has η²_τ = 1 and, if mRNA is produced as an exponential process in the active state, as works for simulations (or as required by the approach of Sánchez et al. (63)), η²_b = 1. So the term that can be reduced to zero was never a dominate source of noise to start with. In a regime where mean mRNA levels are low and the number of proteins made per mRNA is very high, even these modest differences will be amplified to something appreciable in terms of the COV for protein levels (as shown by the simulation in Pedraza and Paulsson (66)).

A much larger noise reduction is observed if one examines promoter architectures that do not assume an active state where mRNA is produced as a random process as proposed in previous work (62,63,66). If the initiation complex disassembles when the polymerase escapes the promoter to start transcription, there will be exactly one mRNA produced per cycle. Variability in firing cycle time will follow Eq. 18. Variation in the number of mRNA produced while the system remains in the firing cycle can be computed with Eqs. 9 and 10,

$${\eta }_{N\left(t\right)}^2\approx \frac{{\sigma }_{\tau }^2}{{\mu }_{\tau }t}={\eta }_{\tau }^2{\mu }_{\tau }.$$ (21)

From the above we know that η²_τ goes to zero as N goes to infinity as long as k_f > k_r, which means the transcription rate variability will also go to zero as the chain gets longer, provided the transition rates are just a little faster so that μ_τ remains finite.

A more realistic model of metazoan promoters contains two transcription loops, one in which the core elements of the preinitiation complex are assembled, and a second in which new polymerases bind, interact with the polymerase initiation complex, and are launched on their round of transcription (65) (see Fig. 3 E). This second loop describes the molecular details of bursting and it is one of the transitions in this loop that produces mRNA. This architecture can strongly benefit from gestation effects reducing expression noise if forward rates dominate. Provocatively, it has been shown experimentally that polymerase reinitiation involves multiple sequential, near-irreversible phosphorylation events of the polymerase-II (Pol II) C-terminal domain tail before productive elongation (65). Consequently this aspect of metazoan transcription may come close to realizing the γ-distributed noise, with a COV that scales as the inverse of number of phosphorylation events.

Example 3: when postponed choices are more reliable

Our final example of the utility of the finite Markov frameworks for modeling stochastic effects in gene expression comes from recent research aimed at understanding kinetic variability in transcription response times to new cell signals (64). In particular, we are interested in seeing if there is any fundamental difference between two distinct schemes of transcription regulation common in embryonic development (29,74). In the first scheme, tissue-specific factors bind distal regulatory DNAs (called enhancers) to allow subsequent binding of polymerase to the promoter and thence transcription initiation. Let us call this method “initiation regulation”. In the second scheme, all promoters may bind polymerase and initiate short transcripts, but productive elongation (leading to mature mRNA production) occurs only in cells that also have bound tissue-specific factors at the enhancer. Let us call this “elongation regulation”. These two models are shown schematically in Fig. 3 F, where the two gated-transition arrows indicate the transitions regulated in the two respective cases; these transitions occur at nonzero rate only when the enhancer is in its active state (state B). The corresponding Markov chains that track the binding occupancy at both the enhancer and promoter are shown in Fig. 3 G. To focus on the different promoter architectures, we follow Boettiger et al. (64) and considered only simple two-state enhancers.

Solving Eq. 9 for the variance in time to go from an unbound state to a productively transcribing one for both the initiation-regulated and elongation-regulated scheme results in two cumbersome expressions (both ratios of polynomials). However, both expressions are fully determined by the same set of parameters (see Fig. 3 G). Consequently we can rapidly compute the difference between the moments for each model (analytically) for a uniform sampling through all parameter space (normalized by the slowest reaction in the system). Surprisingly, the difference is not strictly positive or strictly negative. It is, however, highly skewed such that almost any parameter choice will result in more-synchronous activation by using the elongation-regulation scheme (see Fig. 3 H), consistent with experimental measurements from many genes of each type (29). Only within a highly constrictive range of parameters, when all the back reaction rates are especially small and enhancer activation particularly frequent, will initiation regulation be more synchronous. Equation 10 further shows that synchronous induction will result in less cell-cell variation in mRNA produced.

These results illustrate a few additional advantages in the ability to compute analytic solutions for noise properties. Even when the solutions and their limits do not come out to something easily interpretable, the ability to deeply explore parameter regimes and detect rare cases and qualitative shifts is not easily achieved with much slower stochastic simulations. One can also rapidly test the consequences of altering the reaction structure to identify which critical features contribute to the observed differences. It turns out the essential feature for the bias between these schemes is the irreversibility of promoter escape (k₃₂ = 0). When enhancer-promoter interactions are rare, elongation-regulated promoters accumulate in the initiated state, ready to launch at the first enhancer stimulation. In contrast, the initiation-regulated promoters require the enhancer activation before even beginning the preinitiation complex assembly. Some of these promoters will successfully proceed to the elongating state, but some will fall back into the inactive state and have to wait for a second enhancer interaction before restarting, increasing the spread of transition times (64). Consequently, regulating downstream of an irreversible transition can substantially reduce variability in reaction chains that are not highly processive.

Discussion

It is becoming apparent that for many multicellular animals, a greater portion of the genome is dedicated to controlling gene expression than to encoding genes themselves. A few studies now suggest that control of the probability distribution of expression states—and not just the average expression state—may be a central logical theme driving the organization and evolution of these numerous gene regulatory elements. Here we have reviewed a few experimental examples where differences in regulatory sequence have been linked to distribution properties of expression. We have also seen through theoretical analysis how mechanisms from a deterministic treatment appearing to be identical (e.g., weak binding sites for an abundant TF versus strong binding sites for a rare TF, or rapid multistep assembly versus one-step assembly) may differ substantially in terms of the distribution of expression among many cells, or regulating up or downstream of irreversible transitions.

To date, there have been very few studies that tackle the question of how the highly varied and complex regulatory mechanisms that have been found so far in multicellular systems effect the expected distribution of gene expression. Increasingly sophisticated approaches using both thermodynamic and heuristic models have identified important concepts of how regulatory architecture affects average expression properties (11–20), such as how cooperative binding interactions affect the sensitivity of the average transcription level to the concentration of an activating or repressing factor (14,15,17,19,20), or how different mechanisms of repression affect the integration of signals from multiple enhancers (20). How these architectures compare in terms of their ability to produce or minimize cell variation is, however, largely unexplored.

Yet the progress we have seen in stochastic modeling of transcription now presents a basic toolset with which these regulatory mechanisms, studied in a deterministic way, can be easily explored to determine their anticipated impact on transcription variability as well. Such effort may lead to new insights on why certain types of genes have evolved particular, common regulatory structures. It may also identify key limitations in our existing descriptions of regulation by finding where contemporary models, which successfully match expression averages from experiment, fail to capture experimentally observed properties of the higher moments. Such observations can help guide and refine further experimentation to lead to a more complete and predictive understanding of the mechanisms of gene regulation.

Although early applications of Markov modeling approaches have been restricted to specialists, the general results reviewed here make it straightforward to go from a complex biochemical cartoon of states, and transitions to analytic expressions for the moments of some key properties of the process, without an extended background in probability (just a little help from a computer algebra system). Further theoretical work may soon improve the reach of these methods. In particular, describing the steady-state distribution of mRNA without the need to make assumptions about reinitiation is likely a tractable though currently unsolved challenge.

Improving analytic tools will help develop new hypotheses and insights from the wealth of rapidly growing and improving experimental tools that measure variation in single cells. Together, these improvements may help us develop a deeper understanding of the regulatory roles played by noncoding sequences and the mechanisms by which precision and coherence are maintained in multicellular organisms.