Information-Optimal Transcriptional Response to Oscillatory Driving

Abstract

Intracellular transmission of information via chemical and transcriptional networks is thwarted by a physical limitation: The finite copy number of the constituent chemical species introduces unavoidable intrinsic noise. Here we solve for the complete probabilistic description of the intrinsically noisy response to an oscillatory driving signal. We derive and numerically verify a number of simple scaling laws. Unlike in the case of measuring a static quantity, response to an oscillatory signal can exhibit a resonant frequency which maximizes information transmission. Furthermore, we show that the optimal regulatory design is dependent on biophysical constraints (i.e., the allowed copy number and response time). The resulting phase diagram illustrates under what conditions threshold regulation outperforms linear regulation.

It has long been recognized [1] that the ability to measure biochemical quantities, e.g., concentrations, is intrinsically thwarted by the small copy numbers present at the scale of the cell. This observation has launched considerable experimental investigation as to how high-fidelity signal transmission can occur within single cells [2,3], along with an associated literature in mathematical and computational techniques for modeling such noisy information transmission [4–6]. From the perspective of biological design—either to understand the mechanisms which lead to observed biology or to create synthetic systems with desirable properties—these works investigate how regulatory elements which comprise biological systems function in the presence of intrinsic noise [7].

We consider the simplest probabilistic model of a regulatory element with a dynamical input, illustrated in Fig. 1(a), in which a single transcription factor (the “parent”) with copy number n is driven by an oscillatory creation rate f(t) = g + α cosωt and regulates the expression of a second species (the “child”) with copy number m; the regulation is modeled via the child’s creation rate q_n. This model captures the noisy downstream response to oscillation, e.g., the cell cycle, without limiting the results to a particular mechanism for generating oscillations (e.g., via cell division [8], repressive cycles [3], or activation-repression circuits [9]). We show how the optimal design—i.e., the choice of linear-vs-cooperative and up-vs-downregulation—is determined by the physical demands in terms of allowed copy number and response time. Furthermore, while our intuition from understanding how best to measure static signals suggests that slower response time is always more accurate [1], we illustrate how oscillatory driving leads to an information-optimal driving frequency and compute how this frequency depends on copy number.

FIG. 1.

(a) A transcription factor (the parent) with copy number n is driven by an oscillatory creation rate f(t) = g + α cosωt and regulates via q_n the expression of a second species (the child) with copy number m. (b),(c) Numerical verifications (data points) of analytic expressions (lines) derived in the small-information limit: Eqs. (9) (circles) and (13) top line (squares) and bottom line (triangles) in (b), and Eq. (14) for both up- (up triangles) and down-regulation (down triangles) in (c). Parameters are g = n₀ = 1 and c = 0.1 for (b), ω= 1 for (c), and α = ρ = Δ = 1 and q₀ = 0 for both, yielding small parameters |ν₁| ≤ 0.5 [Eq. (5)], |μ₁| ≤ 0.05 [linear; Eq. (12)], and |μ₁| ≤ 0.184 [threshold; Eq. (12)].

We use the “spectral method” [10], which exploits the linearity of the master equation ṗ_nm = −{ℒ_n[f(t)] + ρℒ_m[q_n]}p_nm, the equation of motion for the joint probability of observing n and m copies of the parent and child, respectively, by expanding its solution in terms of the natural eigenfunctions of the birth-death process with constant creation and decay. For a birth-death process expressed in terms of an arbitrary creation rate h and copy number s, the positive semidefinite operator ℒ_s acts as ℒ_s[h]p_s = sp_s − (s + 1)p_s+1 + hp_s − hp_s−1; time is normalized via the parent decay rate (in these units ρ is the child decay rate). To study dynamics we also Fourier transform in harmonics of the driving frequency ω:

$$p_{nm}(t)=\sum _{j=0}^{\infty }\sum _{k=0}^{\infty }\sum _{z=-\infty }^{\infty }{p}_{jk}^z\langle n|j\rangle \langle m|k\rangle e^{-iz\omega t},$$ (1)

where the parent and child eigenfunctions (or “spectral modes”) enjoy ℒ_n[g]〈n | j〉 = j〈n | j〉 and ℒ_m[q̄] × 〈m | k〉 = k〈m | k〉, respectively. Just as in Refs. [10,11], we introduce a gauge q̄ to define the basis; the analytic results below are independent of this choice. The master equation then becomes an algebraic relation among the expansion coefficients $p_{jk}^z$:

$$-i\omega z{p}_{jk}^z=-(j+\rho k)p_{jk}^z+\frac{\alpha }{2}\sum _{\pm }{p}_{j-1,k}^{z\pm 1}-\rho \sum _{j\prime }{\mathrm{\Delta }}_{jj\prime }{p}_{j\prime ,k-1}^z,$$ (2)

where Δ_jj′ = ∑_n〈j | n〉(q̄ − q_n)〈n | j′〉. Algorithmically we (i) initialize with $G_{00}^z={\delta }_{z0}$ (set by normalization), (ii) exploit the subdiagonality in k, and (iii) for each k, exploit the subdiagonality in j; no matrices need be inverted.

Efficient computation of p_nm(t) allows optimization of the mutual information I(ϕ, n) between the input variable—the phase ϕ = ωt of the driving oscillation—and the output variable—the copy number of either the parent or the child:

$$I(\varphi ,n)={\int }_0^{2\pi }d\varphi \sum _{n}p(n|\varphi )p(\varphi )\text{log}\frac{p(n|\varphi )}{p_n^0},$$ (3)

where p(n | ϕ) ≡ p_n(t), p(ϕ) = 1/2π, and $p_n^0={\int }_0^{2\pi }d\varphi p(n|\varphi )p(\varphi )$ is the time-averaged distribution [12].

The dynamics of the parent can be found exactly by using either the method of characteristics or spectral decomposition [13]: p_n(t) is a Poisson distribution with time-dependent mean ν(t) = ν₀ + 2|ν₁| cos(ωt − γ), with

$${\nu }_0=g,$$ (4)

$$|{\nu }_1|=\frac{\alpha }{2\sqrt{1+{\omega }^2}},$$ (5)

and phase shift γ = tan⁻¹ω. The Fourier transform coefficients $p_n^z={\int }_0^{2\pi }d\varphi e^{iz\varphi }p(n|\varphi )/(2\pi )$ are then computed by expanding the exponential in p(n | ϕ) and identifying the modes [13]:

$$p_n^z=e^{iz\gamma }\sum _j\frac{{|{\nu }_1|}^{2j+|z|}}{j!(j+|z|)!}\langle n|2j+|z|\rangle .$$ (6)

In the limit of weak (α ≪ 1) or fast (ω ≫ 1) driving, an approximation for I(ϕ, n) may be obtained by expanding in the small parameter |ν₁|. We first express Eq. (3) in terms of the Fourier transform $p(n|\varphi )={\sum }_z{p}_n^z{e}^{-iz\varphi }$:

$$I(\varphi ,n)=\sum _{n,z}{p}_n^z{\int }_0^{2\pi }\frac{d\varphi }{2\pi }{e}^{-iz\varphi }\text{log}(1+\sum _{z\prime \ne 0}\frac{p_n^{z\prime }}{p_n^0}{e}^{-iz\prime \varphi }).$$ (7)

Then we note that, for small |ν₁|, Eq. (6) is dominated by the j = 0 term, i.e., $p_n^z\approx e^{iz\gamma }|{\nu }_1{|}^{|z|}\langle n||z|\rangle /|z|!$. Since this is itself small for z ≠ 0, we expand the log in Eq. (7) as log(1 + x) = x − x²/2 + ⋯ for small x. The first two terms in the log expansion [13] contain the leading-order behavior in $p_n^z$ (proportional to $p_n^1{p}_n^{-1}=|p_n^1{|}^2$); employing ${\int }_0^{2\pi }d\varphi e^{i(z-z\prime )\varphi }=2\pi {\delta }_{zz\prime }$ one obtains

$$I(\varphi ,n)\approx \sum _n\frac{{|p_n^1|}^2}{p_n^0}\approx {|{\nu }_1|}^2\sum _n\frac{{\langle n|1\rangle }^2}{\langle n|0\rangle }=\frac{{|{\nu }_1|}^2}{{\nu }_0}$$ (8)

$$=\frac{{\alpha }^2}{4g}\frac{1}{1+{\omega }^2},$$ (9)

where the second to last step uses 〈n | 0〉 = e^−ggⁿ/n! and 〈n | 1〉 = 〈n | 0〉(n − g)/g [11] to evaluate the sum. Equation (8) shows that mutual information asymptotes to the square of the amplitude of the oscillation over the mean. Equation (9) scales like ω⁰ at low frequency and ω⁻² at high frequency, demonstrating that the parent acts as a low-pass filter of information; this scaling is numerically verified in Fig. 1(b).

Although the child distribution p_m(t) is not analytically accessible in general, its mean μ(t) is exactly calculable: Summing the master equation over both indices against m and Fourier transforming yields μ(t) = ∑_zμ_ze^−izωt, where

$${\mu }_z=\frac{1}{1-iz\omega /\rho }\sum _n{q}_n{p}_n^z.$$ (10)

In the limit of weak regulation (i.e., when q_n is near constant) we may approximate p_m(t) as a Poisson distribution with oscillatory mean parameterized by the first and second Fourier modes of the exact mean, i.e., μ(t) ≈ μ₀ ± 2|μ₁| cos(ωt − θ) for up- (down-) regulation, where θ = phase(μ₁) = tan⁻¹ ω/ρ + tan⁻¹ω. Under this approximation, as in Eq. (8), the information between the phase of the driving oscillation and the copy number of the child is the oscillation amplitude squared over the mean, i.e., I(ϕ, m) = |μ₁|²/μ₀ for small |μ₁|.

To compare the transmission properties of both non- and highly cooperative regulation, we study both the linear function q_n = q₀ + cn and the threshold function q_n = q₀ + Δχ (n ∈ Ω_±), respectively, where χ is a characteristic function equal to 1 when n is in the set Ω₊ = {n > n₀} (up-regulation) or Ω₋ = {n ≤ n₀} (down-regulation) and 0 otherwise. In these cases, the mean of the child distribution oscillates about the point

$${\mu }_0=\sum _n{q}_n{p}_n^0=q_0+\{\begin{array}{cc}cg\hfill & \text{linear}\hfill \\ \mathrm{\Delta }{p}_{\pm }^0\hfill & \text{threshold}.\hfill \end{array}$$ (11)

Here the linear result exploits the fact that the mean of the time-averaged parent distribution $p_n^0$ is g (which can be seen from the relationship between distribution moments and spectral modes [13]). In the threshold result we define $p_{\pm }^0\equiv {\sum }_{n\in {\mathrm{\Omega }}_{\pm }}{p}_n^0={\pi }_{\pm }\pm {\sum }_{j>0}|{\nu }_1{|}^{2j}\langle n_0|2j-1\rangle /{(j!)}^2\approx {\pi }_{\pm }$, where π_± ≡ ∑_n∈Ω±〈n | 0〉; the second to last step exploits the result ∑_n∈Ω±〈n | j〉 = ±〈n₀ | j − 1〉 for j > 0 [13], and the last step takes j = 0 in the small |ν₁| limit. The amplitude of the oscillation of the child mean is

$$|{\mu }_1|=\frac{{\sum }_n{q}_n|p_n^1|}{\sqrt{1+{(\omega /\rho )}^2}}=\frac{1}{\sqrt{1+{(\omega /\rho )}^2}}\times \{\begin{array}{c}c|{\nu }_1|\hfill \\ \mathrm{\Delta }|p_{\pm }^1|,\hfill \end{array}$$ (12)

where once more the linear result (top) uses the relationship between moments and modes and in the threshold result (bottom) we define and approximate $|p_{\pm }^1|\equiv {\sum }_{n\in {\mathrm{\Omega }}_{\pm }}|p_n^1|={\sum }_j|{\nu }_1{|}^{2j+1}\langle n_0|2j\rangle /[j!(j+1)!]\approx |{\nu }_1|\langle n_0|0\rangle$ [13]. Equations (11) and (12) yield the following approximations for linear (top) and threshold (bottom) regulation:

$$I(\varphi ,m)\approx \frac{gI(\varphi ,n)}{1+{(\omega /\rho )}^2}\times \{\begin{array}{c}{c}^2/(q_0+cg)\hfill \\ {\mathrm{\Delta }}^2{\langle n_0|0\rangle }^2/(q_0+\mathrm{\Delta }{\pi }_{\pm }),\hfill \end{array}$$ (13)

where I(ϕ, n) is as in Eq. (9). Equation (13) shows that the child I(ϕ, m) is a sharper low-pass filter than the parent I(ϕ, n), falling off like ω⁻⁴ at high frequency instead of ω⁻²; this scaling is verified numerically in Fig. 1(b). We note that since t, n, and m are not Markov related, i.e., p(m | t) ≠ ∑_np(m | n)p(n | t), Eq. (13) is not bound by the data-processing inequality [14], and it is possible for the child to transmit more information than the parent, i.e., I(ϕ, m) > I(ϕ, n) (e.g., for linear regulation with ω → 0, q₀ = 0, and c > 1), which we have verified numerically [13].

Equation (13) also offers analytic intuition about the optimal placement of the parent distribution with respect to a threshold regulation function. The derivative of Eq. (13) (bottom) with respect to g vanishes at g^*, the information-optimal mean of the parent distribution:

$$g^{*}=\frac{n_0}{1\pm \mathrm{\Delta }\langle n_0|0\rangle /[2(q_0+\mathrm{\Delta }{\pi }_{\pm })]}.$$ (14)

As verified in Fig. 1(c), Eq. (14) shows that the parent distribution is shifted below the threshold for up-regulation and above the threshold for down-regulation. These shifts account for the ability of up-regulation to outperform down-regulation when the copy number is highly constrained (see Fig. 3), an effect we observed previously [10] when numerically optimizing steady-state information between the first and last species in a regulatory cascade.

FIG. 3.

Phase diagram showing information-optimal regulation among linear, up-threshold, and down-threshold regulation in the space of driving period and copy number. Phases are separated by solid lines and marked by sample regulation functions (insets). Also shown is (2π over) resonant frequency ω^* [see point c in Fig. 2(a)] as a function of copy number for both up- (dashed line) and down-regulation (dot-dashed line).

The above analytic approximations offer guidance during a full optimization of I(ϕ, m) via numerical integration of Eq. (3). As suggested by Eq. (13), numerical optimization confirms that I(ϕ, m) increases when (i) the amplitude of the driving oscillation is maximal (α = g) and (ii) the dynamic range is maximal (q₀ = 0 and c → ∞ or Δ → ∞). The slope c or discontinuity Δ, however, is constrained by the average copy number of the child μ₀ [Eq. (11)]. Therefore, for a fixed driving frequency and fixed total average copy number N = 〈n〉 + 〈m〉 = g + μ₀, we optimize over the single parameter g by setting α = g, q₀ = 0, and c = μ₀/g = (N − g)/g or $\mathrm{\Delta }={\mu }_0/p_{\pm }^0=(N-g)/p_{\pm }^0$; additionally, we set ρ = 1 for equal parent and child decay rates [10]. For threshold regulation, an optimization over g is done at each of a set of values of the (discrete) parameter n₀, and the global optimum is selected.

At low copy number, optimal information I^*(ϕ, m) behaves as one might expect from the small-oscillation limit [Eq. (13)]: It decreases monotonically with frequency [Fig. 2(a), bottom curves]. At high copy number, I^*(ϕ, m) decreases monotonically with frequency for linear regulation but for threshold regulation exhibits a maximum at a resonant frequency [Fig. 2(a), top curves]. Careful examination of the child distribution at different phases [Figs. 2(b)–2(d)] (or simply its mean [13]) reveals the origin of this maximum as follows. As the parent oscillates about the threshold, the child distribution is switchlike, with two long-lived switch states centered at the threshold’s low and high rates and brief intermediate states in between. At high copy number, the threshold rates are far apart (one is zero and the other is large), making the switch states well separated and transmitting (slightly more than, due to the intermediate states) the strict switch limit [10] of I^*(ϕ, m) ~ 1 bit. For slow oscillations the intermediate states are symmetric [Fig. 2(b)], but for faster oscillations there is a lag in transitioning from one switch state to the other, making the intermediate states distinguishable [Fig. 2(c)] and transmitting more information about phase. Thus the resonant frequency ω^* balances the slowness required to avoid time averaging [Fig. 2(d)] with the speed required for distinguishable intermediate states. The high-copy-number behavior described here has been verified for N as high as 50 [13].

FIG. 2.

(a) At high copy number (N = 20, top curves) the optimal information I^*(ϕ, m) exhibits a resonant driving frequency (point c) for up- (dashed line) and down-threshold (dot-dashed line) regulation but not for linear (solid line) regulation; at low copy number (N = 2, bottom curves), there is no resonant frequency, and slowest (ω → 0) is best. (b)–(d) correspond to marked points in (a) and show the optimal child distribution for down-threshold regulation at phases Θ ≡ ωt − θ = 0, π/2, π, and 3π/2 [legend in (d) applies to (b)–(d)]: (b) Slow driving produces switchlike behavior, with long-lived low- (Θ = 0) and high-copy-number (Θ = π) states and brief intermediates (Θ = π/2, 3π/2) in between; (c) moderate driving produces switchlike behavior with distinguishable intermediates, transmitting the most information; and (d) fast driving time-averages the parent, and thus the child, distribution.

The phase diagram (Fig. 3) shows the information-optimal regulation among linear, up-threshold, and down-threshold regulation across a range of copy numbers N and periods T = 2π/ω. Linear regulation is best when both N and T are large, ultimately surpassing threshold regulation’s limit of ~1 bit. Down-threshold regulation is best at values of T near 2π/ω^* because its intermediate states are more distinguishable (i.e., have a larger Jensen-Shannon divergence) than those of a similarly parameterized up threshold. Up-threshold regulation is best at low N due to its tendency, as discussed above [Eq. (14)] and in Ref. [10], to require fewer proteins to match the transmission across a similarly parameterized down threshold.

Finally, we turn to comparing our predictions with real biological parameters. In simpler systems, where decay rates are bounded from below by dilution from cell division, the ratio of the cell cycle frequency to a species’ decay rate is ω ≤ 2π/ ln2 ≈ 9. Our results predict that slightly smaller ratios are information-optimal (ω^* ~ 2π/10 ≈ 0.6 at high N; see Fig. 3), as in, e.g., yeast glycolysis oscillations (ω ~ 0.5 [15]) or bacterial circadian rhythms, when, through either active degradation [16] or rapid cell division [17], decay rates are several times larger than the circadian frequency, making ω order 1.

The low-pass behavior revealed in Eqs. (9) and (13) is consistent with our intuition from measuring static quantities in the presence of intrinsic noise [1]: The longer we wait, the more accurate our estimate. However, in the presence of an oscillatory driving signal, we find that threshold regulation can lead to an information-optimal frequency, and waiting longer is not necessarily the optimal strategy. Furthermore, we have shown that, at a fixed allowed copy number and allowed integration time, one may find that a different regulation strategy (linear, threshold up-regulation, or threshold down-regulation) is optimal for responding to oscillatory driving. Absent from this analysis are intriguing questions such as whether the diversity of other network topologies observed in nature—including cascades and feedback circuits—are consistent with these observations.