Information-theoretic analysis of the directional influence between cellular processes

Abstract

Inferring the directionality of interactions between cellular processes is a major challenge in systems biology. Time-lagged correlations allow to discriminate between alternative models, but they still rely on assumed underlying interactions. Here, we use the transfer entropy (TE), an information-theoretic quantity that quantifies the directional influence between fluctuating variables in a model-free way. We present a theoretical approach to compute the transfer entropy, even when the noise has an extrinsic component or in the presence of feedback. We re-analyze the experimental data from Kiviet et al. (2014) where fluctuations in gene expression of metabolic enzymes and growth rate have been measured in single cells of E. coli. We confirm the formerly detected modes between growth and gene expression, while prescribing more stringent conditions on the structure of noise sources. We furthermore point out practical requirements in terms of length of time series and sampling time which must be satisfied in order to infer optimally transfer entropy from times series of fluctuations.

Introduction

Quantifying information exchange between variables is a general goal in many studies of biological systems because the complexity of such systems prohibits mechanistic bottom-up approaches. Several statistical methods have been proposed to exploit either the specific dependence of the covariances between input and output variables with respect to a perturbation applied to the network [1], or the information contained in 3-point correlations [2]. These methods are potentially well suited for datasets obtained from destructive measurements, such as RNA sequencing or immunohistochemistry.

However, none of these methods exploits the information contained in time-lagged statistics, which is provided for instance by non-destructive measurements obtained from time-lapse microscopy of single cells. Such experimental data should be quite relevant to understand functional relationships since they merely reflect the time delays present in the dynamics of the system. Time-delayed cross-correlations between gene expression fluctuations have indeed been shown to discriminate between several mechanistic models of well characterized genetic networks [3]. However, such methods become difficult to interpret in the presence of feedback.

This situation is illustrated in reference [4] where the fluctuations in the growth rate and in the expression level of metabolic enzymes have been measured as a function of time by tracking single cells of E. coli with time-lapse microscopy. The interplay between these variables has been characterized using cross-correlations as proposed in [3]. To circumvent the difficulty of discriminating between many complex and poorly parametrized metabolic models, the authors reduced functional relations to effective linear responses with a postulated form of effective couplings.

In the present work, we instead use a time-lagged and information-based method to analyze the interplay between the two fluctuating variables. A crucial feature in this method is that it is model-free and it is able to disentangle the two directions of influence between the two variables, unlike the cross-correlations discussed above. This type of approach was first proposed by Granger [5] in the field of econometrics and found applications in a broader area. More recently, transfer entropy [6], which is a non-linear extension of Granger causality, has become a popular information-theoretic measure to infer directional relationships between jointly dependent processes [7]. It has been successfully applied to various biomedical time series (see for instance [8]) and used extensively in the field of neurobiology, as shown in Ref. [9] and in references therein. This is the tool that will be used in this work.

The plan of this paper is as follows. We first introduce two measures of information dynamics, transfer entropy (TE) and information flow (IF). We then illustrate our numerical method on a well controlled case, namely a simple linear Langevin model, and show that we can properly estimate these quantities from the generated time series. We then analyze experimental data on the fluctuations of metabolism of E. coli taken from Ref. [4]. We provide analytical expressions for the transfer entropy and information flow rates for the model proposed in that reference. After identifying a divergence in one TE rate as the sampling time goes to zero, we introduce a simplified model which is free of divergences while still being compatible with the experimental data. We conclude that the inference of information-theoretic dynamical quantities can be helpful to build physically sound models of the various noise components present in chemical networks.

Information theoretic measures

Unlike the mutual information I(X : Y) that only quantifies the amount of information exchanged between two random variables X and Y as defined in the section on Methods, the transfer entropy (TE) is an asymmetric measure that can discriminate between a source and a target [6]. Consider two sampled time series {‥x_i−1, x_i, x_i+1‥} and {‥y_i−1, y_i, y_i+1‥}, where i is the discrete time index, generated by a source process X and a target process Y. The transfer entropy T_X→Y from X to Y is a conditional, history-dependent mutual information defined as

$$\begin{array}{ccc}\hfill T_{X\to Y}& =& \sum P(y_{i+1},{\mathit{y}}_i^{\left(k\right)},{\mathit{x}}_i^{\left(l\right)})ln\frac{P\left(y_{i+1}\right|{\mathit{y}}_i^{\left(k\right)},{\mathit{x}}_i^{\left(l\right)})}{P\left(y_{i+1}\right|{\mathit{y}}_i^{\left(k\right)})},\hfill \\ & =& \sum _i[H\left(y_{i+1}\right|{\mathit{y}}_i^{\left(k\right)})-H\left(y_{i+1}\right|{\mathit{y}}_i^{\left(k\right)},{\mathit{x}}_i^{\left(l\right)})]\hfill \end{array}$$ (1)

where ${\mathit{y}}_i^{\left(k\right)}=\{y_{i-k+1},\cdots ,y_i\}$ and ${\mathit{x}}_i^{\left(l\right)}=\{x_{i-l+1},\cdots ,x_i\}$ denote two blocks of past values of Y and X of length k and l respectively, $P(y_{i+1},{\mathit{y}}_i^{\left(k\right)},{\mathit{x}}_i^{\left(l\right)})$ is the joint probability of observing $y_{i+1},{\mathit{y}}_i^{\left(k\right)},{\mathit{x}}_i^{\left(l\right)}$, and $P\left(y_{i+1}\right|{\mathit{y}}_i^{\left(k\right)},{\mathit{x}}_i^{\left(l\right)}),P\left(y_{i+1}\right|{\mathit{y}}_i^{\left(k\right)})$ are conditional probabilities. In the second line, H(.|.) denotes the conditional Shannon entropy (see Section on Methods for definition). In the first equation, the summation is taken over all possible values of the random variables $y_{i+1},{\mathit{y}}_i^{\left(k\right)},{\mathit{x}}_i^{\left(l\right)}$ and over all values of the time index i.

To put it in simple terms, T_X→Y quantifies the information contained from the past of X about the future of Y, which the past of Y did not already provide [7, 8]. Therefore, it should be regarded as a measure of predictability rather than a measure of causality between two time-series [10]. For instance, when ${\mathit{x}}_i^{\left(l\right)}$ does not bring new information on y_i+1, then $P\left(y_{i+1}\right|{\mathit{y}}_i^{\left(k\right)},{\mathit{x}}_i^{\left(l\right)})=P\left(y_{i+1}\right|{\mathit{y}}_i^{\left(k\right)})$ and the transfer entropy vanishes because the prediction on y_i+1 is not improved. With a similar definition for T_Y→X, one can define the net variation of transfer entropy from X to Y as ΔT_X→Y ≡ T_X→Y − T_Y→X. The sign of ΔT_X→Y informs on the directionality of the information transfer.

The statistics required for properly evaluating the transfer entropy rapidly increases with k and l, which in practice prohibits the use of large values of k and l. The most accessible case thus corresponds to k = l = 1, which we denote hereafter as ${\overline{T}}_{X\to Y}$. This quantity is then simply defined as

$$\begin{array}{c}\hfill {\overline{T}}_{X\to Y}=\sum _i[H\left(y_{i+1}\right|y_i)-H\left(y_{i+1}\right|y_i,x_i)],\end{array}$$ (2)

When the dynamics of the joint process {X, Y} is Markovian, one has $P\left(y_{i+1}\right|{\mathit{y}}_i^{\left(k\right)},{\mathit{x}}_i^{\left(l\right)})=P\left(y_{i+1}\right|y_i,x_i)$ and since $H\left(y_{i+1}\right|{\mathit{y}}_i^{\left(k\right)})\le H\left(y_{i+1}\right|y_i)$ one has ${\overline{T}}_{X\to Y}\ge T_{X\to Y}$ (see Ref. [11]). Therefore, ${\overline{T}}_{X\to Y}$ represents an upper bound on the transfer entropy. In the case of stationary time series, which is the regime we consider in this work, it is natural to also introduce the TE rate

$$\begin{array}{ccc}\hfill {\overline{\mathcal{T}}}_{X\to Y}& =& \underset{\tau \to 0}{lim}\frac{H\left(y_{t+\tau }\right|y_t)-H\left(y_{t+\tau }\right|x_t,y_t)}{\tau }\hfill \\ & =& \underset{\tau \to 0}{lim}\frac{I(y_{t+\tau }:y_t,x_t)-I(y_{t+\tau }:y_t)}{\tau },\hfill \end{array}$$ (3)

where the continuous time variable t replaces the discrete index i. In practice ${\overline{\mathcal{T}}}_{X\to Y}\simeq {\overline{T}}_{X\to Y}/\tau$, but only for sufficiently small time step τ.

The most direct strategy to evaluate Eq (1) would be to construct empirical estimators of the probabilities from histograms of the data. Although this procedure works well for evaluating other quantities, for instance the entropy production in small stochastic systems [12], it completely fails in the case of transfer entropy. Indeed, such a method leads to a non-zero TE even between uncorrelated signals, due to strong biases in standard estimators based on data binning. In order to overcome this problem, we used the Kraskov-Stögbauer-Grassberger (KSG) estimator which does not rely on binning, as implemented in the software package JIDT (Java Information Dynamics Toolkit) [13]. Using estimators of this kind is particularly important for variables that take continuous values.

In the following, the inference method will be applied to time series generated by diffusion processes. It will then be interesting to compare the TE rate ${\overline{\mathcal{T}}}_{X\to Y}$ to another measure of information dynamics, the so-called information flow [14–16] (also dubbed learning rate in the context of sensory systems [11, 17]), which is defined as the time-shifted mutual information [18]

$$\begin{array}{c}\hfill {\mathcal{I}}_{X\to Y}^{flow}=\underset{\tau \to 0}{lim}\frac{I(y_t:x_t)-I(y_t:x_{t+\tau })}{\tau }.\end{array}$$ (4)

In the special case where the two processes X and Y experience independent noises (the system is then called bipartite) [15], one has the inequality ${\mathcal{I}}_{X\to Y}^{flow}\le {\mathcal{T}}_{X\to Y}$ [17], which in turn implies that

$$\begin{array}{c}\hfill {\mathcal{I}}_{X\to Y}^{flow}\le {\overline{\mathcal{T}}}_{X\to Y}\end{array}$$ (5)

when the joint process is Markovian. Observing a violation of this inequality is thus a strong indication that the noises on X and Y are correlated. As will be seen later, this is indeed the situation in biochemical networks, due the presence of the so-called extrinsic noise generated by the stochasticity in the cell and in the cell environment [19] which acts on all chemical reactions within the cell, and thus induces correlations.

Results

Test of the inference method on a Langevin model

In order to benchmark our inference method and perform a rigorous test in a controlled setting, we first applied it on times series generated by a simple model for which the transfer entropy and the information flow can be computed analytically. The data were obtained by simulating the two coupled Langevin equations

$$\begin{array}{ccc}\hfill m\dot{v}& =& -\gamma v-ay+\xi ,\hfill \\ \hfill {\tau }_r\dot{y}& =& v-y+\eta \hfill \end{array}$$ (6)

that describe the dynamics of a particle of mass m subjected to a velocity-dependent feedback that damps thermal fluctuation [16, 20, 21] (in these equations, the dependence of the variables on the time t is implicit). Here, ξ(t) is the noise generated by the thermal environment with viscous damping γ and temperature T, while η(t) is the noise associated with the measurement of the particle’s velocity v(t). The two noises are independent and Gaussian with zero-mean and variances ⟨ξ(t)ξ(t′)⟩ = 2γk_BTδ(t − t′) and ⟨η(t)η(t′)⟩ = σ² δ(t − t′). a is the feedback gain and τ_r is a time constant.

The two Langevin equations were numerically integrated with the standard Heun’s method [22] using a time step Δt = 10⁻³, and the transfer entropy in the steady state was estimated from 100 time series of duration t = 2000 with a sampling time (i.e., the time between two consecutive data points) τ = Δt. We first checked that the TE in the direction Y → V does vanish in the absence of feedback, i.e. for a = 0, whereas it is non-zero as soon as a > 0. We then tested the influence of the measurement error σ² for a fixed value of the gain a. As can be seen in Fig 1, T_V→Y diverges as σ² → 0, a feature that will play an important role in our discussion of the model for the metabolic network. In the figure, the color of the symbols correspond to three different values of the parameter k which represents the history length in the definition of the transfer entropy (see Eq (1)). One can see that the estimates of T_V→Y for k = 1 are in very good agreement with the theoretical prediction for ${\overline{T}}_{V\to Y}$ (upper solid line). Moreover, the estimates decrease as k is increased from 1 to 5, and one can reasonably expect that the theoretical value of T_V→Y (lower solid line) computed in Ref. [16] and given by Eq (21) in the section on Methods would be reached in the limit k → ∞.

Fig 1. Transfer entropy T_Y→V for the feedback model governed by Eq (6) as a function of the noise intensity σ² for k = 1 (blue circles), k = 3 (green circles) and k = 5 (red circles).

The parameter l present in the definition of Eq (1) is fixed to 1. The lower red (resp. upper blue) solid line represents the value of T_Y→V (resp. ${\overline{T}}_{Y\to V}$) obtained by multiplying the theoretical rate ${\mathcal{T}}_{Y\to V}$ (resp. ${\overline{\mathcal{T}}}_{Y\to V}$) given by Eq (21) (resp. Eq (23) by the sampling time τ = 10⁻³. The parameters of the model are T = 5, γ = m = 1, τ_r = 0.1, and a = 8.

Finally, by estimating the information flow and the transfer entropy, we checked that inequality (5) holds, as a result of the independence of the two noises ξ and η (see section on Methods).

Analysis of stochasticity in a metabolic network

Experimental time series

We are now in position to analyze the fluctuations in the metabolism of E. coli at the single cell level obtained in Ref. [4] using the information-theoretic notions introduced and tested in the previous section. Since there are a multitude of reactions and interactions involved in the metabolism of E. coli, a complete mechanistic description is not feasible, and our model-free inference method has a crucial advantage. In Ref. [4], the length of the cells was recorded as a function of time using image analysis, and the growth rate was then obtained by fitting this data over subparts of the cell cycle. In the same experiment, the fluorescence level of GFP, which is co-expressed with growth enzymes LacY and LacZ was recorded. Three set of experiments were carried out corresponding to three levels of an inducer IPTG: low, intermediate and high.

The two time series have a branching structure due to the various lineages, which all start from a single mother cell as shown in Fig 2. The experimental data thus come in the form of a large ensemble of short times series which represent a record of all the cell cycles. There are about ∼3000 time series, with 2 to 8 measurement points in each of them which are represented as colored points in Fig 2. In order to correctly estimate the transfer entropy from such data, we have analyzed the multiple time series as independent realizations of the same underlying stochastic process. For the present analysis, we fix the history length parameters k and l to the value k = l = 1, which means that we focus on $\overline{T}$ rather than T. We infer the values of $\overline{T}$ in the two directions, from growth (denoted μ) to gene expression (denoted E) and vice versa. The results obtained for the three concentrations of IPTG are represented in Table 1. The negative value of ${\overline{T}}_{\mu \to E}$ which is found in the intermediate case is due to the numerical inference method and should be regarded as a value which cannot be distinguished from zero.

Fig 2. Pedigree tree representing the evolution of the colony of E. coli. studied in Ref. [4].

The splitting of the branches corresponds to cell division events, each colored point is associated to a measurement of a single cell and the colors represent the growth rates as shown in the bar in the lower part of the figure.

Table 1. Inferred values of the transfer entropies in the directions E → μ and μ → E, and the difference $\Delta {\overline{T}}_{E\to \mu }={\overline{T}}_{E\to \mu }-{\overline{T}}_{\mu \to E}$ for low, medium and high concentrations of IPTG based on the data of ref. [4].

The TE are given in nats.

Conc. of IPTG	Low	Intermediate	High
${\overline{T}}_{E\to \mu }$	2.35 ⋅ 10−2	1.37 ⋅ 10−2	1.06 ⋅ 10−3
${\overline{T}}_{\mu \to E}$	2.16 ⋅ 10−2	−4.08 ⋅ 10−3	9.94 ⋅ 10−3
$\Delta {\overline{T}}_{E\to \mu }$	1.84 ⋅ 10−4	1.78 ⋅ 10−2	−8.88 ⋅ 10−3

Based on this analysis, we conclude that the influence between the variables is directed primarily from enzyme expression to growth in the low and intermediate IPTG experiments, while it mainly proceeds in the reverse direction in the high IPTG experiment. Such results are in line with the conclusions of Ref. [4] based on the measured asymmetry of the time-lagged cross-correlations. Moreover, the present analysis provides an estimate of the influence between the two variables separately in the two directions from E to μ and from μ to E. In particular, we observe for the low experiment that the values of TE in the two directions are of same order of magnitude, whereas in the intermediate experiment the TE from E to μ is larger, a feature which could not have been guessed from measured time delays.

Theoretical models

We now turn to the analysis of the model proposed in Ref. [4] to account for the experimental data. The question we ask is whether the model correctly reproduces the above results for the transfer entropies, in particular the change in the sign of $\Delta {\overline{T}}_{E\to \mu }$ for the high concentration of IPTG.

The central equation of the model describes the production of the enzyme as

$$\begin{array}{c}\hfill \dot{E}=p-\mu ·E,\end{array}$$ (7)

where E is the enzyme concentration, p its production rate, and μ the rate of increase in cell volume. Although the function p is typically non-linear, its precise expression is irrelevant because (7) is linearized around the stationary point defined by the mean values E = E₀ and μ = μ₀. This linearization then yields

$$\begin{array}{c}\hfill \delta \dot{E}=\delta p-\delta \mu E_0-{\mu }_0\delta E,\end{array}$$ (8)

in terms of perturbed variables δX(t) = X(t) − X₀, where X₀ denotes the mean of X.

The model of Ref. [4] is essentially phenomenological in nature because it approximates the noises as Gaussian processes. Although this approximation is often done in this field, it may not always hold since fluctuations due to low copy numbers are generally not Gaussian [23]. In any case, the model contains three Gaussian noises: N_G is a common component while N_E and N_μ are component specific to E and μ. These noises are assumed to be independent Ornstein-Uhlenbeck noises with zero mean and autocorrelation functions $\langle N_i\left(t\right)N_i\left(t^{\prime }\right)\rangle ={\eta }_i^2{e}^{-{\beta }_i|t-t^{\prime }|}$ (i = E, μ, G). As commonly done, the three Ornstein-Uhlenbeck noises are generated by the auxiliary equations

$$\begin{array}{c}\hfill {\dot{N}}_i=-{\beta }_i{N}_i+{\xi }_i,\end{array}$$ (9)

where the ${\xi }_i^{\prime }s$ are zero-mean Gaussian white noises satisfying $\langle {\xi }_i\left(t\right){\xi }_j\left(t^{\prime }\right)\rangle ={\theta }_i^2\delta (t-t^{\prime }){\delta }_{ij}$ with ${\theta }_i={\eta }_i\sqrt{2{\beta }_i}$. Introducing the constant logarithmic gains T_XY that represent how a variable X responds to the fluctuations of a source Y, the equations of the model read [4]

$$\begin{array}{ccc}\hfill \frac{\delta p}{E_0{\mu }_0}& =& T_{EE}\frac{\delta E}{E_0}+T_{EG}{N}_G+N_E,\hfill \\ \hfill \frac{\delta \mu }{{\mu }_0}& =& T_{\mu E}\frac{\delta E}{E_0}+T_{\mu G}{N}_G+N_{\mu },\hfill \end{array}$$ (10)

where specifically T_Eμ = −1 and T_μG = 1. Then, eliminating δp from Eqs (8) and (10), one obtains the coupled equations

$$\begin{array}{ccc}\hfill \dot{x}& =& {\mu }_0[(T_{EE}-1)x+T_{E\mu }y+T_{EG}{N}_G+N_E]\hfill \\ \hfill y& =& T_{\mu E}x+T_{\mu G}{N}_G+N_{\mu },\hfill \end{array}$$ (11)

where we have defined the reduced variables x = δE/E₀, y = δμ/μ₀. We stress that N_G is an extrinsic noise that affects both the enzyme concentration and the growth rate, whereas N_E (resp. N_μ) is an intrinsic noise that only affects E (resp. μ). Note that the two effective noises T_EGN_G + N_E and T_μGN_G + N_μ acting on $\dot{x}$ and y are colored and correlated, which makes the present model more complicated than most stochastic models studied in the current literature. In fact, since we are mainly interested in the information exchanged between x and y, it is convenient to replace one of the noises, say N_G, by the dynamical variable y. Differentiating the second equation in Eq (11), using Eq (9) and performing some simple manipulations, one then obtains a new set of equations for the four random variables x, y, u ≡ N_E, v ≡ N_μ:

$$\begin{array}{ccc}\hfill \dot{x}& =& a_{1}x+a_{2}u+a_{3}v+a_{4}y\hfill \\ \hfill \dot{y}& =& b_{1}x+b_{2}u+b_{3}v+b_{4}y+{\xi }_y\hfill \\ \hfill \dot{u}& =& -{\beta }_{E}u+{\xi }_E\hfill \\ \hfill \dot{v}& =& -{\beta }_{\mu }v+{\xi }_{\mu },\hfill \end{array}$$ (12)

where the coefficients a_j and b_j (j = 1…4) are defined by Eq (24) in the section on Methods and ξ_y = ξ_μ + ξ_G is a new white noise satisfying $\langle {\xi }_y\left(t\right){\xi }_y\left(t^{\prime }\right)\rangle =({\theta }_{\mu }^2+{\theta }_G^2)\delta (t-t^{\prime })$ and $\langle {\xi }_y\left(t\right){\xi }_{\mu }\left(t^{\prime }\right)\rangle ={\theta }_{\mu }^2\delta (t-t^{\prime })$.

The calculation of the transfer entropy rate ${\overline{\mathcal{T}}}_{X\to Y}$ (which coincides with ${\overline{\mathcal{T}}}_{E\to \mu }$ since the TE is invariant under the change of variables from E to x and μ to y) is detailed in the section on Methods, together with the calculation of the information flows. The final expression reads

$$\begin{array}{ccc}\hfill {\overline{\mathcal{T}}}_{X\to Y}& =& \frac{1}{4({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2)}\int dxdyp(x,y)[{\overline{g}}_y^2(x,y)-{\overline{\overline{g}}}_y^2\left(y\right)]\hfill \end{array}$$ (13)

where p(x, y) is the steady state probability distribution and the functions ${\overline{g}}_y$ and ${\overline{\overline{g}}}_y$ are defined in Eqs (40) and (43), respectively. This result agrees with that obtained in Refs. [11, 18] and in [24] in special cases.

In Table 2, we show the results of the analysis of the time series generated by Eq (12) using our numerical inference method with a sampling time τ = 1min (equal to the time step Δt used to numerically integrate the model). One can see that the estimates of ${\overline{\mathcal{T}}}_{E\to \mu }$ are in good agreement with the predictions of Eq (13), with the values of the model parameters taken from Table S1 in Ref. [4]. Note that the negative number given by the inference method in the high IPTG experiment signals that the actual value of ${\overline{\mathcal{T}}}_{E\to \mu }$ cannot be distinguished from zero, which is indeed the theoretical prediction. In contrast, the estimated and theoretical results for ${\overline{\mathcal{T}}}_{\mu \to E}$ do not agree, as the inference method yields finite values in all cases whereas the theoretical values diverge.

Table 2. Comparison between the theoretical values of the transfer entropy rates ${\overline{\mathcal{T}}}_{E\to \mu }$ and ${\overline{\mathcal{T}}}_{\mu \to E}$ for the model of Ref. [4] and the values inferred from simulation data.

Averages are taken over 100 times series of duration 10⁶ min, sampled every 1 min.

Conc. of IPTG	Low	Intermediate	High
${\overline{\mathcal{T}}}_{E\to \mu }$(in h−1) (theo.)	0.033	0.034	0
${\overline{\mathcal{T}}}_{E\to \mu }$ (simul.)	0.031	0.034	−0.011
${\overline{\mathcal{T}}}_{\mu \to E}$ (theo.)	∞	∞	∞
${\overline{\mathcal{T}}}_{\mu \to E}$ (simul.)	0.202	0.123	0.347

This behavior is due to the absence of a white noise source directly affecting the dynamical evolution of x in the set of Eq (12). Indeed, as pointed out in Ref. [6] and also observed above in Fig 1, a TE rate diverges when the coupling between the variables is deterministic. In the model of Ref. [4], this feature can be traced back to the fact that the noise N_E affecting the enzyme concentration is colored with a finite relaxation time ${\beta }_E^{-1}$. Therefore, when taking the limit τ → 0 in Eq (3), one explores a time interval $\tau <{\beta }_E^{-1}$ where N_E is not really random. This is illustrated in Fig 3a that corresponds to the low IPTG experiment: we see that the estimate of ${\overline{\mathcal{T}}}_{\mu \to E}$ with the inference method is indeed diverging when the sampling time τ approaches zero. On the other hand, as expected, ${\overline{\mathcal{T}}}_{E\to \mu }$ remains finite and the points nicely lie on the plateau determined by Eq (13).

Fig 3. Transfer entropy rates ${\overline{\mathcal{T}}}_{E\to \mu }$ and ${\overline{\mathcal{T}}}_{\mu \to E}$ in the low IPTG experiment.

(a) Original model of Ref. [4] (b) Modified model where N_E is a white noise. The symbols are the estimates from the inference method when varying the sampling time τ, and the solid lines are the theoretical predictions from Eq (13) in (a) and from Eq (60) in (b). Note that ${\overline{\mathcal{T}}}_{\mu \to E}$ diverges as τ goes to zero in (a) but not (b).

The obvious and simplest way to cure this undesirable feature of the original model is to treat N_E as a purely white noise, which amounts to taking the limit ${\beta }_E^{-1}\to 0$. In fact, it is noticeable that the values of ${\beta }_E^{-1}$ extracted from the fit of the correlation functions in Ref. [4] (resp. ${\beta }_E^{-1}=10.7,9.9$ and 8.15 min for the low, intermediate, and high IPTG concentrations) are significantly smaller than the time steps τ_exp used for collecting the data (resp. τ_exp = 28, 20 and 15.8 min). Therefore, it is clear that the experimental data are not precise enough to decide whether N_E is colored or not. This issue does not arise for the other relaxation times in the model, ${\beta }_{\mu }^{-1}={\beta }_G^{-1}$ and ${\mu }_0^{-1}$, which are much longer (at least for the low and intermediate IPTG concentrations), and can be correctly extracted from the experimental data.

We thus propose to modify the model of Ref. [4] by describing N_E as a Gaussian white noise with variance 〈N_E(t)N_E(t′)〉 = 2D_Eδ(t − t′) and the same intensity as the colored noise in the original model, i.e. $D_E={\eta }_E^2/{\beta }_E$ (which yields D_E ≈ 0.188h, 0.100h, 0.031h for the three IPTG concentrations). Unsurprisingly, this modification does not affect the auto and cross-correlation functions used to fit the data, as shown in Fig 4 (see also section on Methods for a detailed calculation). On the other hand, the values of ${\overline{\mathcal{T}}}_{E\to \mu }$ are changed (compare Tables 2 and 3) and, more importantly, ${\overline{\mathcal{T}}}_{\mu \to E}$, given by Eq (60) is now finite. As a result, the model predicts that the difference $\Delta {\overline{\mathcal{T}}}_{E\to \mu }={\overline{\mathcal{T}}}_{E\to \mu }-{\overline{\mathcal{T}}}_{\mu \to E}$ is positive at low and intermediate IPTG concentrations and becomes negative at high concentration, which is in agreement with the direct analysis of the experimental data in Table 1. In contrast, $\Delta {\overline{\mathcal{T}}}_{E\to \mu }$ was always negative in the original model as ${\overline{\mathcal{T}}}_{\mu \to E}$ is infinite.

Fig 4.

(a) Autocorrelation function R_μμ(τ) for the three IPTG concentrations. Black lines: original model of Ref. [4], red circles: simplified model where N_E is a white noise. (b) Same as (a) for R_EE(τ). (c) Same as (a) for R_Eμ(τ).

Table 3. Theoretical values of the transfer entropy rates ${\overline{\mathcal{T}}}_{E\to \mu }$ and ${\overline{\mathcal{T}}}_{\mu \to E}$ and their difference in the modified model.

Conc. of IPTG	Low	Intermediate	High
${\overline{\mathcal{T}}}_{E\to \mu }$ (h−1)	1.23 ⋅ 10−2	8.2 ⋅ 10−3	0
${\overline{\mathcal{T}}}_{\mu \to E}$ (h−1)	1.9 ⋅ 10−3	5 ⋅ 10−4	2.97 ⋅ 10−2
$\Delta {\overline{\mathcal{T}}}_{E\to \mu }$ (h−1)	1.04 ⋅ 10−2	7.7 ⋅ 10−3	−2.97 ⋅ 10−2

This new behavior of the TE rates is also manifest when the inference method is applied to the time series generated by the model and the sampling time τ is varied. As observed in Fig 3b, the inferred value of ${\overline{\mathcal{T}}}_{\mu \to E}$ no longer diverges as τ → 0 (compare the vertical scale with that in Fig 3a). The estimates of ${\overline{\mathcal{T}}}_{E\to \mu }$ and ${\overline{\mathcal{T}}}_{\mu \to E}$ are also in good agreement with the theoretical predictions, except for the shortest value of τ which is equal to the time step Δt = 1 min used to numerically integrate the equations. It worth mentioning, however, that the error bars increase as τ is decreased.

While the change in the sign of $\Delta {\overline{\mathcal{T}}}_{E\to \mu }$ is now confirmed by the model, which is the main outcome of our analysis, one may also wonder whether the numerical values in Table 1 are recovered. This requires to multiply the rates in Table 3 by the experimental sampling times τ_exp which are different in each experiment, as indicated above. One then observes significant discrepancies for the low and intermediate IPTG experiments. We believe that the problem arises from the presence of many short time series in the set of experimental data. This is a important issue that needs to be examined in more detail since it may be difficult to obtain long time series in practice.

To this aim, we have studied the convergence of the estimates of $\Delta {\overline{\mathcal{T}}}_{E\to \mu }$ to the exact asymptotic value as a function of N, the length of the time series generated by the model in the stationary regime. As shown in Fig 5, the convergence with N is slow, which means that one can make significant errors in the estimation of $\Delta {\overline{\mathcal{T}}}_{E\to \mu }$ if N is small. On the other hand, the convergence can be greatly facilitated by choosing a value of the sampling time which is not too short (but of course shorter than the equilibration time of the system), for instance τ = 6min instead of 1 min in the case considered in Fig 5. The important observation is that the sign of $\Delta {\overline{\mathcal{T}}}_{E\to \mu }$ is then correctly inferred even with N ≈ 1000. In contrast, with τ = 1min, this is only possible for much longer series, typically N ≈ 50000. This is an encouraging indication for experimental studies, as the overall acquisition time of the data can be significantly reduced.

Fig 5. Inferred values of $\Delta {\overline{\mathcal{T}}}_{E\to \mu }$ for the low IPTG experiment as a function of the length N of the time series generated by the modified model.

Panels (a) and (b) correspond to sampling times τ = 6 min and τ = 1 min, respectively. $\Delta {\overline{\mathcal{T}}}_{E\to \mu }\left(\infty \right)$ is the exact asymptotic value.

Finally, we briefly comment on the results for the information flows ${\mathcal{I}}_{E\to \mu }^{flow}$ and ${\mathcal{I}}_{\mu \to E}^{flow}$. As already pointed out, the fact that the noises acting on the two random variables are correlated invalidates inequality (5). This is indeed what is observed in Table 4. It is also noticeable that ${\mathcal{I}}_{E\to \mu }^{flow}\ne -{\mathcal{I}}_{\mu \to E}^{flow}$, except in the high IPTG experiment where T_μE = 0.

Table 4. Comparison between the theoretical values of the TE rates and the information flows for the modified model and the values inferred from simulation data (all quantities are expressed in h⁻¹).

The analysis was performed with a sampling τ = 6 min and 100 time series of 10⁶ points.

Conc. of IPTG	Low	Intermediate	High
${\overline{\mathcal{T}}}^{E\to \mu }$, analytical	0.0123	0.0082	0
${\overline{\mathcal{T}}}^{E\to \mu }$, simulation	0.0128 ± 6 ⋅ 10−4	0.0064 ± 6 ⋅ 10−4	−0.0002 ± 5 ⋅ 10−4
${\overline{\mathcal{T}}}^{\mu \to E}$, analytical	0.0019	0.0005	0.0297
${\overline{\mathcal{T}}}^{\mu \to E}$, simulation	0.0023 ± 6 ⋅ 10−4	0.0012 ± 6 ⋅ 10−4	0.0215 ± 7 ⋅ 10−4
${\mathcal{I}}_{E\to \mu }^{flow}$, analytical	0.0751	0.092	−0.0214
${\mathcal{I}}_{E\to \mu }^{flow}$, simulation	0.076 ± 10−3	0.09 ± 8 ⋅ 10−4	−0.018 ± 8 ⋅ 10−4
${\mathcal{I}}_{\mu \to E}^{flow}$, analytical	0.0455	0.0743	0.0214
${\mathcal{I}}_{\mu \to E}^{flow}$, simulation	0.047 ± 10−3	0.072 ± 10−3	0.015 ± 10−3

Discussion and conclusion

A challenge when studying any biochemical network is to properly identify the direction of information. In this work, using the notion of transfer entropy, we have characterized the directed flow of information between the single cell growth rate and the gene expression, using a method that goes beyond what could be obtained from correlation functions, or from other inference techniques which do not exploit dynamical information.

Another crucial challenge in the field is to properly model the various noise components. It turns out that biological systems are generally non-bipartite due the presence of an extrinsic component in the noise. The present work provides on the one hand analytical expressions for the magnitude of the transfer entropy (or at least an upper bound on it) and of the information flow when the system is not bipartite, and, on the other hand a numerical method to infer the TE in all cases. Furthermore, we have shown that one can correctly infer the sign of the TE difference even with short time series by properly choosing the sampling time (see Ref. [25] for more details on the dependence of TE on the sampling time).

To conclude, we would like to emphasize that the transfer entropy is a general tool to identify variables which are relevant for time series prediction [26]. As such, the method has a lot of potential beyond the particular application covered in this paper: Predicting the current or future state of the environment by sensing it is an adaptation strategy followed by biological systems which can be understood using information-theoretic concepts [11, 27]. Similarly, during evolution, biological systems accumulate information from their environment, process it and use it quasi-optimally to increase their own fitness [28, 29]. In this context, transfer entropy-based methods have the potential to identify the directional interactions in co-evolution processes, which could be for instance the genomic evolution of a virus compared to that of its antigenes [30]. With the recent advances in high-throughput techniques and experimental evolution, we might soon be able to predict reliably the evolution of biological systems [31], and without doubt tools of information theory will play a key role in these advances.

Methods

In this section, we provide a detailed analysis of the information-theoretic quantities for the various models considered in this paper. The section is organized as follows:

• Basic information-theoretic measures

• Transfer entropy and information flow in the feedback cooling model

• Transfer entropy rates and information flows in the model of Ref. [4] for a metabolic network

• Transfer entropy rates and information flows in the modified model for the metabolic network

Basic information-theoretic measures

Below we briefly recall some definitions and properties of the information-theoretic measures. A fundamental quantity is the Shannon entropy which quantifies the uncertainty associated with the measurement x of a random variable X:

$$\begin{array}{c}\hfill H\left(X\right)=-\sum _{x}P\left(x\right)lnP\left(x\right),\end{array}$$ (14)

where P(x) is the probability that event x is realized, given an ensemble of possible outcomes. With this convention, the entropy is measured in nats. Similarly, for two random variables X and Y, one defines the joint Shannon entropy

$$\begin{array}{c}\hfill H(X,Y)=-\sum _{x,y}P(x,y)lnP(x,y),\end{array}$$ (15)

and the conditional Shannon entropy

$$\begin{array}{c}\hfill H\left(X\right|Y)=-\sum _{x,y}P(x,y)lnP\left(x\right|y),\end{array}$$ (16)

where P(x, y) and P(x|y) are joint and conditional probability distribution functions, respectively. The mutual information I(X : Y) is then a symmetric measure defined as

$$\begin{array}{cc}\hfill I(X:Y)& =\sum _{x,y}P(x,y)ln\frac{P(x,y)}{P\left(x\right)P\left(y\right)},\hfill \\ & =H\left(X\right)-H\left(X\right|Y)\hfill \\ & =H\left(Y\right)-H\left(Y\right|X),\hfill \end{array}$$ (17)

which quantifies the reduction of the uncertainty about X (resp. Y) resulting from the knowledge of the value of Y (respX). The more strongly X and Y are correlated, the larger I(X : Y) is.

These notions can be readily extended to random processes X = {X_i} and Y = {Y_i} viewed as collections of individual random variables sorted by an integer time index i. The mutual information between the ordered time series {x_i} and {y_i}, realizations of X and Y, is then defined as

$$\begin{array}{c}\hfill I(X:Y)=I(Y:X)\equiv \sum _{\{x_i,y_i\}}P(x_i,y_i)ln\frac{P(x_i,y_i)}{P\left(x_i\right)P\left(y_i\right)},\end{array}$$ (18)

and characterizes the undirected information exchanged between the two processes. The conditional mutual information is defined similarly.

In contrast, the transfer entropy T_X→Y is a information-theoretic measure that is both asymmetric and dynamic as it captures the amount of information that a source process X provides about the next state of a target process Y. More precisely, as defined by Eq (1) in the introduction,

$$\begin{array}{c}\hfill T_{X\to Y}=\sum _i[I(Y_{i+1}:{\mathit{X}}_i^{\left(l\right)},{\mathit{Y}}_i^{\left(k\right)})-I(Y_{i+1}:{\mathit{Y}}_i^{\left(k\right)})],\end{array}$$ (19)

where k and l define the lengths of the process histories, i.e., ${\mathit{Y}}_i^{\left(k\right)}=\{Y_{i-k+1},\cdots ,Y_i\}$ and ${\mathit{X}}_i^{\left(l\right)}=\{X_{i-l+1},\cdots ,X_i\}$. In this work, we have focused on a history length of 1 (i.e. k = l = 1) and denoted the corresponding TE by ${\overline{T}}_{X\to Y}$. Hence, ${\overline{T}}_{X\to Y}={\sum }_i[H\left(Y_{i+1}\right|Y_i)-H\left(Y_{i+1}\right|X_i,Y_i)]$, which is an upper bound to T_X→Y(k, l) for l = 1 when the joint process {X, Y} obeys a Markovian dynamics [11].

On the other hand, the information flow from X to Y is defined as the time-shifted mutual information

$$\begin{array}{c}\hfill {\mathcal{I}}_{X\to Y}^{flow}=\sum _i[I(Y_i:X_i)-I(Y_i:X_{i+1})],\end{array}$$ (20)

and informs on the reduction of uncertainty in Y_i when knowing about X_i+1 as compared to what we had with X_i only. In practice, ${\mathcal{I}}_{X\to Y}^{flow}$ can be obtained by shifting in time one time series with respect to the other one. Contrary to the transfer entropy which is always a positive quantity, the information flow ${\mathcal{I}}_{X\to Y}^{flow}$ may be negative or positive, depending on whether X sends information to Y (or X gains control of Y), or Y sends information to X (or X looses control over Y). In a bipartite system one has ${\mathcal{I}}_{X\to Y}^{flow}=-{\mathcal{I}}_{Y\to X}^{flow}$ in the stationary regime. This is no longer true when the system is non-bipartite.

Transfer entropy and information flow in the feedback cooling model

We first recall the theoretical expressions of the transfer entropy rates and the information flows for the feedback-cooling model described by Eq (6). These quantities were computed in Ref. [16]. The transfer entropy rates in the stationary state are given by

$$\begin{array}{ccc}\hfill {\mathcal{T}}_{V\to Y}& =& \frac{\gamma }{2m}(\sqrt{1+\frac{2T}{\gamma {\sigma }^2}}-1)\hfill \\ \hfill {\mathcal{T}}_{Y\to V}& =& \frac{1}{2{\tau }_r}(\sqrt{1+\frac{a^2{\sigma }^2}{2\gamma T}}-1).\hfill \end{array}$$ (21)

Note that 2T/(γσ²) is the signal-to-noise ratio that quantifies the relative size of the measurement accuracy to the thermal diffusion of the velocity. Accordingly, the TE rate ${\mathcal{T}}_{V\to Y}$ diverges when the control is deterministic. The information flow ${\mathcal{I}}_{V\to Y}^{flow}$ is given by

$$\begin{array}{c}\hfill {\mathcal{I}}_{V\to Y}^{flow}=\frac{\gamma }{m}(\frac{T⟨y^2⟩}{m|\mathbf{\Sigma }|}-1)\end{array}$$ (22)

where $|\mathbf{\Sigma }|$ is the determinant of the covariance matrix. The analytical expressions of the elements of the matrix, 〈v²〉, 〈y²〉 and 〈vy〉, are given by Eqs (A2) in Ref. [16]. In contrast with ${\mathcal{T}}_{V\to Y}$, the information flow ${\mathcal{I}}_{V\to Y}^{flow}$ remains finite as the noise intensity vanishes.

The upper bounds to the transfer entropies (see Eq (2)) were computed in Ref. [24] in the general case of coupled linear Langevin equations. For the feedback cooling model, one obtains

$$\begin{array}{ccc}\hfill {\overline{\mathcal{T}}}_{V\to Y}& =& \frac{1}{2{\sigma }^2⟨y^2⟩}|\mathbf{\Sigma }|\hfill \\ \hfill {\overline{\mathcal{T}}}_{Y\to V}& =& \frac{a^2}{4\gamma k_{B}T⟨v^2⟩}|\mathbf{\Sigma }|.\hfill \end{array}$$ (23)

As shown in Fig 1, the estimate of the transfer entropy obtained by the inference method is in good agreement with the theoretical value (we stress that the figure shows the rates multiplied by the sampling time τ = 10⁻³). In Fig 6, we also obtain satisfactory agreement between inferred value of the information flow ${\mathcal{I}}_{V\to Y}^{flow}$ and theoretical value, when representing these quantities against the noise intensity σ². These results of this figure confirm the inequalities ${\mathcal{I}}_{V\to Y}^{flow}\le {\mathcal{T}}_{V\to Y}\le {\overline{\mathcal{T}}}_{V\to Y}$.

Fig 6. ${\mathcal{T}}_{V\to Y},{\overline{\mathcal{T}}}_{V\to Y}$ and ${\mathcal{I}}_{V\to Y}^{flow}$ as a function of the noise intensity σ².

The parameters of the model are T = 5, γ = m = 1, τ_r = 0.1 and a = −0.7.

Transfer entropy rates and information flows in the model of Ref. [4] for a metabolic network

Stationary distributions and correlation functions

We first compute the stationary probability distributions (pdfs) associated with Eq (12) were the coefficients a_j and b_j are given by

$$\begin{array}{ccc}\hfill a_1& =& -[{\mu }_E+{\mu }_0{T}_{\mu E}(T_{EG}-1)]\hfill \\ \hfill a_2& =& {\mu }_0\hfill \\ \hfill a_3& =& -{\mu }_0{T}_{EG}\hfill \\ \hfill a_4& =& {\mu }_0(T_{EG}-1)\hfill \\ \hfill b_1& =& T_{\mu E}[{\beta }_G-{\mu }_E-{\mu }_0{T}_{\mu E}(T_{EG}-1)]\hfill \\ \hfill b_2& =& {\mu }_0{T}_{\mu E}\hfill \\ \hfill b_3& =& {\beta }_G-{\beta }_{\mu }-{\mu }_0{T}_{\mu E}{T}_{EG}\hfill \\ \hfill b_4& =& {\mu }_0{T}_{\mu E}(T_{EG}-1)-{\beta }_G.\hfill \end{array}$$ (24)

We recall that μ_E = μ₀(1 + T_μE − T_EE) sets the timescale of E-fluctuations [4]. Since Eq (12) describe a set of coupled Markovian Ornstein-Uhlenbeck processes, the stationary pdf p_xuvy(x, u, v, y) is Gaussian and given by

$$\begin{array}{c}\hfill p_{xuvy}(x,u,v,y)=\frac{1}{{\left(2\pi \right)}^2\sqrt{|\mathbf{\Sigma }|}}{e}^{-\frac{1}{2}(x,u,v,y).{\mathbf{\Sigma }}^{-1}.{(x,u,v,y)}^T},\end{array}$$ (25)

where $\mathbf{\Sigma }$ is the covariance matrix which obeys the Lyapunov equation [32]

$$\begin{array}{c}\hfill \mathbf{A}\mathbf{\Sigma }+\mathbf{\Sigma }{\mathbf{A}}^T=2\mathbf{D},\end{array}$$ (26)

where

$$\begin{array}{c}\hfill \mathbf{A}=\left(\begin{array}{cccc}-a_1& -a_2& -a_3& -a_4\\ 0& {\beta }_E& 0& 0\\ 0& 0& {\beta }_{\mu }& 0\\ -b_1& -b_2& -b_3& -b_4\end{array}\right),\text{and}\mathbf{D}=\left(\begin{array}{cccc}0& 0& 0& 0\\ 0& {\beta }_E{\eta }_E^2& 0& 0\\ 0& 0& {\beta }_{\mu }{\eta }_{\mu }^2& {\beta }_{\mu }{\eta }_{\mu }^2\\ 0& 0& {\beta }_{\mu }{\eta }_{\mu }^2& {\beta }_G{\eta }_G^2+{\beta }_{\mu }{\eta }_{\mu }^2\end{array}\right).\end{array}$$

The solution of Eq (26) reads

$$\begin{array}{ccc}\hfill {\sigma }_{11}& =& \frac{{\mu }_0^2}{{\mu }_E}[\frac{{\eta }_E^2}{{\mu }_E+{\beta }_E}+\frac{{\eta }_{\mu }^2}{{\mu }_E+{\beta }_{\mu }}+\frac{{(T_{EG}-1)}^2}{{\mu }_E+{\beta }_G}{\eta }_G^2]\hfill \\ \hfill {\sigma }_{12}& =& {\sigma }_{21}=\frac{{\mu }_0}{{\mu }_E+{\beta }_E}{\eta }_E^2\hfill \\ \hfill {\sigma }_{13}& =& {\sigma }_{31}=\frac{-{\mu }_0}{{\mu }_E+{\beta }_{\mu }}{\eta }_{\mu }^2\hfill \\ \hfill {\sigma }_{14}& =& {\sigma }_{41}=\frac{{\mu }_0}{{\mu }_E}[\frac{{\mu }_0{T}_{\mu E}}{{\mu }_E+{\beta }_E}{\eta }_E^2+\frac{({\mu }_0{T}_{\mu E}-{\mu }_E)}{{\mu }_E+{\beta }_{\mu }}{\eta }_{\mu }^2\hfill \\ & & +\frac{(T_{EG}-1)[{\mu }_0{T}_{\mu E}(T_{EG}-1)+{\mu }_E]}{{\mu }_E+{\beta }_G}{\eta }_G^2]\hfill \\ \hfill {\sigma }_{22}& =& {\eta }_E^2\hfill \\ \hfill {\sigma }_{23}& =& 0\hfill \\ \hfill {\sigma }_{24}& =& {\sigma }_{42}=\frac{{\mu }_0{T}_{\mu E}}{{\mu }_E+{\beta }_E}{\eta }_E^2\hfill \\ \hfill {\sigma }_{33}& =& {\eta }_{\mu }^2\hfill \\ \hfill {\sigma }_{34}& =& {\sigma }_{43}=\frac{{\mu }_E+{\beta }_{\mu }-{\mu }_0{T}_{\mu E}}{{\mu }_E+{\beta }_{\mu }}{\eta }_{\mu }^2\hfill \\ \hfill {\sigma }_{44}& =& \frac{{\mu }_0^2{T}_{\mu E}^2}{{\mu }_E({\mu }_E+{\beta }_E)}{\eta }_E^2+\frac{[{({\mu }_0{T}_{\mu E}-{\mu }_E)}^2+{\mu }_E{\beta }_{\mu }]}{{\mu }_E({\mu }_E+{\beta }_{\mu })}{\eta }_{\mu }^2\hfill \\ & & +\frac{{\mu }_0^2{T}_{\mu E}^2{(T_{EG}-1)}^2+{\mu }_E[{\mu }_E+{\beta }_G]}{{\mu }_E({\mu }_E+{\beta }_G)}{\eta }_G^2\hfill \\ & & +\frac{2{\mu }_0{T}_{\mu E}(T_{EG}-1)]}{{\mu }_E({\mu }_E+{\beta }_G)}{\eta }_G^2\hfill \end{array}$$ (27)

From this we can compute all marginal pdfs, in particular

$$\begin{array}{ccc}\hfill p_{xy}(x,y)& =& \frac{1}{2\pi \sqrt{{\sigma }_{11}{\sigma }_{44}-{\sigma }_{14}^2}}{e}^{-\frac{1}{2}\frac{{\sigma }_{44}{x}^2-2{\sigma }_{14}xy+{\sigma }_{11}{y}^2}{{\sigma }_{11}{\sigma }_{44}-{\sigma }_{14}^2}},\hfill \end{array}$$ (28)

and

$$\begin{array}{ccc}\hfill p_x\left(x\right)& =& \frac{1}{\sqrt{2\pi {\sigma }_{11}}}{e}^{-\frac{x^2}{2{\sigma }_{11}}}\hfill \\ \hfill p_y\left(y\right)& =& \frac{1}{\sqrt{2\pi {\sigma }_{44}}}{e}^{-\frac{y^2}{2{\sigma }_{44}}}.\hfill \end{array}$$ (29)

As an illustration, the steady-state pdf $p\left(\mu \right)=\frac{1}{{\mu }_0}{p}_y(y=\frac{\mu -{\mu }_0}{{\mu }_0})$ is plotted in Fig 7 for the three different IPTG concentrations (low, intermediate, and high). The agreement with the experimental curves displayed in Fig 1d of Ref. [4] is satisfactory.

Fig 7. Steady-state probability distribution of the growth rate for the three IPTG concentrations: Low (black), intermediate (red), high (blue).

For completeness, we also quote the expressions of R_pp(0) and R_pμ(0) (properly normalized) obtained from the definition $\delta p/\left({\mu }_0{E}_0\right)=\delta \dot{E}/\left({\mu }_0{E}_0\right)+\delta \mu /{\mu }_0+\delta E/E_0=(T_{EE}-T_{EG}{T}_{\mu E})x+u-T_{EG}(v-y)$:

$$\begin{array}{ccc}\hfill R_{pp}\left(0\right)& =& {(T_{EE}-T_{EG}{T}_{\mu E})}^2{\sigma }_{11}+{\sigma }_{22}+T_{EG}^2({\sigma }_{33}+{\sigma }_{44})\hfill \\ & & +2(T_{EE}-T_{EG}{T}_{\mu E})[{\sigma }_{12}+T_{EG}({\sigma }_{14}-{\sigma }_{13})]\hfill \\ & & +2T_{EG}{\sigma }_{24}-2T_{EG}^2{\sigma }_{34}\hfill \end{array}$$ (30)

$$\begin{array}{ccc}\hfill R_{p\mu }\left(0\right)& =& \frac{(T_{EE}-T_{EG}{T}_{\mu E}){\sigma }_{14}+{\sigma }_{24}+T_{EG}({\sigma }_{44}-{\sigma }_{34})}{\sqrt{R_{pp}\left(0\right)R_{\mu \mu }\left(0\right)}}\hfill \end{array}$$ (31)

with R_μμ(0) = σ₄₄.

The correlation functions R_μμ(τ), R_EE(τ), and R_Eμ(τ), obtained by taking the inverse Fourier transform of Eqs (6) in the Supplementary Information of [4] are plotted in Fig 4. In passing, we correct a few misprints in these equations: i) The correct expression of R_μμ(τ) is obtained by replacing A_E(τ) by R_EE(τ) in the first term of Eq (12) in the Supplementary Information of [4]. ii) Eq (10) corresponds to R_Eμ(τ) and not to R_μE(τ) = R_Eμ(−τ). Eq (8) then gives the correct expression of R_Eμ(τ) (and not of R_μE(τ)) provided the function A_X(τ) defined in Eq (10) is altered. For τ ≥ 0, one should have

$$\begin{array}{c}\hfill A_X\left(\tau \right)={\theta }_X^2\frac{{\mu }_0}{2{\beta }_X({\beta }_X+{\mu }_E)}{e}^{-{\beta }_{X}t}.\end{array}$$ (32)

Transfer entropy rates

We now address the computation of the conditional probabilities $p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau |x^{\prime },y^{\prime },t)$ and $p_{y^{\prime }}^y(y,t+\tau |y^{\prime },t)$ at first order in τ. This will allow us to obtain the expressions of the upper bounds to the transfer entropy rates defined by

$$\begin{array}{ccc}\hfill {\overline{\mathcal{T}}}_{X\to Y}& =& \underset{\tau \to 0}{lim}\frac{I[y_{t+\tau }:x_t,y_t]-I[y_{t+\tau }:y_t]}{\tau }\hfill \\ \hfill {\overline{\mathcal{T}}}_{Y\to X}& =& \underset{\tau \to 0}{lim}\frac{I[x_{t+\tau }:x_t,y_t]-I[x_{t+\tau }:x_t]}{\tau },\hfill \end{array}$$ (33)

where I is the mutual information, for instance $I[y_{t+\tau }:x_t,y_t]=\int dyd{x}^{\prime }d{y}^{\prime }{p}_{x^{\prime }{y}^{\prime }}^y(y,t+\tau ;x^{\prime },y^{\prime },t)ln[p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau ;x^{\prime },y^{\prime },t)/\left[p_y\left(y\right)p_{xy}(x^{\prime },y^{\prime })\right]$ in the steady state (where p_xy(x′, y′) and p_y(y) become time independent pdfs). Therefore,

$$\begin{array}{ccc}\hfill {\overline{\mathcal{T}}}_{X\to Y}& =& \underset{\tau \to 0}{lim}\frac{1}{\tau }\int dyd{x}^{\prime }d{y}^{\prime }{p}_{x^{\prime }{y}^{\prime }}^y(y,t+\tau ;x^{\prime },y^{\prime },t)\hfill \\ \hfill & & \times ln\frac{p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau |x^{\prime },y^{\prime },t)}{p_{y^{\prime }}^y(y,t+\tau |y^{\prime },t)}\hfill \\ \hfill {\overline{\mathcal{T}}}_{Y\to X}& =& \underset{\tau \to 0}{lim}\frac{1}{\tau }\int dyd{x}^{\prime }d{y}^{\prime }{p}_{x^{\prime }{y}^{\prime }}^x(x,t+\tau ;x^{\prime },y^{\prime },t)\hfill \\ & & \times ln\frac{p_{x^{\prime }{y}^{\prime }}^x(x,t+\tau |x^{\prime },y^{\prime },t)}{p_{x^{\prime }}^x(x,t+\tau |x^{\prime },t)}.\hfill \end{array}$$ (34)

Note that the actual transfer entropy rates are defined as

$$\begin{array}{ccc}\hfill {\mathcal{T}}_{X\to Y}& =& \underset{\tau \to 0}{lim}\frac{I[y_{t+\tau }:x_t,{\left\{y_{t^{\prime }}\right\}}_{t^{\prime }\le t}]-I[y_{t+\tau }:{\left\{y_{t^{\prime }}\right\}}_{t^{\prime }\le t}]}{\tau }\hfill \\ \hfill {\mathcal{T}}_{Y\to X}& =& \underset{\tau \to 0}{lim}\frac{I[x_{t+\tau }:{\left\{x_{t^{\prime }}\right\}}_{t^{\prime }\le t},y_t]-I[x_{t+\tau }:{\left\{x_{t^{\prime }}\right\}}_{t^{\prime }\le t}]}{\tau }.\hfill \end{array}$$ (35)

where {x_t′}_t′≤t and {y_t′}_t′≤t denote the full trajectories of x_t and y_t in the time interval [0, t]. Since the present model is not bipartite, the calculation of these quantities is a nontrivial task that is left aside.

The two-time distributions $p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau ;x^{\prime },y^{\prime },t)$ and $p_{x^{\prime }{y}^{\prime }}^x(x,t+\tau ;x^{\prime },y^{\prime },t)$ are given by

$$\begin{array}{ccc}\hfill p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau ;x^{\prime },y^{\prime },t)& =& \int dxdudvd{u}^{\prime }d{v}^{\prime }{p}_{{\mathbf{z}}^{\prime }}^{\mathbf{z}}(\mathbf{z},t+\tau |{\mathbf{z}}^{\prime },t)p_{xuvy}\left({\mathbf{z}}^{\prime }\right)\hfill \\ \hfill p_{x^{\prime }{y}^{\prime }}^x(x,t+\tau ;x^{\prime },y^{\prime },t)& =& \int dydudvd{u}^{\prime }d{v}^{\prime }{p}_{{\mathbf{z}}^{\prime }}^{\mathbf{z}}(\mathbf{z},t+\tau |{\mathbf{z}}^{\prime },t)p_{xuvy}\left({\mathbf{z}}^{\prime }\right)\hfill \end{array}$$ (36)

where $p_{{\mathbf{z}}^{\prime }}^{\mathbf{z}}(\mathbf{z},t+\tau |{\mathbf{z}}^{\prime },t)$ is the transition probability from the state ${\mathbf{z}}^{\prime }=(x^{\prime },u^{\prime },v^{\prime },y^{\prime })$ at time t to the state $\mathbf{z}=(x,u,v,y)$ at time t + τ. From the definition of the Fokker-Planck operator ${\mathcal{L}}_{FP}$ associated with the 4-dimensional diffusion process described by Eq (12), the transition probability for small times is given by [32]

$$\begin{array}{ccc}\hfill p_{{\mathbf{z}}^{\prime }}^{\mathbf{z}}(\mathbf{z},t+\tau |{\mathbf{z}}^{\prime },t)& =& [1+\tau {\mathcal{L}}_{FP}(\mathbf{z},t)+\mathcal{O}\left({\tau }^2\right)]\delta (\mathbf{z}-{\mathbf{z}}^{\prime })\hfill \\ & =& \delta (\mathbf{z}-{\mathbf{z}}^{\prime })-\tau \sum _{i=1}^4{\partial }_{z_i}[g_i\left({\mathbf{z}}^{\prime }\right)-\sum _j\frac{{\theta }_{i,j}^2}{2}{\partial }_{z_j}]\delta (\mathbf{z}-{\mathbf{z}}^{\prime })\hfill \end{array}$$ (37)

where $g_i\left(\mathbf{z}\right)$ is the drift coefficient in the equation for z_i (with z₁ = x, z₂ = u, z₃ = v, z₄ = y), ${\theta }_{2,2}={\theta }_E,{\theta }_{3,3}={\theta }_{3,4}={\theta }_{\mu },{\theta }_{4,4}=\sqrt{{\theta }_{\mu }^2+{\theta }_G^2}$ and all other θ_{i, j} being equal to 0.

Let us first consider the calculation of ${\overline{\mathcal{T}}}_{X\to Y}$. By integrating $p_{{\mathbf{z}}^{\prime }}^{\mathbf{z}}(\mathbf{z},t+\tau |{\mathbf{z}}^{\prime },t)$ over x, u, and v, we readily obtain

$$\begin{array}{c}\hfill p_{{\mathbf{z}}^{\prime }}^y(y,t+\tau |{\mathbf{z}}^{\prime },t)=\delta (y-y^{\prime })-\tau {\partial }_y[g_y\left({\mathbf{z}}^{\prime }\right)-{\beta }_{\mu }{\eta }_{\mu }^2{\partial }_v-({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2){\partial }_y]\delta (y-y^{\prime })+\mathcal{O}\left({\tau }^2\right)\end{array}$$

where the terms involving ∂_x, ∂_u, ∂_v cancel due to natural boundary conditions. Hence,

$$\begin{array}{ccc}\hfill p_{{\mathbf{z}}^{\prime }}^y(y,t+\tau ;{\mathbf{z}}^{\prime },t)& =& p_{{\mathbf{z}}^{\prime }}^y(y,t+\tau |{\mathbf{z}}^{\prime },t)p_{xuvy}\left({\mathbf{z}}^{\prime }\right)\hfill \\ & =& \delta (y-y^{\prime })p\left({\mathbf{z}}^{\prime }\right)-\tau p_{xuvy}\left({\mathbf{z}}^{\prime }\right)\times \hfill \\ & & {\partial }_y[g_y\left({\mathbf{z}}^{\prime }\right)-{\beta }_{\mu }{\eta }_{\mu }^2{\partial }_v-({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2){\partial }_y]\delta (y-y^{\prime }),\hfill \end{array}$$ (38)

which yields

$$\begin{array}{ccc}\hfill p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau ;x^{\prime },y^{\prime },t)& =& \delta (y-y^{\prime })p_{xy}(x^{\prime },y^{\prime })-\tau p_{xy}(x^{\prime },y^{\prime }){\partial }_y[{\overline{g}}_y(x^{\prime },y^{\prime })\hfill \\ & & -({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2){\partial }_y]\delta (y-y^{\prime }).\hfill \end{array}$$ (39)

after integration over u′ and v′, where we have defined the averaged drift coefficient

$$\begin{array}{c}\hfill {\overline{g}}_y(x,y)=\frac{1}{p_{xy}(x,y)}\int dudv{g}_y\left(\mathbf{z}\right)p_{xuvy}\left(\mathbf{z}\right).\end{array}$$ (40)

We thus finally obtain

$$\begin{array}{ccc}\hfill p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau |x^{\prime },y^{\prime },t)& =& \delta (y-y^{\prime })-\tau {\partial }_y[{\overline{g}}_y(x^{\prime },y^{\prime })\hfill \\ & & -({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2){\partial }_y]\delta (y-y^{\prime })+\mathcal{O}\left({\tau }^2\right).\hfill \end{array}$$ (41)

Similarly, by also integrating $p_{{\mathbf{z}}^{\prime }}^y(y,t+\tau ;x^{\prime },y^{\prime },t)$ over x′, we obtain

$$\begin{array}{ccc}\hfill p_{y^{\prime }}^y(y,t+\tau |y^{\prime },t)& =& \delta (y-y^{\prime })-\tau {\partial }_y[{\overline{\overline{g}}}_y\left(y^{\prime }\right)-({\beta }_{\mu }{\eta }_{\mu }^2\hfill \\ & & +{\beta }_G{\eta }_G^2\left){\partial }_y\right]\delta (y-y^{\prime })+\mathcal{O}\left({\tau }^2\right).\hfill \end{array}$$ (42)

where

$$\begin{array}{ccc}\hfill {\overline{\overline{g}}}_y\left(y\right)& =& \frac{1}{p_y\left(y\right)}\int dxdudv{g}_y\left(\mathbf{z}\right)p_{xuvy}\left(\mathbf{z}\right)\hfill \\ & =& \frac{1}{p_y\left(y\right)}\int dx{\overline{g}}_y(x,y)p_{xy}(x,y).\hfill \end{array}$$ (43)

Due to the linearity of Eq (12) and the Gaussian character of the pfds, one simply has ${\overline{g}}_y(x,y)=ax+by$ and ${\overline{\overline{g}}}_y\left(y\right)=cy$, where a, b, c are complicated functions of the model parameters which we do not display here.

Eq (41) (resp. Eq (42)) merely shows that $p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau |x^{\prime },y^{\prime },t)$ (resp. $p_{y^{\prime }}^y(y,t+\tau |y^{\prime },t)$) at the lowest order in τ is identical to the transition probability associated with an Ornstein-Uhlenbeck process with drift coefficient ${\overline{g}}_y(x,y)$ (resp. ${\overline{\overline{g}}}_y\left(y\right)$) and diffusion coefficient ${\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2$. To proceed further, it is then convenient to use to the Fourier integral representation of the δ function and re-express $p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau |x^{\prime },y^{\prime },t)$ and $p_{y^{\prime }}^y(y,t+\tau |y^{\prime },t)$ for small times as

$$\begin{array}{c}\hfill p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau |x^{\prime },y^{\prime },t)=\frac{1}{2\sqrt{\pi ({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2)\tau }}{e}^{-\frac{1}{4({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2)\tau }{[y-y^{\prime }-\tau {\overline{g}}_y(x^{\prime },y^{\prime })]}^2}\end{array}$$ (44)

and

$$\begin{array}{c}\hfill p_{y^{\prime }}^y(y,t+\tau |y^{\prime },t)=\frac{1}{2\sqrt{\pi ({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2)\tau }}{e}^{-\frac{1}{4({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2)\tau }{[y-y^{\prime }-\tau {\overline{\overline{g}}}_y\left(y^{\prime }\right)]}^2}.\end{array}$$ (45)

up to corrections of the order τ² [32]. This leads to

$$\begin{array}{ccc}\hfill ln\frac{p_{x^{\prime }{y}^{\prime }}^y(y,t+\tau |x^{\prime },y^{\prime },t)}{p_{y^{\prime }}^y(y,t+\tau |y^{\prime },t)}& =& \frac{1}{4({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2)}[2(y-y^{\prime })-\tau [{\overline{g}}_y(x^{\prime },y^{\prime })+{\overline{\overline{g}}}_y\left(y^{\prime }\right)]]\hfill \\ & & \times [{\overline{g}}_y(x^{\prime },y^{\prime })-{\overline{\overline{g}}}_y\left(y^{\prime }\right)],\hfill \end{array}$$ (46)

and from Eq (39) and the definition of the transfer entropy rate [Eq (34)],

$$\begin{array}{ccc}\hfill 4({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2){\overline{\mathcal{T}}}_{X\to Y}& =& \underset{\tau \to 0}{lim}\frac{1}{\tau }\int dyd{x}^{\prime }d{y}^{\prime }{p}_{x^{\prime }{y}^{\prime }}^y(y,t+\tau ;x^{\prime },y^{\prime },t)[2(y-y^{\prime })\hfill \\ & & -\tau [{\overline{g}}_y(x^{\prime },y^{\prime })+{\overline{\overline{g}}}_y\left(y^{\prime }\right)]][{\overline{g}}_y(x^{\prime },y^{\prime })-{\overline{\overline{g}}}_y\left(y^{\prime }\right)]\hfill \\ & =& \underset{\tau \to 0}{lim}\frac{1}{\tau }\int dyd{x}^{\prime }d{y}^{\prime }{p}_{xy}(x^{\prime },y^{\prime })[\delta (y-y^{\prime })-\tau {\partial }_y[{\overline{g}}_y(x^{\prime },y^{\prime })\hfill \\ & & -({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2){\partial }_y\left]\delta (y-y^{\prime })\right]\hfill \\ & & \times [2(y-y^{\prime })-\tau [{\overline{g}}_y(x^{\prime },y^{\prime })+{\overline{\overline{g}}}_y\left(y^{\prime }\right)]][{\overline{g}}_y(x^{\prime },y^{\prime })-{\overline{\overline{g}}}_y\left(y^{\prime }\right)]\hfill \end{array}$$ (47)

We then use

$$\begin{array}{c}\hfill \int dy(y-y^{\prime })[\delta (y-y^{\prime })-\tau {\partial }_y[{\overline{g}}_y(x^{\prime },y^{\prime })-({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2){\partial }_y]\delta (y-y^{\prime })]=\tau {\overline{g}}_y(x^{\prime },y^{\prime }),\end{array}$$ (48)

and

$$\begin{array}{ccc}\hfill \int d{x}^{\prime }{p}_{xy}(x^{\prime },y^{\prime }){\overline{g}}_y(x^{\prime },y^{\prime })& =& p_y\left(y^{\prime }\right){\overline{\overline{g}}}_y\left(y^{\prime }\right)=\int d{x}^{\prime }{p}_{xy}(x^{\prime },y^{\prime }){\overline{\overline{g}}}_y\left(y^{\prime }\right),\hfill \end{array}$$ (49)

to finally arrive at Eq (13), namely

$$\begin{array}{ccc}\hfill {\overline{\mathcal{T}}}_{X\to Y}& =& \frac{1}{4({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2)}\int dxdy{p}_{xy}(x,y)[{\overline{g}}_y^2(x,y)-{\overline{\overline{g}}}_y^2\left(y\right)].\hfill \end{array}$$ (50)

A similar expression can be found in Ref. [11] (see Eq (A.31) in that reference). Note also that the result given in Ref. [24] is obtained as a special case.

Inserting into Eq (13) the values of the parameters given in Table S1 of Ref. [4], we obtain the values given in Table 2. Note that ${\overline{\mathcal{T}}}_{E\to \mu }=0$ for the high IPTG concentration because T_μE = 0, and therefore μ(t) no longer depends on E(t) as can be seen from Eq (10).

There is no need to detail the calculation of ${\overline{\mathcal{T}}}_{\mu \to E}$ (i.e. ${\overline{\mathcal{T}}}_{Y\to X}$) because it goes along the same line, with y replaced by x. The crucial difference is that there is no white noise acting on $\dot{x}$. Therefore, the denominator in Eq (13), which is the variance of the noise ξ_y, is replaced by 0. This implies that ${\overline{\mathcal{T}}}_{\mu \to E}$ is infinite.

Information flows

The information flows ${\mathcal{I}}_{X\to Y}^{flow}$ and ${\mathcal{I}}_{Y\to X}^{flow}$ are derived from the time-shifted mutual informations I[x_t+τ : y_t] and I[y_t+τ : x_t]. Specifically,

$$\begin{array}{ccc}\hfill {\mathcal{I}}_{X\to Y}^{flow}& =& \underset{\tau \to 0}{lim}\frac{I[x_t:y_t]-I[x_{t+\tau }:y_t]}{\tau }\hfill \\ \hfill {\mathcal{I}}_{Y\to X}^{flow}& =& \underset{\tau \to 0}{lim}\frac{I[y_t:x_t]-I[y_{t+\tau }:x_t]}{\tau }.\hfill \end{array}$$ (51)

Let us first consider the second flow ${\mathcal{I}}_{Y\to X}^{flow}$ which requires the knowledge of $p_{x^{\prime }}^y(y,t+\tau ;x^{\prime },t)$ whose expression is obtained by integrating Eq (39) over x′. This yields

$$\begin{array}{ccc}\hfill p_{x^{\prime }}^y(y,t+\tau ;x^{\prime },t)& =& p_{xy}(x^{\prime },y)-\tau {\partial }_y[{\overline{g}}_y(x^{\prime },y)\hfill \\ & & -({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2){\partial }_y]p_{xy}(x^{\prime },y)+\mathcal{O}\left({\tau }^2\right).\hfill \end{array}$$ (52)

Hence

$$\begin{array}{ccc}\hfill I[y_{t+\tau }:x_t]& =& \int d{x}^{\prime }dy{p}_{x^{\prime }}^y(y,t+\tau ;x^{\prime },t)\hfill \\ & & \times ln\frac{p_{x^{\prime }}^y(y,t+\tau ;x^{\prime },t)}{p_y\left(y\right)p_x\left(x^{\prime }\right)}\hfill \\ & =& I[y_t:x_t]-\tau \int dxdy{\partial }_y[{\overline{g}}_y(x,y)\hfill \\ & & -({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2){\partial }_y]p_{xy}(x,y)ln\frac{p_{xy}(x,y)}{p_y\left(y\right)p_x\left(x\right)}.\hfill \end{array}$$ (53)

We finally obtain

$$\begin{array}{ccc}\hfill {\mathcal{I}}_{Y\to X}^{flow}& =& \int dxdy{\partial }_y[{\overline{g}}_y(x,y)p_{xy}(x,y)\hfill \\ & & -({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2){\partial }_y{p}_{xy}(x,y)]ln\frac{p_{xy}(x,y)}{p_y\left(y\right)p_x\left(x\right)}.\hfill \end{array}$$ (54)

A similar calculation yields

$$\begin{array}{ccc}\hfill {\mathcal{I}}_{X\to Y}^{flow}& =& \int dxdy{\partial }_x\left[{\overline{g}}_x(x,y)p_{xy}(x,y)\right]ln\frac{p_{xy}(x,y)}{p_y\left(y\right)p_x\left(x\right)},\hfill \end{array}$$ (55)

where

$$\begin{array}{c}\hfill {\overline{g}}_x(x,y)=\frac{1}{p_{xy}(x,y)}\int dudv{g}_x\left(\mathbf{z}\right)p_{xuvy}\left(\mathbf{z}\right)\end{array}$$ (56)

is an averaged drift coefficient. Contrary to the case of the transfer entropy rate ${\overline{\mathcal{T}}}_{Y\to X}$, the absence of a white noise acting on $\dot{x}$ does not lead to an infinite result for ${\mathcal{I}}_{Y\to X}^{flow}$. In fact, one has the symmetry relation

$$\begin{array}{c}\hfill {\mathcal{I}}_{X\to Y}^{flow}=-{\mathcal{I}}_{Y\to X}^{flow},\end{array}$$ (57)

which is readily obtained by noting that p_xy(x, y), the stationary solution of the Fokker-Planck equation, satisfies the equation

$$\begin{array}{ccc}& & {\partial }_x\left[{\overline{g}}_x(x,y)p_{xy}(x,y)\right]+{\partial }_y\left[{\overline{g}}_y(x,y)p_{xy}(x,y)\right]\hfill \\ & & -({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2)\frac{{\partial }^2}{\partial y^2}{p}_{xy}(x,y)=0.\hfill \end{array}$$ (58)

Inserting the numerical values of the parameters given in Table S1 of Ref. [4], we obtain the values given in Table 5 below. Interestingly, ${\mathcal{I}}_{E\to \mu }^{flow}$ decreases as the IPTG concentration increases and that it becomes negative at high concentration.

Table 5. Theoretical values of ${\mathcal{I}}_{X\to Y}^{flow}=-{\mathcal{I}}_{Y\to X}^{flow}$ in the original model of Ref. [4].

Conc. of IPTG	Low	Intermediate	High
${\mathcal{I}}_{E\to \mu }^{flow}$(in h−1)	0.0148	0.0088	-0.0243

Transfer entropy rates and information flows in the modified model for the metabolic network

We now repeat the above calculations for the modified model where N_E is treated as a white noise. Eliminating again the variable w (i.e. N_G) in favor of y, the new set of equations that describe the stochastic dynamics and replace Eq (12) reads

$$\begin{array}{ccc}\hfill \dot{x}& =& -[{\mu }_E+{\mu }_0{T}_{\mu E}(T_{EG}-1)]x-{\mu }_0{T}_{EG}v\hfill \\ & & +{\mu }_0(T_{EG}-1)y+{\xi }_x\hfill \\ \hfill \dot{v}& =& -{\beta }_{\mu }v+{\xi }_{\mu }\hfill \\ \hfill \dot{y}& =& T_{\mu E}[{\beta }_G-{\mu }_E-{\mu }_0{T}_{\mu E}(T_{EG}-1)]x+[{\beta }_G-{\beta }_{\mu }\hfill \\ & & -{\mu }_0{T}_{\mu E}{T}_{EG}]v+[{\mu }_0{T}_{\mu E}(T_{EG}-1)-{\beta }_G]y+{\tilde{\xi }}_y,\hfill \end{array}$$ (59)

where we have defined the white noises ξ_x = μ₀N_E and ${\tilde{\xi }}_y={\xi }_y+T_{\mu E}{\xi }_x$ satisfying $\langle {\xi }_x\left(t\right){\xi }_x\left(t^{\prime }\right)\rangle =2D_E{\mu }_0^2\delta (t-t^{\prime })$ and $\langle {\tilde{\xi }}_y\left(t\right){\tilde{\xi }}_y\left(t^{\prime }\right)\rangle =({\theta }_{\mu }^2+{\theta }_G^2+2D_E{\mu }_0^2{T}_{\mu E}^2)\delta (t-t^{\prime })$, respectively. These two noises are correlated, with $\langle {\xi }_x\left(t\right){\tilde{\xi }}_y\left(t^{\prime }\right)\rangle =2D_E{\mu }_0^2{T}_{\mu E}\delta (t-t^{\prime })$.

The pdfs and the correlation functions can be computed as before. In fact, it is clear that this simply amounts to taking the limit β_E → ∞ with $D_E={\eta }_E^2/{\beta }_E$ finite in the previous equations (for instance in Eq (27) for the covariances). The new correlation functions are plotted in Fig 4. As expected, they are almost indistinguishable from those obtained with the original model and they fit the experimental data just as well (this of course is also true for the pdfs).

Much more interesting are the results for the transfer entropy rates and the information flows. Again, there is no need to repeat the calculations as they follow the same lines as before. We now obtain

$$\begin{array}{ccc}\hfill {\overline{\mathcal{T}}}_{X\to Y}& =& \frac{1}{4({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2+D_E{\mu }_0^2{T}_{\mu E}^2)}\times \hfill \\ & & \int dxdy{p}_{xy}(x,y)[{\overline{g}}_y^2(x,y)-{\overline{\overline{g}}}_y^2\left(y\right)]\hfill \end{array}$$ (60)

$$\begin{array}{ccc}\hfill {\overline{\mathcal{T}}}_{Y\to X}& =& \frac{1}{4D_E{\mu }_0^2}\int dxdy{p}_{xy}(x,y)[{\overline{g}}_x^2(x,y)-{\overline{\overline{g}}}_x^2\left(x\right)],\hfill \end{array}$$ (61)

where

$$\begin{array}{ccc}\hfill {\overline{g}}_x(x,y)& =& \frac{1}{p_{xy}(x,y)}\int du{g}_x(x,v,y)p_{xvy}(x,v,y)\hfill \end{array}$$ (62)

$$\begin{array}{ccc}\hfill {\overline{g}}_y(x,y)& =& \frac{1}{p_{xy}(x,y)}\int du{g}_y(x,v,y)p_{xvy}(x,v,y),\hfill \end{array}$$ (63)

and

$$\begin{array}{ccc}\hfill {\overline{\overline{g}}}_x\left(x\right)& =& \frac{1}{p_x\left(x\right)}\int dy{\overline{g}}_x(x,y)p_{xy}(x,y)\hfill \end{array}$$ (64)

$$\begin{array}{ccc}\hfill {\overline{\overline{g}}}_y\left(y\right)& =& \frac{1}{p_y\left(y\right)}\int dx{\overline{g}}_y(x,y)p_{xy}(x,y).\hfill \end{array}$$ (65)

(Again, g_x(x, v, y) and g_y(x, v, y) denote the drift coefficients in Eq (59)). The crucial difference with the results for the original model is that ${\overline{\mathcal{T}}}_{Y\to X}$ is now finite. Similarly, we have

$$\begin{array}{ccc}\hfill {\dot{I}}_{X\to Y}^{flow}& =& \int dxdy{\partial }_x[{\overline{g}}_x(x,y)p_{xy}(x,y)\hfill \\ & & -D_E{\mu }_0^2{\partial }_x{p}_{xy}(x,y)]ln\frac{p_{xy}(x,y)}{p_y\left(y\right)p_x\left(x\right)}\hfill \end{array}$$ (66)

$$\begin{array}{ccc}\hfill {\dot{I}}_{Y\to X}^{flow}& =& \int dxdy{\partial }_y[{\overline{g}}_y(x,y)p_{xy}(x,y)\hfill \\ & & -({\beta }_{\mu }{\eta }_{\mu }^2+{\beta }_G{\eta }_G^2+D_E{\mu }_0^2{T}_{\mu E}^2){\partial }_y{p}_{xy}(x,y)]\hfill \\ & & \times ln\frac{p_{xy}(x,y)}{p_y\left(y\right)p_x\left(x\right)}.\hfill \end{array}$$ (67)

The numerical values of ${\overline{\mathcal{T}}}_{E\to \mu }$ and ${\overline{\mathcal{T}}}_{\mu \to E}$ are given in Table 3. For completeness, we also compare these values with the estimates obtained by the inference method in Table 4. We see that satisfactory results are obtained by properly choosing the sampling time τ. This is also true for the information flows ${\mathcal{I}}_{E\to \mu }^{flow}$ and ${\mathcal{I}}_{\mu \to E}^{flow}$. It is worth noting that the symmetry relation ${\dot{I}}_{E\to \mu }^{flow}=-{\dot{I}}_{\mu \to E}^{flow}$ no longer holds, except for the high IPTG concentration (as T_μE = 0). This contrasts with the preceding case where N_E was modeled by an Ornstein-Uhlenbeck noise. We also observe that the information flows are not always smaller than the transfer entropy rates, contrary to what occurs in bipartite systems. Therefore, the concept of a “sensory capacity” as introduced in Ref. [11] is here ineffective.

Data Availability

All relevant data are within the paper.

Funding Statement

S.L. thanks the Institute of Complex Systems (ISC-PIF), the Region Ile-de-France, and the Labex CelTisPhyBio (No. ANR-10- LBX-0038) part of the IDEX PSL (No. ANR-10-IDEX-0001-02 PSL) for financial support (to DL). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.