Energy Dissipation and Information Flow in Coupled Markovian Systems

Abstract

A stochastic system under the influence of a stochastic environment is correlated with both present and future states of the environment. Such a system can be seen as implicitly implementing a predictive model of future environmental states. The non-predictive model complexity has been shown to lower-bound the thermodynamic dissipation. Here we explore these statistical and physical quantities at steady state in simple models. We show that under quasi-static driving this model complexity saturates the dissipation. Beyond the quasi-static limit, we demonstrate a lower bound on the ratio of this model complexity to total dissipation, that is realized in the limit of weak driving.

1. Introduction

Information theory has long been recognized as fundamentally linked to statistical mechanics [1]. Perhaps most prominently, Landauer showed that information processing can require unavoidable dissipative costs [2]; for example, bit erasure requires that some free energy be dissipated [3,4].

A stochastic system processes information through interaction with its environment: through environment-dependent dynamics the system responds to environmental changes and thereby gains information about the environment [5,6]. For an environment exhibiting temporal correlations, the system carries information about the past, present, and future environmental states. In this way, the system implicitly implements a predictive model of future environmental states [7].

One can quantify this model’s inefficiency by the unnecessary model complexity: information the model retains about the past that does not aid in predicting the future. Recent work established the equivalence between this predictive inefficiency and thermodynamic inefficiency [7], providing another fundamental connection between information theory and statistical mechanics. This connection hints at a design principle for molecular machines operating out of equilibrium [8,9].

These results are potentially applicable to many systems. Biology (where there is a presumptive selective advantage associated with energetic efficiency) furnishes examples of organisms [10], neurons [11], and reaction networks [12] that are capable of learning statistical patterns in their respective environments.

To further illuminate this abstract connection between model complexity and thermodynamic dissipation, here we analytically and numerically explore these statistical and physical quantities in illustrative models. We demonstrate the information learned by the system about its environment per unit energy dissipated (equivalently the ratio of dissipation during system and environmental dynamics) in the limits of quasi-static driving (Table 1) and weak driving (8), which forms the lower bound for generic driving. The dependence of these quantities on the system and environmental parameters motivates a potential guiding principle for functional performance.

Table 1.

Limiting behavior of dissipation ratio. Steady-state dissipation ratio ${\varphi }^{\mathrm{ss}}$ in the various limits of driving strength and speed. These limits are given by the bound in Equation (10), valid in the limit of continuous time.

	Driving Strength	Weak	Strong
Driving Speed		($\mathit{\beta }\mathbf{\Delta }\mathit{E}\ll \mathbf{1}$)	($\mathit{\beta }\mathbf{\Delta }\mathit{E}\gtrsim \mathbf{1}$)
Quasi-static	(${\kappa }_{\mathrm{env}}\ll k_{\mathrm{sys}}$)	${\varphi }^{\mathrm{ss}}=1$	${\varphi }^{\mathrm{ss}}=1$
Intermediate	(${\kappa }_{\mathrm{env}}\sim k_{\mathrm{sys}}$)	${\varphi }^{\mathrm{ss}}={(1+{\kappa }_{\mathrm{env}}/k_{\mathrm{sys}})}^{-1}$	${(1+{\kappa }_{\mathrm{env}}/k_{\mathrm{sys}})}^{-1}\le {\varphi }^{\mathrm{ss}}\le 1$
Fast	(${\kappa }_{\mathrm{env}}\gg k_{\mathrm{sys}}$)	${\varphi }^{\mathrm{ss}}=k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}$	$k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}\le {\varphi }^{\mathrm{ss}}\le 1$

2. Theoretical Background

Consider a stochastic process $\left\{X_t\right|t\in \{0,\Delta t,\dots ,\tau -\Delta t,\tau \}\}$ representing the dynamical evolution of some environmental variable. At a given time, the environment can occupy any of the states $\mathcal{X}$. The time evolution of the environment, $X_t$, is governed by the transition probabilities $p(x_t|{\left\{x_{t^{\prime }}\right\}}_{t^{\prime }=0}^{t-\Delta t})\equiv p(X_t=x_t|{\{X_{t^{\prime }}=x_{t^{\prime }}\}}_{t^{\prime }=0}^{t-\Delta t})$ for $x_t,x_{t^{\prime }}\in \mathcal{X}$. Let another stochastic process $\left\{Y_t\right|t\in \{0,\Delta t,\dots ,\tau -\Delta t,\tau \}\}$ represent the system of interest, which can occupy states $\mathcal{Y}$. Take the dynamics of $Y_t$ to depend on the environmental state via the time-independent conditional transition probabilities $p(y|y^{\prime },x)\equiv p(Y_{t+\Delta t}=y|Y_t=y^{\prime },X_{t+\Delta t}=x)$, where $y,y^{\prime }\in \mathcal{Y}$ and $x\in \mathcal{X}$. We model the evolution of these two stochastic processes using an alternating time step pattern illustrated in Figure 1. For computational simplicity, we take the system and environment to evolve in discrete time steps. However, we set these time steps to be very small compared to the system and environment evolution time-scales in order to closely approximate continuous time evolution. One complete time step is composed of two sub-steps: one work step of environmental dynamics, when the environment does work on the system, followed by one relaxation step of system dynamics, when the system exchanges heat with a thermal bath maintained at temperature T and inverse temperature $\beta \equiv {(k_{\mathrm{B}}T)}^{-1}$.

Figure 1.

Discrete-time system and environmental dynamics. The system $Y_t$ and environment $X_t$ alternate steps, with system evolution during relaxation steps, and environment evolution during work steps.

System dynamics $Y_t$ obey the principle of microscopic reversibility [13]. Reference [7] used such a framework to study the relationship between thermodynamic and information-theoretic quantities. One prominent information-theoretic quantity is the nostalgia$I_{\mathrm{nos}}(t)\equiv I_{\mathrm{mem}}(t)-I_{\mathrm{pred}}(t)$, where the mutual information $I_{\mathrm{mem}}(t)\equiv I[X_t,Y_t]$ [14] between the current system state and past environmental state represents the memory stored by the system about the environment, and the mutual information $I_{\mathrm{pred}}(t)\equiv I[X_{t+\Delta t},Y_t]$ between current system state and future environmental state represents the ability of the system to predict future environmental states. Reference [7] showed that

$$\beta 〈W_{\mathrm{diss}}(t)〉=I_{\mathrm{mem}}(t)-I_{\mathrm{pred}}(t)-\beta 〈\Delta F_{\mathrm{neq}}^{\mathrm{relax}}(t)〉,$$ (1)

where $〈W_{\mathrm{diss}}(t)〉$ is the average total dissipation (defined as the average work done on the system, minus the average change in nonequilibrium free energy of the system) over consecutive work and relaxation steps from t to $t+\Delta t$, and $〈\Delta F_{\mathrm{neq}}^{\mathrm{relax}}(t)〉$ is the average change in nonequilibrium free energy of the system over the relaxation step from t to $t+\Delta t$. The angled brackets indicate that the average system energy and the system entropy are calculated for a particular environmental state, followed by averaging over environmental states. Upon calculating these quantities at each time step, they are combined to calculate the average free energy difference, average work, and average dissipation. Since $\beta 〈\Delta F_{\mathrm{neq}}^{\mathrm{relax}}(t)〉\le 0$ [15],

$$\beta 〈W_{\mathrm{diss}}(t)〉\ge I_{\mathrm{mem}}(t)-I_{\mathrm{pred}}(t).$$ (2)

3. Results

We explore the tightness of the bound (2) through the ratio of nostalgia to dissipation,

$$\varphi (t)\equiv \frac{I_{\mathrm{mem}}(t)-I_{\mathrm{pred}}(t)}{\beta 〈W_{\mathrm{diss}}(t)〉}.$$ (3)

This nostalgia-dissipation ratio is bounded by $0\le \varphi (t)\le 1$ and (after substituting Equation (14) from [7]) can be interpreted as the fraction of dissipation which occurs over work steps,

$$\varphi (t)=\frac{〈W_{\mathrm{diss}}[x_t\to x_{t+\Delta t}]〉}{〈W_{\mathrm{diss}}(t)〉}$$ (4)

where $〈W_{\mathrm{diss}}[x_t\to x_{t+\Delta t}]〉$ is the average dissipation during the work step $x_t\to x_{t+\Delta t}$, and $〈W_{\mathrm{diss}}(t)〉$ is the sum of the average dissipation during consecutive single work and relaxation ($y_t\to y_{t+\Delta t}$) steps. When the environment and system reach steady state, $\varphi$ can be rewritten as:

$${\varphi }^{\mathrm{ss}}=\frac{\ell (t)}{-\beta 〈Q〉},$$ (5)

where $\ell (t)\equiv I[X_{t+\Delta t},Y_{t+\Delta t}]-I[X_{t+\Delta t},Y_t]$ is a learning rate which quantifies the information gained by the system about the current environmental state [16]. The denominator follows from the facts that at steady state −$〈Q〉=〈W〉$ (due to energy conservation) and $〈W〉=〈W_{\mathrm{diss}}〉$ [7]. Refs. [17,18] identify the ratio in Equation (5) as an informational efficiency quantifying the rate at which the system learns about the environment, relative to the total thermodynamic entropy production. By considering (4), these results can be recast in terms of dissipative energy flows.

In order to explore the physical implications of (1) and (2), we investigate the behavior of the relevant information-theoretic and thermodynamic quantities in concrete models that provide physical intuition. We initially restrict our attention to a simple environment model, consisting of two states with a constant transition probability ${\kappa }_{\mathrm{env}}\Delta t$ in each time step.

3.1. Alternating Energy Levels

One of the simplest possible system models with non-trivial behavior is a two-state system with dynamics described by two kinetic rates, $k_{+}$ and $k_{-}$ (Figure 2a). Since we are using discrete time steps, we define the rate k of a given transition to be that of a continuous time model, which is then discretized by choosing $\Delta t$ such that $k\Delta t$ is small, and this transition occurs with probability $k\Delta t$ in each time step. This model possesses a symmetry such that it is unchanged when both the system-state labels and environment-state labels are interchanged. Due to this symmetry, we take $k_{+}\ge k_{-}$ without loss of generality.

Figure 2.

Model kinetics. States and transition rates for models with two system states and two environment states. (a) System equilibration rate and energy gap magnitude and environment transition rate are independent of environment state, but the direction of the energy gap switches with environment state; (b) System equilibration rate and energy gap vary with environment state. Environment transition rate is fixed; (c) System equilibration rate and energy gap and environment transition rate vary with environment state.

Given the constraint of detailed balance [13], such a model describes a two-state system with an energy gap (normalized by temperature) $\beta \Delta E=ln\frac{k_{+}}{k_{-}}$ that flips according to the environment state. System states $y^1$ and $y^2$ are separated by $\Delta E_{12}^A=\Delta E$ when the environment is in state $x^A$ and $\Delta E_{12}^B=-\Delta E$ for environmental state $x^B$. A characteristic rate at which the system reaches equilibrium, and thus becomes correlated with the current environment (and decorrelated with past environmental states), is the minimum transition rate,

$$k_{\mathrm{sys}}\equiv k_{-},$$ (6)

the rate of the rate-limiting step for rearrangement of system probability among its states. The transition ratio $k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}$ expresses this rate relative to the environmental transition rate. Figure 3 shows the steady-state nostalgia $I_{\mathrm{nos}}^{\mathrm{ss}}$, which increases with both $k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}$ and $\beta \Delta E$, and tends to 0 as either $k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}$ or $\beta \Delta E$ approach 0.

Figure 3.

Nostalgia increases with energy gap and system equilibration rate. Nostalgia $I_{\mathrm{nos}}^{\mathrm{ss}}$ as a function of the energy gap $\beta \Delta E$ and transition ratio $k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}$ (${\kappa }_{\mathrm{env}}\Delta t={10}^{-12}$).

The dissipation ratio $\varphi (t)$ approaches a steady-state value ${\varphi }^{\mathrm{ss}}$ for each choice of parameters. Figure 4 shows that ${\varphi }^{\mathrm{ss}}$ follows the same general trends as $I_{\mathrm{nos}}^{\mathrm{ss}}$, increasing with both energy gap magnitude $\beta \Delta E$ and transition ratio $k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}$.

Figure 4.

Dissipation ratio increases with energy gap and system equilibration rate. Steady-state dissipation ratio ${\varphi }^{\mathrm{ss}}\equiv I_{\mathrm{nos}}^{\mathrm{ss}}/\beta 〈W_{\mathrm{diss}}^{ss}〉$ as a function of the energy gap $\beta \Delta E$ and transition ratio $k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}$ (${\kappa }_{\mathrm{env}}\Delta t={10}^{-12}$).

In the limit of large temperature, when the energy gap is small compared to the ambient thermal energy ($\beta \Delta E\ll 1$), ${\varphi }^{\mathrm{ss}}$ reduces to a positive function of the equilibration rates of the system ($k_{\mathrm{sys}}$) and environment (${\kappa }_{\mathrm{env}}$):

$${\varphi }^{\mathrm{ss}}=\frac{1-{\kappa }_{\mathrm{env}}\Delta t}{1-2{\kappa }_{\mathrm{env}}\Delta t+{\kappa }_{\mathrm{env}}/k_{\mathrm{sys}}},\beta \Delta E\ll 1.$$ (7)

This is found by explicitly calculating the steady-state probability distribution. In moving from discrete-time steps to a continuous-time parameter, time step size becomes small compared to system and environment transition times, reducing (7) to

$${\varphi }^{\mathrm{ss}}=\frac{1}{1+{\kappa }_{\mathrm{env}}/k_{\mathrm{sys}}},{\kappa }_{\mathrm{env}}\Delta t,k_{\mathrm{sys}}\Delta t,\beta \Delta E\ll 1.$$ (8)

Thus, in the weak driving (high-temperature) limit ($\beta \Delta E\ll 1$), if the system evolves quickly compared to the environment, most of the dissipation occurs during work steps, the learning rate approaches the total thermodynamic entropy production, and the bound (2) approaches saturation. Conversely (still restricting to high temperature), when the system evolves slowly compared to the environment, most of the dissipation occurs during relaxation steps, the learning rate is small compared to the total thermodynamic entropy production, and the nostalgia is small compared to the bound in (2).

Further, Figure 4 shows that ${\varphi }^{\mathrm{ss}}$ increases with $\beta \Delta E$. Thus, this weak-driving limit gives a non-zero lower bound on ${\varphi }^{\mathrm{ss}}$,

$$\frac{1-{\kappa }_{\mathrm{env}}\Delta t}{1-2{\kappa }_{\mathrm{env}}\Delta t+{\kappa }_{\mathrm{env}}/k_{\mathrm{sys}}}\le {\varphi }^{\mathrm{ss}}\le 1,$$ (9)

or in the limit of small time steps,

$$\frac{1}{1+{\kappa }_{\mathrm{env}}/k_{\mathrm{sys}}}\le {\varphi }^{\mathrm{ss}}\le 1,{\kappa }_{\mathrm{env}}\Delta t,k_{\mathrm{sys}}\Delta t\ll 1.$$ (10)

If the system evolves quickly compared to its environment, nostalgia is the dominant form of dissipation, regardless of $\beta \Delta E$. The limit of quasi-static driving is defined by $k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}\gg 1$. In this limit, ${\varphi }^{\mathrm{ss}}=1$, and therefore the nostalgia (the implicit predictive model inefficiency) is equal to the total dissipation (the thermodynamic inefficiency). The bounds in Equations (9) and (10) therefore hold beyond the quasi-static limit. The bound in Equation (2) can be looser for systems farther from the limit of quasi-static driving. These limits on ${\varphi }^{\mathrm{ss}}$ are laid out in Table 1.

The transition ratio $k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}$ is also equal to the ratio of characteristic timescales ${\tau }_{\mathrm{env}}/{\tau }_{\mathrm{sys}}$. Thus the bound for steady-state dissipation ratio (10) can be recast as

$$\frac{1}{1+N}\le {\varphi }^{\mathrm{ss}}\le 1,{\kappa }_{\mathrm{env}}\Delta t,k_{\mathrm{sys}}\Delta t\ll 1,$$ (11)

for N independent ‘measurements’ the system makes during each environment state [19]. From this perspective, the bound is proportional (up to a multiplicative constant) to the Berg-Purcell lower bound on environmental measurement precision of a single receptor [20].

3.2. Arbitrary System Rates

The results of the previous section were derived for a simple two-state system, in which the energy difference between system states flips with environment transitions, and the system’s equilibration rate is independent of the environment state. We generalize this model to a two-state system with arbitrary rates and hence—by detailed balance—arbitrary energies (Figure 2b). Given the four transition rates $k_{12}^A$, $k_{21}^A$, $k_{12}^B$, and $k_{21}^B$, when the environment is in state $X=x^A$ the system has energy gap (normalized by temperature) $\beta \Delta E_{12}^A=ln\frac{k_{21}^A}{k_{12}^A}$ between state $y^1$ and $y^2$, and a characteristic equilibration rate $k^{\mathrm{A}}=min(k_{12}^A,k_{21}^A)$. Similarly, when the environment is in state $X=x^B$, the corresponding parameters are $\beta \Delta E_{12}^B=ln\frac{k_{21}^B}{k_{12}^B}$ and $k^{\mathrm{B}}=min(k_{12}^B,k_{21}^B)$. Let $\Delta E^A=|\Delta E_{12}^A|$ and $\Delta E^B=|\Delta E_{12}^B|$ be the magnitudes of the energy gaps in environment states $x^A$ and $x^B$, respectively. The energy gaps $\Delta E^A$ and $\Delta E^B$ are free to be aligned ($\Delta E_{12}^A\Delta E_{12}^B>0$) or anti-aligned ($\Delta E_{12}^A\Delta E_{12}^B<0$). A characteristic equilibration rate of the system is thus

$$k_{\mathrm{sys}}=\frac{2}{\frac{1}{k^A}+\frac{1}{k^B}}.$$ (12)

Equations (7) and (8) also apply in this case of arbitrary system rates. Figure 5 shows that across the explored parameter space, the steady-state dissipation ratio ${\varphi }^{\mathrm{ss}}$ lies above the bound (9), with ${\varphi }^{\mathrm{ss}}$ approaching the bound in the weak-driving limit, $\beta (\Delta E^A+\Delta E^B)\ll 1$. We conclude that Equations (9) and (10) apply for arbitrary system rates.

Figure 5.

Lower bound on dissipation ratio for fixed environment transition rate. The steady-state dissipation ratio ${\varphi }^{\mathrm{ss}}$ is lower-bounded by the black curve (9) for all values of the transition ratio $k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}$. Each point corresponds to a particular set of parameters $k^A$, $k^B$, $\beta \Delta E^A$, and $\beta \Delta E^B$. (a) Models in which the energy gaps $\Delta E^A$ and $\Delta E^B$ are anti-aligned; (b) Models in which the energy gaps $\Delta E^A$ and $\Delta E^B$ are aligned (${\kappa }_{\mathrm{env}}\Delta t={10}^{-12}$).

3.3. Arbitrary Environment Rates

Here we generalize our previous assumption of a fixed environmental transition rate ${\kappa }_{\mathrm{env}}$, independent of the present environmental state. We now allow for two different transition rates, ${\kappa }^{AB}$ and ${\kappa }^{BA}$, out of the two states A and B (Figure 2c).

As above, we define the characteristic system equilibration rates $k^A$ and $k^B$ when the environment is in states $X=x^A$ and $X=x^B$, respectively. A characteristic equilibration rate for the system is the harmonic mean of the system transition rates for each environment state, weighted by the rate of switching out of that environmental state:

$$k_{\mathrm{sys}}=\frac{{\kappa }^{AB}+{\kappa }^{BA}}{\frac{{\kappa }^{AB}}{k^A}+\frac{{\kappa }^{BA}}{k^B}}.$$ (13)

For a uniform environmental transition rate (independent of environment state), this reduces to the previous un-weighted harmonic mean (12). Here we define a characteristic environmental rate ${\overline{\kappa }}_{\mathrm{env}}$ as the arithmetic mean of the transition rates between the environment states

$${\overline{\kappa }}_{\mathrm{env}}=\frac{{\kappa }^{AB}+{\kappa }^{BA}}{2}.$$ (14)

With these definitions, Equations (7) and (8) (replacing ${\kappa }_{\mathrm{env}}$ with ${\overline{\kappa }}_{\mathrm{env}}$) apply to this case of arbitrary transition probabilities. Figure 6 shows that across a range of system and environment parameter values, bounds (9) and (10) hold. The proposed bound depends on the system only through $k_{\mathrm{sys}}$, and hence the environmental-state-dependent equilibration rates $k^A,k^B$.

Figure 6.

Lower bound on dissipation ratio for varying environment transition rate. The steady-state dissipation ratio ${\varphi }^{\mathrm{ss}}$ is lower-bounded by the black curve (9) for all values of the transition ratio $k_{\mathrm{sys}}/{\kappa }_{\mathrm{env}}$. Each point corresponds to a particular set of parameters $k^A$, $k^B$, $\beta \Delta E^A$, and $\beta \Delta E^B$. The environment transition rates are ${\kappa }^{AB}=1.8{\overline{\kappa }}_{\mathrm{env}}$ and ${\kappa }^{BA}=0.2{\overline{\kappa }}_{\mathrm{env}}$. (a) Models in which the energy gaps $\Delta E^A$ and $\Delta E^B$ are anti-aligned; (b) Models in which the energy gaps $\Delta E^A$ and $\Delta E^B$ are aligned (${\overline{\kappa }}_{\mathrm{env}}\Delta t={10}^{-12}$).

4. Discussion

Reference [7] described a relationship between dissipation and nostalgia, a novel abstract information-theoretical concept quantifying the information the system stores about its environment that fails to be predictive of future environmental states. Energetically efficient performance requires avoiding this nostalgia. This framework suggests applications in biology, where living things are influenced by, and thus learn about, their environments. Recent explorations of the implications of this relationship have illuminated its behavior in model neurons [21], its relation to sensor performance [18], and the variation of it and related quantities across several biophysical model systems [16].

Here we focused on a physical understanding of the relationships between the information-theoretic and thermodynamic quantities. We calculated the nostalgia and nostalgia-dissipation ratio in some model systems. Calculating these quantities over the parameter space of simple systems helps to establish an intuitive picture: when the system is quick to relax and strongly driven by the environment (energy gaps vary strongly with environment state), the nostalgia provides a tight lower bound on the steady-state dissipation (8); equivalently, the system learns more about the environment per unit heat dissipated.

For fixed equilibration rates, we found the ratio of nostalgia to total dissipation is minimized in the weak-driving limit. Further, the ratio of nostalgia to total dissipation is bounded from below by this weak-driving limit (10), which depends on the system only through its overall equilibration rate. If the system is driven quasi-statically by its environment, this bound dictates that the predictive inefficiency (nostalgia) is responsible for all thermodynamic inefficiency (dissipation). Contexts further from the quasi-static limit can be further from saturating the bound in Equation (2), and hence have a smaller relative contribution from model inefficiency.

One could explore more complex models than the simple Markovian two-state systems and environments described here. One could expand the system to more states [17], or expand the environmental behavior through additional states or non-Markovian dynamics, since this theoretical framework does not restrict the form of these transitions.

Author Contributions

M.E.Q. and D.A.S. conceived and designed the study; M.E.Q. performed the analytic and numerical calculations; M.E.Q. analyzed the data; M.E.Q. and D.A.S. wrote the paper.

Funding

This work is supported by a Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant (D.A.S.) and a Tier-II Canada Research Chair (D.A.S.).

Conflicts of Interest

The authors declare no conflict of interest. The funding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.