Emergence of local irreversibility in complex interacting systems

Abstract

Living systems are fundamentally irreversible, breaking detailed balance and establishing an arrow of time. But how does the evident arrow of time for a whole system arise from the interactions among its multiple elements? We show that the local evidence for the arrow of time, which is the entropy production for thermodynamic systems, can be decomposed. First, it can be split into two components: an independent term reflecting the dynamics of individual elements and an interaction term driven by the dependencies among elements. Adapting tools from nonequilibrium physics, we further decompose the interaction term into contributions from pairs of elements, triplets, and higher-order terms. We illustrate our methods on models of cellular sensing and logical computations, as well as on patterns of neural activity in the retina as it responds to visual inputs. We find that neural activity can define the arrow of time even when the visual inputs do not, and that the dominant contribution to this breaking of detailed balance comes from interactions among pairs of neurons.

I. INTRODUCTION

Living systems consume energy in order to maintain order and function. Being away from equilibrium, we expect that their microscopic dynamics violate detailed balance. Macroscopically, their behaviors define an arrow of time. Despite recent progress in nonequilibrium statistical physics [1-3], there remain basic questions about how irreversibility at one scale emerges from collective dynamics at the scale below. To what extent does the irreversibility of a system arise from interactions between elements, rather than the independent dynamics of the elements themselves? Can simple dynamics involving pairs or triplets of elements build upon one another to generate large-scale irreversibility, thereby defining a macroscopic arrow of time, or do complex biological systems depend on higher-order combinatorial interactions?

To answer these questions, we propose a framework for decomposing the local evidence for the arrow of time in systems with many degrees of freedom. We demonstrate that the local irreversibility can be divided into two nonnegative components: one that reflects the independent irreversibilities of the individual elements, and another that reflects the irreversibility due to interactions between elements. We then show that the interaction term can be further decomposed into contributions from groups of elements of different sizes, from pairs of elements to triplets to complex higher-order terms. In this way, one can determine not only whether the arrow of time arises from the dependencies between elements, but also the specific types of dynamics from which it emerges [4]. This decomposition is similar in spirit to the idea of connected correlations in the decomposition of the entropy itself [5].

We apply our methods to investigate the arrow of time in neural activity. Our visual perception is built out of the patterns of electrical activity of cells in the retina, and evidence for the arrow of time must be found in these patterns. Recent experiments that record the activity of many retinal neurons simultaneously [6,7] make it possible for us to estimate all the relevant quantities directly, without introducing any model assumptions, in groups of up to five cells. We find that roughly two-thirds of these groups exhibit significant irreversibility, even when the movies shown to the retina are completely reversible. Thus, collective neural activity can define an arrow of time even when the visual inputs do not. Moreover, across distinct stimulus ensembles, we consistently find that the local irreversibility is dominated by the dynamics of neuron pairs. Together, these results demonstrate that neuronal populations can define an arrow of time that (1) emerges primarily from pairwise dynamics and (2) does not merely reflect the irreversibility of the stimulus.

The paper is organized as follows. In Sec. II we define the local irreversibility and multipartite dynamics. In Sec. III we show analytically that the local irreversibility of a multipartite system can be split into two nonnegative terms, the first stemming from the independent elements and the second arising from the interactions between elements. In Sec. IV we compare these independent and interaction irreversibilities in a simple model sensing system. In Sec. V we show that the irreversibility due to interactions can be further decomposed into a series of contributions from pairs of elements, triplets, and higher-order terms. In Sec. VI we illustrate this decomposition using a minimal model of logical computations. In Sec. VII we apply the above methods to investigate the irreversibility of neuronal dynamics in the vertebrate retina. Finally, in Sec. VIII we provide conclusions and outlook, highlighting directions for future work.

II. LOCAL IRREVERSIBILITY AND MULTIPARTITE DYNAMICS

When a system is reversible, its dynamics obey detailed balance, and there are no net fluxes between states [Fig. 1(a)]. By contrast, for an irreversible system, fluxes from one state to another break detailed balance [Fig. 1(b)]. Critically, such irreversible dynamics establish an arrow of time: Just by observing the evolution of the system, one can distinguish whether time is flowing forward or backward.

FIG. 1.

Irreversibility and multipartite dynamics. (a, b) A simple three-state system, with states x represented as circles and joint transition probabilities P(x → x′) as arrows. (a) In a reversible system, there are no net fluxes of transitions between states, the dynamics obey detailed balance, and there is no evidence for the arrow of time. (b) Irreversible systems exhibit net fluxes between states, thereby breaking detailed balance and establishing an arrow of time. (c) A multipartite system composed of two binary spins. Only one spin is allowed to change state at a time, thus disallowing the transitions indicated by red arrows.

To quantify irreversibility, consider a system with joint transition probabilities

$$P(x\to x^{\prime })\equiv \text{Prob}[x_t=x,x_{t+1}=x^{\prime }],$$ (1)

where x_t is the state of the system at time t. In words, this is the probability of observing the state x followed by the state x′, and should not be confused with the conditional transition probabilities Prob[x_t+1 = x′ ∣ x_t = x]. The evidence that these dynamics carry about the arrow of time is quantified by the relative entropy, or Kullback-Leibler divergence, between the forward- and reverse-time transition probabilities [8],

$$\stackrel{.}{I}=\sum _{x,x^{\prime }}P(x\to x^{\prime })\log \left[\frac{P(x\to x^{\prime })}{P(x^{\prime }\to x)}\right],$$ (2)

where if we choose base two for the logarithms then the evidence is measured in bits. If a system obeys detailed balance, such that P(x → x′) = P(x′ → x) for all pairs of states x and x′, then this local irreversibility vanishes [Fig. 1(a)]. Conversely, any violation of detailed balance, such that P(x → x′) ≠ P(x′ → x), leads to an increase in the local irreversibility [Fig. 1(b)].

For Markov systems, the transition probabilities P(x → x′) completely define the dynamics, and so İ captures all available information about the arrow of time. Notably, if the states x and x′ include all of the microscopic degrees of freedom in a system, then, under reasonable assumptions, Eq. (2) defines the physical rate at which the system produces entropy [9,10]. In general, if we don’t observe all the relevant degrees of freedom then the dynamics of the observable states x will be non-Markovian, but İ still has a precise meaning: it represents the local evidence for the arrow of time.

We are interested in systems where the overall state x consists of states {x_i} for many interacting elements, i = 1, 2, … , N. Given sufficient temporal resolution, no two elements will change state at exactly the same time. In this limit, the dynamics are defined by the joint probabilities $P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})$ of one element i transitioning from x_i to $x_{\mathrm{i}}^{\prime }$ and the rest of the system remaining in the same state, denoted x_−i [Fig. 1(c)]. Such dynamics, which are referred to as multipartite, exhibit a number of useful properties [11,12]. Chief among these properties is the fact that the local irreversibility simplifies to a sum over the individual elements:

$$\stackrel{.}{I}=\sum _{x,x^{\prime }}P(x\to x^{\prime })\log \left[\frac{P(x\to x^{\prime })}{P(x^{\prime }\to x)}\right]$$ (3)

$$=\sum _x\sum _{\mathrm{i}=1}^N\sum _{x_{\mathrm{i}}^{\prime }}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})\log \left[\frac{P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})}{P(x_{\mathrm{i}}^{\prime }\to x_{\mathrm{i}},x_{-\mathrm{i}})}\right]$$ (4)

$$=\sum _{\mathrm{i}=1}^N\sum _{x_{-\mathrm{i}}}\sum _{x_{\mathrm{i}},x_{\mathrm{i}}^{\prime }}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})\log \left[\frac{P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})}{P(x_{\mathrm{i}}^{\prime }\to x_{\mathrm{i}},x_{-\mathrm{i}})}\right]$$ (5)

$$=\sum _{\mathrm{i}=1}^N{\stackrel{.}{I}}_{\mathrm{i}},$$ (6)

where

$${\stackrel{.}{I}}_{\mathrm{i}}=\sum _{x_{-\mathrm{i}}}\sum _{x_{\mathrm{i}},x_{\mathrm{i}}^{\prime }}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})\log \left[\frac{P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})}{P(x_{\mathrm{i}}^{\prime }\to x_{\mathrm{i}},x_{-\mathrm{i}})}\right]$$ (7)

is the local irreversibility associated with element i.

III. INDEPENDENT AND INTERACTION IRREVERSIBILITY

We are now prepared to investigate the impact of interactions between elements on the irreversibility of a system. To begin, consider a hypothetical system in which the elements do not interact. In this case, the transitions of each element i are completely defined by the marginal transition probabilities

$$P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })=\sum _{x_{-\mathrm{i}}}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}}),$$ (8)

and thus the independent irreversibility of element i is given by

$${\stackrel{.}{I}}_{\mathrm{i}}^{\text{ind}}=\sum _{x_{\mathrm{i}},x_{\mathrm{i}}^{\prime }}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })\log \left[\frac{P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })}{P(x_{\mathrm{i}}^{\prime }\to x_{\mathrm{i}})}\right].$$ (9)

How does this independent irreversibility compare to the true irreversibility in Eq. (7)? To answer this question, we consider the difference ${\stackrel{.}{I}}_{\mathrm{i}}-{\stackrel{.}{I}}_{\mathrm{i}}^{\text{ind}}$, which reflects the local irreversibility of element i due to interactions with the rest of the system. Notably, we find that this difference—which we refer to as the interaction irreversibility ${\stackrel{.}{I}}_{\mathrm{i}}^{\text{int}}$ of element i—is itself an average of KL divergences,

$$\begin{gathered} {\stackrel{.}{I}}_{\mathrm{i}}^{\text{int}}={\stackrel{.}{I}}_{\mathrm{i}}-{\stackrel{.}{I}}_{\mathrm{i}}^{\text{ind}} \\ =\sum _{x_{-\mathrm{i}}}\sum _{x_{\mathrm{i}},x_{\mathrm{i}}^{\prime }}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})(\log \left[\frac{P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})}{P(x_{\mathrm{i}}^{\prime }\to x_{\mathrm{i}},x_{-\mathrm{i}})}\right]\phantom{)} \\ -\phantom{(}\log \left[\frac{P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })}{P(x_{\mathrm{i}}^{\prime }\to x_{\mathrm{i}})}\right]) \end{gathered}$$ (10)

$$=\sum _{x_{-\mathrm{i}}}\sum _{x_{\mathrm{i}},x_{\mathrm{i}}^{\prime }}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})\log \left[\frac{P(x_{-\mathrm{i}}\mid x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })}{P(x_{-\mathrm{i}}\mid x_{\mathrm{i}}^{\prime }\to x_{\mathrm{i}})}\right]$$ (11)

$$=\sum _{x_{\mathrm{i}},x_{\mathrm{i}}^{\prime }}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })D_{KL}[P(x_{-\mathrm{i}}\mid x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })\Vert P(x_{-\mathrm{i}}\mid x_{\mathrm{i}}^{\prime }\to x_{\mathrm{i}})],$$ (12)

where $P(x_{-\mathrm{i}}\mid x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })=P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})∕P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })$ is the conditional probability of the state x_−i of the rest of the system given a transition $x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime }$ in element i.

Equation (12) immediately tells us that ${\stackrel{.}{I}}_{\mathrm{i}}^{\text{int}}⩾0$, thereby establishing that the presence of interactions can only increase the local irreversibility of a system. Moreover, the interaction irreversibility ${\stackrel{.}{I}}_{\mathrm{i}}^{\text{int}}$ of element i admits an insightful information-theoretic interpretation: it is the amount of information that one gains about the state x_−i of the rest of the system by observing the forward-time dynamics of element i rather than the reverse-time dynamics [8]. Thus, if i’s forward- and reverse-time dynamics contain the same information about the rest of the system, then interactions with element i do not contribute to the arrow of time (${\stackrel{.}{I}}_{\mathrm{i}}^{\text{int}}=0$), and all of i’s local irreversibility arises from independent dynamics (${\stackrel{.}{I}}_{\mathrm{i}}={\stackrel{.}{I}}_{\mathrm{i}}^{\text{ind}}$). Importantly, we note that Eqs. (10)–(12) require multipartite dynamics; if multiple elements can change state at once, then the interaction irreversibility İ^int is ill-defined (see Appendix A).

Together, Eqs. (9)–(12) establish our first result: that the local irreversibility of a system can be split into two nonnegative components,

$$\stackrel{.}{I}={\stackrel{.}{I}}^{\text{ind}}+{\stackrel{.}{I}}^{\text{int}},$$ (13)

where ${\stackrel{.}{I}}^{\text{ind}}={\sum }_{\mathrm{i}=1}^N{\stackrel{.}{I}}_{\mathrm{i}}^{\text{ind}}$ is the independent irreversibility of the system (reflecting the local irreversibilities of the individual elements) and ${\stackrel{.}{I}}^{\text{int}}={\sum }_{\mathrm{i}=1}^N{\stackrel{.}{I}}_{\mathrm{i}}^{\text{int}}$ is the interaction irreversibility (reflecting the local irreversibility due to the dependencies between elements).

IV. DECOMPOSING IRREVERSIBILITY IN A SENSING SYSTEM

To illustrate the decomposition in Eq. (13), we examine a sensing system, wherein a sensing variable y attempts to copy an environmental variable x [Fig. 2(a)]. Such sensing networks have been a topic of significant focus in nonequilibrium statistical mechanics [3,11,13-15], revealing the thermodynamic costs of simple computations in living systems [3,13,15-18].

FIG. 2.

Independent and interaction irreversibility in a sensing system. (a) Sensing system with an environmental variable x (left) and a sensing variable y (right), each with three states. At each point in time, one of the two variables is updated at random. With probability p_x, the environmental variable x rotates clockwise, and with probability p_y, the sensing variable y copies x. (b) Fluxes between states of the sensing system (for p_x, p_y > 1/3) induced by the environmental variable (blue) or the sensing variable (red). (c, d) Irreversibility İ (c), independent irreversibility İ^ind (d), and interaction irreversibility İ^int (e) as functions of p_x and p_y. While İ grows with both p_x and p_y, İ^ind only increases with p_x, and İ^int mostly increases with p_y, thereby distinguishing the two sources of irreversibility in the system.

Here we consider an environmental variable x with three states and dynamics defined by

$$\begin{gathered} x^{\prime } \\ P(x^{\prime }\mid x)=\left(\begin{array}{ccc}\hfill \frac{1}{2}(1-p_x)\hfill & \hfill p_x\hfill & \hfill \frac{1}{2}(1-p_x)\hfill \\ \hfill \frac{1}{2}(1-p_x)\hfill & \hfill \frac{1}{2}(1-p_x)\hfill & \hfill p_x\hfill \\ \hfill p_x\hfill & \hfill \frac{1}{2}(1-p_x)\hfill & \hfill \frac{1}{2}(1-p_x)\hfill \end{array}\right)x, \end{gathered}$$ (14)

where P_x is the probability of x increasing from one state to the next [Fig. 2(a), left]. Meanwhile, the dynamics of the sensing variable y are given by

$$\begin{gathered} y^{\prime } \\ P(y^{\prime }\mid x)=\left(\begin{array}{ccc}\hfill p_y\hfill & \hfill \frac{1}{2}(1-p_y)\hfill & \hfill \frac{1}{2}(1-p_y)\hfill \\ \hfill \frac{1}{2}(1-p_y)\hfill & \hfill p_y\hfill & \hfill \frac{1}{2}(1-p_y)\hfill \\ \hfill \frac{1}{2}(1-p_y)\hfill & \hfill \frac{1}{2}(1-p_y)\hfill & \hfill p_y\hfill \end{array}\right)x, \end{gathered}$$ (15)

where p_y is the probability that y copies x [Fig. (2(a), right]. Randomly picking one variable to update at a time, one can solve for the joint transition probabilities of the combined system P(x, y → x′, y′); for details see Appendix B. Notably, since the dynamics are Markovian, İ reflects the full (not just local) irreversibility of the system.

If p_x = p_y = 1/3, then both variables behave randomly, and the system obeys detailed balance. By contrast, if p_x or p_y > 1 /3, then the tendencies for x to increase and y to copy x give rise to fluxes between the states of the system [Fig. 2(b)], thereby breaking detailed balance. Indeed, the irreversibility İ increases with both p_x and p_y [Fig. 2(c)]. The independent irreversibility İ^ind, however, only increases with p_x, capturing the quickening dynamics of x [Fig. 2(d)]. Meanwhile, the interaction irreversibility İ^int primarily increases with p_y, capturing the strengthening dependence of y on x [Fig. 2(e)]. We therefore find that the independent irreversibility is generated by the individual motion of the environmental variable, while the interaction irreversibility arises predominantly from the dependence of the sensing variable on the environment. In this way, the decomposition in Eq. (13) reveals the distinct ways that the environmental and sensing variables generate irreversibility.

V. IRREVERSIBILITY DUE TO kth-ORDER DYNAMICS

Can we tell whether the arrow of time emerges from the dynamics of two or three elements at a time, or whether we require higher-order information about the system as a whole? Answering this question requires further decomposing the local irreversibility into contributions from pairs of elements, triplets, and so on. For now, consider the marginal dynamics of pairs of elements i and j, namely, the marginal transition probabilities

$$P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{\mathrm{j}})=\sum _{x_{-\{\mathrm{i},\mathrm{j}\}}}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}}),$$ (16)

$$P(x_{\mathrm{j}}\to x_{\mathrm{j}}^{\prime },x_{\mathrm{i}})=\sum _{x_{-\{\mathrm{i},\mathrm{j}\}}}P(x_{\mathrm{j}}\to x_{\mathrm{j}}^{\prime },x_{-\mathrm{j}}).$$ (17)

Imagine a hypothetical system that matches these marginal dynamics for all pairs i and j, but otherwise contains minimal information about the arrow of time, so that the dynamics are maximally reversible. This minimal irreversibility, which we denote İ(2), sets a lower bound on the true local irreversibility İ, capturing all of the local irreversibility in pairs of elements and nothing more. In this way, by casting our decomposition as an optimization problem, we are able to directly translate knowledge about a system into a lower bound on its irreversibility. From a practical perspective, the local irreversibility İ is convex (see Appendix C), and so there exist efficient algorithms for computing global minima. In fact, the equivalent problem of minimizing entropy production has garnered significant attention in nonequilibrium physics [12,16,19,20], dating back to the foundational work of Onsager and Prigogine [21,22].

In general, one can compute the minimum irreversibility İ^(k) consistent with the dynamics of k elements at a time. Since these kth-order dynamics contain all of the information about smaller groups of size 1, 2, … , k – 1, the minimum irreversibilities İ^(k) form a hierarchy of lower bounds that increase toward the true local irreversiblity İ:

$$0⩽{\stackrel{.}{I}}^{(1)}⩽{\stackrel{.}{I}}^{(2)}⩽\cdots \le {\stackrel{.}{I}}^{(N-1)}⩽{\stackrel{.}{I}}^{(N)}=\stackrel{.}{I},$$ (18)

where N is the size of the system. There are several things to note about these inequalities. First, for thermodynamic systems, the zeroth-order bound (İ ⩾ 0) is the second law of thermodynamics, which follows from the fact that İ is a KL divergence without any knowledge of the system dynamics. Second, as one might suspect, the first-order irreversibility İ⁽¹⁾—that is, the minimum irreversibility consistent with individual dynamics—is equivalent to the independent irreversibility İ^ind (see Appendix D). Finally, since the Nth-order dynamics contain a full description of the transition probabilities $P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})$, we have İ^(N) = İ.

Inspecting the hierarchy in Eq. (18), we see that the local irreversibility due to kth-order dynamics alone can be captured captured by the difference ${\stackrel{.}{I}}_{\text{int}}^{(k)}={\stackrel{.}{I}}^{(k)}-{\stackrel{.}{I}}^{(k-1)}⩾0$, which we refer to as the interaction irreversibility of order k. Indeed, combining these contributions from ${\stackrel{.}{I}}_{\text{int}}^{(1)}={\stackrel{.}{I}}^{(1)}={\stackrel{.}{I}}^{\text{ind}}$ to ${\stackrel{.}{I}}_{\text{int}}^{(N)}$, we arrive at a full decomposition of the local irreversibility:

$$\stackrel{.}{I}=\underset{{\stackrel{.}{I}}^{\text{ind}}}{\underbrace{{\stackrel{.}{I}}_{\text{int}}^{(1)}}}+\underset{{\stackrel{.}{I}}^{\text{int}}}{\underbrace{{\stackrel{.}{I}}_{\text{int}}^{(2)}+{\stackrel{.}{I}}_{\text{int}}^{(3)}+\cdots +{\stackrel{.}{I}}_{\text{int}}^{(N)}}},$$ (19)

which is our main contribution. We note that this decomposition is in many ways similar to the decomposition of the entropy itself into connected components [5].

VI. DECOMPOSING IRREVERSIBILITY OF LOGICAL FUNCTIONS

To illustrate how irreversibility arises from dynamics of different orders, we apply the decomposition in Eq. (19) to a class of noisy logical functions. Specifically, we consider binary variables x and y that change state at each time step with probability p_flip, and third binary variable z that is the output of a logical function with a probability of error p_error [Fig. 3(a); see Appendix E for a full description]. As for the sensing system in Fig. 2, because the dynamics are Markovian the local irreversibility İ represents the full irreversibility of the system.

FIG. 3.

Decomposing irreversibility in noisy logical functions. (a) System of three binary variables x, y, and z, where z performs a noisy logical function on the inputs x and y. At each point in time, one of the variables is updated at random. With probability p_flip, the inputs x and y change value, and with probability p_error, the output z fails to perform the specified function (see Appendix E). (b, c) Fluxes between states of the system when z either copies x (b) or copies y (c). (d) Irreversibilities İ (full), ${\stackrel{.}{I}}_{\text{int}}^{(2)}$ (pairwise interaction), and ${\stackrel{.}{I}}_{\text{int}}^{(3)}$ (triplet interaction) vs p_error for different values of p_flip. For both of the copy functions, irreversibility arises entirely from pairwise dynamics $\stackrel{.}{I}={\stackrel{.}{I}}_{\text{int}}^{(2)}$, while triplet dynamics do not contribute ${\stackrel{.}{I}}_{\text{int}}^{(3)}=0$.

We note that binary variables in steady state, such as those considered here, cannot break detailed balance on their own (Appendix F). Thus, for binary steady-state systems, the independent irreversibility vanishes (İ^ind = 0), such that the arrow of time arises entirely from the interactions between the elements, $\stackrel{.}{I}={\stackrel{.}{I}}^{\text{int}}={\stackrel{.}{I}}_{\text{int}}^{(2)}+\cdots +{\stackrel{.}{I}}_{\text{int}}^{(N)}$. Specifically for the logical functions [Fig. 3(a)], there are only two contributions to the irreversibility: that due to pairwise dynamics ${\stackrel{.}{I}}_{\text{int}}^{(2)}$ and that due to the full triplet dynamics ${\stackrel{.}{I}}_{\text{int}}^{(3)}$.

To begin, consider a simple function where z copies either x or y while ignoring the other input [Figs. 3(b) and 3(c)]; these are binary simplifications of the sensing system in Fig. 2. As _perror increases—that is, as the accuracy of the function decreases—we find that the irreversibility İ decreases [Fig. 3(d)]. Indeed, as p_error approaches 1/2, the output z completely decouples from the inputs x and y, and the system becomes reversible (İ = 0). Additionally, the arrow of time vanishes if the inputs x and y are static (p_flip = 0) and grows as the inputs become more dynamic [that is, as p_flip increases; Fig. 3(d)]. Visualizing the fluxes between states of the system, we see that the tendency for z to copy x (equivalently, y) induces fluxes only in the x – z (or y – z) plane [Figs. 3(b) and 3(c)]. Accordingly, for all values of p_flip and p_error, the irreversibility arises entirely from pairwise dynamics ($\stackrel{.}{I}={\stackrel{.}{I}}_{\text{int}}^{(2)}$), while triplet dynamics do not contribute to the irreversibility [${\stackrel{.}{I}}_{\text{int}}^{(3)}=0$; Fig. 3(d)].

For comparison, consider the and and or functions [Figs. 4(a) and 4(b)]. Just as for the copy functions (Fig. 3), the irreversibilities of and and or (which we note are identical) increase both with the accuracy of the system (as p_error decreases) and with the speed of dynamics [as p_flip increases; Fig. 4(c)]. However, in contrast to the copy functions, the full dynamics of and and or cannot be deduced from pairs of variables alone. Thus, the irreversibility arises from a combination of both pairwise and triplet dynamics [Fig. 4(c)]. Finally, for the xor function, the behavior of the system becomes apparent only when all three variables are observed simultaneously [Fig. 4(d)]. As such, the irreversibility of xor arises entirely from triplet dynamics ($\stackrel{.}{I}={\stackrel{.}{I}}_{\text{int}}^{(3)}$), while the pairwise dynamics are completely reversible [${\stackrel{.}{I}}_{\text{int}}^{(2)}=0$; Fig. 4(e)]. This is consistent with the status of xor as the prototype of combinatorial interactions.

FIG. 4.

Decomposing irreversibility in AND, OR, and XOR functions. (a, b) For logical systems defined as in Fig. 3(a), we illustrate the fluxes between states when z executes either and (a) or or (b). (c) Irreversibilities İ (full), ${\stackrel{.}{I}}_{\text{int}}^{(2)}$ (pairwise interaction), and ${\stackrel{.}{I}}_{\text{int}}^{(3)}$ (triplet interaction) of the and and or systems vs p_error for different values of p_flip. Irreversibility arises from a combination of pairwise $({\stackrel{.}{I}}_{\text{int}}^{(2)}>0)$ and triplet $({\stackrel{.}{I}}_{\text{int}}^{(3)}>0)$ dynamics. (d, e) Fluxes (d) and irreversibilities (e) when z performs xor. Irreversibility arises entirely from triplet dynamics $(\stackrel{.}{I}={\stackrel{.}{I}}_{\text{int}}^{(3)})$, while pairwise dynamics are reversible $({\stackrel{.}{I}}_{\text{int}}^{(2)}=0)$. (f, g) Minimum irreversibilities İ^(k) (f) and interaction irreversibilities ${\stackrel{.}{I}}_{\text{int}}^{(k)}$ (g), normalized by the full irreversibility İ, as functions of the order k for different logical functions. The errorbars for and and or reflect the small variability in İ^(k)/İ and ${\stackrel{.}{I}}_{\text{int}}^{(k)}∕\stackrel{.}{I}$ over the range of different p_error and p_flip values.

The results of this section are summarized in Figs. 4(f) and 4(g), where we plot the minimum irreversibilities İ^(k) [Fig. 4(f)] and interaction irreversibilities ${\stackrel{.}{I}}_{\text{int}}^{(k)}$ [Fig. 4(g)] of the different logical functions, normalized by the full irreversibilities İ. Since the systems all consist of binary steady-state dynamics, the first-order irreversibilities ${\stackrel{.}{I}}^{(1)}={\stackrel{.}{I}}_{\text{int}}^{(1)}$ vanish, and therefore the independent dynamics do not define an arrow of time (İ^ind = 0). For the copy functions [Figs. 3(b) and 3(c)], irreversibility is driven entirely by second-order dynamics; for the and and or functions [Figs. 4(a) and 4(b)], the arrow of time arises from a combination of second- and third-order dynamics; and for the xor function [Fig. 4(d)], irreversibility is driven entirely by third-order dynamics [see Fig. 4(g)]. In this way, the decomposition in Eq. (19) can be used to uncover the order of the dynamics that generate irreversibility in interacting systems.

VII. DECOMPOSING IRREVERSIBILITY IN NEURONAL POPULATIONS

Using the framework developed above, we are ultimately interested in understanding how irreversibility emerges in biological systems. Here we study electrical activity in groups of neurons at the output of the retina. These ganglion cells provide all the data that the brain has about the visual world, and hence their state provides the ingredients out of which visual perceptions are synthesized, including our perception of the arrow of time. Importantly, information about visual stimuli is encoded not just in the firing of individual neurons, but also in the web of dependencies between neurons [7,23,24]. It remains unknown, however, whether groups of neurons exhibit fluxes between collective states—thereby breaking detailed balance—and if so, whether such irreversibility arises from pairs of neurons or from complicated higher-order dynamics.

Here we analyze experiments on the salamander retina [Fig. 5(a)], where it is possible to record form many neurons simultaneously as they respond to complex visual stimuli [6]. These experiments explored three very different kinds of visual inputs: natural movies [Fig. 5(b)], a single horizontal bar whose vertical motion is equivalent to a Brownian particle on a spring [Fig. 5(c)], and the Brownian bar with precise repetitions of the same trajectory. Although this was not the goal of the original experiments, we note that the natural movies violate time-reversal invariance, being easily recognized when played forward vs backward, while the Brownian bar is an equilibrium system and obeys detailed balance. Appendix G gives a more detailed description of the experimental setup and procedures from Ref [6].

FIG. 5.

Broken detailed balance in a group of neurons. (a) Dots mark the times of action potentials (“spikes”) from 53 neurons in the salamander retina responding to a visual stimulus (see Appendix G for experimental details). (b, c) The same 53 neurons are exposed to three different stimuli: a natural movie of fish swimming (b), a horizontal bar whose movement is defined by a Brownian particle on a spring (c), and the same Brownian bar in panel (c) but with one trajectory repeated multiple times. (d) To record multipartite transitions [see Fig. 1(c)], we slide a window of width Δt = 20 ms along the time series. A neuron transitions to active when a spike enters the front of the window (left), or inactive when a spike exits the back of the window (center). Self-transitions can occur when a cell spikes twice within the same window (right). (e–h) A random group of three neurons responding to a natural movie. (e) Probabilities P(x → x′), where black entries indicate transitions that are disallowed under multipartite dynamics. (f) Changes in state probabilities ΔP(x) are small relative to their standard deviations σ_ΔP(x), indicating that the system is in steady state. (g, h) Fluxes P(x → x′) − P(x′ → x) illustrated as a matrix (g) and as a directed network (h). The presence of fluxes demonstrates that the neurons break detailed balance, defining an arrow of time.

A. Broken detailed balance in neuronal dynamics

The problems of detecting and quantifying irreversibility in data have garnered significant attention in the statistical mechanics of living systems [3,20,25-28]. To detect irreversibility, one must simply search for violations of detailed balance; namely, fluxes between the states of a system [3,25,26]. To quantify the irreversibility of a system, however, one must estimate or bound İ from time-series measurements [20,26-28]. Here, in addition to estimating the local irreversibility İ, we further wish to decompose İ into contributions from dynamics of various orders [as in Eq. (19)]. In order to do so—that is, in order to compute the minimum irreversibilities I^(k) consistent with kth-order dynamics—we must begin by estimating the transition probabilities $P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})$ themselves.

We consider a neuron i active (x_i = 1) if it generates an action potential (“spike”) at least once within a time window of width Δt = 20 ms, or inactive (x_i = 0) if it is silent. In this way, the collective state of N neurons is a binary vector x = {x₁, x₂, x_N}. As we slide the window along the time series, it is almost always the case that only one cell i changes state at a time, either by having a spike enter the front of the window [Fig. 5(d), left window] or exit the back of the window [Fig. 5(d), center window]. Each time this occurs, we record a new transition between states x → x′. For completeness, we remark that self-transitions can occur when a cell spikes twice within the same window [Fig. 5(d), right window]; however, we note that the all-silent state {0, … , 0} cannot have self-transitions, since no spike enters or exits the sliding window. In the rare instances when two spikes enter or exit the window at exactly the same time (within the experimental resolution of 0.1ms), we break ties by adding small random noise to the spike times, thus yielding multipartite dynamics wherein only one cell changes state at a time.

For example, in Fig. 5(e) we illustrate the probabilities of transitions between the states of N = 3 neurons responding to a natural movie [Fig. 5(b)]. Notably, the changes in state probabilities ΔP(x) = ∑_x′ P(x′ → x) – P(x → x′) are small relative to errors [Fig. 5(f)], indicating that the group of neurons is in a stochastic steady state. As discussed above (and in Appendix F), binary steady-state variables cannot break detailed balance on their own. Thus, even though neurons violate detailed balance at the subcellular scale, at the coarse-grained level of binary activity the individual neurons in Figs. 5(e) and 5(f) do not define a local arrow of time. However, when examined as a group, we find that the three cells exhibit fluxes between collective states [Figs. 5(g)-5(h)], thereby breaking detailed balance. In combination, these results establish that the group of neurons operates at a nonequilibrium steady state.

B. Local irreversibility depends on stimulus

We are now prepared to estimate the collective irreversibility of groups of neurons. We note that neurons—indeed, biological systems generally—can have long-range temporal dependencies. Thus, in contrast to the Markov systems examined in previous sections (Figs. 2-4), here İ reflects the local (rather than total) irreversibility of the system. As with other information-theoretic quantities, estimating the local irreversibility from data is challenging, and prone to systematic errors due to finite data. As described in Appendix H, we find that these can be controlled using the strategy of Ref. [23] if we restrict our attention to groups of no more than N = 5 cells.

After correcting for finite-data effects, out of 100 random five-cell groups, across the different stimuli we find that 60% – 68% exhibit significant local irreversibility İ, thereby defining an arrow of time. Moreover, for all cell groups and all stimuli, we find that the dynamics are in steady state (see Appendix J), indicating that individual cells do not break detailed balance, and therefore that any local arrow of time arises from the collective dynamics of multiple neurons. Surprisingly, despite the fact that the Brownian bar is completely reversible, neuronal dynamics are more irreversible when responding to this stimulus than the natural movie [Fig. 6(a)]. Moreover, the local irreversibility is even larger when the same Brownian trajectory is repeated multiple times [Fig. 6(a)], suggesting that a repeated input can induce a stronger arrow of time in the neuronal responses. We confirm that these differences in local irreversibility hold even after accounting for variations in the overall rate of spiking across the stimulus ensembles [Fig. 6(b)]. Additionally, the same ordering of stimuli holds for all group sizes from N = 2 to N = 5 cells [Figs. 6(c) and 6(d)]. These results demonstrate that the arrow of time in neuronal activity does not simply reflect the irreversibility of the stimulus. Instead, neuronal dynamics can define an arrow of time even when the stimulus does not.

FIG. 6.

Stimulus dependence of local irreversibility. (a) Distributions of local irreversibilities İ for five-cell groups responding to a natural movie (blue), a Brownian bar (red), and a repeated Brownian bar (green). (b) The same as panel (a), but normalized to bits per second to account for variations in spike rates across stimuli. In panels (a, b), out of 100 random five-cell groups, we include only those with significant local irreversibility (see Appendix H). (c, d) Local irreversibility (c) and after normalizing to bits per second (d) for different stimuli as functions of the number of cells in a group. Data points are averaged over 100 random groups.

C. Local irreversibility arises from low-order dynamics

To implement the decomposition of local irreversibility from Eq. (19), we need numerical methods to construct the probability distributions that minimize İ while matching the observed kth-order dynamics. We provide one such method for binary systems in Appendix I.

To recall, for binary steady-state systems, the independent irreversibility İ^ind vanishes, and so the local irreversibility arises entirely from the collective dynamics of two or more variables (see Appendix F). For groups of N = 5 cells responding to the natural movie, we find that pairwise dynamics account for much more of the local irreversibility than higher-order dynamics [Fig. 7(a)]. In fact, across all stimuli, pairwise dynamics generate 66% – 74% of the local irreversibility [Fig. 7(b)], more than third-, fourth-, and fifth-order dynamics combined [Fig. 7(c)]. Moreover, the fraction of the irreversibility captured by pairwise dynamics increases significantly with the local irreversibility itself [Fig. 7(d)], demonstrating that groups of neurons that operate further from detailed balance do so in an even more pairwise fashion. Perhaps most notably, despite the fact that the magnitude of the local irreversibility varies significantly from one stimulus to another (Fig. 6), we find that the proportions of irreversibility captured by different types of dynamics remain consistent across stimuli [Figs. 7(b) and 7(c)].

FIG. 7.

Decomposing local irreversibility in neuronal activity. (a) Distributions of interaction irreversibilities ${\stackrel{.}{I}}_{\text{int}}^{(k)}$ of different orders k for five-cell groups responding to a natural movie. (b, c) Minimum irreversibilities İ^(k) (b) and interaction irreversibilities ${\stackrel{.}{I}}_{\text{int}}^{(k)}$ (c), normalized by the true local irreversibilities İ, as functions of the order k averaged over five-cell groups. (d) The fraction of pairwise irreversibility ${\stackrel{.}{I}}_{\text{int}}^{(2)}∕\stackrel{.}{I}$ increases significantly with the local irreversibility İ for five-cell groups (Spearman coefficient r = 0.31, p < 10⁻³). In all panels, out of 100 random groups, we include only those with significant local irreversibility for each stimulus.

In combination, the results of this section indicate that the arrow of time in retinal neurons (1) varies depending on the specific stimulus (Fig. 6), yet (2) does not simply reflect the irreversibility of the stimulus, and (3) consistently arises from the same combination of low-order dynamics, driven primarily by pairs of neurons (Fig. 7).

VIII. CONCLUSIONS

Irreversible dynamics support a wide range of biological functions, yet it remains unclear how macroscopic irreversibility arises from the microscopic dynamics of individual components. In this study, we propose a framework to uncover how irreversibility emerges in complex interacting systems. To do so, we develop analytic and numerical techniques for decomposing the information-theoretic evidence for the arrow of time into contributions from individual elements, pairs, and higher-order dynamics. We illustrate our methods on the examples of irreversible dynamics in models for sensing systems (Fig. 2) and logical functions (Figs. 3 and 4). Moving to real data, we find that the irreversibility of retinal neurons varies from one stimulus to another, but consistently arises from pairwise dynamics (Figs. 5-7).

These results suggest several future directions. For example, given that the irreversibility of retinal neurons does not simply reflect that of the stimulus, it is natural to wonder which stimulus properties are, in fact, responsible for inducing irreversibility in groups of cells. Additionally, in the process of decomposing the local irreversibility, one must compute a hierarchy of minimally irreversible models consistent with observed kth-order dynamics. Just as maximum entropy models have been successful in describing distributions over states at single moments in time [24,29-31], might these minimum irreversibility models provide insights into the dynamical flow of living systems from one state to another? More generally, we remark that the proposed framework is noninvasive, applying to any system with time-series data. Thus, the methods can be used to examine irreversible dynamics in a wide range of other biological systems, from molecular and cellular networks [3,13,15,32-34], to large-scale recordings in the brain [26,35], to entire populations of animals and humans [29,31,36].

The data were collected by Marre et al. in Ref. [6] and are additionally described in Ref. [7]. The data and code used to perform the analyses in this paper are openly available at [37].

We report in the Supplemental Material [38] an analysis of the gender makeup of the authors we cited.

APPENDIX A: MULTIPARTITE DYNAMICS REQUIRED TO DECOMPOSE IRREVERSIBILITY

In Sec. III we show that the local irreversibility İ of a multipartite system can be decomposed into two nonnegative components [Eq. (13)]: the independent irreversibility İ^ind and the interaction irreversibility İ^int. Here we show that multipartite dynamics are necessary for this decomposition. Specifically, we establish that if multiple elements are allowed to change state at the same time, then the decomposition in Eq. (13) can break down and the interaction irreversibility can become ill-defined.

Consider, for example, a system with two identical elements x and y, such that x = y at all moments in time. Given the joint transition probabilities of one of the variables P(x → x′), the dynamics of the combined system are given by

$$P(x,y\to x^{\prime },y^{\prime })=P(x\to x^{\prime }){\delta }_{x,y}{\delta }_{x^{\prime },y^{\prime }}.$$ (A1)

For such a system, the irreversibility is given by

$$\stackrel{.}{I}=\sum _{x,x^{\prime },y,y^{\prime }}P(x,y\to x^{\prime },y^{\prime })\log \frac{P(x,y\to x^{\prime },y^{\prime })}{P(x^{\prime },y^{\prime }\to x,y)}$$ (A2)

$$=\sum _{x,x^{\prime }}P(x\to x^{\prime })\log \frac{P(x\to x^{\prime })}{P(x^{\prime }\to x)}.$$ (A3)

To compute the independent irreversibility İ^ind, we note that the marginal dynamics of y are identical to that of x:

$$P(y\to y^{\prime })=\sum _{x,x^{\prime }}P(x,y\to x^{\prime },y^{\prime })$$ (A4)

$$=\sum _{x,x^{\prime }}P(x\to x^{\prime }){\delta }_{x,y},{\delta }_{x^{\prime },y^{\prime }}$$ (A5)

$$=P(x\to x^{\prime }).$$ (A6)

Thus, the independent irreversibility is given by

$${\stackrel{.}{I}}^{\text{ind}}={\stackrel{.}{I}}_x^{\text{ind}}+{\stackrel{.}{I}}_y^{\text{ind}}$$ (A7)

$$=\sum _{x,x^{\prime }}P(x\to x^{\prime })\log \frac{P(x\to x^{\prime })}{P(x^{\prime }\to x)}$$ (A8)

$$+\sum _{y,y^{\prime }}P(y\to y^{\prime })\log \frac{P(y\to y^{\prime })}{P(y^{\prime }\to y)}$$ (A9)

$$=2\sum _{x,x^{\prime }}P(x\to x^{\prime })\log \frac{P(x\to x^{\prime })}{P(x^{\prime }\to x)}$$ (A10)

$$=2\stackrel{.}{I}.$$ (A11)

Since İ^int = İ − İ^ind = − İ, we find that the interaction irreversibility is negative, thus violating the decomposition of the local irreversibility into nonnegative terms.

APPENDIX B: SOLVING THE SENSING SYSTEM

Consider a sensing system composed of an environmental variable x and a sensing variable y, each with three states. The environmental variable x increases with probability p_x, and the sensing variable y copies x with probability p_y, yielding the dynamics in Eqs. (14) and (15). Randomly choosing one variable to update at each point in time, the dynamics of the combined system are defined by the conditional transition probabilities

$$P(x^{\prime },y^{\prime }\mid x,y)=\frac{1}{2}(P(x^{\prime }\mid x){\delta }_{y,y^{\prime }}+P(y^{\prime }\mid x){\delta }_{x,x^{\prime }}).$$ (B1)

Using the stationary condition π(x, y) = ∑_x′,y′ P(x, y ∣ x∲, y′)π(x′, y′), one can solve for the stationary distribution:

$$\pi (x,y)\propto \left(\begin{array}{c}\hfill 2+7p_y+p_x(1+3p_y+3p_x)\hfill \\ \hfill 5-2p_y+p_x(4-6p_y+3p_x)\hfill \\ \hfill 6-5p_y+p_x(1+3p_y+3p_x)\hfill \\ \hfill 6-5p_y+p_x(1+3p_y+3p_x)\hfill \\ \hfill 2+7p_y+p_x(1+3p_y+3p_x)\hfill \\ \hfill 5-2p_y+p_x(4-6p_y+3p_x)\hfill \\ \hfill 5-2p_y+p_x(4-6p_y+3p_x)\hfill \\ \hfill 6-5p_y+p_x(1+3p_y+3p_x)\hfill \\ \hfill 2+7p_y+p_x(1+3p_y+3p_x)\hfill \end{array}\right)\begin{array}{c}\hfill (1,1)\hfill \\ \hfill (1,2)\hfill \\ \hfill (1,3)\hfill \\ \hfill (2,1)\hfill \\ \hfill (2,2)\hfill \\ \hfill (2,3)\hfill \\ \hfill (3,1)\hfill \\ \hfill (3,2)\hfill \\ \hfill (3,3)\hfill \end{array}.$$ (B2)

Combining Eqs. (B1) and (B2), we arrive at the joint transition probabilities P(x, y → x′, y′) = P(x′, y′ ∣ x, y)π(x, y), which are used to perform the calculations in Sec. IV.

APPENDIX C: CONVEXITY OF LOCAL IRREVERSIBILITY

In order to compute İ^(k), one must minimize the local irreversibility İ subject to constraints on the kth-order dynamics of the system. Here we show that the local irreversibility is convex with respect to the transition probabilities P(x → x′), and thus can be minimized using efficient techniques.

The gradient of the local irreversibility [Eq. (2)] is given by

$$\frac{\partial \stackrel{.}{I}}{\partial P(x\to x^{\prime })}=\log \frac{P(x\to x^{\prime })}{P(x^{\prime }\to x)}-\frac{P(x^{\prime }\to x)}{P(x\to x^{\prime })}+1,$$ (C1)

where for simplicity log(·) is natural logarithm. Since Eq. (C1) depends only on P(x → x′) and P(x′ → x), we see that the Hessian of İ takes the block diagonal form

$$H=\left(\begin{array}{ccc}\hfill \ddots \hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill H(x,x^{\prime })\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill \ddots \hfill \end{array}\right),$$ (C2)

where

$$H(x,x^{\prime })=\left(\begin{array}{cc}\hfill \frac{{\partial }^2\stackrel{.}{I}}{\partial P(x\to x^{\prime }{)}^2}\hfill & \hfill \frac{{\partial }^2\stackrel{.}{I}}{\partial P(x\to x^{\prime })\partial P(x^{\prime }\to x)}\hfill \\ \hfill \frac{{\partial }^2\stackrel{.}{I}}{\partial P(x\to x^{\prime })\partial P(x^{\prime }\to x)}\hfill & \hfill \frac{{\partial }^2\stackrel{.}{I}}{\partial P(x^{\prime }\to x{)}^2}\hfill \end{array}\right)$$ (C3)

$$=\left(\begin{array}{cc}\hfill \frac{P(x\to x^{\prime })+P(x^{\prime }\to x)}{P(x\to x^{\prime }{)}^2}\hfill & \hfill -\frac{P(x\to x^{\prime })+P(x^{\prime }\to x)}{P(x\to x^{\prime })P(x^{\prime }\to x)}\hfill \\ \hfill -\frac{P(x\to x^{\prime })+P(x^{\prime }\to x)}{P(x\to x^{\prime })P(x^{\prime }\to x)}\hfill & \hfill \frac{P(x\to x^{\prime })+P(x^{\prime }\to x)}{P(x^{\prime }\to x{)}^2}\hfill \end{array}\right)$$ (C4)

is the 2 × 2 Hessian for the pair of states (x, x′). The eigenvalues of H(x, x′) are ${\lambda }_1=\frac{(P(x\to x^{\prime })+P(x^{\prime }\to x))(P(x\to x^{\prime }{)}^2+P(x^{\prime }\to x{)}^2)}{P(x\to x^{\prime }{)}^{2}P(x^{\prime }\to x{)}^2}$ and λ₂ = 0. Since λ₁, λ₂ ⩾ 0, and since the eigenvalues of H are simply the eigenvalues of the different blocks H(x, x′) combined, we have established that H is positive semidefinite, and therefore that the local irreversibility İ is convex.

APPENDIX D: EQUIVALENCE BETWEEN INDEPENDENT AND FIRST-ORDER IRREVERSIBILITIES

Here we establish that the independent irreversibility İ^ind is equivalent to the first-order minimum irreversibility İ⁽¹⁾. To do so, consider a hypothetical system Q($Q(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})$, x_−i) that is consistent with the observed first-order dynamics $P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })={\sum }_{x_{-\mathrm{i}}}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})$. Since İ^ind(Q) = İ^ind(P), we have

$$\stackrel{.}{I}(Q)={\stackrel{.}{I}}^{\text{ind}}(Q)+{\stackrel{.}{I}}^{\text{int}}(Q)$$ (D1)

$$={\stackrel{.}{I}}^{\text{ind}}(P)+{\stackrel{.}{I}}^{\text{int}}(Q)$$ (D2)

$$\ge {\stackrel{.}{I}}^{\text{ind}}(P),$$ (D3)

where the inequality follows from that fact that İ^int(Q) ⩾ 0. Thus, the independent irreversibility İ^ind(P) is a lower bound on the local irreversibility İ (Q) of any hypothetical system Q consistent with the observed first-order dynamics. Since the first-order irreversibility İ⁽¹⁾ is just the minimum of İ(Q) among all such systems Q, we have found that İ ⁽¹⁾ ⩾ İ^ind.

In order to establish that İ ⁽¹⁾ = İ^ind, all that remains is to identify a hypothetical system Q that achieves the lower bound in Eqs. (D1)-(D3). Specifically, we seek a system Q that is consistent with the observed first-order dynamics, yet has interaction irreversibility İ^int(Q) = 0. Consider, for example, a system Q in which the dynamics of each element i are independent from the rest of the system, such that $Q(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})\propto Q(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })=P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })$ for all x_−i. Using Eqs. (10)-(12), one can verify that such a system has zero interaction irreversibility, thereby saturating the lower bound in Eqs. (D1)-(D3). We have therefore shown that İ ⁽¹⁾ (the minimum local irreversibility consistent with first-order dynamics) is equivalent to the independent irreversibility İ^ind.

APPENDIX E: NOISY LOGICAL FUNCTIONS

In Sec. VI we examine a system of three binary variables: two inputs x and y that flip with probability p_flip, and an output variable z that performs a logical function on x and y, but with error rate p_error [see Fig. 3(a)]. Specifically, the dynamics of the input variables are defined by the conditional transition probabilities

$$P(x^{\prime }\mid x)=P(y^{\prime }\mid y)=\left(\begin{array}{cc}\hfill 1-p_{\text{flip}}\hfill & \hfill p_{\text{flip}}\hfill \\ \hfill p_{\text{flip}}\hfill & \hfill 1-p_{\text{flip}}\hfill \end{array}\right),$$ (E1)

and the dynamics of the output variable are defined by

$$P(z^{\prime }\mid x,y)=\{\begin{array}{cc}1-p_{\text{error}},\hfill & z^{\prime }=f(x,y)\hfill \\ p_{\text{error}},\hfill & z^{\prime }\ne f(x,y)\hfill \end{array}\phantom{\}},$$ (E2)

where f (x, y) is the logical function performed by z. Randomly picking one variable to update at each point in time, the conditional transition probabilities for the entire system are given by

$$\begin{gathered} P(x^{\prime },y^{\prime },z^{\prime }\mid x,y,z)=\frac{1}{3}(P(x^{\prime }\mid x){\delta }_{y,y^{\prime }}{\delta }_{z,z^{\prime }} \\ +P(y^{\prime }\mid y){\delta }_{x,x^{\prime }}{\delta }_{z,z^{\prime }}+P(z^{\prime }\mid x,y){\delta }_{x,x^{\prime }}{\delta }_{y,y^{\prime }}). \end{gathered}$$ (E3)

Using Eq. (E3), one can solve for the stationary distribution π(x, y, z) and then compute the joint transition probabilities P(x, y, z → x′, y′, z′) = P(x′, y′, z′ ∣ x, y, z)π(x, y, z).

APPENDIX F: INDEPENDENT IRREVERSIBILITY VANISHES FOR BINARY STEADY-STATE SYSTEMS

For binary steady-state systems, such as the logical functions in Sec. VI and the neurons in Sec. VII, the independent irreversibility İ^ind is zero. To see this, note that the marginal dynamics of any binary steady-state variable are defined by the conditional transition probabilities

$$P(x_{\mathrm{i}}^{\prime }\mid x_{\mathrm{i}})=\left(\begin{array}{cc}\hfill 1-p_{\mathrm{i}}\hfill & \hfill p_{\mathrm{i}}\hfill \\ \hfill q_{\mathrm{i}}\hfill & \hfill 1-q_{\mathrm{i}}\hfill \end{array}\right),$$ (F1)

where 0 ⩽ p_i, q_i ⩽ 1 are the probabilities of i switching between its two states. The marginal steady-state distribution for i is $\pi (x_{\mathrm{i}})=\frac{1}{p_{\mathrm{i}}+q_{\mathrm{i}}}(q_{\mathrm{i}},p_{\mathrm{i}}{)}^T$, and thus the marginal joint transition probabilities are given by

$$P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })=P(x_{\mathrm{i}}^{\prime }\mid x_{\mathrm{i}})\pi (x_{\mathrm{i}})$$ (F2)

$$=\frac{1}{p_{\mathrm{i}}+q_{\mathrm{i}}}\left(\begin{array}{cc}\hfill (1-p_{\mathrm{i}})q_{\mathrm{i}}\hfill & \hfill p_{\mathrm{i}}{q}_{\mathrm{i}}\hfill \\ \hfill q_{\mathrm{i}}{p}_{\mathrm{i}}\hfill & \hfill (1-q_{\mathrm{i}})p_{\mathrm{i}}\hfill \end{array}\right).$$ (F3)

Since the above transition probabilities are symmetric, the marginal dynamics of each element i obey detailed balance (such that ${\stackrel{.}{I}}_{\mathrm{i}}^{\text{ind}}=0$). Thus, we find that the independent irreversibility of the entire system ${\stackrel{.}{I}}^{\text{ind}}={\sum }_{\mathrm{i}=1}^N{\stackrel{.}{I}}_{\mathrm{i}}^{\text{ind}}$ is zero. We emphasize that this holds only for the local irreversibility; if we consider non-Markov effects in strings of 3, 4, or more points in time, then binary steady-state variables can break time-reversal symmetry and define an arrow of time [39].

APPENDIX G: NEURONAL RECORDINGS

The neuronal data examined in Sec. VII were recorded from larval tiger salamander retina, which were dissected, perfused with Ringer’s solution, and pressed onto dense arrays of 252 electrodes with 30-μm spacing, as described in Ref. [7]. In the experiments, which lasted 4–6 hours, movies were projected onto the photoreceptor layer of the retina via an objective lens, and voltages were recorded at 10 kHz. Spikes were sorted conservatively (as described in Ref. [6]), yielding 53 reliable cells from which groups were randomly selected for analysis.

The stimuli were presented on a 360 × 600 display, with pixels of size 3.81 μm on the retina and a frame rate of 60 frames per second. All stimuli were normalized to the same average light intensity. The natural movie depicted a fish swimming in a tank, repeated 102 times. The moving bar was 11 pixels wide and black on a gray background, with trajectories displayed 62 times. The trajectory of the bar’s vertical position was generated by a stochastic process equivalent to a Brownian particle on a spring attached to the center of the display. Specifically, the vertical position x_t and velocity v_t of the bar were updated at each time t according to the equations of motion:

$$x_{t+\tau }=x_t+{\upsilon }_t\tau$$ (G1)

and

$${\upsilon }_{t+\tau }=(1-\Gamma \tau ){\upsilon }_t-{\omega }^2{x}_t\tau +{\xi }_t\sqrt{D\tau },$$ (G2)

where τ = 1/60 s is the time step (which matches the frame rate of the visual display), ω = 3π s⁻¹ is the natural frequency, Γ = 20 s⁻¹ parameterizes the damping (chosen such that the dynamics are slightly overdamped), and D = 2.7 × 10⁶ pixel²/s³ is chosen to allow reasonable range of motion. For the repeated bar stimulus, the same trajectory was repeated 62 times.

APPENDIX H: CORRECTING FOR FINITE DATA

For time-series data, such as the neuronal spiking examined in Sec. VII, in order to estimate quantities of interest—such as transition rates, flux rates, and changes in state probabilities (Fig. 5); local irreversibilities İ (Fig. 6); and interaction irreversibilities ${\stackrel{.}{I}}_{\text{int}}^{(k)}$ (Fig. 7)—one must correct for finite-data effects [7,23,24]. To do so, for a given stimulus and group of neurons, we begin with a list of the observed transitions {x(t) → x(t + 1)}. For a given set of data fractions f, we subsample the transitions (without replacement) in a hierarchical fashion, such that each subsample of transitions is a subset of the larger subsamples. For the neuronal data in Sec. VII, we find that data fractions f = {1, 0.9, 0.8, 0.7, 0.6, 0.5} are sufficient.

For each data fraction f, we estimate the quantity of interest. For example, for the local irreversibility, we use the estimate

$$\stackrel{.}{I}=\sum _{x,x^{\prime }}\tilde{P}(x\to x^{\prime })\log \frac{\tilde{P}(x\to x^{\prime })}{\tilde{P}(x^{\prime }\to x)},$$ (H1)

where

$$\tilde{P}(x\to x^{\prime })=\frac{N(x\to x^{\prime })+1}{{\sum }_{y,y^{\prime }}(N(y\to y^{\prime })+1)}$$ (H2)

are the maximum likelihood probabilities with one pseudocount for each transition, and N(x → x′) is the number of times that the transition x → x′ was observed in the data. We include pseudocounts to avoid infinities in Eq. (H1), but we confirm that the naïve estimator without pseudocounts yields the same results. After estimating the quantity of interest for all fractions f, we then extrapolate to the infinite-data limit using a linear fit with respect to the inverse data fraction 1/f [Fig. 8(a)]. Repeating this process 100 times, we arrive at both an average and standard deviation for the infinite-data estimates of the desired quantity [Fig. 8(b)].

FIG. 8.

Correcting for finite-data effects on local irreversibility. (a) Estimated local irreversibility İ (black markers) vs inverse data fraction for one group of N = 5 neurons responding to a natural movie. Gray line indicates a linear fit, and red marker indicates the extrapolation to infinite data. (b) Estimated local irreversibility (black markers), linear fits (gray lines), and extrapolation to infinite data (red marker) after repeating the process in panel (a) 100 times for the same group of neurons. Data points and error bars reflect averages and standard deviations over the 100 repetitions.

To check that the above procedure gives accurate estimates for the local irreversibility İ, we note that randomizing the timing of spikes should destroy the arrow of time. Thus, for time-randomized data, the estimated local irreversibility should vanish in the infinite-data limit. Consider the 100 groups of N = 5 neurons analyzed in Figs. 6 and 7. Among these groups, after correcting for finite-data effects, we find that 60% – 68% exhibit significant local irreversibility İ, depending on the stimulus [Fig. 9(a)]. By contrast, after randomizing the spike times, the local irreversibility estimates are centered around zero, with only 0% – 2% of groups exhibiting significant local irreversibility [Fig. 9(b)]. Examining different group sizes, we find that the percentage of groups with significant local irreversibility increases from ~10% for N = 2 cells to ~100% for N ⩾ 8 cells [Fig. 9(c)]. Importantly, after randomizing spike times, groups of N ⩽ 5 cells are almost always locally reversible, as desired [Fig. 9(d)]. However, even for time-randomized data, we find that some groups of N ⩾ 6 cells exhibit significant local irreversibility [Fig. 9(d)], demonstrating finite-data effects cannot be adequately accounted for. We therefore conclude that N = 5 is the largest number of cells for which we can consistently estimate local irreversibility İ in our dataset.

FIG. 9.

Estimated local irreversibilities of real and time-shuffled data. (a, b) Distributions of local irreversibility estimates İ, normalized by standard deviations σ_İ, for real time series (a) and after randomizing spike times (b). Distributions are over the same 100 groups of N = 5 cells analyzed in Figs. 6 and 7, and the dashed line indicates the threshold for significance. (c, d) Fraction of cell groups with significant local irreversibility as a function of group size N for real time series (c) and after randomizing spike times (d). For each size, fractions are computed over 100 random groups.

APPENDIX I: MINIMIZING LOCAL IRREVERSIBILITY

Computing the kth-order minimum irreversibility İ^(k) requires finding a hypothetical system $Q(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})$ that matches the observed kth-order marginal dynamics, but otherwise has minimum local irreversibility İ(Q). We remark that the local irreversibility İ is convex (see Appendix C), and thus computing İ^(k) is a constrained convex minimization problem for which there exist efficient optimization methods. From a practical perspective, there are only two main hurdles to overcome: (1) adapting an existing convex minimization technique for our problem and (2) specifying the constraints on the kth-order dynamics. Here we address these challenges for binary systems.

1. Frank-Wolfe algorithm

To minimize the local irreversibility İ given a set of constraints, we employ the Frank-Wolfe algorithm, which efficiently converges to a local (and therefore global) minimum. Specifically, we initialize Q using any dynamics that match the observed constraints (for example, one can begin with the observed dynamics Q = P). We then iterate the following steps:

(1) First, we compute the gradient of the local irreversibility:

$$\begin{gathered} \frac{\partial \stackrel{.}{I}}{\partial Q(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})}=\log \frac{Q(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})}{Q(x_{\mathrm{i}}^{\prime }\to x_{\mathrm{i}},x_{-\mathrm{i}})} \\ -\frac{Q(x_{\mathrm{i}}^{\prime }\to x_{\mathrm{i}},x_{-\mathrm{i}})}{Q(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})}+1, \end{gathered}$$ (I1)

where log(·) represents the natural logarithm for simplicity.

(2) Second, we solve for the dynamics Q* that obey the desired constraints while minimizing the inner product with the gradient:

$$\sum _{\mathrm{i}=1}^N\sum _{x_{-\mathrm{i}}}\sum _{x_{\mathrm{i}},x_{\mathrm{i}}^{\prime }}Q(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})\frac{\partial \stackrel{.}{I}}{\partial Q(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})}.$$ (I2)

We note that this constrained linear minimization problem is a linear program, and thus can be efficiently solved using standard techniques (e.g., the linprog function in MATLAB).

(3) Finally, we take a step toward Q*, such that Q ← Q + α_t(Q* − Q), where α_t = α₀/t is the step size, which decreases with the number of iterations t.

An implementation of the above algorithm is available at [37].

2. Constraining kth-order dynamics

We seek to constrain the kth-order dynamics of a binary, multipartite system with joint transition probabilities $P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})$. For each element i, consider a group of k − 1 of the remaining elements K ⊆ {1,…, i − 1, i + 1,…, N}. Let x_K denote the states of the elements in K, and x_{−{i, K}} the states of the elements not in i nor K. The marginal dynamics of i with the elements in K held fixed are then given by

$$P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_K)=\sum _{x_{-\{\mathrm{i},K\}}}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}}).$$ (I3)

Constraining the kth-order dynamics amounts to constraining the marginal probabilities $P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_K)$ for all elements i and all groups of the remaining elements K of size k − 1. For example, if K is empty, then we arrive at the independent (first-order) dynamics $P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime })$. If K consists of one element j, then we have the pairwise (second-order) dynamics $P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{\mathrm{j}})$ discussed in Sec. V. We remark, however, that these marginal probabilities are not all independent, and therefore the set of constraints is overdetermined.

To write down independent constraints that fully define the kth-order dynamics, it helps to consider an analogy with Ising systems. Consider a binary system with state probabilities P(x). It is known that the kth-order marginal probabilities P(x_K) = ∑_x−K P(x) are completely defined by the correlations between groups of elements up to size k: ⟨x_i⟩, ⟨x_ix_j⟩, …, ⟨Π_i∈K x_i⟩, where ⟨·⟩ represents an average over P(x) [5]. Moreover, these correlations are independent, thus forming a basis for the kth-order probabilities P(x_K).

Here, we wish to constrain the kth-order transition probabilities $P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_K)$ for all elements i and all groups of the remaining elements K of size k − 1. For a given transition $x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime }$, we denote the correlation between a set of the remaining elements K by

$${\langle \prod _{\mathrm{j}\in K}{x}_{\mathrm{j}}\rangle }_{x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime }}=\sum _{x_{-\mathrm{i}}}\prod _{\mathrm{j}\in K}{x}_{\mathrm{j}}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}}).$$ (I4)

If K is empty, then we simply arrive at the independent transition probabilities:

$$\langle 1{\rangle }_{x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime }}=\sum _{x_{-\mathrm{i}}}P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_{-\mathrm{i}})=P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime }).$$ (I5)

By analogy with Ising systems, for each transition $x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime }$, the kth-order marginal probabilities $P(x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime },x_K)$ can be defined by the correlations between groups of elements (not including i) from the empty set up to size $k-1:\langle 1{\rangle }_{x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime }},\langle x_{\mathrm{j}}{\rangle }_{x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime }},\dots ,\langle {\prod }_{\mathrm{j}\in K}{x}_{\mathrm{j}}{\rangle }_{x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime }}$. We can then constrain the kth-order dynamics of the entire system by computing the above correlations for each of the 2N transitions $x_{\mathrm{i}}\to x_{\mathrm{i}}^{\prime }$ (not including self-transitions). We remark that we do not need to constrain self-transitions x → x because they do not contribute to the local irreversibility [Eq. (2)]. Code for constraining the kth-order dynamics of binary, multipartite systems is available at [37].

APPENDIX J: GROUPS OF NEURONS OPERATE AT STEADY STATE

In Fig. 5 we see that a group of N = 3 neurons operates at a nonequilibrium steady state. Here we demonstrate that steady-state dynamics are not specific just to this group, but are instead a general feature of all groups of neurons analyzed in this paper. To determine if a system operates at steady state, one must examine whether its state probabilities are stationary in time. The change in the probability P(x) of a state x during one time step is given by ΔP(x) = ∑_x′ P(x′ → x) − P(x → x′). In steady state this should be zero, but more precisely we expect that it will be a random number with a variance set by the errors in sampling the underlying distributions. In Fig. 10 we plot the distributions of ΔP(x), normalized by the relevant standard deviation σ_ΔP(x), for the groups of N = 5 cells analyzed in Figs. 6 and 7. We note that these quantities are estimated using the same finite-data correction techniques described in Appendix H. Across all stimuli, we find that the changes in state probabilities ΔP(x) for all cell groups are small relative to errors; that is, for all stimuli, all groups of neurons appear to operate at steady state.

FIG. 10.

Neurons operate at stochastic steady states. (a–c) Distributions of changes in state probabilities ΔP(x) = ∑_x′ P(x′ → x) − P(x → x′), normalized by the standard deviation σ_ΔP(x), for groups of N = 5 cells responding to a natural movie (a), moving bar stimulus (b), and a repeated moving bar stimulus (c). Distributions are over the 2^N = 32 different states for the 100 random groups analyzed in Figs. 6 and 7.