A local–global principle for nonequilibrium steady states

Significance

Fundamentals of statistical physics explain that systems in thermal equilibrium exhibit spontaneous order because orderly configurations have low energy. This fact is remarkable, and powerful, because energy is a “local” property of configurations. Nonequilibrium systems, including engineered and living systems, can also exhibit order, but there is no property analogous to energy that generally explains why orderly configurations of these systems often emerge. However, recent experiments suggest that a local property called “rattling” predicts which configurations are favored, at least for a broad class of nonequilibrium systems. We develop a theory of rattling that explains for which systems it works and why, and we demonstrate its application across scientific domains.

Abstract

The global steady state of a system in thermal equilibrium exponentially favors configurations with lesser energy. This principle is a powerful explanation of self-organization because energy is a local property of configurations. For nonequilibrium systems, there is no such property for which an analogous principle holds, hence no common explanation of the diverse forms of self-organization they exhibit. However, a flurry of recent empirical results has shown that a local property of configurations called “rattling” predicts the steady states of some nonequilibrium systems, leading to claims of a far-reaching principle of nonequilibrium self-organization. But for which nonequilibrium systems is rattling accurate, and why? We develop a theory of rattling in terms of Markov processes that gives simple and precise answers to these key questions. Our results show that rattling predicts a broader class of nonequilibrium steady states than has been claimed and for different reasons than have been suggested. Its predictions hold to an extent determined by the relative variance of, and correlation between, the local and global “parts” of a steady state. We show how these quantities characterize the local-global relationships of various random walks on random graphs, spin-glass dynamics, and models of animal collective behavior. Surprisingly, we find that the core idea of rattling is so general as to apply to equilibrium and nonequilibrium systems alike.

Self-organization abounds in nature, from the spontaneous assembly of fractal protein complexes (1), to nests of army ants, or bivouacs, formed by entangling potentially millions of their own, living bodies (2), to “marching bands” of desert locusts that can span hundreds of square kilometers (3). These varied phenomena warrant a common explanation because each entails a system that occupies relatively few configurations^* among an overwhelming number of alternatives. For physical systems in thermal equilibrium, the Boltzmann distribution provides such an explanation: Systems preferentially occupy configurations of lower energy. Specifically, the probability of a configuration $x$ of a physical system in thermal equilibrium satisfies

$$\begin{array}{c}\hfill \text{Prob}(x)\propto e^{-\beta E(x)}\end{array}$$ [1]

in terms of a constant $\beta$ and the configuration’s energy $E(x)$ (4).

The Boltzmann distribution is remarkable because the energy of a configuration is “local” in the sense that it does not depend on the dynamics that connect other configurations. This fact makes it possible to understand the equilibrium configurations of many-body systems, like proteins, far more efficiently than is possible with methods that apply to nonequilibrium systems (5). In contrast, there can be no local property of configurations that generally determines their weight in the global, steady-state distribution of a nonequilibrium system (6–8). Recent empirical evidence, however, suggests that a local property called “rattling” predicts the configurational weights of a broad class of nonequilibrium steady states (9–15).

Rattling.

Chvykov et al. offer a compelling heuristic: If the motion of a system is “so complex, nonlinear, and high-dimensional that no global symmetry or constraint can be found for its simplification,” then it amounts to diffusion in an abstract state space (10). A function of the motion’s effective diffusivity in the vicinity of a configuration $x$, called the rattling, then predicts the steady-state weight of $x$.

As a simple example, consider a one-dimensional diffusion with variable diffusivity $D(x)$, which has a steady-state distribution $\text{Prob}(x)$ that is proportional to $1/D(x)$. In this case, Chvykov et al. define the rattling $R(x)$ to be $logD(x)$ and the heuristic is literally true: The steady-state distribution has the form of the Boltzmann distribution with $\beta =1$, written in terms of $R(x)$ as

$$\begin{array}{c}\hfill \text{Prob}(x)\propto e^{-\beta R(x)}.\end{array}$$ [2]

For more general dynamics, Chvykov et al. replace $D(x)$ with an estimate of the rate at which the mean squared displacement from $x$ grows in each dimension and replace the rattling $R(x)$ with $(logdetD(x))/2$, which is a higher-dimensional analogue of $logD(x)$. Moreover, instead of the strict proportionality in Eq. 2, which implies that the logarithms of $\text{Prob}(x)$ and $1/R(x)$ are perfectly linearly correlated with a slope of $\beta =1$, the heuristic vaguely predicts that their correlation $\mathrm{\rho }$ is high and that the slope $\beta$, which may vary by system, is “of order $1$.”

Qualitatively, the rattling heuristic says that systems spend more time in configurations that they exit more slowly. This claim may seem trivial at first, but it is not—the steady-state distribution is global, while rattling is local—and it cannot be true in general. A theory of rattling should explain precisely for which systems Eq. 2 approximately holds, in terms of formulas for $\mathrm{\rho }$ and $\beta$.

Although the initial body of work on rattling does not identify the class of nonequilibrium systems to which the heuristic applies, it demonstrates that this class is broad. Examples include experiments with swarms of robots (10) and collectives of active microparticles (13); simulations of spin glasses (11) and stochastic dynamics with strong timescale separation (9); and numerical studies of the Lorenz equations (12) and mechanical networks (15). Strikingly, in many cases, the slope $\beta$ in Eq. 2 is nearly equal to $1$ (10).

Summary of Our Results.

The remarkable accuracy and scope of the rattling heuristic suggest that there is a simple, underlying mathematical theory of rattling. Our main results provide such a theory in terms of Markov chains, which serve as a general model of nonequilibrium steady states (16–21). Specifically, we derive formulas for Markov chain analogues of the correlation $\mathrm{\rho }$ and slope $\beta$ in terms of two further quantities, denoted $\widehat{\text{ρ}}$ and $r$, that respectively characterize the correlation between the stationary distribution’s local and global “parts” and the relative number of exponential scales over which these parts vary. The quantities $\widehat{\mathrm{\rho }}$ and $r$ together determine the accuracy of rattling’s predictions for any Markov chain.^†

Even when the rattling heuristic holds, in the sense that $\mathrm{\rho }$ and $\beta$ are close to $1$, the steady-state probabilities of some states can vastly differ from those predicted by Eq. 2. This discrepancy limits the value of the heuristic to some applications, like the estimation of observables. With this in mind, we also consider a stronger claim than the rattling heuristic makes: that the ratios of the steady-state probabilities and those in Eq. 2 are uniformly close to $1$. Our results identify conditions on the rates $Q(x,y)$ of a Markov chain that guarantee that its stationary probabilities $\mathrm{\pi }(x)$ are within a small constant factor of those specified by an analogue of Eq. 2:

$$\begin{array}{c}\hfill \mathrm{\pi }(x)\propto q{(x)}^{-\beta }.\end{array}$$ [3]

Here, $q(x)=\sum _{y\ne x}Q(x,y)$ denotes the exit rate of state $x$, the logarithm of which is the Markov chain analogue of rattling (see SI Appendix, Text for details). When Eq. 3 holds, $\mathrm{\pi }$ is akin to a Boltzmann distribution because, like the energy of a configuration, the exit rate of a state is “local” (Fig. 1). We show that for Eq. 3 to hold up to a small constant factor, it suffices for the global part of the stationary distribution to vary little.

Fig. 1.

Exit rates are local. A state $x$ of a Markov chain abstracts a configuration of a physical system, like the robot swarm of Chvykov et al. (10). The relative probability of $x$ in the stationary distribution is purely a function of the rates of the chain. The sum of the rates leaving $x$ is experimentally accessible, because this sum is the reciprocal of the average amount of time that it takes the system to leave $x$. This quantity is “local” to $x$ because it does not depend on the rates leaving any $y\ne x$.

Applications.

We briefly highlight three important applications of our results: to explain self-organization in broad classes of nonequilibrium systems, to estimate the observables of some nonequilibrium steady states, and to analyze the local-global relationship in systems across domains.

First, if a nonequilibrium system has special states of low rattling, then rattling explains self-organization to the extent that the correlation $\mathrm{\rho }$ is close to $1$. Our results show that there are essentially two distinct classes of systems, defined in terms of the quantities $\widehat{\mathrm{\rho }}$ and $r$, for which $\mathrm{\rho }\approx 1$. We later discuss how these quantities might be estimated from data.

Second, our results explain that, if the global part of a nonequilibrium steady state is hypothesized or inferred to vary little, then the configurational weights $q{(x)}^{-\beta }$ can be used to estimate observables, in the same way that Boltzmann weights $e^{-\beta E(x)}$ are used to estimate observables of equilibrium steady states. For example, these weights could be used to efficiently generate one-shot samples from the nonequilibrium steady states of some many-body systems (5).

Third, our results are even useful for analyzing systems with known steady-state distributions, because they relate the local and global parts of arbitrary steady states. This is especially interesting when the rates of a Markov chain are functions of a parameter. (For example, this is the context of fundamental results in stochastic thermodynamics, like the Jarzynski equality (22) and the Crooks fluctuation theorem (23, 24), where the parameter models the influence of external forces.) Because we have not precisely defined $\widehat{\mathrm{\rho }}$ and $r$ yet, we only explain the high-level idea. Every Markov chain defines a pair $(r,\widehat{\mathrm{\rho }})$ in the strip $[0,\infty )\times [-1,1]$, and changing a parameter of the chain amounts to placing a dynamics on these pairs. Our results allow us to interpret certain regions of the strip as classes of steady states that have local weights. Later, we demonstrate how this idea can be used to analyze bifurcations in models of animal collective behavior.

Broader Perspective and Related Work.

Markov chains that satisfy detailed balance^‡ are analogous to equilibrium systems because they are statistically time-reversible and have stationary distributions that can be expressed in the Boltzmann form (19). Accordingly, studies that model nonequilibrium steady states as Markov chains typically emphasize how and to what extent these chains violate detailed balance. For example, the conservation of probability requires that violations of detailed balance arise from net fluxes of probability around cycles of states. These circular fluxes form the basis of Hill’s thermodynamic formalism (25, 26), Schnakenberg’s network theory (16, 27), and cycle representations of Markov chains (28–31), and they appear in the Helmholtz–Hodge decomposition of Markov chains (32). Moreover, probability fluxes—and functions of them, like thermodynamic forces and entropy production—are the subject of nonequilibrium fluctuation theorems (33–36), thermodynamic uncertainty relations (37–39), and results concerning the response of nonequilibrium steady states to rate perturbations (40–43).

It may come as a surprise, then, that our results make no reference to detailed balance or the nature of its possible failure. Instead, we view the search for nonequilibrium analogues of the Boltzmann distribution as part of a broader effort to understand when local information about the rates of a Markov chain suffices to determine or predict its stationary distribution (Fig. 1). From this perspective, chains that satisfy detailed balance are not an ideal from which other chains depart; while their stationary distributions can be expressed in the Boltzmann form, the associated weights are nonlocal functions of the rates (19, 31) (Fig. 2).

Fig. 2.

The relationship between the rates of an irreducible Markov chain $Q$ and its stationary distribution $\mathrm{\pi }$ specializes for certain classes of rate matrices. In general, all of the rates are necessary to determine $\mathrm{\pi }(x)$; the Markov chain tree theorem specifies how to do so, by summing products of the rates along the spanning trees of the underlying adjacency graph (Eq. 4). However, if the chain satisfies detailed balance, then it suffices to know the ratio of the product of forward rates to the product of reverse rates along a path ${\mathcal{P}}_x$ to $x$ from a fixed, reference state. If the chain has a doubly stochastic jump chain, then it suffices to know the exit rate $q(x)$ of each state $x$.

In general, if $Q$ is an irreducible rate matrix, then its stationary distribution $\mathrm{\pi }$ is the unique solution to $\mathrm{\pi }Q=\mathbf{0}$, which implicates all of the rates $Q(u,v)$. In fact, according to the Markov chain tree theorem (44), $\mathrm{\pi }(x)$ is proportional to a sum over spanning trees of products of rates:

$$\begin{array}{c}\hfill \mathrm{\pi }(x)\propto {\displaystyle \sum _{\mathcal{T}}}{\displaystyle \prod _{(u,v)\in {\mathcal{T}}_x}}Q(u,v).\end{array}$$ [4]

Here, the sum ranges over spanning trees $\mathcal{T}$ of the chain’s transition graph and ${\mathcal{T}}_x$ is the set of directed edges that results from “pointing” all of the edges of $\mathcal{T}$ toward $x$ (Fig. 2). Our results identify special classes of chains for which relatively little and, importantly, exclusively local information about their rates suffices to determine their stationary distributions. For example, for chains that satisfy Eq. 3, it suffices to know the exit rates $q(x)$ to determine $\mathrm{\pi }$.

Results

We model the steady state of a physical system as a stationary, continuous-time Markov chain with a finite number of states $N$ (16, 18, 19).^§ For simplicity, we refer to the chain by the matrix $Q$ of its transition rates. We assume that the chain is irreducible, meaning that it can eventually reach any state from any other state; this guarantees that $Q$ has a unique stationary distribution $\mathrm{\pi }$.

Our results emphasize the relationship between $\mathrm{\pi }$, the exit rates

$$q(x):={\displaystyle \sum _{y\ne x}}Q(x,y),$$

and the stationary distribution of a chain $\widehat{Q}$ that is closely related to $Q$, called the jump chain of $Q$. The jump chain is part of a standard construction of continuous-time Markov chains (45, Section 2.6). We define it to be the discrete-time Markov chain with transition probabilities

$$\begin{array}{c}\hfill \widehat{Q}(x,y):=\left\{\begin{array}{cc}Q(x,y)/q(x)\hfill & x\ne y,\hfill \\ 0\hfill & x=y.\hfill \end{array}\right.\end{array}$$

The jump chain is irreducible because $Q$ is, hence it has a unique stationary distribution $\widehat{\mathrm{\pi }}$.

We view the logarithms of $1/q$ and $\widehat{\mathrm{\pi }}$ as the local and global “parts” of $\mathrm{\pi }$. In this view, Markov chain analogues of the Boltzmann distribution (Eq. 1) and the rattling heuristic (Eq. 2) would approximate $\mathrm{\pi }$ by a distribution with weights that are a function of $q$, with an error that depends on the relationship between $q$ and $\widehat{\mathrm{\pi }}$. Indeed, a simple heuristic suggests that the analogue of rattling $R$ is $logq$ (SI Appendix, Text). Eq. 2 therefore suggests that we approximate $\mathrm{\pi }(x)$ by

$${\mathrm{\nu }}_{\beta }(x):=\frac{q{(x)}^{-\beta }}{{\sum }_{y}q{(y)}^{-\beta }}$$

for a real number $\beta$. Informally, our results identify two distinct ways for ${\mathrm{\nu }}_{\beta }$ to approximate $\mathrm{\pi }$ for some $\beta$, i.e., for $\mathrm{\pi }$ to have local weights: Either 1) the global part of $\mathrm{\pi }$ is approximately uniform or 2) the local and global parts of $\mathrm{\pi }$ are approximately collinear.

The first way for $\mathrm{\pi }$ to have local weights is for $\widehat{\mathrm{\pi }}$ to be close to uniform. To state this more precisely, we will say that two numbers $a$ and $b$ are within a factor of $k\ge 1$ if

$$k^{-1}\le \frac{a}{b}\le k.$$

Additionally, we will say that two vectors or matrices $A$ and $B$ of the same size are within a factor of $k\ge 1$ if all of their corresponding entries $A(x,y)$ and $B(x,y)$ are within a factor of $k$.

Theorem 1

If $\widehat{\mathrm{\pi }}$ is within a factor of $k\ge 1$ of the uniform distribution, then $\mathrm{\pi }$ and ${\mathrm{\nu }}_1$ are within a factor of $k^2$. In particular, if $\widehat{\mathrm{\pi }}$ is uniform, then $\mathrm{\pi }={\mathrm{\nu }}_1$.

In fact, $\widehat{\mathrm{\pi }}$ is uniform if and only if the jump chain $\widehat{Q}$ is doubly stochastic, i.e., each of its rows and columns sums to $1$. Since small perturbations of $\widehat{Q}$ produce correspondingly small perturbations of $\widehat{\mathrm{\pi }}$ (46, 47), the conclusion of Theorem 1 applies whenever the original chain $Q$ has a jump chain that is nearly doubly stochastic.

Theorem 2

If there is a doubly stochastic matrix $D$ that is within a factor of $k\ge 1$ of the jump chain’s transition probability matrix $\widehat{Q}$, then $\mathrm{\pi }$ and ${\mathrm{\nu }}_1$ are within a factor of $k^{2N}$. In particular, if $\widehat{Q}$ is doubly stochastic, then $\mathrm{\pi }={\mathrm{\nu }}_1$.

Theorem 2 makes it easy to generate examples of nonequilibrium steady states with local weights (Fig. 3). Simply multiply the rows of a doubly stochastic matrix by any exit rates $q(x)>0$, and replace the diagonal entries with $-q(x)$. This yields a Markov chain $Q$ with local weights $-logq(x)$ and the stationary distribution ${\mathrm{\nu }}_1$.

Fig. 3.

A variety of Markov chains that satisfy $\mathrm{\pi }(x)\propto 1/q(x)$. (Transition graphs with red edges depict doubly stochastic matrices; edge widths are proportional to transition probabilities.) Doubly stochastic matrices are varied, ranging from sparsely connected, directed cycles ($P_1$) to densely connected, undirected graphs ($P_2$). Moreover, they include arbitrary convex combinations, like $P$, which is a convex combination of $P_1$ and $P_2$. According to Theorems 1 and 2, scaling the rows $P(x,·)$ by any positive numbers $q(x)$, and subsequently choosing the diagonal entries to make the row sums equal zero, produces the transition rate matrix of a continuous-time Markov chain $Q(x,y)=q(x)P(x,y)$ that has a stationary distribution $\mathrm{\pi }(x)\propto 1/q(x)$. (The blue edges have widths proportional to the corresponding rates of $Q$.) This requires $P$ to have entries of zero on its diagonal (no “self-loops”) and an adjacency graph that is strongly connected. The variety of doubly stochastic matrices, and the freedom to scale their rows by any exit rates, means that an even greater variety of chains satisfy $\mathrm{\pi }(x)\propto 1/q(x)$.

Even when $\widehat{\mathrm{\pi }}$ is far from uniform, the exit rates can “predict” the stationary probabilities, in the sense that the logarithms of $1/q(X)$ and $\mathrm{\pi }(X)$ are highly collinear, for a random state $X$. This is true, for example, of many Markov chains with random rates (Fig. 4). Our next results explain why the correlations are high and the slopes are close to $1$.

Fig. 4.

Extent to which exit rates predict the stationary distributions of Markov chains with random transition rates. Each panel includes a scatter plot of $log\mathrm{\pi }(x)$ versus $-logq(x)$, paired with a corresponding plot of $log\widehat{\mathrm{\pi }}(x)$ versus $-logq(x)$, for one of four Markov chains $Q$. Each Markov chain has $2^{10}$ states that correspond to the vertices of either the complete graph (circular marks in A and B) or the $10$-dimensional hypercube (square marks in C and D). In (A and C), independent and identically distributed (i.i.d.) exponential rates $Q(x,y)$ and $Q(y,x)$ with mean $1$ connect every pair of states $x,y$ that are adjacent in the underlying graph (green). In (B and D), the rates instead have i.i.d. log-normal distributions with parameters $\mu =0$ and $\sigma =5$ (blue). (A) Observe that, for exponential rates on the complete graph, the logarithms of $\mathrm{\pi }(x)$ and $1/q(x)$ are highly collinear, while those of $\widehat{\mathrm{\pi }}(x)$ and $1/q(x)$ are not. (B) The same is true of log-normal rates on the complete graph, but the collinearity of $log\mathrm{\pi }(x)$ and $-logq(x)$ is greater than in (A) because $\widehat{\mathrm{\pi }}(x)$ varies over relatively fewer exponential scales than $q(x)$ in the case of log-normal rates. (In the language of Theorem 3, $r$ is smaller in B than in A.) (C and D) On the hypercube, $log\mathrm{\pi }(x)$ and $-logq(x)$ remain highly collinear, but to a lesser extent than in (A and B), because the variance of $\widehat{\mathrm{\pi }}(x)$ is greater relatively in (C and D). The slopes of the lines of best fit are nearly equal to $1$ in all cases, because the correlation $\widehat{\mathrm{\rho }}$ between the local and global “parts” of $\mathrm{\pi }$ is close to $0$ (see Theorem 4).

We measure the strength of collinearity using the linear correlation coefficient, defined for random variables $U$ and $V$ that have finite, positive variances as

$$\mathbf{C}\mathbf{o}\mathbf{r}\mathbf{r}(U,V):=\frac{\mathbf{E}(\mathit{UV})-\mathbf{E}(\mathit{U})\mathbf{E}(\mathit{V})}{\sqrt{\mathbf{V}\mathbf{a}\mathbf{r}(U)\mathbf{V}\mathbf{a}\mathbf{r}(V)}}.$$

Specifically, we consider the correlation

$$\begin{array}{c}\hfill \mathrm{\rho }:=\mathbf{C}\mathbf{o}\mathbf{r}\mathbf{r}\left(-logq(X),log\mathrm{\pi }(X)\right).\end{array}$$ [5]

While the value of $\mathrm{\rho }$ depends on the distribution $\mathbf{P}$ of the random state $X$, our results hold for any distribution that makes $\mathrm{\rho }$ defined. In other words, we merely assume that the logarithms of $\mathrm{\pi }(X)$ and $q(X)$ have finite, positive variances. For example, $\mathbf{P}$ could be the uniform distribution over states or the stationary distribution $\mathrm{\pi }$.

Our next result is a formula for $\mathrm{\rho }$ in terms of the correlation between the local and global parts of $\mathrm{\pi }$

$$\widehat{\mathrm{\rho }}:=\mathbf{C}\mathbf{o}\mathbf{r}\mathbf{r}\left(-logq(X),log\widehat{\mathrm{\pi }}(X)\right),$$

and the relative number of exponential scales over which they vary

$$r:=\sqrt{\frac{\mathbf{V}\mathbf{ar}(log\widehat{\mathrm{\pi }}(X))}{\mathbf{V}\mathbf{ar}(logq(X))}}.$$

Technically, $\widehat{\mathrm{\rho }}$ is undefined when $\widehat{\mathrm{\pi }}(X)$ is constant. However, in this case, we will adopt the useful convention that $\widehat{\mathrm{\rho }}=0$.

Theorem 3

The correlation coefficient $\mathrm{\rho }$ satisfies

$$\begin{array}{c}\hfill \mathrm{\rho }=\frac{1+\widehat{\mathrm{\rho }}r}{\sqrt{1+2\widehat{\mathrm{\rho }}r+r^2}}.\end{array}$$ [6]

According to Eq. 6, the correlation $\mathrm{\rho }$ is strictly positive unless $\widehat{\mathrm{\rho }}$ is at most $-1/r$, which is impossible if $r$ is less than $1$ because $\widehat{\mathrm{\rho }}\in [-1,1]$. Plots of the contours of $\mathrm{\rho }$ in the $(r,\widehat{\mathrm{\rho }})$ strip (SI Appendix, Fig. S1) further show that $\mathrm{\rho }$ is close to $1$ to the extent that $\widehat{\mathrm{\rho }}$ is close to $1$ or $r$ is close to $0$. A simple lower bound of $\mathrm{\rho }$, which makes no reference to $\widehat{\mathrm{\rho }}$, reflects the latter fact (SI Appendix, Corollary S1).

Theorem 3 explains the relatively high correlations exhibited by Markov chains with random rates (Fig. 4). Specifically, when the random rates have exponential distributions, the local and global parts of $\mathrm{\pi }$ are approximately uncorrelated and have nearly equal variances (Fig. 4 A and C), meaning that $\widehat{\mathrm{\rho }}\approx 0$ and $r\approx 1$, hence $\mathrm{\rho }\approx 1/\sqrt{2}\approx 0.71$ by Eq. 6. When the rates instead have log-normal distributions, the local and global parts remain approximately uncorrelated, but the local part varies relatively more, leading to a lesser $r$ and a correspondingly greater $\mathrm{\rho }$ (Fig. 4 B and D). The difference between the cases of exponential and log-normal rates is not due to the exponential rates’ lesser variance; the key quantities in Fig. 4A remain constant as the variance of the exponential rates increases over hundreds of scales (SI Appendix, Fig. S2). Note that the correlations in Fig. 4 concern a uniformly random state $X$. If $X$ is instead distributed according to $\mathrm{\pi }$, then the experiments in Fig. 4A–C have qualitatively similar results, but the case of log-normal rates on the hypercube (Fig. 4D) is markedly different: the correlation $\widehat{\mathrm{\rho }}$ equals $-0.40$, and the linear relationship between the logarithms of $1/q(X)$ and $\mathrm{\pi }(X)$ has a lesser slope of $0.72$ (SI Appendix, Fig. S3).

In contrast to the conclusions of Theorems 1 and 2, there may be no $\beta$ for which $\mathrm{\pi }$ and ${\mathrm{\nu }}_{\beta }$ are within a small factor, even when $\mathrm{\rho }$ is close to $\pm 1$. (See SI Appendix, Proposition S1 for a bound on this factor.) However, in this case, there is a $\beta$ for which ${\mathrm{\nu }}_{\beta }$ accurately predicts the relative probabilities of states, in the sense of making the following, log-ratio error small:

$$\begin{array}{c}\hfill \mathbf{L}(\beta ):=\mathbf{E}\left(\frac{1}{2}log{\left(\frac{\mathrm{\pi }(X)}{\mathrm{\pi }(Y)}/\frac{{\mathrm{\nu }}_{\beta }(X)}{{\mathrm{\nu }}_{\beta }(Y)}\right)}^2\right).\end{array}$$ [7]

Note that the exponent in the expectation modifies the logarithm, not its argument, and the expectation is with respect to $X$ and $Y$, which are independent and distributed according to $\mathbf{P}$. In fact, while $\mathbf{L}(\beta )$ is distinct from the mean squared error that is used to fit the lines in Fig. 4, its minimizer ${\beta }^{\ast }$ coincides with the slopes of these lines (See SI Appendix Proposition S2). Our final result therefore explains that these slopes are close to $1$ because $\widehat{\mathrm{\rho }}$ is close to $0$.

Theorem 4

The log-ratio error $\mathbf{L}(\beta )$ is minimized by ${\beta }^{\ast }=1+\widehat{\mathrm{\rho }}r$ and $\mathbf{L}({\beta }^{\ast })$ equals

$$\begin{array}{c}\hfill \left(1-{\mathrm{\rho }}^2\right)\mathbf{V}\mathbf{a}\mathbf{r}(log\mathrm{\pi }(X))=\left(1-{\widehat{\mathrm{\rho }}}^2\right)\mathbf{V}\mathbf{a}\mathbf{r}(log\widehat{\mathrm{\pi }}(X)).\end{array}$$

Theorem 4 states that the local distribution ${\mathrm{\nu }}_{\beta }$ that best approximates $\mathrm{\pi }$ in the sense of preserving its relative probabilities is ${\mathrm{\nu }}_{{\beta }^{\ast }}$. While the Markov chains with random rates in Fig. 4 have slopes ${\beta }^{\ast }\approx 1$, many familiar Markov chains have ${\beta }^{\ast }\not\approx 1$ and even ${\beta }^{\ast }<0$. For example, consider the random energy model on the $N$-dimensional hypercube, which assigns i.i.d. random energies $E(x)$ to states $x\in {\{0,1\}}^N$ (48, 49). These energies determine a Boltzmann probability distribution $\mathrm{\pi }(x)\propto exp(-E(x))$. The two Markov chains defined by the rates $Q(x,y)=exp(E(x))$ and $Q(x,y)=exp(-E(y))$ are in detailed balance with $\mathrm{\pi }$, as are the chains determined by the weighted geometric means of these rates, defined for $\lambda \in [0,1]$ by $Q(x,y)=exp(\lambda E(x)-(1-\lambda )E(y))$. [Markov chains of this kind are known as Glauber dynamics (50).] As $\lambda$ ranges from $0$ to $1$, the correlation $\mathrm{\rho }$ increases from $0$ to $1$, while the slope ${\beta }^{\ast }$ remains nonnegative (Fig. 5, red points). In contrast, under the Sherrington–Kirkpatrick (SK) model, i.i.d. random couplings $g_{\mathit{ij}}$ of the coordinates $x_i$ determine the energies $E(x)=\sum _{i,j}{g}_{\mathit{ij}}{x}_i{x}_j$ of each state (51). Due to the dependence of nearby states’ energies, the correlations and slopes take negative values when $\lambda$ is less than roughly $0.4$ (Fig. 5, blue points).

Fig. 5.

The relationships between exit rates and stationary probabilities for spin glasses with reversible dynamics. Each Markov chain has $2^{10}$ states corresponding to the vertices of a $10$-dimensional hypercube, random rates satisfying detailed balance, and a Boltzmann stationary distribution given by $\mathrm{\pi }(x)\propto exp(-E(x))$, where the rates and stationary distribution depend on energies $E(x)$ assigned to each state. In (A–C), these energies are given by the random energy model (REM)—namely, i.i.d. normal random variables with a mean of $0$ and a variance of $10$. In (D–F), the energies are those of the Sherrington–Kirkpatrick (SK) model, which are correlated through random couplings $g_{\mathit{ij}}$, according to $E(x)=\sum _{i,j}{g}_{\mathit{ij}}{x}_i{x}_j$, with couplings that are i.i.d. normal random variables with a mean of $0$ and a variance of $1/10$. (A) We can see that the first set of rates has exit rates that exactly satisfy a linear relationship between the logarithms of $\mathrm{\pi }(x)$ and $1/q(x)$. (B) The other extreme, transition rates that depend on the energy of the destination, produce no detectable linear relationship. In (C), the geometric mean of the first two rates produces high correlation, with a slope greater than $1$. (D) For the SK model, the logarithms of $\mathrm{\pi }(x)$ and $1/q(x)$ are exactly collinear, as in (A). (E) However, because adjacent states $x,y$ have correlated energies under the SK model, $log\mathrm{\pi }(x)$ and $-logq(x)$ can be highly negatively correlated, unlike in (B). (F) The geometric mean of the rates again produces high, positive correlation, but with an even greater slope. Finally, (G and H) show that transition from local-global anticorrelation to correlation occurs in the SK model when the weighting $\lambda$ in the geometric mean of the extreme rates is roughly $0.4$.

Theorems 3 and 4 characterize the relationship between the exit rates and stationary probabilities of a Markov chain, in terms of $\widehat{\mathrm{\rho }}$ and $r$. We can use these results to better understand how the local-global relationship changes as the parameters of the chain vary. Fig. 6 demonstrates this idea by comparing two models of ant colony behavior that undergo stochastic bifurcations as colony size varies. The Föllmer–Kirman (FK) model describes the number of ants that choose one of two identical paths to a food source, under the effects of random switching and recruitment (52–54). It predicts that, as colony size $N$ increases, the colony abruptly transitions from alternately concentrating on one path to splitting evenly between the two. The Beekman–Ratnieks–Sumpter (BRS) model concerns the number of ants that follow and reinforce a pheromone trail, by spontaneously finding it or by being recruited to it (55, 56).^¶ It predicts that there is a critical colony size above which the colony sustains a trail, but below which it does not. Despite the apparent similarities of the FK and BRS models, the relationships between their local and global parts change in entirely different ways as colony size increases, as the quantities $\widehat{\mathrm{\rho }}$ and $r$ show.

Fig. 6.

Stochastic bifurcations in two models of ant collective behavior, characterized in terms of $\mathrm{\rho }$, $\widehat{\mathrm{\rho }}$, and $r$. The Föllmer–Kirman (FK) and Beekman–Ratnieks–Sumpter (BRS) models are Markov chains with nearest-neighbor jumps on the state space $\{0,1,\cdots ,N\}$. (A) The local and global parts of the FK model’s stationary distribution $\mathrm{\pi }$ are essentially perfectly anticorrelated for every $N$. However, as $N$ increases, the global part begins to vary more than the local part, which is reflected by $r$ going from below $1$ to above $1$, and which causes the correlation $\mathrm{\rho }$ between $log\mathrm{\pi }(x)$ and $-logq(x)$ to abruptly change from $1$ to $-1$, as shown in (B). (C) In contrast, the BRS model generally has $r$ much larger than $1$, so the correlation $\mathrm{\rho }$ between $log\mathrm{\pi }(x)$ and $-logq(x)$ is essentially determined by the correlation $\widehat{\mathrm{\rho }}$ between the local and global parts. The correlation $\widehat{\mathrm{\rho }}$ decreases as $N$ increases to the bifurcation point, but recovers to a moderate value for larger $N$. The same is true of $\mathrm{\rho }$ (B). (A–C) To make the bifurcation points of both models approximately $N=50$, we used the parameters $(k_1,k_2)=(0.02,1)$ and $(b_1,b_2,b_3)=(0.03,0.002,2)$.

Under the FK model, $\mathrm{\rho }$ sharply decreases from $1$ to $-1$, as the colony size $N$ increases through the bifurcation point (Fig. 6B). The abrupt transition from correlation to anticorrelation in fact arises from a gradual increase in $r$, the variance of the steady state’s global part relative to its local part (Fig. 6A). The slope ${\beta }^{\ast }$ decreases linearly during this transition, as Theorem 4 predicts (SI Appendix, Fig. S4A). The exit rates and steady state of the BRS model are also highly anticorrelated in the vicinity of the critical colony size (Fig. 6B). However, $\mathrm{\rho }$ changes more gradually and the correlation recovers to a positive value at larger colony sizes. Fig. 6C shows that, in contrast to the FK model, the variation in $\widehat{\mathrm{\rho }}$ drives the decrease in $\mathrm{\rho }$ near the critical colony size. Since $r$ is much larger than $1$ for all but the smallest colonies, its variation matters little to $\mathrm{\rho }$, which is reflected by the flatness of the contours in Fig. 6C. Instead, the variation of $r$ greatly affects ${\beta }^{\ast }$, which takes values as low as roughly $-15$ and as high as roughly $30$, as the colony size increases (SI Appendix, Fig. S4B). We note that the results in Fig. 6 concern a uniformly random state $X$. The curves of the BRS model differ substantially when $X$ is distributed according to $\mathrm{\pi }$, but a sharp decrease in $\mathrm{\rho }$ near the bifurcation point arises in largely the same way (SI Appendix, Figs. S5 and S6).

Discussion

The conceptual heart of rattling is the simple observation that many systems tend to spend more time in configurations that they exit more slowly. While systems can exhibit this tendency regardless of whether detailed balance holds, it fails to be universal because the time that a system spends in a configuration depends on both the duration and the frequency of its visits, which can oppose one another. The former is a local property of a configuration, while the latter is generally a global property of the system. In this sense, the failure of the rattling heuristic requires “adversarial global structure,” as Chvykov et al. claimed (10). Theorem 3 implies a precise version of this claim: $\mathrm{\rho }$ is positive unless the local and global parts of the stationary distribution are sufficiently anticorrelated, that is, unless $\widehat{\mathrm{\rho }}$ is at most $-1/r$. In particular, the rattling heuristic always holds when the local part varies more than the global part ($r<1$).

Together, $r$ and $\widehat{\mathrm{\rho }}$ determine the accuracy of the rattling heuristic as well as the form of the weights that it predicts (Theorems 3 and 4). We view each pair $(r,\widehat{\mathrm{\rho }})$ as defining a class of chains, say ${\mathcal{Q}}_{(r,\widehat{\mathrm{\rho }})}$, for which a certain local-global principle applies (Fig. 2). The simplest class ${\mathcal{Q}}_{(0,0)}$ consists of chains with uniform global part, which therefore satisfy $\mathrm{\rho }=\beta =1$ and $\mathrm{\pi }(x)\propto 1/q(x)$. Theorem 2 explains that it is simple to make these chains by scaling the entries of doubly stochastic matrices (Fig. 3). More generally, the class ${\mathcal{Q}}_{(r,0)}$ consists of chains with uncorrelated local and global parts, hence they satisfy $\mathrm{\rho }=1/\sqrt{1+r^2}$ and $\beta =1$, and the rattling heuristic predicts their weights to be $1/q(x)$. As Fig. 4 shows, many chains with random rates essentially belong to this class. The chains in ${\mathcal{Q}}_{(·,0)}$ are analogous to the systems that Chvykov et al. considered to be the domain of rattling (10). However, rattling also perfectly predicts the weights $1/q{(x)}^{\beta }$ of chains in the class ${\mathcal{Q}}_{(r,\pm 1)}$, with an exponent $\beta =1\pm r$ that is generally not of order $1$ and which can even be negative. Figs. 5 and 6 include relevant examples.

These key quantities depend on the distribution of the random state $X$, which models how the system is observed. The preceding discussion and Figs. 4–6 emphasize the case when $X$ is uniformly random, primarily because $\mathrm{\rho }$ then coincides with the empirical correlations that prior work observed and sought to explain (10, 13). However, Theorems 3 and 4 are far more general—they work for any distribution that makes $\mathrm{\rho }$ defined (Eq. 5). For example, $X$ can be chosen according to the stationary distribution $\mathrm{\pi }$, so that $\mathrm{\rho }$ emphasizes the correlation among states with the greatest steady-state weight. The comparison of Figs. 4D and 6 with SI Appendix, Figs. S3D and S5 shows that this choice can lead to qualitatively different results when $\mathrm{\pi }$ is far from uniform.

While our results confirm the general sense that rattling should predict the steady states of a broad class of nonequilibrium systems (9, 10, 13, 15), they also show that the original rationale for its success was not entirely correct. Specifically, Chvykov et al. attributed the correlation in examples like Fig. 4A and B to the large number of transition rates into and out of each state and their independence (10). However, these properties fail to explain the similar correlations in Fig. 4C and D, where the states are sparsely connected, and those in Fig. 5, where the rates are strongly dependent. Theorem 3 reveals that these properties are neither necessary nor sufficient for high correlation.

To fully realize rattling’s promise as a principle of self-organization, future work should further develop its methodology, broaden its applications, and deepen its theoretical foundations. First, detecting the failure of the rattling heuristic, like detecting the failure of detailed balance, generally requires global information about the dynamics. Practical approaches to the latter are the focus of recent work (57–61) and we anticipate parallel developments for rattling. One possible approach is to estimate the local and global parts of $\mathrm{\pi }$ by adapting, for example, the algorithm of ref. 62. Second, applications of rattling can take inspiration from Yang et al. (13), which engineered active microparticles that self-organize into a collective oscillator using rattling to infer a key design parameter. Finally, theoretical efforts can address the properties of rattling that are relevant to practice, like its behavior under transformations of the state space, possible analogues of our formulas for systems with continuous state spaces, and the relation of rattling to central topics of Markov chain theory, like mixing times (63) and metastability (64).

Materials and Methods

We prove Theorems 1–4 in this section. These results stem from the standard fact that the fraction of time $\mathrm{\pi }(x)$ that a Markov chain $Q$ spends in state $x$ in the long run is proportional to the fraction of visits $\widehat{\mathrm{\pi }}(x)$ that it makes to state $x$, multiplied by the expected duration $1/q(x)$ of each visit. Throughout this section, we continue to assume that $Q$ is an irreducible Markov chain on a finite number $N$ of states.

Proposition 0.1

The stationary distributions of a Markov chain $Q$ and its jump chain $\widehat{Q}$ are related by

$$\begin{array}{c}\hfill \mathrm{\pi }(x)=\frac{\widehat{\mathrm{\pi }}(x)/q(x)}{{\sum }_y\widehat{\mathrm{\pi }}(y)/q(y)}.\end{array}$$ [8]

Proof: Since $\widehat{\mathrm{\pi }}$ is the stationary distribution of $\widehat{Q}$, by the definition of the jump chain, $\widehat{\mathrm{\pi }}$ satisfies

$$\widehat{\mathrm{\pi }}(x)={\displaystyle \sum _{y\ne x}}\widehat{\mathrm{\pi }}(y)\widehat{Q}(y,x)={\displaystyle \sum _{y\ne x}}\frac{\widehat{\mathrm{\pi }}(y)}{q(y)}Q(y,x).$$

In other words, $\widehat{\mathrm{\pi }}/q$ is an invariant measure for $Q$:

$$(\widehat{\mathrm{\pi }}/q)Q=\mathbf{0}.$$

Normalizing $\widehat{\mathrm{\pi }}/q$ therefore gives the stationary distribution $\mathrm{\pi }$.

Proof of Theorem 1.

Recall that we are given $k\ge 1$ such that

$$\begin{array}{c}\hfill 1/(kN)\le \widehat{\mathrm{\pi }}(x)\le k/N,\end{array}$$ [9]

for every state $x$, and we must show that $\mathrm{\pi }(x)$ and ${\mathrm{\nu }}_1(x)$ are within a factor of $k^2$:

$$k^{-2}\le \mathrm{\pi }(x)/{\mathrm{\nu }}_1(x)\le k^2.$$

For the upper bound, we use Eq. 8 to write $\mathrm{\pi }(x)$ in terms of $\widehat{\mathrm{\pi }}(x)$ and then use Eq. 9 twice to find that

$$\begin{array}{c}\hfill \mathrm{\pi }(x)=\frac{\widehat{\mathrm{\pi }}(x)/q(x)}{{\sum }_y\widehat{\mathrm{\pi }}(y)/q(y)}\le k^2\frac{1/q(x)}{{\sum }_{y}1/q(y)}=k^2{\mathrm{\nu }}_1(x).\end{array}$$ [10]

The lower bound follows in a similar way.

The proof of Theorem 2 combines a perturbation bound for Markov chain stationary distributions with Theorem 1.

Proof of Theorem 2.

We are given a doubly stochastic matrix $D$ of the same size as $\widehat{Q}$, and a number $k\ge 1$ such that $\widehat{Q}$ and $D$ are within a factor of $k$, and we must show that $\mathrm{\pi }$ and ${\mathrm{\nu }}_1$ are within a factor of $k^{2N}$. We use Theorem 1 of ref. 46 to relate the ratio of the entries of $\widehat{Q}$ and $D$ to the ratio of their stationary probabilities. It states that, if irreducible stochastic matrices on $N$ states are within a factor of $k$, then their stationary distributions are within a factor of $k^N$. This result applies to $\widehat{Q}$ and $D$ because we assumed that $\widehat{Q}$ is irreducible and that $\widehat{Q}$ and $D$ are within a factor of $k$, hence $D$ is also irreducible. Since $D$ is doubly stochastic, it has a uniform stationary distribution. Theorem 1 of ref. 46 and the fact that $\widehat{Q}$ and $D$ are within a factor of $k$ therefore imply that $\widehat{\mathrm{\pi }}$ and the uniform distribution are within a factor of $k^N$. Using Theorem 1, we conclude that $\mathrm{\pi }$ and ${\mathrm{\nu }}_1$ are within a factor of $k^{2N}$.

The next proof uses the basic fact that, for any two random variables $U$ and $V$ with positive, finite variances, the correlation of $U$ and $U+V$ equals

$$\mathbf{C}\mathbf{o}\mathbf{r}\mathbf{r}(U,U+V)=\frac{\mathbf{V}\mathbf{ar}(U)+\mathbf{C}\mathbf{ov}(U,V)}{\sqrt{\mathbf{V}\mathbf{ar}(U)\left(\mathbf{V}\mathbf{ar}(U)+2\mathbf{C}\mathbf{ov}(U,V)+\mathbf{V}\mathbf{ar}(V)\right)}}.$$

With some algebra, we can rewrite this formula in terms of $s=\mathbf{C}\mathbf{o}\mathbf{r}\mathbf{r}(U,V)$ and $t^2=\mathbf{V}\mathbf{ar}(V)/\mathbf{V}\mathbf{ar}(U)$ as

$$\begin{array}{c}\hfill \mathbf{C}\mathbf{o}\mathbf{r}\mathbf{r}(U,U+V)=\frac{1+st}{\sqrt{1+2st+t^2}}.\end{array}$$ [11]

Recall that $\mathrm{\rho }$ denotes the correlation between the logarithms of $\mathrm{\pi }(X)$ and $1/q(X)$ for a random state $X$ distributed according to $\mathbf{P}$. The proof of Theorem 3 is a direct calculation of $\mathrm{\rho }$, using Eq. 8.

Proof of Theorem 3.

By Eq. 8, the probability $\mathrm{\pi }(X)$ satisfies

$$log\mathrm{\pi }(X)=-logq(X)+log\widehat{\mathrm{\pi }}(X)-log\left({\displaystyle \sum _y}\widehat{\mathrm{\pi }}(y)/q(y)\right).$$

Since the last term is constant, it does not affect the correlation $\mathrm{\rho }$, which therefore equals

$$\begin{array}{c}\hfill \mathbf{C}\mathbf{o}\mathbf{r}\mathbf{r}(-logq(X),-logq(X)+log\widehat{\mathrm{\pi }}(X))=\mathbf{C}\mathbf{o}\mathbf{r}\mathbf{r}(U,U+V)\end{array}$$

in terms of $U=-logq(X)$ and $V=log\widehat{\mathrm{\pi }}(X)$. We then obtain the formula for $\mathrm{\rho }$ from Eq. 11, where $s=\widehat{\mathrm{\rho }}$ and $t=r$.

Recall the definition of the log-ratio error $\mathbf{L}(\beta )$ in Eq. 7. The proof of Theorem 4 entails a calculation of $\mathbf{L}(\beta )$, using Eq. 8.

Proof of Theorem 4.

The log-ratio error $\mathbf{L}(\beta )$ equals

$$\begin{array}{c}\hfill \frac{1}{2}\mathbf{E}\left({(A-B)}^2\right)=\frac{1}{2}\left(\mathbf{E}\left(A^2\right)-2\mathbf{E}(AB)+\mathbf{E}\left(B^2\right)\right)=\mathbf{V}\mathbf{ar}(A),\end{array}$$

in terms of the auxiliary variables

$$\begin{array}{c}\hfill A=log\left(\mathrm{\pi }(X)/{\mathrm{\nu }}_{\beta }(X)\right)\text{and}B=log\left(\mathrm{\pi }(Y)/{\mathrm{\nu }}_{\beta }(Y)\right),\end{array}$$

due to the linearity of expectation and because $A$ and $B$ are i.i.d. By Eq. 8, the variance of $A$ equals

$$\begin{array}{c}\hfill \mathbf{V}\mathbf{ar}\left(log\widehat{\mathrm{\pi }}(X)+(\beta -1)logq(X)\right)=\mathbf{V}\mathbf{ar}(V-(\beta -1)U),\end{array}$$

where $U=-logq(X)$ and $V=log\widehat{\mathrm{\pi }}(X)$. We write the variance as

$$\begin{array}{c}\hfill \mathbf{V}\mathbf{ar}(V)-2(\beta -1)\mathbf{C}\mathbf{ov}(U,V)+{(\beta -1)}^2\mathbf{V}\mathbf{ar}(U),\end{array}$$

and then identify factors of $\widehat{\mathrm{\rho }}$ and $r$, to find that

$$\mathbf{L}(\beta )=\left(r^2-2\widehat{\mathrm{\rho }}r(\beta -1)+{(\beta -1)}^2\right)\mathbf{V}\mathbf{ar}(U).$$

The error is minimized by ${\beta }^{\ast }=1+\widehat{\mathrm{\rho }}r$, which then satisfies

$$\mathbf{L}({\beta }^{\ast })=r^2\left(1-{\widehat{\mathrm{\rho }}}^2\right)\mathbf{V}\mathbf{ar}(logq(X))=\left(1-{\widehat{\mathrm{\rho }}}^2\right)\mathbf{V}\mathbf{ar}(log\widehat{\mathrm{\pi }}(X)).$$

A simple but tedious manipulation of this equation using Eqs. 6 and 8 shows that $\mathbf{L}({\beta }^{\ast })$ further equals $(1-{\mathrm{\rho }}^2)\mathbf{V}\mathbf{ar}(log\mathrm{\pi }(X))$. See SI Appendix for details.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

There are no data underlying this work.