Emergent complexity in slowly driven stochastic processes

Abstract

We consider the distribution of first passage time events in the presence of non-ergodic modes that drive otherwise ergodic dynamics on a potential landscape. We find that in the limit of slow and large enough fluctuations the distribution of first passage time events, f(t), exhibits heavy tails dominated by a power law with exponent f(t) ~ t⁻², and corrections that depend on the strength and the nature of fluctuations. We support our theoretical findings through direct numerical simulations in illustrative examples.

Introduction.—

Complex dynamics are ubiquitous in the natural world. Despite their intrinsic irregularity and unpredictability, they can nonetheless exhibit coherent and universal emergent properties. Of particular importance in the study of complex systems is the understanding of the time it takes for rare events to occur [1–3]. Notable examples include natural disasters [4] or the spreading of a virus [5]. In fact, first passage times are central to many fields within physics and beyond, with important examples stemming from chemistry, biology and finance (see, e.g., [6–12] and references therein). Biology in particular is ripe with examples where time is of the essence [13], such as fertilization [14], intracellular events [15–21], search processes [22–24], neural activity [25, 26] or population dynamics [27].

We here consider the estimation of first passage time distributions (FPTDs) from finite-time observations in an experimental context. In particular, we are interested in systems with intrinsic time scales comparable to the observation time, for which weak ergodicity breaking becomes evident [28, 29]. Such dynamics can be found for instance in glassy systems [30–33], where the time scales of equilibration are so long that one can decompose the dynamics into a stationary component and an “aging” component that breaks time-translation invariance.

Our main inspiration comes from the less traditional branch of the physics of animal behavior [34, 35]. Remarkably, recent advances in machine vision (see, e.g., [36–39]) have resulted in an explosion of high spatiotemporal resolution behavioral data. Analysis of fine-scale posture movements shows that much like the runand-tumble behavior of bacteria [40], more complex organisms also exhibit stereotyped behaviors, albeit with a more intricate structure [41–48]. The notion of stereotypy in behavior inherently stems from the time scale separation between variations on what is defined as a behavioral state, and the transitions between behavioral states, much like a particle hopping between wells in a potential landscape. For example, while foraging for food the nematode worm C. elegans transitions between coarsegrained “runs” and “pirouettes”, which are stereotyped sequences of finer scale movements [48, 49]. However, unlike the particle hopping among potential wells which has a characteristic exponential distribution of transition times, the time spent in a given behavior can be heavy-tailed (see, e.g. Fig.4E of [48] or Fig.3 of [50]). We here hypothesize that such heavy-tailed distributions reflect the slow continuous modulation of behavior on longer time scales, resulting from environmental factors or fluctuating internal states driven by neuromodulation, such as hunger, stress or arousal (see, e.g., [51–53]). Indeed, it has been shown that C. elegans continuously modulates its rate of reorientation events to explore larger and larger arenas in search of food [54]. In order to truly capture the multiscale nature of behavior, we therefore need to account for the fact that it can be modulated on time scales comparable to the observation time.

In this Letter, we introduce a general model of behavior in which the pose dynamics evolves in potential landscapes that fluctuate over time. We then study how these dynamics impact the estimation of the distribution of times spent in a given behavior. In the first section, we introduce our phenomenological description of the behavioral dynamics, decomposing it into ergodic dynamics on a potential landscape and the non-ergodic modulation of the landscape. We then derive a general result for the distribution of first passage times, and illustrate it through direct numerical simulations in three example systems.

Slowly driven ergodic dynamics.—

Given a set of observations of animal locomotion (e.g. from video imaging), we consider that the dynamics can be decomposed into ergodic and non-ergodic components. The former are the state-space variables that mix sufficiently well and define the potential wells that correspond to the stereotyped behaviors; the latter non-ergodic components evolve on time scales comparable to the observation time and slowly modulate the potential landscape. The full dynamics is thus given by

$$\{\begin{array}{l}\dot{\overrightarrow{X}}=F(\overrightarrow{X},\overrightarrow{\lambda })\hfill \\ {\tau }_{\lambda }\dot{\overrightarrow{\lambda }}=G(\overrightarrow{X},\overrightarrow{\lambda })\hfill \end{array},$$ (1)

where $\overrightarrow{X}\in {ℝ}^D$ represents the ergodic components, $\overrightarrow{\lambda }\in {ℝ}^{D_{\lambda }}$ represents the non-ergodic degrees of freedoms, F and G are nonlinear, possibly noisy, functions, and τ_λ is assumed to be of the order of the measurement time T_exp, τ_λ = 𝒪(T_exp), such that the $\overrightarrow{\lambda }$ dynamics do not mix. Given the time scale separation between the dynamics of $\overrightarrow{X}$ and $\overrightarrow{\lambda }$, we assume that the dynamics of $\overrightarrow{X}$ is well approximated by quasi-stationary Fokker-Planck dynamics $\dot{\rho }=ℒ\rho$, where ℒ represents the Fokker-Planck operator. Since we are primarily interested in the long time scale behavior of the system, we consider a projection of the dynamics onto the slowest mode of ℒ, yielding a generalized Langevin equation [55, 56] with history-dependent friction and fluctuations. Assuming that we can sample the system on a time scale longer than the noise correlation time, we obtain an effective overdamped description:

$$\dot{\overrightarrow{X}}=F(\overrightarrow{X},\overrightarrow{\lambda })\Rightarrow \dot{x}=-{\partial }_{x}U(x,\lambda )+\sqrt{2T_x}{\eta }_x(t),$$ (2)

where T_x captures the effective temperature, η_x is Gaussian white noise, and λ is a slow control parameter that modulates the effective potential landscape on slow time scales. Similarly, we consider that λ also obeys an effective overdamped Langevin equation,

$$\dot{\lambda }=-{\tau }_{\lambda }^{-1}{\partial }_{\lambda }V(\lambda )+\sqrt{2T_{\lambda }{\tau }_{\lambda }^{-1}}{\eta }_{\lambda }(t),$$ (3)

where V is assumed to be uncoupled from the dynamics of x for simplicity, T_λ captures the degree of fluctuations in λ and η_λ is Gaussian white noise.

First passage time distributions.—

We are primarily interested in studying the time spent in a given behavioral state. Within the context of the Langevin dynamics of Eq.2, this is given by the first passage time to reach an energy barrier x_f from the bottom of the potential x₀, defined as,

$${\tau }_{x_0,x_f}(\lambda )=\text{inf}\left\{\tau :x(t+\tau ,\lambda )=x_f\mid x(t,\lambda )=x_0\right\}.$$ (4)

Despite the general interest in this concept, finding analytical expressions for the density of first passage time events is generally a formidable task [57]. Remarkably few closed-form expressions for the FPTD are known, with most results concerning only the mean first passage time (MFPT) which is more tractable (see, e.g., [1, 6, 9]). However, the MFPT provides only limited information, especially when multiple time scales are involved [15]. Here, we are interested in studying the behavior of the full first passage time distribution, with particular focus on its long time behavior in the presence of weakly non-ergodic dynamics, Eqs.2 and 3.

As previously discussed, the measurement time T_exp essentially separates ergodic from non-ergodic dynamics. In addition, it also sets a lower bound on the slowest observed hopping rates ${\omega }_{\text{min}}~T_{\text{exp}}^{-1}$, such that when τ_λ = 𝒪(T_exp) we can make an adiabatic approximation and assume that transition events occur within a nearly static potential. For a given hopping rate ω, the first passage time distribution is given by

$$f(t,\omega )=\omega e^{-\omega t},$$

where $\omega (\lambda )=1/{\tau }_{x_0,x_f}(\lambda )$ is the dominating slow kinetic transition rate which implicitly depends on the dynamics of λ. When we allow λ to fluctuate slowly, the distribution of first passage times f(t) is given by the expectation value of f(t, ω) over the distribution of ω, p(ω), weighted by the effective number of transition observed within T_exp, which is proportional to ω. Marginalizing over ω we get

$$f(t)~{\int }_{{\omega }_{\text{min}}}^{{\omega }_{\text{max}}}p(\omega ){\omega }^2{e}^{-\omega t}d\omega .$$ (5)

While the barrier height is going to depend on the dynamics of a slow control parameter λ, the tail of the distribution is going to be dominated by instances in which the barrier height is the largest, motivating the use of Kramers approximation (see, e.g., [2]),

$$\omega (\lambda )={\omega }_0\text{\hspace{0.17em}exp}\left\{-\frac{\Delta U(\lambda )}{T_x}\right\},$$ (6)

where ΔU(λ) = U(x_f, λ)−U(x₀, λ) and ω₀ is a constant. For multiple realization of Eq.3 with different initial conditions, the distribution of λ is given by the Boltzmann weight [58],

$$p(\lambda )~\text{exp}\left\{-\frac{V(\lambda )}{2T_{\lambda }}\right\}.$$ (7)

Leveraging Eqs.S1,6,7 we can obtain an asymptotic approximation of the FPTD in the large t limit (see Supplemental Material),

$$f(t)~t^{-2}\text{\hspace{0.17em}exp}\left\{-\frac{V\left(\Delta U^{-1}\left(T_x\text{\hspace{0.17em}log}\left({\omega }_{0}t\right)\right)\right)}{2T_{\lambda }}\right\},$$ (8)

where ΔU⁻¹(·) represents the inverse function of the potential difference defined by Eq. 6 and we have kept only the dominant order of the asymptotic approximation (see Supplemental Material). For very general conditions on V (λ) and U(x, λ), we thus get f(t) ~ t⁻² for t → ∞ and T_λ ≫ 1. In the following section we will demonstrate the validity of this result in three illustrative examples.

Illustrative examples: Slowly-driven harmonic oscillator.—

Consider that x evolves in a harmonic potential, U(x, s) = (x − sx_f)², that is driven by a slow parameter s that fluctuates within V (s) = s²/2, pushing U(x, s) closer or further from x_f in a time scale τ_s, Fig.1(a). The equations of motion are given by a set of Ito stochastic differential equation, corresponding to coupled Ornstein-Uhlenbeck processes,

$$\{\begin{array}{l}d{x}_t=-\left(x_t-s_t{x}_f\right)dt+\sqrt{2T_x}d{W}_t\hfill \\ d{s}_t=-{\tau }_s^{-1}{s}_t+\sqrt{2T_s{\tau }_s^{-1}}d{W}_t\hfill \end{array},$$ (9)

where T_x and T_s captures the degree of fluctuations, dW_t is a Wiener Gaussian white noise process. We are interested in the density of first passage time events from the minimum of the potential x₀ = s to x_f = 1, for which it is challenging to find a closed form analytical expression, even when $s(t)=s\in ℝ$ [57]. In the Supplemental Material, we derive the FPTD in Laplace space [59] and leverage it to estimate the FPTD through numerical inversion [60] for varying values of τ_s (as in Ref. [61]), see Fig.S2. We find that when s fluctuates fast enough, τ_s → 0, we can average out s and get the simpler dynamics $d{x}_t=-\left(x_t-\langle s\rangle x_f\right)dt+\sqrt{2T_x}d{W}_t$. In this case, the FPTD is well approximated by f(t) ≈ f(t, 〈ω〉) = 〈ω〉e^−〈ω〉t, where 〈ω〉 is the average hopping rate which is set by 〈s〉. Even when τ_s > 0 but short, it is possible to obtain a self-consistent Markovian dynamics for x(t) (see e.g., [1]). In this case, the distribution of first passage times is still dominantly exponential, but with a corrected first passage time which depends on the ratio of temperatures T_s/T_x and the slow time scale τ_s. However, as we have shown in the previous section τ_s is large enough, τ_s ~ T_exp, the distribution of first passage times becomes heavy-tailed. In this limit, we can leverage Eq.8 to derive an asymptotic approximation to the distribution of first passage times. The tail of the distribution will be dominated by low ω values, which correspond to |s| ≫ 1. In this limit, the barrier height primarily behaves as ΔU(s) = s²/2 + 𝒪(s). In addition, since V (s) = s²/2, we see that V (ΔU⁻¹(x)) = x and Eq.8 yields (see Supplemental Material),

$$f(t)~t^{-2-\frac{T_x}{2T_s}},$$ (10)

which matches what we obtain from direct numerical simulations of Eq.9, Figs.1(b),S2,S3(a).

FIG. 1.

Heavy-tailed first passage time distributions for a slowly-driven overdamped harmonic oscillator. (a) We simulate the dynamics of a particle in a harmonic oscillator while slowly driving the potential landscape, and estimate the distribution of times it takes to reach x_f. The gray line represents the minimum of potential, x₀ = s, and the color scheme different values of s. (b) FPTDs obtained from direct numerical simulations of Eq.9 for different values of the temperature T_s that controls the level of fluctuations for the parameter driving the slow variations of the potential landscape. As predicted, the tail of the distribution behaves as a power law with an exponent f(t) ~ t^−2−α, with $\alpha =\frac{T_x}{2T_s}$. The color scheme represents different ratios of temperatures, and the black dashed line the T_s →∞ limit.

Illustrative examples: Slowly-driven double-well potential.—

We now consider a symmetric double-well potential in which the barrier height is slowly modulated according to an Ornstein-Uhlenbeck process, Fig.2(a),

$$\{\begin{array}{l}d{x}_t=-4s_t^2{x}_t{\left(x_t-1\right)}^{2}dt+\sqrt{2T_x}d{W}_t\hfill \\ d{s}_t=-{\tau }_s^{-1}\left(s_t-{\mu }_s\right)dt+\sqrt{2T_s{\tau }_s^{-1}}d{W}_t\hfill \end{array},$$ (11)

where all the parameters are the same as in Eq.9 with an extra μ_s that represents the expectation value of s, which we set as μ_s = 1. In this case, we have a quartic potential for x, U(x, s) = s²(x² − 1)², which yields ΔU(s) = s². Since V (s) = s²/2, we see that V (ΔU⁻¹(x)) = x/2 and Eq.8 yields (see Supplemental Material),

$$f(t)~t^{-2-\frac{T_x}{4T_s}},$$ (12)

matching what we find through direct numerical simulations of Eq.11, Figs.2(b),S3(b)

FIG. 2.

Heavy-tailed first passage time distribution of a slowly-driven double-well potential. (a) Schematic of the variation in the double-well potential with s (colored from blue to red; the black line represents s = μ_s). (b) FPTDs from direct numerical simulations of Eq.11 for different values of T_s. As expected, the tail of the distribution behaves as a power law f(t) ~ t^−2−α, where $\alpha =\frac{T_x}{4T_s}$ (colored line). The black dashed line represents the T_s →∞ limit.

Illustrative examples: Slowly-driven rugged parabolic potential.—

Finally, we consider a rugged parabolic potential as a simple model of the rough energy landscapes found across complex systems, from glasses to proteins (see, e.g., [19, 20, 62]). We construct a rugged landscape by superimposing a sinusoidal perturbation onto a harmonic potential [63], U(x, s) = U₀(x, s) + U₁(x), where U₀(x, s) = (x − s)²/2 and U₁(x) = −cos(2πkx)/(kπ). The corresponding dynamics are given by,

$$\{\begin{array}{l}d{x}_t=-\left(x_t-s_t+2\text{\hspace{0.17em}sin}\left(2\pi k{x}_t\right)\right)dt+\sqrt{2T_x}d{W}_t\hfill \\ d{s}_t=-{\tau }_s^{-1}{s}_t+\sqrt{2T_s{\tau }_s^{-1}}d{W}_t\hfill \end{array},$$ (13)

where k sets the number of smaller barriers between the global minimum of the potential and x_f = 1. We set k = 10 resulting in a rugged potential as illustrated in Fig.3(a). In this case, since U(x, s) is not as simple as before, it is more challenging to derive the correction terms to the power law. However, it has been shown [63] that by spatial averaging of U₁(x) = −cos(2πkx)/(kπ) over one period, the resulting hopping rate is simply corrected by a constant prefactor $\omega =I_0^{-2}\left(k^{-1}{\pi }^{-1}{T}_x^{-1}\right){\omega }_0$, where I₀ is the modified Bessel function and ω₀ is the hopping rate in the absence of the sinusoidal perturbation (from U₀(x, s) = (x − s)²/2). As such, we expect the asymptotic behavior of f(t) to be the same as for the slowly driven harmonic potential, Eq.10. Indeed, this is what we observe in Figs.3(b),S3(c).

FIG. 3.

Heavy-tailed first passage time distribution in slowly driven rugged parabolic potential. (a) We estimate the first passage time to reach x_f from the global minimum of a rugged parabolic potential. (b) FPTDs from direct numerical simulations of Eq.13 for different values of T_s. As expected, the tail of the distribution behaves as a power law f(t) ~ t^−2−α (colored lines) with $\alpha =\frac{T_x}{2T_s}$. The black dashed line corresponds to the T_s →∞ limit.

Discussion.—

Inspired by quantitative analysis of animal behavior, we here examined how the existence of slow non-ergodic modes impacts the statistics collected experimentally, focusing on the distribution of first passage time events. Our results show the emergence of heavy-tailed distributions. In particular, we find that the distribution asymptotes to a power law with an exponent f(t) ~ t⁻² in the limit of large fluctuations, regardless of the details of the dynamics. As remarked in the Introduction, our results have important implications to a wide variety of fields, and we here discuss some of these in detail.

In the context of animal behavior, heavy-tailed first passage times with an exponent f(t) ≈ t⁻² have been found extensively across multiple species, from bacteria [64], termites [65] and rats [66] to marine animals [67, 68], humans [69] and even fossil records [70]. In the context of search behaviors (e.g., when foraging for food), such observations have led researchers to hypothesize that Lévyflights (power law distributed run lengths) are efficient search strategies and thus evolutionarily favorable [71–75]. However, we here show that such fat tails may emerge when the animal is continuously adapting its behavior (slowly modulating the potential landscape), even in the absence of external drives. We therefore predict that disrupting the internal mechanisms for slow modulation of behavior (e.g. neuromodulatory pathways) should result in distribution of first passage times that have exponential tails.

Power laws have been observed in a wide variety of systems, from solar flares [76, 77] to the brain [78] and different hypotheses have been put forward to explain their emergence (for a review see e.g. [79]). Among these, work inspired by phase transitions in statistical mechanics associates power laws to “criticality”, mostly due to the fact that models inferred from the data appear to require fine-tuning of the model parameters to a special regime between two qualitatively different “phases” (see, e.g., [80]). However, as we have shown here, power laws can emerge without fine tuning and far from “criticality”. Indeed, slow modes that evolve on time scales comparable to the observation time are challenging to infer from data, and can give rise to best-fit models that appear “critical”. While some of the arguments we have put forward have also been proposed in other contexts [22, 81–83], we here place them into the framework of out-ofequilibrium statistical mechanics, explicitly connecting the long time scale emergent behavior with the underlying effective fluctuations. In addition, unlike other approaches [82, 84], our framework does not require explicit external drives, but simply collective modes that evolve in a weakly non-ergodic fashion.

Our starting point is an effective description of the long time scale dynamics, and further work will be required to fully bridge between microscopic dynamics, and the emergent long time behavior of the first passage time distribution that we uncovered. For example, we find that for intermediate values of 1 ≪ T_λ ≪ T_exp the FPTD behaves as a truncated power law with an effective exponent that is slightly smaller that −2 (see Supplemental Material), which goes beyond arguments presented here. What are the minimum τ_λ and T_λ for power laws to be measurable, and how do simple exponentials (T_λ ≪ T_exp) transition to power law behavior? These are important questions if one hopes to test our predictions in an experimental context (using for example a set-up akin to the ones used to test stochastic resonance [85, 86]). Additionally, we note that when T_λ ≫ T_exp, the distribution of initial conditions determines the emergent behavior, see Fig.S4. Inspired by experiments in animal behavior, which are typically done with multiple animals, we here assume that the initial condition for the slow mode is sampled according to its Boltzmann distribution $\lambda (t=0)~e^{-\frac{V(\lambda )}{2T_{\lambda }}}$, reflecting the variability across individuals. In this case, the emergent behavior we have derived holds true from τ_λ = T_exp to τ_λ → ∞. However, if the variability across experiments is smaller than that of the Boltzmann distribution, the τ_λ → ∞ limit will differ from the behavior at τ_λ ~ T_exp. Indeed, if the variance of the initial distribution of λ is smaller than that of the Boltzmann distribution, the temperature T_λ in our derivation should be changed to a new effective temperature $T_{\lambda }^0<T_{\lambda }$ reflecting the lower variance of the initial conditions. Making this transformation we still get a power law distribution of first passage times, but with a modified exponent that reflects the lower variance (see Supplemental Material).

To conclude, we have considered the effect of slow non-ergodic modulations and theoretically captured their effects on the distribution of first passage times, a result that we believe is widely relevant to a range of natural systems.