Non-equilibrium early-warning signals for critical transitions in ecological systems

Significance

Complex systems can be multistable and exhibit transitions between alternative locally stable states. We employ the landscape-flux theory from nonequilibrium statistical mechanics as a general framework to quantify the global stability of ecological systems. The average flux as the nonequilibrium driving force, the entropy production as the thermodynamic driving force, and time irreversibility of the cross-correlation functions can serve as warning signals for critical transitions between alternative stable states much earlier than other conventional predictors. We utilize a classical shallow lake model as an exemplar for our early warning prediction based on the landscape-flux approach. The method we propose is general and can be readily applied to assess the resilience of many other complex ecological systems.

Abstract

Complex systems can exhibit sudden transitions or regime shifts from one stable state to another, typically referred to as critical transitions. It becomes a great challenge to identify a robust warning sufficiently early that action can be taken to avert a regime shift. We employ landscape-flux theory from nonequilibrium statistical mechanics as a general framework to quantify the global stability of ecological systems and provide warning signals for critical transitions. We quantify the average flux as the nonequilibrium driving force and the dynamical origin of the nonequilibrium transition while the entropy production rate as the nonequilibrium thermodynamic cost and thermodynamic origin of the nonequilibrium transition. Average flux, entropy production, nonequilibrium free energy, and time irreversibility quantified by the difference in cross-correlation functions forward and backward in time can serve as early warning signals for critical transitions much earlier than other conventional predictors. We utilize a classical shallow lake model as an exemplar for our early warning prediction. Our proposed method is general and can be readily applied to assess the resilience of many other ecological systems. The early warning signals proposed here can potentially predict critical transitions earlier than established methods and perhaps even sufficiently early to avert catastrophic shifts.

Complex systems such as ecological systems show rich dynamical behaviors due to the diverse underlying interactions involving competition, exploitation, and mutualism (1). The nature of these dynamics and the stability of equilibria and other asymptotic characteristics are crucial for the fates of species, populations, and ecosystems (1–5).

Under the influence of climate change, land use, and human activities, the state of an ecosystem can change from one steady state to another. The complexities of the environmental pressure, nonlinearity, stochasticity, and other characteristics often lead to state transitions characterized by sudden changes or jumps. Accurately defining the tipping point or threshold point at which the state transition occurs is an important practical challenge. Similarly, it is of evident interest to capture changes in the ecosystem structure and features before critical tipping points as early warning signals (6–9).

Ecosystems may lose stability and have bifurcations or phase transitions between alternative locally stable states (6–12). To avoid such transition events, tipping point identification and other early warning signals are crucial for prevention and control (6–9). In some cases, ecosystem transitions may be desirable, such as when a system is trapped in an unfavorable state, and early warning signals are helpful as we seek to promote regime shifts. Critical slowing down has long been considered as one of the most significant early warning signal for certain classes of transitions (6–8). However, such prediction is often too close to the transition point and may not be early enough to allow an intervention to avert an undesirable transition. Indeed, in other situations, it may not apply due to the nature of the transition (13).

One example of the aforementioned critical transition scenario is when nutrient concentrations in a shallow lake ecosystem undergo significant changes. At the level of low nutrient concentration, the lake water is typically clear and transparent, and submerged plants are dominant, showing a clear lake state. At high levels of nutrient concentration, the phytoplankton in the water body is dominant, and water transparency decreases markedly, a so-called turbid state. Once the nutrient concentration in the lake water exceeds a certain threshold, the submerged plants will disappear, the algae will grow in large quantities, and the lake will change from the clear water state to the turbid water state. When the input of exogenous nutrient reaches a certain threshold, a small increment of nutrient can lead to major changes of the ecosystem structure. The lake suddenly changes from a clear water state to a turbid water state, and the ecosystem function is seriously reduced (9). Within a certain nutrient range, the clear water and turbid water states are bistable, with the possibility of noise-induced transitions between the two alternative stable states.

In this study, we apply the landscape-flux theory from nonequilibrium statistical mechanics to provide a general framework to quantify the global stability of the ecological systems and provide warning signals for critical transitions. For complex systems subject to random noise, the “driving force” of the system can be profitably decomposed into the gradient of the potential landscape and a rotational flux term. The gradient of the potential pushes the system toward the attractor, while the flux measures the degree of detailed-balance breaking and, being rotational in nature, tends to destabilize the point attractor (14–17). Therefore, the curl flux should play a significant role for the instability of the current state and the emergence of the new state, giving rise to the nonequilibrium phase transitions. Furthermore, the flux also provides the source for the associated nonequilibrium thermodynamic cost in terms of the entropy production. From this perspective, the flux gives rise to the dynamical origin while the entropy production gives rise to the thermodynamic origin for the phase transition. In this framework, the average flux plays the role of the nonequilibrium driving force, and the entropy production rate gives the nonequilibrium thermodynamic cost and the nonequilibrium free energy. Crucially, from an applied perspective, time irreversibility of the cross-correlation functions can serve as the warning signals for critical transitions between alternative stable states. We use a well-known shallow lake model to demonstrate the utility of this approach to predicting regime shifts, and additional results for a savanna-forest model are shown in SI Appendix.

It is worth mentioning that effective one-dimensional (1D) approach has attracted extensive attention for the study of the phase transitions/bifurcations in the ecological systems of finding the early warning signals (6–9, 18, 19). Critical slowing down refers to that a system has slowing down response to the perturbations as it approaches the tipping point, with often greater variance, autocorrelation, and return time (13, 20). Critical slowing down has been widely applied to the models with saddle-node bifurcations. Most of the studies have so far concentrated on the effective 1D approach, and the results can often be applied to the effective equilibrium systems where the global stability can be quantified by the landscape alone without considering the key nonequilibrium ingradient, the flux (6–9, 18, 19).

In the pure 1D systems, under natural boundary condition, there is no net flux. The system obeys detailed balance. Thus, such 1D dynamics is dictated by the landscape alone. However, the real ecological systems are often open, nonequilibrium, complex, and high-dimensional. The system may not always be easily treated as effective 1D system (18, 19). It is thus a great challenge to explore the global stability and the early warning signals for critical transitions. Furthermore, the critical slowing down is only one of the possible early warning signals, since regime shifts do not all exhibit critical slowing down.

The potential-flux landscape theory of nonequilibrium statistical mechanics we proposed here can provide and quantify non-equilibrium early warning signals for multidimensional system, two-dimensional (2D) system in this study. For a higher-dimensional system, the nonequilibrium characterized by the nonzero curl flux can lead to a much richer complex dynamics with detailed balance breaking while the equilibrium dynamics are solely determined by the gradient of the potential landscape. The curl flux breaking the detailed balance plays an important role in driving the nonequilibrium dynamics of the system.

Here, we propose the fully vectorized high-dimensional formulation of the potential landscape, in contrast to the almost ubiquitous focus on 1D landscapes throughout the ecological literature on critical transitions, early warning signals, and critical slowing down (6–9, 18, 19). Thus, our theory provides a method for exploring critical transitions in higher dimensions with both the rotational component of the flux and the gradient component of the potential landscape. For the multidimensional systems, the curl flux component of the driving force has not been considered as early warning indicators by the critical slowing down approach. The curl flux or the vectorized formulation of the potential landscape is usually absent in the 1D system (6–9, 18, 19) (under the natural boundary conditions). The contribution of the curl flux present often in multidimensional systems in addition to the potential landscape provides the basis for the emergence of the non-equilibrium early warning signals beyond the currently often used critical slowing down.

Methods

Model of Shallow Lake with Bream and Pike.

We explore a shallow lake with bream and pike (SLBP) model to illustrate our approach. Turbid water characterized by high algal biomass that is predominantly dominated by phytoplankton generally contains a large number of bream, while clear water generally contains relatively low populations of bream, some pike (which predate the bream), and well-developed aquatic vegetation. Observations show that the transparency of the water has a strong relationship with the ratio of pike to bream (10); the interactions among these two variables are shown via a schematic diagram in Fig. 1.

Fig. 1.

The schematic diagram for the SLBP model. Arrows with flat ends denote inhibition and arrows with cuspidal ends denote activation. Bream and pike are both also subject to natural mortality.

The SLBP system can be described by a set of nonlinear ordinary differential equations for these species interactions (10):

$$\begin{array}{cc}\hfill \frac{\mathit{dX}}{\mathit{dt}}& =i_b+rX\frac{N}{N+H_1}-c_b{X}^2-p_r{F}_{R}Y,\hfill \\ \hfill \frac{\mathit{dY}}{\mathit{dt}}& =i_p+c_e{p}_r{F}_{R}Y\frac{V}{V+H_2}-m_{p}Y-c_p{Y}^2,\hfill \end{array}$$ [1]

where X represents the bream population density, Y represents the pike population density, and N is the nutrient level. Table 1 summarizes the mathematical definitions and notations, while the interpretation of the parameters and their default values are given in Table 2. Increasing nutrient loading of shallow lakes can change the topology of the dynamics, leading the shallow lake system to critical transitions or to flickering between the states or the basins of attraction. In this study, we use the immigration rate of bream i_b = 3 × 10⁻⁴ and the immigration rate of pike i_p = 3 × 10⁻⁴ (which are larger than the original model) in order to avoid the steady-state solution of the associated Fokker–Planck equation becoming overcrowded at the boundary.

Table 1.

Mathematical variables

Symbol	Interpretation
x	System state
F(x)	Driving force
D	Scale factor representing the magnitude of the fluctuations
G	Diffusion matrix
P(x, t)	Probability of system state x at time t
J(x, t)	Probability flux
J s s	Probability flux of steady state
U(x)	Population potential landscape
ϕ 0	Intrinsic potential landscape
V	Intrinsic flux velocity
L(x)	Lagrangian
ℱ	Intrinsic free energy
𝒵	Partition function
e p	Entropy production rate
e p i n	Intrinsic entropy production rate
$\overline{J}$	The average flux
${\overline{J}}_{\mathit{in}}$	The intrinsic average flux
CXY(τ′)	The cross-correlation function forward in time
${\tilde{C}}_{\mathit{XY}}(\tau \prime )$	The cross-correlation function backward in time
Δ C	The average difference in cross-correlations between The forward in time and the backward in time
τ r e l a x	Relaxation time from autocorrelation
f ω	The frequency of the flickering

Table 2.

Parameters interpretation and default values (10)

Symbol	Ecological interpretation	Default value
i b	Immigration rate of bream	3 × 10−4
i p	Immigration rate of pike	3 × 10−4
r	Maximum growth rate of bream	7.5 × 10−3
H 1	Half saturation constant	0.5
H 2	Half saturation constant	0.1
H 3	Half saturation constant	20
H 4	Half saturation constant	15
c b	Intraspecific competition constant for bream	7.5 × 10−5
c p	Intraspecific competition constant for pike	2.75 × 10−4
p r	Maximum predation rate of pike	5 × 10−2
c e	Pike food conversion efficiency to growth	0.1
m p	Mortality rate of pike	2.25 × 10−3
K	Maximum vegetation coverage	100%
V	Percentage of lake covered with vegetation	$\frac{K\ast H_3^2}{H_3^2+X^2}$
F R	Functional response of pike	$\frac{X^2}{X^2+H_4^2}$

Landscape and Flux Theory for Nonequilibrium SLBP Model.

Stochastic fluctuations are common across a wide array of natural systems (16, 21). We can write the nonlinear dynamics subject to a randomly fluctuating environment as: $\dot{\mathbf{x}}=\mathbf{F}(\mathbf{x})+\zeta$, where F(x) represents the deterministic force, and the vector x denotes a state of the system, for example, x = {X, Y} in this study. ζ represents Gaussian fluctuating force, whose autocorrelation function is given as < ζ (x, t) ζ (x, 0) > = 2D(x)δ(t), where D(x) represents the diffusion coefficient matrix. Set D(x) = DG(x), where D is the diffusion coefficient representing the noise intensity and G is the scaled diffusion matrix describing the anisotropy. As the scales of X and Y are very different, G(x) is set as the anisotropic matrix with G₁₁ = 1, G₁₂ = G₂₁ = 0, G₂₂ = 0.1 throughout. The Langevin equations give the information on the stochastic trajectories, thus they are not predictable. However, the statistical patterns of the trajectories and the distributions are predictable and follow the Fokker–Planck diffusion equation (22–24).

The evolution of probability density function of the system, P(x, t), obeys the local conservation law: ∂P/∂t = −∇⋅J(x, t). The change of the probability in time is equal to the net flux J in or out. The probability flux J is defined as: J(x, t) = F(x) P(x, t)−D∇(GP (x, t)). The driving force for the dynamics can then be decomposed as: F = −DG ⋅ ∇U  +  J_ss/P_ss + D∇⋅G, where U = −lnP_ss is the nonequilibrium potential landscape which is related to the steady-state probability distribution (P_ss) and the steady-state probability flux J_ss.

The steady-state probability flux can then be either zero, constant, or rotational (having the curl nature at nonequilibrium steady-state ∇ ⋅ J_ss = 0). The equilibrium state has zero flux, i.e., it obeys the so-called detailed balance condition, with no net input or output. On the other hand, the nonzero flux denotes the net flow in or out of the system, while the magnitude of the flux measures the degree of the detailed balance breaking (away from the equilibrium).

Nonequilibrium open systems often exchange energy, materials, and information with their environments. The time evolution of the system entropy can be decomposed to the entropy production rate and heat dissipation rate as (15, 25–28): $\dot{S}={\dot{S}}_t-{\dot{S}}_e$. The population entropy production rate can be represented with $e_p={\dot{S}}_t=\int d\mathbf{x}(\mathbf{J}·{(D\mathbf{G})}^{-1}·\mathbf{J})/P$. And ${\dot{S}}_e=\int d\mathbf{x}(\mathbf{J}·{(D\mathbf{G})}^{-1}·\mathbf{F}\prime )$ denotes the heat dissipation rate of the environment. Thus, the entropy production rate can be seen as the total entropy change of the system and environment $e_p={\dot{S}}_t=\dot{S}+{\dot{S}}_e$. The entropy production rate must be nonnegative but the heat dissipation rate may be either positive or negative. This measure can quantify the entropy flow rate between the environment and the nonequilibrium system. At steady state, the entropy production rate and the heat dissipation rate are equal (15, 25–27). The entropy production rate provides a global thermodynamic characterization for the nonequilibrium system. We define the average magnitude of the flux as $\overline{J}=\int |\mathbf{J}|d\mathbf{x}$ to quantify how far a system is from the equilibrium.

Lyapunov Function for the SLBP Model Under Zero Fluctuations.

The steady-state probability and the related population potential can be used to explore the global stability under finite fluctuations. The population potential is not a Lyapunov function (29) and finding Lyapunov functions is often a challenging problem for complex nonequilibrium systems. To this end, we show that the intrinsic potential landscape ϕ₀ in the zero-noise limit is a Lyapunov function of ecological dynamics (26, 29).

The probability density function P is first expanded in a power series in the diffusion coefficient D (assuming fluctuation is relatively small) as follows: P(x) = exp(−(ϕ₀(x)/D + ϕ₁(x) + Dϕ₂(x)+⋯))/Z where Z = ∫ exp(−U(x))dx. By substituting this expansion into the Fokker–Planck equation, we obtain the D⁻¹ order expansion of the Fokker–Planck equation. This is called the Hamilton–Jacobi equation (HJE) and is given by:

$$\begin{array}{c}\hfill H=\mathbf{F}·\mathrm{\nabla }{\varphi }_0+\mathrm{\nabla }{\varphi }_0·\mathbf{G}·\mathrm{\nabla }{\varphi }_0=0.\end{array}$$ [2]

Differentiation of ϕ₀ with respect to time then reveals that ϕ₀(x) obeys the differential equation ${\dot{\varphi }}_0(\mathbf{x})=\dot{\mathbf{x}}·\mathrm{\nabla }{\varphi }_0=\mathbf{F}·\mathrm{\nabla }{\varphi }_0$. Combining this equation with the HJE shows that ${\dot{\varphi }}_0(\mathbf{x})=-\mathrm{\nabla }{\varphi }_0·\mathbf{G}·\mathrm{\nabla }{\varphi }_0\le 0$ and hence ϕ₀(x) monotonically decreases along the deterministic trajectories under the zero fluctuation limit if G is positive definite. Therefore, ϕ₀ is a Lyapunov function for the system. ϕ₀ is then referred to the intrinsic potential of the system (26, 29).

The force F can be decomposed into a gradient term and a curl term in the zero fluctuation limit as: F = −G ⋅ ∇ϕ₀ + (J_ss/P_ss)|_{D → 0} = −G ⋅ ∇ϕ₀ + V, where −G ⋅ ∇ϕ₀ is the gradient of the nonequilibrium intrinsic potential. V = (J_ss/P_ss)_{D → 0} is called the intrinsic steady-state flux velocity. J_ss|_{D → 0} is the steady-state intrinsic divergence free curl flux (since ∇ ⋅ V = 0). The relationship between ϕ₀ and the intrinsic flux is described by the relation (J_ss/P_ss)|_{D → 0} ⋅ ∇ϕ₀ = V ⋅ ∇ϕ₀ = 0, highlighting that the gradient of the intrinsic potential and the intrinsic flux are perpendicular to each other in the zero fluctuation limit (26, 29). The average magnitude of the intrinsic flux ${\overline{J}}_{\mathit{in}}$ is defined as ${\overline{J}}_{\mathit{in}}=\int |\mathbf{V}|exp(-{\varphi }_0(\mathbf{x}))d\mathbf{x}$ to quantify how far a system is from the equilibrium. The intrinsic entropy production rate is defined as e_pin = ∫V ⋅ (DG)⁻¹ ⋅ V exp(−ϕ₀(x))dx.

Nonequilibrium Thermodynamics: Entropy, Energy, and Free Energy of General Dynamical Systems.

The intrinsic potential ϕ₀ in the nonequilibrium systems can be related to the steady-state probability distribution under the zero-fluctuation limit: 𝒫_ss(x)=P_ss(x)|_{D → 0} = exp(−ϕ₀/𝒟)/𝒵, where 𝒟 = D|_{D → 0}. The partition function is defined as 𝒵 = ∫exp(−ϕ₀/𝒟)dx. Therefore, ϕ₀ = −𝒟ln(𝒵𝒫_ss). The entropy of the nonequilibrium system under the zero-fluctuation limit is defined as: 𝒮 = −∫𝒫(x, t)ln𝒫(x, t)dx. The intrinsic energy is defined as: ℰ = ∫ϕ₀𝒫(x, t)dx = −𝒟∫ln(𝒵𝒫_ss)𝒫(x, t)dx. Thus, we define the intrinsic free energy as ℱ = ℰ − 𝒟𝒮 = 𝒟(∫𝒫ln(𝒫/𝒫_ss)dx−ln𝒵).

It can be shown that

$$\begin{array}{cc}\hfill \frac{d\mathcal{F}}{\mathit{dt}}& =-{\mathcal{D}}^2(\int [\mathrm{\nabla }ln(\frac{\mathcal{P}}{{\mathcal{P}}_{\mathit{ss}}})·\mathbf{G}·\mathrm{\nabla }ln(\frac{\mathcal{P}}{{\mathcal{P}}_{\mathit{ss}}})\left]\mathcal{P}d\mathbf{x}\right)\le 0,\hfill \end{array}$$ [3]

and hence the nonequilibrium intrinsic free energy ℱ always decreases and is a Lyapunov function for the system. It also follows from Eq. 3 that the minimum value of ℱ is ℱ = −𝒟ln𝒵.

Kinetic Speed and Dominant Paths Between the Clear State and the Turbid State.

We use the path-integral approach to identify transitions between stable states. The probability of the path from initial state x_i at t = 0 to final state x_f at time t is given by $P({\mathbf{x}}_f,t|{\mathbf{x}}_i,0)=\int D\mathbf{x}exp[-{\int }_0^{t}dt(\frac{1}{2}\mathrm{\nabla }·\mathbf{F}(\mathbf{x})+\frac{1}{4}(d\mathbf{x}/dt-\mathbf{F}(\mathbf{x}))·{(D\mathbf{G})}^{-1}·(d\mathbf{x}/dt-\mathbf{F}(\mathbf{x})))]=\int D\mathbf{x}exp[-A(\mathbf{x})]=\int D\mathbf{x}exp[-{\int }_0^{t}L(\mathbf{x}(t))dt]$, where L(x(t)) is the Lagrangian and A(x) is the action for each path on the potential landscapes (29, 30). The Dx term gives a weighted sum over all possible paths connecting x_i at time zero to x_f at time t. The dominant paths with the optimal weights can thus be found by minimization of the action A(x) or the Lagrangian L(x(t)).

The Cross-Correlation Function.

Simulated time series data from the SLBP model exhibits noise-induced switching between the high and the low pike or the bream level, mimicking observations from real ecological systems (12, 31). Practical early warning signals based on the landscape-flux potential theory demand that we quantify the nonequilibrium nature of the system based on such time series data and hence we define the cross-correlation function as follows: C_XY(τ′) = ⟨X(0)Y(τ′)⟩ where X and Y denote the time trajectories of variables X and Y with time interval τ′ (32, 33). C_XY(τ′) represents the cross-correlation function forward in time and ${\tilde{C}}_{\mathit{XY}}(\tau \prime )$ represents the cross-correlation function backward in time. The average difference in cross-correlations between the forward in time and the backward in time, defined as $\mathrm{\Delta }C=\sqrt{\frac{1}{t_f}{\int }_0^{t_f}{(C_{\mathit{XY}}(\tau \prime )-{\tilde{C}}_{\mathit{XY}}(\tau \prime ))}^{2}d\tau \prime }$, can be used as a quantification of the time irreversibility of the system and the degree to which it is out of equilibrium.

Results

Shallow lakes can have two alternative stable states: a Clear state dominated by aquatic vegetation, and a Turbid state dominated by high algal biomass (34). Vegetation tends to improve water transparency, while high turbidity, on the other hand, prevents underwater plant growth. Adverse effects of turbidity on vegetation growth are due to light limitation. We now explore the shallow lake bream-pike model (SLBP model) (10) under both finite and zero fluctuations using the landscape-flux approach. Often state changes occur dramatically with the system remaining unchanged for a long time before a sudden transition. Shallow lakes can shift to alternative stable states; such changes in the system state are often termed critical transitions or regime shifts and have been studied extensively (9, 34).

Fig. 2A shows the deterministic phase (bifurcation) diagram vs. the nutrient level, N. There are two stable states, Clear and Turbid, for a significant range of N, with saddle-node bifurcations starting and ending the bistable regime. We solve the Fokker–Planck equation of the corresponding stochastic SLBP model to obtain the steady-state probability distribution and thus the population landscape through: U = −lnP_ss. Fig. 2B shows the three-dimensional population-potential landscapes (U) under finite fluctuations. The population-potential landscape initially has one stable state that evolves from the Clear state with lower nutrient loading level to the Turbid state upon increasing the nutrient loading level. As N increases, the stable Turbid state emerges. As N increases even further, the shallow lake system switches from the Clear state with pike domination to the Turbid state with bream domination; and eventually, the pikes almost disappear at a sufficiently high nutrient loading level. In the end, the Turbid state becomes dominant while the Clear state disappears. SI Appendix, Fig. S1 shows the 2D intrinsic potential landscape ϕ₀ for the shallow lake model under the zero fluctuation limit for another view in contrast to the three-dimensional (3D) figure. We can see that the intrinsic potential landscape ϕ₀ has almost the same tendency as the population-potential landscapes (U) upon increasing the nutrient level.

Fig. 2.

(A) The phase diagram vs. N. (B) The quantitative population potential landscape U for the shallow lake model. (C) The quantitative one-dimensional population potential landscape U by the projection vs. X with integral Y at certain increasing N. (D) The continuous population potential landscape U vs. the N and X with integral Y.

For real ecosystems, disturbances and stochastic fluctuations are almost inevitable. When the system has two alternative states, even if the external environmental conditions remain unchanged, when the intensity of the disturbance is large enough, it can drive the state of the system from a local stable basin to go across the unstable equilibrium state and fall into another stable state basin and exhibit regime shifting behavior. This steady-state transition can be intuitively described by the ubiquitous “ball in the valley” conceptual model (Fig. 2 C and D): the ball is at the bottom of the valley, which characterizes the basin of attraction in the dynamical system, indicating that the system is in a steady state. When the system is subjected to small fluctuations, it can deviate from the steady state and the ball is at the hillside location, but the system can return to the steady state. Thus the ball returns to the valley bottom location under small fluctuations. When the fluctuations are strong enough, the ball may go across a ridge, which is an unstable saddle point, into an adjacent alternative stable valley. Thus, the system falls into another stable state basin (6–9).

Fig. 2C shows the population potential landscape U projected on X with increasing N. Fig. 2D shows the continuous population potential landscape U vs. the N and X with integral Y. The quantitatively accurate Fig. 2C is qualitatively similar to the schematic diagram known as the marble-in-a-cup model of ecosystem stability (34, 35). Fig. 2C is a potential landscape quantified by the probability distribution of the stochastic SLBP model.

There may be two different internal mechanisms for the emergence of the steady-state transition. One is that an exogenous disturbance (random noise in our model) causes the system to go over an unstable saddle point as shown in Fig. 2C, with the potential landscape qualitatively unchanged for different values of the nutrient level N. This mechanism is also referred to as noise-induced attractor switching or N-tipping in the literature (36). The other steady-state transition mechanism involves changes in the external environmental conditions (changes to the nutrient level) that lead to the loss of the internal stability of the system, i.e., one of the steady states disappears completely in a saddle-node bifurcation. In this case, the potential landscape qualitatively changes, which is mainly manifested in the reduction of the attraction domain, as shown in Fig. 2 B and D. The potential landscape characterizes and also helps to visually represent the basins of attraction of the SLBP system.

In Fig. 2C, the states of the system are indicated by the positions of the red ball on the blue terrace which are quantified from the probability distribution. The two basins in one subfigure indicate the size of the domain of attraction, with larger and deeper domains showing higher stability of the system, and shallower and narrower domains indicating lower stability of the system. The depth of each basin quantifies the potential required for the system to leave the current basin of attraction and go over the threshold given by the unstable saddle point to reach the alternative stable state. To achieve this transition, systems with higher stability require more cost (i.e., larger disturbances) than the ones with lower stability. In Fig. 2D, the continuous population potential landscape U follows the phase diagram lines; as the basin of the Clear state becomes shallower, the Turbid state emerges and its basin of attraction deepens upon N increase. The ball (state of the system) can then shift between these two stable states as the N level changes.

The landscape topography represented by the barrier height between the two stable states can be used as a quantitative measure of ecosystem stability. Fig. 3A shows the barrier heights of the population-potential landscape vs. N under finite fluctuations. ΔU_SC = U_S − U_C represents the barrier height from Clear to Turbid and ΔU_ST = U_S − U_T represents the barrier height from Turbid to Clear, where U_S is the value of population potential U at the saddle point between Turbid state and Clear state, U_C is the minimum value of population potential at the Clear state, and U_T is the minimum value of population potential at the Turbid state. When nutrient level increases, the population barrier height ΔU_ST increases while the population barrier height ΔU_SC decreases. At lower nutrient level, the Clear state is much more stable than the Turbid state. Near N = 1.5 at the intersection, the Clear state and Turbid state have the same depth of their basins of attraction showing the equal chance of appearance of these two states. As the nutrient level increases further, the stabilities of the two states shift. Thus, the Clear state becomes less stable while the Turbid state becomes much more stable. We can also use the initial appearance of the Turbid state, characterized by the variations of ΔU_ST, as an early warning signal for a critical transition. This can be significantly earlier than the nutrient level at which the Turbid state becomes dominant. However, the probability of the system actually occupying the Turbid state at this stage is vanishingly small, making its emergence difficult to capture for traditional early warning indicators.

Fig. 3.

(A) The population barrier heights vs. parameter N. (B) The logarithm of MFPT vs. N. (C) The logarithm of MFPT vs. barrier heights for N. (D) The dominant population paths and fluxes on the population-potential landscape U with N = 1.5. The purple lines represent the dominant population paths from the Turbid state to Clear state. The black lines represent the dominant population paths from the Clear state to Turbid state. The white arrows represent the steady-state probability fluxes while the black arrows represent the negative gradient of U.

The escape time from one local stable state to another can also be used as a quantitative measure for global stability. One can estimate the mean exit time from the basin of attraction (15, 29, 37, 38). We can then obtain the mean first passage time (MFPT) for the escape shown as ref. (22): F ⋅ ∇τ + D∇²τ = −1. Here, F represents the driving force of the system, while D represents the diffusion coefficient measuring the fluctuation level. We assume that τ_CT represents the MFPT from the Clear state to the Turbid state, while τ_TC represents the MFPT from the Turbid state to the Clear state. Fig. 3B shows the logarithm of MFPT vs. N. We found that lnτ_TC increases while lnτ_CT decreases as N increases. Fig. 3C shows that the logarithm of MFPT has a positive correlation with its corresponding barrier height ΔU, which reveals the close relationship between the MFPT and the corresponding barrier height. In particular, we have τ ∼ e^−ΔU. As the barrier height becomes higher, it is harder to switch from the original state to the other state. Therefore, both the barrier height and the escape time provide the landscape topography and quantitative measures of the global stability of the ecosystem.

Fig. 3D shows the fluxes (white arrows) and the negative gradient of the population potential landscape (black arrows) on the population-potential landscapes for N = 1.5. The fluxes supported by the nutrient level going around the stable states can increase the communications between the two stable states, Clear and Turbid. We also show the dominant paths on the population potential landscape U under finite fluctuations. The purple line denotes the dominant path from the Turbid state to the Clear state, while the black line represents the dominant path from the Clear state to the Turbid state. The white arrows guide the dominant paths on the population-potential landscape under finite fluctuations. Due to the rotational nature of the nonequilibrium driving force, the forward and backward paths follow different routes and are therefore irreversible. The spiral shapes of the fluxes around the basins provide the origin for the nature of nonequilibrium dynamics. This can generate instability and create possible new states.

Fig. 4A shows the phase diagram for the SLBP model again for comparison purposes. Fig. 4B shows the population entropy production rate e_p (black squares) and the population average flux $\overline{J}$ (red circles) vs. N. As the N increases, both the $\overline{J}$ and e_p increase first and then decrease, undergoing significant changes near the left deterministic saddle-node bifurcation at the transition N = 1.34 (purple dashed lines). This is the nutrient level at which Clear state is dominant and the Turbid state first appears. The right deterministic saddle-node bifurcation is near N = 3.038 (cyan dashed lines) in Fig. 4B, where the Turbid state is dominant and the Clear state disappears. The sharp peaks of the $\overline{J}$ and the e_p are distinct near the transition N = 1.34 at the initial appearance of the Turbid state. In fact, the nonzero rotational flux breaks the detailed balance and provides a direct quantitative measurement of the degree to which the system is out of equilibrium (14–17, 39, 40). Thus, the nonequilibrium influences on the dynamics are dictated by the flux. While the gradient of the landscape tends to maintain the stability of the points attractors, the flux (being purely rotational) tends to destabilize the point attractor. This provides a dynamical origin for the instability of the current attractor and the possible bifurcation and phase transition. The flux is also closely related to the entropy production rate and thus gives rise to the thermodynamic cost for realizing the nonequilibrium bifurcation and phase transition. Therefore, we expect that the rotational flux characterizing the “nonequilibriumness” of the system plays the key role in the emergence of nonequilibrium states, bifurcations, and phase transitions. This suggests that rotational flux can provide the dynamic source for bifurcation or phase transition, while entropy production gives rise to the thermodynamic source for bifurcation or phase transition in nonequilibrium systems. Thus, the origin of instability and new state formation can be identified, and the onset of the critical transition can potentially be predicted.

Fig. 4.

(A) The phase diagram vs. N. (B) The population entropy production rate (black squares), the population average flux (red circles), the average change of the forward and backward in time cross-correlation function (blue triangles) and the free energy (green stars) vs. N. The purple dashed line is at N = 1.34, while the cyan dashed line is at N = 3.038. (C) The relaxation time of the autocorrelations for state Clear (τ_relaxC) (blue squares) vs. N. The frequency of the flickering from Clear to Turbid (f_ωCT) (red circles) vs. N.

According to the critical slowing down theory, under gradual changes in the external environmental conditions, the ecosystem will show critical slowing when it approaches the right bifurcation point (Fig. 2A); this is characterized by a smaller area of attraction and flatter landscape, as shown in Fig. 2C (Left attractor). The Clear state basin of attraction with N = 1.6 has a smaller width and lower depth. The left Clear basins in Fig. 2C with N = 1.7 or N = 3.0 are even smaller and shallower, both of which are markers of critical slowing down. At this time, the “resilience” of the system decreases, and the risk of the system switching to another steady state increases (9). Thus, the critical slowing down has been suggested as a way to capture the switching at tipping points for a saddle-node bifurcation, which can be characterized by the slower recovery from the perturbations, increasing autocorrelation and increased variance (9). After a system in equilibrium is subjected to an external transient disturbance, it will return to its original equilibrium state after a certain period of time, called the relaxation time; this relaxation time is the time it takes for the system to adapt itself to the changes in the environment. We show the relaxation time from autocorrelation in Fig. 4C (blue squares). We can clearly see that the relaxation time has a sharp increase near the right bifurcation N = 3.038 with small fluctuations. Upon external perturbation, it takes much longer time to return to its original equilibrium state near the right bifurcation N = 3.038.

The flickering frequency is the number of state transitions per unit time. For example, f_ωCT represents the number of transitions from the Clear state to the Turbid state per unit time. We also show the frequency of the flickering from Clear to Turbid (f_ωCT) in Fig. 4C (red circles) with fluctuation strength D = 1 × 10⁻⁴. We can see that the frequency of the flickering f_ωCT increases sharply as the N increases. This can be understood since the basin of attraction of the state Clear becomes shallower and less stable. The system then has a larger chance or frequency to flicker to the state Turbid. The frequency of the flickering is a well-known early warning signal from previous studies (8, 41). The tipping points derived from both the critical slowing down and the frequency of the flickering are near the bifurcation for the SLBP model when the clear water state becomes flat (unstable) and in the mean time, the turbid water state becomes dominant (9, 10). Critical slowing down is an indicator that the Clear state is losing its resilience and the potential landscape of Clear state has a small basin of attraction, while the Turbid state becomes dominant. However, the real transition can occur far from this bifurcation point due to larger fluctuations and the critical slowing down near this bifurcation may not begin early enough to give a practically useful warning of the impending regime shift. We propose to use the $\overline{J}$ and e_p to quantify the driving force of the nonequilibrium dynamics and thermodynamic costs and thereby serve as earlier warning signals than those employed in the extant literature.

The flux and the entropy production are not easy to directly extract from experimentally obtained real-time data. For practical purposes, one can study the time irreversibility of the time series to quantify the “nonequilibriumness” of the system, reflecting the degree of the flux and the entropy production. The biomass time series can be obtained from the observation in the experiments, for example, the time series of the bream population or the pike population. The data simulated from the Langevin equation shown in SI Appendix, Fig. S2A mimics the experimentally obtained real-time series. We show the long-time trajectories of X and Y in the stochastic SLBP model with the noise-induced attractor switching between Clear and Turbid.

SI Appendix, Fig. S2B shows the two-point cross-correlations forward and backward in time and highlights significant differences between C_XY(τ′) and ${\tilde{C}}_{\mathit{XY}}(\tau \prime )$. The differences between forward and backward in time cross-correlation are shown directly in SI Appendix, Fig. S2C, which confirms the time irreversibility of the underlying system.

The average difference in cross-correlations between the forward in time and the backward in time ΔC is shown in Fig. 4B (blue triangles) for our SLBP model, which can be used as a quantification of the time irreversibility. C_XY(τ′) and ${\tilde{C}}_{\mathit{XY}}(\tau \prime )$ are equal to each other with zero flux, corresponding to the time reversibility of the system at equilibrium (33). Since time irreversibility is directly related to the degree of the detailed balance breaking (32, 33, 42), the cross-correlation difference measures the degree of nonequilibriumness and also reflects the strength of the flux. However, the flux is not easy to quantify directly from the experiments. Therefore, the difference in cross-correlations forward and backward in time among the experimentally observed real-time traces of the observables provides a practical way to quantify the nonequilibrium driving force of the system. ΔC increases significantly near the left saddle-node bifurcation regime. A steep fall in ΔC is noticeable near the left bifurcation at N = 1.34 (purple dashed line), where the Turbid state initially appears. This provides a practical method from the observed temporal trajectory of the shallow lake system to give earlier warning signals at the beginning of the appearance of the Turbid state at N = 1.34, which is far before the bifurcation representing the dominance of the Turbid state at N = 3.038.

Fig. 4B (green stars) shows the nonequilibrium intrinsic free energy ℱ vs. N. Although the free energy is continuous, the slope of the free energy changes significantly near the transition point around N = 1.34 (purple dashed line). Thus, the intrinsic free energy can also serve as an early warning signal for potential regime shift, far before bifurcation for the dominance of the Turbid state at N = 3.038 (cyan dashed line).

The critical slowing down theory generally applies to continuous second-order phase transitions, but is less suitable for exploring discontinuous or first-order phase transitions. Here, the nonequilibrium driving force in terms of the flux, the thermodynamic cost in terms of the e_p, the intrinsic free energy, and the time irreversibility all give rise to early warning signals which are far earlier than the predictions from currently available methods. Therefore, they can provide a feasible method for predicting bifurcations/phase transition to avoid catastrophic regime shifts. In the SLBP model, we study here, we predict the switch from the regime in which the Clear state is dominant to that in which the Turbid state dominates at the initial appearance of the Turbid state, rather than near the N level where the Turbid state is becoming dominant. This leads to much earlier warning signals than previous investigations have uncovered (6–8).

Critical slowing down can identify the second bifurcation where the “bad” (turbid) attractor state for the ecological system becomes dominant and the “good” (current clear water) attractor state for the ecological system becomes flat (the right saddle node bifurcation), while it will miss the first bifurcation where the “good” (current) attractor state is dominant (the left saddle node bifurcation in the phase diagram) and the “bad” attractor state has just appeared and is shallow. This is because critical slowing down emerges only when the landscape around the “good” (current) attractor state becomes flat. When the second bifurcation emerges where the “bad” attractor state dominates, the “good” (current) attractor state where the system currently resides flattens, leading to the appearance of critical slowing down. However, when the “good” (current) state is near the first bifurcation point, it becomes dominant against others, but the associated landscape around it is not flat at all. Therefore, one does not anticipate any critical slowing down around this point. Thus, the critical slowing down cannot be used to predict the first bifurcation when the “good” (current) attractor state is dominant. The non-equilibrium landscape-flux warning signals which appear near the first bifurcation are much earlier than the critical slowing down indicators emerging near second bifurcation where the “bad” attractor becomes dominant and the basin of attraction of the “good” steady state flattens.

Our results for early warning signals apply to multidimensional dynamical systems, but we now consider a 1D dynamical system to illustrate how our approach performs in this special case. For 1D dynamical system with natural boundary conditions, detailed balance is preserved and the steady-state probability flux is zero. Thus the system is in equilibrium and the entropy production rate is also zero. In this case, the time series is reversible in time and the nonequilibrium indicators we employed previously are no longer effective. One must then rely on the other indicators of an impending regime shift, such as critical slow down or flickering.

In the SLBP model, if one assumes that Y changes at a time scale much faster than that of X, so that it quickly reaches the steady state, the system dynamics are effective 1D along the slow observable X. In other words, in the limit of total separation of time scales between the two dynamic variables, we may set dY/dt = 0 so that we can solve for Y as a function of X. The effective 1D steady-state flux is then J(x)=J(X, Y)=J(X, f(X)). Thus, the effective 1D population entropy production is given by:

$$\begin{array}{cc}\hfill e_{p1d}(X,Y)& =e_{p1d}(X,f(X))\hfill \\ \hfill & =\int dX(\mathbf{J}(X,f(X))·{(D\mathbf{G})}^{-1}·\mathbf{J}(X,f(X)))/P_{\mathit{ss}}.\hfill \end{array}$$

The effective 1D average flux is given by:

$$\begin{array}{cc}\hfill {\overline{J}}_{1d}(X,Y)& ={\overline{J}}_{1d}(X,f(X))\hfill \\ \hfill & =\int dX\int dY|\mathbf{J}(X,Y)|=\int dX|\mathbf{J}(X,f(X))|.\hfill \end{array}$$

The normalized entropy production rate e_p1d and ${\overline{J}}_{1d}$ average flux for the SLBP model are shown in Fig. 5A. We can see that both the e_p1d and ${\overline{J}}_{1d}$ have the peaks near the left bifurcation. They are both consistent with the predictions in 2D case. This is likely due to the fact that every Y = f(X) line at each parameter N in the XY space always passes through the maximum location of the steady-state probability (equivalent to the minimum of the basins potential on the landscapes) since the maximum of the probability is around dX/dt = 0 and dY/dt = 0. When N is around N = 1.5, the Y = f(X) passes through two local maximum of the probability (dX/dt = 0 and dY/dt = 0 have two stable solutions). Near N = 1.5 with these two equal basins, the landscape gradient is close to zero and thus does not contribute to the flux. The flux then can be approximated as J = F(X, Y = f(X))P_ss. The flux is therefore directly related to the steady-state probability at around N = 1.5 near these two potential basins or probability peaks. The steady-state probability peaks lead to the flux peaks and then the peaks of the e_p1d and ${\overline{J}}_{1d}$ at around N = 1.5. Therefore, we can see that the predictions of early warning indicators are powerful and adaptive for both high-dimensional and effective 1D dynamical systems, which is better than the critical slow down prediction at around N = 3.038.

Fig. 5.

(A) The effective one-dimensional population production rate e_p1d and the effective one dimensional average flux ${\overline{J}}_{1d}$ vs. N. (B) The effective one-dimensional three-point correlation function C₃ vs. N.

The cross-correlation can only be calculated from the trajectories involving two different observables, but for an effective 1D case, this is clearly no longer possible. Thus, we use the time irreversibility of the three-point auto-correlation function C₃ = < X(τ)X(τ′)X(τ″)> to reflect the degree of detailed balance breaking in the system. The three-point auto-correlation function C₃ has a peak around N = 1 in Fig. 5B, indicating that the time irreversibility of the effective 1D three-point auto-correlation function can also be used as an early warning signal indicator which is beyond the current critical slow down prediction near the bifurcation at N = 3.038. Therefore, even for an effectively 1D system, indicators based on the landscape-flux potential theory provide robust early warning signals of regime shifts.

We also consider an extreme case when the fluctuations are negligible and can be neglected. We show the intrinsic barrier heights, the intrinsic ${\overline{J}}_{\mathit{in}}$ and intrinsic e_pin (SI Appendix, Fig. S3), and intrinsic paths on potential landscape (SI Appendix, Fig. S4) under zero fluctuation limit for the deterministic dynamics in SI Appendix, which have almost the same tendencies with the results under finite fluctuations. We show the relaxation time from autocorrelation and flickering frequency in (SI Appendix, Fig. S5). We also discuss the hysteresis loop in shallow lake in (SI Appendix, Fig. S6). We have also shown another well-known forest-savanna model as an example of illustration for our landscape and flux theory in providing the early warning signals and predicting the tipping points of the ecological systems in (SI Appendix, Figs. S7 and S8).

Conclusion and Discussion

Stability and dynamics are crucial to understanding the structure and function of ecological systems. In this study, we use a landscape and flux approach from statistical physics to explore the global stability and quantification of ecological systems, using a shallow-water lake model as an illustrative example. This approach and the associated methods can be used to explore general complex ecological systems, both from the modeling and data-driven perspectives. The dynamics of ecological systems subject to random forcing are determined by both the gradient of the potential landscape and the rotational flux (which breaks the detailed balance condition for equilibrium systems when it is nonzero). We uncover both the population potential landscapes under finite fluctuations and the intrinsic potential landscapes under zero fluctuations limit, which can be used to quantify the stability and robustness of ecological systems.

We considered a simple well-known shallow lake ecological model with two species, bream and pike. Two alternative stable states, the Clear and Turbid states, emerge in this model. We explored several crucial physical measurements to quantify the stability and robustness of the shallow lake system. Barrier heights (as the potential difference) quantify the difficulty of state switching, giving rise to the landscape topography measure of the global stability. Mean first passage time, the characterization of escape time from one stable to the alternative one, quantifies the length of time for the state switching giving rise to the kinetic measure of the global stability.

In Fig. 4B, we observe sharp changes in several key physical quantities related to the degree to which the system is out of equilibrium around the left saddle-node bifurcation (where the Turbid state emerges). The average flux represents the nonequilibrium driving force, while the entropy production rate represents the thermodynamic cost. This indicates that flux and entropy production rate can identify the dynamic and thermodynamic sources for the bifurcation of the shallow lake ecosystem, respectively. A generalized free energy can provide a global quantitative description for the nonequilibrium system in analogy to the equilibrium system. These measurements can all be used to give quantitative early warning signals for the critical phase transitions from the dominance of the Clear state to the dominance of the Turbid state for the shallow lake ecological system. The average differences between the cross-correlation forward and backward in time from the time series of the system quantifies the time irreversibility of the system. It can be used as a practical early warning signal directly from the observational real-time traces for the initial appearance of the Turbid state at left saddle-node bifurcation (N = 1.34), far earlier than the appearance of the dominance of the Turbid state at the right saddle-node bifurcation (N = 3.038). This can provide a primary practical marker for predicting the bifurcation of the system directly from the experimental time series observations. Given the types and quality of data typically available for ecological systems, it is less clear how certain theoretical indicators highlighted above, such as the entropy production rate, can be readily applied in real-world scenarios. However, different concepts of entropy have been introduced to study transitions in biological applications, such as in gene expression data (43, 44), and it may be possible to apply similar ideas in the present context if sufficiently resolved data are available.

Understanding dimensionality, and where dimensionality reduction is possible, is crucial for exploring the mechanism of critical transition (45). Our potential-flux landscape theory is broadly suitable for higher-dimensional systems and avoids some pitfalls of methods that require effectively 1D dynamics for best results. We can obtain the steady-state probability by directly solving the Fokker–Planck equation for 2D or 3D systems (16, 46) and also use the self-consistent mean field method to solve the Fokker–Planck equation or use the stochastic Langevin equations to obtain the steady-state probability for even higher-dimensional systems (16, 46). It is also possible to quantify the pathways, the MFPT, and the indicators of the early warning signals for addressing the stability and dynamics for high-dimensional systems, which appear to be more powerful for the ecological systems considered here than the early warning signal indicators from the effective 1D approaches (greater variance, autocorrelation, and relaxation time). Although we only considered models with saddle-node bifurcations in this work, the potential-flux landscape theory has been used to study other types of bifurcations in other applied fields; for example, pitchfork bifurcations in multilocus evolution (47) and Hopf bifurcations in game theory and evolution (14, 26). It has been shown that slowing down of recovery rates can occur only very close to a threshold for Hopf bifurcations (48), further motivating the investigation of the non-equilibrium early warning indicators proposed here for applications to ecological systems with cyclic dynamics.

The dominant paths that are calculated by the path integral method do not follow the expected steepest descent gradient path as in equilibrium system. The forward and backward dominant paths separate from each other and their irreversibility is due to the nonzero flux. These results can help us to design more stable ecological systems. It may also help to design effective strategies or interventions for lake management. The observed critical transitions and bifurcations have been empirically observed repeatedly in lakes, savanna and forest, climate, or ocean circulation (1, 8). The landscape and flux theory can potentially be used to predict the tipping points and transitions in many complex systems much earlier beyond the previous studies (6–9). Further research on early warning signals can not only improve the existing theory but also provide practical and effective guidance for the protection and management of complex and socially valued ecosystems.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

All study data are included in the article and/or SI Appendix.