High-order correlations in species interactions lead to complex diversity-stability relationships for ecosystems

Abstract

How ecosystems maintain stability is an active area of research. Inspired by applications of random matrix theory in nuclear physics, May showed decades ago that in an ecosystem model with many randomly interacting species, increasing species diversity decreases the stability of the ecosystem. There have since been many additions to May’s efforts, one being an improved understanding the effect of mutualistic, competitive, or predator-prey-like correlations between pairs of species. Here we extend a random matrix technique developed in the context of spin-glass theory to study the effect of high-order correlations among species interactions. The resulting analytically solvable models include next-to-nearest-neighbor correlations in the species interaction network, such as the enemy of my enemy is my friend, as well as higher-order correlations. We find qualitative differences from May and others’ models, including nonmonotonic diversity-stability relationships. Furthermore, inclusion of particular next-to-nearest-neighbor correlations in predator-prey as opposed to mutualist-competitive networks causes the former to transition to being more stable at higher species diversity. We discuss potential applicability of our results to microbiota engineering and to the ecology of interpredator interactions, such as cub predation between lions and hyenas as well as companionship between humans and dogs.

I. INTRODUCTION AND BACKGROUND

Many ecosystems are composed of a large number of species that interact with each other in a plethora of ways. The extreme case of the Amazon rainforest is estimated to house several million species, ranging from carnivorous plants [1] to freshwater sharks [2]. Our own gut microbiome contains hundreds of microbial species that coevolve with us [3–5]. The complexity of such ecosystems makes it challenging to model and understand them [6].

A celebrated attempt to understand large ecosystems was made by May in the early 1970s [7–9], which we detail here. Consider an ecosystem with N species, each with continuous, time-dependent abundance z_i(t) where i = 1,2, …, N, which represent the elements of the abundance vector $\overrightarrow{z}$. The species interact with each other, which determines how their abundances change over time. Assume that the dynamics of z_i(t) are governed by a system of coupled ordinary differential equations admitting a steady-state vector of species abundances $\overrightarrow{z}=\overrightarrow{s}$, around which the system can be linearized. Thus, the introduction of small changes in species abundances around the steady state, denoted x_i(t) = z_i(t) − s_i with corresponding vector $\overrightarrow{x}$, will dictate the dynamics according to

$$\frac{d\overrightarrow{x}}{dt}=M\overrightarrow{x},$$ (1)

where M is the Jacobian matrix evaluated at $\overrightarrow{s}$. Following precedent, we will refer to this matrix as the community matrix [8] where element m_ij of M encodes an interaction between the species pair j and i.

May considered a community matrix model of the form M₁ = G − R. Here −R is a diagonal matrix representing the self-regulation of the species, which for convenience is taken to be −R = −RI where I is the identity matrix and R is a positive constant. G is a random matrix with independent and identically distributed entries: The entry distribution has mean 0 and finite standard deviation σ and is otherwise arbitrary. The parameter σ sets the range of the interaction strength between species pairs and we will simply refer to it as the interaction strength [10]. Note that M₁ is not symmetric; the effect of species i on j will differ from that of j on i. May focused on the local stability (hereafter referred to as stability) of the steady-state species abundance of this ecosystem model. In general, small perturbations of abundances around a stable steady state are suppressed and the system returns back to the steady state, while those around an unstable one lead to divergence from the steady state. The stability can simply be determined from the eigenvalues of M₁: If none of its eigenvalues have a real part greater than 0, then the steady state is stable. Thus, one can address the question of stability in May’s model through the distribution of eigenvalues of M₁.

According to the circular law conjecture [11,12], in the limit of large N, the eigenvalues of G are uniformly distributed on the disk of radius $\sigma \sqrt{N}$ centered at the origin in the complex plane, with the largest real part of its eigenvalues consequently being $\sigma \sqrt{N}$. The circular law conjecture has been proven in its most general form only a decade ago by Tao, Vu, and Krishnapur [13], who provide a universal result agnostic to the details of the matrix entry distribution. Thus, a simple change of variables in the eigenvalue distribution for G leads to $\sigma \sqrt{N}-R$ for the largest real eigenvalue part of M₁. This implies that the ecosystem is stable when $\sigma \sqrt{N}<R$. One way to incorporate the fact that not all species pairs necessarily interact with each other is to introduce a connectivity fraction C, which randomly sets a fraction C of the entries of G to 0. This simply modifies the inequality to $\sigma \sqrt{NC}<R$, bringing us to the particular diversity – stabilityrelationship proposed by May [7], which emphasizes a trade-off among σ, N, and C within this ecosystem model. Thinking of the species interaction strength σ as a control parameter, we can define a criticalinteractionstrength σ_c after which the ecosystem is always unstable [10]. For May’s model, ${\sigma }_c=\frac{R}{\sqrt{NC}}$ [see Fig. 1(a) where connectivity C is dropped following rest of the manuscript; see Results]. It is important to note that this analysis does not say anything about “feasibility,” i.e., having physically allowable (positive) values for the abundances in the steady-state vector around which the system is linearized. The study of feasibility requires consideration of the original system before linearization, which the models in this manuscript will be mostly agnostic to. Relatedly, the models discussed in this manuscript, including those by May [7] and Ref. [14], do not take into account any effect of the steady-state vector of abundances on the Jacobian matrix at steady state and hence any effect of it on stability [8]. It is interesting to contrast this to the case of simple generalized Lotka-Volterra models, where species interactions are typically linear with respect to species abundances [8,15] and the steady-state Jacobian matrix accordingly has an explicit dependence on the steady-state abundances. We refer interested readers to a fraction of many revealing pieces of work on different models regarding feasibility and stability by Refs. [16–20].

FIG. 1.

May’s model and three-species correlations. (a) The eigenvalues of May’s community matrix M₁ are uniformly distributed in a disk of radius $\sigma \sqrt{N}$ centered at the point (−R, 0) on the real axis in the complex plane. We have dropped the connectivity fraction C in May’s original model for the remainder of the manuscript. As drawn, the system is stable. When σ > σ_c (purple radius larger than orange radius), the largest real eigenvalue component is larger than 0, and the ecosystem is unstable. (b) The interspecies interactions encoded in G can be represented in a weighted directed graph (left panel) where each node represents a species and each directed link weight equals the interaction effect of one species on another. Here only one of the two links (one direction) between species pairs is shown. Left panel shows an arbitrary subset of the three-species paths from species i to k. In the right panel, h_ik is the modification to the interaction effect of k on i introduced when going from May’s model M₁ to M_2,1.

While the assumptions underlying May’s approach have been questioned [8,9,17,21–23], it provides an interesting null model and has inspired further investigations using random matrix approaches. An avenue of interest in ecology is the study of the effects of correlations among species interactions, which is assumed to not exist in May’s model. This is potentially one way nature avoids May’s diversity-stability constraint, with species interactions in different ecosystems having empirically different correlations [21,24–26]. A common example of correlations in an ecosystem can be seen in ecosystems rich in predator-prey (also known as exploitive interaction) pairs, where it is expected to find more pairs of species that directly affect each other in an opposite manner, that is, through an interaction beneficial for one but detrimental to the other. Beyond that, it is possible that there are prevalent higher-order correlations in the direct interactions of species, such as those between the common predators of a prey, though this is not well understood at the ecosystem level, potentially due to difficulties in experimental ecology (see Discussion).

An important case of correlations in ecosystems was addressed by Allesina and Tang [14] in a May-like model where diagonally symmetric elements of the community matrix were set to have pairwise correlations by drawing the pair of elements from a general bivariate distribution with correlation ρ. Through theory and simulations, they showed that the functional form of May’s diversity-stability relationship undergoes simple modifications in the presence of such correlations. For example, a perfect predator-prey limit of the community matrix (ρ chosen such that m_ijm_ji < 0 with marginal means of interactions 0) was shown to increase the critical interaction strength σ_c to $\frac{R}{\sqrt{NC}(1-|\rho |)}$ compared to the $\frac{R}{\sqrt{NC}}$ in May’s model. Empirical evidence suggests that correlations contribute to stability in more ways than described in Ref. [14], but the theoretical basis is not clear [24]. There have indeed been other computational and theoretical advances on the effect of correlations among species interactions [26–31]. However, the topic is far from exhausted and enjoys only a small collection of analytical results [21]. More generally, the addition of complexity to models of large numbers of interacting species has proven challenging but is critical to understand and make predictions regarding complex ecosystems [3,5,6].

While random matrix theory has helped achieve milestones in the modeling of ecological systems [9,10,14,17,20,26], it has also been extensively applied in finance [32], neural information processing [33], and throughout physics [34–39]. One application includes the introduction of a “squared interaction matrix” to study the effect of next-to-nearest-neighbor correlations in a fully connected spin-glass model in a simple, analytically tractable manner [40]. Here we extend this approach to study three-species (next-to-nearest-neighbor) and higher-order correlations in direct interspecies interactions using a particular set of correlation structures, which are easily generated by incorporating random matrices raised to integer powers in our models. For clarity, we would like to note that we do not study indirect interactions [41–43]. Indirect interactions arise, for example, when species A directly interacts with species B and alters the abundance of species B, which then leads to species A indirectly affecting the other species, C, that is directly interacting with species B (“linked chains of direct interactions”) [42]. We also note that the high-order correlations we study are distinct from what are defined as high-order “interactions,” where there is a direct interaction between two species that is governed by a third species through, for example, its abundance [10,44–46] (this at times has been referred to as an indirect interaction as well [42]; for a rigorous discussion of high-order interactions, see Ref. [47]). In contrast, we instead study a system of linearized differential equations, with high-order correlations among terms corresponding to the direct pairwise interactions of species. We show that with one correlation structure, the addition of high-order correlations beyond three species to May’s model (such as the friend of the enemy of my enemy helps me) can cause the critical interaction strength to vary nonmonotonically with species diversity. Thus the maximum critical interaction strength is achieved at finite values of species diversity, unlike in May’s model. We then show that for a related correlation structure, predator-prey networks transition to being more stable than mutualist-competitive networks at higher species diversity. Overall, the correlation structures in our linear models give rise to analytically tractable, complex diversity-stability relationships. We discuss how our results may fit into broader ecological and evolutionary frameworks.

II. RESULTS

We build on May’s model [7], where we drop the connectivity fraction C, which gives the diversity-stability relationship $\sigma \sqrt{N}<R$. Following literature, we refer to σ as the interaction strength and focus on the critical interaction strength σ_c above which the ecosystem is always unstable for given N [10]. In May’s model, ${\sigma }_c=\frac{R}{\sqrt{N}}$ where the critical interaction strength is seen to decrease linearly with increasing $\sqrt{N}$ [Fig. 1(a)]. Throughout the manuscript, we analyze modifications to May’s model, and keep working in the large-N limit. Finally, we refer to entries of G in May’s model as pairwise interactions between species unless we need to distinguish g_ij from g_ji. In this case, we refer to g_ij as the interaction effect of species j on i.

A. Positive n-species correlations

Now, instead of May’s model M₁ = −R + G, we first analyze a modified ecosystem with community matrix

$$M_{2,\beta }=-R+G+\frac{\beta }{N}{G}^2,$$ (2)

where, again, each matrix is of dimensions N × N with N being species diversity. β is a positive constant (0 < β ⩽ 1) determining the level of mixing of the rightmost term with the rest of the terms. We will be looking at the effect of increasing β, which we will show to correspond to increasing next-to-nearest-neighbor correlations in M_2,β. Note that for a given instantiation of the random community matrix M_2,β, G² is simply the square of the random matrix G whose entries are already drawn. First, let us focus on the rightmost term, $\frac{\beta }{N}{G}^2$. The introduction of $+\frac{\beta }{N}{G}^2$ implies a particular next-to-nearest-neighbor correlation structure in the ecosystem. Each element h_ik of the matrix $\frac{1}{N}{G}^2$ by definition equals to $\frac{1}{N}{\sum }_j{g}_{ij}{g}_{jk}$ where g_ijg_jk is the product of the interaction effect of species j on i (in G) and the interaction effect of species k on j (in G). Thus the element h_ik of $\frac{1}{N}{G}^2$ represents the average of all pairwise interaction effects (in G) of k on other species, each weighted by the interaction effect (in G) of those species on species i. N corresponds to the number of three-species paths between the two species i and k [Fig. 1(b)]. Finally, note that the operator norm of $\frac{1}{N}{G}^2$ is $\frac{\sigma }{\sqrt{N}}$ fold that of G. Dean and Ritort, in a similar vein, introduced GG^T (instead of G²) to the respective Hamiltonian in the squared interaction matrix Sherrington-Kirkpatrick model of spin glasses to study another form of next-to-nearest-neighbor correlations [40].

Coming back to the community matrix $M_{2,\beta }=-R+G+\frac{\beta }{N}{G}^2$, we see that increasing β decreases the critical interaction strength σ_c at which the ecosystem destabilizes [Fig. 2(a)]. This is not surprising since we are simply increasing the largest real eigenvalue component of May’s community matrix by βσ². This is seen from a fact that we will be repeatedly using throughout the manuscript: In general, if v is an eigenvalue of G, and χ is a polynomial, then χ(v) is an eigenvalue of χ(G) (see the Appendix). In this case, ${\sigma }_c=\frac{1}{2\beta }(\sqrt{4\beta R+N}-\sqrt{N})$ which for large N behaves like the critical interaction strength of May’s model, $\frac{R}{\sqrt{N}}$. Now let us set β = 1, M₂ ≡ M_2,1 and consider the inclusion of higher-order correlations in May’s model, that is, longer species paths when modifying the direct interspecies interactions. Instead of only including three-species correlations, we will include up to (F+1)-species correlations with the analogous structure, which gives the community matrix

$$M_F=-R+G+\sum _{p=2}^F\frac{1}{N^{p-1}}{G}^p.$$ (3)

FIG. 2.

Nonmonotonic stability relationships arise from beyond three-species correlations. (a) Increasing β decreases the critical interaction strength σ_c of community matrix M_2,β. As β approaches 0, σ_c for May’s community matrix M₁ is recovered (dashed blue). (b) The critical interaction strengths of M₂ and May’s community matrix M₁ monotonically decrease, while the community matrices including beyond three-species correlations have a nonmonotonic relationship between their critical interaction strength and species diversity. The inset shows the same plot with different scaling of the axes. R is set to 10⁶ for all curves.

The term G^p takes into account all possible (p+1)-species paths between a pair of species, which total to N^p−1 terms. In the case of M₃ we find that σ_c no longer monotonically decreases with N, as opposed to the case of M₂, and it peaks at a nonzero species diversity [Fig. 2(b); see the Appendix]. This is interesting in part due to its qualitative difference from May’s model where the highest possible critical interaction strength occurs at species diversity N = 0.

We should note that the observed nonmonotonicity in σ_c in the case of F = 3 is, as expected, due to the emergence of a high integer power of σ, with a cooperating factor of N, in the equations to solve for the largest eigenvalue real component and thus is not particularly special to the inclusion of correlations (see the Appendix 1). However, it is possible that the correlation structure we use is a natural way in which those terms can arise (see Discussion). In any case, we see that this approach allows for relatively easy analytical treatments of random matrix models with high-order correlations among the direct interspecies interactions.

We now look at the community matrix containing up to even higher-order correlations, where we take lim F → ∞ in M_F. If we assume ${\sigma }_c<\sqrt{N}$, then, using the Taylor series for (1 − x)⁻¹, we easily find ${\sigma }_c=\frac{\sqrt{N}R}{N+R}$ (see the Appendix). This limit exhibits a broader peak for σ_c than in the M₃ case [Fig. 2(b)]. One may be further interested in modifying the community matrix M_F, such as with certain prefactors to the terms. A simple case is when we penalize each $\frac{1}{N^{p-1}}{G}^p$ term (p!)⁻¹ fold, resulting in the community matrix $M_E=-R+G+{\sum }_{p=2}^F\frac{1}{N^{p-1}}\frac{G^p}{p!}$. Here, using the Taylor series for the matrix exponential of G, it is straightforward to find ${\sigma }_c=\sqrt{N}\text{ log}\left(1+\frac{R}{N}\right)$ for M_E [Fig. 2(b); see the Appendix].

B. Negative three-species correlations in predator-prey, mutualistic, and competitive networks

We move on to analyze a different correlation structure. First, instead of May’s community matrix M₁ = −R + G, following Allesina and Tang [14], we start with a more general community matrix, M_c = −R + G_c. The N × N matrix G_c has pairwise correlation ρ among its diagonally symmetric entries g_ij and g_ji (i ≠ j). In the construction of G_c, its entries g_ij and g_ji (i ≠ j) are sampled from a general bivariate distribution with correlation −1 < ρ < 1, marginal means 0 and marginal standard deviation (interaction strength) σ. When ρ < 0, M_c is biased toward having more predatorprey-like (g_ijg_ji < 0) interactions. When ρ > 0, M_c is instead biased toward having more mutualistic (g_ij > 0; g_ji > 0) and competitive (g_ij < 0; g_ji < 0) interactions which we call mutualist-competitive biased. The case of ρ = 0 reduces to May’s model.

To investigate the stability of the ecosystem governed by the community matrix M_c, we need to determine its largest real eigenvalue components. This was done by Allesina and Tang [14] with the help of preceding theoretical results [48]. The eigenvalues of G_c were shown to be distributed uniformly inside an ellipse centered at the origin in the complex plane, where the ellipse intercepts the real axis at $\pm \sigma \sqrt{N}(1+\rho )$ and the imaginary axis at $\pm \sigma \sqrt{N}(1-\rho )$. The stability criterion for M_c is thus $\sigma \sqrt{N}(1+\rho )<R$ and the critical interaction strength ${\sigma }_c=\frac{R}{\sqrt{N}(1+\rho )}$. This analysis reveals a simple relationship where predator-prey biased interactions provide more stability than mutualist-competitive biased interactions.

We will now modify M_c to introduce a different form of three-species correlations than we studied before, which is the addition of $-\frac{1}{N}{G}_c^2$ to M_c where we note the choice of the minus sign. This choice is partially due to the rich stability behavior it will be shown to generate. We will thus focus on the stability of the ecosystem governed by the community matrix

$$M_{c,-2}=-R+G_c-\frac{1}{N}{G}_c^2.$$ (4)

Following a similar procedure to the previous section leads to a piecewise analytical solution for the critical interaction strength σ_c for M_c,−2 (Fig. 3; see the Appendix). An important feature of this solution is that as the species diversity decreases, including more positive as opposed to more negative pairwise correlation ρ in G_c will transition from decreasing σ_c to increasing σ_c [Fig. 3(b)]. This is as opposed to M_c where decreasing ρ increases σ_c for all N [Fig. 3(a)].

FIG. 3.

Mutualist-competitive and predator-prey correlations shape effect of a three-species correlation structure. (a) Critical interaction strength σ_c for community matrix M_c decreases with increasing ρ for all N. (b) σ_c for community matrix M_c,−2 is described by a piecewise function, with the two pieces joining at the respective threshold species diversity for the given curve, marked with a black dot. For M_c,−2, increasing ρ increases σ_c for small N while it decreases σ_c for large N. R is set to 3000 for all curves.

III. DISCUSSION

A. Summary

In this manuscript, we modified May’s random community matrix model [7] with simple high-order correlations among species interactions by using random matrices raised to integer powers. We showed in Sec. IIA that high-order correlations can lead, in a number of ways, to maximum possible critical interaction strengths occurring at finite values of species diversity. One implication of this is that, unlike May’s model, systems with fewer species can be more unstable. A similar effect has also been observed when high-order species interactions, instead of correlations, were included in May-like models [10]. In Sec. IIB, we studied a three-species correlation structure which resulted in predator-prey networks transitioning to being more stable than mutualist-competitive networks at higher species diversity. Our work adds to the limited number of analytical studies that involve application of random matrices with high-order correlations among their elements [14,40,49].

B. Motivating the models

It is interesting to think about how high-order correlations in species interactions may arise in nature. Lions, hyenas, and cheetahs, who compete for many prey species, actively kill each others’ cubs, suggestive of similarities with high-order (three-species) negative correlations as in Sec. IIA. To illustrate how parts of our models could potentially map to such biological settings, we provide a few speculative examples. Let us focus on a hypothetical scenario inspired by the interaction between lions and hyenas and show one specific way increased interspecies aggression may come about in an ecosystem where high-order correlations are developing in a manner consistent with our models. What we will effectively be doing is restating the mathematics of our model in words. The species names are mostly symbolic for our purposes. Interspecies aggression is known to have evolved between lions and hyenas. Let us assume that they exist in an ecosystem with some “initial” community matrix Q (where we will ignore the self-regulation term), and that lions eat prey A (q_LA > 0; q_AL < 0 where L is the index for lion, A is the index for prey A, and q is an element of the matrix Q) and hyenas also eat prey A (q_HA > 0; q_AH < 0 where H is the index for hyenas). The lions and hyenas also exhibit some initial but small level of direct interaction, such that q_HL and q_LH are approximately zero.

We are interested in high-order correlations in an ecological network. High-order correlations could in theory come about in numerous ways—for example, new interactions may simultaneously emerge through evolution and change predator-prey and predator-predator interactions altogether to new “values” that are unrelated to the pre-evolved values of the interactions that lacked high-order correlations. Or high-order correlations may have already been part of the ecosystem from its original formation and could continue remaining within an ecosystem, as the ecosystem evolves.

However, perhaps an alternative attractive possibility is that ecological networks vary in their level of interspecies interaction correlation over time, and correlations arise due to previous (pre-evolved) interactions feeding into how the new interactions evolve. So in the lion and hyena example, one way this could happen is if q_LA and q_AH feed into the evolved k_LH (where k is an element of the newly evolved matrix K) in this new, evolved ecological network and likewise for q_HA, q_AL, and k_HL. A simple model would be to take the new k_LH to be proportional to q_LA and q_AH, which gives k_LH ~ q_LAq_AH—that is, the interaction of each with their common prey feeds into the interaction they are developing, and it does so in a directly proportional manner. If there are multiple common prey species (prey A, B, C, and D), then a simple model would be to take the average of q_LAq_AH, q_LBq_BH, q_LCq_CH, q_LDq_DH. At the same time, k_HL would also evolve in an analogous manner, in addition to interactions of other pairs of species. Thus, this reasoning shows how one can arrive to a term similar to the addition of $\frac{1}{N}{Q}^2$ in our random matrix models [Fig. 1(b)]. If the mechanism described here for the arising of correlations repeats over time as the ecosystem (and consequently its community matrix) evolves further, then higher powers of Q than 2 will also arise. This can be seen through the following logic: Say the evolved ecosystem with correlations as above has the community matrix αQ + βQ² (where α, β are arbitrary constants), as opposed to the original starting ecosystem with interaction matrix αQ (the addition of βQ² is responsible for the already evolved correlations). If this evolved ecosystem with community matrix αQ+ βQ² undergoes another “step” of evolution to generate correlations (just like the starting interaction matrix did), then the new community matrix in this second step of evolution will include a term proportional to (αQ+ βQ²)² and thus will include higher powers of Q, namely Q³ and Q⁴. Mechanistically, when considering the evolution of interspecies aggression among predators, a high-order correlation, it is interesting to consider the historical selection pressure of indirect interactions (e.g., depletion of the common prey population by the other predator) and high-order interactions (e.g., driven by physical encounters of the common prey, hyena and lion). Similar arguments can be made for potential prey species that share a common predator and need to compete for sheltering territory (a high-order interaction), which may be expected to later correlate with evolution of interspecies aggression among the prey species. However, whether and how our models may apply to the full variety of species interactions in large ecosystems is an interesting avenue for future investigation.

We believe the “mappings” in this section are extremely speculative, despite being mathematically interesting. The hypothetical lion-hyena example helps motivate the correlation structure in Sec. IIA of our manuscript, and in particular the positive sign in front of G² as well as the integer powers of our matrices. Overall, it would be interesting to study the interspecies aggression behavior of other higher predators that occur independent of the abundance of their common prey species and look for patterns within ecosystems across many predators for high-order negative correlations in species interactions and their effects on ecological stability (see patterns discussed in Ref. [50]).

On the other hand, ancient humans and ancient dogs who shared many prey species likely hunted together: a high-order interaction, as opposed to a high-order correlation in their direct interaction, since it depends on presence of another species. However, they later seem to have developed a direct interaction of companionship independent of their shared prey; such companionship among predators could be an interesting angle to investigate high-order positive correlations as in Sec. IIB. Thus, in order to motivate the matrix structure in Sec. IIB, we focus on another hypothetical scenario inspired by the interaction of ancient humans and dogs, which again is meant to be mostly symbolic. We will take rabbits as their common prey. Let us assume ancient humans fed on rabbits (q_HR > 0, q_RH < 0) and dogs fed on rabbits (q_DR > 0, q_RD < 0). Then, cooperative hunting evolved (a high-order interaction, not correlation: humans hunted rabbits more efficiently in the presence of dogs). However, humans then over time also evolved a direct companionship with dogs, where even in the absence of rabbits, humans provided shelter for dogs. Here, if we take the same approach as the lion and hyena model to calculate the newly evolved k_HD, then we see that q_HRq_RD would be negative. Thus, for a direct favorable interaction between humans and dogs, we take k_HD to be proportional to −q_HRq_RD. This motivates the negative sign in front of G² in Sec. IIB.

In this hypothetical ancient human-dog example, we reasoned that the evolution of high-order interactions (e.g., cooperative hunting) could lead to evolution of high-order correlations in direct interactions. We are not aware of any conclusive evidence for this being a realistic mechanism, but it seems like an attractive possibility.

C. Future avenues

We dropped the connectivity fraction from May’s model, effectively setting it to 1, since it becomes more subtle in our framework—the connectivity in general increases as we raise our matrices to different powers, which introduces interactions among otherwise noninteracting (disconnected) species. A future avenue for investigation would thus be the implications of different levels as well as patterns of connectivity in our community matrices and the matrices that are used to construct them. Furthermore, it would be interesting to study the introduction of matrices with high-order correlations as community matrices into generalized Lotka-Volterra models. Likewise, comparing our matrices with steady-state Jacobian matrices from different models of high-order interactions should also be informative.

Importantly, as mentioned, future investigations are required to understand whether and how our models make sense when the full variety of species interactions in a large ecosystem are considered. We also acknowledge that even if high-order correlations do arise in a qualitatively similar fashion to our models, the appropriate quantitative description for them may still be different. For example, the proportionality we describe above as well as the way we do our averaging with N (for example, the $\frac{1}{N}$ factor in front of G²) are, after all, particular choices in our model that made sense to us [see Fig. 1(b) and description in Sec. IIA]. Additionally, while it seems likely that high-order correlations among species interactions do exist in some form in nature, it is of course unknown whether high-order correlations in practice even produce any of the various ecological stability behaviors that we predict. So our model is a very speculative one. Additionally, our models are phenomenological in nature, just like many other ecological models, and thus mechanistic studies (e.g., with evolutionary game theory) are required to check if they at all make sense in the context of evolution.

Although our models can perhaps be modified in the future, in their current form they involve introducing high-order correlations throughout the ecological network for all species, instead of for a few select group of species. In this light, it is interesting to consider the case of Amazonian birds, where many species of birds sharing similar resources, as opposed to just a few pairs, seem to have developed interspecies aggression [51] where they fight, for example, over territory. We think these kinds of settings with many species being involved in interspecies aggression could overall be fitting starting points to study the effect of high-order correlations on stability.

An important orthogonal avenue for consideration for biological relevance of our results is when whole ecosystem properties are under effective selective pressure. Typically, each species within an ecosystem is considered to be individually under selection. However, additional effective selection pressures on whole ecosystems can potentially be the case in microbiota attached to hosts (where the microbiota are taken to be ecosystems on their own): Here, selective pressures may perhaps act on not only individual microbial species but also on the microbial ecosystem as a whole through outputs of the ecosystem [5]. Thus, for example, if stability of the microbial ecosystem is a favorable aspect to the host, then our frameworks in Secs. IIA and IIB show recipes for achieving such favorable regimes that could thus emerge due to selection because of the advantage provided to the host, whatever the mechanistic basis. However, this argument would require evolutionary considerations, such as evolutionary game theory, to test its merits.

Perhaps a particularly important use for our framework could be in reverse, that is, to utilize the forms of correlation structures we propose to achieve stability in engineered ecosystems. While this seems out of reach with current technology, microbiota engineering is a rapidly developing field [52]. It seems reasonable to think that having stable microbiota in the human body could be a desirable feature. As bioengineering progresses, perhaps an improved version of the framework we propose could be utilized to help determine what modifications to make in human microbiota to make it more stable. Our framework is potentially attractive in this sense because it gives a recipe for modifying the stability of the ecosystem by introducing the same operation to each species interaction within the ecosystem, instead of having to specifically target few strongor weak species or perhaps having to know details of the species. It is, however, unclear to us what physical form (e.g., details of what should be included in a potential plasmid library) such a perturbation would take.

Overall, like many ecological models, ours are phenomenological and await consolidation and complementation via input from metabolics, genetics, as well as branches of evolutionary theory [5,53–55]. Understanding finite-size effects in our models is another avenue for future investigation [56,57]. We hope that as experimental challenges in studying species interactions are resolved [6,43,58,59], for example by using microbial communities as models of large ecosystems [5,60–65], our theoretical predictions will be rigorously tested.

APPENDIX

1. Squared interaction matrix Sherrington-Kirkpatrick model

The Sherrington-Kirkpatrick spin-glass model is an Ising model where infinite-ranged random interactions couple the spins in the absence of an external field [66]. The random spin-spin interaction matrix J (the Hessian of the Ising Hamiltonian) has entries that are independent and identically Gaussian distributed. In order to study next-to-nearest-neighbor correlations, Dean and Ritort [40] replaced the interaction matrix in the Sherrington-Kirkpatrick model with a “squared interaction matrix,” namely JJ^T.

2. Section IIA details

When we refer to the eigenvalue distribution of a finite-sized random matrix, we consider the approximation of the ensemble average eigenvalue distribution from many instantiations of the matrix. For large N [13], the largest eigenvalue real part in the eigenvalue distribution of $M_{2,\beta }=-R+G+\frac{\beta }{N}{G}^2$, where G is defined as in the manuscript, can be found as follows. Note that the last two terms share the same set of eigenvectors, which are also eigenvectors of R = RI where I is the identity matrix. In general, if v is an eigenvalue of G and χ is a polynomial, then χ(v) is an eigenvalue of χ(G). Thus the largest eigenvalue real part in the eigenvalue distribution of M_2,β approaches $-R+\sigma \sqrt{N}+\beta {\sigma }^2$ for large N. To find the critical interaction strength σ_c above which the ecosystem becomes unstable, we equate this limiting largest eigenvalue real part to 0 and solve for σ_c, which gives ${\sigma }_c=\frac{1}{2\beta }(\sqrt{4\beta R+N}-\sqrt{N})$ for M_2,β.

We will approach the community matrix

$$M_F=-R+G+\sum _{p=2}^F\frac{1}{N^{p-1}}{G}^p$$ (A1)

similarly. For F = 3, that is, M₃, the largest eigenvalue real part approaches $-R+\sigma \sqrt{N}+{\sigma }^2+\frac{{\sigma }^3}{\sqrt{N}}$ for large N. Equating this to 0 and solving for the critical interaction strength gives

$${\sigma }_c=\frac{1}{6}\sqrt{N}[2^{2/3}\sqrt[3]{\frac{27R}{N}+\sqrt{{\left(\frac{27R}{N}+7\right)}^2+32}+7}-\frac{4\sqrt[3]{2}}{\sqrt[3]{\frac{27R}{N}+\sqrt{{\left(\frac{27R}{N}+7\right)}^2+32}+7}}-2].$$ (A2)

In the case of lim F → ∞, we rewrite M_F as $-R-NI+N{\sum }_{p=0}^F\frac{1}{N^p}{G}^p$. Thus, using the Taylor series $\frac{1}{1-x}={\text{lim}}_{F\to \infty }{\sum }_{p=0}^F{x}^p$ (convergent for −1 < x < 1), we get the approached largest eigenvalue real part for lim_F→∞ M_F as $-R-N+N{\left(1-\frac{\sigma }{N}\sqrt{N}\right)}^{-1}$ for large N as long as $\sigma <\sqrt{N}$. Under the condition $\sigma <\sqrt{N}$, we can solve for when the largest eigenvalue real part, for large N, is equal to 0 which gives ${\sigma }_c=\frac{\sqrt{N}R}{N+R}$.

In the case of the community matrix $M_E=-R+G+{\sum }_{p=2}^F\frac{1}{N^{p-1}}\frac{G^p}{p!}$, we can rearrange to get $M_E=-R-NI+N{\sum }_{p=0}^F\frac{1}{N^p}\frac{G^p}{p!}$. The last term is recognized as simply N times the matrix exponential of G, which leads to the largest eigenvalue real part approaching $-R-N+N\text{ exp}\left(\frac{{\sigma }_c}{N}\sqrt{N}\right)$ for large N. Thus, we can solve $-R-N+N\text{ exp}\left(\frac{{\sigma }_c}{N}\sqrt{N}\right)=0$ to get ${\sigma }_c=\sqrt{N}\text{ log}\left(1+\frac{R}{N}\right)$.

3. Section IIB details

As described in Appendix 2, we again consider how points map to each other in the eigenvalue distributions. The eigenvalues of G_c lie inside the ellipse ${(\frac{x}{1+\rho })}^2+{(\frac{y}{1-\rho })}^2=N{\sigma }^2$ in the complex plane for large N. We can calculate real part of eigenvalues of $M_{c,-2}=-R+G_c-\frac{1}{N}{G}_c^2$ as follows. The eigenvalue x + iy in the eigenvalue distribution of G_c will map to $\Psi =-R+x+iy-\frac{1}{N}{(x+iy)}^2=-R+x+iy-\frac{1}{N}{x}^2+\frac{1}{N}{y}^2-\frac{2}{N}ixy$, the corresponding eigenvalue of M_c,−2. At the ellipse boundary of the approached eigenvalue distribution of G_c for large N, $y=\pm (1-\rho )\sqrt{N{\sigma }^2-{(\frac{x}{1+\rho })}^2}$ and the real part of Ψ becomes $-R+x-\frac{1}{N}{x}^2+\frac{1}{N}{[(1-\rho )\sqrt{N{\sigma }^2-{(\frac{x}{1+\rho })}^2}]}^2=-R+{(1-\rho )}^2{\sigma }^2+x-[1+\frac{{(1-\rho )}^2}{{(1+\rho )}^2}]\frac{x^2}{N}$. This is maximized when $1-[1+\frac{{(1-\rho )}^2}{{(1+\rho )}^2}]\frac{2}{N}x=0$ giving $x_{\text{max}}=\frac{N}{2(1+\frac{{(1-\rho )}^2}{{(1+\rho )}^2})}$. However, the range of possible x is restricted to $-(1+\rho )\sigma \sqrt{N}⩽x⩽(1+\rho )\sigma \sqrt{N}$, so when $x_{\text{max}}>(1+\rho )\sigma \sqrt{N}$, that is, $\frac{1}{(1+\rho )}\frac{\sqrt{N}}{2(1+\frac{{(1-\rho )}^2}{{(1+\rho )}^2})}>\sigma$, the maximization within the range occurs at $(1+\rho )\sigma \sqrt{N}$.

The largest eigenvalue real part, for large N, thus approaches

$$m_1=-R+{(1-\rho )}^2{\sigma }^2+\frac{\gamma N}{2}$$

for $\sigma ⩾\frac{1}{(1+\rho )}\gamma \sqrt{N}$, and

$$m_2=-R+{(1-\rho )}^2{\sigma }^2+(1+\rho )\sigma \sqrt{N}-\frac{1}{2\gamma }{(1+\rho )}^2{\sigma }^2$$

for $\sigma <\frac{1}{(1+\rho )}\gamma \sqrt{N}$ where $\gamma =\frac{1}{2[1+\frac{{(1-\rho )}^2}{{(1+\rho )}^2}]}$.

Thus, when $\sigma ⩾\frac{1}{(1+\rho )}\gamma \sqrt{N}$, the ecosystem is stable under the condition m ⩽ 0. This gives a critical interaction strength ${\sigma }_c=\sqrt{{(1-\rho )}^{-2}(R-\frac{\gamma N}{2})}=\frac{1}{2}\sqrt{\frac{-N+4R}{{(\rho -1)}^2}+\frac{N}{2(1+{\rho }^2)}}$ down to the threshold species diversity N = N_T where $\sqrt{{(1-p)}^{-2}(R-\frac{\gamma N_T}{2})}=\frac{1}{(1+\rho )}\gamma \sqrt{N_T}$. Rearranging, we find $N_T={[\frac{\gamma }{2}+\frac{{(1-\rho )}^2}{{(1+\rho )}^2}{\gamma }^2]}^{-1}R=\frac{16R{({\rho }^2+1)}^2}{{(\rho +1)}^2(\rho (3\rho -2)+3)}$.

When $\sigma <\frac{1}{(1+\rho )}\gamma \sqrt{N}$, the ecosystem is stable as long as m₂ ⩽ 0. which gives strength, we start with solving m₂ = 0, which gives the curves $\sigma ={\omega }_{\pm }=\frac{1}{2({(1-\rho )}^2-\frac{1}{2\gamma }{(1+\rho )}^2)}\{-[(1+\rho )\sqrt{N}]\pm \sqrt{{(1+\rho )}^{2}N+4R[{(1-\rho )}^2-\frac{1}{2\gamma }{(1+\rho )}^2]}\}=\frac{\sqrt{N}\mp \sqrt{(N-4R)}}{2(\rho +1)}$. Note that when σ = ω₋, $\sigma <\frac{1}{(1+\rho )}\gamma \sqrt{N}$ is not satisfied. Thus in this regime, σ_c = ω₊.

To summarize, after rearranging and simplifications, the final piecewise function for the critical interaction strength σ_c is

$$\begin{gathered} \frac{1}{2}\sqrt{\frac{4R-N}{{(\rho -1)}^2}+\frac{N}{2\left(1+{\rho }^2\right)}},N⩽N_T \\ \frac{\sqrt{\text{N}}-\sqrt{(\text{N}-4R)}}{2(\rho +1)},N>N_T \end{gathered}$$

for −1 < ρ < 1.

4. Software

Analytical results were found by hand, using Wolfram Mathematica version 12 and WolframAlpha search engine. Plots were made using Matlab version R2019a and further labeled in Microsoft PowerPoint. Figure 1 was partially made in BioRender (biorender.com).