How similar can co-occurring species be in the presence of competition and ecological drift?

Abstract

If two species live on a single resource, the one with a slight advantage will out-compete the other: complete competitors cannot coexist. This is known as the competitive exclusion principle. If no extinction occurs, it is because evolutionary adaptation to slightly different niches takes place. Therefore, it is widely accepted that ecological communities are assembled by evolutionary differentiation and progressive adaptation of species to different niches. However, some ecologists have recently challenged this classic paradigm highlighting the importance of chance and stochasticity. Using a synthetic framework for community dynamics, here we show that, while deterministic descriptors predict coexistence, species similarity is limited in a more restrictive way in the presence of stochasticity. We analyse the stochastic extinction phenomenon, showing that extinction occurs as competitive overlap increases above a certain threshold well below its deterministic counterpart. We also prove that the extinction threshold cannot be ascribed only to demographic fluctuations around small population sizes. The more restrictive limit to species similarity is, therefore, a consequence of the complex interplay between competitive interactions and ecological drift. As a practical implication, we show that the existence of a stochastic limit to similarity has important consequences in the recovery of fragmented habitats.

1. Introduction

The main goal of community ecology is to understand the underlying forces that determine the identity and number of species and their relative abundances in any given set of geographical locations across space and time. Ecological communities result from a number of processes occurring at different spatio-temporal scales. New species arise via speciation and immigration. Species abundances are shaped by drift and selection, as well as ongoing dispersal. Therefore, selection, speciation, dispersal and ecological drift (understood as changes in discrete species abundances caused by the stochastic processes at play) can be considered the four fundamental pillars of community ecology [1].

Classically, if two species compete for the same resource, the one with a slight advantage will out-compete the other. This is known as the competitive exclusion principle—or Gause's law [2]. In cases where no extinction occurs, this is because adaptation to slightly different niches takes place. Accordingly, ecological communities are assembled by species evolutionary differentiation and progressive adaptation to different niches. Core ideas in community ecology, such as adaptation, niche differentiation and limiting similarity, all rely on this principle.

The theoretical and experimental developments studying the coexistence of similar species in the last century led to the competitive exclusion principle stated as ‘complete competitors occupying identical niches cannot coexist’. [3] However, a large number of species seemingly coexist on few resources in natural communities. For instance, phytoplankton diversity may reach high values despite the limited range of resources (e.g. light, nitrate, phosphate, silicic acid, iron) that microscopic algae compete for. In addition, classical coexistence theory implicitly assumes that the strength of competition correlates positively with the degree of similarity between competing species [4] (figure 1). As contemporary coexistence theory suggests [6,7], MacArthur's assumption should be qualified in order to explain the empirical observation that similar species may coexist in the natural world.

Figure 1.

The ratio ρ between inter- and intraspecific competition can be recast in terms of species overlaps in their use of shared resources. Here we show a simple example with two species consuming a single resource represented by a continuous variable x with probability density p(x). The probability p(x)dx that a species consumes resources over the resource interval [x, x + dx) is illustrated with a Gaussian density function with mean m and deviation s. (a) A situation where the overlap in resource use is moderate. (b) Depicts a scenario of large resource overlap (shaded areas). Following [5], the probability that individuals of species 1 and 2 encounter themselves and compete for the same amount of resource is p₁(x)p₂(x)dx. Hence ρ can be measured as the probability of co-occurrence of both species relative to the probability that two individuals of the same species encounter each other, The larger the resource overlap (as in b), the closer to 1 the value of competitive overlap (ρ) will be. Therefore, a limit to competition compatible with species coexistence finds its counterpart in a limit to species similarity in the use of shared resources. (Online version in colour.)

Beyond experimental approaches, mathematical modelling has permitted ecologists to address Hutchinson's paradox [8] with remarkable success and key results have been established, such as the limiting values to species similarity that permit coexistence [5], the promotion of competitive coexistence because of interspecific differences [9,10], trait-based mediated coexistence in phytoplankton communities [11,12], the role of mutualism at enhancing diversity [13] or the role of spatial structure and dispersal limitation in favouring coexistence of competitors [14,15].

Although further theoretical elaborations of these central ideas still use essentially deterministic approaches, important conceptual advances have been made to assess the distinctive roles of the four essential processes in the construction of ecological communities. For instance, seminal work by MacArthur studies the interplay between competition and stochastic extinction and concludes that even modest competition could significantly elevate extinction risk [16]. Tilman [17] focuses on the role of stochasticity and competition for resources at conditioning coexistence. Hubbell [18] develops a theory to study how communities can result only from speciation, ecological drift and dispersal limitation. Despite disregarding selection, one of the merits of Hubbell's neutral theory is to consider the interplay of speciation, dispersal and ecological drift in a unifying framework for the first time [19].

However, community ecology still lacks a body of unifying theory that considers how selection forces interact with the other fundamental processes. Selection forces are at the base of niche theories while stochasticity plays a central role in neutral, dispersal–assembly theories. Random and selection processes have been re-framed in terms of stabilizing mechanisms (niches) versus fitness equivalence (neutrality) [14,20], but, to our knowledge, the first attempt to present a common quantitative framework for niche and neutral theories was Haegeman & Loreau's (HL) work [21]. Although these authors disregarded speciation, their synthesis shows how typical community patterns emerge from an underlying stochastic dynamics that makes particular assumptions about how local competition occurs (see also [22]).

Here we show that HL stochastic competitive dynamics introduces a natural limit to the strength of competition compatible with stable coexistence. This threshold is significantly smaller than its deterministic counterpart. In ecological communities where niche differences are a relevant driver of competitive interactions, the threshold in competition translates into a reduction of species limiting similarity in the presence of ecological drift. This means that the stochastic and deterministic predictions for the composition of a competing ecological community are utterly different. As a consequence, stochasticity can impose severe limits to similarity in order to sustain stable species coexistence. We also show that the extinction threshold persists for large population sizes, and the phenomenon here described is not caused by demographic fluctuations of small-sized populations. We fully characterize the exclusion phenomenon, showing that our conclusions are recovered under distinct theoretical assumptions. This leads to a re-definition of the competitive exclusion principle that applies under rather robust and realistic conditions. As a practical implication in current biodiversity research, our results can be relevant when it comes to assessing how competition-induced extinction limits local diversity in natural settings.

2. Models

2.1. Deterministic competitive dynamics

Classical coexistence theory is based on deterministic models under the common assumption of Lotka–Volterra (LV) dynamics. Most of the theoretical studies on species coexistence still rely on LV dynamics [13,23]. Using LV equations, mathematical conditions for the exclusion of competing species can be derived as inequalities relating carrying capacities and competition strengths (see [24–26] and the electronic supplementary material). These conditions state that coexistence is stable when intraspecific competitive interactions overcome interspecific ones [6]. In situations where competition arises through species overlaps in their use of shared resources (niche differences), the ratio ρ between inter- versus intraspecific competition measures species similarity, the larger the ratio the stronger the similarity (figure 1). Therefore, in a framework acknowledging the importance of stabilizing niche differences for the coexistence of similar species [6], the limiting similarity condition is expressed as ρ < 1.

This condition arises under a purely deterministic framework. It can be recovered, in particular, under the assumption of species equivalence. To keep things simple we consider the fully symmetric version of the LV competitive model with immigration (at rate μ),

2.1

Here r is the intrinsic growth rate (uniform for all species), K the carrying capacity, ρ measures the strength of interspecific competition relative to intraspecific competition and S is the potential species richness of the community. Although immigration has been explicitly included to evaluate the impact of dispersal processes, we will focus in what follows on the low immigration regime.

The deterministic dynamics (2.1) presents a single equilibrium point whose densities are strictly positive. Moreover, when the equilibrium point is globally stable for ρ < 1 (see the electronic supplementary material). Hence, as long as interspecific interactions are weaker than intraspecific ones, the symmetric, deterministic model permits the packing of an arbitrary number of species, even in the presence of low immigration. Once the deterministic threshold ρ_d = 1 is crossed over, stable coexistence is impossible and competitive exclusion takes place.

2.2. Stochastic framework for competition

Most ecological interactions are driven by processes that are purely stochastic [27]. Individuals are discrete entities. The arrival of new individuals (dispersal limitation) or the succession of local births and deaths (ecological drift) are events that inherently occur at random instants. Thus, the intrinsic discrete nature and randomness of ecological processes must be taken into account to have a reliable picture of coexistence. For that purpose we use HL stochastic model of competitive communities [21], whose deterministic limit is precisely given by the LV dynamics (2.1).

HL model implements local dynamics as a birth–death stochastic process. It can be recast in terms of the ‘mainland–island’ paradigm [18,28], which assumes that diversity in an island stems from a balance between two stochastic processes: local extinction and the arrival of new species from the mainland. The interplay between in situ local competition within system boundaries and ongoing dispersal across boundaries from a regional species pool (metacommunity) is an important driver of community dynamics. In the high immigration limit, regional processes trivially determine local community properties through random sampling from the pool [29,30]. Local competition should become more important as the immigration rate decreases.

At any time t, the state of a local community is described by a vector n = (n₁, …, n_S), where n_i represents the population size of species i, S being the metacommunity diversity. Changes in the population size of species i occur through four elementary processes: (i) Intrinsic (local) birth and death processes, which are density-independent and occur at per capita rates r⁺ and r⁻, respectively. (ii) External immigration (at rate μ) of new individuals from the metacommunity. (iii) Intraspecific competition takes place at a per capita rate rn_i/K, where K is interpreted as a carrying capacity and r = r⁺ − r⁻. (iv) Interspecific competition comes about at a per capita rate ρr ∑_i≠jn_j/K, where ρ quantifies interspecific competition (relative to intraspecific competition) for every pair of species at the metacommunity level. The model is fully symmetric, as model parameters are uniform across all species.

The stochastic process is mathematically described by the probability P(n, t) of observing a population-size vector n at time t. A simple probability balance yields the master equation for P(n, t),

2.2

where is the overall birth probability per unit time for species i, the overall death rate and e_i = (0,…,1,…,0) the vector whose entries are 1 in the ith position and zero otherwise. The probability of visiting state n at time t increases at rate for the transition n − e_i → n, where a new individual of species i has arrived either by immigration or by an intrinsic birth, and also increases through the transition n + e_i → n at rate when an individual of species i is removed either by intrinsic death or by competition with individuals present in the community. The negative terms in the right-hand side of (2.2) simply balance the transitions from n to any other state, which force a decrease in the probability P(n, t). In this work, we focus on time-independent (steady-state) solutions of the master equation, which satisfy

In mathematical terms, even for modest values of the immigration rate μ > 0 there is a non-zero probability that an individual of any species arrives at the community, so complete extinction never occurs on average in the steady state. On the other hand, when μ = 0 the birth rate is linear in population sizes and the death rate is quadratic. This implies that the complete extinction state n* = (0, …, 0) is the unique absorbing state of the process, hence complete extinction will occur with probability one. Therefore, to understand the interplay between ecological drift and competition in a non-trivial steady-state regime, we consider here positive but small immigration rates. Although the probability of observing individuals of a given species is always positive, it can be negligible. We will associate such small probability with a scenario where the species is, as a matter of fact, extinct. In addition, as we describe in the next section, we are interested in characterizing the strength of stochastic competitive exclusion as the average fraction of species that coexist together in a local community.

In the limit of negligible fluctuations, the master equation reduces to the fully symmetric version of the deterministic LV competitive model with immigration for S species (2.1), see the electronic supplementary material. However, reliable limits to coexistence in the presence of stochasticity should be derived from the solutions of the master equation.

3. Results

We first fully characterize the competitive exclusion phenomenon that the HL model predicts, and compare it with the deterministic condition for coexistence. The explicit consideration of stochasticity introduces a threshold in competition, well below the deterministic one, above which the stable coexistence of all the species in the metacommunity is impossible. Therefore, in ecological communities where similarity (niche) differences drive competitive interactions, stochastic variability in population sizes can induce severe limits to species similarity. In other words, the limit to species similarity is more restrictive when competitive interactions are recast in stochastic terms. After proving that the threshold cannot be explained by fluctuations of small population sizes, we assess the implications of a smaller bound in species similarity on the conservation and recovery of natural communities.

3.1. Stochastic thresholds

We begin by showing that, for a community formed by two potential species, there is a threshold in competition ρ_s at which the probability of coexistence starts decreasing, and a second threshold ρ_c above which coexistence is no longer possible. Both thresholds are smaller than that of the deterministic model—i.e. ρ_s < ρ_c < ρ_d = 1.

Figure 2 depicts the joint probability distribution P(n₁, n₂) that the two species have population sizes n₁ and n₂ at the steady state as competition increases. The steady-state joint probability distribution P(n₁, n₂) can be calculated numerically using the embedded Markov chain associated with the continuous-time stochastic birth–death process, see details in the electronic supplementary material. When ρ = 0 the distribution exhibits a single maximum, associated with coexistence. At some ρ_s < 1 two new maxima emerge symmetrically with coordinates (n, 0) and (0, n). For ρ > ρ_s up to a certain value ρ_c < 1 the system exhibits a bistable situation in which it may change between a two-species coexistence state and a state where only one species survives. Finally, the coexistence peak disappears when competition reaches ρ_c, a value strictly lower than the threshold ρ_d = 1 portrayed by the deterministic model (2.1). Above ρ_c only communities with one or two extinct species can be found. Therefore, the shape of the joint probability distribution reveals three coexistence regimes separated by two threshold values in competition: (i) if 0 ≤ ρ ≤ ρ_s, the coexistence of the two species in a local community is stochastically stable; (ii) if ρ_s ≤ ρ ≤ ρ_c, competitive exclusion starts and the system alternates between communities formed by two or one species; and (iii) if ρ_c ≤ ρ ≤ 1, only a single species (or none) persists in local communities.

Figure 2.

Surface plot of the stationary joint distribution for S = 2 potential species, P(n₁, n₂), for different values of ρ. (a–d) The values are 0.05, 0.4, 0.6 and 0.95. Remaining model parameters are r⁺ = 50, r⁻ = 0.1, μ = 1 and K = 50. (Online version in colour.)

These three regimes can be more clearly pictured when the joint probability distribution is aggregated over convenient regions of the (n₁,n₂) space to represent coexistence or the extinction of 1 or 2 species (figure 3a). As shown in figure 3, in the intermediate region ρ_s ≤ ρ ≤ ρ_c, local communities may have 0, 1 or 2 species interchangeably. When fluctuations are accounted for, not only competitive exclusion starts to operate when interspecific competition is still lower than intraspecific competition, but also both states—coexistence and exclusion—alternate within a range of ρ. Once the limit ρ_c has been crossed over, maximal (two species) coexistence is impossible.

Figure 3.

(a) The saddle points (marked as white circles) of the joint distribution P(n₁, n₂) can be used to partition the configuration space into three regions associated with where coexistence is the most probable state (the square that contains the coexistence peak), or to the extinction of either one species (the two rectangles containing the boundary maxima) or two species (the small square close to the origin). Saddle points have been calculated numerically extending the discrete distribution P(n₁, n₂) to be real-valued through cubic-spline interpolation in both axes. We then solve numerically the critical point condition, ∂P(n₁, n₂)/∂n₁ = ∂P(n₁, n₂)/∂n₂ = 0 and use the Hessian matrix to single out saddle points. We finally aggregate the probabilities P(n₁, n₂) to calculate the curves represented in b and c. Aggregated coexistence probability is defined as the sum of P(n₁, n₂) over the square that contains the coexistence peak. Aggregated one-species extinction probability is obtained by summing over the two rectangles that contain the boundary maxima around (n, 0) and (0, n). The rest of the configuration space represents the probability of total extinction. Model parameters are the same as in figure 2. (b) Aggregated probabilities as a function of ρ for r⁺ = 50, r⁻ = 0.1, μ = 1 and K = 50. A vertical, dashed line shows the value ρ_c at which maximal coexistence (two species in a local community) is impossible. The threshold ρ_s at which the aggregated probability of coexistence starts declining is marked with arrows. The inset depicts the same curves for r⁺ = 0.11, r⁻ = 0.1, μ = 1 and K = 500. We observe that increasing the carrying capacity brings the threshold closer to 1. (c) Same as (b) for r⁻ = 27 and K = 23. (Online version in colour.)

Remarkably, figure 3 shows that, for small values of the carrying capacity K and large death rates r⁻, coexistence is not the only possible state even at ρ = 0. This puts a practical limit on the maximum number of coexisting species which does not have a deterministic counterpart—recall that the deterministic model permits the packing of an arbitrary number of species for 0 ≤ ρ < 1.

3.2. Multispecies competitive communities

In order to investigate the stochastic extinction transition for realistic metacommunity sizes, we simulate numerically the continuous-time process using a standard Gillespie stochastic simulation algorithm (see [31] and the electronic supplementary material). This method generates a trajectory n(t) in the space of population sizes through elementary changes in the number of individuals at each step. The species abundance distribution is calculated as the fraction P(n) of species that have n individuals once the steady state has been reached. The probability of coexistence is then given by 1 − P(0), and the average local community richness is S[1 − P(0)]. The threshold in competition ρ_s at which the system starts being stochastically unstable cannot be calculated using the multivariate distribution, which is difficult to sample. For large metacommunity sizes we define ρ_s as the overlap value at which, on average, one of the S species is extinct in local communities—equivalently, as the value of ρ for which coexistence probability equals 1 − 1/S. This is an operational, reasonable way to estimate numerically the threshold ρ_s, aimed at representing configurations close to full coexistence (see the electronic supplementary material).

Figure 4a shows the decline in biodiversity upon increases of the competitive overlap ρ, for different values of metacommunity diversity S. We observe that species diversity remains constant—equal to S—up to the threshold ρ_s. From that point on it steadily decreases and the coexistence of the S species in local communities is precluded. The same pattern found for two and three species holds for multispecies communities: demographic fluctuations limit average species richness, so that if competitive overlap increases beyond a threshold some species go extinct, the more extinctions the larger the overlap. For ρ ∼ 1 this effect can reduce local diversity well below S.

Figure 4.

Stochastic competitive exclusion and limiting similarity. (a) Average community richness for different metacommunity diversities (S) as a function of species overlap. Model parameters as in figure 2. We have pictured populations as randomly distributed points in a two-dimensional (implicit) space for S = 50 (insets), where colours stand for different species. The deterministic competitive exclusion region, ρ > 1, has been shadowed. The threshold value ρ_s in competition has been marked with a vertical, dashed line for S = 50. In the region where stochastic exclusion holds, several metacommunity diversities are compatible with the same level of realized diversity (horizontal dashed line). In this regime, the strength of competition that a community can tolerate should adapt accordingly (vertical arrows). For a given value of realized diversity, if metacommunity diversity is larger (right arrow) competition strength will be compatible with a more neutral scenario (ρ closer to 1). Conversely, if potential diversity is smaller, the strength of competition (ρ < 1) will highlight the importance of niche-based processes in shaping local diversity with a larger departure from pure neutrality. (b) Coexistence probability (S = 50) when pair-wise competitive interactions are drawn from a truncated normal distribution (mean ρ, deviation σ_ρ). Inset: immigration rates are proportional to species abundances in the metacommunity, μ_i = 0.1N_i (log-series parameter α = 0.22). The dashed line corresponds to the symmetric case (σ_ρ = 0, constant μ). (Online version in colour.)

Figure 4a also shows that a given local diversity can be compatible with several values of metacommunity diversity (S) and competition. If S is small, then competition must be low. Hence, assuming that competition arises via niche differences [6], large species differences (leading to small competition) underlie long-term coexistence. If, on the contrary, S is large, the same level of diversity is reached with a larger competitive overlap (smaller niche differences). In other words, the most diverse regions in the world, such as the tropics, may host local communities characterized by higher (nearly neutral) competitive overlaps between species.

3.3. Model structural robustness

A natural question that arises from our analysis is to what extent the conclusions we draw depend on the symmetry assumption. To answer this question we have introduced variability in competition. In realistic scenarios, pair-wise competitive interactions ρ_ij emerge from underlying ecological processes, such as interference competition, or explicit competition for resources. Thus, the symmetric competition overlap ρ that we have used until now has to be replaced by a competition matrix (ρ_ij), where ρ_ij measures the competitive overlap between species i ≠ j. Here we assume that functional ecological equivalence precludes the competition matrix (ρ_ij) from reaching a well-defined structure [32] but rather pair-wise interactions show a random pattern for all species in the metacommunity. As similarity measures, the entries of the competition matrix are symmetric, ρ_ij = ρ_ji for i ≠ j. Then, the S(S−1)/2 independent, off-diagonal matrix entries are drawn from a truncated normal distribution of mean ρ and variance Truncation ensures 0 ≤ ρ_ij ≤ 1 but implies that the average across off-diagonal matrix entries is different from ρ. Accordingly, we have represented in figure 4b the probability of coexistence as a function of the average This figure illustrates that the effect of variability in competition is to lower the threshold at which diversity starts declining.

In addition, until now we have assumed that immigration rates are uniform for all species. When immigrants enter at the local community proportionally to their abundances in the metacommunity, the extinction threshold persists (figure 4b, inset). The immigration rate for invader i has been taken as μ_i = κN_i in each realization, N_i being the (regional) abundance of species i in the metacommunity. According to most observations [18], regional abundances (N_i) are drawn from a log-series distribution. We conclude that the qualitative picture, i.e. the existence of a more restrictive threshold for species similarity in the presence of ecological drift, remains unchanged.

3.4. Threshold persistence for large population sizes

It is well known that demographic stochasticity has negative effects on community persistence when population sizes are small [33]. However, the existence of a stochastic limit to species similarity ρ_s is by no means a consequence of small population sizes. We have computed the average population size in local communities,

3.1

and its variance

3.2

at the threshold ρ_s for increasing carrying capacity K and metacommunity richness S, keeping the ratio K/S fixed (figure 5). We maintain K/S fixed because increasing K at constant S will trivially displace the threshold towards 1.

Figure 5.

Fluctuations in population sizes (σ_n, circles) compared with their averages (, squares) for increasing K and S while K/S = 2. Inset: extinction threshold ρ_s (defined as the value of competition at which on average one of the S species is extinct) as a function of S. Power law fits to data satisfying 10² ≤ K ≤ 10³ (50 ≤ S ≤ 500) are shown. (Online version in colour.)

It can be proved (cf. the electronic supplementary material) that fluctuations in population sizes calculated at ρ_s scale as σ_n ∼ K^1/2, whereas average population sizes are expected to scale as as long as the extinction threshold decays as ρ_s ∼ S^−γ. We observe in figure 5 that those scalings are well reproduced numerically, with γ ∼ 0.6. Therefore, the fluctuations of the population size relative to its average scale as and tend to zero as At the same time, the threshold value ρ_s that limits species similarity decreases. For large K, fluctuations are negligible compared with population sizes and the threshold persists, so extinctions cannot be ascribed to large fluctuations around small-sized populations. On the contrary, the threshold on species similarity may well be even more restrictive for large population sizes.

3.5. Habitat fragmentation and recovery

We have shown that ecological drift undermines the coexistence of an assemblage of species by lowering the threshold at which competitive exclusion starts to operate. This drift-induced, more restrictive threshold can be interpreted as a smaller limit to species similarity in stable coexistence. A practical implication of the fast diversity decay once the threshold has been crossed over concerns habitat destruction and later recovery. In general, habitat destruction should translate into a decrease in carrying capacity or an increase in competitive overlap or both, which brings about a decrease of local diversity as a result of stochastic competitive exclusion. Conversely, a decrease in competition through niche restoration favours species coexistence. However, as the immigration rate controls the arrival of new species, the recovery of the initial diversity level is a much slower process.

To illustrate this point we have simulated the stochastic dynamics when competitive overlap ρ is linearly increased and later decreased in time down to the initial value. The time-dependent coexistence probability is shown in figure 6a. We can see in this figure that it takes longer for the system to recover than to decrease its diversity. Different timescales in fragmentation and restoration regimes yield a dynamical effect similar to hysteresis [34] when parameter changes occur dynamically, before reaching the steady state. In figure 6b,c we have simulated cycles of increase (respectively decrease) and later decrease (respectively increase) of competition (figure 6b) and carrying capacity (figure 6c). In the simulations, mild variations in ρ or K take place after regular time intervals Δt in the destruction and restoration pathways. For Δt that are small enough, the system has not reached the steady state when competition or carrying capacity vary, hence the fragmentation and restoration trajectories do not necessarily coincide and the system may not recover its initial state after a complete cycle. Both pathways, however, eventually coincide as Δt → ∞. Note the hysteresis exhibited by the coexistence probability, which is stronger the faster the change in the parameters.

Figure 6.

(a) Temporal evolution of the coexistence probability as competitive overlap varies as shown in the profile (dashed lines). Dark-shaded areas mark intervals where competition increases or decreases. There is an additional time (light-shaded areas) to recover initial diversity levels after the period of niche destruction (increasing competitive overlap). Remaining parameters are S = 50, r⁺ = 50, r⁻ = 0.1, K = 50 and μ = 1. Averages have been calculated using 400 model realizations. Transient dynamics from the initial condition to steady state is not shown. (b) Increases (respectively decreases) in competitive overlap occur at regular time intervals Δt. As the steady state is not necessarily attained, the initial state is not necessarily recovered in the restoration pathway. In the limit Δt → ∞, the fragmentation and restoration curves coincide. Here S = 50, r⁺ = 50, r⁻ = 0.1, μ = 1 and K = 50. (c) Same as (b) but showing the effect of changing carrying capacity (here ρ = 0.1 and other parameters remain unchanged as K varies). (Online version in colour.)

4. Discussion

Our work points to three general conclusions. First, we emphasize the distinct role of ecological drift in community ecology [27]. Second, we warn about the dangers of regarding stochastic effects as a slight perturbation of deterministic predictions. Due to the discrete nature of ecological interactions, stochastic and deterministic predictions may be irreconcilably different. Accordingly, coexistence theory should be reformulated in stochastic terms. And third, we emphasize the value of unifying theory to consider the interplay between ecological drift, dispersal limitation and selection forces at explaining diversity patterns in local communities. In particular, here we have explored the role of competition as the selection process at work because of its potential leading role to explain how diversity is maintained in the world. However, other interactions surely show the same kind of purely stochastic thresholds.

We have based our analysis on a synthetic theory introduced by HL [21] to bridge the gap between niche and neutral theories using a single parameter that encodes niche overlap. As this parameter tends to 1, intra- and interspecific competition become equal, so competitive interactions become neutral at the individual per capita level, and community properties at stationarity are the same as those predicted by the standard neutral model [21,29,35].

Our main result uncovers the existence of a stochastic extinction threshold ρ_s < 1 in the transition from full niche separation (ρ = 0) to full niche overlap (ρ = 1). Above this threshold, competitive exclusion holds. In situations where competition can be interpreted in terms of species similarity, this threshold is equivalent to a more restrictive limit on similarity than its deterministic counterpart. Ecological drift restricts the capacity of the ecosystem to maintain an elevated number of coexisting species. Once the threshold has been crossed over, species are driven to extinction leading to a considerable reduction of local diversity levels. As we have proved this phenomenon is intrinsically stochastic and is not a trivial consequence of finite population sizes. It is relevant to remark that the classical competitive exclusion principle as well as previous approaches to species coexistence do not account for this extinction phenomenon. Therefore, classical coexistence conditions should be recast in stochastic terms.

A paradigmatic example that seemingly disproves competitive exclusion is the so-called ‘paradox of the plankton’. Our results seem to contradict the empirical observation that realized plankton diversity is larger than expected under the deterministic competitive exclusion principle. In the presence of drift the expected diversity will become even lower. Other mechanisms have been proposed to solve the paradox, for example, planktonic species could avoid competition by using resources at different windows of time via oscillating, out-of-phase trajectories [36]. As suggested by Hutchinson [37], time is a relevant component of the niche, so unstable dynamics can promote coexistence. One of our assumptions is that local communities have reached a steady state, which is not necessarily true in natural settings. An important implication of out-of-equilibrium scenarios is the emergence of significant hysteresis in competition and carrying capacity. Another hypothesis of our framework is the assumption of implicit-space interactions. Spatially explicit settings are known to favour coexistence [14,15]. It remains an open question to quantify the extent to which the inclusion of stochasticity in a spatially explicit version of the LV deterministic dynamics would change deterministic predictions either favouring coexistence or not. These situations will probably inspire new extensions of our modelling approach, the implications of which could be significantly important to the contemporary theoretical understanding of species coexistence and biodiversity.

In other contexts, the difference between deterministic and stochastic thresholds has long been recognized. For instance, to study the maintenance of an infectious disease in a population, the concept of ‘critical community size’ was introduced [38]. Below this critical size, stochastic fluctuations lead to disease extinction even though deterministic approaches predict disease maintenance [39]. Major dynamical transitions in epidemics from regular to irregular cycles can also be better explained if stochasticity is accounted for [40]. Here we show that stochastic extinction phenomena are not trivially introduced by a finite population size, but they arise through the interplay of the processes driving the system: competitive interactions are responsible for the displacement of the threshold in similarity when ecological drift is explicitly considered.

Our results have potential applications in situations where agents compete for shared resources. For example, financial markets and banking systems share remarkable analogies with ecological systems [41]. Knowledge on the assessment of systemic risk has been recently increased thanks to analogies with the dynamics of ecological food webs and banking systems [42–44]. In financial markets, banks compete for shared resources, and the degree of overlap between entities can be determinant in the competition–stability trade-off [45]. Limits to species similarity in their use of common resources, such as the ones found here, can be potentially applicable to ‘ecosystems’ of banks.

The role of a theory is to provide a framework to interpret the real world. Are our theoretical explorations about stochastic coexistence in competitive communities able to yield new insights to understand variable levels of diversity maintenance in the world? Our results point towards at least two empirically testable predictions. We highlight them in order.

First, as human general appropriation of the Earth's ecosystems involves a homogenization of natural habitats, it should have the overall effect of increasing species niche overlap. Community assembly through repeated invasion and species loss through stochastic extinction are highly asymmetrical processes. The introduction of new species into the local community occurs at low pace but, once the extinction threshold is overcome, species loss may be very fast. This timescale separation implies a significant hysteresis effect in the process of biodiversity loss and potential restoration. If niche overlap is increased through habitat simplification beyond the extinction threshold, niche restoration efforts can take a long time to cause any effect as coexistence cannot be restored through habitat amelioration unless it goes all the way down to the values where coexistence is the only possible state.

Second, species evolving under ecological drift should become less similar, that is, more ecologically segregated than they would be the case if ecological drift was negligible, in order to make coexistence possible. In particular, the model predicts a negative relationship between realized diversity in a local site and the strength of competitive overlap. In terms of species similarities, the larger the niche separation the larger the probability of stochastic coexistence. As a consequence, potential diversity could be inferred from actual diversity and the average level of interspecific competition. Conversely, accurate empirical knowledge about the potential species richness of a community becomes crucial to assess the relevance of niche differences in competing communities (figure 4a).

In support of our approach, we have used the HL model to generate several new predictions, some of which could be potentially tested against empirical data. These are preliminary explorations to show how a synthetic theoretical framework can guide further research and generate new insights into the causes and processes controlling local diversity across regions. HL initial stochastic synthesis [21] opened the door to study the extent to which competitive overlap versus ecological drift drive local diversity patterns in a quantitative way. A better assessment of the relative importance of the different processes that shape ecological communities is crucial to improve our ability to map, monitor and control changes in biodiversity geographical distributions across the globe.

Authors' contributions

J.A.C., S.C. and D.A. conceived research and designed the study; J.A.C. and S.C. performed research; J.A.C., S.C. and D.A. analysed the results; J.A.C. and D.A. wrote the manuscript. All authors gave final approval for publication.

Competing interests

We declare we have no competing interests.

Funding

J.A.C. and D.A. acknowledge financial support from the Spanish Ministerio de Economía y Competitividad under the project SITES (CGL2012-39964).