Cyclic dominance in evolutionary games: a review

Abstract

Rock is wrapped by paper, paper is cut by scissors and scissors are crushed by rock. This simple game is popular among children and adults to decide on trivial disputes that have no obvious winner, but cyclic dominance is also at the heart of predator–prey interactions, the mating strategy of side-blotched lizards, the overgrowth of marine sessile organisms and competition in microbial populations. Cyclical interactions also emerge spontaneously in evolutionary games entailing volunteering, reward, punishment, and in fact are common when the competing strategies are three or more, regardless of the particularities of the game. Here, we review recent advances on the rock–paper–scissors (RPS) and related evolutionary games, focusing, in particular, on pattern formation, the impact of mobility and the spontaneous emergence of cyclic dominance. We also review mean-field and zero-dimensional RPS models and the application of the complex Ginzburg–Landau equation, and we highlight the importance and usefulness of statistical physics for the successful study of large-scale ecological systems. Directions for future research, related, for example, to dynamical effects of coevolutionary rules and invasion reversals owing to multi-point interactions, are also outlined.

1. Introduction

Games of cyclic dominance play a prominent role in explaining the intriguing biodiversity in nature [1–3], and they are also able to provide insights into Darwinian selection [4], as well as into structural complexity [5] and prebiotic evolution [6]. Cyclical interactions have been observed in different ecological systems. Examples include marine benthic systems [7], plant systems [8–12] and microbial populations [2,13–16]. Cyclic dominance also plays an important role in the mating strategy of side-blotched lizards [17], the overgrowth of marine sessile organisms [18], the genetic regulation in the repressilator [19], and in explaining the oscillating frequency of lemmings [20] and of the Pacific salmon [21]. The list of examples where the puzzle of biological diversity can be explained by the interaction topology of the food web among those who struggle for life is, indeed, impressively long and inspiring [22,23].

In addition to the closed loops of dominance in the food webs that govern the evolution of competing species and strategies, experiments have shown that the key to the sustenance of biodiversity is often spatial structure. In vitro experiments with Escherichia coli, for example, have shown that arranging the bacteria on a Petri dish is crucial for keeping all three competing strands alive [2,24,25]. On the other hand, in vivo experiments with bacterial colonies in the intestines of co-caged mice, which can be considered as locally well-mixed populations, have revealed that mobility alone is sufficient to maintain coexistence [14]. Solely because the bacteria were able to migrate from one mouse to another, the biodiversity was preserved. Motivated by such intriguing experimental observations, theoretical explorations of the relevance of spatial structure in evolutionary games have also received ample attention in the recent past [26–31].

The importance of spatial structure has been put into the spotlight by the discovery that interaction networks such as the square lattice can promote the evolution of cooperation in the Prisoner's Dilemma game through the mechanism that is now known as network reciprocity [32,33]. In essence, network reciprocity relies on the fact that cooperators do best if they are surrounded by other cooperators. More generally, the study of spatial games has revealed the need for realistic models to go beyond well-mixed populations [34–36]. To understand the fascinating diversity in nature and to reveal the many hidden mechanisms that sustain it, we cannot avoid the complexity of such systems. Cyclical interactions are characterized by the presence of strong fluctuations, unpredictable and nonlinear dynamics, multiple scales of space and time and frequently also some form of emergent structure. Because many of these properties are also inherent to problems of non-equilibrium statistical physics, theoretical research on spatial rock–paper–scissors (RPS) and related games of cyclic dominance has a long and fruitful history [37–65], much of which is firmly rooted in methods of statistical physics that can inform relevantly on the outcome of evolutionary games on structured populations.

Cyclical interactions are in many ways the culmination of evolutionary complexity, and they may also arise spontaneously in evolutionary games where the number of competing strategies is three or more. For example, cyclic dominance has been observed in public goods games with volunteering [66,67], peer punishment [35,66,68,69], pool punishment [70,71] and reward [72–74], but also in pairwise social dilemmas with coevolution [75,76] or with jokers [77]. Counterintuitive complex phenomena that are due to cyclic dominance include the survival of the weakest [41,49], the emergence of labyrinthine clustering [64] and the segregation along interfaces that have internal structure [78], to name but a few examples. Importantly, the complexity of solutions increases further and fast if the competing strategies are more than three. Figure 1 depicts a transition towards more and more complex food webs from left to right, which support the formation of defensive alliances [81] and subsystem solutions [82] that are not possible in the simplest RPS game.

Figure 1.

Examples of food webs of competing strategies. The number of competing strategies and the topological complexity of the food web increases from left to right, and so does the complexity and number of possible stable solutions: (i) the traditional RPS food web. Here rock (R) is wrapped by paper, paper (P) is cut by scissors, and scissors (S) are broken by a rock. (ii) Four-strategy food web with different invasion rates between predator–prey pairs, which already supports the emergence of defensive alliances [79]. (iii) Six-species food web with heterogeneous invasion rates, supporting a multitude of different defensive alliances as well as noise-guided evolution [80]. (iv) Topologically complex food web describing bacterial warfare with two toxins. Here nine different strains compete, giving rise to a multitude of stable solutions that amaze with their complexity and beauty [44]. Three-strategy alliances that can successfully avert an external invasion are denoted by broken coloured lines.

Although the outcome of evolutionary games with cyclic dominance may depend on the properties of the interaction network, the topology of the food web, and on the number of competing strategies, there, nevertheless, exists fascinatingly robust universal behaviour that is independent on model particularities. While we may have only begun to scratch the surface of the actual importance of cyclic dominance in nature [83,84], the time is ripe for a survey of theoretical advances that may guide future experimental efforts. Like by the evolution of cooperation [85], in the games of cyclic dominance, the evolutionary game theory [86–90] is also proving to be the theory of choice for the study of these beautifully simple yet fascinating systems.

The organization of this review is as follows. In §2, we focus on mean-field and zero-dimensional RPS models to give a comprehensive prologue to the overview of the spatial RPS game in §3. Within the latter, we separately review the impact of different interaction networks and mobility, and we describe the derivation and the proper use of the complex Ginzburg–Landau equation (CGLE) to describe the spatio-temporal properties and the stability of the spiralling patterns. In §4, we depart from the RPS game where the cyclic dominance is explicitly encoded in the food web by considering social dilemmas and other evolutionary games. We separately review games where cyclical interactions emerge spontaneously, the role of time-dependent interactions, the impact of voluntary participation, and we survey evolutionary games where an alliance between two strategies acts as an additional independent strategy. From there, we focus on cyclical interaction between four and more competing strategies, which opens the door towards reviewing the emergence of defensive alliances and large networks of competitive species. We conclude with a summary and an outlook towards promising future research efforts.

2. Cyclic dominance in well-mixed systems

While the emphasis of this review is on cyclic dominance in structured populations, it is useful to start by considering the simplest setting to study RPS games which is the mean field limit of well-mixed populations where all individuals can interact with each other.

A generic RPS game between individuals of three species, A, B and C, in a population of total size N entails cyclic competition and reproduction according to the following reaction schemes

2.1

2.2

2.3

where the reactions (2.1) and (2.2) describe two forms of cyclic dominance of species A over B, B over C and in turn, C over A, and (2.3) accounts for reproduction into an empty space (denoted by ∅︀) with a rate q [28,46,91]. The reaction given by equation (2.1) corresponds to one species dominating and displacing another (dominance-removal) with rate p, whereas, in equation (2.2), there is a zero-sum dominance-replacement with rate z. Recent experiments have shown that in addition to cyclic dominance, some species can mutate. For instance, E. coli bacteria are known to mutate [2], and it was found that the side-blotched lizards Uta stansburiana are not only engaged in cyclic competition, but they can also undergo throat colour transformations [92]. It is therefore also instructive to assume that each species can mutate into one of the other species with a rate μ [53,91], according to

2.4

Additional evolutionary processes that should be specified are those encoding the movement of individuals. However, these are irrelevant in the absence of spatial structure and is therefore introduced in §3. In the mean field limit of population size N → ∞, the dynamics of the system evolving according to equations (2.1)–(2.4) is aptly described by the following rate equations for the densities a(t) of As, b(t) of Bs and c(t) of Cs

2.5

where ρ_∅︀ = 1 − (a + b + c) denotes the density of empty sites. These rate equations admit a steady-state s* = (a*, b*, c*) in which A, B and C coexist at the same density a* = b* = c* = q/(p + 3q) encompass three types of oscillatory dynamics:

1. When p, q > 0, z ≥ 0 and μ = 0, the equation (2.5) can be rewritten as the rate equations of the May–Leonard model [93]. In this case, the above coexistence state is unstable, and there are also three absorbing stationary states corresponding to the population being composed of only A, B or C. As in the May–Leonard model, the dynamics is characterized by heteroclinic cycles connecting these three absorbing states, with the population being composed almost exclusively of either A, B or C, in turn. It is worth noting that the heteroclinic cycles are degenerate when z = 0 [93].

2. When p, q > 0, z ≥ 0 and μ > 0, a supercritical Hopf bifurcation arises at the value μ = μ_H = pq/(6(p + 3q)) of the bifurcation parameter: the coexistence steady-state is a stable focus when μ > μ_H, whereas it is unstable when μ < μ_H, and the dynamics is thus characterized by a stable limit cycle of frequency [53,91].

3. When z > 0 and p = q = μ = 0, the state ∅︀ plays no role in the dynamics, and one recovers the cyclic Lotka–Volterra model [39,94]. The sum and product of the densities, a(t) + b(t) + c(t) and a(t)b(t)c(t), are conserved by (2.5) which results in nested neutrally stable closed orbits around the coexistence fixed point which is a neutrally stable centre [95]. It is worth noting that the cyclic dominance-replacement scheme with asymmetric rates is also characterized by a marginally stable coexistence fixed point s* = [1/(z_A + z_B + z_C)](z_B, z_C, z_A) surrounded by neutrally stable orbits on which the mean field dynamics takes place (figure 2) [42,50].

Figure 2.

Phase portrait of the RPS dynamics with asymmetric dominance-replacement rates z_A = 2, z_B = 1, z_C = 1.5 in a well-mixed population. The mean field rate equations predict closed orbits, here shown in black solid lines. Their linearization around the neutrally stable coexistence steady state s* = (2/9, 1/3, 4/9) transform the orbits into circles in the coordinates y_A, y_B (blue). The red flow that erratically spirals out from the coexistence fixed point is a single trajectory obtained by simulating the stochastic dynamics in a finite population of size N = 200. This illustrates how demographic fluctuations are responsible for the extinction of two species (here A and C) after a time of order N. See the main text for details, as well as [42], from where this figure is adapted.

The evolution in well-mixed populations of finite size, N < ∞, is usually described in terms of birth–death Markov chains [96]. In this case, the interactions between a finite number of discrete individuals lead to fluctuations that may drastically affect the dynamics. In particular, in the presence of absorbing states, fluctuations are responsible for the extinction of two strategies and the fixation of the surviving one (figure 2) [42,50]. Questions concerning the mean time necessary for two species to go extinct and the probability that one specific strategy survives have recently attracted significant attention and were mostly addressed by means of stochastic simulations. These are often efficiently carried out using the Gillespie algorithm [97]: for instance, cyclic competition and reproduction according to equations (2.1)–(2.3) are implemented by allowing at each time increment a reproduction event with probability q/(p + q + z), a dominance displacement move with probability p/(p + q + z) and a dominance-replacement update with probability z/(p + q + z). When p > 0 and μ = 0 and the mean field dynamics is characterized by heteroclinic cycles, the mean extinction time has been found to be of order . The existence of quantities conserved by the mean field dynamics allows some analytical progress in the case of cyclic dominance with replacement (p = q = μ = 0) for which it was shown that two species die out after a mean time that scales linearly with the system size N [42,98,99]. In this class of models with asymmetric rates, it was also shown the survival probability follows an intriguing ‘law of the weakest’: the strategy with the smallest dominance rate is the most likely to survive and to fixate the population [41,49], as illustrated in figure 2. Some other aspects of extinction in RPS games in finite and well-mixed populations have been considered in references [100,101] and studied in further details in [102].

3. Rock–paper–scissors game in structured populations

In a structured population, or alternatively, in a spatial system, the interactions between players are defined by an interaction network and are thus limited. Because players have only limited interactions with their neighbours, a strategy (or species) will not necessarily meet its superior (or predator). This restriction has far-reaching consequences because it enables pattern formation, and through that the survival of all three competing strategies with a time-dependent frequency akin to the red queen [103–105]. For example, it is possible to observe propagating fronts and spiral waves between the competing strategies [40,47,91], which are particularly fascinating and complex if the invasion rates between different pairs of strategies are significantly different [64]. In general, self-sustained oscillations of all three strategies are possible in spatially homogeneous populations [106,107] and result from periodic dynamics around a long-lived metastable coexistence state whose lifetime is determined by the system size (or the size of the population) [108,109]. Hence, if the system size is sufficiently large all possible combinations of invasion fronts appear, resulting in a time-independent constant frequency of strategies, depending only on the invasion rates between the three competing strategies.

In comparison with a well-mixed population, a spatial system allows us to introduce an additional microscopic process that has no effect if everybody interacts with everybody else, namely site exchange between strategies or between an individual player and an empty site. This gives rise to mobility, which arguably has a profound impact on the outcome of the RPS game as it can both promote and impede biodiversity [3]. In contrast to well-mixed systems, where as seen intrinsic noise can jeopardize the coexistence of species [42], depending on the level of mobility, species coexistence can be more easily maintained than under well-mixed conditions [3,45], and stochastic effects can have counterintuitive effects at some bifurcation point be reduced by stable patterns in spatial systems [46].

The mentioned mixing of strategies (directly or via empty sites) can also influence the cyclic competition in aquatic media [110]. Furthermore, the degree of mixing determines the way of extinction, such as the annihilation of domains, heteroclinic orbits or travelling waves in a finite system [111]. The RPS game can also be regarded as a cyclical Lotka–Volterra model, and indeed, it has been shown that a two-species Lotka–Volterra system with empty cites can give rise to global oscillations [112]. In general, however, it is important to acknowledge that the structure of the interaction network and mobility can both have a significant impact on the evolutionary outcome of cyclical interactions, and in the following subsections, we review this in more detail as well as survey the usefulness of the CGLE.

3.1. Interaction networks

In a structured population, species should contact in real-space and their interaction is characterized by an appropriate graph. Lattices are the simplest among these interaction networks, where every player has the same number of neighbours. By far the most widely used is the square lattice, although alternatives such as the honeycomb lattice, the triangular lattice and the kagome lattice are frequently employed as well to study the relevance of the degree of players and the role of clustering. While the square and the honeycomb lattice both have the clustering coefficient equal to zero, the kagome and the triangular lattice both feature percolating overlapping triangles, such that their clustering coefficient is and , respectively. In general, lattices can be regarded as an even field for all competing strategies where the possibility of network reciprocity is given [32]. Furthermore, as there are many different types of lattices, it is possible to focus on very specific properties of cyclical interactions and test what their role is in the evolutionary process. In some cases, such as competition between bacteria [2,16,28,113,114] or for the description of parasite–grass–forb interactions [12], lattices are also an apt approximation of the actual interaction network. In more complex systems, however, the description of interactions requires the usage of more intricate networks, which typically have broad-scale degree distribution and small-world properties [115,116]. The question how the topology of the interaction network affects the outcome of cyclical interaction, and game-theoretical models, in general, [26,27,29,31], is one of great interest.

As an initial departure from lattices towards more complex interaction topologies, it is possible to consider networks with dispersed degrees. As demonstrated by Masuda and Konno, the resulting heterogeneity can help to maintain the stable coexistence of species in cyclic competition [117]. Another possibility is to introduce permanent shortcut links that connect distant players, thereby effectively introducing small-world properties. In contrast to lattices, on small-world networks, the frequencies of competing strategies oscillate even in the large system size limit, as shown in [118–121]. The explanation behind this observation lies in the fact that the introduction of shortcuts introduces quenched randomness into the interaction network. If the magnitude of this randomness exceeds a threshold value, i.e. if the number of shortcut links is sufficiently large, the amplitude of frequency oscillations can be so large that the system will always terminate into an absorbing state during the evolutionary process. These results lead to the conclusion that not only the limited interaction range that is imposed by structured populations, but even more so the explicit spatial distribution of competing species is the crucial property that determines coexistence and biodiversity [122]. This observation becomes particularly interesting if we compare it with the observations made in the realm of spatial social dilemma games, where the limited interaction range was found to be the decisive factor for the promotion of cooperation based on network reciprocity [26,33].

Properties that are typical of small worlds can be introduced not only by means of permanent shortcut links between distant players, but also by means of temporary long-range links that replace nearest-neighbour links with probability P. This process is typically referred to as annealed randomness [118]. But while topological properties of annealed and quenched small worlds may be very similar, there is a relevant difference in the evolutionary outcomes of cyclical interactions taking place on them. Namely annealed randomness no longer gives rise to oscillatory frequencies of the three competing strategies in the RPS game. For sufficiently large values of P, the oscillations are replaced by steady-state behaviour [123], as is commonly observed on regular lattices. Interestingly, similar results were also reported for a four-strategy cyclic dominance game in [124]. But as we review later, the transitions between stationary and oscillatory states in cyclical interactions can, in fact, be driven by many different mechanisms, even if the interaction network thereby remains unchanged.

The topology of the interaction network may also change as a result of a particular coevolutionary rule, as studied frequently before in the realm of social dilemma games. Note that when not only individual strategy, but also other player-specific character (such as connectivity, strategy adopting or passing capacity, etc.) may change in time then we can observe parallel evolutions, coevolution of strategies and other quantities, see [29] for a review. In particular, rewiring could be the consequence of invasions between strategies [75,125]. Although games of cyclic dominance received comparatively little attention in this respect, there is a study by Demirel et al. [126], which builds exactly on this premise. Namely the loser of each particular instance of the game can either adopt the winning strategy (an invasion occurs), or it can rewire the link that brought the defeat. It was shown that non-equilibrium phase transitions occur as a function of the rewiring strength, as illustrated in figure 3. The observation of the ‘stationary → oscillatory → absorbing → stationary’ phase transitions is conceptually similar to those observed in the realm of interaction networks with annealed randomness. In general, it is thus possible to conclude that oscillatory behaviour in coevolving networks is a robust phenomenon—a conclusion that is further corroborated by agent-based simulations showing that coevolving networks can influence the cycling of host sociality and pathogen virulence [127].

Figure 3.

Transitions between ‘stationary → oscillatory → absorbing → stationary’ phases in a coevolutionary RPS game where an invaded player can sever the link towards its predator with a probability P. For further details, we refer to [126]. The reported non-equilibrium phase transitions are qualitatively similar to those observed in interaction networks with annealed randomness [118], as well as those observed in coevolutionary social dilemma games [75,125]. The emergence of oscillatory behaviour akin to the red queen thus appears to be a robust phenomenon in coevolving networks.

3.2. Mobility

As briefly explained above, besides the properties of the interaction network, mobility, as is the case in general for evolutionary game-theoretical models [128–132], can also play a decisive role in the maintenance of biodiversity in games of cyclic dominance. The consideration of mobility has both theoretical and practical aspects. Theoretically, the question is whether mobility affects the stability of spatial patterns that typically emerge in structured populations, and if yes, how? From a practical point of view, it is important to acknowledge the fact that in real-life prey and predators are frequently on the move in order to maximize their chances of survival. Accordingly, their neighbourhoods change, which is in agreement with the effect that is brought about by mobility in simulations. The crucial point to clarify is whether mobility simply shifts the evolutionary outcomes towards those observed in well-mixed populations, or whether there emerge unexpected phenomena that disagree with mean-field predictions. As we review in what follows, the introduction of mobility in structured populations is certainly much more than simply a transition towards well-mixed conditions.

Pioneering work on the subject has been done by Reichenbach et al. [3,133], who showed that mobility (or migration) has a critical impact on biodiversity. In particular, small mobility that is below a critical threshold promotes species coexistence, whereas large mobility that is above the threshold destroys biodiversity. Behind these observations are spontaneously emerging spiral patterns, which, in general, sustain the coexistence of all three competing strategies [40,134]. Under the influence of mobility, however, the spirals grow in size, and above the critical threshold, they outgrow the system size causing the loss of biodiversity. Accordingly, in [3], it was thus argued that mobility can both promote and jeopardize biodiversity in RPS games.

As most studies on this subject, in this section, we focus on RPS games on two-dimensional square lattices of total size N (here, linear size is ). The population's composition changes according to the dominance-removal reactions given by equation (2.1) and reproduction reactions given by equation (2.3) (i.e. q, p > 0 and z = μ = 0, which yields degenerate heteroclinic cycles at mean field level; see §2). Furthermore, it is assumed that individuals can move by swapping their position with any neighbour at rate γ according to the scheme

3.1

where . In many references, mobility is measured in continuum limit by introducing the mobility or diffusion coefficient m = 2γ/N, where N is the system size [3].

To focus on the effects of mobility m on pattern formation and biodiversity, the rate of competition p and reproduction q can initially be set fixed at 1. Starting from random initial conditions where each lattice site is with equal probability either occupied by rock A, paper B, scissor C or left empty, we find that the emerging spatial patterns depend sensitively on mobility. As shown in the top row of figure 4, as mobility increases from left to right, the characteristic scale of spiral waves also increases. It was found that at a critical value of mobility, m = m_c = (4.5 ± 0.5) × 10⁻⁴ for p = q = 1, the length scale of the spiralling patterns outgrows the system size resulting in two species dying out and the dynamics settling in the absorbing of the remaining species [45–47,136]. The diversity is lost, whereby the extinction probability is defined as the disappearance of two species after a waiting time that is proportional to the system size. As the system size increases, this probability approaches zero if m < m_c, and it tends to 1 if m > m_c.

Figure 4.

Typical spatial patterns, which emerge spontaneously from random initial conditions on a square lattice. Top panels illustrate how the characteristic length scale of spiral waves increases with increasing mobility m from left to right. Here the competition and reproduction rate are both p = q = 1. Bottom panels show the reverse transition, where the characteristic length scale of spiral waves decreases as the competition rate p increases from left to right. Here, the mobility m = 10⁻⁴ and the production rate q = 1.0 are fixed. Colours red, blue and green denote rock (species A), paper (species B) and scissor (species C), respectively. Empty sites are grey and denoted by ⊗. The system size is N = 512². For further simulation details, we refer to [135].

In contrast to mobility, competition promotes coexistence of species because the typical length of the spatial patterns decreases with increasing competition rate p. This is illustrated in the bottom row of figure 4 from left to right. Macroscopic spiral waves emerge at p = 0.1, but they become smaller and fragmented as p increases. The fragmentation is due to the fact that large competition rates introduce empty sites around small patches that contain all three competing strategies (i.e. at the spiral core), which, in turn, leads to the disintegration of macroscopic spirals typically observed away from the core. Diversity is therefore favoured at larger values of p, as they effectively prevent the spirals outgrowing the system size. Similar phenomenon can also be observed in the generic metapopulation model. In the latter case, however, the fragmentation is a consequence of the meeting of spiral fronts, as illustrated figure 8 [91,137].

Figure 8.

Influence of nonlinear mobility on spiralling patterns in the metapopulation RPS model defined by (2.1)–(2.4) and (3.3) with (γ_d, γ_e) = (0.05, 0.05), (0.20, 0.05) from left to right, respectively: typical snapshots in lattice simulations starting from ordered initial conditions and with parameters L² = 128², N = 1024, p = q = 1, z = 0.1 and (far below the Hopf bifurcation point). (a) Nonlinear mobility enhances the convective instability and far field breakup of spiral waves. See [91] and movies [145] for further details.

Because pattern formation obviously plays a critical role in warranting the coexistence of the three competing strategies, it is particularly important to understand how the spatial patterns evolve. Prepared initial conditions are strongly recommended in such cases, as they can help identify critical processes that may either enhance or destroy the stability of coexistence. The emergence of spirals can be engineered if the evolution is initiated from three roundish areas, each holding a particular strategy, whereas the rest of the lattice is empty [138]. Based on such a procedure, it has been shown that there exists a critical competition rate, p_c, below which single-armed spirals in finite populations are stable. For p > p_c, the spirals break up and form disordered spatial structures, because too many empty sites allow the emergence of competing mini spirals [135]. This transition is demonstrated in the top row of figure 5 from left to right.

Figure 5.

Disintegration of single-armed spiral waves as the competition rate p increases past the critical values p_c. Depicted are typical spatial patterns that emerge from a prepared initial state (see fig. 2 in [135] for details) for different values of p (see legend). The top row depicts results obtained with Monte Carlo simulations for m = 5.0 × 10⁻⁵ and q = 1, while the bottom row depicts results obtained with partial differential equations describing the spatial RPS game (see equation (3.2)) for D_m = 5.0 × 10⁻⁵ and q = 1. Disordered spatial patterns emerge as soon as p > p_c = 2.3. Colours red, blue and green denote rock (species A), paper (species B) and scissor (species C), respectively. Empty sites are grey and denoted by ⊗. For further simulation details, we refer to [135].

Although Monte Carlo simulations are applied most frequently for studying spatial evolutionary games (see reviews [26] and [31] for details, and [91] for the use of the spatial Gillespie algorithm [97]), when density fluctuations are neglected, the dynamics of the spatial RPS game on a domain of unit size can be described by the following partial differential equations (PDEs) obtained by letting the lattice spacing 1/N vanish in the continuum limit N → ∞:

3.2

where r labels the two-dimensional position, whereas ρ_Ø and D_m denote the density of empty site and the diffusion coefficient (on a domain of unit size, D_m coincides with m), respectively. This set of PDEs can be solved numerically by standard finite difference methods, and as illustrated in the bottom row of figure 5, the results are similar to those obtained via Monte Carlo simulations.

By exploiting further the potential of prepared initial conditions, it is also possible to observe multi-armed spirals, as well as pairs of anti-spirals with a finite number of arms and pairs [57]. We show examples of this fascinating pattern formation in figure 6. An in-depth qualitative analysis reveals that such multi-armed spirals and anti-spirals are more robust to mobility than single-armed spirals, persisting not only at small, but also at intermediate mobility rates. In addition to prepared initial conditions, periodic currents (or localized forcing) can also be applied to evoke specific spatial patterns. If a periodic current of all three strategies is applied at the centre of the lattice, target waves can be observed, as reported in [139]. Such phenomena emerge owing to the interplay between local and global system dynamics, in particular, the intermittent synchronization between the periodic current and the global oscillations of the density of the three competing strategies on the spatial grid. In figure 7, we show how the global target wave (top row from left to right) emerges owing to the emergence of resonance between the periodic current and the global oscillations (bottom row from left to right).

Figure 6.

Beautiful examples of multi-armed spirals with two, three and four arms (top row from left to right), as well as anti-spirals with one, two, and three pairs (bottom row from left to right panels). Special initial conditions have been used, which may provide further insight with regards to the robustness of biodiversity against mobility. Colours red, blue, green and grey denote rock (species A), paper (species B), scissor (species C) and empty sites, respectively, same as in figures 4 and 5. Here the mobility rate is m = 6.0 × 10⁻⁵ and the system size is N = 512². For further simulation details and the special conditions we refer to [57].

Figure 7.

Emergence of target waves owing to the application of a periodic current of all three strategies at the centre of a square lattice. The top panel shows characteristic snapshots of the spatial grid over time from left to right (note how the initial random state is gradually transformed into a global target wave). The bottom panel shows the corresponding space–time plot, where the emergence of the cone in the travelling front indicates the emergence of resonance between the periodic current and the global oscillations. Colours red, blue, green and grey denote rock (species A), paper (species B), scissor (species C) and empty sites, respectively, as in figures 4 and 5. The system size is N = 512². For further details, we refer to [139].

The impact of mobility can also be studied in off-lattice simulations, where the radius of the interaction range crucially affects coexistence. In particular, a transition from coexistence to extinction is uncovered with a strikingly non-monotonous behaviour in the coexistence probability [140]. Close to the minimal value of the coexistence probability and under the influence of intermediate mobility, it is even possible to observe transitions between spiral and plane waves over time, as depicted in fig. 2 of [140]. In general, strong mobility can either promote or impair coexistence depending on the interaction range, although diversity is quite robust across large regions of the parameter space [52]. Importantly, these phenomena are absent in any lattice-based model, and it is possible to explain them using PDEs similar to equation (3.2). Off-lattice simulations of RPS and related evolutionary games are still relatively unexplored. Partly, this is surely owing to significantly higher computational resources that are needed to simulate such systems in comparison with traditional spatial games, yet, on the other hand, the efforts may be well worthwhile as the off-lattice set-up is closer to some actual conditions than lattice-based simulations.

Another interesting phenomenon to consider that is directly related to mobility is epidemic spreading. The effects of epidemic spreading on species coexistence in a spatial RPS game have been studied in [51,54,141]. It has been shown that in the absence of epidemic spreading, there exist extinction basins that are spirally entangled for m > m_c, whereas basins of coexistence emerge for m < m_c. Moreover, it has been discovered that intraspecies infection promotes coexistence, whereas interspecies spreading fails to evoke the same effect [51,54,141].

To close this section, we note that many more interesting and counterintuitive phenomena are observable when more than three strategies compete for space. In a four-strategy system, for example, when a two-strategy neutral alliance may emerge, the intensity of mobility is able to directly select the winning solution of the subsystem [142]. More specifically, if the mobility is small the RPS-type cyclic dynamics is dominant, whereas if the mobility is large, the two-strategy neutral alliances take over. Further fascinating examples related to cyclical interactions between more than three strategies are reviewed in §5.

3.3. Metapopulation and nonlinear mobility

In §3.2, as in the vast part of the literature, the two-dimensional RPS models were implemented on square lattices whose nodes were either empty or occupied by an A, B or C player, and the interactions described by equations (2.1), (2.3), (3.1) involved agents on neighbouring sites. Inspired by the experiments of [2,16,24], an alternative metapopulation modelling approach allowing further analytical progress has recently been put forward [91,136,143]. In the metapopulation formulation, the lattice consists of an array of L × L patches each of which comprises a well-mixed (sub)population of constant size N (playing the role of the carrying capacity) that consists of A, B or C individuals and empty spaces. The RPS game is then implemented according to the general reactions equations (2.1)–(2.4) between individuals within the same patch (intrapatch reactions), whereas the spatial degrees of freedom are granted by allowing the individuals to move between neighbouring patches. At this point, it is interesting to divorce the pair-exchange (with rate γ_e) from the hopping (with rate γ_d) process, according to the scheme (X, Y = A, B, C)

3.3

As shown in [91,144], this leads to nonlinear mobility when γ_e ≠ γ_d (see also [50,109]). In biology, organisms are, in fact, known not to simply move diffusively, but to sense and respond to their environment. Here, equation (3.3) allows us to discriminate between the movement in crowded regions, where mobility is dominated by pair-exchange, and mobility in diluted regions where hopping can be more efficient.

The generic two-dimensional metapopulation model of equations (2.1)–(2.4), supplemented by equation (3.3), is ideally suited to capture stochastic effects via size expansion in N [96] and, to lowest order (where all fluctuations are neglected) and in the continuum limit [144], yields the following PDEs

3.4

with γ_nl = γ_d − γ_e and the notation a ≡ a(r, t), b ≡ b(r, t) and c ≡ c(r, t). We note that nonlinear diffusive terms of the form a∇²(b + c) − (b + c)∇²a arise when γ_nl ≠ 0. In [91,137,144], the solutions of equation (3.4) have been shown to accurately match the stochastic simulations of the metapopulation model performed using a spatial version of the Gillespie algorithm [97]. It has been shown that the PDEs (3.4) accurately capture the properties of the lattice metapopulation model as soon as the carrying capacity is , whereas their derivation assumes [91,144]. Quite remarkably, it turns out that certain outcomes of stochastic simulations with N = 2−16 are qualitatively reproduced by the solutions of equation (3.4) [145].

In §3.4, we shall discuss how the PDEs (3.4) can be used to derive a CGLE, with an effective linear diffusive term, from which many of the spatio-temporal properties of the spiralling patterns that we have encountered can be predicted. Yet, some intriguing features of the model triggered by nonlinear diffusion cannot be captured by the CGLE. An interesting effect of nonlinear mobility appears in the RPS model of equations (2.1)–(2.4) at low mutation rates when γ_d > γ_e: as illustrated in figure 8, in this case, the nonlinear mobility promotes the far field breakup of spiral waves and enhances their convective instability [91,144,145].

3.4. Complex Ginzburg–Landau equation

The CGLE is a celebrated and well-studied nonlinear equation commonly used in the physics community to describe a variety of phenomena spanning from superconductivity, superfluidity, liquid crystals to field theory [146]. After the introduction of a phenomenological time-dependent Ginzburg–Landau theory for superconductors [147], the CGLE was derived in the context of fluid dynamics [148,149] and is known for its ability to predict the emergence of complex coherent structures, such as spiral waves in two-spatial dimensions, as well as chaotic behaviour. Its fascinating properties have received considerable interest and are the subject of numerous reviews [146,150]. It has recently been realized that many of the spatio-temporal properties of the two-dimensional RPS models can be understood by means of a suitable mapping onto a CGLE [3,45,133].

The first attempts to use the CGLE in the context of spatial RPS games have been carried out in [3,45,133] for the model with dominance-removal reactions (2.1), whereas z = μ = 0, and pair exchange in equation (3.1). At mean field level, this model is characterized by an unstable coexistence fixed point s* with one stable eigenvector, and the dynamics quickly approaches a manifold tangent to the plane normal to the stable eigenvector of s*. In [3,45,133], that manifold was computed to quadratic order around s*. On such manifold, the dynamics becomes two-dimensional, and the mean field flows approach the absorbing boundaries of the phase portrait where they linger and form a heteroclinic cycle [87,93]. Exploiting a suitable nonlinear (‘near-identity’) transformation, effectively mapping (a,b,c) − s* onto the new variables z = (z₁,z₂) and by retaining terms up to cubic order in z, the rate equations (2.5) were then recast in the normal form of a Hopf bifurcation. As explained in [28,133], where details of the derivation can be found, space and movement were finally reinstated by adding a linear diffusive term to rate equations in the z variables yielding a CGLE. It has to be noted that such an approach relies on three uncontrolled approximations:

1. It consists of approximating heteroclinic cycles by stable limit cycles resulting from a fictitious Hopf bifurcation.

2. While the new variables z₁ and z₂ are assumed to be small quantities, and therefore the mapping is expected to be suitable near s*, its use is not restricted to the vicinity of s*.

3. Even though mobility is mediated by pair exchange in equation (3.1), which results in linear diffusion in the PDEs (3.2), the nonlinear mapping (a,b,c) − s* → z generates nonlinear diffusive terms that are ignored by the CGLE.

Notwithstanding, this approach has been able to explain many qualitative features of the above model and, upon adjusting one fitting parameter, even obtain approximations for the wavelength of spiralling patterns and the velocity of the propagating fronts [3,45,133]. This treatment was then extended to the case of dominance-removal and dominance-replacement cyclic competitions with linear mobility and no mutations (p, z, γ > 0, μ = 0) [46,143]. Recently, a class of RPS-like models with more than three species (with dominance-removal and hopping but no mutations) has also been considered by generalizing this approach [151].

Recently, an alternative analytical treatment has been devised to derive perturbatively the CGLE associated with the generic class of RPS models defined by the reactions equations (2.1)–(2.4), and with mobility mediated by pair-exchange and hopping in equation (3.3) [91,144]. This alternative derivation has been carried out within the metapopulation formulation outlined in §3.3. The key observation is that the mean field dynamics of the model defined by equations (2.1)–(2.4) is characterized by a Hopf bifurcation at μ = μ_H (see §2) and by stable limit cycles when μ < μ_H. A multiscale expansion around the Hopf bifurcation is appropriate to derive a CGLE that aptly describes the fascinating oscillatory patterns arising on two-dimensional lattices when μ < μ_H. As explained in [91,137], a space and time perturbation expansion in the parameter is performed by introducing the ‘slow variables’ and by expanding the densities in powers of ε [152]. Hence, after a suitable linear transformation (a, b, c) − (a*, b*, c*) → (u, v, w), where w decouples from u and v (to linear order), one writes , and , where the functions are of order . Substituting these expressions into equation (3.4) with , where is a (complex) modulated amplitude, the CGLE is thus derived by imposing the removal of the secular term arising at order , which yields [91,137,144]

3.5

where and, after rescaling by a constant [137],

3.6

It is worth stressing that the CGLE (3.5) is a controlled approximation of the PDEs (3.4) around the Hopf bifurcation point, and it is noteworthy that it involves a linear (real) effective diffusion coefficient. The CGLE (3.5) permits an accurate characterization of the spatio-temporal patterns in the vicinity of the Hopf bifurcation point. For the sake of simplicity, we here restrict σ and ζ into [0,3]. Using the well-known phase diagram of the two-dimensional CGLE [146,153], one thus distinguishes four phases separated by the three critical values (c_AI, c_EI, c_BS) ≈ (1.75, 1.25, 0.845), as illustrated in figure 9a: the absolute instability (AI) phase in which stable spiral waves cannot be sustained (when c > c_AI), the Eckhaus instability (EI) phase in which spiral waves are convectively unstable, with spirals' arms that far from the cores first distort and then break up (when c_EI < c < c_AI), the bound state (BS) phase characterized by stable spiral waves with well-defined wavelength and frequency (when c_BS < c < c_EI), and a phase in which spiral waves annihilate when they collide (SA phase, when 0 < c < c_BS).

Figure 9.

(a) Typical snapshots of the phases AI, EI, BS, SA (from left to right) obtained from lattice simulations with parameters q = γ_e = γ_d = 1, μ = 0.02, L = 128, N = 64 and, from left to right, z = (1.8, 1.2, 0.6, 0). (b) Phase diagram of the two-dimensional RPS system (2.1)–(2.4) around the Hopf bifurcation point with contours of c = (c_AI, c_EI, c_BS) in the p − z plane, see text. We distinguish four phases: spiral waves are unstable in AI, EI and SA phases, whereas they are stable in BS phase. The boundaries between the phases have been obtained using (3.6). See [91] for full details.

The properties of the CGLE (3.5) have also be exploited to gain insights into the properties of the spatio-temporal patterns arising away from the Hopf bifurcation, where the SA phase appears to be generally replaced by an extended BS phase [137,144,145]. Yet, it has also been found that when an Eckhaus-like far-field breakup of the spiral waves occurs away from the Hopf bifurcation, very much like in figure 4 (bottom panel; see [144]).

4. Evolutionary games with spontaneously emerging cyclic dominance

Cyclic dominance occurs not only in RPS and related evolutionary games where the closed loop of dominance is explicitly imposed on the competing strategies through the directions of invasions between them (figure 1). Quite remarkably, cyclical interactions may occur spontaneously in any evolutionary game where the competing strategies are three or more, and it is also possible to observe them in two-strategy games if certain player-specific properties are time-dependent [76]. Well-known examples include the classic public goods game [31], where in addition to cooperators and defectors the third competing strategy is volunteering [66,67,154–157], rewarding [73,74,158–160] or punishing [68,69,161–164]. Further examples include the ultimatum game with discrete strategies [165], the public goods game with pool punishment [71,166–168] as well as the public goods game with correlated positive (reward) and negative (punishment) reciprocity [169].

As in the original RPS game, in these evolutionary games, the spontaneous emergence of cyclic dominance also promotes the coexistence of competing strategies, and through that it stabilizes solutions that would otherwise be unstable. As such, the spontaneous emergence of cyclic dominance is one of the main driving forces behind complex pattern formation, which, in turn, is responsible for the many differences between evolutionary outcomes reported in well-mixed and structured populations. Examples include the stabilization of reward [73] and punishment [68] in structured populations, the evolution of empathy and fairness in human bargaining [165], as well as an oscillatory coexistence of defection and cooperation with the ‘tit-for-tat’ (T) strategy [170–172]. The latter example is particularly instructive, as there is a transition from a three-strategy stationary state to a three-strategy oscillatory state that depends on the cost of the ‘tit-for-tat’ strategy. This phase transition is possible only in a structured population, and because there is a spontaneous emergence of cyclic dominance only between the three competing strategies. The complete phase diagram is shown in figure 10, where the dotted blue line delineates to two three-strategy phases. It is worth emphasizing that an oscillatory state emerges not because of a time-varying interaction network (the importance of which we have emphasized in §31 in the realm of the RPS game), but solely owing to the relations between the competing strategies as defined via the pay-off matrix.

Figure 10.

Phase diagram of an evolutionary Prisoner's Dilemma game with the ‘tit-for-tat’ (T) strategy, as obtained on a random regular graph. In addition to the pure defection (D) phase, the stationary two-strategy C + D phase, and the stationary three-strategy C + D + T phase, there also exists a parameter region where all three competing strategies coexist in an oscillatory phase (O). The spontaneous emergence of cyclic dominance therefore sustains cooperation (C) even at prohibitively high temptations to defect. For further details, we refer to [170].

Another example that merits attention is the stabilization of punishment in the realm of the public goods game. In general, punishers bear an additional cost if they wish to sanction defectors, and thus face evolutionary pressure not only from defectors, but also from pure cooperators, which effectively become second-order free-riders [173–175]. Accordingly, in well-mixed populations, punishment is evolutionarily unstable in the absence of additional mechanisms or further strategic complexity. In structured populations, however, punishment alone introduces cyclic dominance such that punishing cooperators are able to invade defectors, defectors are able to invade pure cooperators, and pure cooperators are able to invade punishers [68]. Even more complex evolutionary outcomes are possible in the case of pool punishment [71], where one ‘strategy’ in the closed loop of dominance can be an alliance of two strategies (for further details, we refer to §4.3). Interestingly, the introduction of self-organized punishment, when the fine increases proportional to the activity of defectors, prevents the spontaneous emergence of cyclic dominance and results in a more effective way of punishment [176]. In the latter case, however, the stability of punishers is not directly challenged by pure cooperators (second-order free-riders), as the punishing activity (and the additional cost of sanctioning) ceases in the absence of defectors.

4.1. Time-dependent learning

As we have noted previously, cyclical interactions may occur spontaneously not only if the competing strategies are three or more, but also in two-strategy games if certain player-specific properties are time-dependent. One such example concerns time-dependent strategy learning ability of players in the spatial Prisoner's Dilemma game [76]. In a preceding work [177], it has been shown that a limited teaching activity of players after a successful strategy pass modifies the propagation of strategies in a biased way and can promote the evolution of cooperation in social dilemmas. A significantly different effect, however, can be observed if the learning ability of players is time-dependent [76]. A temporarily impaired learning activity generates cyclic dominance between defectors and cooperators, which helps to maintain the diversity of strategies via propagating waves, as illustrated in figure 11. Consequently, cooperators can coexist with defectors even under extremely adverse conditions. This result is particularly inspiring because it indicates the possibility of a time-dependent player-specific property to effective replace a third strategy in a closed loop of dominance. Hence, cyclic dominance can emerge only between two competing strategies. Other works on dynamical learning in games with cyclic interactions, where agents sample a finite number of moves of their opponents between each update, have shown how demographic fluctuations can lead to noise-sustained cyclic orbits [178,179].

Figure 11.

Spontaneous emergence of cyclic dominance due to the introduction of time-dependent learning ability of players in the spatial Prisoner's Dilemma game. Cooperators (blue) and defectors (red) become entailed in a close loop of dominance, which is mediated by the time-varying learning ability. Pale (dark) shades of blue and red encode low (high) learning ability of cooperators and defectors, respectively. Travelling waves emerge from an initial cluster of cooperators located in the centre of the square lattice (not shown). From there on, the waves spread across the whole population (from (a) towards (c)), thus allowing the survival of cooperators even at very high temptations to defect. For further details, we refer to [76].

4.2. Voluntary participation

Evolutionary games with voluntary participation were among the first to record the spontaneous emergence of cyclic dominance outside the realm of the traditional RPS formalism. In particular, the coexistence of cooperators and defectors owing to the presence of volunteers (or loners) was initially reported in well-mixed populations [66,67], where the cyclic invasions among the three competing strategies resulted in an oscillatory state. Inspired by Alice's Adventures in Wonderland, where the red queen found that it takes a lot of running to stand still, the emergence of oscillations that were needed to sustain cooperation was dubbed accordingly as the ‘red queen’ effect. Notably, the terms were proposed in the early 1970s [103] as an evolutionary hypothesis based on which organisms must constantly adapt and evolve, not merely to gain a reproductive advantage, but also simply to survive.

Today, volunteering is known as a viable mechanisms to avert the tragedy of the commons [180] within the realm of the public goods game. Hauert et al. [66] studied this phenomenon by assuming that individuals have three available strategies, namely unconditional cooperation, defection and volunteering, which resulted in a RPS-like evolutionary dynamics. In particular, if there are n_c cooperators, n_d defectors and n_l loners in the group, then actually only k = n_c + n_d individuals are involved in the public goods game. Defectors thus get the pay-off rn_c/k, where r > 1 is the multiplication factor that takes into account the synergetic effects of a group effort, whereas cooperators gain (rn_c/k) − 1 (here −1 is due to their contribution to the common pool, which defectors withhold). At the same time, loners who do not participate in the game, get a fixed small pay-off σ, such that 0 < σ < r − 1. Based on numerical simulations and the analysis of the replicator equation, it quickly follows that frequencies of all three strategies exhibit oscillations over time if r > 2. These theoretical results have been confirmed by human experiments performed by Semmann et al. [67], which revealed that volunteering reduces the size of groups engaged in the public goods game, and that cooperation is indeed promoted through time-dependent oscillations. The rationale is that defectors outperform cooperators in a large predominantly cooperative group, yet as soon as defectors become the predominant force, it pays more to be a loner. But as loners grow in numbers and the actual size of the group participating in the game decreases, cooperation again becomes viable, and so the loop of dominance closes.

Motivated by the pioneering human experiments of [67], a new experiment has been designed and conducted with a total of 90 undergraduate students from Wenzhou University, who participated in repeated public goods games with volunteering in groups of five. The principal goal was to measure the average frequencies of strategies over time. The pay-offs were the same as described in the preceding paragraph, thus designating σ and the multiplication factor r as the two main parameters. The results are summarized in figure 12, which confirm that all three strategies coexist within a wide range of parameter values. However, the time-averaged frequencies depend on the actual values of σ and r. As shown in figure 12a,b, defectors are most common for σ = 0.5 and r = 3.0, which reverses in favour of loners for σ = 1.5 and r = 3.0, thus experimentally confirming the expected promotion of loners as σ increases. The dominance of loners is even more remarkable for σ = 2.5 and r = 3.0, but shifts towards cooperators for σ = 2.5 and r = 6.5, as shown in figure 12b. This, in turn, confirms the evolutionary advantage of cooperators that is warranted by higher multiplication factors. Interestingly, the outcome of these experiments indicates that the coexistence of the three strategies is more widespread than predicted by theory. For example, all three strategies coexist even for σ > r − 1 (see σ = 2.5 and r = 3.0 case in figure 12b), which may be attributed to the bounded rationality and emotions of participants that are not accounted for in simulations.

Figure 12.

Widespread coexistence of cooperators, loners and defectors in an experimental public goods game with volunteering. Columns displays the time-averaged frequencies that were chosen by the 90 participants of this human experiment, depending on the pay-off of loners σ and the multiplication factor r. (a) The multiplication factor r = 3.0 was set fixed, whereas two different values of σ were considered (see figure legend). (b) The pay-off of loners σ = 2.5 was set fixed, whereas two different values of r were considered (see figure legend). Importantly, the coexistence of all three strategies is significantly more widespread than predicted by theory, which may be attributed to the bounded rationality and emotions that are not accounted for in simulations. The whiskers in both panels show the standard error of the displayed time-averaged frequencies (rates).

Aside from the theoretical explorations of the public goods game with volunteering in well-mixed populations and the above-described human experiments, the public goods game with volunteering in structured populations also received its fair share of attention. In the light of the reviewed results on the spatial RPS game, it is perhaps little surprising that structured populations also maintain the survival of all three competing strategies, but they do so not through the emergence of oscillations, but rather through time-wise constant fractions of cooperators, defectors and loners [154]. However, the topology of the interaction network can play a key role, much the same as by the RPS game [118,119]. Specifically, if the interaction network has sufficiently strongly pronounced small-world properties (there exists a threshold in the fraction of shortcut links that must be exceeded), then the stationary three-strategy phase gives way to an oscillatory three-strategy phase [155,181]. Further strengthening the analogies between the RPS game and the public goods game with volunteering, we note that oscillations in the latter can also be evoked not only through changes in the topology of the interaction network, but also through changes in the pay-offs, as illustrated in fig. 1 of [181].

Interestingly, the so-called joker strategy, introduced by Arenas et al. [182], is conceptually different from the loner strategy, but has a similar impact on the evolution of cooperation in social dilemmas. In particular, in finite well-mixed populations oscillations emerge spontaneously, and this is irrespective of the system size and the type of strategy updating [77,182], thus adding yet further support to the relevance of cyclic dominance in evolutionary games.

4.3. When three competing strategies are more than three

If the competing strategies in an evolutionary game are more than two (with some exceptions, as described in §4.1), this opens up the possibility for the spontaneous emergence of cyclic dominance with much the same properties that characterize the classical RPS game. Sometimes, however, three competing strategies are more than three, because some subsystem solutions, typically composed of a subset of the three original strategies, manifest as an additional ‘strategy’. The fourth ‘strategy’ is not exactly equivalent to the original three strategies because it only emerges during the evolutionary process, yet its involvement in the closed loop of dominance is just as important. These rather exotic solutions, which can be observed only in structured populations, have recently been reported in the spatial public goods game with pool punishment [71], and in the spatial ultimatum game with discrete strategies [165].

In the spatial public goods game with pool punishment, we initially have three competing strategies, which are cooperators (C), defectors (D) and pool punishers (P). Quite remarkably, at certain parameter values (see [71] for details), defectors and pool punishers form a stable alliance, which effectively manifests as a strategy, and this strategy forms a closed loop of dominance with cooperators and defectors. Cyclic dominance ensues, where in addition to the two pure strategies C and D, the D + P alliance is the third competing strategy. As a consequence, the characteristic spatial patterns feature propagating fronts made up of all the mentioned competitors, as illustrated in figure 13a.

Figure 13.

In structured populations, three competing strategies can in fact be more than three. The spontaneous emergence of cyclic dominance can be driven by pure strategies, as well as by stable alliances of a subset of pure strategies, which in the closed loop of dominance act just the same. (a) Cyclic dominance between cooperators (blue), defectors (red), and a stable alliance between defectors and pool punishers (red and green). Here, defectors beat cooperators, who beat the alliance of defectors and pool punishers, who in turn beat pure defectors [71]. Note that one ‘strategy’ in the closed loop of dominance is thus an alliance of two strategies. (b) Cyclic dominance in the spatial ultimatum game with discrete strategies [165]. Two emphatic strategies, E₁ (light blue) and E₂ (navy blue), compete with the strategy A (red) that is characterized by a particular (p, q) pair. Here, E₁ beats the stable alliance of E₂ + A, who in turn beats E₂, who in turn beats E₁, thus closing the loop of dominance. As in the spatial public goods game with pool punishment, here one ‘strategy’ in the loop is an alliance of two strategies.

The peculiarity of the above-described solution indicates that it may not be satisfactory to describe the actual state of a three-strategy system by a single ‘point’ in a ternary phase diagram. Namely just the fractions of the competing strategies do not necessarily determine the actual state of the system in an unambiguous way, and this is because spatiality offers an additional degree of freedom in comparison with solutions that can be observed in well-mixed populations. As described in [71], this new degree of freedom may give rise to an additional ‘strategy’ that is as such not present at the start of the evolutionary process.

Importantly, although such evolutionary solutions may at first glance appear as special and perhaps even limited outcomes of a particular evolutionary game, we emphasize that they may in fact be common place. To corroborate this, we review a completely different evolutionary game, namely the spatial ultimatum game with discrete strategies [165]. Unlike in the public goods game, in the ultimatum game, the focus is not on the aversion of the tragedy of the commons, but rather on the evolution of empathy and fairness [183]. While discrete strategies still cover the whole (p, q) proposal–acceptance parameter plane, the discreetness takes into consideration the fact that in reality these levels are coarse-grained rather than continuous. Thus, instead of the infinite number of continuous strategies, we have N discrete strategies E_i, defined according to their (p, q) interval (see [165] for details). If p = q, we have N different classes of empathetic players who can compete with an additional strategy A that is characterized by an arbitrary (p, q) pair. Most crucially, the finite number of competing strategies enables pattern formation, and as illustrated in figure 13b, cyclic dominance may emerge spontaneously where a member in the closed loop of dominance can be an alliance of two strategies. This example, in addition to the previously described one concerning the spatial public goods game with pool punishment [71], thus corroborates the fact that in structured populations three competing strategies can in fact be more than three.

5. Cyclic dominance between more than three strategies

Extensions of the classical RPS game to more than three competing strategies are straightforward (figure 1), and in addition to their obvious appeal from the theoretical point of view [81,94,121,151,184], such games also find actual applicability in describing competition in microbial populations [82] and larger ecological food webs and communities [22,23,185–188]. Based on the review of results concerning the RPS and related three-strategy evolutionary games with cyclic dominance, it is little surprising that the complexity of spatial patterns, and indeed the complexity of evolutionary solutions in general, increases drastically as we increase the number of strategies that form a closed loop of dominance. The spontaneous emergence of defensive alliances and numerous stable spatial distributions that are separated by first and second-order phase transitions are just some of the phenomena that one is likely to observe.

The reason behind the mushrooming number of stable solutions with the rising number of competing strategies lies in the fact that the solutions of subsystems, where some strategies are missing, can also be solutions of the whole system. The final stationary state can thus be determined not only by the competition of individual strategies, but also by the competition of subsystem solutions (defensive alliances) that are characterized by their own composition and spatio-temporal distribution of strategies (we have already reviewed two such examples in figure 13). Consequently, the general understanding of evolutionary games with many competing strategies requires the systematic analysis and comparison of all possible subsystem solutions, i.e. solutions that are formed by just a subset of all available strategies.

The simplest generalization of the RPS game is towards a four-strategy cyclic dominance game [47,189–191]. As reported in [47], the formation of spiral patterns takes place only without a conservation law for the total density, and like in the three-strategy case, strong mobility can destroy species coexistence. Cyclic dominance among four strategies also gives rise to so-called neutral strategies, which do not invade each other—an option that is not available among three competing strategies. In the absence of mixing, neutral strategies can easily result in population-wide frozen states, where only those strategies that do not invade each other are present. More precisely, the coexistence of all four strategies is limited to a rather narrow range of invasion rates, whereas, otherwise, the system evolves into a frozen state where only neutral strategies are present [79]. This observation further supports the fact that the topology of the food web alone cannot unambiguously determine the evolutionary outcome of cyclical interactions. In addition, the invasion rates and the interaction range are likewise crucial parameters that determine the transitions from coexistence to uniformity among cyclically competing strategies [188,192,193].

Globally synchronized oscillations in a four-strategy cyclic dominance game are also possible, as reported in [121] and presented in figure 14. In fact, Rulquin & Arenzon [121] have shown that such coexistence phases require less long-range interactions than three-strategy phases, although intuitively one would expect the opposite. Because the emergence of long distance, global synchronization states is thus possible even when the food web has multiple subloops and several possible local cycles, this leads to the conjecture that global oscillations might be a general characteristic, even for large, realistic food webs.

Figure 14.

Globally synchronized oscillations in a four-strategy cyclic dominance game [121]. Depicted are simplex representations of the time evolution of the four densities for three different combinations of the fraction of long-range links Q and the heterogeneity of the food web χ (see legend). Starting from the initially random homogeneous state (black dot), the system evolves (left) towards finite size, stochastic fluctuations around the asymptotically stable fixed point (the amplitude of these fluctuations decreases with increasing system size [192]), (middle) towards global oscillations, where the asymptotic state is a limit cycle whose perimeter may be used as an order parameter for the transition, or (right) towards global oscillations, where the orbit approaches the heteroclinic one and the average perimeter is close to the maximum value. We refer to [121] for further details.

In addition to the coexistence of all competing strategies, the emergence of ‘frozen’ states, and global oscillations mentioned above, curvature-driven domain growth is also possible when the evolutionary dynamics is governed by interactions of equal strength among more than three strategies [56,59,60]. In this case, the time-dependence of the areas of different domains, which can be quantified effectively by means of the length of domain walls, confirms to the well-known algebraic law of the domain growth process [194].

5.1. Alliances

The subject that deserves particular attention within the realm of cyclic dominance between more than three strategies is the formation of alliances. As described in the introductory paragraphs of this section, alliances play a key role in the fascinating complexity and multitude of solutions that grace such systems, particularly through the formation of subsystems solutions that together with the pure strategies form the complete solutions of the whole system.

Alliances can emerge in strikingly different ways, although the most common one is the formation of so-called defensive alliances. For example, in a four-strategy predator–prey system, two neutral strategies can defend each other from the two external invaders [195], hence the name a ‘defensive alliance’. The protective role of an alliance can also be observed when three cyclically dominating strategies are present. In this case, they can protect each other from an external invader by invading the predator of their prey [184]. We also emphasize that cyclically dominant strategies may not necessarily form defensive alliances, even if the governing food web is complex and large. For example, in figure 1d, the strategies RS + KS + SS form a subsystem solution, but this is not an alliance as it is vulnerable to the invasion of an external strategy. On the other hand, the strategies RS + KR + SK form an actual alliance, and a membership in the latter can effectively protect all three strategies within the loop from an external invasion.

Moreover, if the number of competing strategies is high enough, it is also possible to observe the competition between the different alliances [81]. Alliance-specific invasion rates, however, can break the symmetry of the food web, and subsequently only a single alliance survives [196]. The complexity of a six-species predator–prey food web with alliance-specific invasions rates also enables noise-guided evolution within cyclical interactions [80], which is deeply rooted in short-range spatial correlations and is conceptually related to the coherence resonance phenomenon in dynamical systems via the mechanism of threshold duality [197–200].

When it comes to the study of alliances within games of cyclic dominance, the general question to answer is which strategies are capable of forming alliances? As we have reviewed above, two strategies are sufficient to form an alliance, but of course three or even more strategies could also form one. The phase diagram presented in figure 15 demonstrates that, in fact, all the mentioned alliances can be present in the stable solution of the system, depending also on the parameter values that determine invasion and mixing in the studied six-species predator–prey system [201]. The key property that determines whether an alliance is viable or not is its ability to invade other alliances or resist such an invasion itself. Similar to the competition among pure strategies, here too the appropriate approach is to measure directly the velocity of invasion between competing domains that contain specific subsystem solutions [63]. The use of special initial conditions, like the one illustrated in figure 16 (remnants of the straight vertical interface that separated two three-strategy cyclic alliances can still be clearly inferred), is thereby very useful and recommended. Unlike pure strategies, however, here we first need to partition the population and wait for the appropriate subsystem solutions to emerge before removing the barriers along the interfaces.

Figure 15.

Who can form an alliance? The displayed phase diagram of stable solutions in dependence on the invasion rate (α) and the strength of mixing (X) in a six-species predator–prey system reveals that in fact all possible alliances between two or more competing strategies can be viable at specific parameter regions [201]. In the upper left corner, the three-strategy cyclic alliance 0 + 2 + 4 or the three-strategy cyclic alliance 1 + 3 + 5 will dominate the system. In the parameter region marked by T ^C(D), on the other hand, three alliances, each consisting of two-neutral strategies, play the RPS game at a higher level, i.e. at the level of competing alliances (rather than pure strategies). Within the shadowed parameter region all the mentioned alliances coexist.

Figure 16.

Alliances may also form spontaneously, without support from specific relations within the governing food web. This is particularly common in games that are played in groups [31], where the pay-offs stemming from the multi-point interactions may give rise to relations between the competing strategies that make the formation of an alliance evolutionary most advantageous. One particular such example concerns the spatial public goods game with pool (O; light green) and peer (E; dark green) punishment, where the three-strategy cyclic alliance D + C + O and the three-strategy cyclic alliance E + O + D emerge spontaneously. Subsequently, these two alliances compete for space effectively as ‘strategies’, thereby giving rise to complex yet beautiful spatial patterns. Cooperators are blue, whereas defectors are denoted red.

Importantly, the formation of alliances is not always the consequence of specific relations within the governing food web. Alliances may also emerge spontaneously during the evolutionary process, for example owing to pay-off relations between the competing strategies that are owing to multi-point (group) interactions. The phenomenon was observed in the spatial public goods game, where in addition to cooperators and defectors both pool (O) and peer (E) punishers competed for space [202]. Figure 16 illustrates an example of the spontaneous emergence of two three-strategy cyclic alliance. Notably, either the D + C + O or the E + O + D alliance can be stable solutions of the whole system at specific values of the punishment cost and fine. But if they meet, as displayed in figure 16, the E + O + D alliance will invade the D + C + O alliance. Remarkably, the invasion process between the two alliances is effectively the same as it would be between two pure strategies.

As by the competition among pure strategies [3], by the competition of alliances too the role of mobility (or diffusion) is likewise important. Within the framework of a nine-species predator–prey model [184], it has been shown that below a critical mobility rate, the system exhibits expanding domains of three different defensive alliances, each consisting of three cyclically dominant species. Owing to the neutral relationship between these three defensive alliances, a voter-model-type coarsening starts [203], and after a sufficiently long time, only one alliance survives if the system has finite size. Above the critical mobility rate, however, the three defensive alliances can all coexist [44]. In a way, the effect of high mobility is thus the opposite to the one observed in the RPS game. There high mobility rates jeopardize diversity and lead to the survival of a single strategy [3], whereas the coexistence of defensive alliances may receive unexpected support. Further related to the impact of mobility on the survival of alliances, a recent study has demonstrated that mobility-dependent selection can prefer either a three-strategy cyclic alliance or a neutral pair of two-strategy alliances within a four-strategy system [142].

At this point, we again emphasize that it is not enough to simply separate random mixtures of selected strategies that are involved in particular alliances and monitor the evolution. Before the barriers along the interfaces are removed, a sufficiently long relaxation time has to be taken into account, which allows the actual subsystem solutions (rather than just the random mixtures of the involved strategies) to emerge. Only then will the actual competition between the alliances unfold and give correct insights into the power relations between them. In the absence of such a procedure, it may easily happen that one of the considered subsystem solutions cannot evolve fast enough, for example owing to requiring a longer relaxation time than another subsystem solution, and thus it becomes impossible to properly compare their viability. A specific example to that effect concerns a five-strategy predator–prey system [63,204], where if the mobility is strong enough the dominance between the competing alliances can vanish simultaneously. This phenomenon is then also accompanied by a divergence in the density fluctuations of individual strategies. In the absence of mobility, however, the dominance between the alliances does not vanish simultaneously, and only a sharp peak in the density fluctuations can be observed instead of the power-law divergence. The phenomena observed in the realm of this five-strategy predator–prey system in fact outline a more general type of behaviour, namely the mobility-induced reversal of the direction of invasion between the competing alliances, thus highlighting yet again that the topology of the food web does not always (in fact, it does so only rarely) determine the final evolutionary outcome in structured populations.

6. Conclusion and outlook

As we hope this review clearly shows, cyclical interactions are a fascinating subject, which have the power to captivate with the complexity and beauty of the governing evolutionary dynamics, and which in addition to this has many real-life applications that we have only begun to fully appreciate. We have reviewed main results in well-mixed populations with the focus on oscillatory dynamics and the extinction to absorbing states owing to finite system size. We have also reviewed results on the classical RPS game in structured populations. We have first focused on the important role of the topology of the interaction networks, which may induce oscillatory states, although the latter can also be evoked by changes in the game parametrization. We have also focused extensively on mobility, which has many practical implications as in real-life prey and predators are frequently on the move in order to maximize their chances of survival. We have emphasized that mobility can either promote or jeopardize biodiversity, and it may give rise to fascinatingly rich pattern formation, including spiral and target waves, multi-armed spirals with two, three and four arms, as well as anti-spirals. Mobility can also affect the basins of coexistence and extinction through intraspecies and interspecies epidemic spreading. An alternative metapopulation modelling approach is also reviewed, where the distinction of pair-exchange and hopping processes allows us to discriminate movement in crowded and in diluted regions. The resulting nonlinear mobility promotes the far field breakup of spiral waves and enhances their convective instability. Within this metapopulation formulation, we have also derived perturbatively the CGLE.

In addition to the review of the well-mixed and spatial RPS game, we have also given special attention to evolutionary games where cyclic dominance emerges spontaneously, for example between only two competing strategies through time-dependent learning, among three strategies in public goods game with volunteering, punishment or reward as well as by means of the introduction of the joker strategy. We have reviewed how alliances may effectively act as strategies and close the loop of dominance in three-strategy evolutionary games, including the public goods game with pool punishment and the ultimatum game with discrete strategies. From there onwards, we have surveyed cyclic dominance when the competing strategies are more than three, where we have emphasized the possibility of competition between alliances. We have highlighted the importance of the correct preparation of special initial conditions that allow proper monitoring of the invasion relations between the alliances, and we have given prominent exposure to the fact that alliances may also form spontaneously, without support from specific relations within the governing food web.

There are still unexplored problems related to cyclic dominance in evolutionary games that merit further attention. While physics-inspired studies account for the majority of recent theoretical advances on this topic, there also exist much experimental and theoretical work that was outside the scope of this review. Competitive intransitivity and the coexistence of species in intransitive networks [205,206], as well as community-scale biodiversity in ecological systems with a large number of species [207] and the governing large competitive networks [185–187], are just some of the topics that we did not cover in detail. While we have mentioned several experiments and empirical data that corroborate the importance of cyclical interaction in the Introduction, covering this in detail could easily amount to an independent review. In addition to the often-cited examples mentioned in the Introduction, most recent experimental research suggests that we may have only just begun to scratch the surface of the actual importance of cyclic dominance in nature [83,84]. It would be desirable for the fundamental theoretical advances to become more in sync with the experiments by means of more dedicated and precise mathematical modelling. It is worth noting that cyclic dominance could be crucial not just to understand biodiversity, but also social diversity [208–210].

In terms of future fundamental research, we would like to point towards the basically unexplored subject of cyclic dominance in group interactions. In social dilemma games, the transition from pairwise to group interactions [211,212] brings with it many qualitative changes in the evolutionary dynamics (see [31] for a review), and the reasonable expectation is that the same would also hold true in games of cyclic dominance. We have recently considered the impact of different interaction ranges in the spatial RPS game, and found that the transition from pairwise to group interactions can decelerate and even revert the direction of the invasion between the competing strategies [193]. These results thus indicate that, in addition to the invasion rates and other properties of cyclic dominance games already reviewed here, the interaction range is at least as important for the maintenance of biodiversity among cyclically competing strategies, and is thus surely deserving of further attention.

Motivation for future research can also be gathered from coevolutionary games [29], where cyclical interactions can still be considered as being at an early stage of development. While initially many studies that were performed within the realm of social dilemma games appeared to be valid also for games that are governed by cyclical interactions, recent research has made it clear that this is by far not the case. The incentives are thus clearly there to reexamine the key findings, which were so far reported only for pairwise games on coevolutionary networks, also for games that are governed by cyclical interactions. We conclude by noting that devising and studying cyclic dominance systems with greater practical applicability and predictive power, including more complex interaction networks and larger numbers of competing strategies, should also be well within the scope of viable forthcoming research.

Funding statement

This work was supported by the Hungarian National Research Fund (grant K-101490), the National Natural Science Foundation of China (grants no. 61203145 and 11047012) and the Slovenian Research Agency (grants no. J1-4055 and P5-0027). B.S. is thankful for the support of an EPSRC studentship (grant no. EP/P505593/1).