A complete classification of evolutionary games with environmental feedback

Abstract

A tragedy of the commons, in which rational behavior of individuals to maximize their own payoffs depletes common resources, is one of the most important research topics in game theory. To better understand the social dilemma problem, recent studies have developed a theoretical framework of feedback-evolving game where individual behavior affects an environmental (renewable) resource and the environmental resource changes individual payoffs. While previous studies assumed that the frequency of defectors increases (prisoner's dilemma [PD] game) when the environmental resource is abundant to investigate an oscillating tragedy of the commons, it is also possible for other types of game to produce the social dilemma. In this paper, we extend the feedback-evolving game by considering not only PD game, but also the other three game structures when the environmental resource is replete for a reasonably complete classification. The three games are Chicken game where defectors and cooperators coexist through minority advantage, Stag-Hunt (SH) game with minority disadvantage, and Trivial game where the frequency of cooperators increases. In addition, we utilize a dilemma phase plane to visually track (transient) dynamics of game structure changes. We found that an emergent initial condition dependence (i.e. bistability) is pervasive in the feedback-evolving game when the three games are involved. We also showed that persistent oscillation dynamics arise even with Chicken or SH games in replete environments. Our generalized analysis will be an important step to further extend the theoretical framework of feedback-evolving game to various game situations with environmental feedback.

Significance Statement.

Game theory has been studied to understand mechanisms that promote (or prevent) cooperation in various societies. Recently, a new theoretical framework, ‘feedback-evolving game,’ has been proposed in which an individual's behavior modifies both the social construct and the environmental context. This is useful for considering a tragedy of the commons, where individual rational behaviors deplete common resources. In this paper, we provide a complete classification of possible dynamics generated by the feedback-evolving game utilizing a dilemma phase plane where we can visually track dynamic changes of game structures. It will be fruitful to combine the feedback-evolving game and the dilemma phase plane to understand various game theoretical dynamics with environmental feedback including natural resource management and voluntary vaccination in pandemics.

Introduction

Evolutionary game theory has been used to understand how cooperation arises: specifically, researchers investigate how frequencies of two behavioral strategies (‘cooperation’ and ‘defection’) dynamically change when an individual reward (fitness) is determined by a combination of its own and opponent's strategies (i.e. social makeup) (1–3). For example, in prisoner's dilemma (PD) game, the defection strategy always obtains a higher reward than the cooperation strategy. Thus, the defection strategy increases its frequency and will eventually become dominant in a population/society. The rewards of strategies (or the ‘game structure’) can be expressed by a payoff matrix, and frequency dynamics is described by replicator dynamics (2, 3). This theoretical framework is important to understand the social dilemma in which immediate self-interest is in conflict with long-term collective interests and disturb the promotion of cooperative behavior. Since these social dilemma situations arise not only in human societies but also in biological dynamics, evolutionary game theory has been extensively studied in diverse disciplines including economics, politics, psychology, epidemiology, public health, evolutionary biology, and ecology (1–8).

Recently, a new theoretical framework called ‘feedback-evolving game’ was proposed and developed to understand an individual action that modifies both the social makeup and an environmental variable (9, 10) (Fig. 1A). The framework considers two payoff matrices in ‘replete’ and ‘depleted’ environments and assumes that the environmental variable can affect the rewards by changing the relative contributions of the two matrices. The environmental variable, on the other hand, is affected by the strategy frequency: the cooperation strategy increases the environmental resource, while the defection strategy decreases it (Fig. 1A). Such strategy-dependent feedback is common in various settings in biological populations/communities as well as human societies (11) including microbial fixation of inorganic nutrients (12, 13), production of extracellular enzymes (e.g. siderophores: 14, enzymes that break down sugars: 15, antibiotic compounds: 16), growth of cancer cells in the tumor microenvironment (17), car driving in traffic flow (18), and voluntary vaccination during a pandemic (19). Weitz et al. (9) particularly emphasized that the feedback-evolving game is useful to consider a ‘tragedy of the commons' where individual actions to increase their own rewards eventually deplete ‘public goods/common-pool resources’ such as natural resources in fisheries and forestry as well as clean air and water in polluted environments (20). They proposed that their framework can capture an oscillating tragedy of the commons: cooperation increases the environmental resource; this in turn increases the frequency of the defection strategy; the increased defection strategy decreases the environmental resource; this promotes cooperation, and so on (9, 10) (Fig. 1A). The theoretical framework is also useful for understanding complex eco-evolutionary dynamics where rapid evolution affects contemporary ecological processes and vice versa (21–23). Therefore, the evolutionary game theory with environmental feedback (or ‘coevolutionary game theory’ (9), ‘eco-evolutionary game theory’ (10)) is becoming an important tool for understanding complex dynamics with feedbacks between organismal behavior and the surrounding environments in this era of global environmental changes (10).

Fig. 1.

A) The basic concept of feedback-evolving game. (Left) The abundance of the environmental resource (n) is increased by cooperators and decreased by defectors (Eq. 6). (Right) The payoff of the two strategies change depending on the environmental resource, and the frequency of cooperator (x) may be decreased in replete environments and increased in depleted environments (Eq. 5). The updated frequency of cooperator in turn affects the environment. B) The four game structures and dilemma strengths of multiplayer game (modified from (7)). The combination of positive and negative values of the strengths of GID ($D_g^{\mathrm{\prime }}$) and RAD ($D_r^{\mathrm{\prime }}$) shows the four game structures in the dilemma phase plane. The first, second, third, and fourth quadrants represent the PD (red), Chicken (yellow), Trivial (blue), and SH (green) game structures, respectively.

In this manuscript, we extend the evolutionary game theory with environmental feedback by utilizing a dilemma phase plane to understand how environmental feedback changes the game structure and how feedbacks with environments produce novel game dynamics (24) (Fig. 1B). The dilemma phase plane is a useful tool to visualize the game structure by using two indicators of a payoff matrix that represent players' incentives to exploit their opponents (gamble-intending dilemma [GID], the vertical axis of Fig. 1B) and not to be exploited by their opponents (risk-averting dilemma RAD the horizontal axis of Fig.1,B: 25, 26). By using the dilemma phase plane, we can classify game structures to four types: (i) PD game where the defection strategy dominates (when GID and RAD are positive in Fig. 1B), (ii) Chicken (also known as snowdrift or hawk-dove) game where the two strategies coexist via negative frequency dependence (when GID is positive and RAD is negative in Fig. 1B), (iii) Stag-Hunt (SH) game where the initial condition determines the dominant strategy via positive frequency dependence (i.e. bistability: when GID is negative and RAD is positive in Fig. 1B), and (iv) Trivial game with no social dilemma where the cooperation strategy dominates (when GID and RAD are negative in Fig. 1B). The original model of Weitz et al. (9) considered a situation where the defection strategy is favored (i.e. PD game) in replete environments and the cooperation strategy is favored (i.e. Trivial game) in depleted environments (Fig. 1A) to understand a tragedy of the commons (Figs. 2–4 of Weitz et al. (9); A₁ is PD and A₀ is Trivial in Fig. 2). To explore conditions in which a tragedy of the commons is averted, they also analyzed situations where PD, Chicken, and SH games appear in depleted environments (Fig. 5 of Weitz et al. (9); column where A₁ is PD in Fig. 2). To further develop the theoretical framework and understand the basic characteristics of dynamics in feedback-evolving game, it is natural and essential to extend the analyses to include situations where non-PD games appear in replete environments (columns where A₁ is Chicken, SH, and Trivial in Fig. 2). Indeed, it is possible to consider a situation where the cooperators and defectors (i.e. ‘free riders') coexist in replete environments (27–30). When the environmental resource is not completely ‘public’ (i.e. accessible by anyone in a society) due to, for example, spatial heterogeneity in bacteria populations where cooperators produce extracellular enzymes, the cooperators may obtain higher benefits when rare and stable coexistence arises (i.e. Chicken game) even in ‘replete’ (in this case, enzyme-rich) environments (31). Similarly, some people would not exploit fish populations voraciously even when the number of fish individuals is very large, which results in coexistence of cooperators and defectors in ‘replete’ environments. For understanding a complete picture of game dynamics with environmental feedback, we consider situations where the game structure in replete environments is Chicken, SH, or Trivial games instead of PD game (Fig. 2). We revealed that assuming Chicken game (or SH/Trivial game) instead of PD game in replete environments drastically changes dynamics of feedback-evolving game: it produces rich dynamics of bistability, which was not found in the previous studies on evolutionary games with environmental feedback, suggesting that a tragedy of the commons may be averted depending on the initial condition (Fig. 2). We also found persistent oscillation dynamics with Chicken or SH games in replete environments (Fig. 2). We further clarified how combining two games through environmental feedback can produce another game by using the dilemma phase plane: for example, combining PD and Trivial games in replete and depleted environments, respectively, can result in Chicken game (Fig. 3). Our analyses will be an important step for developing a game theoretical framework of environmentally mediated social dilemma (11) and eco-evolutionary dynamics (14, 22).

Fig. 2.

Overview of the feedback-evolving game dynamics generated by the combination of game structures $A_0$ (in depleted environments) and $A_1$ (in replete environments). ‘TOC’ stands for a tragedy of the commons and indicates dynamics toward $n=0$. ‘O-TOC’ refers to an oscillating tragedy of the commons as described by Weitz et al. (9) and indicates heteroclinic cycles. ‘PO’ denotes a persistent oscillation, indicating neutrally stable oscillations. ‘Bistability’ indicates the presence of two alternative stable states and the final outcome depends on initial conditions. ‘Averted’ signifies dynamics that avoid the tragedy of the commons, resulting in $n\ne 0$. See Supplementing Material and Tables S1–S4 for more detailed conditions. Combinations of game structures presented in Weitz et al. (9) are marked in blue text, whereas we verified all game structures, which are displayed in diagrams with pink text.

Fig. 3.

Persistent oscillations (left: A), SH-mediated heteroclinic cycles (center: B), and Chicken-type mixed strategy (right: C) in the Trivial–PD combination. The parameters are sets as $ϵ=0.1$, $\theta =2$. A) Under the payoff setting shown as Eq. (9), the state dynamics and dilemma strength show persistent oscillations with a neutral interior fixed point $(x^{*},n^{*})=\left(\frac{1}{3},\frac{1}{2}\right)$. B) Under the payoff setting shown as Eq. (10), the state dynamics and dilemma strength show a heteroclinic cycles (i.e. an oscillating tragedy of the commons) with an unstable interior fixed point $(x^{*},n^{*})=\left(\frac{1}{3},\frac{2}{3}\right)$. C) Under the payoff setting shown as Eq. (11), the state dynamics and dilemma strength converge to Chicken game with a stable fixed point $(x^{*},n^{*})=\left(\frac{1}{3},\frac{17}{42}\right)$. A1), B1), C1) The state dynamics of the frequency of cooperator (x, blue) and the abundance of the environmental resource (n, orange). A2), B2), C2) The x–n phase plane shows the dynamics with each initial conditions. The red line corresponds to the dynamics indicated in (A1), (A3), and (A4), (B1), (B3), and (B4), and (C1), (C3), and (C4). A3), B3), C3) The state dynamics of dilemma strengths of GID ($D_g^{\mathrm{\prime }}$, red) and RAD ($D_r^{\mathrm{\prime }}$, black) with game structure transition. A4), B4), C4) The dilemma phase planes in the feedback-evolving games. The shaded background colors indicate the regions of PD (red), Chicken (yellow), Trivial (blue), and SH (green) games. A4) The black line connecting $A_0$ and $A_1$ passes through the origin. $A^{*}$ denotes the dilemma strength coordinate of $A(n^{*})$that satisfies the condition of $r_1=r_2$. B4) The coordinate of $A^{*}$ is located in the SH region. C4) The coordinate of $A^{*}$ is located in the Chicken region.

Materials and methods

The game structure, or rewards determined by a pairwise combination of an individual's own strategy and an opponent's strategy, is represented by 2 × 2 payoff matrix (1–3, 32). Here, we consider the payoff matrix of pairwise games in an infinite population: we model a situation where two players are selected from an infinite population at random and they play the game. Players receive a reward depending on the selected strategies, C (cooperation) or D (defection) according to the following payoff matrix:

$$A\equiv [a_{ij}]=\begin{array}{cc}& \begin{array}{cc}\mathrm{C}& \mathrm{D}\end{array}\\ \begin{array}{c}\mathrm{C}\\ \mathrm{D}\end{array}& \left(\begin{array}{cc}R& S\\ T& P\end{array}\right).\end{array}$$ (1)

If both players cooperate, they receive the ‘reward’ R; if both defect, they get ‘punishment’ P; if one chooses cooperation while the other defects, the defector gets the ‘temptation’ T, whereas the cooperator gets the payoff of ‘sucker’ S (1–3, 32). The game structure and the social dilemma strengths of the game depend on the relative magnitudes of the payoff matrix elements P, R, S, and T (26, 33–36). Particularly, the signs of T−R and P−S determine the structure of the game (26, 33–36).

Thus, we use two indicators based on a payoff matrix that represent players' incentives to exploit their opponents (GID) and not to be exploited by their opponents (RAD) for visualization in the dilemma phase plane (Fig. 1B). The strengths of GID and RAD, $D_g^{\mathrm{\prime }}$ and $D_r^{\mathrm{\prime }}$, respectively, can be calculated from the elements of the payoff matrix (Eq. 1) as follows (26):

$$D_g^{\mathrm{\prime }}={\displaystyle \frac{T-R}{R-P},}$$ (2)

and

$$D_r^{\mathrm{\prime }}={\displaystyle \frac{P-S}{R-P}}$$ (3)

Because their denominators (R−P) are always positive by definition, the numerators determine the signs. The strength of GID is proportional to the difference of rewards between a defector and a cooperator when their opponents have the cooperative strategy (i.e. T−R: players' incentives to exploit their opponents). In the same way, the strength of RAD is determined by the difference of rewards between a defector and a cooperator when their opponents have the defective strategy (i.e. P−S: players' incentives not to be exploited by their opponents).

There are four possible combinations depending on the signs of $D_g^{\mathrm{\prime }}$ and $D_r^{\mathrm{\prime }}$: PD game ($D_g^{\mathrm{\prime }}>0$ and $D_r^{\mathrm{\prime }}>0$); Chicken game ($D_g^{\mathrm{\prime }}>0$ and $D_r^{\mathrm{\prime }}<0$); SH game ($D_g^{\mathrm{\prime }}<0$ and $D_r^{\mathrm{\prime }}>0$); and Trivial game ($D_g^{\mathrm{\prime }}<0$ and $D_r^{\mathrm{\prime }}<0$) (Fig. 1B) (7, 26, 33–36). When both of them are positive (negative), defectors (cooperators) always obtain higher rewards than their opponents because T > R and P > S (T < R and P < S), and this results in PD (Trivial) game. When $D_g^{\mathrm{\prime }}$ is positive and $D_r^{\mathrm{\prime }}$ is negative, the game between different strategies (i.e. the game between defectors and cooperators) results in higher rewards than the game between the same strategies because T > R and P < S and it promotes stable coexistence (i.e. Chicken game). When $D_g^{\mathrm{\prime }}$ is negative and $D_r^{\mathrm{\prime }}$ is positive, on the other hand, the game between the same strategies results in higher rewards because T < R and P > S and it leads to bistability (i.e. SH game). Based on the strengths of GID and RAD, we can draw a dilemma phase plane and place any payoff matrix on the phase plane for understanding the game structure (24) (Fig. 1B). By utilizing the dilemma phase plane, we will be able to better understand how game structure is changing through feedback-evolving game.

The theoretical framework of feedback-evolving game (Fig. 1A) considers two payoff matrices $A_0=\left[\begin{array}{cc}{R}_0& S_0\\ T_0& P_0\end{array}\right]$ and $A_1=\left[\begin{array}{cc}{R}_1& S_1\\ T_1& P_1\end{array}\right]$ for the depleted and replete environments, respectively (9). The relative importance of the two payoff matrices varies according to the abundance of the environmental resource (n):

$$\begin{array}{rl}A(n)& =\left[\begin{array}{cc}{A}_{11}(n)& A_{12}(n)\\ A_{21}(n)& A_{22}(n)\end{array}\right]=(1-n)A_0+n{A}_1\\ & =(1-n)\left[\begin{array}{cc}{R}_0& S_0\\ T_0& P_0\end{array}\right]+n\left[\begin{array}{cc}{R}_1& S_1\\ T_1& P_1\end{array}\right]\end{array}$$ (4)

The frequency dynamics of cooperators x ($0\le x\le 1$) is described by a replicator equation as follows (1–3):

$${\displaystyle \frac{\mathrm{d}x}{\mathrm{d}t}=x(1-x)[r_1(x,n)-r_2(x,n)]}$$ (5)

where $r_1(x,n)=x{A}_{11}(n)+(1-x)A_{12}(n)$ and $r_2(x,n)=x{A}_{21}(n)+(1-x)A_{22}(n)$ denote the frequency- and environment-dependent fitness of cooperative player 1 and of defective player 2, respectively. The frequency of the cooperation strategy increases when the fitness of cooperators is larger than that of defectors, and the speed of dynamics becomes slower when the frequency is close to 0 or 1.

The resource is assumed to be increased by cooperators and decreased by defectors as $\theta x-(1-x)$, where θ ($>0$) is the ratio of the enhancement rates to degradation rates of cooperators and defectors, respectively. Then, the dynamics of the environmental resource n ($0\le n\le 1$) is represented by:

$${\displaystyle \frac{\mathrm{d}n}{\mathrm{d}t}=ϵn(1-n)[(1+\theta )x-1]}$$ (6)

The rate of environmental dynamics is partly set by the dimensionless quantity, $ϵ$. The speed of dynamics becomes slower when the resource abundance is close to 0 or 1, or when the parameter $ϵ$ is small.

By combining the game theoretical dynamics (Eq. 5) and environmental resource dynamics (Eq. 6), we can calculate an internal equilibrium of the feedback-evolving game by assuming $r_1=r_2$:

$$(x^{*},n^{*})=({\displaystyle \frac{1}{1+\theta }},{\displaystyle \frac{\theta (P_0-S_0)+(T_0-R_0)}{\theta (P_0-S_0)+(T_0-R_0)+\theta (S_1-P_1)+(R_1-T_1)}}),$$ (7)

where $x^{*}$ and $n^{*}$ denote the equilibrium of x and n, respectively. The position of the internal equilibrium is important for classifying and understanding dynamics of feedback-evolving game in the following sections. Here, we define $A^{*}$ as the payoff matrix when the amount of environmental resource is $n^{*}$as follows:

$$A^{*}=A(n^{*})=(1-n^{*})\left[\begin{array}{cc}{R}_0& S_0\\ T_0& P_0\end{array}\right]+n^{*}\left[\begin{array}{cc}{R}_1& S_1\\ T_1& P_1\end{array}\right]$$ (8)

We analyzed the model dynamics by using analytical techniques including local stability analyses (see Supplementing Material) as well as numerical simulations of the ordinary differential equations (Eqs. 5 and 6). For understanding the effects of parameter values on feedback-evolving game dynamics in the analyses, we carefully classified R_i, S_i, T_i, and P_i (i = 0, 1) according to the four game types (Fig. 1B) and how they are combined. In addition, we checked how the θ value affects the internal equilibrium, $(x^{*},n^{*})$, and change game dynamics. We also examined how changing the $ϵ$ value affects game dynamics numerically.

Results

Trivial–PD combinations

In this section, we replicate the results of the feedback-evolving game of Weitz et al. (9) by setting $A_0$ to Trivial and $A_1$ to PD and visually demonstrate dynamical changes in game structures by utilizing the dilemma phase plane (Fig. 3). As Trivial game ($A_0$) prefers cooperation in depleted environments whereas PD ($A_1$) promotes defection in replete environments, there is an oscillating tragedy of the commons (Fig.1,A: 9). In the following three examples, we show that (i) dynamical changes of game structures can be easily understood by using the dilemma phase plane, and (ii) (transient) dynamics with the combination of two games can produce a third game, which can affect dynamics. Note that the payoff matrices presented here are examples used in the previous study (9) and we used them for ease of comparison. See Supplementing Material for more general results. We also examined the effects of the parameter value $ϵ$ on dynamics (Fig. S1) and found that the qualitative results do not depend on the speed of the feedback, $ϵ$, as shown by Weitz et al. (9).

Persistent oscillations in the Trivial–PD case

First, we consider parameter values that show persistent oscillations (neutral cycles) of the frequency of cooperators (x) and the abundance of environmental resource (n) (Fig. 3A1–2):

$$A(n)=(1-n)\left[\begin{array}{cc}{R}_0& S_0\\ T_0& P_0\end{array}\right]+n\left[\begin{array}{cc}{R}_1& S_1\\ T_1& P_1\end{array}\right]=(1-n)\left[\begin{array}{cc}5& 1\\ 3& 0\end{array}\right]+n\left[\begin{array}{cc}3& 0\\ 5& 1\end{array}\right]$$ (9)

The persistent oscillation by the feedback mechanism occurs as follows: (i) When the resource is abundant, the matrix $A_1$ (PD) becomes dominant, and thus the defectors increase (x approaches 0). (ii) The increased defectors exploit and deplete the resource (n approaches 0). (iii) The reduced resource makes the matrix $A_0$ (Trivial) dominant and the frequency of cooperators increases (x approaches 1). (iv) The cooperators increase the resource (n approaches 1), and then the state goes back to the process (i) (Figs. 1A and 3A). In such a persistent oscillation, various periodic orbits are obtained on the x–n phase plane depending on initial conditions by numerical simulations (black and red circles in Fig. 3A2). Along the cycles, the strengths of GID ($D_g^{\mathrm{\prime }}$) and RAD ($D_r^{\mathrm{\prime }}$) also show synchronized persistent oscillations (Fig. 3A3). This results in alternations of the game structure between PD and Trivial (Fig. 3A3). Mathematically, when the eigenvalues of the Jacobian matrix around the internal equilibrium are purely imaginary (i.e. the trace is 0 and the determinant is positive in the Jacobian matrix), the equilibrium point is a center and persistent oscillations around the center arise (see Supplementing Material for details). As the determinant is always positive in the Trivial–PD case, the condition for persistent oscillations is $\frac{(S_0-P_0)}{(R_0-T_0)}=\frac{(S_1-P_1)}{(R_1-T_1)}$. In the dilemma phase plane, the black line connecting $A_0$ and $A_1$ dilemma coordinates passes through the origin because the parameter setting in Eq. (9) satisfies a condition ($R_0=T_1$, $S_0=P_1$, $T_0=R_1$, and $P_0=S_1$, and thus $\frac{(S_0-P_0)}{(R_0-T_0)}=\frac{(S_1-P_1)}{(R_1-T_1)}$), which makes the dilemma strengths (Eqs. 2 and 3) at the internal equilibrium (Eq. 7) the origin (Fig. 3A4).

SH-mediated heteroclinic cycle in the Trivial–PD case

Second, we examine dynamics with the following parameter values in the Trivial–PD combination:

$$A(n)=(1-n)\left[\begin{array}{cc}3.5& 1\\ 2& 0.75\end{array}\right]+n\left[\begin{array}{cc}4& 1\\ 7& 2\end{array}\right]$$ (10)

In this case, dynamics of x, n, $D_g^{\mathrm{\prime }}$, and $D_r^{\mathrm{\prime }}$ show heteroclinic cycles (i.e. an oscillating tragedy of the commons) where cycle periods and amplitudes gradually increase (Fig. 3B1 and B2). Here, three types of game structures (i.e. PD, Trivial, and SH) emerge (Fig. 3B3) and the order in which the three game structures appear (i.e. SH→PD→SH→Trivial→SH→PD→SH→…) (Fig. 3B3) may seem strange, but this can be easily understood by the dilemma phase plane. In this parameter setting, the black line connecting dilemma coordinates of $A_0$ and $A_1$ passes through the SH (green) region (Fig. 3B4). The SH region in which the coordinate of $A^{*}$ is located shows bistable dynamics where $x=0$ and $x=1$ are locally stable equilibria and the outcome depends on the initial proportion of cooperators. This example shows that a third game (SH) emerges by combining two games (PD and Trivial), and the third game can affect dynamics of the feedback-evolving game. In this case, the bistable dynamics of SH game can produce heteroclinic cycles of the feedback-evolving game.

Chicken-type stable mixed strategy in the Trivial–PD case

Finally, we examine the following parameters in the Trivial–PD combination:

$$A(n)=(1-n)\left[\begin{array}{cc}3.5& 1\\ 2& 0.05\end{array}\right]+n\left[\begin{array}{cc}4& 1\\ 7& 2\end{array}\right]$$ (11)

Here, the dynamics converges to a stable fixed point (Fig. 3C1 and C2) where the game structure is Chicken. In transient dynamics, the order in which the three game structures appear (i.e. PD→Chicken→Trivial→Chicken→PD→Chicken→Trivial→Chicken) (Fig. 3C3) may seem strange again, but this can be understood in the dilemma phase plane. The black line passes through the Chicken (yellow) region and the game dynamics eventually converges to the Chicken region where the two strategies stably coexist (Fig. 3C4). This example shows that a third game (Chicken) emerges by combining two games (PD and Trivial), and the third game affects dynamics of the feedback-evolving game by producing a stable equilibrium.

SH–Chicken combinations

Hereafter, we expand the analysis of the previous study (9) and analyze situations where the game structure in replete environments is Chicken, SH, or Trivial instead of PD. Previous studies have assumed that defectors increase their frequency (i.e. PD game) in replete environments for considering an oscillating tragedy of the commons (9), but it is possible for cooperators and defectors (i.e. free riders) coexist in replete environments (27–30). We demonstrate that bistability is pervasive by relaxing the assumption (i.e. PD game in replete environments). We also show that the persistent oscillation dynamics occur even when the game structure in replete environments is Chicken or SH.

Our analyses in the previous section showed that there are diverse feedback-evolving game dynamics when two games ($A_0$ and $A_1$) are in diagonal positions on the dilemma phase plane (i.e. the Trivial–PD combination: Fig. 3) depending on the position of the line connecting them (either crossing the origin [Fig. 3A], the SH region [Fig. 3B], or the Chicken region [Fig. 3C]). Thus, here we focus on the SH–Chicken combination where the game structures are SH and Chicken in depleted and replete environments, respectively (see Supporting Information for detailed analyses and dynamics of other game combinations: Figs. S2–S5). We show how this combination produces diverse dynamics depending on the line connecting them (i.e. crossing the origin, the PD region, or the Trivial region) (Fig. 4).

Fig. 4.

The four patterns of dynamics in the SH–Chicken feedback-evolving game. The game structures of $A_0$ and $A_1$ are SH and Chicken, respectively. (Top: A1–D1) The x–n phase plane shows the dynamics with each initial condition where x is the frequency of cooperators and n is the amount of environmental resource. Arrows indicate the direction of state dynamics. Black dots indicate stable fixed points, white dots are unstable interior fixed points, and a gray dot is a neutral fixed point ($x^{*},n^{*}$). The white triangle on the base line (when n = 0) connected to the dotted line indicates an unstable equilibrium point $x_{m,0}$ (in SH game in depleted environments). The black triangle on the upper base line (when n = 1) connected to the dashed line indicates a stable equilibrium point $x_{m,1}$ (in Chicken game in replete environments). (Bottom: A2–D2) The dilemma phase planes in the SH–Chicken feedback-evolving game show the combinations of game structures ($A_0$ and $A_1$) where X-axis is the strength of RAD and Y-axis is the strength of GID. The red, yellow, blue, and green regions indicate PD, Chicken, Trivial, and SH games, respectively. The parameters are follows: A) $(R_0,S_0,T_0,P_0)=(8,1,7,4)$, $(R_1,S_1,T_1,P_1)=(5,4,6,1)$, $ϵ=1$, $\theta =2$; B) $(R_0,S_0,T_0,P_0)=(10,1,7,4)$, $(R_1,S_1,T_1,P_1)=(5,4,6,1)$, $ϵ=1$, $\theta =0.2$; C) $(R_0,S_0,T_0,P_0)=(8,1,7,4)$, $(R_1,S_1,T_1,P_1)=(5,4,6,1)$, $ϵ=1$, $\theta =0.2$; and D) $(R_0,S_0,T_0,P_0)=(8,1,7,4)$, $(R_1,S_1,T_1,P_1)=(5,4,10,1)$, $ϵ=1$, $\theta =0.2$.

We start the analysis by considering the situation where the environmental resource is 0 or 1 in the SH–Chicken combination. When $n=0$, SH game ($A_0$) has an unstable internal fixed point $x_{m,0}^{*}=\frac{(S_0-P_0)}{[(S_0-P_0)+(T_0-R_0)]}$, and there is positive frequency dependence (i.e. minority disadvantage: Fig. 1B). On the other hand, when $n=1$, Chicken game ($A_1$) has a stable internal fixed point $x_{m,1}^{*}=\frac{(S_1-P_1)}{[(S_1-P_1)+(T_1-R_1)]}$, and dynamics converges to the equilibrium (i.e. minority advantage: Fig. 1B). Depending on the positions of $x_{m,0}$, $x_{m,1},$ and $x^{*}$ as well as whether $\frac{S_0-P_0}{R_0-T_0}$ is larger or smaller than $\frac{S_1-P_1}{R_1-T_1}$ (i.e. whether the line connecting the two games crosses the origin, the PD region, or the Trivial region), there are four possible situations (Fig. 4). Here, nine parameters (the eight elements of the payoff matrices, $A_0$ and $A_1$, and the value of θ) determine dynamics of the feedback-evolving game. See Discussion for the effects of the parameter $ϵ$.

Simple bistability in the SH–Chicken combination

If the internal equilibrium of Chicken game in replete environments is larger than the internal equilibrium with environmental feedback ($x_{m,1}>x^{*}$), there is bistable dynamics and the initial values of x and n determine whether the feedback-evolving game converges to a depleted equilibrium with no cooperators (i.e. a tragedy of the commons: $n=0$) or a replete equilibrium where cooperators and defectors coexist ($(x,n)=(x_{m,1}^{*},1)$) (Fig. 4A). Unlike the Trivial–PD combination (Fig. 3), even when $A^{*}$ is at the origin in the dilemma phase plane (Fig. 4A2), the internal fixed point $(x^{*},n^{*})$ is unstable and there are no persistent oscillations (Fig. 4A: but see Bistability with a persistent oscillation section).

Bistability with an internal equilibrium in the SH–Chicken combination

If the internal equilibria in depleted and replete environments are smaller than the internal equilibrium with environmental feedback ($x_{m,0},x_{m,1}<x^{*}$) and $\frac{S_0-P_0}{R_0-T_0}>\frac{S_1-P_1}{R_1-T_1}$, the feedback-evolving game shows bistability with an internal equilibrium. Depending on the initial values of x and n, the feedback-evolving game reaches a depleted equilibrium with no cooperators (i.e. a tragedy of the commons: $n=0$) or an internal equilibrium where cooperators and defectors coexist ($(x,n)=(x^{*},n^{*}))$ (Fig. 4B). The line connecting $A_0$ and $A_1$ passes through the Trivial region and $A^{*}$ is at the Chicken region (Fig. 4B2). This is phenomenologically similar to the results of the laboratory experiments of Sanchez and Gore (15) where a cooperative yeast strain produces an extracellular enzyme that breaks down sugars while a defective strain exploits the sugar without extracting the enzyme. They observed bistable eco-evolutionary dynamics to population extinction due to the dominance of the defective strain or coexistence of the cooperative and defective strains as our theoretical dynamics (Fig. 4B1). Notably, there was transient spiral dynamics to the coexistence equilibrium in the experiments, which agrees with our simulations (Fig. 4B1).

Bistability with a persistent oscillation

If the internal equilibria in depleted and replete environments are smaller than the internal equilibrium with environmental feedback ($x_{m,0},x_{m,1}<x^{*}$) and $\frac{S_0-P_0}{R_0-T_0}=\frac{S_1-P_1}{R_1-T_1}$, again, there is bistable dynamics. But in this case, one equilibrium is at the origin, $(x,n)=(0,0)$, and the other one is persistent oscillations with a center at $(x,n)=(x^{*},n^{*})$ (Fig. 4C). This clearly shows that a persistent oscillation arises even with Chicken (instead of PD) in replete environments. The cyclic dynamics occurs when the initial proportion of cooperator is high (Fig. 4C1). The mechanism behind the oscillations is as follows: when values of x and n are large, Chicken game decreases the proportion of cooperators to the internal equilibrium without environmental feedback, $x_{m,1}$, but it decreases the environmental resource as well. Importantly, the equilibrium proportion of cooperators ($x_{m,1}$) is larger than the unstable equilibrium of SH game without environmental feedback ($x_{m,0}$), and thus, SH game increases the proportion of cooperators in depleted environments and it results in the increased environmental resource. As a result, SH and Chicken games occur alternately by crossing the origin in the dilemma phase plane (Fig. 4C).

A tragedy of the commons in the SH–Chicken combination

Finally, if $x_{m,0},x_{m,1}<x^{*}$ and $\frac{S_0-P_0}{R_0-T_0}<\frac{S_1-P_1}{R_1-T_1}$, $A^{*}$ locates in the SH region in the dilemma phase plane, and $(x^{*},n^{*})$ is an unstable interior fixed point (Fig. 4D). The line connecting $A_0$ and $A_1$ passes through the PD region and the feedback-evolving game shows the dominance of defectors in deplete environments (i.e. a tragedy of the commons: $n=0$, Fig. 4D2). Although we combined SH and Chicken here, it results in dynamics that is similar to PD where defectors dominate irrespective of the initial conditions (Fig. 4D1).

Bistability and a persistent oscillation

As shown in the previous section, combining various game structures can produce diverse dynamics (see Supplementing Material for a detailed analyses and dynamics of other game combinations) and bistable dynamics may be common (Fig. 4A–C). In this section, we introduce all game structure combinations that lead to bistable dynamics that were not observed in the previous studies (9, 10). Here, we show game combinations with bistability on the dilemma phase planes (Fig. 5C, G, and L) and the actual bistable dynamics in the x–n planes (Fig. 5A–B, D–F, and H–K). Note that we already showed bistable dynamics of Fig. 5H–J in Fig. 4A–C.

Fig. 5.

The nine patterns of bistability in the feedback-evolving game. A), B), D)–F), H)–K) Bistabilities are classified based on dynamics with each initial condition in the x–n phase plane where x is the frequency of cooperators and n is the amount of environmental resource. Arrows indicate the direction of state dynamics. Black dots indicate stable fixed points, white dots are unstable interior fixed points ($x^{*},n^{*}$), and gray dots are neutral fixed points ($x^{*},n^{*}$). C), G), L) Dilemma phase planes indicate the combinations of game structures ($A_0$ and $A_1$) in which each bistability occurs. Here, X-axis is the strength of RAD and Y-axis is the strength of GID. The red, yellow, blue, and green regions indicate PD, Chicken, Trivial, and SH games, respectively. A) An unstable internal equilibrium point and bistable equilibrium points at $(x,n)=(0,0)$ and $(1,1)$. B) Bistable equilibrium points at $(0,0)$ and $(1,1)$. D) Bistable equilibrium points at $(x^{*},n^{*})$ and $(1,1)$. E) Bistability of neutral cycles (i.e. persistent oscillations) and a stable equilibrium point at $(1,1)$. F) Bistable equilibrium points at $(x_{m,0},0)$ and $(1,1)$. H) Bistable equilibrium points at $(0,0)$ and $(x^{*},n^{*})$. I) Bistability of a stable equilibrium point at $(0,0)$ and neutral cycles (i.e. persistent oscillations). J) Bistable equilibrium points at $(0,0)$ and $(x_{m,1},1)$. K) Bistable equilibrium points at $(x_{m,0},0)$ and $(x_{m,1},1)$. A)–C) The SH-like bistability of $(x,n)=(0,0)$ and $(1,1)$. D)–G) Bistability with a stable point of $(x,n)=(1,1)$ (i.e. a cooperative society). A), B), F), H–L) Bistability with a stable point of $n=0$ (i.e. a tragedy of the commons).

The simplest dynamics with bistability would be the one with two locally stable points at the depleted point without cooperators, $(x,n)=(0,0)$, and the replete point without defectors, $(x,n)=(1,1)$, as like SH game (Fig. 5A–C). The difference between Fig. 5A and B is the presence of an unstable internal equilibrium point. This bistability occurs in the PD–Trivial, PD–SH, and SH–Trivial combinations (Fig. 5C). It is interesting to see that the SH-like dynamics emerges by combining PD and Trivial even when the line connecting the two games does not cross the SH region.

Diverse bistable dynamics arise when $A_0$ or $A_1$ is Chicken game (Fig. 4D–F and H–K). Bistability where one of the locally stable points has no defectors in replete environments, $(x,n)=(1,1)$, appears in the Chicken–SH and Chicken–Trivial combinations where $A_0$ is Chicken (Fig. 5D–G). Among them, a neutral cycle with the center at $(x,n)=(x^{*},n^{*})$ can occur when $\frac{S_0-P_0}{R_0-T_0}=\frac{S_1-P_1}{R_1-T_1}$ in the Chicken–SH combination, and this also indicates that a persistent oscillation does not need PD in replete environments (Fig. 5E).

When $A_1$ is Chicken game, on the other hand, bistability where one of the locally stable points has no cooperators in depleted environments, $(x,n)=(0,0)$, appears in the PD–Chicken and SH–Chicken combinations (Fig. 5H–J). Please note that we can see flipped dynamics of Fig. 5D, E, and F in Fig. 5H, I, and J, respectively. Bistability can also occur in the Chicken–Chicken combination (Fig. 5K) because each Chicken game has an equilibrium point of mixed strategies when $n=0$ and 1.

A complete classification of Chicken-base feedback-evolving game

For a complete classification of feedback-evolving game, we show an example where the payoff matrix of $A_1$ is fixed to Chicken game structure (Fig. 6) and we call it ‘Chicken-base feedback-evolving game.’ The PD-based feedback-evolving game was already analyzed in Fig. 5 of Weitz et al. (9), and see Supporting Information for details of SH-based and Trivial-based dynamics. Depending on the choice of A₀, there are four possibilities (Fig. 6). We showed the results of the SH–Chicken combination in the previous section (Figs. 5 and 6, lower right), and thus, we focus on the other three combinations: PD–Chicken, Chicken–Chicken, and Trivial–Chicken.

Fig. 6.

A complete classification of Chicken-base feedback-evolving game. The game structure of $A_1$ (the game structure in replete environments) is fixed to Chicken game. The x–n phase planes show dynamics with each initial condition in feedback-evolving game where x is the frequency of cooperators and n is the amount of environmental resource. Arrows indicate the direction of state dynamics. Black dots indicate stable fixed points, white dots are unstable interior fixed points ($x^{*},n^{*}$), and a gray dot is a neutral fixed point ($x^{*},n^{*}$). The white triangle on the base line (when n = 0) connected to the dotted line indicates an unstable equilibrium point $x_{m,0}$ (in SH game in depleted environments). The black triangle on the upper base line (when n = 1) connected to the dashed line indicates a stable equilibrium point $x_{m,1}$ (in Chicken game in replete environments). The dilemma phase planes show dilemma coordinates of $A_0$ and $A_1$ where X-axis is the strength of RAD and Y-axis is the strength of GID. The red, yellow, blue, and green regions indicate PD, Chicken, Trivial, and SH games, respectively. Games of $A_0$ and $A_1$ are connected by black lines on which the game structure moves along the line by environmental feedback. The first, second, third, and fourth quadrants, area-divided from the black arrow axis, represent when $A_0$ is PD, Chicken, Trivial, and SH game structures, respectively.

In the PD–Chicken combination, the stability of the fixed point of Chicken game $A_1$  $(x_{m,1},1)$ depends on the relative position to the internal equilibrium, $x^{*}$ (Fig. 6, upper right). When $x_{m,1}<x^{*}$, the feedback-evolving game results in a tragedy of the commons with the PD game structure. On the other hand, the condition $x_{m,1}>x^{*}$ can avoid a tragedy of the commons depending on the initial conditions.

Second, in the Chicken–Chicken combination, the game outcome depends on the relative positions of $x_{m,0}$, $x_{m,1}$, and $x^{*}$ (Fig. 6, upper left). When $x_{m,0}<x^{*}$, the state where the cooperative and defective strategies coexist in depleted environments, ($x_{m,0},0$), becomes (locally) stable. When $x_{m,1}>x^{*}$, on the other hand, the state where the two strategies coexist in replete environments, $(x_{m,1},1)$, becomes stable. If both conditions are satisfied, the bistable dynamics emerges (Fig. 5K), and if they are not satisfied (i.e. $x_{m,0}>x^{*}>x_{m,1}$), the interior equilibrium becomes stable.

Finally, in the Trivial–Chicken combination, the game outcome depends on the relative positions $x_{m,1}$ and $x^{*}$ (Fig. 6, lower left). When $x_{m,1}>x^{*}$, the feedback-evolving game converges to the coexistence equilibrium in replete environments, $(x_{m,1},1)$, and otherwise to the internal equilibrium with environmental feedback.

Discussion

In this manuscript, we conducted a reasonably complete classification of evolutionary games with environmental feedback by (i) utilizing the dilemma phase plane that can visually track the transition of game structures and (ii) extending the previous analysis (9) through considering Chicken, SH, and Trivial (instead of PD) games in replete environments. While previous studies tended to focus on a situation where the defection strategy is favored (i.e. PD game) in replete environments (9), coexistence of cooperators and defectors should be possible as well (i.e. Chicken game, (31)). In the same way, bistable game situation that depends on initial conditions may occur in various situations in a society, such as social contract, collective behavior, and opinion dynamics (37–39). Our analyses revealed various types of bistable dynamics (Figs. 2 and 4–6), which was not observed in previous studies on feedback-evolving game (9, 10). This suggests that a tragedy of the commons may be averted depending on the initial condition. Furthermore, we demonstrated that persistent oscillation dynamics do not need PD in replete environments: it can arise with Chicken or SH (Figs. 4 and 5). We further show that the Trivial–PD combination can generate Chicken game (Fig. 3C). Thus, environmental feedback can work as a generating function of a different game structure from inputs of two games. This change of game structures can be revealed by measuring the strengths of the two social dilemmas.

The nine types of bistable dynamics we found (Fig. 5) occur when the game structure in replete environments is Chicken, Trivial, or SH (Figs. S2–S4), and this is why the previous studies that assumed PD game in replete environments did not find bistable dynamics (9). Our analyses revealed that bistability does not need SH game, which is inherently bistable with positive frequency dependence. Note that the PD–Trivial combination in Fig. 5A has the same characteristics of dynamics in SH game with stable depleted and replete environments without cooperators and defectors, respectively. In other words, it is possible to generate SH-like dynamics by combining PD and Trivial games even without crossing the SH region in the dilemma phase plane (Fig. 5C), but this may not be captured as Chicken game produced by the Trivial–PD combination (Fig. 3C). We also found bistable dynamics with neutral cycles (i.e. persistent oscillations: Figs. 4C and 5E and I). The neutral cycles can occur when $\frac{S_0-P_0}{R_0-T_0}=\frac{S_1-P_1}{R_1-T_1}$ in the SH–Chicken and Chicken–SH combinations (see detailed conditions in Tables S2 and S3). Therefore, the persistent oscillations do not need PD in replete environments, and the game combinations that pass through the origin on the dilemma phase plane may produce a neutral cycle. Note that, however, the neutral cycles do not occur in the PD–Trivial combination as it produces the SH-like dynamics.

The assumption that Chicken game arises (instead of PD game) in replete environments is important as suggested by Taylor (29). He claimed that the public goods provision problem can be better represented by Chicken game (29). For example, ecological systems such as lakes, rivers, atmosphere, and oceans for fisheries are continuously used and exploited by humans. The problem of free riders coexisting with cooperators is unavoidable, and this applies to other issues in economics, politics, psychology, epidemiology, public health, and evolutionary biology. This is why we need to better understand Chicken-based feedback-evolving game by using theoretical models.

Recent studies have demonstrated that evolution is rapid enough to affect contemporary ecological dynamics and promote eco-evolutionary dynamics (23). Although previous studies tended to focus on adaptive evolution of exploiters (predators, parasites, parasitoids, or herbivores) or victims (prey, hosts, or plants) (23), recent studies examined how rapid evolution of cooperation (or mutualistic interactions) affects ecological dynamics and vice versa (15, 21, 22, 40–43). Thus, it will be interesting to utilize the theoretical framework of feedback-evolving game for understanding rapid evolution and eco-evolutionary dynamics with mutualistic and exploitative interactions in the wild as well as human societies. Indeed, we found indirect evidence of Chicken game in replete environments in microbial experiments. We showed that bistable dynamics (Fig. 4B) that is phenomenologically similar to results of Sanchez and Gore (15) where laboratory yeast experiments show bistable eco-evolutionary dynamics that leads to population extinction due to the dominance of the defective strain or coexistence of the cooperative and defective strains. Although they constructed specific theoretical models to explain the experimental dynamics, our results suggest that the game structures in depleted and replete environments might be SH and Chicken, and the feedback-evolving game can be a useful tool for explaining and understanding microbial experiments (15).

Feedback-evolving game is an elegant framework of game theory that generates diverse dynamics by combining just two payoff matrices and resource dynamics. However, all models have underlying assumptions and their potential limitations. In the case of feedback-evolving game, we assumed coupling of simple game dynamics between two players and simple ecological dynamics where cooperators increase the resource by using deterministic differential equations. This simple eco-evolutionary game dynamics may not be able to represent more complex situations in nature and human societies with, for example, demographic stochasticity, spatial structure, and individual heterogeneity. Therefore, it will be possible to develop the framework further to various directions by modifying the assumptions. One important topic is an integration of feedback-evolving game with ‘reciprocity rules' for understanding the evolution of cooperation with environmental feedback (44). Ito and Tanimoto (24) showed that the famous five reciprocity rules (i.e. direct and indirect reciprocity group selection kin selection and network reciprocity: 44) change the dilemma strengths of game structures on the dilemma phase plane (24). Therefore, introducing the reciprocity rules to feedback-evolving game will not only change the coordinates of $A_0$ and $A_1$, but also may change the hidden game structure that occurs during the environmental feedback.

It will also be possible to make dynamics of the environmental resource more complex as suggested by Tilman et al. (10). While the previous study considered general equations where cooperators increase the resource (Eq. 6), it will be interesting to consider more specific models for common resources. For example, we will be able to analyze how voluntary vaccination or social distancing (19, 45, 46) as a cooperative behavior can affect epidemiological dynamics by using the SIR model (47). In this case, the common environmental resource (n) will be inversely proportional to the number of infected individuals, and thus, the parameter that determines the timescale of ecological dynamics, $ϵ$, will be crucial for feedback-evolving game unlike the simple dynamics (Eq. 6, (10)). Here, replete environments indicate the (near) absence of infected individuals, but even in that situation, there will be some people who are willing to have voluntary vaccination or social distancing. This further underlines the potential importance of considering Chicken game (i.e. coexistence of cooperators and defectors) in addition to PD game in replete environments. However, it should be noted that there is significant heterogeneity among individuals in the costs associated with infection and vaccination. Therefore, using only two payoff matrices may not be able to capture the complexity in human populations. In this case, a more complex individual-based model may be more appropriate to represent feedbacks between infection and behavior dynamics in heterogeneous populations. It will also be possible to make the evolutionary component more complex by considering demographic stochasticity (48), nonlinear coupling of two payoff matrices, $A_0$ and $A_1$, as well as coupling of three or more payoff matrices for producing more complex game theoretical dynamics in future studies by using, for example, individual-based simulations estimating parameters from empirical data. In addition, as our generalized analyses revealed the potential importance of alternative stable states in feedback-evolving game, it will be fruitful to integrate theoretical techniques developed in the context of catastrophic regime shift and tipping points such as early warning signals (49, 50). As the current framework considers game dynamics between two players, it will also be important to integrate it and an N-player game or a population game (51) to understand multiple aspects of a tragedy of the commons and common-pool resource problems. Either way, our dilemma phase plane analysis will be a useful approach to visualize dynamical changes of game structures and applicable not only to a tragedy of the commons but also to various game situations with environmental feedback in diverse research disciplines.

Supplementary Material

Supplementary material is available at PNAS Nexus online.

Funding

This work was partially supported by the Japan Society for the Promotion of Science (JSPS) Grant-in-Aid for Scientific Research (KAKENHI) (nos. JP17H04731, JP19KK0262, JP21H01575, JP22H01713, and JP23KK0210 to H.I. and JP19K16223, JP20KK0169, JP21H02560, JP22H02688, and JP22H04983 to M.Y.), NIG-JOINT (96A2023 and 73A2024 to H.I.), and Australian Research Council (ARC) Discovery Project (DP220102040 to M.Y.).

Author Contributions

Conceptualization, methodology, writing—original draft, and writing—review and editing: HI and MY; investigation and visualization: HI; and supervision: MY.

Data Availability

There are no data underlying this work.