The replicator equation and other game dynamics

Abstract

The replicator equation is the first and most important game dynamics studied in connection with evolutionary game theory. It was originally developed for symmetric games with finitely many strategies. Properties of these dynamics are briefly summarized for this case, including the convergence to and stability of the Nash equilibria and evolutionarily stable strategies. The theory is then extended to other game dynamics for symmetric games (e.g., the best response dynamics and adaptive dynamics) and illustrated by examples taken from the literature. It is also extended to multiplayer, population, and asymmetric games.

Game dynamics model how individuals or populations change their strategy over time based on payoff comparisons. This contrasts with classical noncooperative game theory that analyzes how rational players will behave through static solution concepts such as the Nash equilibrium (NE) (i.e., a strategy choice for each player whereby no individual has a unilateral incentive to change his or her behavior). In general, game dynamics assume that strategies with higher payoff do better. As we will see, the limiting behavior of these dynamics (i.e., the evolutionary outcome) is often a NE with additional stability properties.

The most important game dynamics is the replicator equation, defined for a single species by Taylor and Jonker (1) and named by Schuster and Sigmund (2). The replicator equation is the first game dynamics studied in connection with evolutionary game theory, a theory that was developed by Maynard Smith and Price (3) (see also ref. 4) from the biological perspective in order to predict the evolutionary outcome of population behavior without a detailed analysis of such biological factors as genetic or population size effects. With payoff translated as fitness (i.e., reproductive success), the frequency of a strategy in a large, well-mixed single species changes under the (continuous-time) replicator equation at a per capita rate equal to the difference between its expected payoff and the average payoff of the population (Eq. 1). If each strategy payoff is constant (in particular, independent of strategy frequency), the ultimate outcome of evolution is that everyone plays the strategy with highest payoff, a result that is true for all game dynamics and not only the replicator equation. In biological terms, we have Darwin’s survival of the fittest through natural selection.

Of more interest is what happens when individual payoff depends on the actions of others (i.e., when there is an actual game involved). An early success of evolutionary game theory is then the result that an evolutionarily stable strategy (ESS) is dynamically stable for the single species replicator equation described above (1, 5). The ESS, an intuitive concept of uninvadability originally generalizing the “unbeatable” sex ratios analyzed by Hamilton (6), can be defined solely in terms of payoff comparisons (4, 7). Strategic (i.e., game-theoretic) reasoning has since taken on an increasingly prominent role in predicting both individual and population behaviors in biological systems (8).

Evolutionary game theory has long since expanded beyond its biological roots and become increasingly important for analyzing human and/or social behavior. Here, changes in strategy frequencies do not result from natural selection; rather, individuals (or societies) alter their behavior based on payoff consequences. The replicator equation then emerges from, for instance, individuals making rational decisions on how to imitate observed strategies that currently receive higher payoff. Depending on what information these decision makers have (and how they use this information), a vast array of other game dynamics are possible (9–11).

Evolutionary game theory and its corresponding game dynamics have also expanded well beyond their initial emphasis on single-species (i.e., symmetric) games with a finite set of (pure) strategies where payoffs result from random one-time interactions between pairs of individuals (also called “two-player symmetric normal form games” or, more simply, “matrix games”). In this paper, we briefly highlight some features of matrix games at the beginning of the following section before generalizing to other classes of symmetric games. These are population games, games with a continuum of pure strategies, and multiplayer games. We then consider two-player asymmetric games, including extensive form games (where pairs of individuals have a series of interactions with each other and the set of actions available at later interactions may depend on what choices were made in earlier ones) as well as asymmetric population games and games with continuous strategy spaces.

Symmetric Games

Matrix games have a finite set of m pure strategies, $\left\{e_1,e_2,\dots ,e_m\right\}$, available for individuals to play. The $m\times m$ payoff matrix A has entries $a_{ij}=\pi \left(e_i,e_j\right)$ (the payoff to $e_i$ when playing against $e_j$), for $i,j=1,\dots ,m$. To obtain the continuous-time, pure-strategy replicator equation (Eq. 1) following the original fitness approach (1), the per capita growth rate in the number $n_i$ of individuals using strategy $e_i$ at time t is taken as the expected payoff of $e_i$ from a single interaction with a random individual in the large population. That is, ${\dot{n}}_i=n_i{\displaystyle {\sum }_{j=1}^m\pi \left(e_i,e_j\right)p_j}\equiv n_i\pi \left(e_i,p\right)$, where p is the population state in the (mixed) strategy simplex ${\mathrm{\Delta }}^m\equiv \left\{(p_1,p_2,\dots ,p_m\}|{\displaystyle {\sum }_{j=1}^m{p}_j=1,0\le p_j\le 1}\right\}$ with $p_i=n_i/{\displaystyle {\sum }_{j=1}^m{n}_j}$ the proportion of the population using strategy e_i at time t. A straightforward calculus exercise yields the replicator equation on ${\mathrm{\Delta }}^m$:

$${\dot{p}}_i=p_i\left(\pi \left(e_i,p\right)-\pi \left(p,p\right)\right)\hspace{1em}\mathrm{for}\hspace{1em}i=1,\dots ,m,$$ [1]

where $\pi \left(p,p\right)={\displaystyle {\sum }_{j=1}^m{p}_i\pi \left(e_i,p\right)}$ is the average payoff of an individual chosen at random (i.e., the population mean payoff). From the theory of dynamical systems, trajectories of [1] leave the interior of ${\mathrm{\Delta }}^m$ forward invariant as well as each of its faces (12).

Theorem 1. The replicator equation for a matrix game satisfies:

• a)A stable rest point is a NE.
• b)A convergent trajectory in the interior of the strategy space evolves to a NE.
• c)A strict NE is locally asymptotically stable.

Theorem 1 is the Folk Theorem of Evolutionary Game Theory (9, 12, 13) applied to the replicator equation [see SI Appendix for definitions of technical terms in the statement of the theorem (SI Appendix, section 1) and throughout the paper]. The three conclusions are true for many matrix game dynamics (in either discrete or continuous time) and serve as a benchmark to test dynamical systems methods applied to general game dynamics and to nonmatrix evolutionary games such as those considered in the remaining sections of this paper.

The Folk Theorem means that biologists can predict the evolutionary outcome of their stable systems by examining NE behavior of the underlying game. It is as if individuals in these systems are rational decision makers when in reality it is natural selection through reproductive fitness that drives the system to its stable outcome. This has produced a paradigm shift toward strategic reasoning in population biology. The profound influence it has had on the analysis of behavioral ecology is greater than earlier game-theoretic methods applied to biology such as Fisher’s (14) argument [see also Darwin (15) and Hamilton (6)] for the prevalence of the 50:50 sex ratio in diploid species and Hamilton’s (16) theory of kin selection.

The importance of strategic reasoning in population biology is further enhanced by Theorem 2 that is based on the intuitive concept of an ESS (3) defined by Maynard Smith (ref. 7, p. 10) as a “strategy such that, if all members of a population adopt it, then no mutant strategy could invade the population under the influence of natural selection.” Maynard Smith goes on to say on the same page that the “definition of an ESS as an uninvadable strategy can be made more precise … if precise assumptions are made about the evolving population.” In a matrix game, if most individuals in the population use $\mathit{p}*\in {\mathrm{\Delta }}^m$ with the rest using a mutant strategy p, then the mutant will go extinct in this two-strategy model (SI Appendix, section 1) if and only if

• i)$\pi \left(p,\mathit{p}*\right)\le \pi \left(\mathit{p}*,\mathit{p}*\right)$ (NE condition) and
• ii)$\pi \left(\mathit{p}*,p\right)>\pi \left(p,p\right)$ if $\pi \left(p,\mathit{p}*\right)=\pi \left(\mathit{p}*,\mathit{p}*\right)$ (stability condition).

$\mathit{p}*\in {\mathrm{\Delta }}^m$ is then an ESS if it satisfies these two conditions for all other $p\in {\mathrm{\Delta }}^m$.

Theorem 2.

• a)$\mathit{p}*$ is an ESS of a matrix game if and only if $\pi \left(\mathit{p}*,p\right)>\pi \left(p,p\right)$ for all $p\in {\mathrm{\Delta }}^m$ sufficiently close (but not equal) to $\mathit{p}*$.
• b)An ESS $\mathit{p}*$ is a locally asymptotically stable rest point of the replicator equation.
• c)An ESS $\mathit{p}*$ in the interior of ${\mathrm{\Delta }}^m$ is a globally asymptotically stable rest point of the replicator equation.

The equivalent condition for an ESS contained in Theorem 2a is the more useful characterization when generalizing the ESS concept to other evolutionary games. It is called “locally superior” (17), “neighborhood invader strategy” (NIS) (18), or “neighborhood superior” (19). One reason for the different names for this concept is that there are several ways to generalize local superiority to other evolutionary games and these have different stability consequences.

The most elegant proof (5) of the stability statements in Theorem 2 b and c shows that $V\left(p\right)\equiv {\displaystyle \prod p_i^{{\mathit{p}}_i^{\ast }}}$, where the product is taken over $\left\{i:{\mathit{p}}_i^{\ast }>0\right\}$ is a strict local Lyapunov function [i.e., $V\left(\mathit{p}*\right)>V\left(p\right)$ and $\dot{V}\left(p\right)=V\left(p\right)\left(\pi \left(\mathit{p}*,p\right)-\pi \left(p,p\right)\right)>0$ for all $p\in {\mathrm{\Delta }}^m$ sufficiently close but not equal to an ESS $\mathit{p}*$]. It is tempting to add these stability statements to the Folk Theorem because they remain valid for many matrix game dynamics through the use of other Lyapunov functions. There are several reasons to avoid this temptation.

First, these statements are not true for discrete-time matrix game dynamics (SI Appendix, section 1) as shown already for three-strategy games that exhibit cyclic dominance. Secondly, global stability of an interior ESS in these games is not true for important classes of game dynamics such as the monotone selection dynamics (17). Furthermore, the three-strategy games of Example 1 demonstrate that trajectories for matrix games may converge to a NE that is not an ESS or approach a heteroclinic cycle around the boundary of the strategy simplex.

Example 1 (Generalized Rock–Scissors–Paper Game): Consider the three-strategy rock–scissors–paper (RSP) matrix game with payoff matrix A given by

$$\begin{array}{l}\begin{array}{cc}& \begin{array}{ccc}R& S& P\end{array}\\ \begin{array}{c}R\\ S\\ P\end{array}& \left[\begin{array}{rrr}\hfill 0& \hfill b_2& \hfill -a_3\\ \hfill -a_1& \hfill 0& \hfill b_3\\ \hfill b_1& \hfill -a_2& \hfill 0\end{array}\right]\end{array}=\begin{array}{c}\begin{array}{ccc}R& S& P\end{array}\\ \left[\begin{array}{rrr}\hfill 0& \hfill 6& \hfill -4\\ \hfill -4& \hfill 0& \hfill 4\\ \hfill 2& \hfill -2& \hfill 0\end{array}\right]\end{array}.\end{array}$$ [2]

All such games with positive parameters $a_i$ and $b_i$ exhibit cyclic dominance, whereby R beats S (i.e., R strictly dominates S in the two-strategy game based on these two strategies), S beats P, and P beats R. This dominance implies that there is no NE on the boundary of ${\mathrm{\Delta }}^3$. In fact the unique NE for [2] is the completely mixed strategy $\mathit{p}*=\left(10/29,8/29,11/29\right)$ in the interior. It is globally asymptotically stable under the replicator equation (Fig. 1) but $\mathit{p}*$ is not an ESS because $\pi \left(e_1,\mathit{p}*\right)=\pi \left(\mathit{p}*,\mathit{p}*\right)=4/29$ and $\pi \left(\mathit{p}*,e_1\right)=-\left(10/29\right)<0=\pi \left(e_1,e_1\right)$.

Fig. 1.

Trajectories of the replicator equation for the RSP game with payoff matrix 2.

The matrix game with payoff matrix $-A$ also exhibits cyclic dominance and its trajectories under the replicator equation are the same as Fig. 1 except the direction is reversed. That is, all interior trajectories of [1] (except the one initially at $\mathit{p}*$) approach a heteroclinic cycle around the boundary that joins the three pure strategies in the order $e_3,e_2,e_1,e_3$. It is also well known (12) that the standard RSP game (with all $a_i=b_i=1$ in [2]) has periodic orbits of [1] around the unique NE $\mathit{p}*=\left(1/3,1/3,1/3\right)$.

Cyclic behavior is common not only in biology (e.g., predator–prey systems) but also in human behavior (e.g., business cycles, the emergence and subsequence disappearance of fads, etc.). Thus, it is not surprising that evolutionary game dynamics include cycles as well. In fact, as the number of strategies increases, even more rich dynamical behavior such as chaotic trajectories can emerge (12).

What may be more surprising is the many classes of matrix games (10) for which these complicated dynamics do not appear (e.g., potential, stable, supermodular, zero-sum, doubly symmetric games) and for these the evolutionary outcome is often predicted through rationality arguments underlying Theorems 1 and 2. The emphasis in the remainder of the paper is on such situations for nonmatrix games.

Before doing so, it is important to mention that the replicator equation for doubly symmetric matrix games is formally equivalent to the continuous-time model of natural selection at a single (diploid) locus with m alleles $A_1,\dots ,A_m$ (12, 20). Specifically, if $a_{ij}$ is the fitness of genotype $A_i{A}_j$ and $p_i$ is the frequency of allele $A_i$ in the population, then [1] is the continuous-time selection equation of population genetics (14). It can then be shown that population mean fitness $\pi \left(p,p\right)$ is increasing (this is one part of the fundamental theorem of natural selection). Furthermore, the locally asymptotically stable rest points of [1] correspond precisely to the ESSs of the symmetric payoff matrix A and all trajectories in the interior of ${\mathrm{\Delta }}^m$ converge to a NE of A (20). Analogous results hold for the nonoverlapping generation (viability) selection model (12).

Population Games.

Symmetric population games are the most straightforward generalizations of matrix games. In a symmetric population game at fixed population size with finitely many pure strategies, the payoff $\pi \left(e_i,p\right)$ of strategy $e_i$ is an arbitrary continuous function of the population state $p\in {\mathrm{\Delta }}^m$. Matrix games then correspond to the case where $\pi \left(e_i,p\right)$ is linear in the components of p. If payoff is nonlinear, these are called “playing-the-field” evolutionary games (7). In either case, equating fitness to reproductive success again leads to the replicator equation (Eq. 1) on ${\mathrm{\Delta }}^m$. The Folk Theorem is true for these population games as is Theorem 2b when an ESS is defined as locally superior using Theorem 2a. [The use of the term ESS becomes problematic for nonmatrix games because it often has several different possible meanings. For instance, an ESS as in Theorem 2b is called a “local ESS” by Hofbauer and Sigmund (9) to emphasize that the condition $\pi \left(\mathit{p}*,p\right)>\pi \left(p,p\right)$ holds only for $p\in {\mathrm{\Delta }}^m$ that are sufficiently close (but not equal) to $\mathit{p}*$. When there is ambiguity, we will instead use the phrase “locally superior” or “neighborhood superior” as appropriate.] On the other hand, an interior ESS need no longer be globally asymptotically stable (compare with Theorem 2c) because there may be more than one such ESS.

Population games have important applications to biology [e.g., sex-ratio game; Maynard Smith (7)] as well as to human behavior [e.g., congestion games; Sandholm (10)]. The following example, taken from behavioral ecology, illustrates that important game dynamics other than the replicator equation arise naturally.

Example 2 (Habitat Selection Game and Ideal Free Distribution): The foundation of the habitat selection game for a single species was laid by Fretwell and Lucas (21) before evolutionary game theory appeared. They were interested in predicting how a species (specifically, a bird species) of fixed population size should distribute itself among several resource patches if individuals would move to patches with higher fitness. They argued the outcome would be an ideal free distribution (IFD) defined as a patch distribution whereby the fitness of all individuals in any occupied patch would be the same and at least as high as what would be their fitness in any unoccupied patch (otherwise some individuals would move to a different patch). If there are H patches (or habitats) and an individual’s pure strategy $e_i$ corresponds to being in patch i (for $i=1,2,\dots ,H$), we have a population game by equating the payoff of $e_i$ to the fitness in this patch. The verbal description of an IFD in this “habitat selection game” is then none other than that of a NE.

If patch fitness is decreasing in patch density (i.e., in the population size in the patch), Fretwell and Lucas proved that there is a unique IFD at each fixed total population size. Moreover, the IFD is an ESS that is globally asymptotically stable under the replicator equation (22). To see this, let $p\in {\mathrm{\Delta }}^H$ be a distribution among the patches and $\pi \left(e_i,p\right)$ be the fitness in patch i. Then $\pi \left(e_i,p\right)$ depends only on the proportion $p_i$ in this patch [i.e., has the form $\pi \left(e_i,p_i\right)$]. Because the vector field $\left(\pi \left(e_1,p\right),\dots ,\pi \left(e_H,p\right)\right)$ is the gradient of a real-valued function $F\left(p\right)$ defined on ${\mathrm{\Delta }}^H$, we have a potential game (SI Appendix, section 1.1). Following Sandholm (10), it is a strictly stable game and so has a unique ESS $\mathit{p}*$ which is globally asymptotically stable under the replicator equation (as well as many other game dynamics).

Although Fretwell and Lucas (21) did not attach any dynamics to their model, movement among patches is discussed implicitly. Following Krivan et al. (22), let $I_{ij}\left(p\right)$ be the probability an individual in patch j moves to patch i per unit time if the current patch distribution is p. Then the corresponding continuous-time migration (or dispersal) dynamics in vector form is

$$\dot{p}=I\left(p\right)p-p,$$ [3]

where $I\left(p\right)$ is the $H\times H$ migration matrix with entries $I_{ij}\left(p\right)$. The following result (22) uses the (decreasing) Lyapunov function $W\left(p\right)\equiv {\max }_{1\le i\le H}\hspace{1em}\pi \left(e_i,p\right)$.

Theorem 3. Suppose patch fitness is a decreasing function of patch density in a single-species habitat selection game. Then any migration dynamics 3 that satisfies the following two conditions evolves to the unique IFD.

• a)Individuals never move to a patch with lower fitness.
• b)If there is a patch with higher fitness than some occupied patch, some individuals move to a patch with highest fitness.

We illustrate Theorem 3 when there are three patches. Suppose that at p, patch fitnesses are ordered $\pi \left(e_1,p\right)>\pi \left(e_2,p\right)>\pi \left(e_3,p\right)$ and consider the two migration matrices

$$I^1\left(p\right)\equiv \left[\begin{array}{ccc}1& 1& 1\\ 0& 0& 0\\ 0& 0& 0\end{array}\right]\hspace{0.5em}{I}^2\left(p\right)\equiv \left[\begin{array}{lll}1\hfill & 1/3\hfill & 1/3\hfill \\ 0\hfill & 2/3\hfill & 1/3\hfill \\ 0\hfill & 0\hfill & 1/3\hfill \end{array}\right].$$

$I^1\left(p\right)$ corresponds to a situation where individuals who move go to patch 1 because they know it has highest fitness. The corresponding games dynamics

$$\dot{p}=BR\left(p\right)-p,$$ [4]

where $BR\left(p\right)$ is the best response strategy $e_1$ to p is called the best response dynamics (SI Appendix, section 1.1).

On the other hand, $I^2\left(p\right)$ models individuals who only gain fitness information by sampling one patch at random, moving to this patch if it has higher fitness than its current patch [e.g., an individual in patch 2 moves if it samples patch 1 and otherwise stays in its own patch (with probabilities $1/3$ and $2/3$, respectively)]. Several trajectories for each of these two migration dynamics are illustrated in Fig. 2 (see also SI Appendix, Fig. S1). As can be seen, all converge to the IFD as they must by Theorem 3, even though their paths to this rational outcome are quite different.

Fig. 2.

Trajectories for the habitat selection game with patch fitness functions $\pi \left(e_1,p\right)=1-p_1,\pi \left(e_2,p\right)=0.8\left[1-\left(10p_2/9\right)\right]$ and $\pi \left(e_3,p\right)=0.6\left[1-\left(10p_3/8\right)\right]$. (A) Best response dynamics with migration matrices of the form $I^1\left(p\right)$ and (B) dynamics for nonideal animals with migration matrices of the form $I^2\left(p\right)$.

Finally, suppose information is gained by sampling a random individual, moving to its patch with probability proportional to the fitness difference only if the sampled individual has higher fitness. It is well known (13) that this “proportional imitation rule” leads to the replicator equation (SI Appendix, section 1.1). Since the proportional imitation rule satisfies the two conditions of Theorem 3, the unique IFD is globally asymptotically stable under [1].

Fretwell and Lucas (21) briefly consider their IFD concept when patch fitness increases with patch density when density is low (the so-called Allee effect). Although Theorem 3 no longer applies, these habitat selection games are still potential games (but not strictly stable). Thus, all interior trajectories under many game dynamics (including the replicator equation and best response dynamics) converge to a NE (10). Several NE are already possible for two-patch models, some of which are locally asymptotically stable and some not. There is a difference of opinion whether to define IFD as any of these NE or restrict the concept to only those that are locally superior and/or asymptotically stable (23).

Habitat selection games also provide a natural setting for the effect of evolving population sizes, a topic of obvious importance in population biology that has so far received little attention in models of social behavior. A population-migration dynamics emerges if population size N evolves through fitness taken literally as reproductive success (SI Appendix, section 1.1). As discussed in SI Appendix, section 1.1, if patch fitness is positive when unoccupied, decreases with patch density and eventually becomes negative, then the system evolves to carrying capacity whenever the migration matrix $I\left(p;N\right)$ satisfies the two conditions in Theorem 3 for each N. In particular, the evolutionary outcome is independent of the time scale of migration compared with that of changing population size, a notable result since it is often not true when two dynamical processes are combined.

Games with Continuous Strategy Spaces.

Game dynamics when players can choose from a continuum of pure strategies S become complicated, especially if individual payoff depends on the population state [which is now a distribution in the set of probability measures $\mathrm{\Delta }\left(S\right)$ on S]. One approach to avoid these complications is to assume that the population is always monomorphic at its mean $x\in S$. When S is a convex compact subset of R (i.e., a closed and bounded interval), it is further assumed that x evolves through trait substitution in the direction y of nearby mutants that can invade due to their higher payoff than x when playing against this monomorphism. If $\pi \left(y,x\right)$ is the payoff of a mutant using strategy y in an otherwise monomorphic population x, then x increases (decreases) if $\pi \left(y,x\right)$ is an increasing (decreasing) function of y for y close to x.

The most elementary dynamics to model these assumptions is called the “canonical equation of adaptive dynamics” which has the form (up to a change in time scale)

$${\dot{x}=\frac{\partial \pi \left(y,x\right)}{\partial y}|}_{y=x}\equiv {\pi }_1\left(x,x\right).$$ [5]

A rest point $\mathit{x}*$ [i.e., ${\pi }_1\left(\mathit{x}*,\mathit{x}*\right)=0$] is convergence stable (24) if it is asymptotically stable under [5]. $\mathit{x}*$ is convergence stable if and only if ${d/dx\left[{\partial \pi \left(y,x\right)/\partial y|}_{y=x}\right]|}_{x=\mathit{x}*}={\pi }_{11}\left(\mathit{x}*,\mathit{x}*\right)+{\pi }_{12}\left(\mathit{x}*,\mathit{x}*\right)<0$. On the other hand, $\mathit{x}*$ is a neighborhood-strict NE if and only if ${\pi }_{11}\left(\mathit{x}*,\mathit{x}*\right)<0$ (SI Appendix, section 1.2).

Thus, under adaptive dynamics, a strict NE is not necessarily attainable from nearby populations (i.e., it need not be convergence stable) and, conversely, a convergence stable rest point need not be a NE. That is, parts a and c of the Folk Theorem are not true. A convergence stable $\mathit{x}*$ that is not a neighborhood-strict NE is called an “evolutionary branching point” (25) since, in a dimorphic population with some individuals using pure strategies on either side of $\mathit{x}*$, these nearby strategies evolve away from $\mathit{x}*$ [as can be shown by the second order Taylor polynomial expansion of $\pi \left(y,x\right)$ about $\left(\mathit{x}*,\mathit{x}*\right)$]. For this reason, only those $\mathit{x}*$ that satisfy both conditions i and ii given by

• i)${\pi }_{11}\left(\mathit{x}*,\mathit{x}*\right)<0$ (neighborhood-strict NE condition) and
• ii)${\pi }_{11}\left(\mathit{x}*,\mathit{x}*\right)+{\pi }_{12}\left(\mathit{x}*,\mathit{x}*\right)<0$ (convergence stability condition)

are considered to be stable for models based on adaptive dynamics.

Conditions i and ii respectively are equivalent to $\pi \left(x,\mathit{x}*\right)<\pi \left(\mathit{x}*,\mathit{x}*\right)$ and $\pi \left(y,x\right)>\pi \left(x,x\right)$ if x is close to $\mathit{x}*$ and y is near x but closer to $\mathit{x}*$. These define the concept of a continuously stable strategy (CSS) in terms of static payoff comparisons as introduced earlier by Eshel (26) to generalize the ESS to one-dimensional (1D) continuous strategy spaces. A CSS is then stable for models based on adaptive dynamics.

However, a CSS is not stable under the replicator equation (Eq. 6) for continuous strategy spaces. When payoffs result from pairwise interactions between individuals and $\pi \left(x,y\right)$ is interpreted as the payoff to x against y, then the expected payoff to x in a random interaction is $\pi \left(x,P\right)\equiv {\displaystyle {\int }_S\pi \left(x,y\right)P\left(dy\right)}$, where P is the probability measure on S corresponding to the current distribution of the population’s strategies. With $\pi \left(P,P\right)\equiv {\displaystyle {\int }_S\pi \left(x,P\right)P\left(dx\right)}$ the mean payoff of the population and B a Borel subset of S, the replicator equation (27)

$$\frac{d{P}_t}{dt}\left(B\right)={\displaystyle {\int }_B\left(\pi \left(x,P_t\right)-\pi \left(P_t,P_t\right)\right)P_t\left(dx\right)}$$ [6]

has a unique solution given any initial $P_0$ in the infinite dimensional space $\mathrm{\Delta }\left(S\right)$ of Borel probability measures over the strategy space S (28) (SI Appendix, section 1.2). The replicator equation describes the evolution of the population strategy distribution $P\in \mathrm{\Delta }\left(S\right)$. From this perspective, the canonical equation becomes a heuristic tool that approximates the evolution of the population mean by ignoring effects due to the diversity of strategies in the population.

From SI Appendix, section 1.2, to guarantee $\mathit{x}*$ is stable under [6], $\mathit{x}*$ must be a neighborhood-strict NE as well as a NIS defined as $\pi \left(\mathit{x}*,x\right)>\pi \left(x,x\right)$ for x near $\mathit{x}*$ (29). These two conditions are equivalent to $\mathit{x}*$ being neighborhood superior [i.e., $\pi \left(\mathit{x}*,P\right)>\pi \left(P,P\right)$ for all distributions P whose support is sufficiently close but not equal to $\mathit{x}*$] (19).

The replicator equation as well as concepts of neighborhood-strict NE, NIS, and neighborhood superiority have straightforward generalizations to multidimensional continuous strategy spaces where the stability results remain true (19). The CSS and canonical equation of adaptive dynamics have also been generalized (30, 31) but these depend on the direction(s) in which mutants are more likely to appear. If $\mathit{x}*$ is stable for all such directions (called “strong convergence stability”), the corresponding CSS has similar strong stability properties as the ESS does for matrix games.

Multiplayer Games.

Matrix games are special types of population games where individuals interact in two-player contests. In multiplayer games with a finite set S of pure strategies, interactions are formed among $n$ players where $n>2$ is fixed. For example, if $n=3$, the expected payoff to $e_i$ in a random interaction if the population has state $p\in {\mathrm{\Delta }}^3$ is

$$\pi \left(e_i;p\right)={\displaystyle \sum _{j,k=1}^m{p}_j{p}_k\pi \left(e_i;e_j,e_k\right)},$$

where the payoff $\pi \left(e_i;e_j,e_k\right)$ to $e_i$ in the symmetric three-player game is assumed to depend on the other two players but not their order [i.e., $\pi \left(e_i;e_j,e_k\right)=\pi \left(e_i;e_k,e_j\right)$].

Multiplayer games are then a class of population games where payoffs are nonlinear in the population state. Thus, by the section Population Games, the Folk Theorem and Theorem 2b hold for any multiplayer game with finitely many pure strategies under the replicator equation (Eq. 1). Bukowski and Miekisz (32) (see also ref. 8) characterize an ESS in terms of uninvadability [i.e., local superiority as in Theorem 2a]. Then, for three-player games, $\mathit{p}*$ is an ESS if and only if, for all $p\ne \mathit{p}*$,

• i)$\pi \left(p;\mathit{p}*,\mathit{p}*\right)\le \pi \left(p^{\mathrm{\ast }};\mathit{p}*,\mathit{p}*\right)$ (NE condition),
• ii)if $\pi \left(p;\mathit{p}*,\mathit{p}*\right)=\pi \left(\mathit{p}*;\mathit{p}*,\mathit{p}*\right)$, then $\pi \left(p;\mathit{p}*,p\right)\le \pi \left(\mathit{p}*;\mathit{p}*,p\right)$,
• iii)if $\pi \left(p;\mathit{p}*,\mathit{p}*\right)=\pi \left(\mathit{p}*;\mathit{p}*,\mathit{p}*\right)$ and $\pi \left(p;\mathit{p}*,p\right)=\pi \left(\mathit{p}*;\mathit{p}*,p\right)$, then $\pi \left(p;p,p\right)<\pi \left(\mathit{p}*;p,p\right)$,

where $\pi \left(p;q,r\right)\equiv {\displaystyle {\sum }_{j,k=1}^m{p}_i{q}_j{r}_k\pi \left(e_i;e_j,e_k\right)}$. That is, an ESS satisfies the NE condition i as well as two stability conditions ii and iii.

Bukowski and Miekisz (32) go on to classify the ESS structure of all three-player two-strategy games, showing in particular that an interior ESS need not be globally asymptotically stable (compare with Theorem 2c ). In fact, they also provide an example of a four-player two-strategy supersymmetric game [i.e., $\pi \left(e_i;e_j,e_k,e_{\ell }\right)$ is the same for all permutations of the fixed indices $i,j,k,\ell \in \left\{1,2\right\}$] that has two interior ESSs. Furthermore, for all two-strategy multiplayer games, $\mathit{p}*$ is locally asymptotically stable if and only if $\mathit{p}*$ is an ESS.

The above discussion assumes that the multiplayer game has finitely many pure strategies. In the following example, this is no longer the case.

Example 3 (Public Goods Game): The public goods game (PGG) is a multiplayer game where each player is given an initial endowment $E>0$ and then decides how much x of this endowment to contribute to a common pool (i.e., $0\le x\le E$). All contributions to the common pool are multiplied by a factor $r>1$ and then evenly distributed among all n players. A player’s payoff is then the remainder of his endowment $E-x$ plus what he receives from the public pool. If he contributes $x_1$ and the other $n-1$ players contribute $x_2,x_3,\dots ,x_n$, respectively, his payoff $\pi \left(x_1;x_2,x_3,\dots ,x_n\right)$ is given as

$$E-x_1+\frac{r}{n}{\displaystyle \sum _{i=1}^n{x}_i}=E+\left(\frac{r}{n}-1\right)x_1+\frac{r}{n}{\displaystyle \sum _{i=2}^n{x}_i}.$$ [7]

It is assumed that $1<r<n$ and so each player receives only part of his own contribution to the common pool (i.e., $r/n<1$).

The only NE is for each player to contribute 0 (i.e., to free-ride) since $\pi \left(0;x_2,x_3,\dots ,x_n\right)>\pi \left(x_1;x_2,x_3,\dots ,x_n\right)$ for all $x_1>0$. On the other hand, each player receives his maximum possible payoff of $rE$ when all players contribute E. PPG is the multiplayer version of the prisoner’s dilemma (PD) game (33) with free-riding (respectively, contributing E) corresponding to defect (respectively, cooperate). PGG and PD have been used as the standard examples of social dilemmas to investigate the evolution of cooperation in human and other societies (33, 34), both from a theoretical and empirical perspective.

Since PGG has a continuous strategy space, the Folk Theorem may no longer apply (as we saw above). It is therefore important to analyze stability under game dynamics such as the replicator equation and adaptive dynamics. From [7], the expected payoff $\pi \left(y;P\right)$ of an individual playing y in a group whose other $n-1$ players are chosen at random from a population with state $P\in \mathrm{\Delta }\left(\left[0,E\right]\right)$ is $E+\left[\left(r/n\right)-1\right]y+\left[r\left(n-1\right)/n\right]\overline{x}$, where $\overline{x}\equiv {\displaystyle {\int }_{\left[0,E\right]}xP\left(dx\right)}$ is the average contribution of an individual in the population (SI Appendix, section 1.3). Furthermore, the population mean payoff is $\pi \left(P;P\right)\equiv {\displaystyle {\int }_{\left[0,E\right]}\pi \left(y;P\right)P\left(dy\right)}=E+\left[\left(r/n\right)-1\right]\overline{x}+\left[r\left(n-1\right)/n\right]\overline{x}=E+\left(r-1\right)\overline{x}$.

Following Cressman et al. (35), the evolution of $\overline{x}$ under the replicator equation (Eq. 6) is $d\overline{x}/dt=\left[\left(r/n\right)-1\right]{\displaystyle {\int }_{\left[0,E\right]}{\left(x-\overline{x}\right)}^{2}P\left(dx\right)}\le 0$ with equality if and only if P is a distribution ${\delta }_x$ that has all its weight on some $x\in \left[0,E\right]$. Since $P_t$ has the same support as $P_0$ for all $t\ge 0$, $\overline{x}$ evolves to the smallest element $\mathit{x}*$ in the support of $P_0$. In particular, if there are some free-riders in the original population distribution, $P_t$ evolves to ${\delta }_0$ in the weak topology. This result also follows from the fact that $\pi \left(0;P\right)>\pi \left(P;P\right)$ if $P\ne {\delta }_0$ [i.e., 0 is (globally) neighborhood superior in PGG].

The corresponding adaptive dynamics is $dx/dt={\left[d\pi \left(y;{\delta }_x\right)/dy\right]|}_{y=x}=\left(r/n\right)-1<0$ for x in the interior of $\left[0,E\right]$ and so x evolves to 0 for this game dynamics as well. That is, neither game dynamics predicts cooperative behavior will emerge in the PGG social dilemma.

Asymmetric Games

An asymmetric game is a multiplayer game where the players are assigned roles with a certain probability and, to each role, there is a set of strategies. If there is only one role, then we have a symmetric game. Here we concentrate on two-player two-role asymmetric games with finite pure strategy sets $\left\{e_1,e_2,\dots ,e_m\right\}$ and $\left\{f_1,f_2,\dots ,f_n\right\}$ respectively. These are also called “two-species games” (roles correspond to species) with intraspecific (respectively, interspecific) interactions among players in the same role (respectively, different roles). We also assume that the expected payoffs ${\pi }_1\left(e_i;p,q\right)$ and ${\pi }_2\left(f_j;p,q\right)$ to $e_i$ in species 1 and to $f_j$ in species 2 are linear in the components of the population states $p\in {\mathrm{\Delta }}^m$ and $q\in {\mathrm{\Delta }}^n$. One interpretation of linearity is that each player engages in one intraspecific and one interspecific random pairwise interaction per unit time.

The corresponding replicator equation on the $\left(m-1\right)+\left(n-1\right)$ dimensional strategy space ${\mathrm{\Delta }}^m\times {\mathrm{\Delta }}^n$ is then

$$\begin{array}{l}{\dot{p}}_i=p_i\left({\pi }_1\left(e_i;p,q\right)-{\pi }_1\left(p;p,q\right)\right)\hspace{1em}\mathrm{for}\hspace{1em}i=1,\dots ,m\\ {\dot{q}}_j=q_j\left({\pi }_2\left(f_j;p,q\right)-{\pi }_2\left(q;p,q\right)\right)\hspace{1em}\mathrm{for}\hspace{1em}j=1,\dots ,n,\end{array}$$ [8]

where, for example, ${\pi }_1\left(p;p,q\right)\equiv {\displaystyle {\sum }_{i=1}^m{p}_i{\pi }_1\left(e_i;p,q\right)}$ is the mean payoff of species 1. The Folk Theorem is valid under [8], where a NE is a strategy pair $\left(\mathit{p}*,\mathit{q}*\right)$ such that ${\pi }_1\left(p;p*,\mathit{q}*\right)\le {\pi }_1\left(p*;p*,\mathit{q}*\right)$ for all $p\ne \mathit{p}*$ and ${\pi }_2\left(q;\mathit{p}*,q*\right)\le {\pi }_2\left(q*;p*,q*\right)$ for all $q\ne \mathit{q}*$ (it is strict if both inequalities are strict).

To generalize Theorem 2, consider the uninvadability approach of Maynard Smith and Price (3) for the resident–mutant system, where residents and mutants use strategy pairs $\left(\mathit{p}*,q*\right)$ and $\left(p_0,q_0\right)$, respectively. $\left(\mathit{p}*,q*\right)$ is called a “two-species ESS” (36) if it is locally asymptotically stable under the corresponding 2D (mixed-strategy) replicator equation for all $\left(p_0,q_0\right)\ne \left(\mathit{p}*,q*\right)$. The following result (9, 12, 20) corresponds to Theorem 2.

Theorem 4.

• a)
  $\left(\mathit{p}*,q*\right)$ is a two-species ESS if and only if

  $$\begin{array}{l}\mathit{either}\hspace{1em}{\pi }_1\left(\mathit{p}*;p,q\right)>{\pi }_1\left(p;p,q\right)\\ \mathit{or}\hspace{1em}{\pi }_2\left(\mathit{q}*;p,q\right)>{\pi }_2\left(q;p,q\right)\end{array}$$ [9]

  for all strategy pairs $\left(p,q\right)$ that are sufficiently close (but not equal) to $\left(\mathit{p}*,\mathit{q}*\right)$.
• b)A two-species ESS $\left(\mathit{p}*,\mathit{q}*\right)$ is a locally asymptotically stable rest point of the replicator equation (Eq. 8).
• c)A two-species ESS $\left(\mathit{p}*,\mathit{q}*\right)$ in the interior of ${\mathrm{\Delta }}^m\times {\mathrm{\Delta }}^n$ is a globally asymptotically stable rest point of the replicator equation.

Condition 9 for $\left(\mathit{p}*,\mathit{q}*\right)$ is thus the logical extension to two-species games of local superiority. It has been further extended to asymmetric games having continuous strategy spaces where it is called “neighborhood superior” (19) when applied to pure strategy pairs $\left(\mathit{x}*,y*\right)$ and distributions $\left(P,Q\right)$ with nearby support. A neighborhood superior $\left(\mathit{x}*,y*\right)$ is then stable under the measure-theoretic replicator equation that extends [6] to two-species systems. A related condition (called “neighborhood half-superior”) corresponds to the CSS concept and stability under adaptive dynamics (19, 31).

The theory with finitely many pure strategies has also been extended to two-species population games with nonlinear payoffs (10). For the habitat selection game with two competitive species, individual fitness decreases as the density of either species in its patch increases. Krivan et al. (22) show that the IFD [defined as a distribution where the fitness of species 1 in all patches occupied by this species are equal and at least as high as in any patch unoccupied by this species (and the same for species 2)] is not always stable unless it is also a two-species ESS. This result brings into question how the single-species IFD of Fretwell and Lucas (21) should be defined for two (or more) species.

Asymmetric games with no intraspecific interactions (i.e., the probability individuals in the same role interact is 0) were considered early on by Selten (37) who called them “truly asymmetric games.” With two players, two roles and our linearity assumption on expected payoffs, these become bimatrix games because ${\pi }_1\left(e_i;p,q\right)={\pi }_1\left(e_i,q\right)\equiv {\displaystyle {\sum }_{\ell =1}^n{A}_{i\ell }{q}_{\ell }}$ and ${\pi }_2\left(f_j;p,q\right)={\pi }_2\left(f_j,p\right)\equiv {\displaystyle {\sum }_{k=1}^m{B}_{kj}{p}_k}$, where, for example, A is the $m\times n$ matrix with entries $A_{ij}={\pi }_1\left(e_i,f_j\right)$.

It is well known (12) that $\left(\mathit{p}*,\mathit{q}*\right)$ is a locally asymptotically stable rest point of the bimatrix replicator equation (Eq. 8) if and only if it is a strict NE (SI Appendix, section 2). Moreover (38), $\left(\mathit{p}*,\mathit{q}*\right)$ is a strict NE if and only if it is a two-species ESS [e.g., if $\left(p,q\right)=\left(p,\mathit{q}*\right)$, then [9] implies ${\pi }_1\left(\mathit{p}*,q*\right)>{\pi }_1\left(p,q*\right)$ for $p\ne \mathit{p}*$]. Unfortunately, many bimatrix games have no strict NE and so game dynamics may not predict a NE outcome for them. However, more can be said when these bimatrix normal form games come from a corresponding extensive form (e.g., Theorem 5 in the following section).

Asymmetric Extensive Form Games.

Extensive form games whose decision trees describe finite series of interactions between the same two players (with the set of actions available at later interactions possibly depending on what choices were made previously) were introduced alongside normal form games by von Neumann and Morgenstern (39). From an evolutionary game perspective, differences with normal form intuition already emerge for games of perfect information (SI Appendix, section 2.1) with short decision trees as illustrated in this section (see also ref. 13).

Consider the elementary perfect information game of Fig. 3A that goes by several names [e.g., the “chain store game” (38), the “entry deterrence game” (17)]. Player 1 has one decision node u where he chooses between the actions L and R. If he takes action L, player 1 gets payoff 1 and player 2 gets 4. If he takes action R, then we reach the decision node v of player 2 who then chooses between $\ell$ and r leading to both players receiving payoff 0 or both payoff 2, respectively.

Fig. 3.

The extensive form (A) of the chain store game and trajectories (B) of the replicator equation (Eq. 8).

The corresponding bimatrix normal form [e.g., 1 and 4 are the payoffs to players 1 and 2, respectively when they play $\left(L,\ell \right)$]

$$\begin{array}{c}L\\ R\end{array}\stackrel{\ell \hspace{1em}\hspace{1em}\hspace{1em}r}{{\displaystyle \left[\begin{array}{cc}1,4& 1,4\\ 0,0& 2,2\end{array}\right]}}$$

has two NE outcomes. One is the two-species ESS and strict NE pair $\left(R,r\right)$. On the other hand, if player 1 chooses L, then player 2 is indifferent to what strategy he uses since his payoff is always 4. Furthermore, player 1 is no better off by playing R with positive probability if and only if player 2 plays $\ell$ at least half the time. Thus,

$$G\equiv \left\{(L,q_1\ell +\left(1-q_1\right)r\left)\right|\frac{1}{2}\le q_1\le 1\right\}$$

is a set of NE (called a “NE component” since it is a connected set of NE that is not contained in any larger connected set of NE) all corresponding to the same NE outcome; namely, the path L that leads to payoffs 1 and 4.

The trajectories of the replicator equation (Eq. 8) are shown in Fig. 3B. All points on the vertical edge $p_1=1$ ($p_1$ is the probability player 1 plays L) are rest points although only those in G are limit points of interior trajectories. The six results below for this example follow since $q_1$ is always strictly decreasing and $p_1$ is strictly increasing (decreasing) if and only if $q_1>1/2$ $\left(q_1<1/2\right)$ for any interior trajectory. They hold in general by Theorem 5 (13).

• 1)Every NE outcome is a single terminal node.
• 2)Every NE component G includes a pure strategy pair.
• 3)The outcomes of all elements of G are the same.
• 4)Every interior trajectory converges to a NE.
• 5)If a NE component is interior attracting, it includes the subgame perfect NE (SPNE) defined below.
• 6)A NE is locally asymptotically stable if and only if it is a strict SPNE if and only if it is pervasive (i.e., it reaches every player decision point).

Theorem 5. These six results are true for all generic perfect information games without moves by nature.

Some game theorists argue that these games have only one rational NE equilibrium outcome and this can be found by backward induction. This procedure starts at a final player decision node (i.e., a player decision node that has no player decision points following it) and decides which unique action this player chooses there to maximize his payoff in the subgame with this as its root. The original game tree is then truncated at this node by creating a terminal node there with payoffs to the two players given by this action. The process yields the SPNE when it is continued until the game tree has no player decision nodes left. That is, the strategy constructed by backward induction produces a NE in each subgame ${\mathrm{\Gamma }}_u$ corresponding to the subtree with root at the decision node u. This is a pure strategy pair and is indicated by the double lines in the decision tree as in Fig. 3A. If a NE is not subgame perfect, then this perspective argues that there is some player decision node where an incredible threat has been used such as player 2 forcing payoff 0 by playing $\ell$ if node v is reached (38).

Results 5 and 6 support this argument although, as the decisions tree becomes more complex than Fig. 3A, the SPNE (component) need no longer be stable (13). Instability of the SPNE often arises when a player has two (or more) decision nodes along a path in the tree or when the game does not have perfect information as in the following example.

Example 4 (The Prey Recognition Game): Two prey types are distributed among a large number of patches (or microhabitats) with at most one prey in each patch. Let $p_i$ be the probability a patch chosen at random by a single predator contains prey of type i for $i=1,2$ and $p_0$ the probability the patch is empty (i.e., contains no prey) (Fig. 4, level 1). If the predator finds a prey while spending searching time ${\tau }_s$ in the chosen patch, it decides immediately whether to attack, move to another microhabitat to begin a new search, or spend recognition time to determine the type of prey encountered with probabilities $q_A,q_M,q_R$ respectively [i.e., $q\equiv \left(q_A,q_M,q_R\right)\in {\mathrm{\Delta }}^3$]. The horizontal dashed line in Fig. 4 joining these two encounter events (called an “information set”) indicates that this decision must be made without knowing the type of prey.

Fig. 4.

The decision tree for the prey recognition game. In the reduced tree, the dotted edges are deleted.

If the predator decides to spend recognition time ${\tau }_r$ to determine the encountered prey is of type i, then it must subsequently decide whether to attack this prey or not with probabilities $q_{Ai}$ and $1-q_{Ai}$ respectively (Fig. 4, level 3). If ${\tau }_{ih}$ is the handling time for prey of type i and ${\pi }_i$ is their nutritional value, then the total predator nutritional value per unit time $f\left(\left(q_A,q_R,q_M\right),q_{A1},q_{A2}\right)$ is

$$\frac{p_1{\pi }_1\left(q_A+q_R{q}_{A1}\right)+p_2{\pi }_2\left(q_A+q_R{q}_{A2}\right)}{\tau },$$ [10]

where $\tau ={\tau }_s+p_1{q}_A{\tau }_{1h}+p_1{q}_R\left(q_{A1}{\tau }_{1h}+{\tau }_r\right)+p_2{q}_A{\tau }_{2h}+p_2{q}_R\left(q_{A2}{\tau }_{2h}+{\tau }_r\right)$ (SI Appendix, section 2.2).

Optimal foraging theory (40) postulates the predator maximizes f as a function of $q,q_{A1},q_{A2}$. Game-theoretic methods can be used to find the optimal foraging behavior (41). Specifically, the agent normal form of Fig. 4 is a three-player foraging game where player 1 chooses $q\in {\mathrm{\Delta }}^3$ at the information set, player 2 chooses $q_{A2}$ at the node labeled “recognized prey 2” and player 3 chooses $q_{A1}$ at “recognized prey 1.” Optimal foraging behavior is a NE of this game.

At a NE, the predator should never move to another patch when it first finds a prey since, by abandoning this prey, the predator wastes the time spent searching for it. Furthermore, if prey type 1 is more profitable than type 2 (i.e., ${\pi }_1/{\tau }_{1h}>{\pi }_2/{\tau }_{2h}$ as assumed throughout this example), then the predator must attack any prey 1 that it recognizes (SI Appendix, section 2.2). Thus, we assume that $q_M=0$ and $q_{A1}=1$ in the decision tree of Fig. 4 and analyze the truncated foraging game that eliminates the three edges indicated by dotted lines there.

Following Cressman et al. (41), if the profitabilities of both prey types are nearly equal or recognition time is long, the only NE behavior of player 1 is to immediately attack any prey encountered. In fact, the only NE component is then $G\equiv \left\{\left(q,q_{A2}\right)|q_A=1,0\le q_{A2}\le 1\right\}$.

Otherwise, a subset of G (see the red line segment in SI Appendix, Fig. S2) is a NE component and another NE outcome emerges where the predator spends the time to recognize the type of prey it encountered and then only attacks the most profitable type (i.e., $q_A=0$ and $q_{A2}=0$ is a NE). This latter NE is strict and is the only one that corresponds to optimal foraging behavior. It is also the only NE outcome that is locally asymptotically stable under the adaptive dynamics (SI Appendix, section 2.2 and Fig. S2)

$$\begin{array}{l}{\dot{q}}_A=q_A\left(1-q_A\right)\frac{\partial }{\partial q_A}f\left(q_A,q_{A2}\right)\\ {\dot{q}}_B=q_{A2}\left(1-q_{A2}\right)\frac{\partial }{\partial q_{A2}}f\left(q_A,q_{A2}\right)\end{array}$$ [11]

that keeps the unit square invariant.

Although the prey recognition game is not a conventional extensive form game because payoffs are nonlinear in the predator’s strategy (and the game has a continuous strategy space), the game dynamics 11 is remarkably similar to that of the replicator equation for the chain store game (Fig. 3B and SI Appendix, Fig. S2).

Discussion

The replicator equation and other deterministic game dynamics have become essential tools over the past 40 years in applying evolutionary game theory to behavioral models in the biological and social sciences. The theory that we have summarized for these dynamics assumes large homogeneous populations with random interactions. Stochastic effects (e.g., based on finite populations) and the effects of nonrandom interactions (e.g., games on graphs), which have become increasingly important in game dynamic models (10, 11), are beyond the scope of this paper.

The examples given in this paper show that static game-theoretic solution concepts (e.g., NE and ESS) play a central role in predicting the evolutionary outcome of game dynamics. Conversely, game dynamics that arise naturally in analyzing behavioral evolution lead to a more thorough understanding of issues connected to the static concepts. That is, both the classical and evolutionary approaches to game theory benefit through this interplay between them.

For instance, the prey recognition game illustrates anew the potential of game-theoretic methods to gain a better understanding of issues that arise in behavioral ecology. Here, it suggests how the predator “learns” its optimal foraging behavior through game dynamics, a question that is not often considered in optimal foraging theory. On the other hand, the analysis of this game also raises important questions in game-theoretic applications to human behavior. Specifically, this example considers the effect time spent on different interactions has on rational behavior, an aspect that becomes increasingly central when individuals have a series of interactions such as those modeled by an extensive form.