Abstract
Repetition is one of the key mechanisms to maintain cooperation. In long-term relationships, in which individuals can react to their peers׳ past actions, evolution can promote cooperative strategies that would not be stable in one-shot encounters. The iterated prisoner׳s dilemma illustrates the power of repetition. Many of the key strategies for this game, such as ALLD, ALLC, Tit-for-Tat, or generous Tit-for-Tat, share a common property: players using these strategies enforce a linear relationship between their own payoff and their co-player׳s payoff. Such strategies have been termed zero-determinant (ZD). Recently, it was shown that ZD strategies also exist for multiplayer social dilemmas, and here we explore their evolutionary performance. For small group sizes, ZD strategies play a similar role as for the repeated prisoner׳s dilemma: extortionate ZD strategies are critical for the emergence of cooperation, whereas generous ZD strategies are important to maintain cooperation. In large groups, however, generous strategies tend to become unstable and selfish behaviors gain the upper hand. Our results suggest that repeated interactions alone are not sufficient to maintain large-scale cooperation. Instead, large groups require further mechanisms to sustain cooperation, such as the formation of alliances or institutions, or additional pairwise interactions between group members.
Keywords: Cooperation, Repeated games, Zero-determinant strategies, Evolutionary game theory, Public goods game
Highlights
-
•
We explore the evolution of direct reciprocity in groups of n players.
-
•
We show why it is instructive to consider zero-determinant (ZD) strategies.
-
•
ZD strategies include AllD, AllC, Tit-for-Tat, extortionate and generous strategies.
-
•
In small groups, generosity allows the evolution of cooperation.
-
•
In large groups, cooperation is unlikely to evolve.
1. Introduction
One of the major questions in evolutionary biology is why individuals cooperate with each other. Why are some individuals willing to pay a cost (thereby decreasing their own fitness) in order to help someone else? During the last decades, researchers have proposed several mechanisms that are able to explain why cooperation is abundant in nature (Nowak, 2006; Sigmund, 2010). One such mechanism is repetition: if I help you today, you may help me tomorrow (Trivers, 1971). Among humans, this logic of reciprocal giving has been documented in numerous behavioral experiments (e.g., Wedekind and Milinski, 1996; Keser and van Winden, 2000; Fischbacher et al., 2001; Dreber et al., 2008; Grujic et al., 2014). Moreover, it has also been suggested that direct reciprocity is at work in several other species, including vampire bats (Wilkinson, 1984), sticklebacks (Milinski, 1987), blue jays (Stephens et al., 2002), and zebra finches (St. Pierre et al., 2009). From a theoretical viewpoint, these observations lead to the question under which circumstances direct reciprocity evolves, and which strategies can be used to sustain mutual cooperation.
The main model to explore these questions is the iterated prisoner׳s dilemma, a stylized game in which two individuals repeatedly decide whether they cooperate or defect (Rapoport and Chammah, 1965; Doebeli and Hauert, 2005). The payoffs of the game are chosen such that mutual cooperation is preferred over mutual defection, but each individual is tempted to defect at the expense of the co-player. Theoretical studies have highlighted several successful strategies for this game (Axelrod and Hamilton, 1981; Molander, 1985; Kraines and Kraines, 1989; Nowak and Sigmund, 1992, 1993b). Evolution often occurs in dynamical cycles (Boyd and Lorberbaum, 1987; Nowak and Sigmund, 1993a; van Veelen et al., 2012): unconditional defectors (ALLD) can be invaded by reciprocal strategies like Tit-for-Tat (TFT), which in turn often catalyze the evolution of more cooperative strategies like generous Tit-for-Tat (gTFT) and unconditional cooperators (ALLC). Once ALLC is common, ALLD can reinvade, thereby closing the evolutionary cycle (Nowak and Sigmund, 1989; Imhof et al., 2005; Imhof and Nowak, 2010).
The above mentioned strategies for the iterated prisoner׳s dilemma share an interesting mathematical property: they enforce a linear relationship between the players׳ payoffs in an infinitely repeated game (Press and Dyson, 2012). For example, when player 1 adopts the strategy Tit-for-Tat, the players׳ payoffs πj will satisfy the equation , irrespective of player 2׳s strategy. Similarly, when player 1 adopts ALLD, payoffs will satisfy (where c and b denote the cost and the benefit of cooperation, respectively; this version of the prisoner׳s dilemma is sometimes called the donation game, see e.g. Sigmund, 2010). Finally, when player 1 applies gTFT, the enforced payoff relation becomes . Strategies that enforce such linear relationships between payoffs have been called zero-determinant strategies, or ZD strategies (this name is motivated by the fact that these strategies let certain determinants vanish, see Press and Dyson, 2012). After Press and Dyson׳s discovery, several studies have explored how ZD strategies for the repeated prisoner׳s dilemma fare in an evolutionary context (Akin, 2013; Stewart and Plotkin, 2012, 2013; Hilbe et al., 2013a,b; Adami and Hintze, 2013; Szolnoki and Perc, 2014a,b; Chen and Zinger, 2014), and in behavioral experiments (Hilbe et al., 2014a).
Zero-determinant strategies are not confined to pairwise games; they also exist in the iterated public goods game (Pan et al., 2014), and in fact in any repeated social dilemma, with an arbitrary number of involved players (Hilbe et al., 2014b). In this way, it has become possible to identify the multiplayer-game analogues of the above mentioned strategies. For example, the multiplayer-version of TFT in a repeated public goods game is proportional Tit-for-Tat (pTFT): if j of the other group members cooperated in the previous round, then a pTFT-player cooperates with probability in the next round, with n being the size of the group. Herein, we will explore the role of these recently discovered multiplayer ZD strategies for the evolution of cooperation.
We consider two evolutionary scenarios. First, we consider a conventional setup, in which the members of a well-mixed population are engaged in a series of repeated public goods games, and where successful strategies reproduce more often. In line with previous studies (Boyd and Richerson, 1988; Hauert and Schuster, 1997; Grujic et al., 2012), our simulations confirm that the prospects of cooperation depend on the size of the group. Small groups promote generous ZD strategies that allow for high levels of cooperation, whereas larger groups favor the emergence of selfish ZD strategies such as ALLD. For our second evolutionary scenario, we consider a player with a fixed ZD strategy whose co-players are allowed to adapt their strategies over time. Similar to the case of the repeated prisoner׳s dilemma (Press and Dyson, 2012; Chen and Zinger, 2014), the resulting group dynamics then depends on the applied ZD strategy of the focal player. But also here, the possibilities of a single player to generate a positive group dynamics diminishes with group size, irrespective of the strategy applied by the focal player.
Taken together, these results suggest that larger groups make it more difficult to sustain cooperation. In the discussion, we will thus argue that there are three potential mechanisms that can help individuals solving their multiplayer social dilemmas: they can either provide additional incentives on a pairwise basis (Rand et al., 2009; Rockenbach and Milinski, 2006); they can coordinate their actions and form alliances (Hilbe et al., 2014b); or they can implement central institutions which enforce mutual cooperation (Ostrom, 1990; Sigmund et al., 2010; Sasaki et al., 2012; Cressman et al., 2012; Traulsen et al., 2012; Zhang and Li, 2013; Schoenmakers et al., 2014).
2. Model
2.1. Iterated multiplayer dilemmas and memory-one strategies
In the following, we consider a group of n individuals, which is engaged in a repeated multiplayer dilemma. In each round of the game, players can decide whether to cooperate (C) or to defect (D). The payoffs in a given round depend on the player׳s own decision, and on the number of cooperators among the remaining group members. That is, in a round in which j of the other group members cooperate, the focal player receives aj for cooperation, and bj for defection (see also Table 1). We suppose that the multiplayer game takes the form of a social dilemma, such that payoffs satisfy the following three conditions (see also Kerr et al., 2004): (a) individuals prefer their co-players to be cooperative, and for all j; (b) within a mixed group, defectors outperform cooperators, for all j; (c) mutual cooperation is favored over mutual defection, . Several well-known examples of multiplayer games satisfy these criteria, including the public goods game (see e.g. Ledyard, 1995), the volunteer׳s dilemma (Diekmann, 1985; Archetti, 2009), or the collective-risk dilemma (Milinski et al., 2008; Santos and Pacheco, 2011; Abou Chakra and Traulsen, 2014).
Table 1.
Number of cooperating co-players | n−1 | n−2 | … | 0 |
Payoff for cooperation | … | |||
Payoff for defection | … |
We assume that the multiplayer game is repeated, such that the group members face the same dilemma situation over multiple rounds. Herein, we will focus on infinitely repeated games, but the theory of ZD strategies can also be developed for games with finitely many rounds, or when future payoffs are discounted (Hilbe et al., 2014a, 2015). In repeated games, players can react on their co-players׳ previous behavior. In the simplest case, players only consider the outcome of the last round, that is, they apply a so-called memory-one strategy. Memory-one strategies consist of two parts: a rule that tells the player what to do in the first round, and a rule for what to do in all subsequent rounds, depending on the previous round׳s outcome. In infinitely repeated games, the first-round play can typically be neglected (see also Appendix A.1). In that case, memory-one strategies can be written as a vector . The entries correspond to the player׳s cooperation probability in the next round, given that the player used in the previous round, and that j of the other group members cooperated. Using this notation, the strategy ALLD can be written as (); the strategy ALLC takes the form (); and the strategy proportional Tit-for-Tat is given by pTFT=().
When all players in a group apply memory-one strategies, one can directly calculate the resulting payoffs for each group member, using a Markov chain approach (Nowak and Sigmund, 1993b; Hauert and Schuster, 1997). A detailed description is given in Appendix A.1. However, it is worth noting that the computation of payoffs is numerically expensive, because one needs to calculate the entries of a transition matrix (and the left eigenvector thereof). The exponential increase in computation time for large groups makes it difficult to attain evolutionary results beyond a certain group size (for example, in Hauert and Schuster, 1997, the maximum group size considered is n=5).
2.2. Zero-determinant strategies
Only recently, Press and Dyson (2012) have described a particular subclass of memory-one strategies for the repeated prisoner׳s dilemma. With these so-called ZD strategies, a player can enforce a linear relationship between her own payoff and the co-player׳s payoff. Such strategies do also exist in multiplayer social dilemmas (Hilbe et al., 2014b): a memory-one strategy is called a ZD strategy if there are constants l, s, and such that the entries of can be written as
(1) |
By adopting such a strategy, player i can enforce the payoff relationship
(2) |
where πi is the payoff of player i, and is the average payoff of i׳s co-players (Press and Dyson, 2012; Hilbe et al., 2014b). We call s the slope of the ZD strategy, as it controls how the co-players׳ payoffs π−i change with the focal player׳s payoff πi. Moreover, we call l the baseline payoff: when all players adopt the same ZD strategy, then , and Eq. (2) implies that each player obtains the payoff . The parameter ϕ in the definition of ZD strategies does not have a direct impact on the enforced payoff relationship (Eq. (2)). However, the value of ϕ determines how fast payoffs converge over the course of the game (Hilbe et al., 2014a). Thus, we call ϕ the convergence factor.
It is instructive to consider a few examples of ZD strategies for the public goods game. One example is the strategy proportional Tit-for-Tat with cooperation probabilities . The strategy pTFT results from the definition of ZD strategies (1) by setting s=1 and . Since s=1, it follows from Eq. (2) that a player using pTFT enforces the fair relationship , that is, a pTFT player ensures that he always gets exactly the average payoff of the group. In a similar way, many well-known memory-one strategies can be represented as ZD strategies, including ALLD, ALLC, extortionate strategies (EXT), and generous strategies (GEN), as shown in Table 2 and Fig. 1.
Table 2.
Strategy | Baseline payoff l | Slope s | Convergence factor ϕ |
---|---|---|---|
ALLD | 0 | ||
EXT | 0 | ϕ | |
pTFT | 1 | ||
GEN | rc− c | ϕ | |
ALLC | rc− c |
Compared to groups with memory-one players, the calculation of payoffs becomes considerably more simple when all players adopt ZD strategies. To see this, suppose each of the n group members applies some ZD strategy with parameters li, si and ϕi. As a result, each player enforces a linear payoff relationship as in Eq. (2). Overall, this leads to n linear equations in the n unknown payoffs πi. This system of equations can be solved explicitly (for details, see Appendix A.2); the payoffs of the players are given by
(3) |
where
(4) |
This formula allows a fast calculation of payoffs even in large groups.
The representation of ZD strategies in Eq. (1) has one apparent disadvantage. Because the definition requires three free parameters l, s, and ϕ, it may be difficult to decide whether or not a given memory-one strategy can be written as a ZD strategy. To solve this difficulty, one can derive an alternative representation of ZD strategies. In Appendix A.3, we show that a memory-one strategy is a ZD strategy for the public goods game if and only if the entries satisfy
(5) |
For general group sizes n, these conditions define the 3-dimensional subspace of ZD strategies, within the 2n-dimensional space of memory-one strategies. When we consider a public goods game between three players only, we can illustrate the resulting space of ZD strategies. To this end, let us assume that players only use reactive strategies (i.e., their cooperation probabilities only depend on the actions of the co-players, but not on their own action, such that for all j). For groups of three players, reactive strategies thus take the form , where pi is the probability to cooperate when i of the co-players cooperated in the previous round. Since , the space of all reactive strategies takes the form of a cube (Fig. 2). The conditions for ZD strategies (5) simplify to the condition , which is a two-dimensional plane in the cube of reactive strategies. This plane has the four corners ALLD, pTFT, ALLC, and the anti-reciprocal strategy . Extortionate strategies are on the edge between pTFT and ALLD; in particular, they all have (extortioners never cooperate after mutual defection). In contrast, generous strategies are on the edge between pTFT and ALLC; in particular, they must have (generous players always cooperate after mutual cooperation).
3. Evolution of zero-determinant strategies
In the following, we want to explore the role of these various ZD strategies in evolutionary processes. To get an intuitive understanding of the possible transitions, let us first focus on a restricted strategy set. Specifically, we consider the strategies ALLD, ALLC, and pTFT; moreover, we include a particular instance of an extortionate strategy (for which we set the slope s=0.8, as depicted in Fig. 1B), and a particular instance of a generous strategy (also having a slope s=0.8, as depicted Fig. 1D). Using other instances of extortionate or generous strategies would leave the main conclusions unchanged, as described in more detail below.
For the evolutionary dynamics, we consider a population with N individuals. Let ND, NE, NT, NG, and NC denote the number of unconditional defectors, extortioners, pTFT players, generous players, and unconditional cooperators, respectively, such that . In each time step, groups of size are randomly formed (by sampling group members from the population without replacement). Given the composition of the group, we can calculate the payoff of each player using the payoff formula (3). By summing up over all possible group compositions, this yields the expected payoff for each strategy in the population. To model the spread of successful strategies, we consider a pairwise comparison process (Blume, 1993; Szabó and Tőke, 1998; Traulsen et al., 2006; Hilbe et al., 2013a; Stewart and Plotkin, 2013). In each time step, some randomly chosen player is given the chance to imitate the strategy of some other randomly chosen group member. If the focal player׳s expected payoff is , and the role model׳s payoff is , then the focal player adopts the role model׳s strategy with probability
(6) |
The parameter denotes the strength of selection. In the limit , selection is neutral and the imitation probability simplifies to . In the limit of strong selection () the role model is imitated only if its strategy is sufficiently beneficial. In addition to these imitation events, we assume that subjects sometimes explore new strategies: in each time step, a randomly chosen player may switch to another strategy with probability (with all other strategies having the same chance to be chosen). Overall, these assumptions lead to a stochastic selection-mutation process, in which successful strategies have a higher chance to be adopted (Nowak et al., 2004; Imhof and Nowak, 2006; Antal et al., 2009).
To explore the role of different strategies for the evolutionary dynamics, we have run simulations for different subsets of ZD strategies, and for two different group sizes (as shown in Fig. 3). When groups are small and the population consists only of defectors and generous players, cooperation cannot emerge when initially rare (Fig. 3A). Instead, the emergence of cooperation is dependent on additional strategies that are able to invade ALLD. For example, extortionate strategies can serve as a catalyst for cooperation: extortioners are able to subvert defectors, and once the fraction of extortioners has surpassed a certain threshold, generous ZD strategies can invade and fixate in the population (Fig. 3B). A similar effect can be observed by adding pTFT to the population, which also promotes the evolution of generosity (Fig. 3C). Compared to pTFT, generous ZD strategies have the advantage that they are less prone to errors, as they are more likely to accept a co-player׳s accidental defection. Adding unconditional cooperators, however, can destabilize populations of generous players (Fig. 3D). ALLC players are able to subvert a generous population by neutral drift, which in turn allows for the re-invasion of defectors. As in the case of the iterated prisoner׳s dilemma, the dynamics of the repeated public goods game may result in cycles between cooperation and defection.
Larger group sizes further impede the evolution of cooperation: when the group size is above a certain threshold, evolution either settles at a population of defectors, or at a population of extortioners (the lower panels in Fig. 3 depict the case n=8). This effect of group size is also illustrated in Fig. 4, which shows the average abundance of each of the five considered ZD strategies as a function of group size n. Whereas generous strategies are most abundant when , more selfish strategies succeed in large groups.
To obtain an analytical understanding for these results, let us calculate under which conditions a mutant ZD strategy can invade into a population of defectors. If the mutant applies a ZD strategy with parameters and , we can use the payoff equation (3) to calculate the mutant׳s payoff in a group of defectors
(7) |
Because baseline payoffs satisfy , and because slopes fulfill (Hilbe et al., 2014b), it follows that , i.e., no single mutant has a selective advantage in an ALLD population. In particular, for generous mutants (with and ) we get , and hence they are disfavored when rare. However, two strategy classes are able to invade ALLD by neutral drift: when the mutant either applies an extortionate strategy (with ), or pTFT (with ), then . These calculations confirm that both pTFT and extortionate strategies can act as a catalyst for cooperation, as they are able to subvert a population of defectors irrespective of the size of the group.
Similarly, we can also explore the stability of a population of generous players. As expected, ALLC mutants are always able to invade by neutral drift (again irrespective of group size). Moreover, using Eq. (3), it follows that the payoff of a single defector exceeds the residents׳ payoff if
(8) |
where is the slope of the generous strategy. Thus, any given generous strategy can be invaded by ALLD, provided that the group size n is sufficiently large. Equivalently, to be stable against defectors, a generous strategy must not be too generous, . In particular, it follows that the set of stable generous strategies shrinks with the size of the group. Taken together, these results suggest that it becomes increasingly difficult to achieve cooperation in large groups.
4. Evolution in the space of memory-one strategies
By focusing on the five ZD strategies above, we have gained insights into the possible transitions from defection to cooperation; moreover, it has allowed us to show how overly altruistic strategies (such as ALLC) and large group sizes can lead to the downfall of cooperation. However, the focus on these five particular strategies also comes with a risk. We may have neglected other important strategies, which may have a critical effect on the evolutionary outcomes. In order to assess how general the above results are, let us explore in the following how the dynamics of repeated social dilemmas change when we allow for all possible memory-one strategies.
Specifically, we apply the adaptive dynamics approach introduced by Imhof and Nowak (2010); that is, we adapt the previously used evolutionary process as follows. Again, we consider a population of size N that is engaged in a repeated public goods game, starting with a homogeneous population of defectors. When a mutation occurs, the mutant strategy is not restricted to a particular subset of ZD strategies; instead, mutants may adopt any memory-one strategy (i.e., a mutant׳s memory-one strategy is created by drawing random numbers uniformly from the unit interval [0,1]). We assume that mutations are sufficiently rare, such that the mutant strategy either fixates, or goes extinct, before the next mutation occurs (this process may take a long time,see Fudenberg and Imhof, 2006; Wu et al., 2012). As a consequence, the dynamics results in a sequence of strategies (, , , …), where the strategy is the strategy applied by the resident after t mutation events. Given this strategy sequence, we can calculate the sequence of resident payoffs (π0, π1, π2,…), using the payoff algorithm described in Appendix A.1. By analyzing these two sequences for different parameter values n, we can analyze the impact of group size on the evolution of strategies, and on the resulting average payoffs.
As shown in Fig. 5A, larger group sizes lead, on average, to lower population payoffs. This is not only in line with our previous results depicted in Fig. 4; it also confirms the results of Boyd (1989), showing that large groups are more likely to end up in selfish states. However, it is worth noting that the previous results were based on the comparison of the ALLD strategy with a handful of other, more cooperative strategies (in Boyd, 1989, defectors were matched against threshold variants of Tit-for-Tat,which only cooperate if at least k of the other players cooperated in the previous round). Fig. 5A shows that this conclusion also holds in the larger (and more general) strategy space of memory-one strategies: larger group sizes impede the evolution of cooperation (which is in line with the simulations presented in Hauert and Schuster, 1997).
To gain further insights into what drives this downfall of cooperation, we have also explored which strategies were used by the residents over the course of the evolutionary process. To this end, we have applied the method introduced by Hilbe et al. (2013a): to measure the relative importance of a given strategy , we have recorded how often the evolutionary process visits the neighborhood of (as the strategy׳s neighborhood, we have taken the 1% of memory-one strategies that are closest to ). Using this method, we call being favored by selection if the evolutionary process spends more than 1% of the time in this neighborhood (i.e., if the process spends more time in the neighborhood than expected under neutrality).
Let us first apply this method to the five ZD strategies considered before. As shown in Fig. 5B, our results reflect the qualitative findings in the previous section. Only in small groups, the generous strategy is favored by selection; as the group size increases, ALLD and the extortionate strategy become increasingly successful. For comparison, we have also explored the evolutionary success of the traditional champion in repeated games, win-stay lose-shift (WSLS, see Nowak and Sigmund, 1993b). WSLS only cooperates if all group members have used the same action in the previous round, i.e., , and otherwise (in Hauert and Schuster, 1997 this strategy is called Pavlov, and in Pinheiro et al., 2014 it is called an All-or-None strategy). In Hilbe et al. (2014b) it is shown that WSLS is a Nash equilibrium if , which is satisfied for the parameters used for the simulations. Indeed, Fig. 5B confirms that WSLS is favored by selection for all considered group sizes, but its relative importance decreases with n. For , the process spends more than 20% of the time in the neighborhood of WSLS, whereas for n=7 the neighborhood is only visited 12% of the time. These results indicate that although WSLS is able to sustain cooperation even in larger groups, evolutionary processes tend to favor ALLD and extortionate strategies instead, which is in line with the downfall of average payoffs as the group size increases.
5. Performance of ZD strategies against adapting opponents
In the previous two sections, we have considered a traditional setup to study evolutionary processes. We have assumed that all players come from the same population, and they all are equally likely to change their strategies over time. However, for the iterated prisoner׳s dilemma it has been suggested that extortioners, and more generally ZD strategies with a positive slope, are particularly successful when they are stubborn (Press and Dyson, 2012; Hilbe et al., 2013a; Chen and Zinger, 2014): they should refrain from switching to other strategies that may be more profitable in the short run, in order to gain a long-run advantage. When a player with a fixed strategy is paired with adapting co-players, the nature of the interaction changes. Instead of a symmetric and simultaneous game, the interaction now takes the form of an asymmetric and sequential game (Bergstrom and Lachmann, 2003; Damore and Gore, 2011): by choosing a fixed strategy, the stubborn player moves first, whereas the adapting players have a chance to evolve, and to move towards a best reply over time.
To investigate such a setup in the context of multiplayer dilemmas, let us modify the evolutionary process as follows: instead of considering a large population of players, let us consider a fixed group of size n that is engaged in a sequence of repeated public goods game. One of the players, called the focal player, is assumed to take a fixed ZD strategy. The other group members are allowed to change their strategies from one repeated game to the next. Specifically, we assume that in each time step, one of the adapting players is chosen at random. This player is then given the chance to experiment with a different strategy. When the payoff of the old strategy is , whereas the new strategy yields , we assume that the player switches to the new strategy with probability ρ as specified in Eq. (6). Overall, these assumptions result in an evolutionary process in which one player sticks to his strategy, whereas the other players can change to better strategies, given the current composition of the group.
In Fig. 6, we show the outcome of such an evolutionary process under the assumption that all players are restricted to the five ZD strategies used before. Independent of the fixed strategy of the focal player, all simulations have in common that the focal player׳s payoff decreases with group size. Nevertheless, the strategy of the focal player still has a considerable impact on the resulting group dynamics. For small group sizes, the simulations confirm that focal players with a higher slope value s tend to gain higher payoffs (see Fig. 6, upper panels). The co-players of a focal ALLD or ALLC player often adapt towards selfish strategies, whereas the co-players of a focal EXT, pTFT, or GEN player tend to adopt cooperative strategies (as depicted in Fig. 6, lower panels). Only as the group size becomes large, this positive effect of the focal player׳s strategy on the group dynamics disappears. For example, in groups of size n=8, the strategy distribution of the remaining group members is largely independent of the fixed strategy of the focal individual. The only exception occurs when the focal player is unconditionally altruistic (in which the remaining group members favor ALLD, independent of the group size). These simulations confirm that stubborn players are most successful when they apply ZD strategies with a high slope value (the most successful strategy in Fig. 6 is pTFT, which is also the strategy that has the maximum value for s). Higher slope values correspond to players that are more conditionally cooperative. Thus, when players aim to have a positive impact on the group dynamics, they need to apply reciprocal strategies.
6. Discussion
Repeated interactions provide an important explanation for the evolution of cooperation: individuals cooperate because they can expect to be rewarded in future (Trivers, 1971; Axelrod and Hamilton, 1981; Doebeli and Hauert, 2005; Nowak, 2006; Sigmund, 2010). The framework of repeated games does not necessarily require sophisticated mental capacities. Several experiments suggest that various animal species are able to use reciprocal strategies (Wilkinson, 1984; Milinski, 1987; Stephens et al., 2002; St. Pierre et al., 2009), and also theory suggests that full cooperation can already be achieved using simple strategies that only refer to the outcome of the last round.
Much research in the past has been devoted to explore conditionally cooperative strategies in pairwise interactions. There has been considerably less effort to understand the evolution of reciprocity in larger groups (some exceptions include Boyd, 1989; Hauert and Schuster, 1997; Kurokawa and Ihara, 2009; Grujic et al., 2012; Van Segbroeck et al., 2012). This is surprising, because using the theory of ZD strategies, most of the successful strategies for the repeated prisoner׳s dilemma can be naturally generalized to other social dilemmas, with an arbitrary number of players (Hilbe et al., 2014b; Pan et al., 2014). In the public goods game, for example, the set of ZD strategies includes ALLD and ALLC, but also reciprocal strategies like proportional Tit-for-Tat (pTFT), extortionate and generous strategies. Herein, we have explored how these strategies fare from an evolutionary perspective.
Our simulations suggest that the evolutionary success of ZD strategies critically depends on the size of the group. In smaller groups, the dynamics of strategies is comparable to the dynamics in the prisoner׳s dilemma (Nowak and Sigmund, 1992; Imhof and Nowak, 2010; Hilbe et al., 2013a): selfish populations can be invaded by extortioners or pTFT, which in turn can give rise to the evolution of generous ZD strategies. Generous strategies, however, can be subverted by unconditional cooperators, which can lead back to populations of defectors. These evolutionary cycles collapse when groups become too large. In large groups, evolution favors selfish strategies instead, resulting in a sharp decrease in population payoffs. To obtain these results, we have sometimes restricted the strategy space, by focusing on players using ZD strategies only. This focus has allowed us to calculate payoffs efficiently. In general, the time to compute payoffs in multiplayer games increases exponentially in the size of the group (which makes it unfeasible to simulate games with more than 5–10 players). But for ZD strategies, payoffs can be computed directly, using the formula in Eq. (3). The focus on ZD strategies, however, may come at the risk of neglecting other important strategies, such as win-stay lose-shift (WSLS). Nevertheless, our main qualitative results remain unchanged even when we consider the more general space of memory-one strategies (as shown in Fig. 5).
Overall, we have observed that repeated interactions can only help sustaining cooperation when groups are sufficiently small. The downfall of cooperation in large groups can be prevented if large-scale endeavors have an efficiency advantage: Pinheiro et al. (2014) observe that WSLS remains successful if is kept constant (and therefore r needs to increase as n becomes large). However, for many examples (such as the management of common resources) such an efficiency advantage seems unfeasible. For such cases, our results suggest that repeated interactions alone are no longer able to sustain cooperation.
Yet, human societies are remarkably successful in maintaining cooperative norms even in groups of considerable size (Fehr and Fischbacher, 2003), suggesting that large-scale cooperation is based on additional mechanisms. Three mechanisms seem to be especially relevant: individuals can maintain cooperation if there are additional pairwise incentives to cooperate (Rand et al., 2009; Rockenbach and Milinski, 2006); they can increase their strategic power by coordinating their actions and by forming alliances (Hilbe et al., 2014b); or they can implement central institutions that enforce mutual cooperation (Sigmund et al., 2010, 2011; Sasaki et al., 2012; Hilbe et al., 2014b; Schoenmakers et al., 2014). Interestingly, each of these additional mechanisms is costly, and thus requires an evolutionary explanation on its own. In particular, these additional mechanisms are only likely to evolve when other, more efficient ways to establish cooperation fail. Herein, we have shown such a failure to establish cooperation when repeated interactions take place in large groups.
Acknowledgments
We would like to thank Ethan Akin for substantial suggestions concerning the material presented in Appendix A.2. Support from the John Templeton Foundation is gratefully acknowledged. C.H. acknowledges generous funding by the Schrödinger scholarship of the Austrian Science Fund (FWF) J3475.
Appendix A.
A.1. Payoffs in groups of memory-one players
In the following, we describe how one can calculate the average payoffs in a group, under the assumption that all players use memory-one strategies. To this end, consider a group of size n and suppose player i applies the memory-one strategy . To calculate payoffs, we use the Markov-chain approach presented in Hauert and Schuster (1997). The states of the Markov chain are the possible outcomes of a given round: if the action of player i in a given round is Si, then we can write the outcome of that round as a vector . Let denote the number of cooperators in σ. Given each player׳s memory-one strategy , and the outcome σ of the previous round, one can calculate the transition probability to observe the outcome in the next round. Since players act independently, is a product with n factors, , where
(9) |
The transition probabilities can be collected in a stochastic transition matrix . In most cases, this transition matrix has a unique left eigenvector with respect to the leading eigenvalue 1, such that , and . In that case, the entries give the fraction of rounds in which the players find themselves in state σ over the course of the game. For each of these states σ, we can define the resulting payoff for player i as
(10) |
As a result, we can calculate the average payoff πi of player i over the course of the repeated multiplayer game as
(11) |
In a few cases, however, the invariant distribution of the transition matrix M may not be unique. This happens, for example, when all players apply the strategy pTFT, such that the payoffs critically depend on the players׳ cooperation probabilities in the initial round. To circumvent these technical difficulties, we make the assumption that players sometimes commit errors with probability ε. In effect, this assumption implies that instead of the intended strategy , players use the memory-one strategy , where is the corresponding vector with all entries being one. For any , the resulting invariant distribution is unique. Payoffs are then defined by considering the limit when the error rate goes to zero (see also Section 3.14 in Sigmund, 2010),
(12) |
As an example, this definition of payoffs implies that a homogeneous group of pTFT-players yields a payoff of . For groups in which is unique, the payoff formulas (11) and (12) give the same result.
A.2. Payoffs in groups of ZD strategists
When all players of a group apply a ZD strategy, the calculation of payoffs becomes considerably more simple. To show this, let us consider a group of n players, where each of the players applies some ZD strategy with parameters li and si. It follows that each of the players enforces the payoff relationship
(13) |
Instead of a relationship between player i׳s payoff πi and the average payoff of the co-players π−i, we can rewrite this equation such that it is a function between πi and the average payoff of the group (including i)
(14) |
with . This confirms that player i׳s payoff can be calculated once the average payoff of the group is known. To calculate , we use elementary transformations to write the relationship (14) as
(15) |
with
(16) |
Summing up over all players then confirms that , and therefore
(17) |
Substituting this result into (15) then leads to the conclusion
(18) |
This allows a direct calculation of the payoffs from the parameters li and si.
A.3. Zero-determinant strategies for the public goods game
In the public goods game, cooperators contribute an amount into a common pool. Total contributions are then multiplied by some factor , and equally divided among all group members. In the following, we aim to provide an alternative characterization of ZD strategies for the public goods game (which does not depend on free parameters such as l, s, and ϕ).
Proposition 1 Characterization of ZD-strategies for the public goods game —
A memory-one strategy for the public goods game is a ZD strategy if and only if
(19)
Proof
- ()
By plugging the values of and into the definition of ZD strategies (1), it follows that any ZD strategy satisfies the following two conditions:Since the two terms on the right hand side coincide, it follows that the two expressions on the left hand side coincide, and thus
(20) for all j. Moreover, since the two terms on the right hand side of (20) are independent of j, we can also conclude that
(21) for and .
(22) - ()
Conversely, for a given memory-one strategy that satisfies Eq. (19), let us defineLet us first show that these parameters are independent of j. For s and ϕ, the independence of j follows immediately from conditions (5). But also the parameter l is independent of j, which can be shown by repeatedly using (5)
(23) Thus, the above parameters l, s, and ϕ are indeed independent of j. Then,
(24) which can be shown by plugging the values of l, s, and ϕ in (23) into the right hand side of (25). Overall, (25) confirms that is a ZD strategy. □
(25)
We note that characterization (19) does not depend on the game parameters: the set of ZD strategies is the same for all public good games (no matter what the multiplication factor r is). It also follows that all unconditional strategies (such as ALLC and ALLD) are ZD strategies.
References
- Abou Chakra M., Traulsen A. Under high stakes and uncertainty the rich should lend the poor a helping hand. J. Theoret. Biol. 2014;341:123–130. doi: 10.1016/j.jtbi.2013.10.004. [DOI] [PubMed] [Google Scholar]
- Adami C., Hintze A. Evolutionary instability of zero-determinant strategies demonstrates that winning is not everything. Nat. Commun. 2013;4:2193. doi: 10.1038/ncomms3193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Akin, E., 2013. Stable Cooperative Solutions for the Iterated Prisoner׳s Dilemma. arXiv, 1211.0969v2.
- Antal T., Traulsen A., Ohtsuki H., Tarnita C.E., Nowak M.A. Mutation-selection equilibrium in games with multiple strategies. J. Theoret. Biol. 2009;258:614–622. doi: 10.1016/j.jtbi.2009.02.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Archetti M. Cooperation as a volunteer׳s dilemma and the strategy of conflict in public goods games. J. Evol. Biol. 2009;11:2192–2200. doi: 10.1111/j.1420-9101.2009.01835.x. [DOI] [PubMed] [Google Scholar]
- Axelrod R., Hamilton W.D. The evolution of cooperation. Science. 1981;211:1390–1396. doi: 10.1126/science.7466396. [DOI] [PubMed] [Google Scholar]
- Bergstrom C.T., Lachmann M. The Red King Effect: when the slowest runner wins the coevolutionary race. Proc. Natl. Acad. Sci. USA. 2003;100:593–598. doi: 10.1073/pnas.0134966100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Blume L.E. The statistical mechanics of strategic interaction. Games Econ. Behav. 1993;5:387–424. [Google Scholar]
- Boyd R. Mistakes allow evolutionary stability in the repeated Prisoner׳s Dilemma game. J. Theoret. Biol. 1989;136:47–56. doi: 10.1016/s0022-5193(89)80188-2. [DOI] [PubMed] [Google Scholar]
- Boyd R., Lorberbaum J. No pure strategy is evolutionary stable in the iterated prisoner׳s dilemma game. Nature. 1987;327:58–59. [Google Scholar]
- Boyd R., Richerson P.J. The evolution of reciprocity in sizeable groups. J. Theoret. Biol. 1988;132:337–356. doi: 10.1016/s0022-5193(88)80219-4. [DOI] [PubMed] [Google Scholar]
- Chen J., Zinger A. The robustness of zero-determinant strategies in iterated prisoner׳s dilemma games. J. Theoret. Biol. 2014;357:46–54. doi: 10.1016/j.jtbi.2014.05.004. [DOI] [PubMed] [Google Scholar]
- Cressman R., Song J.-W., Zhang B.-Y., Tao Y. Cooperation and evolutionary dynamics in the public goods game with institutional incentives. J. Theoret. Biol. 2012;299:144–151. doi: 10.1016/j.jtbi.2011.07.030. [DOI] [PubMed] [Google Scholar]
- Damore J.A., Gore J. A slowly evolving host moves first in symbiotic interactions. Evolution. 2011;65(8):2391–2398. doi: 10.1111/j.1558-5646.2011.01299.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Diekmann A. Volunteer׳s dilemma. J. Confl. Resolut. 1985;29:605–610. [Google Scholar]
- Doebeli M., Hauert C. Models of cooperation based on the prisoner׳s dilemma and the snowdrift game. Ecol. Lett. 2005;8:748–766. [Google Scholar]
- Dreber A., Rand D.G., Fudenberg D., Nowak M.A. Winners don׳t punish. Nature. 2008;452:348–351. doi: 10.1038/nature06723. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Du J., Wu B., Altrock P.M., Wang L. Aspiration dynamics of multi-player games in finite populations. J. R. Soc. Interface. 2014;11(94):1742–5662. doi: 10.1098/rsif.2014.0077. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fehr E., Fischbacher U. The nature of human altruism. Nature. 2003;425:785–791. doi: 10.1038/nature02043. [DOI] [PubMed] [Google Scholar]
- Fischbacher U., Gächter S., Fehr E. Are people conditionally cooperative? Evidence from a public goods experiment. Econ. Lett. 2001;71:397–404. [Google Scholar]
- Fudenberg D., Imhof L.A. Imitation processes with small mutations. J. Econ. Theory. 2006;131:251–262. [Google Scholar]
- Gokhale C.S., Traulsen A. Evolutionary games in the multiverse. Proc. Natl. Acad. Sci. USA. 2010;107:5500–5504. doi: 10.1073/pnas.0912214107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gokhale, C.S., Traulsen, A., 2014. Evolutionary multiplayer games. Dyn. Games Appl. 4 (4), 468-488.
- Grujic J., Cuesta J.A., Sánchez A. On the coexistence of cooperators, defectors and conditional cooperators in the multiplayer iterated prisoner׳s dilemma. J. Theoret. Biol. 2012;300:299–308. doi: 10.1016/j.jtbi.2012.02.003. [DOI] [PubMed] [Google Scholar]
- Grujic, J., Gracia-Lázaro, C., Milinski, M., Semmann, D., Traulsen, A., Cuesta, J.A., Moreno, Y., Sánchez, A., 2014. A comparative analysis of spatial prisoner׳s dilemma experiments: conditional cooperation and payoff irrelevance. Sci. Rep. 4, 4615. [DOI] [PMC free article] [PubMed]
- Hauert C., Schuster H.G. Effects of increasing the number of players and memory size in the iterated prisoner׳s dilemma: a numerical approach. Proc. R. Soc. B. 1997;264:513–519. [Google Scholar]
- Hilbe C., Nowak M.A., Sigmund K. The evolution of extortion in iterated prisoner׳s dilemma games. Proc. Natl. Acad. Sci. USA. 2013;110:6913–6918. doi: 10.1073/pnas.1214834110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hilbe C., Nowak M.A., Traulsen A. Adaptive dynamics of extortion and compliance. PLoS One. 2013;8:e77886. doi: 10.1371/journal.pone.0077886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hilbe C., Röhl T., Milinski M. Extortion subdues human players but is finally punished in the prisoner׳s dilemma. Nat. Commun. 2014;5:3976. doi: 10.1038/ncomms4976. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hilbe C., Wu B., Traulsen A., Nowak M.A. Cooperation and control in multiplayer social dilemmas. Proc. Natl. Acad. Sci. USA. 2014;111(46):16425–16430. doi: 10.1073/pnas.1407887111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hilbe, C., Traulsen, A., Sigmund, K., 2015. Partners or rivals? Strategies for the iterated prisoner׳s dilemma. working paper [DOI] [PMC free article] [PubMed]
- Imhof L.A., Fudenberg D., Nowak M.A. Evolutionary cycles of cooperation and defection. Proc. Natl. Acad. Sci. USA. 2005;102:10797–10800. doi: 10.1073/pnas.0502589102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Imhof L.A., Nowak M.A. Evolutionary game dynamics in a Wright–Fisher process. J. Math. Biol. 2006;52:667–681. doi: 10.1007/s00285-005-0369-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Imhof L.A., Nowak M.A. Stochastic evolutionary dynamics of direct reciprocity. Proc. R. Soc. B. 2010;277:463–468. doi: 10.1098/rspb.2009.1171. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kerr B., Godfrey-Smith P., Feldman M.W. What is altruism? TREE. 2004;19(3):135–140. doi: 10.1016/j.tree.2003.10.004. [DOI] [PubMed] [Google Scholar]
- Keser C., van Winden F. Conditional cooperation and voluntary contributions to public goods. Scand. J. Econ. 2000;102:23–39. [Google Scholar]
- Kraines D.P., Kraines V.Y. Pavlov and the prisoner׳s dilemma. Theory Decis. 1989;26(1):47–79. [Google Scholar]
- Kurokawa S., Ihara Y. Emergence of cooperation in public goods games. Proc. R. Soc. B. 2009;276:1379–1384. doi: 10.1098/rspb.2008.1546. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ledyard J.O. Public goods: a survey of experimental research. In: Kagel J.H., Roth A.E., editors. The Handbook of Experimental Economics. Princeton University Press; Princeton: 1995. pp. 111–194. [Google Scholar]
- Milinski M. Tit For Tat in sticklebacks and the evolution of cooperation. Nature. 1987;325(6103):433–435. doi: 10.1038/325433a0. [DOI] [PubMed] [Google Scholar]
- Milinski M., Sommerfeld R.D., Krambeck H.-J., Reed F.A., Marotzke J. The collective-risk social dilemma and the prevention of simulated dangerous climate change. Proc. Natl. Acad. Sci. USA. 2008;105(7):2291–2294. doi: 10.1073/pnas.0709546105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Molander P. The optimal level of generosity in a selfish, uncertain environment. J. Confl. Resolut. 1985;29:611–618. [Google Scholar]
- Nowak M.A. Five rules for the evolution of cooperation. Science. 2006;314:1560–1563. doi: 10.1126/science.1133755. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nowak M.A., Sasaki A., Taylor C., Fudenberg D. Emergence of cooperation and evolutionary stability in finite populations. Nature. 2004;428:646–650. doi: 10.1038/nature02414. [DOI] [PubMed] [Google Scholar]
- Nowak M.A., Sigmund K. Oscillations in the evolution of reciprocity. J. Theoret. Biol. 1989;137:21–26. doi: 10.1016/s0022-5193(89)80146-8. [DOI] [PubMed] [Google Scholar]
- Nowak M.A., Sigmund K. Tit for tat in heterogeneous populations. Nature. 1992;355:250–253. [Google Scholar]
- Nowak M.A., Sigmund K. Chaos and the evolution of cooperation. Proc. Natl. Acad. Sci. USA. 1993;90:5091–5094. doi: 10.1073/pnas.90.11.5091. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nowak M.A., Sigmund K. A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner׳s Dilemma game. Nature. 1993;364:56–58. doi: 10.1038/364056a0. [DOI] [PubMed] [Google Scholar]
- Ostrom E. Cambridge University Press; Cambridge, UK: 1990. Governing the Commons: The Evolution of Institutions for Collective Action. [Google Scholar]
- Pan, L., Hao, D., Rong, Z., Zhou, T., 2014. Zero-Determinant Strategies in the Iterated Public Goods Game. arXiv, 1402.3542v1. [DOI] [PMC free article] [PubMed]
- Peña J., Lehmann L., Nöldeke G. Gains from switching and evolutionary stability in multi-player matrix games. J. Theoret. Biol. 2014;346:23–33. doi: 10.1016/j.jtbi.2013.12.016. [DOI] [PubMed] [Google Scholar]
- Pinheiro F.L., Vasconcelos V., Santos F.C., Pacheco J.M. Evolution of All-or-None strategies in repeated public goods dilemmas. PLoS Comput. Biol. 2014;10(11):e1003945. doi: 10.1371/journal.pcbi.1003945. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Press W.H., Dyson F.D. Iterated prisoner׳s dilemma contains strategies that dominate any evolutionary opponent. Proc. Natl. Acad. Sci. USA. 2012;109:10409–10413. doi: 10.1073/pnas.1206569109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rand D.G., Dreber A., Ellingsen T., Fudenberg D., Nowak M.A. Positive interactions promote public cooperation. Science. 2009;325:1272–1275. doi: 10.1126/science.1177418. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rapoport A., Chammah A.M. University of Michigan Press; Ann Arbor: 1965. Prisoner׳s Dilemma. [Google Scholar]
- Rockenbach B., Milinski M. The efficient interaction of indirect reciprocity and costly punishment. Nature. 2006;444:718–723. doi: 10.1038/nature05229. [DOI] [PubMed] [Google Scholar]
- Santos F.C., Pacheco J.M. Risk of collective failure provides an escape from the tragedy of the commons. Proc. Natl. Acad. Sci. USA. 2011;108:10421–10425. doi: 10.1073/pnas.1015648108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sasaki T., Brännström Å., Dieckmann U., Sigmund K. The take-it-or-leave-it option allows small penalties to overcome social dilemmas. Proc. Natl. Acad. Sci. USA. 2012;109:1165–1169. doi: 10.1073/pnas.1115219109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schoenmakers S., Hilbe C., Blasius B., Traulsen A. Sanctions as honest signals—the evolution of pool punishment by public sanctioning institutions. J. Theoret. Biol. 2014;356:36–46. doi: 10.1016/j.jtbi.2014.04.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sigmund K. Princeton University Press; Princeton: 2010. The Calculus of Selfishness. [Google Scholar]
- Sigmund K., De Silva H., Traulsen A., Hauert C. Social learning promotes institutions for governing the commons. Nature. 2010;466:861–863. doi: 10.1038/nature09203. [DOI] [PubMed] [Google Scholar]
- Sigmund K., Hauert C., Traulsen A., De Silva H. Social control and the social contract: the emergence of sanctioning systems for collective action. Dyn. Games Appl. 2011;1:149–171. [Google Scholar]
- St. Pierre A., Larose K., Dubois F. Long-term social bonds promote cooperation in the iterated prisoner׳s dilemma. Proc. R. Soc. B. 2009;27(1676):4223–4228. doi: 10.1098/rspb.2009.1156. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stephens D.W., McLinn C.M., Stevens J.R. Discounting and reciprocity in an iterated prisoner׳s dilemma. Science. 2002;298:2216–2218. doi: 10.1126/science.1078498. [DOI] [PubMed] [Google Scholar]
- Stewart A.J., Plotkin J.B. Extortion and cooperation in the prisoner׳s dilemma. Proc. Natl. Acad. Sci. USA. 2012;109:10134–10135. doi: 10.1073/pnas.1208087109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stewart A.J., Plotkin J.B. From extortion to generosity, evolution in the iterated prisoner׳s dilemma. Proc. Natl. Acad. Sci. USA. 2013;110(38):15348–15353. doi: 10.1073/pnas.1306246110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Szabó G., Tőke C. Evolutionary prisoner׳s dilemma game on a square lattice. Phys. Rev. E. 1998;58:69–73. [Google Scholar]
- Szolnoki A., Perc M. Defection and extortion as unexpected catalysts of unconditional cooperation in structured populations. Sci. Rep. 2014;4:5496. doi: 10.1038/srep05496. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Szolnoki A., Perc M. Evolution of extortion in structured populations. Phys. Rev. E. 2014;89(2):022804. doi: 10.1103/PhysRevE.89.022804. [DOI] [PubMed] [Google Scholar]
- Traulsen A., Nowak M.A., Pacheco J.M. Stochastic dynamics of invasion and fixation. Phys. Rev. E. 2006;74:011909. doi: 10.1103/PhysRevE.74.011909. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Traulsen A., Röhl T., Milinski M. An economic experiment reveals that humans prefer pool punishment to maintain the commons. Proc. R. Soc. B. 2012;279:3716–3721. doi: 10.1098/rspb.2012.0937. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Trivers R.L. The evolution of reciprocal altruism. Q. Rev. Biol. 1971;46:35–57. [Google Scholar]
- Van Segbroeck S., Pacheco J.M., Lenaerts T., Santos F.C. Emergence of fairness in repeated group interactions. Phys. Rev. Lett. 2012;108:158104. doi: 10.1103/PhysRevLett.108.158104. [DOI] [PubMed] [Google Scholar]
- van Veelen M., García J., Rand D.G., Nowak M.A. Direct reciprocity in structured populations. Proc. Natl. Acad. Sci. USA. 2012;109:9929–9934. doi: 10.1073/pnas.1206694109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- van Veelen M., Nowak M.A. Multi-player games on the cycle. J. Theoret. Biol. 2012;292:116–128. doi: 10.1016/j.jtbi.2011.08.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wedekind C., Milinski M. Human cooperation in the simultaneous and the alternating prisoner׳s dilemma: Pavlov versus generous tit-for-tat. Proc. Natl. Acad. Sci. USA. 1996;93(7):2686–2689. doi: 10.1073/pnas.93.7.2686. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wilkinson G.S. Reciprocal food-sharing in the vampire bat. Nature. 1984;308:181–184. [Google Scholar]
- Wu B., Gokhale C.S., Wang L., Traulsen A. How small are small mutation rates? J. Math. Biol. 2012;64:803–827. doi: 10.1007/s00285-011-0430-8. [DOI] [PubMed] [Google Scholar]
- Zhang B., Li C., De Silva H., Bednarik P., Sigmund K. The evolution of sanctioning institutions: an experimental approach to the social contract. Exp. Econ. 2013;17:285–303. [Google Scholar]