Cooperation and control in multiplayer social dilemmas

Significance

Many of the world’s most pressing problems, like the prevention of climate change, have the form of a large-scale social dilemma with numerous involved players. Previous results in evolutionary game theory suggest that multiplayer dilemmas make it particularly difficult to achieve mutual cooperation because of the lack of individual control in large groups. Herein, we extend the theory of zero-determinant strategies to multiplayer games to describe which strategies maintain cooperation. Moreover, we propose two simple models of alliances in multiplayer dilemmas. The effect of these alliances is determined by their size, the strategy of the allies, and the properties of the social dilemma. When a single individual’s strategic options are limited, forming an alliance can result in a drastic leverage.

Abstract

Direct reciprocity and conditional cooperation are important mechanisms to prevent free riding in social dilemmas. However, in large groups, these mechanisms may become ineffective because they require single individuals to have a substantial influence on their peers. However, the recent discovery of zero-determinant strategies in the iterated prisoner’s dilemma suggests that we may have underestimated the degree of control that a single player can exert. Here, we develop a theory for zero-determinant strategies for iterated multiplayer social dilemmas, with any number of involved players. We distinguish several particularly interesting subclasses of strategies: fair strategies ensure that the own payoff matches the average payoff of the group; extortionate strategies allow a player to perform above average; and generous strategies let a player perform below average. We use this theory to describe strategies that sustain cooperation, including generalized variants of Tit-for-Tat and Win-Stay Lose-Shift. Moreover, we explore two models that show how individuals can further enhance their strategic options by coordinating their play with others. Our results highlight the importance of individual control and coordination to succeed in large groups.

Cooperation among self-interested individuals is generally difficult to achieve (1–3), but typically the free rider problem is aggravated even further when groups become large (4–9). In small communities, cooperation can often be stabilized by forms of direct and indirect reciprocity (10–17). For large groups, however, it has been suggested that these mechanisms may turn out to be ineffective, as it becomes more difficult to keep track of the reputation of others and because the individual influence on others diminishes (4–8). To prevent the tragedy of the commons and to compensate for the lack of individual control, many successful communities have thus established central institutions that enforce mutual cooperation (18–22).

However, a recent discovery suggests that we may have underestimated the amount of control that single players can exert in repeated games. For the repeated prisoner’s dilemma, Press and Dyson (23) have shown the existence of zero-determinant strategies (or ZD strategies), which allow a player to unilaterally enforce a linear relationship between the own payoff and the coplayer’s payoff, irrespective of the coplayer’s actual strategy. The class of zero-determinant strategies is surprisingly rich: for example, a player who wants to ensure that the own payoff will always match the coplayer’s payoff can do so by applying a fair ZD strategy, like Tit-for-Tat. On the other hand, a player who wants to outperform the respective opponent can do so by slightly tweaking the Tit-for-Tat strategy to the own advantage, thereby giving rise to extortionate ZD strategies. The discovery of such strategies has prompted several theoretical studies, exploring how different ZD strategies evolve under various evolutionary conditions (24–30).

ZD strategies are not confined to the repeated prisoner’s dilemma. Recently published studies have shown that ZD strategies also exist in other repeated two player games (29) or in repeated public goods games (31). Herein, we will show that such strategies exist for all symmetric social dilemmas, with an arbitrary number of participants. We use this theory to describe which ZD strategies can be used to enforce fair outcomes or to prevent free riders from taking over. Our results, however, are not restricted to the space of ZD strategies. By extending the techniques introduced by Press and Dyson (23) and Akin (27), we also derive exact conditions when generalized versions of Grim, Tit-for-Tat, and Win-Stay Lose-Shift allow for stable cooperation. In this way, we find that most of the theoretical solutions for the repeated prisoner’s dilemma can be directly transferred to repeated dilemmas with an arbitrary number of involved players.

In addition, we also propose two models to explore how individuals can further enhance their strategic options by coordinating their play with others. To this end, we extend the notion of ZD strategies for single players to subgroups of players (to which we refer as ZD alliances). We analyze two models of ZD alliances, depending on the degree of coordination between the players. When players form a strategy alliance, they only agree on the set of alliance members, and on a common strategy that each alliance member independently applies during the repeated game. When players form a synchronized alliance, on the other hand, they agree to act as a single entity, with all alliance members playing the same action in a given round. We show that the strategic power of ZD alliances depends on the size of the alliance, the applied strategy of the allies, and on the properties of the underlying social dilemma. Surprisingly, the degree of coordination only plays a role as alliances become large (in which case a synchronized alliance has more strategic options than a strategy alliance).

To obtain these results, we consider a repeated social dilemma between n players. In each round of the game, players can decide whether to cooperate (C) or to defect (D). A player’s payoff depends on the player’s own decision and on the decisions of all other group members (Fig. 1A): in a group in which j of the other group members cooperate, a cooperator receives the payoff $a_j$, whereas a defector obtains $b_j$. We assume that payoffs satisfy the following three properties that are characteristic for social dilemmas (corresponding to the individual-centered interpretation of altruism in ref. 32): (i) irrespective of the own strategy, players prefer the other group members to cooperate ($a_{j+1}\ge a_j$ and $b_{j+1}\ge b_j$ for all j); (ii) within any mixed group, defectors obtain strictly higher payoffs than cooperators ($b_{j+1}>a_j$ for all j); and (iii) mutual cooperation is favored over mutual defection $\left(a_{n-1}>b_0\right)$. To illustrate our results, we will discuss two particular examples of multiplayer games (Fig. 1B). In the first example, the public goods game (33), cooperators contribute an amount $c>0$ to a common pool, knowing that total contributions are multiplied by r (with $1<r<n$) and evenly shared among all group members. Thus, a cooperator’s payoff is $a_j=rc\left(j+1\right)/n-c$, whereas defectors yield $b_j=rcj/n$. In the second example, the volunteer’s dilemma (34), at least one group member has to volunteer to bear a cost $c>0$ in order for all group members to derive a benefit $b>c$. Therefore, cooperators obtain $a_j=b-c$ (irrespective of j), whereas defectors yield $b_j=b$ if $j\ge 1$ and $b_0=0$. Both examples (and many more, such as the collective risk dilemma) (7, 8, 35) are simple instances of multiplayer social dilemmas.

Fig. 1.

Illustration of the model assumptions for repeated social dilemmas. (A) We consider symmetric n−player social dilemmas in which each player can either cooperate or defect. The player’s payoff depends on its own decision and on the number of other group members who decide to cooperate. (B) We will discuss two particular examples: the public goods game (in which payoffs are proportional to the number of cooperators) and the volunteer’s dilemma (as the most simple example of a nonlinear social dilemma). (C) In addition to individual strategies, we will also explore how subjects can enhance their strategic options by coordinating their play with other group members. We refer to the members of such a ZD alliance as allies, and we call group members that are not part of the ZD alliance outsiders. Outsiders are not restricted to any particular strategy. Some or all of the outsiders may even form their own alliance.

We assume that the social dilemma is repeated, such that individuals can react to their coplayers’ past actions (for simplicity, we will focus here on the case of an infinitely repeated game). As usual, payoffs for the repeated game are defined as the average payoff that players obtain over all rounds. In general, strategies for such repeated games can become arbitrarily complex, as subjects may condition their behavior on past events and on the round number in nontrivial ways. Nevertheless, as in pairwise games, ZD strategies turn out to be surprisingly simple.

Results

Memory-One Strategies and Akin’s Lemma.

ZD strategies are memory-one strategies (23, 36); they only condition their behavior on the outcome of the previous round. Memory-one strategies can be written as a vector $\mathbf{p}=\left(p_{C,n-1},\dots ,p_{C,0},p_{D,n-1},\dots ,p_{D,0}\right)$. The entries $p_{S,j}$ denote the probability to cooperate in the next round, given that the player previously played $S\in \left\{C,D\right\}$ and that j of the coplayers cooperated (in the SI Text, we present an extension in which players additionally take into account who of the coplayers cooperated). A simple example of a memory-one strategy is the strategy Repeat, ${\mathbf{p}}^{Rep}$, which simply reiterates the own move of the previous round, $p_{C,j}^{Rep}=1$ and $p_{D,j}^{Rep}=0$. In addition, memory-one strategies need to specify a cooperation probability $p_0$ for the first round. However, our results will often be independent of the initial play, and in that case we will drop $p_0$.

Let us consider a repeated game in which a focal player with memory-one strategy p interacts with $n-1$ arbitrary coplayers (who are not restricted to any particular strategy). Let $v_{S,j}\left(t\right)$ denote the probability that the outcome of round t is $\left(S,j\right)$. Let $\mathbf{v}\left(t\right)=\left[v_{C,n-1}\left(t\right),\dots ,v_{D,0}\left(t\right)\right]$ be the vector of these probabilities. A limit distribution v is a limit point for $t\to \infty$ of the sequence $\left[\mathbf{v}\left(1\right)+\dots +\mathbf{v}\left(t\right)\right]/t$. The entries $v_{S,j}$ of such a limit distribution correspond to the fraction of rounds in which the focal player finds herself in state $\left(S,j\right)$ over the course of the game.

There is a surprisingly powerful relationship between a focal player’s memory-one strategy and the resulting limit distribution of the iterated game. To show this relationship, let $q_C\left(t\right)$ be the probability that the focal player cooperates in round t. By definition of ${\mathbf{p}}^{Rep}$ we can write $q_C\left(t\right)={\mathbf{p}}^{Rep}\cdot \mathbf{v}\left(t\right)=\left[v_{C,n-1}\left(t\right)+\dots +v_{C,0}\left(t\right)\right]$. Similarly, we can express the probability that the focal player cooperates in the next round as $q_C\left(t+1\right)=\mathbf{p}\cdot \mathbf{v}\left(t\right)$. It follows that $q_C\left(t+1\right)-q_C\left(t\right)=\left(\mathbf{p}-{\mathbf{p}}^{Rep}\right)\cdot \mathbf{v}\left(t\right)$. Summing up over all rounds from 1 to t, and dividing by t, yields $\left(\mathbf{p}-{\mathbf{p}}^{Rep}\right)\cdot \left[\mathbf{v}\left(1\right)+\dots \mathbf{v}\left(t\right)\right]/t=\left[q_C\left(t+1\right)-q_C\left(1\right)\right]/t$, which has absolute value at most $1/t$. By taking the limit $t\to \infty$ we can conclude that

$$\left(\mathbf{p}-{\mathbf{p}}^{Rep}\right)\cdot \mathbf{v}=0.$$ [1]

This relation between a player’s memory-one strategy and the resulting limit distribution will prove to be extremely useful. Because the importance of Eq. 1 has been first highlighted by Akin (27) in the context of the pairwise prisoner’s dilemma, we will refer to it as Akin’s lemma. We note that Akin’s lemma is remarkably general, because it neither makes any assumptions on the specific game being played, nor does it make any restrictions on the strategies applied by the remaining $n-1$ group members.

Zero-Determinant Strategies in Multiplayer Social Dilemmas.

As an application of Akin’s lemma, we will show in the following that single players can gain an unexpected amount of control over the resulting payoffs in a multiplayer social dilemma. To this end, we first need to introduce some further notation. For a focal player i, let us write the possible payoffs in a given round as a vector ${\mathbf{g}}^i=\left(g_{S,j}^i\right)$, with $g_{C,j}^i=a_j$ and $g_{D,j}^i=b_j$. Similarly, let us write the average payoffs of i’s coplayers as ${\mathbf{g}}^{-i}=\left(g_{S,j}^{-i}\right)$, where the entries are given by $g_{C,j}^{-i}=\left[j{a}_j+\left(n-j-1\right)b_{j+1}\right]/\left(n-1\right)$ and $g_{D,j}^{-i}=\left[j{a}_{j-1}+\left(n-j-1\right)b_j\right]/\left(n-1\right)$. Finally, let 1 denote the $2n$-dimensional vector with all entries being one. Using this notation, we can write player i’s payoff in the repeated game as ${\pi }^i={\mathbf{g}}^i\cdot \mathbf{v}$, and the average payoff of i’s coplayers as ${\pi }^{-i}={\mathbf{g}}^{-i}\cdot \mathbf{v}$. Moreover, by definition of v as a limit distribution, it follows that $\mathbf{1}\cdot \mathbf{v}=1$. After these preparations, let us assume player i applies the memory-one strategy

$$\mathbf{p}={\mathbf{p}}^{Rep}+\alpha {\mathbf{g}}^i+\beta {\mathbf{g}}^{-i}+\gamma \mathbf{1},$$ [2]

with α, β, and γ being parameters that can be chosen by player i (with the only restriction that $\beta \ne 0$). Due to Akin’s lemma, we can conclude that such a player enforces the relationship

$$0=\left(\mathbf{p}-{\mathbf{p}}^{Rep}\right)\cdot \mathbf{v}=\left(\alpha {\mathbf{g}}^i+\beta {\mathbf{g}}^{-i}+\gamma \mathbf{1}\right)\mathbf{v}=\alpha {\pi }^i+\beta {\pi }^{-i}+\gamma .$$ [3]

Player i’s strategy thus guarantees that the resulting payoffs of the repeated game obey a linear relationship, irrespective of how the other group members play. Moreover, by appropriately choosing the parameters α, β, and γ, the player has direct control on the form of this payoff relation. As in Press and Dyson (23), who were first to discover such strategies for the prisoner’s dilemma, we refer to the memory-one strategies in Eq. 2 as zero-determinant strategies or ZD strategies.

For our purpose, it will be convenient to proceed with a slightly different representation of ZD strategies. Using the parameter transformation $l=-\gamma /\left(\alpha +\beta \right)$, $s=-\alpha /\beta$, and $\varphi =-\beta$, ZD strategies take the form

$$\mathbf{p}={\mathbf{p}}^{Rep}+\varphi \left[\left(1-s\right)\left(l\mathbf{1}-{\mathbf{g}}^i\right)+{\mathbf{g}}^i-{\mathbf{g}}^{-i}\right],$$ [4]

and the enforced payoff relationship according to Eq. 3 becomes

$${\pi }^{-i}=s{\pi }^i+\left(1-s\right)l.$$ [5]

We refer to l as the baseline payoff of the ZD strategy and to s as the strategy’s slope. Both parameters allow an intuitive interpretation: when all players adopt the same ZD strategy p such that ${\pi }^i={\pi }^{-i}$, it follows from Eq. 5 that each player yields the payoff l. The value of s determines how the mean payoff of the other group members ${\pi }^{-i}$ varies with ${\pi }^i$. The parameter ϕ does not have a direct effect on Eq. 5; however, the magnitude of ϕ determines how fast payoffs converge to this linear payoff relationship as the repeated game proceeds (37).

The parameters $l,\hspace{0.5em}s,$ and ϕ of a ZD strategy cannot be chosen arbitrarily, because the entries $p_{S,j}$ are probabilities that need to satisfy $0\le p_{S,j}\le 1$. In general, the admissible parameters depend on the specific social dilemma being played. In SI Text, we show that exactly those relations 5 can be enforced for which either $s=1$ (in which case the parameter l in the definition of ZD strategies becomes irrelevant) or for which l and $s<1$ satisfy

$$\underset{0\le j\le n-1}{\text{max}}\left\{b_j-\frac{j}{n-1}\frac{b_j-a_{j-1}}{1-s}\right\}\le l\le \underset{0\le j\le n-1}{\text{min}}\left\{a_j+\frac{n-j-1}{n-1}\frac{b_{j+1}-a_j}{1-s}\right\}.$$ [6]

It follows that feasible baseline payoffs are bounded by the payoffs for mutual cooperation and mutual defection, $b_0\le l\le a_{n-1}$, and that the slope needs to satisfy $-1/\left(n-1\right)\le s\le 1$. With s sufficiently close to 1, any baseline payoff between $b_0$ and $a_{n-1}$ can be achieved. Moreover, because the conditions in Eq. 6 become increasingly restrictive as the group size n increases, larger groups make it more difficult for players to enforce specific payoff relationships.

Important Examples of ZD Strategies.

In the following, we discuss some examples of ZD strategies. At first, let us consider a player who sets the slope to $s=1$. By Eq. 5, such a player enforces the payoff relation ${\pi }^i={\pi }^{-i}$, such that i’s payoff matches the average payoff of the other group members. We call such ZD strategies fair. As shown in Fig. 2A, fair strategies do not ensure that all group members get the same payoff; due to our definition of social dilemmas, unconditional defectors always outperform unconditional cooperators, no matter whether the group also contains fair players. Instead, fair players can only ensure that they do not take any unilateral advantage of their peers. Our characterization 6 implies that all social dilemmas permit a player to be fair, irrespective of the group size. As an example, consider the strategy proportional Tit-for-Tat (pTFT), for which the probability to cooperate is simply given by the fraction of cooperators among the coplayers in the previous round

$${\mathit{pTFT}}_{S,j}=\frac{j}{n-1}.$$ [7]

For pairwise games, this definition of pTFT simplifies to Tit-for-Tat, which is a fair ZD strategy (23). However, also for the public goods game and for the volunteer’s dilemma, pTFT is a ZD strategy, because it can be obtained from Eq. 4 by setting $s=1$ and $\varphi =1/c$, with c being the cost of cooperation.

Fig. 2.

Characteristic dynamics of payoffs over the course of the game for three different ZD strategies. Each panel depicts the payoff of the focal player ${\pi }^i$ (blue) and the average payoff of the other group members ${\pi }^{-i}$ (red) by thick lines. Additionally, the individual payoffs of the other group members are shown as thin red lines. (A) A fair player ensures that the own payoff matches the mean payoff of the other group members. However, fair strategies cannot ensure that all group members yield the same payoff. (B) For games in which mutual defection leads to the lowest group payoff, extortionate players ensure that their payoffs are above average. (C) In games in which mutual cooperation is the social optimum, generous players let their coplayers gain higher payoffs. The three graphs depict the case of a public goods game with $r=4$, $c=1$, and group size $n=20$. For the strategies of the other group members, we used random memory-one strategies, where the cooperation probabilities were independently drawn from a uniform distribution. For the strategies of the focal player, we used (A) pTFT, (B) ${\mathbf{p}}^{Ex}$ with $s=0.8$, and (C) ${\mathbf{p}}^{Ge}$ with $s=0.8$.

As another interesting subclass of ZD strategies, let us consider strategies that choose the mutual defection payoff as baseline payoff, $l=b_0$, and that enforce a positive slope $0<s<1$. The enforced payoff relation 5 becomes ${\pi }^{-i}=s{\pi }^i+\left(1-s\right)b_0$, implying that on average the other group members only get a fraction s of any surplus over the mutual defection payoff. Moreover, as the slope s is positive, the payoffs ${\pi }^i$ and ${\pi }^{-i}$ are positively related. As a consequence, the collective best reply for the remaining group members is to maximize i’s payoffs by cooperating in every round. In analogy to Press and Dyson (23), we call such ZD strategies extortionate, and we call the quantity $\chi =1/s$ the extortion factor. For games in which $l=b_0=0$, Eq. 5 shows that the extortion factor can be written as $\chi ={\pi }^i/{\pi }^{-i}$. Large extortion factors thus signal a substantial inequality in favor of player i. Extortionate strategies are particularly powerful in social dilemmas in which mutual defection leads to the lowest group payoff (as in the public goods game and in the volunteer’s dilemma). In that case, they enforce the relation ${\pi }^i\ge {\pi }^{-i}$; on average, player i performs at least as well as the other group members (as also depicted in Fig. 2B). As an example, let us consider a public goods game and a ZD strategy ${\mathbf{p}}^{Ex}$ with $l=0$, $\varphi =n/\left[\left(n-r\right)sc+rc\right]$, for which Eq. 4 implies

$$p_{S,j}^{Ex}=\frac{j}{n-1}\left[1-\left(1-s\right)\frac{n\left(r-1\right)}{r+\left(n-r\right)s}\right],$$ [8]

independent of the player’s own move $S\in \left\{C,D\right\}$. In the limit $s\to 1$, ${\mathbf{p}}^{Ex}$ approaches the fair strategy pTFT. As s decreases from 1, the cooperation probabilities of ${\mathbf{p}}^{Ex}$ are increasingly biased to the own advantage. Extortionate strategies exist for all social dilemmas (this follows from condition [6] by setting $l=b_0$ and choosing an s close to 1). However, larger groups make extortion more difficult. For example, in public goods games with $n>r/\left(r-1\right)$, players cannot be arbitrarily extortionate any longer as [6] implies that there is an upper bound on χ (SI Text).

As the benevolent counterpart to extortioners, Stewart and Plotkin described a set of generous strategies for the iterated prisoner’s dilemma (24, 28). Generous players set the baseline payoff to the mutual cooperation payoff $l=a_{n-1}$ while still enforcing a positive slope $0<s<1$. These parameter choices result in the payoff relation ${\pi }^{-i}=s{\pi }^i+\left(1-s\right)a_{n-1}$. In particular, for games in which mutual cooperation is the optimal outcome for the group (as in the public goods game and in the prisoner’s dilemma but not in the volunteer’s dilemma), the payoff of a generous player satisfies ${\pi }^i\le {\pi }^{-i}$ (Fig. 2C). For the example of a public goods game, we obtain a generous ZD strategy ${\mathbf{p}}^{Ge}$ by setting $l=rc-c$ and $\varphi =n/\left[\left(n-r\right)sc+rc\right]$, such that

$$p_{S,j}^{Ge}=\frac{j}{n-1}+\left(1-s\right)\frac{n-j-1}{n-1}\frac{n\left(r-1\right)}{r+\left(n-r\right)s}.$$ [9]

For $s\to 1$, ${\mathbf{p}}^{Ge}$ approaches the fair strategy pTFT, whereas lower values of s make ${\mathbf{p}}^{Ge}$ more cooperative. Again, such generous strategies exist for all social dilemmas, but the extent to which players can be generous depends on the particular social dilemma and on the size of the group.

As a last interesting class of ZD strategies, let us consider players who choose $s=0$. By Eq. 5, such players enforce the payoff relation ${\pi }^{-i}=l$, meaning that they have unilateral control over the mean payoff of the other group members (for the prisoner’s dilemma, such equalizer strategies were first discovered in ref. 38). However, unlike extortionate and generous strategies, equalizer strategies typically cease to exist once the group size exceeds a critical threshold. For the example of a public goods game this threshold is given by $n=2r/\left(r-1\right)$. For larger groups, single players cannot determine the mean payoff of their peers any longer.

Stable Cooperation in Multiplayer Social Dilemmas.

Let us next explore which ZD strategies give rise to a Nash equilibrium with stable cooperation. In SI Text, we prove that such ZD strategies need to have two properties: they need to be generous (by setting $l=a_{n-1}$ and $s>0$), but they must not be too generous [the slope needs to satisfy $s\ge \left(n-2\right)/\left(n-1\right)$]. In particular, whereas in the repeated prisoner’s dilemma any generous strategy with $s>0$ is a Nash equilibrium (27, 28), larger group sizes make it increasingly difficult to uphold cooperation. In the limit of infinitely large groups, it follows that s needs to approach 1, suggesting that ZD strategies need to become fair. For the public goods game, this implies that stable cooperation can always be achieved when players cooperate in the first round and adopt proportional Tit-for-Tat thereafter. Interestingly, this strategy has received little attention in the previous literature. Instead, researchers have focused on other generalized versions of Tit-for-Tat, which cooperate if at least k coplayers cooperated in the previous round (4, 39, 40). Such memory-one strategies take the form $p_{S,j}=0$ if $j<k$ and $p_{S,j}=1$ if $j\ge k$. Unlike pTFT, these threshold strategies neither enforce a linear relation between payoffs, nor do they induce fair outcomes, suggesting that pTFT may be the more natural generalization of Tit-for-Tat in large-scale social dilemmas.

In addition to the stable ZD strategies, Akin’s lemma also allows us to characterize all pure memory-one strategies that sustain mutual cooperation. In SI Text, we show that any such strategy p needs to satisfy the following four conditions

$$p_{C,n-1}=1,\hspace{1em}{p}_{C,n-2}=0,\hspace{1em}{p}_{D,1}\le \frac{a_{n-1}-a_0}{b_{n-1}-a_{n-1}},\hspace{1em}\text{and}\hspace{1em}{p}_{D,0}\le \frac{a_{n-1}-b_0}{b_{n-1}-a_{n-1}},$$ [10]

with no restrictions being imposed on the other entries $p_{S,j}$. The first condition $p_{C,n-1}=1$ ensures that individuals continue to play C after mutual cooperation; the second condition $p_{C,n-2}=0$ guarantees that any unilateral deviation is punished; and the last two conditions describe whether players are allowed to revert to cooperation after rounds with almost uniform defection. Surprisingly, only these last two conditions depend on the specific payoffs of the social dilemma. As an application, condition 10 imply that the threshold variants of Tit-for-Tat discussed above are only a Nash equilibrium if they use the most stringent threshold: $k=n-1$. Such unforgiving strategies, however, have the disadvantage that they are often susceptible to errors: already a small probability that players fail to cooperate may cause a complete breakdown of cooperation (41). Instead, the stochastic simulations by Hauert and Schuster (5) showed that successful strategies tend to cooperate after mutual cooperation and after mutual defection [i.e., $p_{C,n-1}=p_{D,0}=1$ and $p_{S,j}=0$ for all other states $\left(S,j\right)$]. We refer to such a behavior as WSLS, because for pairwise dilemmas it corresponds to the Win-Stay, Lose-Shift strategy described by ref. 36. Because of condition [10], WSLS is a Nash equilibrium if and only if the social dilemma satisfies $\left(b_{n-1}+b_0\right)/2\le a_{n-1}$. For the example of a public goods game, this condition simplifies to $r\ge 2n/\left(n+1\right)$, which is always fulfilled for $r\ge 2$. For social dilemmas that meet this condition, WSLS provides a stable route to cooperation that is robust to errors.

Zero-Determinant Alliances.

In agreement with most of the theoretical literature on repeated social dilemmas, our previous analysis is based on the assumption that individuals act independently. As a result, we observed that a player’s strategic options typically diminish with group size. As a countermeasure, subjects may try to gain strategic power by coordinating their strategies with others. In the following, we thus extend our theory of ZD strategies for single individuals to subgroups of players. We refer to these subgroups as ZD alliances. Because the strategic power of ZD alliances is likely to depend on the exact mode of coordination between the allies, we consider two different models: when subjects form a strategy alliance, they only agree on the set of alliance members and on a common ZD strategy that each ally independently applies. During the actual game, there is no further communication between the allies. Strategy alliances can thus be seen as a boundary case of coordinated play, which requires a minimum amount of coordination. Alternatively, we also analyze synchronized alliances, in which all allies synchronize their actions in each round (i.e., the allies cooperate collectively, or they defect collectively). In effect, such a synchronized alliance thus behaves like a new entity that has a higher leverage than each player individually. Synchronized alliances thus may be considered as a boundary case of coordinated play that requires substantial coordination.

To model strategy alliances, let us consider a group of $n^{\mathcal{A}}$ allies, with $1\le n^{\mathcal{A}}<n$. We assume that all allies make a binding agreement that they will play according to the same ZD strategy p during the repeated game. Because the ZD strategy needs to allow allies to differentiate between the actions of the other allies and the outsiders, we need to consider a more general state space than before. The state space now takes the form $\left(S,j^{\mathcal{A}},j^{-\mathcal{A}}\right)$. The first entry S corresponds to the focal player’s own play in the previous round, $j^{\mathcal{A}}$ gives the number of cooperators among the other allies, and $j^{-\mathcal{A}}$ is the number of cooperators among the outsiders. A memory-one strategy p again needs to specify a cooperation probability $p_{S,j^{\mathcal{A}},j^{-\mathcal{A}}}$ for each of the possible states. Using this state space, we can define ZD strategies for a player i in a strategy alliance as

$$\mathbf{p}={\mathbf{p}}^{Rep}+\varphi \left[\left(1-s\right)\left(l\mathbf{1}-{\mathbf{g}}^i\right)+{\mathbf{g}}^i-\left(n^{\mathcal{A}}-1\right)w^{\mathcal{A}}{\mathbf{g}}^{\mathcal{A}}-\left(n-n^{\mathcal{A}}\right)w^{-\mathcal{A}}{\mathbf{g}}^{-\mathcal{A}}\right].$$ [11]

The vector ${\mathbf{g}}^{\mathcal{A}}$ contains the average payoff of the other allies for each possible state, and ${\mathbf{g}}^{-\mathcal{A}}$ is the corresponding vector for the outsiders. The weights $w^{-\mathcal{A}}\ge 0$ and $w^{\mathcal{A}}\ge 0$ are additional parameters that determine the relative importance of outsiders and other allies, being subject to the constraint $\left(n^{\mathcal{A}}-1\right)w^{\mathcal{A}}+\left(n-n^{\mathcal{A}}\right)w^{-\mathcal{A}}=1$. In the special case of a single player forming an alliance, $n^{\mathcal{A}}=1$, this guarantees that the two definitions of ZD strategies 4 and 11 are equivalent.

Similarly to the case of single individuals, we can apply Akin’s lemma to show that strategy alliances enforce a linear relationship between their own mean payoff ${\pi }^{\mathcal{A}}$ and the mean payoff of the outsiders ${\pi }^{-\mathcal{A}}$ (for details, see SI Text)

$${\pi }^{-\mathcal{A}}=s^{\mathcal{A}}{\pi }^{\mathcal{A}}+\left(1-s^{\mathcal{A}}\right)l,$$ [12]

where the slope of the alliance is given by $s^{\mathcal{A}}=\left[s-\left(n^{\mathcal{A}}-1\right)w^{\mathcal{A}}\right]/\left[1-\left(n^{\mathcal{A}}-1\right)w^{\mathcal{A}}\right].$ A strategy alliance can enforce exactly those payoff relationships 12 for which either $s^{\mathcal{A}}=1$ or for which l and $s^{\mathcal{A}}<1$ satisfy the conditions

$$\underset{0\le j\le n-n^{\mathcal{A}}}{\text{max}}\left\{b_j-\frac{j}{n-n^{\mathcal{A}}}\frac{b_j-a_{j-1}}{1-s^{\mathcal{A}}}\right\}\le l\le \underset{n^{\mathcal{A}}-1\le j\le n-1}{\text{min}}\left\{a_j+\frac{n-j-1}{n-n^{\mathcal{A}}}\frac{b_{j+1}-a_j}{1-s^{\mathcal{A}}}\right\}.$$ [13]

Interestingly, to reach this strategic power, an alliance needs to put a higher weight on the within-alliance payoffs (i.e., $w^{\mathcal{A}}$ needs to exceed $w^{-\mathcal{A}}$; SI Text), such that the allies are stronger affected by what the other allies do, as opposed to the actions of the outsiders. For single player alliances, $n^{\mathcal{A}}=1$, condition 13 again simplifies to the previous condition 6. However, as the alliance size $n^{\mathcal{A}}$ increases, condition 13 becomes easier to satisfy. Larger alliances can therefore enforce more extreme payoff relationships. For the example of a public goods game, we noted that single players cannot be arbitrarily extortionate when $n>r/\left(r-1\right)$. Alliances, on the other hand, only need to be sufficiently large, $n^{\mathcal{A}}/n\ge \left(r-1\right)/r$. Once an alliance has this critical mass, there are no bounds to extortion.

In a similar way, we can also analyze the strategic possibilities of a synchronized alliance. Because synchronized alliances act as a single entity, they transform the symmetric social dilemma between n independent players to an asymmetric game between $n-n_{\mathcal{A}}+1$ independent players. From the perspective of the alliance, the state space now takes the form $\left(S,j\right)$, where $S\in \left\{C,D\right\}$ is the common action of all allies and where $0\le j\le n-n_{\mathcal{A}}$ is the number of cooperators among the outsiders. ZD strategies for the synchronized alliance can be defined analogously to ZD strategies for single players

$$\mathbf{p}={\mathbf{p}}^{Rep}+\varphi \left[\left(1-s^{\mathcal{A}}\right)\left(l\mathbf{1}-{\mathbf{g}}^{\mathcal{A}}\right)+{\mathbf{g}}^{\mathcal{A}}-{\mathbf{g}}^{-\mathcal{A}}\right],$$ [14]

with ${\mathbf{g}}^{\mathcal{A}}$ being the payoff vector for the allies and ${\mathbf{g}}^{-\mathcal{A}}$ being the payoff vector of the outsiders. For a single player alliance, $n^{\mathcal{A}}=1$, this again reproduces the definition of ZD strategies in 4. By applying Akin’s lemma to Eq. 14, we conclude that synchronized alliances enforce ${\pi }^{-\mathcal{A}}=s^{\mathcal{A}}{\pi }^{\mathcal{A}}+\left(1-s^{\mathcal{A}}\right)l$, which is the same as relationship 12 for strategy alliances. Surprisingly, we even find that for reasonable alliance sizes, $n^{\mathcal{A}}\le n/2$, strategy alliances and synchronized alliances have the same set of enforceable parameters l and $s^{\mathcal{A}}$, as given by Eq. 13 (see SI Text for details). Thus, for the two models of ZD alliances considered here, the exact mode of coordination is irrelevant for the alliance’s strategic power unless the alliance has reached a substantial size.

Table 1 gives an overview of our findings on ZD strategies and ZD alliances in multiplayer social dilemmas. It shows that, although generally, ZD strategies exist for all group sizes, the power of single players to enforce particular outcomes typically diminishes or disappears in large groups. Forming ZD alliances then allows players to increase their strategic scope. The impact of a given ZD alliance, however, depends on the specific social dilemma: although ZD alliances can become arbitrarily powerful in public goods games, their strategic options remain limited in the volunteer’s dilemma.

Table 1.

Strategic power of different ZD strategies for three different social dilemmas

Strategy class	Typical property	Prisoner’s dilemma	Public goods game	Volunteer’s dilemma
Fair strategies	${\pi }^{-\mathcal{A}}={\pi }^{\mathcal{A}}$	Always exist	Always exist	Always exist
Extortionate strategies	${\pi }^{-\mathcal{A}}\le {\pi }^{\mathcal{A}}$	Always exist	In large groups, single players cannot be arbitrarily extortionate, but sufficiently large ZD alliances can be arbitrarily extortionate	Even large ZD alliances cannot be arbitrarily extortionate
Generous strategies	${\pi }^{-\mathcal{A}}\ge {\pi }^{\mathcal{A}}$	Always exist	In large groups, single players cannot be arbitrarily generous, but sufficiently large ZD alliances can be arbitrarily generous	Do not ensure that own payoff is below average
Equalizers	${\pi }^{-\mathcal{A}}=l$	Always exist	May not be feasible for single players, but is always feasible for sufficiently large ZD alliances	Only feasible if size of ZD alliance is $n^{\mathcal{A}}=n-1$, can only enforce $l=b-c$

Analogously to the case of individual players, ZD alliances are fair when they set $s^{\mathcal{A}}=1$; they are extortionate when $l=b_0$ and $0<s^{\mathcal{A}}<1$; they are generous for $l=a_{n-1}$ and $0<s^{\mathcal{A}}<1$; and they are equalizers when $s^{\mathcal{A}}=0$. For each of the three considered social dilemmas, we explore whether a given ZD strategy is feasible by examining the respective conditions in Eq. 13. In the repeated prisoner’s dilemma, single players can exert all strategic behaviors (23, 28, 29). Other social dilemmas either require players to form alliances to gain sufficient control (as in the public goods game), or they only allow for limited forms of control (as in the volunteer’s dilemma). These results hold both for strategy alliances and for synchronized alliances.

Discussion

When Press and Dyson (23) discovered the new class of ZD strategies for the repeated prisoner’s dilemma, this came as a big surprise (24, 25): after more than five decades of research, it seemed unlikely that any major property of the prisoner’s dilemma has been overlooked. For repeated multiplayer dilemmas the situation is different. Although various Folk theorems guarantee that cooperation is also feasible in large groups (42, 43), there has been considerably less theoretical research on the evolution of cooperation in repeated multiplayer dilemmas (4, 5, 39, 40). This lack of research may be due to the higher complexity: the mathematics of repeated n-player dilemmas seems to be more intricate, and numerical investigations are impeded because the time to compute payoffs increases exponentially in the number of players (5). Nevertheless, we showed here that many of the results for the repeated prisoner’s dilemma can be directly transferred to general social dilemmas, with an arbitrary number of involved subjects. The foundation for this progress is a new framework, provided by Akin’s lemma and the theory of Press and Dyson.

Using this framework, we extended the theory of repeated multiplayer dilemmas into three directions. The first and most immediate direction is our finding that ZD strategies exist in all social dilemmas. These strategies allow players to unilaterally dictate linear payoff relations, irrespective of the specific social dilemma being played, irrespective of the group size, and irrespective of the counter measures taken by the other group members. In particular, we showed that any social dilemma allows players to be fair, extortionate, or generous. Each of these strategy classes has its own particular strengths: extortionate strategies give a player a relative advantage compared with the other group members; fair strategies help to avoid further inequality within a group; and generous strategies allow players to revert to mutual cooperation when a coplayer defected by accident. At the same time, ZD strategies are remarkably simple. For example, to be fair in a public goods game, players only need to apply a rule called proportional Tit-for-Tat: if j of the $n-1$ other group members cooperated in the previous round, then cooperate with probability $j/\left(n-1\right)$ in the following round. Extortionate and generous strategies can be obtained in a similar way, by slightly modifying pTFT to the own advantage or to the advantage of the others.

As the second direction, we explored which ZD strategies and which pure memory-one strategies can be used to sustain cooperation in multiplayer dilemmas. Among ZD strategies, such strategies need to be generous (such that players never try to outperform their peers) (27, 28), but at the same time they must not be too generous. The right degree of generosity depends on the size of the group but not on the specific social dilemma being played. As a rule of thumb, we obtain that in larger groups, subjects are required to show less generosity.

As the last direction, we extended the concept of zero-determinant strategies from single players to subgroups of players, to which we refer to as ZD alliances. Depending on the degree of coordination, we explored two forms of ZD alliances: members of a strategy alliance only agree on using a common ZD strategy during the game, but they do not coordinate each of their decisions; members of a synchronized alliance, on the other hand, act as a single entity—they either all cooperate or they all defect in a given round. The effect of such ZD alliances depends on the size of the alliance, the applied strategy, and the properties of the underlying social dilemma. In general, we find that by coordinating their play with others, subjects can increase their strategic options considerably. The exact mode of coordination, however, only turns out to play a minor role: As long as the size of the ZD alliance is below half the group size, strategy alliances and synchronized alliances have the same strategic power. In addition to their static properties, ZD strategies for the prisoner’s dilemma also have a remarkable dynamic component (23, 44): when a player commits himself to an extortionate ZD strategy, then adapting coplayers learn to cooperate over time. Numerical simulations in the SI show an analogous result for multiplayer dilemmas: when ZD alliances apply strategies with a positive slope, they can trigger a positive group dynamics among the outsiders. The magnitude of this dynamic effect again depends on the size of the alliance, and on the applied strategy of the allies.

Here, we focused on ZD strategies; but the toolbox that we apply (in particular Akin’s lemma) is more general. As an example, we identified all pure memory-one strategies that allow for stable cooperation, including the champion of the repeated prisoner’s dilemma, Win-Stay Lose-Shift (36, 45). We expect that there will be further applications of Akin’s lemma to come. Such applications may include, for instance, a characterization of all Nash equilibria among the stochastic memory-one strategies or an analysis of how alliances are formed and whether evolutionary forces favor particular alliances over others (46, 47).

Overall, our results reveal how single players in multiplayer games can increase their control by choosing the right strategies and how they can increase their strategic options by joining forces with others.