Participation costs dismiss the advantage of heterogeneous networks in evolution of cooperation

Abstract

Real social interactions occur on networks in which each individual is connected to some, but not all, of others. In social dilemma games with a fixed population size, heterogeneity in the number of contacts per player is known to promote evolution of cooperation. Under a common assumption of positively biased pay-off structure, well-connected players earn much by playing frequently, and cooperation once adopted by well-connected players is unbeatable and spreads to others. However, maintaining a social contact can be costly, which would prevent local pay-offs from being positively biased. In replicator-type evolutionary dynamics, it is shown that even a relatively small participation cost extinguishes the merit of heterogeneous networks in terms of cooperation. In this situation, more connected players earn less so that they are no longer spreaders of cooperation. Instead, those with fewer contacts win and guide the evolution. The participation cost, or the baseline pay-off, is irrelevant in homogeneous populations, but is essential for evolutionary games on heterogeneous networks.

1. Introduction

A variety of creatures from many small organisms to social animals including humans show altruistic behaviour. Altruism occurs even when being selfish is better for an individual, which constitutes the social dilemma (Dawes 1980; Sugden 1986). Emergence of altruism under social dilemmas can be explained by various mechanisms, such as kin selection, direct reciprocity, indirect reciprocity and group selection (Nowak 2006). In the Prisoner's Dilemma, altruism is also promoted by the viscosity of populations (Axelrod 1984; Nowak & May 1992). If players are aligned on a spatially structured graph such as the square lattice, cooperators form close-knit clusters of conspecifics to survive the invasion of selfish defectors. Maintenance of such clusters is impossible in well-mixed populations modelled by the random graph and the all-to-all connected network. Spatial structure affects cooperation in other games as well (e.g. Szabó & Hauert 2002). Particularly, spatial structure can be detrimental to cooperation in the snowdrift game (Hauert 2002; Hauert & Doebeli 2004).

To be more realistic, players often inhabit networks of social contacts more complex than the square lattice, the random graph, and the all-to-all connected network (Newman 2003). First, real social networks are small world, implying the combination of abundant localized interactions, as in the square lattice, and sufficient short cuts, as in the random graph. Second, players are heterogeneous in terms of the number of contacts with others. An extreme case of this is the scale-free network in which the number of neighbours is distributed according to the power law (Barabási & Albert 1999). In contrast, the number of neighbours of conventional networks is the same for everybody (regular lattices and the all-to-all connected network) or distributed with a narrow tail (so-called Erdös–Rényi random graph). Even though not all social networks are scale-free, the number of neighbours is naturally heterogeneous.

Recently, it was shown that heterogeneous networks promote evolution of cooperation in the Prisoner's Dilemma, the snowdrift and stag hunt games. Particularly, scale-free networks are strong amplifiers of altruism (Durán & Mulet 2005; Santos & Pacheco 2005, 2006; Santos et al. 2006a,b).

To explain the mechanism of enhanced cooperation, let us introduce the notion of the temperature of players. In evolutionary graph theory, hot players are those replaced often by others (Lieberman et al. 2005). By modifying this definition slightly, we denote by hot players those who play often, namely players with many neighbours. Cold players are those with a small number of neighbours, such as leaves in a network. Hot players are allowed in more rounds of the game than cold players per generation. If the typical pay-off obtained by playing a game is positively biased, which is a common assumption adopted in the previous studies cited above and others (Abramson & Kuperman 2001; Ebel & Bornholdt 2002; Ifti et al. 2004; Eguíluz et al. 2005; Zimmermann & Eguíluz 2005; Santos et al. 2006c; but see Pacheco et al. 2006; Tomassini et al. 2006 for other set-ups), it is largely worth participation. Then, hot players earn more than cold players because everybody earns positive ‘base’ pay-offs proportional to the number of neighbours. As a result, hot players are more successful in disseminating their strategies. Particularly, cooperation once employed by a hub is stable due to cooperative reactions in its neighbourhood. In addition, if hubs tend to be connected to each other, as in the model proposed by Barabási & Albert (1999, BA model), cooperators on hubs form loose clusters to take advantage of a variant of spatial reciprocity (Santos & Pacheco 2005; Santos et al. 2006a,b).

However, hot players are successful not owing to playing well but owing to the connectivity. In the present paper, we critically re-examine the effect of heterogeneous networks on emergence of cooperation. In real lives, participation in the game may be costly. A link to a neighbour implies building and maintaining communication, and this cost has actually been modelled for studying network formation (Jackson & Wolinsky 1996; Bala & Goyal 2000; Goyal & Vega-Redondo 2005). Expensive entry fees would dismiss the premium of hot players to change the entire scenario.

We study two-person games on networks with participation costs. For simplicity, networks are assumed to be fixed in size and topology. We show that there are three regimes depending on how costly participation is. First, when participation is inexpensive, cooperation is enhanced on heterogeneous networks as discovered previously. Second, when the participation cost is intermediate, the effect of the local pay-off structure, namely the configuration of the two-person game, and that of the network are comparable. Then, altruism does not develop. Third, when participation is very costly, initial strategies of cold players have long transients and propagate to hot players. With small and large participation costs, the network rather than the local pay-off structure determines evolutionary dynamics. In the intermediate regime, evolution is most sensitive to the local pay-offs.

2. Model

We compare effects of two types of networks on the evolution of cooperation by means of Monte Carlo simulations. A diluted well-mixed population is modelled by the regular random graph in which each player has eight neighbours who are chosen randomly from the population. The heterogeneous networks are modelled by scale-free networks generated by the BA model, in which the number of neighbours denoted by k follows the power law p(k)∝k⁻³ (Barabási & Albert 1999). The average number of neighbours in the scale-free networks is set equal to 8 for fair comparison with the regular random graph. Both types of networks consist of n=5000 players.

To probe the network effect, we consider only two simple strategies without memory, namely unconditional cooperation and unconditional defection. The initial fraction of cooperators is set equal to 0.5. In one generation, everybody participates in the two-person game with all the neighbours. The pay-off matrix will be specified in §3.

Each player tends to copy successful strategies in their neighbourhood. We apply the update rule compatible with the replicator dynamics, following the previous literature (Santos & Pacheco 2005; Santos et al. 2006a,b). Suppose that player x with k_x neighbours has obtained generation pay-off P_x. To update the strategy, x selects a player y among the k_x neighbours with equal probability (=1/k_x). Then, x copies y's strategy with probability (P_y−P_x)/(max(k_x, k_y) × (uppermost pay-off in one game−lowermost pay-off in one game)), if P_y>P_x. The denominator is the normalization constant so that the replacement probability ranges between 0 and 1. If P_y≤P_x, the strategy of x is unchanged. All the players experience updating according to this rule synchronously. This completes one generation.

Each evolutionary simulation consists of 5000 generations. The final fraction of cooperators denoted by c_f is calculated as the average fraction of cooperators based on the last 200 generations of five runs with different initializations for each network and five different realizations of the network. When the participation cost is not very large, c_f corresponds to values close to stochastic stationary values. Otherwise, c_f represents transient values.

3. Results

(a) Prisoner's Dilemma

We start with two well-known pay-off matrices of the simplified Prisoner's Dilemma. The first one is given by

$$\begin{array}{ccc}& C& D\\ C& 1& 0\\ D& T& 0\end{array}.$$ (3.1)

The entries of equation (3.1) indicate the pay-off that the row player gains when playing against the column player. The first (second) row and column correspond to cooperation (defection). The Prisoner's Dilemma arises when T>1, and larger T results in more defectors. With participation cost h, the pay-off matrix becomes

$$\left(\begin{array}{cc}1-h& -h\\ T-h& -h\end{array}\right).$$ (3.2)

Note that introducing h does not trespass the notion of the Prisoner's Dilemma as far as T>1.

Actually, the game defined by equation (3.1) or (3.2) lies on the boundary between the snowdrift game analysed in §3b and the Prisoner's Dilemma. Therefore, we also examine another standard pay-off matrix of the generic Prisoner's Dilemma given by

$$\begin{array}{ccc}& C& D\\ C& b-c& -c\\ D& b& 0\end{array},$$ (3.3)

where b>c. With the participation cost, equation (3.3) is transformed to

$$\left(\begin{array}{cc}1-r-h& -r-h\\ 1-h& -h\end{array}\right),$$ (3.4)

where r=c/b is the cost-to-benefit ratio. In the following, we refer to numerical results for the pay-off matrix given by equation (3.2). The results that are qualitatively the same as the following are obtained for equation (3.4) (as shown in the electronic supplementary material figure 1b–d).

The fraction of cooperators c_f is not affected by h on the regular random graph (electronic supplementary material figure 1a). Since each player has the same number of neighbours, participation cost does not differentiate the players. In contrast, h drastically affects c_f for the scale-free networks, as shown in figure 1a. In figure 1b, c_f for the scale-free networks relative to c_f for the regular random graph is plotted. We identify three qualitatively different regimes in terms of h, which roughly correspond to (I) h<0.24, (II) 0.24≤h<2 and (III) h≥2. The transition between (II) and (III) is fairly gradual.

Figure 1.

The Prisoner's Dilemma on the scale-free networks with participation costs. The pay-off matrix is represented by equation (3.2). (a) The final fraction of cooperators c_f. (b) c_f for the scale-free networks shown in (a) −c_f for the regular random graph shown in the electronic supplementary material figure 1a.

(i) Regime (I): costless participation

When h<0.24, participation is inexpensive, and hot players such as hubs are strong competitors regardless of the strategies of their cold neighbours. The pay-off of a player increases linearly with the number of contacts, as shown by thin lines with positive slopes in figure 2a. In particular, cooperation spreads from hot cooperators to their cold neighbours, the local density of cooperators increases, and hot cooperators gain more by mutual cooperation. Cooperation triggered by hot players is self-promotive. Defective hot players may also win for a moment. However, defection then prevails in their neighbourhood so that hot defectors can no longer exploit the neighbours owing to mutual defection. This results in a null generation pay-off of hot defectors so that they can be outperformed by their cold neighbours. A hot player sticks to cooperation but not to defection.

Figure 2.

The effect of the number of neighbours in the Prisoner's Dilemma on the scale-free networks. (a) The average generation pay-off and (b) the average number of flips are plotted in terms of the number of neighbours that a player has. Note that the log scale is used for the abscissa of (b) for clarity. The lines correspond to h=0 (thinnest line), 0.2, 0.23, 0.24, 0.25, 0.3 and 0.5 (thickest line). The pay-off matrix is given by equation (3.2) with T=1.5.

In sum, heterogeneous networks enhance altruism, which recovers the previous work corresponding to h=0 (Santos & Pacheco 2005, 2006; Santos et al. 2006a,b). Note that this regime extends to h<0, i.e. when gifts are given for participation so that everyone wins a positive pay-off.

To illuminate on different dynamics of hot and cold players, we measure how often players flip the strategy. The average number of flips throughout the evolutionary run (including the contribution from both transients and stationary states) is shown in figure 2b. Colder players experience more flips when h<0.24 (thin lines). They myopically follow what hotter players do in both transients and stationary states. Cooperation on hubs is stabilized in an early stage, yielding less flips of hotter players.

(ii) Regime (II): moderately expensive participation

Interestingly, cold players spread their strategies to hot players when h≥0.24 (regimes (II) and (III)), which is opposite to what happens in regime (I). As a result, enhanced cooperation diminishes, even with a relatively small participation cost.

Regime (II) is defined by small to intermediate h(0.24≤h<2). Now, the local pay-off structure as well as the network topology is relevant. When h=0.3, scale-free networks surpass the regular random graph in terms of the number of cooperators only for 1≤T≤1.4. When h=0.6, this range shrinks to 1≤T≤1.1. The privilege of scale-free networks is entirely lost, when h=1. In regime (II), the pay-off of a player linearly decreases with the number of contacts (thick lines with negative slopes in figure 2a). Consistent with this, hot players flip strategies more often than cold players (thick lines in figure 2b). Hubs no longer conduct the dynamics.

(iii) Regime (III): costly participation

When h is roughly greater than 2, participation is really costly. Then, cold players with any strategy surpass hot players and govern the dynamics. This is not because cold players are tactical but because they play less often and lose less than hot players do. Strategies of cold players are not often replaced owing to the small number of neighbours. Consequently, cold players persist in their initial strategies, actually up to more than 300 000 generations. In the meantime, strategies of cold players tend to spread to hot neighbours. As a result, c_f, which is measured at 5000 generations, is almost the same as the initial fraction of cooperators (=0.5) regardless of T (figure 1a). Figure 1b indicates that c_f is larger for the scale-free networks than for the random graph, but this is due to long transients. After a very long period, the fraction of cooperators will be smaller than c_f shown in figure 1a to eradicate the spurious advantage of scale-free networks.

(b) Snowdrift game

The snowdrift game, which is also known as the chicken game or the hawk–dove game (Sugden 1986; Hauert & Doebeli 2004), is illustrated as a situation of two drivers caught in a snowdrift. For the two cars to get out, which is equivalent to pay-off β (more than 1) for each driver, the snow must be shovelled away. A total effort of unity must be invested to this task. Two players may cooperate to share the cost so that each pays half, or one player may cover the full cost. Otherwise, both players miss β. Cooperation benefits not only the opponent but also the focal player itself. Different from the Prisoner's Dilemma in which defection is beneficial, in the snowdrift game, cooperation can deserve even when the opponent defects.

If the participation is free, the pay-off matrix of the snowdrift game is given by

$$\begin{array}{ccc}& C& D\\ C& \beta -1/2& \beta -1\\ D& \beta & 0\end{array}.$$ (3.5)

In this case, heterogeneous networks reinforce evolution of cooperation as in the Prisoner's Dilemma (Santos & Pacheco 2005; Santos et al. 2006a,b). Without dismissing the structure of the snowdrift game, the participation cost translates the pay-off matrix to the following form:

$$\left(\begin{array}{cc}\beta -1/2-h& \beta -1-h\\ \beta -h& -h\end{array}\right).$$ (3.6)

As shown in the electronic supplementary material figure 2, the participation cost h does not influence c_f on the regular random graph. The fraction of cooperators converges to a value close to the theoretical estimate c_f=1−r, where r=1/(2β−1) is the cost-to-benefit ratio (Sugden 1986; Hofbauer & Sigmund 1998; Hauert & Doebeli 2004). If cooperation is relatively costly with a small β (large r), cooperators decrease in number.

On heterogeneous networks, how the fraction of cooperators varies as a function of r depends on the participation cost. We again find three regimes as shown in figure 3. The scale-free networks host more cooperators than the regular random graph only when h is near zero or negative (regime (I)). The advantage of the scale-free networks is neutralized by intermediate h (roughly speaking, h≅1), which defines regime (II). Note that the reduction of cooperation is not as much as that for the Prisoner's Dilemma. With large h (roughly speaking, h≥2), c_f is rather insensitive to the local pay-off structure due to long transients (regime (III)).

Figure 3.

The snowdrift game on the scale-free networks with participation costs. (a) c_f and (b) c_f for the scale-free networks shown in (a) −c_f for the regular random graph shown in the electronic supplementary material figure 2.

(c) General two-person games

With the participation cost incorporated, general symmetrical two-person games with two strategies are represented by

$$\begin{array}{ccc}& C& D\\ C& R-h& S-h\\ D& T-h& P-h\end{array}.$$ (3.7)

In accordance with the previous sections, we denote by cooperation (defection) the strategy corresponding to the first (second) row and column. As T increases, players are tempted to defect, and c_f decreases. As S decreases, players would refuse cooperation to avoid exploitation by defectors. Therefore, c_f decreases. The Prisoner's Dilemma, the snowdrift and stag hunt games are defined by T>R>P>S, T>R>S>P and R>T>P>S, respectively. The Prisoner's Dilemma usually accompanies another condition 2R>T+S so that mutual cooperation is more beneficial than alternating unilateral cooperation.

Multiplying each element of equation (3.7) by a common constant modifies just the time-scale of evolution. Accordingly, there are three free parameters in the pay-off matrix, which are chosen to be T, S and h, while we set R=1 and P=0.

In figure 4a, c_f for h=0 is plotted in the T–S parameter space for the regular random graph. As expected, the number of cooperators decreases with T and increases with S. The results are independent of h. For the scale-free networks, we plot c_f in figure 4b–e for four values of h (also see electronic supplementary material figure 3 for the direct comparison of c_f for the scale-free networks and c_f for the regular random graph). We confirm the existence of the three regimes, extending the results shown in figures 1 and 3. First, as shown in figure 4b, scale-free networks promote evolution of cooperation when the participation is costless (Santos et al. 2006a). Cooperation is strengthened in the Prisoner's Dilemma (T>1, S<0), the snowdrift game (T>1, S>0) and also the stag hunt game (T<1, S<0). Second, the advantage of the heterogeneity is lost for a wide range of T and S, when h=0.5 (figure 4c) and h=1 (figure 4d). Third, the transient is very long when participation is costly (h=2). This allows defectors to survive for a long time even without dilemma (S>0, T<1) and considerable cooperators to survive under the Prisoner's Dilemma (figure 4e).

Figure 4.

c_f in the T–S space for (a) the regular random graph and the scale-free networks with (b) h=0, (c) h=0.5, (d) h=1 and (e) h=2.

4. Discussion

We have discovered that the participation cost, which is irrelevant in well-mixed populations and on homogeneous networks including the regular lattices, casts a dramatic effect on evolutionary dynamics on heterogeneous networks. When participation is nearly free (regime (I)), heterogeneous networks promote cooperation (Durán & Mulet 2005; Santos & Pacheco 2005, 2006; Santos et al. 2006a,b). This is because the cooperation on hot players is robust against invasion of whatever strategies of cold players. Even if a cold player is good at exploiting cooperators in the neighbourhood, the aggregated pay-off would be much smaller than that of a hot player that would earn a lot just by playing the game many times. Hot players are leaders, and cold players are myopic followers. However, this phenomenon is not robust against variation in participation costs, which is consistent with the loss of cooperation under positive affine transformation of the pay-off matrix (Tomassini et al. 2006). When the participation cost is intermediate (regime (II)), cooperators do not really increase and even decrease on heterogeneous networks. When participation is costly (regime (III)), hot players myopically follow cold players. In regimes (I) and (III), not local pay-off structure but network structure governs evolution. The local pay-offs are relevant only in regime (II), for which cooperation is not enhanced by heterogeneous networks. Regimes (II) and (III), which have been largely unexplored, may be relevant in, for example, environmental problems, political conflicts and human relationships. In these situations, players are often forced to play and incur participation costs. In other words, the best one could get may be the least disastrous, but not really wonderful, solution.

For replicator-type and many other update rules in well-mixed populations, evolution is invariant under uniform addition of a constant to the pay-off matrix (e.g. participation cost). Then, a game in regimes (II) and (III) can be translated into a game in regime (I). However, this operation is disallowed for heterogeneous networks. Since multiplying the pay-off matrix by a positive constant alters just the evolutionary time-scale in either case, there are two free parameters for two-person games in homogeneous populations, whereas there are three of them in heterogeneous populations (e.g. S, T and h as used in figure 4).

The present results can be generalized in some aspects. First, two-person games can be asymmetrical. Second, the update rule does not have to be of a replicator type (Hofbauer & Sigmund 1998; Ohtsuki et al. 2006) as far as the reproduction rate is not extremely nonlinear in the generation pay-off. Third, large noise would blur but would not break down the three regimes. For example, irrational actions may cause hot cooperators, which are stable with small participation costs and small noise, to defect and elicit upsurges of defectors nearby. However, noise does not overturn the fact that hot (cold) players are better off for small (large) participation costs. Fourth, network models can be arbitrary. For example, the scale-free networks based on the configuration model (without growth and preferential attachment) and the Erdös–Rényi random graph promote altruism in regime (I), albeit to a lesser extent than the BA model (Santos & Pacheco 2005; Santos et al. 2006a,b). These heterogeneous networks as well as the BA model do not enhance altruism in regimes (II) and (III) (data not shown).

The present results also have limitations. First, we have only assumed memoryless strategies, namely unconditional cooperators and unconditional defectors. Second, the networks have been static. In coevolutionary dynamics in which players form and sever links as well as play games, only regime (I) has been considered, a main conclusion being enhanced cooperation (Skyrms & Pemantle 2000; Eguíluz et al. 2005; Zimmermann & Eguíluz 2005; Santos et al. 2006c). In regime (III), for example, everybody must sever links to be loners (Goyal & Vega-Redondo 2005; for the role of loners, also refer to Hauert et al. 2002). Third, we have used the additive pay-off (Nowak et al. 1994; Abramson & Kuperman 2001; Ebel & Bornholdt 2002; Ifti et al. 2004; Santos & Pacheco 2005, 2006; Durán & Mulet 2005; Santos et al. 2006a,b). An alternative is to use the average pay-off, or division of the generation pay-off of each player by the number of neighbours (Kim et al. 2002; Santos & Pacheco 2006; Taylor & Nowak 2006). The average pay-off is not affected by the number of neighbours and hampers the enhanced altruism on heterogeneous networks (Santos & Pacheco 2006; Tomassini et al. 2006). We have adopted the additive pay-off scheme because its intuitive meaning is clearer than the average pay-off. These topics warrant future work.

Supplementary Material

Electronic supplementary figures

(i) Prisoner, (ii) snowdrift game and (iii) the advantage of the scale-free networks in the T–S space