Skip to main content
PLOS One logoLink to PLOS One
. 2010 Dec 6;5(12):e15117. doi: 10.1371/journal.pone.0015117

Heterogeneous Aspirations Promote Cooperation in the Prisoner's Dilemma Game

Matjaž Perc 1,*, Zhen Wang 2
Editor: James A R Marshall3
PMCID: PMC2997779  PMID: 21151898

Abstract

To be the fittest is central to proliferation in evolutionary games. Individuals thus adopt the strategies of better performing players in the hope of successful reproduction. In structured populations the array of those that are eligible to act as strategy sources is bounded to the immediate neighbors of each individual. But which one of these strategy sources should potentially be copied? Previous research dealt with this question either by selecting the fittest or by selecting one player uniformly at random. Here we introduce a parameter Inline graphic that interpolates between these two extreme options. Setting Inline graphic equal to zero returns the random selection of the opponent, while positive Inline graphic favor the fitter players. In addition, we divide the population into two groups. Players from group Inline graphic select their opponents as dictated by the parameter Inline graphic, while players from group Inline graphic do so randomly irrespective of Inline graphic. We denote the fraction of players contained in groups Inline graphic and Inline graphic by Inline graphic and Inline graphic, respectively. The two parameters Inline graphic and Inline graphic allow us to analyze in detail how aspirations in the context of the prisoner's dilemma game influence the evolution of cooperation. We find that for sufficiently positive values of Inline graphic there exist a robust intermediate Inline graphic for which cooperation thrives best. The robustness of this observation is tested against different levels of uncertainty in the strategy adoption process Inline graphic and for different interaction networks. We also provide complete phase diagrams depicting the dependence of the impact of Inline graphic and Inline graphic for different values of Inline graphic, and contrast the validity of our conclusions by means of an alternative model where individual aspiration levels are subject to evolution as well. Our study indicates that heterogeneity in aspirations may be key for the sustainability of cooperation in structured populations.

Introduction

Understanding the evolution of cooperation among selfish individuals in human and animal societies remains a grand challenge across disciplines. Evolutionary games are employed frequently as the theoretical framework of choice in order to interpret the emergence and survival of cooperative behavior [1][6]. The prisoner's dilemma game, in particular, has attracted considerable interest [7][11] as the essential yet minimalist example of a social dilemma. In the original two-person one-shot game the two players have two strategies to choose from (cooperation and defection), and their payoffs depend on the simultaneous decision of both. If they choose to cooperate they will receive the highest collective payoff, which will be shared equally among them. Mutual defection, on the other hand, yields the lowest collective payoff. Yet to defect is tempting because it yields a higher individual payoff regardless of the opponent's decision. It is thus frequently so that both players choose not to cooperate, thus procreating the inevitable social dilemma. In reality, however, interactions may be repeated and the reputation of players compromised [12], [13]. Additionally, individuals may alter with whom they interact [14], and different behaviors may be expressed when participants in a social interaction occupy different roles [15][18]. Such and similar considerations have been very successful in elucidating why the unadorned scenario of total defection is often at odds with reality [19], where it is clear that both humans and animals cooperate to achieve what would be impossible by means of isolated efforts. Mechanisms supporting cooperation identified thus far include kin selection [20] as well as many others [6], [21][23], and there is progress in place aimed at unifying some of these approaches [24], [25].

Probably the most vibrant of all in recent years have been advances building on the seminal paper by Nowak and May [26], who showed that spatial structure may sustain cooperation without the aid of additional mechanisms or strategic complexity. Although in part anticipated by Hamilton's comments on viscous populations [20], it was fascinating to discover that structured populations, including complex and social networks [27][29], provide an optimal playground for the pursuit of cooperation. Notably, a simple rule for the evolution of cooperation on graphs and social networks is that natural selection favors cooperation if Inline graphic, where Inline graphic is the benefit of the altruistic act, Inline graphic is its cost, while Inline graphic is the average number of neighbors [30]. This is similar to Hamilton's rule stating that Inline graphic should be larger than the coefficient of genetic relatedness between individuals [20]. In fact, on graphs and social networks the evolution of altruism can thus be fully explained by the inclusive fitness theory since the population is structured such that interactions are between genetic relatives on average [25], [31], [32].

According to the “best takes all” rule [33], [34] players are allowed to adopt the strategy of one of their neighbors, provided its payoff is higher than that from the other neighbors as well as from the player aspiring to improve by changing its strategy. Based on this relatively simple setup, it was shown that on a square lattice cooperators form compact clusters and so protect themselves against being exploited by defectors. The “best takes all” strategy adoption rule is, however, just one of the many possible alternatives that were considered in the past. Other examples include the birth-death and imitation rule [35], the proportional imitation rule [36], the reinforcement learning adoption rule [37], or the Fermi-function based strategy adoption rule [38]. The latter received substantial attention, particularly in the physics community, for its compatibility with the Monte Carlo simulation procedure and the straightforward adjustment of the level of uncertainty governing the strategy adoptions Inline graphic. However, with this rule the potential donor of the new strategy is selected uniformly at random from all the neighbors. This is somewhat untrue to what can be observed in reality, where in fact individuals typically aspire to their most successful neighbors rather than just somebody random. In this sense the “best takes all” rule seems more appropriate, although it fails to account for errors in judgment, uncertainty, external factors, and other disturbances that may vitally affect how we evaluate and see our co-players. Here we therefore propose a simple tunable function that interpolates between the “best takes all” and the random selection of a neighbor in a smooth fashion by means of a single parameter Inline graphic. In this sense the parameter Inline graphic acts as an aspiration parameter, determining to what degree neighbors with a higher payoff will be considered more likely as potential strategy sources than other (randomly selected) neighbors.

Aiming to further disentangle the role of aspirations, we also consider two types of players, denoted by type Inline graphic and Inline graphic, respectively. While players of type Inline graphic conform to the aspirations imposed by the value of the aspiration parameter Inline graphic, type Inline graphic players choose whom to potentially copy uniformly at random irrespective of Inline graphic. We denote the fraction of type Inline graphic and Inline graphic players by Inline graphic and Inline graphic, respectively. This additional division of players into two groups is motivated by the overwhelming evidence indicating that heterogeneity, almost irrespective of its origin, promotes cooperative actions. Most notably associated with this statement are complex networks, including small-world networks [39][41], random regular graphs [5], [42], scale-free networks [43][48], as well as adaptive and growing networks [49][56]. Furthermore, we follow the work by McNamara et al. on the coevolution of choosiness and cooperation [57], in particular by omitting the separation of the population on two types of players and introducing the heterogeneity by means of normally distributed individual aspiration levels that are then also subject to evolution.

At present, we thus investigate how aspirations on an individual level affect the evolution of cooperation. Having something to aspire to is crucial for progress and betterment. But how high should we set our goals? Should our role models be only overachievers and sports heroes, or is it perhaps better to aspire to achieving somewhat more modest goals? Here we address these questions in the context of the evolutionary prisoner's dilemma game and determine just how strong and how widespread aspirations should be for cooperation to thrive best. As we will show, a strong drive to excellence in the majority of the population may in fact act detrimental on the evolution of cooperation, while on the other hand, properly spread and heterogeneous aspirations may be just the key to fully eliminating the defectors. We will show that this holds irrespective of the structure of the underlying interaction network, as well as irrespective of the level of uncertainty by strategy adoptions Inline graphic. In addition, the presented results will be contrasted with the output of a simple coevolutionary model, where individual aspirations will also be subject to evolution by means of natural selection. We will conclude that appropriately tuned aspirations may be seen as a universally applicable promoter of cooperation, which will hopefully inspire new studies along this line of research.

Results

Depending on the interaction network, the strategy adoption rule and other simulation details (see e.g. [4], [6], [23], [58]), there always exists a critical cost-to-benefit ratio Inline graphic at which cooperators in the prisoner's dilemma die out. This is directly related to Hamilton's rule stating that natural selection favors cooperation if Inline graphic is larger than the coefficient of genetic relatedness between individuals [20]. If the aspiration parameter Inline graphic (note that then the division of players to those of type Inline graphic and those of type Inline graphic is irrelevant), Inline graphic, and the interaction network is a square lattice, then, in our case, Inline graphic. In what follows, we will typically set Inline graphic slightly below this threshold to Inline graphic and examine how different values of Inline graphic, Inline graphic, Inline graphic, as well as different interaction networks influence the outcome of the prisoner's dilemma game.

It is instructive to first examine characteristic snapshots of the square lattice for different values of Inline graphic and Inline graphic. Results presented in Fig. 1 hint to the conclusion that heterogenous aspiration to the fittest promotes cooperation, although the details of this claim depend somewhat on the value of the aspiration parameter Inline graphic. For small values of Inline graphic it is best if all the players, i.e. Inline graphic, aspire to their slightly (note that Inline graphic is small) fitter neighbors and thus none actually choose the potential strategy sources uniformly at random. This can be deduced from the top three panels of Fig. 1 if compared from left to right. For large Inline graphic, however, it is best if only half of the players, i.e. Inline graphic, aspire to their most fittest neighbors, while the other half chooses their role models randomly. This can be observed if one compares the bottom three panels of Fig. 1 with one another, although the difference in the overall density of cooperators (depicted green and blue) between the middle and the right panel is fairly small. Finally, the role of the aspiration parameter is more clear cut since larger Inline graphic clearly favor the cooperative strategy if compared to small Inline graphic. This can be observed if comparing the snapshots presented in Fig. 1 vertically.

Figure 1. Characteristic snapshots of strategy distributions on the square lattice.

Figure 1

Top row depicts results for the aspiration parameter Inline graphic while the bottom row features results for Inline graphic. In both rows the fraction of type Inline graphic players Inline graphic is Inline graphic, Inline graphic and Inline graphic from left to right. Cooperators of type Inline graphic and Inline graphic are colored green and blue, respectively. Defectors of type Inline graphic and Inline graphic, on the other hand, are colored red and yellow. If comparing the snapshots vertically, it can be observed that larger values of Inline graphic (top Inline graphic, bottom Inline graphic) clearly promote the evolution of cooperation. The scenario from left to right via increasing the fraction of type Inline graphic players is not so clear cut. For Inline graphic (top row) we can conclude that larger Inline graphic favor cooperative behavior, as clearly the cooperators flourish more and more from the left toward the right panel. For Inline graphic (bottom row), however, it seems that for Inline graphic (bottom middle) cooperators actually fare better then for both Inline graphic (bottom left) and Inline graphic (bottom right). Hence, the conclusion imposes that for higher Inline graphic values an intermediate (rather than the maximal, as is the case for lower Inline graphic) fraction of type Inline graphic players (those that aspire to their most fittest neighbors only) is optimal for the evolution of cooperation. Results in all panels were obtained for Inline graphic and Inline graphic.

Since the snapshots presented in Fig. 1 can be used primarily for an initial qualitative assessment of the impact of heterogeneous aspirations, we present in Fig. 2 the fraction of cooperators Inline graphic (left) and the critical cost-to-benefit ratio Inline graphic (right) in dependence on Inline graphic for different values of Inline graphic. It can be observed that the promotion of cooperation for the optimal combination of the two parameters, being Inline graphic and Inline graphic, is really remarkable. The fraction of cooperators rises from Inline graphic to Inline graphic, while the critical cost-to-benefit ratio rises a full order of magnitude from Inline graphic to Inline graphic. As tentatively deduced from the lower three snapshots in Fig. 1, it can also be observed that for high values of Inline graphic an intermediate fraction of type Inline graphic players is optimal for the evolution of cooperation. Conversely, for low Inline graphic the fraction of cooperators Inline graphic and the critical cost-to-benefit ratio Inline graphic both increase monotonously with increasing Inline graphic. If, however, selecting a particular value of Inline graphic, then the impact of the aspiration parameter Inline graphic is always such that cooperation is the more promoted the larger the value of Inline graphic. This can be observed clearly from both panels, and indeed seems like the main driving force behind the elevated levels of cooperation. Fine-tuning the fraction of players making use of the aspiration to the fittest (from Inline graphic downwards since the Inline graphic limit trivially returns the random selection of potential strategy sources) at high Inline graphic can rise the cooperation level further, but more in the sense of minor adjustments, similarly as was observed in the past for the impact of uncertainty by strategy adoptions [42] or the impact of noise [59].

Figure 2. Promotion of cooperation due to heterogenous aspirations on the square lattice.

Figure 2

Left panel depicts the density of cooperators Inline graphic in dependence on the fraction of type Inline graphic players Inline graphic for different values of the aspiration parameter Inline graphic. Right panel depicts the critical cost-to-benefit ratio Inline graphic at which cooperators die out, i.e. Inline graphic, in dependence on Inline graphic for different values of Inline graphic. Results in both panels convey the message that low values of Inline graphic require a high fraction of type Inline graphic players for cooperation to flourish. Conversely, higher values of Inline graphic sustain cooperation optimally if only half (Inline graphic) of the players aspires to their most fittest neighbors while the rest chooses whom to potentially imitate uniformly at random. Optimal conditions for the evolution of cooperation thus require Inline graphic and Inline graphic to be fine-tuned jointly. Depicted results in both panels were obtained for Inline graphic and Inline graphic.

Aiming to generalize the validity of our results, we present in Fig. 3 the fraction of cooperators Inline graphic in dependence on Inline graphic for different values of Inline graphic as obtained on the random regular graph (left) and the small-world network (right). The goal is to test to what extend above conclusions hold also on interaction networks other than the square lattice, in particular such that are more complex and spatially heterogenous. If comparing the obtained results with those presented in the left panel of Fig. 2, it seems save to conclude that they are to a very large degree qualitatively identical. Some differences nevertheless can be observed. The first is that what constitutes a high Inline graphic limit is a bit higher on complex networks than on the regular lattice. Note that for Inline graphic the optimal fraction of type Inline graphic players is practically still Inline graphic. Even for Inline graphic the bell-shaped dependence on Inline graphic is far less pronounced than on the square lattice, and the optimal Inline graphic (the peak of Inline graphic) is closer to Inline graphic than Inline graphic. The second difference is, looking relatively to the starting point at Inline graphic, that the promotion of cooperation due to positive Inline graphic and Inline graphic is somewhat less prolific. This is, however, not that surprising since complex networks in general promote cooperation already on their own [6], and thus secondary promotive mechanisms may therefore become less expressed. Aside from these fairly mild differences though, we can conclude that heterogenous aspirations do promote cooperation irrespective of the underlying interaction network, and that the details of the promotive effect are largely universal and predictable.

Figure 3. Promotion of cooperation due to heterogenous aspirations on the random regular graph (RRG) and the small-world (SW) network.

Figure 3

Left panel depicts the density of cooperators Inline graphic in dependence on the fraction of type Inline graphic players Inline graphic for different values of the aspiration parameter Inline graphic for the RRG. Right panel depicts Inline graphic in dependence on Inline graphic for different values of Inline graphic for the Watts-Strogatz SW network with the fraction of rewired links equalling Inline graphic. These results are in agreement with those presented in Fig. 2, supporting the conclusion that the impact of heterogenous aspirations on the evolution of cooperation is robust against alterations of the interaction network. As on the square lattice, low, but also intermediate, values of Inline graphic require Inline graphic for cooperation to thrive, while higher values of Inline graphic sustain cooperation optimally only if Inline graphic. Depicted results in both panels were obtained for Inline graphic and Inline graphic.

Next, we proceed with examining how positive values of Inline graphic and Inline graphic fare under different levels of uncertainty by strategy adoptions. The latter can be tuned via Inline graphic [see Eq. (3)], which acts as a temperature parameter in the employed Fermi strategy adoption function [38]. Accordingly, when Inline graphic all information is lost and the strategies are adopted by means of a coin toss. Note that this aspect has thus far not received any attention here as Inline graphic was fixed. The matter is not trivial to address because uncertainty and noise can have a rather profound impact on the evolution of cooperation [40], [42], [59][62], and thus care needs to be exercised. The safest and most accurate way to approach the problem is by means of phase diagrams. Since we have two additional parameters (Inline graphic and Inline graphic) against which we want to test the impact of Inline graphic, we determined full Inline graphic phase diagrams for six characteristic combinations of Inline graphic and Inline graphic on the square lattice. Obtained results are presented in Fig. 4. Notably, the phase diagram presented in the top left panel of Fig. 4 is well-known, implying the existence of an optimal level of uncertainty for the evolution of cooperation, as was previously reported in [42], [59]. In particular, note that the Inline graphic transition line is bell shaped, indicating that Inline graphic is the optimal temperature at which cooperators are able to survive at the highest value of Inline graphic. Importantly though, this phenomenon can only be observed on interaction topologies lacking overlapping triangles [63], [64]. Interestingly, increasing Inline graphic from Inline graphic (top row) to Inline graphic (bottom row) completely eradicates (as do interaction networks incorporating overlapping triangles) the existence of an optimal Inline graphic, and in fact qualitatively reverses the dependence. The Inline graphic transition line has an inverted bell-shaped outlay, indicating the existence of the worst rather than an optimal temperature Inline graphic for the evolution of cooperation. The qualitative changes are less profound if Inline graphic is kept constant at Inline graphic (top row) and Inline graphic increases (from left to right). Still, however, the bell-shaped outlay of the Inline graphic transition gives way to a monotonically increasing curve, saturating only for high Inline graphic. These qualitative changes in the phase diagrams imply that increasing the aspiration parameter Inline graphic or the fraction of players abiding to it (type Inline graphic) effectively alters the interaction network. While the square lattice obviously lacks overlapping triangles and thus enables the observation of an optimal Inline graphic for small enough values of Inline graphic and Inline graphic (or a combination thereof, as is the case in the top left panels), trimming the likelihood of who will act as a strategy source and how many players will actually aspire to their fittest neighbors seems to effectively enhance linkage among essentially disconnected triplets and thus precludes the same observation. It is instructive to note that a similar phenomenon was observed recently in public goods games, where the joint membership in large groups was also found to alter the effective interaction network and thus the impact of uncertainly on the evolution of cooperation [64].

Figure 4. Full Inline graphic phase diagrams for the square lattice.

Figure 4

Top row depicts results for the aspiration parameter Inline graphic while the bottom row features results for Inline graphic. In both rows the fraction of type Inline graphic players Inline graphic is Inline graphic, Inline graphic and Inline graphic from left to right. The outline of panels thus corresponds to the snapshots presented in Fig. 1. Thin blue and thick red lines mark the border between stationary pure Inline graphic and Inline graphic phases and the mixed Inline graphic phase, respectively. In agreement with previous works [42], [63], it can be observed that for Inline graphic and Inline graphic (top left) there exists an intermediate uncertainty in the strategy adoption process (an intermediate value of Inline graphic) for which the survivability of cooperators is optimal, i.e. Inline graphic is maximal. Conversely, while the borderline separating the pure Inline graphic and the mixed Inline graphic phase for all the other combinations of Inline graphic and Inline graphic exhibits a qualitatively identical outlay as for the Inline graphic and Inline graphic case, the Inline graphic transition is qualitatively different and very much dependent on the particularities players' aspirations. Note that in all the bottom panels there exist an intermediate value of Inline graphic for which Inline graphic is minimal rather than maximal, while towards the large Inline graphic limit Inline graphic increases, saturating only for Inline graphic (not shown). In the top middle and right panel, on the other hand, the bell-shaped outlay of the Inline graphic transition gives way to a monotonically increasing curve, again saturating only for Inline graphic. It can thus be concluded that, while the aspiration based promotion of cooperation is largely independent of Inline graphic, the details of phase transition are very much affected, which can be attributed to an effective alterations of the interaction network due to preferred strategy sources (see also main text for details).

In terms of the facilitation of cooperation, however, it can be concluded that the promotive impact of positive values of Inline graphic and Inline graphic prevails irrespective of Inline graphic. By comparing the extend of pure Inline graphic and mixed Inline graphic regions for different pairs of the two parameters, we can observe that for small values of Inline graphic (top panels in Fig. 4) it is best if all the players, i.e. Inline graphic, aspire to their slightly (note that Inline graphic is small) fitter neighbors, while for large Inline graphic (bottom panels in Fig. 4) it is best if only approximately half of the players, i.e. Inline graphic, aspire to their most fittest neighbors. The same conclusions were stated already upon the inspection of results presented in Figs. 2 and 3, and with this we now affirm that not only is the promotion of cooperation via heterogeneous aspirations robust against differences in the interaction networks, but also against variations in the uncertainty by strategy adoptions.

It remains of interest to elucidate why then cooperative behavior is in fact promoted by positive values of Inline graphic and Inline graphic. To provide answers, we show in Fig. 5 time courses of Inline graphic for different characteristic combination of the two main parameters that we have used throughout this work. What should attract the attention is the fact that in the most early stages of the evolutionary process (note that values of Inline graphic were recorded also in-between full iteration steps) it appears as if defectors would actually fare better than cooperators. This is actually what one would expect, given that defectors are, as individuals, more successful than cooperators and will thus be chosen more likely as potential strategy donors if Inline graphic is positive. This should in turn amplify their chances of spreading and ultimately result in the decimation of cooperators (indeed, only between 20–30% survive). Quite surprisingly though, the tide changes fairly fast, and as one can observe from the presented time courses, frequently the more so the deeper the initial downfall of cooperators. We argue that for positive values of Inline graphic and Inline graphic a negative feedback effect occurs, which halts and eventually reverts what appears to be a march of defectors toward dominance. Namely, in the very early stages of the game defectors are able to plunder very efficiently, which quickly results in a state where there are hardly any cooperators left to exploit. Consequently, the few remaining clusters of cooperators start recovering lost ground against weakened defectors. Crucial thereby is the fact that the clusters formed by cooperators are impervious to defector attacks even at high values of Inline graphic because of the positive selection towards the fittest neighbors acting as strategy sources (occurring for Inline graphic). In a sea of cooperators this is practically always another cooperator rather than a defector trying to penetrate into the cluster. This newly identified mechanism ultimately results in fairly widespread cooperation that goes beyond what can be warranted by the spatial reciprocity alone (see e.g. [6]), and this irrespective of the underlying interaction network and uncertainty by strategy adoptions.

Figure 5. Time courses of the density of cooperators on the square lattice.

Figure 5

Results are presented for the aspiration parameters Inline graphic (solid lines) and Inline graphic (dashed lines), each for three different fractions of type Inline graphic players Inline graphic, as depicted on the figure. The crucial feature of all time courses is the initial temporary downfall of cooperators, which sets in for all depicted combinations of Inline graphic and Inline graphic. Quite remarkably, what appears to become an ever faster extinction eventually becomes a rise to, at least in some cases, near-dominance. Note that the horizontal axis is logarithmic and that values of Inline graphic were recorded also in-between full iteration steps to ensure a proper resolution. Depicted results were obtained for Inline graphic and Inline graphic.

Finally, it is instructive to examine whether an optimal intermediate value of Inline graphic, determining the aspiration level of player Inline graphic, can emerge spontaneously from an initial array of normally distributed values. This would imply that natural selection indeed favors individuals with a specific aspiration level, which would in turn extend the credibility of thus far presented results that were obtained primarily in a top-down manner [by optimizing a population-level property (cooperation) by means of an appropriate selection of parameters determining the aspiration level of individuals]. For this purpose we omit the division of the population on players of type Inline graphic and Inline graphic, and initially assign to every player a value Inline graphic that is drawn randomly from a Gaussian distribution with a given mean Inline graphic and standard deviation Inline graphic. Then if player Inline graphic adopts the strategy from player Inline graphic also Inline graphic becomes equal to Inline graphic (see Methods for details). Results obtained with this alternative coevolutionary model are presented in Fig. 6. It can be observed that the initial Gaussian distribution sharpens fast around an intermediate value of Inline graphic, which then gradually becomes more and more frequent in the population as the natural selection spontaneously eliminates the less favorable values that warrant a lower individual fitness. The final state is a population where virtually all players have an identical aspiration level Inline graphic, and accordingly, the outcome in terms of the stationary density of cooperators is equal to that obtained with the original model having Inline graphic and Inline graphic. In this sense the preceding results are validated and their generality extended by means of a bottom-up approach entailing a spontaneous coevolution towards an intermediate individually optimal aspiration level. We note, however, that with this simple coevolutionary model the result that heterogeneous aspirations promote cooperation is not exactly reproduced. Further studies on more sophisticated models incorporating coevolving aspirations are required to arrive spontaneously at a heterogeneous distribution of individual aspiration levels. Inspirations for this can be found in the recent review on coevolutionary games [23], and we are looking forward to further developments in this direction.

Figure 6. Spontaneous fixation towards an intermediate aspiration level by means of natural selection.

Figure 6

Presented results were obtained with the alternative model where players are not divided into two groups and initially every player is assigned a random aspiration level Inline graphic drawn from a Gaussian distribution with the mean Inline graphic and standard deviation Inline graphic. The main panel depicts the distributions Inline graphic of individual aspiration levels as recorded at Inline graphic (solid black line), Inline graphic (dashed red line) and Inline graphic (dotted green line) full iteration steps. The fixation towards a dominant average value Inline graphic due to natural selection is evident since the interval of Inline graphic values still present in the population becomes more and more narrow as time progresses. The inset shows the convergence of Inline graphic (solid gray line) and Inline graphic (dotted blue line). The initial temporary downfall of cooperators, followed by the rise to near-dominance, is well-expressed also in the coevolutionary setup, and the stationary density agrees well with the results obtained by means of the original model with Inline graphic and Inline graphic (compare with the dashed cyan line in Fig. 5). Note that in the inset the horizontal axis is logarithmic and that values of Inline graphic and Inline graphic were recorded also in-between full iteration steps to ensure a proper resolution. Depicted results were obtained for Inline graphic and Inline graphic.

Discussion

We have shown that heterogenous aspiration to the fittest, i.e. the propensity of designating the most successful neighbor as being the role model, may be seen as a universally applicable promoter of cooperation that works on different interaction networks and under different levels of stochasticity. For low and moderate values of the aspiration parameter Inline graphic cooperation thrives best if the total population abides to aspiring to the fittest. For large values of Inline graphic, however, it is best if only approximately half of the players persuasively attempt to copy their most successful neighbors while the rest chooses their opponents uniformly at random. The optimal evolution of cooperation thus requires fine-tuning of both, the density of players that are prone to aspiring to the fittest, as well as the aspiration parameter that determines how fit a neighbor actually must be in order to be considered as the potential source of the new strategy. In addition, by studying an alternative model where individual aspiration levels were also subject to evolution, we have shown that an intermediate value of the aspiration level emerges spontaneously through natural selection, thus supplementing the main results by means of a coevolutionary approach.

Notably, the extensions of the prisoner's dilemma game we have considered here seem very reasonable and are in fact easily justifiable with realistic examples. For example, it is a fact that people will, in general, much more likely follow a successful individual than somebody who is just struggling to get by. Under certain adverse circumstances, like in a state of rebelion or in revolutionary times, however, it is also possible that individuals will be inspired to copy their less successful partners or those that seem to do more harm than good. In many ways it seems that the ones who are satisfied with just picking somebody randomly to aspire to are the ones that are most difficult to come by. In this sense the rather frequently adopted random selection of a neighbor, retrieved in our case if Inline graphic (or equivalently Inline graphic), seems in many ways like the least probable alternative. In this sense it is interesting to note that our aspiring to the fittest becomes identical to the frequently adopted, especially in the early seminal works on games on grids [26], [33], [34], “best takes all” adoption rule if Inline graphic, Inline graphic in Eq. (2), and Inline graphic in Eq. (3). Although in our simulations we never quite reach the “best takes all” limit, and thus a direct comparison with the seminal works is somewhat circumstantial, we find here that the intermediate regions of heterogenous aspirations offer fascinating new insights into the evolution of cooperation, and we hope that this work will inspire future studies, especially in terms of understanding the emergence of successful leaders in societies via a coevolutionary process [23].

Methods

An evolutionary prisoner's dilemma game with the temptation to defect Inline graphic (the highest payoff received by a defector if playing against a cooperator), reward for mutual cooperation Inline graphic, the punishment for mutual defection Inline graphic, and the sucker's payoff Inline graphic (the lowest payoff received by a cooperator if playing against a defector) is used as the basis for our simulations. Without loss of generality the payoffs can be rescaled as Inline graphic, Inline graphic, Inline graphic and Inline graphic, where Inline graphic is the cost-to-benefit ratio [5]. For positive Inline graphic we have Inline graphic, thus strictly satisfying the prisoner's dilemma payoff ranking.

As the interaction network, we use either a regular Inline graphic square lattice, the random regular graph (RRG) constructed as described in [65], or the small-world (SW) topology with an average degree of four generated via the Watts-Strogatz algorithm [66]. Each vertex Inline graphic is initially designated as hosting either players of type Inline graphic or Inline graphic with the probability Inline graphic and Inline graphic, respectively. This division of players is performed uniformly at random irrespective of their initial strategies and remains unchanged during the simulations. According to established procedures, each player is initially also designated either as a cooperator (Inline graphic) or defector (Inline graphic) with equal probability. The game is iterated forward in accordance with the sequential simulation procedure comprising the following elementary steps. First, player Inline graphic acquires its payoff Inline graphic by playing the game with all its neighbors. Next, we evaluate in the same way the payoffs of all the neighbors of player Inline graphic and subsequently select one neighbor Inline graphic via the probability

graphic file with name pone.0015117.e305.jpg (1)

where the sum runs over all the neighbors of player Inline graphic. Importantly, Inline graphic is the so-called selection or aspiration parameter that depends on the type of player Inline graphic according to

graphic file with name pone.0015117.e309.jpg (2)

Evidently, if the aspiration parameter Inline graphic then irrespective of Inline graphic (density of type Inline graphic players) the most frequently adopted situation is recovered where player Inline graphic is chosen uniformly at random from all the neighbors of player Inline graphic. For Inline graphic and Inline graphic, however, Eqs. (1) and (2) introduce a preference in all players of type Inline graphic (but not in players of type Inline graphic) to copy the strategy of those neighbors who have a high fitness, or equivalently, a high payoff Inline graphic. Lastly then, after the neighbor Inline graphic that is aspired to by player Inline graphic is chosen, player Inline graphic adopts the strategy Inline graphic from the selected player Inline graphic with the probability

graphic file with name pone.0015117.e325.jpg (3)

where Inline graphic denotes the amplitude of noise or its inverse (Inline graphic) the so-called intensity of selection [38]. Irrespective of the values of Inline graphic and Inline graphic one full iteration step involves all players Inline graphic having a chance to adopt a strategy from one of their neighbors once.

An alternative model, allowing for individual Inline graphic values to be subject to evolution as well, entails omitting the division of the population on two types of players and assigning to every individual an initial Inline graphic value that is drown randomly from a Gaussian distribution having mean Inline graphic and standard deviation Inline graphic, as was done recently in [57], for example. Subsequently, if player Inline graphic adopts the strategy from player Inline graphic following the identical procedure as described above for the original model, then the value of Inline graphic changes to that of Inline graphic as well. The key question that we aim to answer with this model is whether a specific aspiration level is indeed optimal for an individual to prosper, and if yes, does the selection pressure favor it spontaneously. Essentially, we are interested in the distribution of Inline graphic values after the stationary fraction of strategies in the population is reached. A link with the original model can be established by considering in this case Inline graphic to equal one and Inline graphic.

Results of computer simulations were obtained on populations comprising Inline graphic to Inline graphic individuals, whereby the fraction of cooperators Inline graphic was determined within Inline graphic full iteration steps after sufficiently long transients were discarded. Moreover, since the heterogeneous preferential selection of neighbors may introduce additional disturbances, final results were averaged over up to Inline graphic independent runs for each set of parameter values in order to assure suitable accuracy.

Acknowledgments

We have benefited substantially from the insightful comments of the Editor James Arthur Robert Marshall and an anonymous referee, which were instrumental for elevating the quality of this work. Helpful discussions with Professor Lianzhong Zhang are gratefully acknowledged as well.

Footnotes

Competing Interests: The authors have declared that no competing interests exist.

Funding: MP acknowledges support from the Slovenian Research Agency (Grant No. Z1-2032). ZW acknowledges support from the Center for Asia Studies of Nankai University (Grant No. 2010-5) and from the National Natural Science Foundation of China (Grant No. 10672081). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Hofbauer J, Sigmund K. Cambridge, UK: Cambridge University Press; 1998. Evolutionary Games and Population Dynamics. [Google Scholar]
  • 2.Nowak MA. Cambridge, MA: Harvard University Press; 2006. Evolutionary Dynamics. [Google Scholar]
  • 3.Maynard Smith J. Cambridge, UK: Cambridge University Press; 1982. Evolution and the Theory of Games. [Google Scholar]
  • 4.Roca CP, Cuesta JA, Sánchez A. Evolutionary game theory: Temporal and spatial effects beyond replicator dynamics. Phys Life Rev. 2009;6:208–249. doi: 10.1016/j.plrev.2009.08.001. [DOI] [PubMed] [Google Scholar]
  • 5.Hauert C, Szabó G. Game theory and physics. Am J Phys. 2005;73:405–414. [Google Scholar]
  • 6.Szabó G, Fáth G. Evolutionary games on graphs. Phys Rep. 2007;446:97–216. [Google Scholar]
  • 7.Nowak M, Sigmund K. Game-dynamical aspects of the prisoner's dilemma. Appl Math Comp. 1989;30:191–213. [Google Scholar]
  • 8.Moyano LG, Sánchez A. Evolving learning rules and emergence of cooperation in spatial prisoner's dilemma. J Theor Biol. 2009;259:84–95. doi: 10.1016/j.jtbi.2009.03.002. [DOI] [PubMed] [Google Scholar]
  • 9.Doebeli M, Hauert C. Models of cooperation based on prisoner's dilemma and snowdrift game. Ecol Lett. 2005;8:748–766. [Google Scholar]
  • 10.Szabó G, Szolnoki A, Sznaider GA. Segregation process and phase transition in cyclic predator-prey models with even number of species. Phys Rev E. 2007;76:051921. doi: 10.1103/PhysRevE.76.051921. [DOI] [PubMed] [Google Scholar]
  • 11.Szolnoki A, Perc M, Szabó G, Stark HU. Impact of aging on the evolution of cooperation in the spatial prisoner's dilemma game. Phys Rev E. 2009;80:021901. doi: 10.1103/PhysRevE.80.021901. [DOI] [PubMed] [Google Scholar]
  • 12.Milinski M, Semmann D, Krambeck HJ. Reputation helps to solve the ‘tragedy of the commons’. Nature. 2002;415:424–426. doi: 10.1038/415424a. [DOI] [PubMed] [Google Scholar]
  • 13.Fehr E. Don't lose your reputation. Nature. 2004;432:449–450. doi: 10.1038/432449a. [DOI] [PubMed] [Google Scholar]
  • 14.Van Segbroeck S, Santos FC, Lenaerts T, Pacheco JM. Reacting differently to adverse ties promotes cooperation in social networks. Phys Rev Lett. 2009;102:058105. doi: 10.1103/PhysRevLett.102.058105. [DOI] [PubMed] [Google Scholar]
  • 15.Maynard Smith J, Parker GA. The logic of asymmetric contests. Anim Behav. 1976;24:159–175. [Google Scholar]
  • 16.Selten R. A note on evolutionary stable strategies in asymmetric animal conflicts. J Theor Biol. 1980;84:93–101. doi: 10.1016/s0022-5193(80)81038-1. [DOI] [PubMed] [Google Scholar]
  • 17.Hammerstein P. The role of asymmetries in animal contests. Anim Behav. 1981;29:193–205. [Google Scholar]
  • 18.Marshall JAR. The donation game with roles played between relatives. J Theor Biol. 2009;260:386–391. doi: 10.1016/j.jtbi.2009.07.008. [DOI] [PubMed] [Google Scholar]
  • 19.Milinski M. TIT FOR TAT in sticklebacks and the evolution of cooperation. Nature. 1987;325:533–535. doi: 10.1038/325433a0. [DOI] [PubMed] [Google Scholar]
  • 20.Hamilton WD. Genetical evolution of social behavior I & II. J Theor Biol. 1964;7:1–52. doi: 10.1016/0022-5193(64)90038-4. [DOI] [PubMed] [Google Scholar]
  • 21.Nowak MA. Five rules for the evolution of cooperation. Science. 2006;314:1560–1563. doi: 10.1126/science.1133755. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Schuster S, Kreft JU, Schroeter A, Pfeiffer T. Use of game-theoretical methods in biochemistry and biophysics. J Biol Phys. 2008;34:1–17. doi: 10.1007/s10867-008-9101-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Perc M, Szolnoki A. Coevolutionary games – a mini review. BioSystems. 2010;99:109–125. doi: 10.1016/j.biosystems.2009.10.003. [DOI] [PubMed] [Google Scholar]
  • 24.Lehmann L, Keller L. The evolution of cooperation and altruism - a general framework and a classification of models. J Evol Biol. 2006;19:1365–1376. doi: 10.1111/j.1420-9101.2006.01119.x. [DOI] [PubMed] [Google Scholar]
  • 25.Marshall JAR. Ultimate causes and the evolution of altruism. Behav Ecol Sociobiol: in press. 2010.
  • 26.Nowak MA, May RM. Evolutionary games and spatial chaos. Nature. 1992;359:826–829. [Google Scholar]
  • 27.Albert R, Barabási AL. Statistical mechanics of complex networks. Rev Mod Phys. 2002;74:47–97. [Google Scholar]
  • 28.Newman MEJ. The structure and function of complex networks. SIAM Review. 2003;45:167–256. [Google Scholar]
  • 29.Boccaletti S, Latora V, Moreno Y, Chavez M, Hwang D. Complex networks: Structure and dynamics. Phys Rep. 2006;424:175–308. [Google Scholar]
  • 30.Ohtsuki H, Hauert C, Lieberman E, Nowak MA. A simple rule for the evolution of cooperation on graphs and social networks. Nature. 2006;441:502–505. doi: 10.1038/nature04605. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Lehmann L, Keller L, Sumpter D. The evolution of helping and harming on graphs: the return of the inclusive fitness effect. J Evol Biol. 2007;20:2284–2295. doi: 10.1111/j.1420-9101.2007.01414.x. [DOI] [PubMed] [Google Scholar]
  • 32.Grafen A. An inclusive fitness analysis of altruism on a cyclical network. J Evol Biol. 2007;20:2278–2283. doi: 10.1111/j.1420-9101.2007.01413.x. [DOI] [PubMed] [Google Scholar]
  • 33.Nowak MA, May RM. The spatial dilemmas of evolution. Int J Bifurcat Chaos. 1993;3:35–78. [Google Scholar]
  • 34.Nowak MA, Bonhoeffer S, May RM. More spatial games. Int J Bifurcat Chaos. 1994;4:33–56. [Google Scholar]
  • 35.Ohtsuki H, Nowak MA. Evolutionary games on cycles. Proc R Soc Lond B. 2006;273:2249–2256. doi: 10.1098/rspb.2006.3576. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Schlag KH. Why imitate, and if so, how? a bounded rational approach to multi-armed bandits. J Econ Theory. 1998;78:130–156. [Google Scholar]
  • 37.Wang S, Szalay MS, Zhang C, Csermely P. Learning and innovative elements of strategy adoption rules expand cooperative network topologies. PLoS ONE. 2008;3:e1917. doi: 10.1371/journal.pone.0001917. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Szabó G, Tőke C. Evolutionary prisoner's dilemma game on a square lattice. Phys Rev E. 1998;58:69–73. [Google Scholar]
  • 39.Abramson G, Kuperman M. Social games in a social network. Phys Rev E. 2001;63:030901(R). doi: 10.1103/PhysRevE.63.030901. [DOI] [PubMed] [Google Scholar]
  • 40.Ren J, Wang WX, Qi F. Randomness enhances cooperation: coherence resonance in evolutionary game. Phys Rev E. 2007;75:045101(R). doi: 10.1103/PhysRevE.75.045101. [DOI] [PubMed] [Google Scholar]
  • 41.Chen XJ, Wang L. Promotion of cooperation induced by appropriate payoff aspirations in a small-world networked game. Phys Rev E. 2008;77:017103. doi: 10.1103/PhysRevE.77.017103. [DOI] [PubMed] [Google Scholar]
  • 42.Vukov J, Szabó G, Szolnoki A. Cooperation in the noisy case: Prisoner's dilemma game on two types of regular random graphs. Phys Rev E. 2006;73:067103. doi: 10.1103/PhysRevE.73.067103. [DOI] [PubMed] [Google Scholar]
  • 43.Santos FC, Pacheco JM. Scale-free networks provide a unifying framework for the emergence of cooperation. Phys Rev Lett. 2005;95:098104. doi: 10.1103/PhysRevLett.95.098104. [DOI] [PubMed] [Google Scholar]
  • 44.Santos FC, Rodrigues JF, Pacheco JM. Graph topology plays a determinant role in the evolution of cooperation. Proc R Soc Lond B. 2006;273:51–55. doi: 10.1098/rspb.2005.3272. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Poncela J, Gómez-Gardeñes J, Floría LM, Moreno Y. Robustness of cooperation in the evolutionary prisoner's dilemma on complex systems. New J Phys. 2007;9:184. [Google Scholar]
  • 46.Assenza S, Gómez-Gardeñes J, Latora V. Enhancement of cooperation in highly clustered scale-free networks. Phys Rev E. 2008;78:017101. doi: 10.1103/PhysRevE.78.017101. [DOI] [PubMed] [Google Scholar]
  • 47.Szolnoki A, Perc M, Danku Z. Towards effective payoffs in the prisoner's dilemma game on scale-free networks. Physica A. 2008;387:2075–2082. [Google Scholar]
  • 48.Perc M. Evolution of cooperation on scale-free networks subject to error and attack. New J Phys. 2009;11:033027. [Google Scholar]
  • 49.Zimmermann MG, Eguíluz V, Miguel MS. Coevolution of dynamical states and interactions in dynamic networks. Phys Rev E. 2004;69:065102(R). doi: 10.1103/PhysRevE.69.065102. [DOI] [PubMed] [Google Scholar]
  • 50.Pacheco JM, Traulsen A, Nowak MA. Coevolution of strategy and structure in complex networks with dynamical linking. Phys Rev Lett. 2006;97:258103. doi: 10.1103/PhysRevLett.97.258103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Szolnoki A, Perc M, Danku Z. Making new connections towards cooperation in the prisoner's dilemma game. EPL. 2008;84:50007. [Google Scholar]
  • 52.Poncela J, Gómez-Gardeñes J, Floría LM, Sánchez A, Moreno Y. Complex cooperative networks from evolutionary preferential attachment. PLoS ONE. 2008;3:e2449. doi: 10.1371/journal.pone.0002449. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Szolnoki A, Perc M. Resolving social dilemmas on evolving random networks. EPL. 2009;86:30007. [Google Scholar]
  • 54.Poncela J, Gómez-Gardeñes J, Traulsen A, Moreno Y. Evolutionary game dynamics in a growing structured population. New J Phys. 2009;11:083031. [Google Scholar]
  • 55.Szolnoki A, Perc M. Emergence of multilevel selection in the prisoner's dilemma game on coevolving random networks. New J Phys. 2009;11:093033. [Google Scholar]
  • 56.Wu B, Zhou D, Fu F, Luo Q, Wang L, et al. Evolution of cooperation on stochastic dynamical networks. PloS ONE. 2010;5:1–7. doi: 10.1371/journal.pone.0011187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.McNamara JM, Barta Z, Fromhage L, Houston AI. The coevolution of choosiness and cooperation. Nature. 2008;451:189–192. doi: 10.1038/nature06455. [DOI] [PubMed] [Google Scholar]
  • 58.Hauert C. Effects of space in 2×2 games. Int J Bifurc Chaos. 2002;12:1531–1548. [Google Scholar]
  • 59.Perc M. Coherence resonance in spatial prisoner's dilemma game. New J Phys. 2006;8:22. [Google Scholar]
  • 60.Perc M, Marhl M. Evolutionary and dynamical coherence resonance in the pair approximated prisoner's dilemma game. New J Phys. 2006;8:142. [Google Scholar]
  • 61.Tanimoto J. Promotion of cooperation by payoff noise in a 2×2 game. Phys Rev E. 2007;76:041130. doi: 10.1103/PhysRevE.76.041130. [DOI] [PubMed] [Google Scholar]
  • 62.Traulsen A, Nowak MA, Pacheco JM. Stochastic payoff evaluation increases the temperature of selection. J Theor Biol. 2007;244:349–356. doi: 10.1016/j.jtbi.2006.08.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Szabó G, Vukov J, Szolnoki A. Phase diagrams for an evolutionary prisoner's dilemma game on two-dimensional lattices. Phys Rev E. 2005;72:047107. doi: 10.1103/PhysRevE.72.047107. [DOI] [PubMed] [Google Scholar]
  • 64.Szolnoki A, Perc M, Szabó G. Topology-independent impact of noise on cooperation in spatial public goods games. Phys Rev E. 2009;80:056109. doi: 10.1103/PhysRevE.80.056109. [DOI] [PubMed] [Google Scholar]
  • 65.Szabó G, Szolnoki A, Izsák R. Rock-scissors-paper game on regular small-world networks. J Phys A: Math Gen. 2004;37:2599–2609. [Google Scholar]
  • 66.Watts DJ, Strogatz SH. Collective dynamics of ‘small world’ networks. Nature. 1998;393:440–442. doi: 10.1038/30918. [DOI] [PubMed] [Google Scholar]

Articles from PLoS ONE are provided here courtesy of PLOS

RESOURCES