Heterogeneous Aspirations Promote Cooperation in the Prisoner's Dilemma Game

Abstract

To be the fittest is central to proliferation in evolutionary games. Individuals thus adopt the strategies of better performing players in the hope of successful reproduction. In structured populations the array of those that are eligible to act as strategy sources is bounded to the immediate neighbors of each individual. But which one of these strategy sources should potentially be copied? Previous research dealt with this question either by selecting the fittest or by selecting one player uniformly at random. Here we introduce a parameter that interpolates between these two extreme options. Setting equal to zero returns the random selection of the opponent, while positive favor the fitter players. In addition, we divide the population into two groups. Players from group select their opponents as dictated by the parameter , while players from group do so randomly irrespective of . We denote the fraction of players contained in groups and by and , respectively. The two parameters and allow us to analyze in detail how aspirations in the context of the prisoner's dilemma game influence the evolution of cooperation. We find that for sufficiently positive values of there exist a robust intermediate for which cooperation thrives best. The robustness of this observation is tested against different levels of uncertainty in the strategy adoption process and for different interaction networks. We also provide complete phase diagrams depicting the dependence of the impact of and for different values of , and contrast the validity of our conclusions by means of an alternative model where individual aspiration levels are subject to evolution as well. Our study indicates that heterogeneity in aspirations may be key for the sustainability of cooperation in structured populations.

Introduction

Understanding the evolution of cooperation among selfish individuals in human and animal societies remains a grand challenge across disciplines. Evolutionary games are employed frequently as the theoretical framework of choice in order to interpret the emergence and survival of cooperative behavior [1]–[6]. The prisoner's dilemma game, in particular, has attracted considerable interest [7]–[11] as the essential yet minimalist example of a social dilemma. In the original two-person one-shot game the two players have two strategies to choose from (cooperation and defection), and their payoffs depend on the simultaneous decision of both. If they choose to cooperate they will receive the highest collective payoff, which will be shared equally among them. Mutual defection, on the other hand, yields the lowest collective payoff. Yet to defect is tempting because it yields a higher individual payoff regardless of the opponent's decision. It is thus frequently so that both players choose not to cooperate, thus procreating the inevitable social dilemma. In reality, however, interactions may be repeated and the reputation of players compromised [12], [13]. Additionally, individuals may alter with whom they interact [14], and different behaviors may be expressed when participants in a social interaction occupy different roles [15]–[18]. Such and similar considerations have been very successful in elucidating why the unadorned scenario of total defection is often at odds with reality [19], where it is clear that both humans and animals cooperate to achieve what would be impossible by means of isolated efforts. Mechanisms supporting cooperation identified thus far include kin selection [20] as well as many others [6], [21]–[23], and there is progress in place aimed at unifying some of these approaches [24], [25].

Probably the most vibrant of all in recent years have been advances building on the seminal paper by Nowak and May [26], who showed that spatial structure may sustain cooperation without the aid of additional mechanisms or strategic complexity. Although in part anticipated by Hamilton's comments on viscous populations [20], it was fascinating to discover that structured populations, including complex and social networks [27]–[29], provide an optimal playground for the pursuit of cooperation. Notably, a simple rule for the evolution of cooperation on graphs and social networks is that natural selection favors cooperation if , where is the benefit of the altruistic act, is its cost, while is the average number of neighbors [30]. This is similar to Hamilton's rule stating that should be larger than the coefficient of genetic relatedness between individuals [20]. In fact, on graphs and social networks the evolution of altruism can thus be fully explained by the inclusive fitness theory since the population is structured such that interactions are between genetic relatives on average [25], [31], [32].

According to the “best takes all” rule [33], [34] players are allowed to adopt the strategy of one of their neighbors, provided its payoff is higher than that from the other neighbors as well as from the player aspiring to improve by changing its strategy. Based on this relatively simple setup, it was shown that on a square lattice cooperators form compact clusters and so protect themselves against being exploited by defectors. The “best takes all” strategy adoption rule is, however, just one of the many possible alternatives that were considered in the past. Other examples include the birth-death and imitation rule [35], the proportional imitation rule [36], the reinforcement learning adoption rule [37], or the Fermi-function based strategy adoption rule [38]. The latter received substantial attention, particularly in the physics community, for its compatibility with the Monte Carlo simulation procedure and the straightforward adjustment of the level of uncertainty governing the strategy adoptions . However, with this rule the potential donor of the new strategy is selected uniformly at random from all the neighbors. This is somewhat untrue to what can be observed in reality, where in fact individuals typically aspire to their most successful neighbors rather than just somebody random. In this sense the “best takes all” rule seems more appropriate, although it fails to account for errors in judgment, uncertainty, external factors, and other disturbances that may vitally affect how we evaluate and see our co-players. Here we therefore propose a simple tunable function that interpolates between the “best takes all” and the random selection of a neighbor in a smooth fashion by means of a single parameter . In this sense the parameter acts as an aspiration parameter, determining to what degree neighbors with a higher payoff will be considered more likely as potential strategy sources than other (randomly selected) neighbors.

Aiming to further disentangle the role of aspirations, we also consider two types of players, denoted by type and , respectively. While players of type conform to the aspirations imposed by the value of the aspiration parameter , type players choose whom to potentially copy uniformly at random irrespective of . We denote the fraction of type and players by and , respectively. This additional division of players into two groups is motivated by the overwhelming evidence indicating that heterogeneity, almost irrespective of its origin, promotes cooperative actions. Most notably associated with this statement are complex networks, including small-world networks [39]–[41], random regular graphs [5], [42], scale-free networks [43]–[48], as well as adaptive and growing networks [49]–[56]. Furthermore, we follow the work by McNamara et al. on the coevolution of choosiness and cooperation [57], in particular by omitting the separation of the population on two types of players and introducing the heterogeneity by means of normally distributed individual aspiration levels that are then also subject to evolution.

At present, we thus investigate how aspirations on an individual level affect the evolution of cooperation. Having something to aspire to is crucial for progress and betterment. But how high should we set our goals? Should our role models be only overachievers and sports heroes, or is it perhaps better to aspire to achieving somewhat more modest goals? Here we address these questions in the context of the evolutionary prisoner's dilemma game and determine just how strong and how widespread aspirations should be for cooperation to thrive best. As we will show, a strong drive to excellence in the majority of the population may in fact act detrimental on the evolution of cooperation, while on the other hand, properly spread and heterogeneous aspirations may be just the key to fully eliminating the defectors. We will show that this holds irrespective of the structure of the underlying interaction network, as well as irrespective of the level of uncertainty by strategy adoptions . In addition, the presented results will be contrasted with the output of a simple coevolutionary model, where individual aspirations will also be subject to evolution by means of natural selection. We will conclude that appropriately tuned aspirations may be seen as a universally applicable promoter of cooperation, which will hopefully inspire new studies along this line of research.

Results

Depending on the interaction network, the strategy adoption rule and other simulation details (see e.g. [4], [6], [23], [58]), there always exists a critical cost-to-benefit ratio at which cooperators in the prisoner's dilemma die out. This is directly related to Hamilton's rule stating that natural selection favors cooperation if is larger than the coefficient of genetic relatedness between individuals [20]. If the aspiration parameter (note that then the division of players to those of type and those of type is irrelevant), , and the interaction network is a square lattice, then, in our case, . In what follows, we will typically set slightly below this threshold to and examine how different values of , , , as well as different interaction networks influence the outcome of the prisoner's dilemma game.

It is instructive to first examine characteristic snapshots of the square lattice for different values of and . Results presented in Fig. 1 hint to the conclusion that heterogenous aspiration to the fittest promotes cooperation, although the details of this claim depend somewhat on the value of the aspiration parameter . For small values of it is best if all the players, i.e. , aspire to their slightly (note that is small) fitter neighbors and thus none actually choose the potential strategy sources uniformly at random. This can be deduced from the top three panels of Fig. 1 if compared from left to right. For large , however, it is best if only half of the players, i.e. , aspire to their most fittest neighbors, while the other half chooses their role models randomly. This can be observed if one compares the bottom three panels of Fig. 1 with one another, although the difference in the overall density of cooperators (depicted green and blue) between the middle and the right panel is fairly small. Finally, the role of the aspiration parameter is more clear cut since larger clearly favor the cooperative strategy if compared to small . This can be observed if comparing the snapshots presented in Fig. 1 vertically.

Figure 1. Characteristic snapshots of strategy distributions on the square lattice.

Top row depicts results for the aspiration parameter while the bottom row features results for . In both rows the fraction of type players is , and from left to right. Cooperators of type and are colored green and blue, respectively. Defectors of type and , on the other hand, are colored red and yellow. If comparing the snapshots vertically, it can be observed that larger values of (top , bottom ) clearly promote the evolution of cooperation. The scenario from left to right via increasing the fraction of type players is not so clear cut. For (top row) we can conclude that larger favor cooperative behavior, as clearly the cooperators flourish more and more from the left toward the right panel. For (bottom row), however, it seems that for (bottom middle) cooperators actually fare better then for both (bottom left) and (bottom right). Hence, the conclusion imposes that for higher values an intermediate (rather than the maximal, as is the case for lower ) fraction of type players (those that aspire to their most fittest neighbors only) is optimal for the evolution of cooperation. Results in all panels were obtained for and .

Since the snapshots presented in Fig. 1 can be used primarily for an initial qualitative assessment of the impact of heterogeneous aspirations, we present in Fig. 2 the fraction of cooperators (left) and the critical cost-to-benefit ratio (right) in dependence on for different values of . It can be observed that the promotion of cooperation for the optimal combination of the two parameters, being and , is really remarkable. The fraction of cooperators rises from to , while the critical cost-to-benefit ratio rises a full order of magnitude from to . As tentatively deduced from the lower three snapshots in Fig. 1, it can also be observed that for high values of an intermediate fraction of type players is optimal for the evolution of cooperation. Conversely, for low the fraction of cooperators and the critical cost-to-benefit ratio both increase monotonously with increasing . If, however, selecting a particular value of , then the impact of the aspiration parameter is always such that cooperation is the more promoted the larger the value of . This can be observed clearly from both panels, and indeed seems like the main driving force behind the elevated levels of cooperation. Fine-tuning the fraction of players making use of the aspiration to the fittest (from downwards since the limit trivially returns the random selection of potential strategy sources) at high can rise the cooperation level further, but more in the sense of minor adjustments, similarly as was observed in the past for the impact of uncertainty by strategy adoptions [42] or the impact of noise [59].

Figure 2. Promotion of cooperation due to heterogenous aspirations on the square lattice.

Left panel depicts the density of cooperators in dependence on the fraction of type players for different values of the aspiration parameter . Right panel depicts the critical cost-to-benefit ratio at which cooperators die out, i.e. , in dependence on for different values of . Results in both panels convey the message that low values of require a high fraction of type players for cooperation to flourish. Conversely, higher values of sustain cooperation optimally if only half () of the players aspires to their most fittest neighbors while the rest chooses whom to potentially imitate uniformly at random. Optimal conditions for the evolution of cooperation thus require and to be fine-tuned jointly. Depicted results in both panels were obtained for and .

Aiming to generalize the validity of our results, we present in Fig. 3 the fraction of cooperators in dependence on for different values of as obtained on the random regular graph (left) and the small-world network (right). The goal is to test to what extend above conclusions hold also on interaction networks other than the square lattice, in particular such that are more complex and spatially heterogenous. If comparing the obtained results with those presented in the left panel of Fig. 2, it seems save to conclude that they are to a very large degree qualitatively identical. Some differences nevertheless can be observed. The first is that what constitutes a high limit is a bit higher on complex networks than on the regular lattice. Note that for the optimal fraction of type players is practically still . Even for the bell-shaped dependence on is far less pronounced than on the square lattice, and the optimal (the peak of ) is closer to than . The second difference is, looking relatively to the starting point at , that the promotion of cooperation due to positive and is somewhat less prolific. This is, however, not that surprising since complex networks in general promote cooperation already on their own [6], and thus secondary promotive mechanisms may therefore become less expressed. Aside from these fairly mild differences though, we can conclude that heterogenous aspirations do promote cooperation irrespective of the underlying interaction network, and that the details of the promotive effect are largely universal and predictable.

Figure 3. Promotion of cooperation due to heterogenous aspirations on the random regular graph (RRG) and the small-world (SW) network.

Left panel depicts the density of cooperators in dependence on the fraction of type players for different values of the aspiration parameter for the RRG. Right panel depicts in dependence on for different values of for the Watts-Strogatz SW network with the fraction of rewired links equalling . These results are in agreement with those presented in Fig. 2, supporting the conclusion that the impact of heterogenous aspirations on the evolution of cooperation is robust against alterations of the interaction network. As on the square lattice, low, but also intermediate, values of require for cooperation to thrive, while higher values of sustain cooperation optimally only if . Depicted results in both panels were obtained for and .

Next, we proceed with examining how positive values of and fare under different levels of uncertainty by strategy adoptions. The latter can be tuned via [see Eq. (3)], which acts as a temperature parameter in the employed Fermi strategy adoption function [38]. Accordingly, when all information is lost and the strategies are adopted by means of a coin toss. Note that this aspect has thus far not received any attention here as was fixed. The matter is not trivial to address because uncertainty and noise can have a rather profound impact on the evolution of cooperation [40], [42], [59]–[62], and thus care needs to be exercised. The safest and most accurate way to approach the problem is by means of phase diagrams. Since we have two additional parameters ( and ) against which we want to test the impact of , we determined full phase diagrams for six characteristic combinations of and on the square lattice. Obtained results are presented in Fig. 4. Notably, the phase diagram presented in the top left panel of Fig. 4 is well-known, implying the existence of an optimal level of uncertainty for the evolution of cooperation, as was previously reported in [42], [59]. In particular, note that the transition line is bell shaped, indicating that is the optimal temperature at which cooperators are able to survive at the highest value of . Importantly though, this phenomenon can only be observed on interaction topologies lacking overlapping triangles [63], [64]. Interestingly, increasing from (top row) to (bottom row) completely eradicates (as do interaction networks incorporating overlapping triangles) the existence of an optimal , and in fact qualitatively reverses the dependence. The transition line has an inverted bell-shaped outlay, indicating the existence of the worst rather than an optimal temperature for the evolution of cooperation. The qualitative changes are less profound if is kept constant at (top row) and increases (from left to right). Still, however, the bell-shaped outlay of the transition gives way to a monotonically increasing curve, saturating only for high . These qualitative changes in the phase diagrams imply that increasing the aspiration parameter or the fraction of players abiding to it (type ) effectively alters the interaction network. While the square lattice obviously lacks overlapping triangles and thus enables the observation of an optimal for small enough values of and (or a combination thereof, as is the case in the top left panels), trimming the likelihood of who will act as a strategy source and how many players will actually aspire to their fittest neighbors seems to effectively enhance linkage among essentially disconnected triplets and thus precludes the same observation. It is instructive to note that a similar phenomenon was observed recently in public goods games, where the joint membership in large groups was also found to alter the effective interaction network and thus the impact of uncertainly on the evolution of cooperation [64].

Figure 4. Full phase diagrams for the square lattice.

Top row depicts results for the aspiration parameter while the bottom row features results for . In both rows the fraction of type players is , and from left to right. The outline of panels thus corresponds to the snapshots presented in Fig. 1. Thin blue and thick red lines mark the border between stationary pure and phases and the mixed phase, respectively. In agreement with previous works [42], [63], it can be observed that for and (top left) there exists an intermediate uncertainty in the strategy adoption process (an intermediate value of ) for which the survivability of cooperators is optimal, i.e. is maximal. Conversely, while the borderline separating the pure and the mixed phase for all the other combinations of and exhibits a qualitatively identical outlay as for the and case, the transition is qualitatively different and very much dependent on the particularities players' aspirations. Note that in all the bottom panels there exist an intermediate value of for which is minimal rather than maximal, while towards the large limit increases, saturating only for (not shown). In the top middle and right panel, on the other hand, the bell-shaped outlay of the transition gives way to a monotonically increasing curve, again saturating only for . It can thus be concluded that, while the aspiration based promotion of cooperation is largely independent of , the details of phase transition are very much affected, which can be attributed to an effective alterations of the interaction network due to preferred strategy sources (see also main text for details).

In terms of the facilitation of cooperation, however, it can be concluded that the promotive impact of positive values of and prevails irrespective of . By comparing the extend of pure and mixed regions for different pairs of the two parameters, we can observe that for small values of (top panels in Fig. 4) it is best if all the players, i.e. , aspire to their slightly (note that is small) fitter neighbors, while for large (bottom panels in Fig. 4) it is best if only approximately half of the players, i.e. , aspire to their most fittest neighbors. The same conclusions were stated already upon the inspection of results presented in Figs. 2 and 3, and with this we now affirm that not only is the promotion of cooperation via heterogeneous aspirations robust against differences in the interaction networks, but also against variations in the uncertainty by strategy adoptions.

It remains of interest to elucidate why then cooperative behavior is in fact promoted by positive values of and . To provide answers, we show in Fig. 5 time courses of for different characteristic combination of the two main parameters that we have used throughout this work. What should attract the attention is the fact that in the most early stages of the evolutionary process (note that values of were recorded also in-between full iteration steps) it appears as if defectors would actually fare better than cooperators. This is actually what one would expect, given that defectors are, as individuals, more successful than cooperators and will thus be chosen more likely as potential strategy donors if is positive. This should in turn amplify their chances of spreading and ultimately result in the decimation of cooperators (indeed, only between 20–30% survive). Quite surprisingly though, the tide changes fairly fast, and as one can observe from the presented time courses, frequently the more so the deeper the initial downfall of cooperators. We argue that for positive values of and a negative feedback effect occurs, which halts and eventually reverts what appears to be a march of defectors toward dominance. Namely, in the very early stages of the game defectors are able to plunder very efficiently, which quickly results in a state where there are hardly any cooperators left to exploit. Consequently, the few remaining clusters of cooperators start recovering lost ground against weakened defectors. Crucial thereby is the fact that the clusters formed by cooperators are impervious to defector attacks even at high values of because of the positive selection towards the fittest neighbors acting as strategy sources (occurring for ). In a sea of cooperators this is practically always another cooperator rather than a defector trying to penetrate into the cluster. This newly identified mechanism ultimately results in fairly widespread cooperation that goes beyond what can be warranted by the spatial reciprocity alone (see e.g. [6]), and this irrespective of the underlying interaction network and uncertainty by strategy adoptions.

Figure 5. Time courses of the density of cooperators on the square lattice.

Results are presented for the aspiration parameters (solid lines) and (dashed lines), each for three different fractions of type players , as depicted on the figure. The crucial feature of all time courses is the initial temporary downfall of cooperators, which sets in for all depicted combinations of and . Quite remarkably, what appears to become an ever faster extinction eventually becomes a rise to, at least in some cases, near-dominance. Note that the horizontal axis is logarithmic and that values of were recorded also in-between full iteration steps to ensure a proper resolution. Depicted results were obtained for and .

Finally, it is instructive to examine whether an optimal intermediate value of , determining the aspiration level of player , can emerge spontaneously from an initial array of normally distributed values. This would imply that natural selection indeed favors individuals with a specific aspiration level, which would in turn extend the credibility of thus far presented results that were obtained primarily in a top-down manner [by optimizing a population-level property (cooperation) by means of an appropriate selection of parameters determining the aspiration level of individuals]. For this purpose we omit the division of the population on players of type and , and initially assign to every player a value that is drawn randomly from a Gaussian distribution with a given mean and standard deviation . Then if player adopts the strategy from player also becomes equal to (see Methods for details). Results obtained with this alternative coevolutionary model are presented in Fig. 6. It can be observed that the initial Gaussian distribution sharpens fast around an intermediate value of , which then gradually becomes more and more frequent in the population as the natural selection spontaneously eliminates the less favorable values that warrant a lower individual fitness. The final state is a population where virtually all players have an identical aspiration level , and accordingly, the outcome in terms of the stationary density of cooperators is equal to that obtained with the original model having and . In this sense the preceding results are validated and their generality extended by means of a bottom-up approach entailing a spontaneous coevolution towards an intermediate individually optimal aspiration level. We note, however, that with this simple coevolutionary model the result that heterogeneous aspirations promote cooperation is not exactly reproduced. Further studies on more sophisticated models incorporating coevolving aspirations are required to arrive spontaneously at a heterogeneous distribution of individual aspiration levels. Inspirations for this can be found in the recent review on coevolutionary games [23], and we are looking forward to further developments in this direction.

Figure 6. Spontaneous fixation towards an intermediate aspiration level by means of natural selection.

Presented results were obtained with the alternative model where players are not divided into two groups and initially every player is assigned a random aspiration level drawn from a Gaussian distribution with the mean and standard deviation . The main panel depicts the distributions of individual aspiration levels as recorded at (solid black line), (dashed red line) and (dotted green line) full iteration steps. The fixation towards a dominant average value due to natural selection is evident since the interval of values still present in the population becomes more and more narrow as time progresses. The inset shows the convergence of (solid gray line) and (dotted blue line). The initial temporary downfall of cooperators, followed by the rise to near-dominance, is well-expressed also in the coevolutionary setup, and the stationary density agrees well with the results obtained by means of the original model with and (compare with the dashed cyan line in Fig. 5). Note that in the inset the horizontal axis is logarithmic and that values of and were recorded also in-between full iteration steps to ensure a proper resolution. Depicted results were obtained for and .

Discussion

We have shown that heterogenous aspiration to the fittest, i.e. the propensity of designating the most successful neighbor as being the role model, may be seen as a universally applicable promoter of cooperation that works on different interaction networks and under different levels of stochasticity. For low and moderate values of the aspiration parameter cooperation thrives best if the total population abides to aspiring to the fittest. For large values of , however, it is best if only approximately half of the players persuasively attempt to copy their most successful neighbors while the rest chooses their opponents uniformly at random. The optimal evolution of cooperation thus requires fine-tuning of both, the density of players that are prone to aspiring to the fittest, as well as the aspiration parameter that determines how fit a neighbor actually must be in order to be considered as the potential source of the new strategy. In addition, by studying an alternative model where individual aspiration levels were also subject to evolution, we have shown that an intermediate value of the aspiration level emerges spontaneously through natural selection, thus supplementing the main results by means of a coevolutionary approach.

Notably, the extensions of the prisoner's dilemma game we have considered here seem very reasonable and are in fact easily justifiable with realistic examples. For example, it is a fact that people will, in general, much more likely follow a successful individual than somebody who is just struggling to get by. Under certain adverse circumstances, like in a state of rebelion or in revolutionary times, however, it is also possible that individuals will be inspired to copy their less successful partners or those that seem to do more harm than good. In many ways it seems that the ones who are satisfied with just picking somebody randomly to aspire to are the ones that are most difficult to come by. In this sense the rather frequently adopted random selection of a neighbor, retrieved in our case if (or equivalently ), seems in many ways like the least probable alternative. In this sense it is interesting to note that our aspiring to the fittest becomes identical to the frequently adopted, especially in the early seminal works on games on grids [26], [33], [34], “best takes all” adoption rule if , in Eq. (2), and in Eq. (3). Although in our simulations we never quite reach the “best takes all” limit, and thus a direct comparison with the seminal works is somewhat circumstantial, we find here that the intermediate regions of heterogenous aspirations offer fascinating new insights into the evolution of cooperation, and we hope that this work will inspire future studies, especially in terms of understanding the emergence of successful leaders in societies via a coevolutionary process [23].

Methods

An evolutionary prisoner's dilemma game with the temptation to defect (the highest payoff received by a defector if playing against a cooperator), reward for mutual cooperation , the punishment for mutual defection , and the sucker's payoff (the lowest payoff received by a cooperator if playing against a defector) is used as the basis for our simulations. Without loss of generality the payoffs can be rescaled as , , and , where is the cost-to-benefit ratio [5]. For positive we have , thus strictly satisfying the prisoner's dilemma payoff ranking.

As the interaction network, we use either a regular square lattice, the random regular graph (RRG) constructed as described in [65], or the small-world (SW) topology with an average degree of four generated via the Watts-Strogatz algorithm [66]. Each vertex is initially designated as hosting either players of type or with the probability and , respectively. This division of players is performed uniformly at random irrespective of their initial strategies and remains unchanged during the simulations. According to established procedures, each player is initially also designated either as a cooperator () or defector () with equal probability. The game is iterated forward in accordance with the sequential simulation procedure comprising the following elementary steps. First, player acquires its payoff by playing the game with all its neighbors. Next, we evaluate in the same way the payoffs of all the neighbors of player and subsequently select one neighbor via the probability

(1)

where the sum runs over all the neighbors of player . Importantly, is the so-called selection or aspiration parameter that depends on the type of player according to

(2)

Evidently, if the aspiration parameter then irrespective of (density of type players) the most frequently adopted situation is recovered where player is chosen uniformly at random from all the neighbors of player . For and , however, Eqs. (1) and (2) introduce a preference in all players of type (but not in players of type ) to copy the strategy of those neighbors who have a high fitness, or equivalently, a high payoff . Lastly then, after the neighbor that is aspired to by player is chosen, player adopts the strategy from the selected player with the probability

(3)

where denotes the amplitude of noise or its inverse () the so-called intensity of selection [38]. Irrespective of the values of and one full iteration step involves all players having a chance to adopt a strategy from one of their neighbors once.

An alternative model, allowing for individual values to be subject to evolution as well, entails omitting the division of the population on two types of players and assigning to every individual an initial value that is drown randomly from a Gaussian distribution having mean and standard deviation , as was done recently in [57], for example. Subsequently, if player adopts the strategy from player following the identical procedure as described above for the original model, then the value of changes to that of as well. The key question that we aim to answer with this model is whether a specific aspiration level is indeed optimal for an individual to prosper, and if yes, does the selection pressure favor it spontaneously. Essentially, we are interested in the distribution of values after the stationary fraction of strategies in the population is reached. A link with the original model can be established by considering in this case to equal one and .

Results of computer simulations were obtained on populations comprising to individuals, whereby the fraction of cooperators was determined within full iteration steps after sufficiently long transients were discarded. Moreover, since the heterogeneous preferential selection of neighbors may introduce additional disturbances, final results were averaged over up to independent runs for each set of parameter values in order to assure suitable accuracy.