Skip to main content
PLOS Computational Biology logoLink to PLOS Computational Biology
. 2014 Apr 24;10(4):e1003567. doi: 10.1371/journal.pcbi.1003567

Evolutionary Game Dynamics in Populations with Heterogenous Structures

Wes Maciejewski 1,*, Feng Fu 2, Christoph Hauert 1
Editor: Francisco C Santos3
PMCID: PMC3998889  PMID: 24762474

Abstract

Evolutionary graph theory is a well established framework for modelling the evolution of social behaviours in structured populations. An emerging consensus in this field is that graphs that exhibit heterogeneity in the number of connections between individuals are more conducive to the spread of cooperative behaviours. In this article we show that such a conclusion largely depends on the individual-level interactions that take place. In particular, averaging payoffs garnered through game interactions rather than accumulating the payoffs can altogether remove the cooperative advantage of heterogeneous graphs while such a difference does not affect the outcome on homogeneous structures. In addition, the rate at which game interactions occur can alter the evolutionary outcome. Less interactions allow heterogeneous graphs to support more cooperation than homogeneous graphs, while higher rates of interactions make homogeneous and heterogeneous graphs virtually indistinguishable in their ability to support cooperation. Most importantly, we show that common measures of evolutionary advantage used in homogeneous populations, such as a comparison of the fixation probability of a rare mutant to that of the resident type, are no longer valid in heterogeneous populations. Heterogeneity causes a bias in where mutations occur in the population which affects the mutant's fixation probability. We derive the appropriate measures for heterogeneous populations that account for this bias.

Author Summary

Understanding the evolution of cooperation is a persistent challenge to evolutionary theorists. A contemporary take on this subject is to model populations with interactions structured as close as possible to actual social networks. These networks are heterogeneous in the number and type of contact each member has. Our paper demonstrates that the fate of cooperation in such heterogeneous populations critically depends on the rate at which interactions occur and how interactions translate into the fitnesses of the strategies. We also develop theory that allows for an evolutionary analysis in heterogeneous populations. This includes deriving appropriate criteria for evolutionary advantage.

Introduction

Population structure has long been known to affect the outcome of an evolutionary process [1][4]. Evolutionary graph theory has emerged as a convenient framework for modelling structured populations [4], [5]. Individuals reside on vertices of the graph and the edges define the interaction neighbourhoods.

A variety of processes have been investigated on a number of graph classes. However, few analytical results exist in general, since an arbitrary graph may not exhibit sufficient symmetry to aid calculations. The most general class of graphs for which analytical results are known is the class of homogeneous (vertex-transitive) graphs. Such a graph Inline graphic has the property that for any two vertices Inline graphic and Inline graphic there exists a structure-preserving transformation Inline graphic of Inline graphic such that Inline graphic. It is worth noting that not all regular graphs are homogeneous; an extreme example is the Frucht graph [6], which is regular of degree Inline graphic and has only the trivial symmetry. Intuitively, this class consists of graphs that “look” the same from any vertex. The amount of symmetry in such graphs has allowed for a complete set of analytical results for restricted types of evolutionary processes and weak selection [7][9]. Despite the tractability of calculations on homogeneous graphs, natural population structures are seldom homogeneous. Therefore it is important to understand the effects of heterogeneous population structures on evolutionary processes [4], [8], [10] and, in particular, on the evolution of cooperation.

In the simplest case there are two strategic types: cooperators that provide a benefit Inline graphic to their interaction partner at some cost Inline graphic to themselves (Inline graphic), whereas defectors provide neither benefits nor incur costs. This basic setup is known as an instance of the prisoner's dilemma and reflects a conflict of interest because mutual cooperation yields payoff Inline graphic and hence both parties prefer this outcome over mutual defection, which yields a payoff of zero. However, at the same time each party is tempted to defect in order to avoid the costs of cooperation. The temptation of increased benefits for unilateral defection thwarts cooperation – to the detriment of all. This conflict of interest characterizes social dilemmas [11], [12].

More general kinds of interactions between two individuals and two strategic types, Inline graphic and Inline graphic, can be represented in the form of a Inline graphic payoff matrix as in Table 1. The payoffs garnered from these game interactions affect an individual's expected number of offspring by altering their propensity to have offspring (their fitness) or their survival. The expected number of offspring is determined by the fitness of the individuals and some population updating process, which will be made precise in the next section. The offspring produced during the population update have the potential to change the strategy composition of the population. An increase in the abundance of one strategy over a sufficiently large time scale indicates that strategy is favoured by evolution.

Table 1. The payoff matrix for a general Inline graphic by Inline graphic strategy game.

A B
A 1 S
B T 0

Here S and T are real numbers.

It can be shown, for replicator dynamics, for example [13], [14], that any payoff matrix can be reduced to the matrix in Table 1 without loss of generality because adding a constant term to the payoff matrix does not affect the dynamics and multiplying the payoffs by a positive factor merely rescales the time. Therefore we can always shift the payoffs such that Inline graphic-Inline graphic-encounters return a payoff of zero and scale all other payoffs such that Inline graphic-Inline graphic-encounters yield a payoff of Inline graphic. In the Averaged versus Accumulated Payoffs section, we show that the generality of the matrix in Table 1 extends to other forms of stochastic dynamics in finite populations based on the frequency dependent Moran process [15].

The (additive) prisoner's dilemma introduced before corresponds to the special case with Inline graphic and Inline graphic. Rescaling the payoff matrix in Table 1 by Inline graphic yields the traditional form, Table 2. More generally, the prisoner's dilemma requires Inline graphic and Inline graphic to result in the characteristic conflict of interest outline above. The special case of the additive prisoner's dilemma, Table 2, effectively reduces the game to a single parameter with Inline graphic (and Inline graphic). Moreover it has the special property that when an individual changes its strategy, the payoff gain (or loss) is the same, regardless of the opponents' strategy – the so-called equal-gains-from-switching property [16].

Table 2. The payoff matrix for an additive prisoner's dilemma game.

C D
C b−c c
D b 0

In the absence of structure, cooperators dwindle and disappear in the prisoner's dilemma. In contrast, structured populations enable cooperators to form clusters, which ensures that cooperators more frequently interact with other cooperators than they would with random interactions [17], [18]. Such assortment between cooperators is essential for the survival of cooperation [19].

In heterogeneous graphs not all vertices have the same number of connections and hence the fitnesses of individuals may be based on different numbers of interactions. Because of this, some vertices are more advantageous to occupy than others. However, which sites are favourable depends on the type of population dynamics. In particular, for the Moran process in structured populations it is important to distinguish between birth-death and death-birth updating [10], [20], [21], i.e. whether first an individual is randomly selected for reproduction with a probability proportional to its fitness and then the clonal offspring replaces a (uniformly) randomly selected neighbour – or, if first an individual is selected at random to die and then the vacant site is repopulated with the offspring of a neighbouring individual with a probability proportional to its fitness. Even in homogenous populations the sequence of events is of crucial importance but becomes even more pronounced in heterogenous structures [10], [20].

In order to illustrate that the population dynamics may bestow an advantage to individuals occupying certain sites in a heterogeneous population, consider neutral evolution, where game payoffs do not affect the evolutionary process and all individuals have the same fitness. For birth-death updating every individual is chosen to reproduce with the same probability but neighbours of individuals with few connections are replaced more frequently. Hence vertices with fewer neighbours are more favourable than those with many connections. Conversely, for death-birth updating every individual has the same expected life time but highly-connected individuals, or, hubs, get more frequently a chance to produce offspring, since one of their many neighbours dies, and are thus more favourable than vertices with few neighbours [21][23]. A simple example of this is a Inline graphic-line graph, one central vertex connected to two end vertices. In the birth-death process, the central vertex is replaced with probability Inline graphic, while either end vertex is replaced with probability Inline graphic, while in the death-birth process, the central vertex replaces either end vertex with probability Inline graphic and either end replaces the centre with probability Inline graphic [21]. The upshot is, even though the fitness of all individuals is the same, the effective number of offspring produced depends on the dynamics as well as an individual's location in the population.

The intrinsic advantage of some vertices over others can be further enhanced through game interactions leading to differences in fitness that depend on an individual's strategy as well as its position on the graph. For example, a cooperator occupying a favourable vertex can more easily establish a cluster of cooperators, which creates a positive feedback through mutual increases in fitness. Conversely, a favourable vertex also supports the formation of a cluster of defectors but this results in a negative feedback and lowers the fitness of the defector in the favourable vertex. The fact that heterogeneity can promote cooperation was first observed for the prisoner's dilemma and snowdrift games [24], [25] and has subsequently been confirmed for public goods games [26], [27]. However, the detailed effects not only crucially depend on the dynamics but also on how fitnesses are determined. For example, heterogenous population structures favour cooperation if payoffs from game interactions are accumulated but that advantage disappears if payoffs are averaged [28][30].

The effects of population structure on the outcome of evolutionary games is sensitive to a number of factors: population dynamics [10], [20], [31], translation of payoffs into fitness [28], [30], [32][35], the diversity of players [27], [34], [36], and the type of game played – for example, spatial structure tends to support cooperation in the prisoner's dilemma but conversely, in the snowdrift game, spatial structure may be detrimental [37]. Macroscopic features of the evolutionary process on the level of the population, such as frequency and distribution of cooperators, are determined by microscopic processes on the level of individuals. In the current article, we discuss some of these microscopic processes, such as averaging and accumulating payoffs, and the rate at which interactions take place, and illustrate how they affect an evolutionary outcome. Crucially, we also illustrate that the conditions for evolutionary advantage commonly found in the literature are not applicable to evolution in finite, heterogeneous populations. We modify these conditions and develop a general framework to determine evolutionary advantage in finite, heterogeneous populations.

The manuscript is organized as follows. Sections “Accumulated and Averaged Payoffs” and “Criteria for Evolutionary Success” create a critical synthesis of the existing literature concerning evolution on heterogeneous graphs. In these sections we extend existing results to general Inline graphic games and focus on an imitation process. We also discuss the inapplicability of approaches used in homogeneous populations and present our novel conditions for evolutionary success in heterogeneous populations. Interspersed in these sections are new observations and results that aid in establishing a consistent framework on which we base further novel results presented in the section “Stochastic Interactions and Updates”.

Results

Accumulated Versus Averaged Payoffs

In heterogenous population structures individuals naturally engage in different numbers of interactions. This renders comparisons of the performances of individuals more challenging. One natural approach is to simply accumulate the game payoffs. This clearly puts hubs with many neighbours in a strong position as scoring many times even a small payoff may still exceed few large payoffs. To avoid this bias in favour of hubs, game payoffs can be averaged. Interestingly, these two approaches not only play a decisive role for the evolutionary outcome but also entail important biological implications. In this section we extend previous work on payoff accounting [29] to general Inline graphic games and provide a thorough discussion of why different payoff accounting schemes can result in markedly different evolutionary outcomes.

Consider two different ways to translate the total, accumulated payoffs Inline graphic of an individual Inline graphic into its fitness Inline graphic:

graphic file with name pcbi.1003567.e039.jpg (1a)
graphic file with name pcbi.1003567.e040.jpg (1b)

where Inline graphic denotes the strength of selection and Inline graphic is the number of interactions experienced by Inline graphic. The limit Inline graphic recovers the neutral process, where selection does not act. Note that the payoff matrix in Table 1 can still be used without loss of generality because adding a constant Inline graphic merely changes the (arbitrary) baseline fitness from Inline graphic to Inline graphic and multiplying the payoffs by Inline graphic is identical to simply changing the selection strength to Inline graphic.

The exponential form of fitness in the above equations is mathematically convenient since it guarantees that the fitness is always positive, irrespective of the strength of selection and payoff values. It is worth noting that if the strength of selection is weak, that is, Inline graphic, then

graphic file with name pcbi.1003567.e051.jpg (2a)
graphic file with name pcbi.1003567.e052.jpg (2b)

which represents another common form for fitness found in the literature [8].

Homogenous populations

In the past, details of the payoff accounting have received limited attention, or the two approaches have been used interchangeably, because they yield essentially the same results for traditional models of spatial games, which focus on lattice populations [4], [38] or, more generally, on homogenous populations [8], [10], [39]. In fact, the difference in payoff accounting reduces to a change in the selection strength because in homogenous populations each individual has the same degree Inline graphic (number of neighbours) and hence, on average, the same number of interactions Inline graphic per unit time. If each individual interacts with all its neighbours then Inline graphic. Thus, the only difference is that the selection strength for accumulated payoffs is Inline graphic-times as strong as for averaged payoffs.

Therefore, in homogenous populations all individuals engage in the same number of interactions per unit time and consequently accumulating or averaging payoffs merely affects the strength of selection. Naturally, the converse question arises – are uniform interaction rates restricted to homogenous graphs? Or, more generally, which class of graphs supports uniform interaction rates?

To answer this question, let us consider an arbitrary graph Inline graphic with adjacency matrix Inline graphic where Inline graphic indicates the weight or the strength of the (directed) edge from vertex Inline graphic to Inline graphic. Inline graphic if vertex Inline graphic is connected to Inline graphic and Inline graphic if it is not. For example, the natural choice for the edge weights on undirected graphs is Inline graphic. That is, all Inline graphic edges leaving vertex Inline graphic have the same weight and hence Inline graphic for all Inline graphic.

An individual on vertex Inline graphic is selected to interact with vertex Inline graphic with a probability proportional to Inline graphic. In this case we say vertex Inline graphic has initiated the interaction. Interactions with self are excluded by requiring Inline graphic. If there are Inline graphic interactions per unit time, then the average number of interactions Inline graphic that vertex Inline graphic engages in is given by

graphic file with name pcbi.1003567.e079.jpg (3)

where the fraction indicates the probability that vertex Inline graphic participates in one particular interaction either by initiating it (first sum in numerator) or initiated by neighbours of Inline graphic (second sum in numerator). On average each individual engages in Inline graphic interactions. Note that the factor Inline graphic enters because each interaction affects two individuals. Therefore, a graph structure results in uniform interaction rates if and only if

graphic file with name pcbi.1003567.e084.jpg (4)

holds for every vertex Inline graphic, or equivalently, if Inline graphic for all Inline graphic where Inline graphic is an arbitrary positive constant.

If the sum of the weights of all Inline graphic edges leaving vertex Inline graphic, Inline graphic, is the same for all Inline graphic then Inline graphic and Eq. (4) requires that the sum of the weights of all incoming edges, Inline graphic, for all Inline graphic, as well to ensure uniform interaction rates. The class of graphs that satisfies the condition Inline graphic for all Inline graphic are called circulations [5] and, in the special case with Inline graphic, the adjacency matrix Inline graphic is doubly stochastic such that each row and column sums to Inline graphic. A more generic representative of the broad class of circulation graphs is shown in Figure 1 but this does not include heterogenous graphs such as scale-free networks.

Figure 1. A representative example of the broad class of circulation graphs.

Figure 1

Note that the weights of edges entering as well as those leaving any vertex all sum to Inline graphic.

In order to illustrate that the number of interactions experienced by an individual depends on which vertex they reside, let us consider a mean-field approximation for sufficiently large networks, based on the degree distribution Inline graphic and degree-degree correlations Inline graphic [40]. Specifically, Inline graphic denotes the probability of a randomly chosen vertex having degree Inline graphic, and Inline graphic denotes the conditional probability of a vertex Inline graphic with degree Inline graphic connected to vertices Inline graphic with degree Inline graphic. With this notation, the connectivity between a vertex of degree Inline graphic and another vertex of degree Inline graphic is Inline graphic. Further averaging this quantity over all vertices having degree Inline graphic, we obtain Inline graphic, which indicates the weight of the connection between a vertex of degree Inline graphic and another vertex of degree Inline graphic,

graphic file with name pcbi.1003567.e118.jpg (5)

The formula above omits higher-order correlations than two-point correlations, and works for large, sparse networks (Inline graphic and Inline graphic for all Inline graphic) [41]. In this case, Inline graphic can be interpreted as the probability that vertices Inline graphic and Inline graphic are connected at the mean-field level.

For random, undirected, and degree-uncorrelated graphs, Inline graphic does not depend on Inline graphic and is thus given by Inline graphic, where Inline graphic is the average degree of the network, Inline graphic. This applies even if the network is not sparse. Accordingly, Inline graphic can be simplified:

graphic file with name pcbi.1003567.e131.jpg (6)

Inserting into Eq. (4) yields

graphic file with name pcbi.1003567.e132.jpg (7)

Hence, the number of interactions of one vertex scales linearly with its degree.

Similarly, each vertex can initiate the same number of interactions, Inline graphic. Then, with probability Inline graphic the neighbouring vertex Inline graphic initiates an interaction with Inline graphic:

graphic file with name pcbi.1003567.e137.jpg (8)

Again, vertices with a degree greater (less) than the average degree are expected to have more (fewer) interactions than on average. Interaction rates on various heterogenous networks are shown in Figure 2. As shown in Figures 2a–c, this approximation above works well for a variety of networks where degree of adjacent vertices are uncorrelated. However, when the network is strongly degree-correlated, like the two-star graphs [27], [34], this approximation works poorly (also see Figure 2d for such an example of highly clustered scale free networks). In this case, we may use Eq.(5) to calculate Inline graphic as long as the function for degree-degree correlations, Inline graphic, is explicitly known.

Figure 2. Average number of interactions as a function of the degree of the vertex for different types of random heterogenous population structures:

Figure 2

(A) Erdós-Rényi random graphs [53], (B) Newman-Watts small-world networks [54]. (C) Barabási-Albert scale-free networks [42], and (D) Klemm-Eguiluz highly-clustered scale-free networks [55]. All graphs have size Inline graphic and an average degree of Inline graphic. At each time step a randomly chosen individual interacts with a randomly selected neighbour. The average number of interactions is shown for simulations (blue dots) and an analytical approximation for graphs where the degrees of adjacent vertices are uncorrelated (red line, see Eq. (8)).

This indicates that on undirected graphs uniform interaction rates can be achieved only on regular graphs, where all vertices have the same number of neighbours.

Heterogenous populations

In recent years the focus has shifted from homogenous populations to heterogenous structures and, in particular, to small-world or scale-free networks because they capture intriguing features of social networks [42]. On these structures the accounting of payoffs becomes important and, indeed, a crucial determinant of the evolutionary outcome. If payoffs are accumulated, heterogenous structures further promote the evolution of cooperation [24], [25], [27], [36]. In contrast, averaging the game payoffs can remove the ability for scale-free graphs to sustain higher levels of cooperation [28][30].

So far our discussion has focused on interactions between individuals and the translation of payoffs into fitness. The next step is to specify how differences in fitness affect the population dynamics. The most common updating rules in evolutionary games on graphs fall into three categories: Moran birth-death and death-birth, and imitation processes. The evolutionary outcome can be highly sensitive to the choice of update rule. For example, supposing weak selection, cooperation in the prisoner's dilemma may only thrive under death-birth but not under birth-death updating [8], [10], [20].

In heterogenous populations the range of payoffs depends on the payoff accounting: if payoffs are averaged, the range is determined by the maximum and minimum values in the payoff matrix but if payoffs are accumulated the range additionally depends on the size and structure of the population. In particular, this difference may also affect the updating rule: for example, the pairwise comparison process Inline graphic represents the probability that vertex Inline graphic adopts the strategy of vertex Inline graphic based on their fitnesses of Inline graphic, Inline graphic, respectively [43], [44]. This represents an imitation process where Inline graphic denotes a sufficiently large normalization constant to ensure that the expression indeed remains a probability. Since Inline graphic needs to be at least twice the range of possible fitness values, a generic choice of Inline graphic becomes impossible for accumulated payoffs.

Here we focus on a related imitation process where an individual Inline graphic is chosen at random to reassess its strategy by comparing its performance to a randomly chosen neighbour Inline graphic. Individual Inline graphic then imitates the strategy of Inline graphic with probability

graphic file with name pcbi.1003567.e154.jpg (9)

where Inline graphic and Inline graphic are the fitnesses of Inline graphic and Inline graphic. This variant is convenient as it includes an appropriate normalization factor and hence works regardless of how the fitnesses are calculated. In particular, for exponential payoff-to-fitness mapping (see Eq. (1)) the imitation rule, Eq. (9), recovers the Fermi-update [45]:

graphic file with name pcbi.1003567.e159.jpg (10a)
graphic file with name pcbi.1003567.e160.jpg (10b)

For a comparison between averaged and accumulated payoffs in homogenous and heterogenous populations, see Figure 3.

Figure 3. Average fraction of strategy Inline graphic for accumulated (top row) versus averaged (bottom row) payoffs in homogenous (left column) and heterogeneous (middle column) populations as well as the difference between them (right column) as a function of the game parameters Inline graphic and Inline graphic (see Table 1).

Figure 3

In each panel the four quadrants indicate the four basic types of generalized social dilemmas: prisoner's dilemma (upper left), snowdrift or co-existence games (upper right), stag hunt or coordination games (lower left) and harmony games (lower right). Homogenous populations are represented by Inline graphic lattices with von Neumann neighbourhood (degree Inline graphic) and heterogenous populations are represented by Barabási-Albert scale-free networks (size Inline graphic, average degree Inline graphic). The population is updated according to the imitation rule Eq. (9). The colours indicate the equilibrium fraction of strategy Inline graphic (left and middle columns) ranging from Inline graphic dominates (blue), equal proportions (green), to Inline graphic dominates (red). Increases in equilibrium fractions due to heterogeneity are shown in blue shades (right column) and decreases in shades of red. The intensity of the colour indicates the strength of the effect. Accumulated payoffs in heterogenous populations shift the equilibrium in support of the more efficient strategy Inline graphic except for harmony games where Inline graphic dominates in any case (bottom right quadrant). Conversely, for averaged payoffs the support of strategy Inline graphic is much weaker and even detrimental for Inline graphic. Parameters: initial configuration is a random distribution of equal proportions of strategies Inline graphic and Inline graphic; each simulation run follows Inline graphic updates and the equilibrium frequency of Inline graphic is averaged over the last Inline graphic updates; results are averaged over Inline graphic independent runs; for scale-free networks the network is regenerated every Inline graphic runs. No mutations occured during the simulation run.

On a microscopic level averaging or accumulating payoffs in heterogenous populations turns out to have important biological implications: when averaging payoffs, individuals play different games depending on their location on the graph, whereas for accumulated payoffs everyone plays the same game but at different rates – again based on the individuals' locations. These intriguing differences are illustrated and discussed for the simplest heterogenous structure, the star graph, in a subsequent section. First we develop a framework that aids in analyzing an evolutionary process in heterogeneous, graph-structured populations.

Criteria for Evolutionary Success

In order to determine the evolutionary success of a strategic type in a finite population we consider three fixation probabilities: Inline graphic and Inline graphic. The first, Inline graphic, indicates the probability that a single Inline graphic type in an otherwise Inline graphic population goes on to supplant all Inline graphics, while the second, Inline graphic, refers to the probability of the converse process where a single Inline graphic type takes over a population of Inline graphic types. These fixation probabilities are important whenever mutations can arise in the population during reproduction or through errors in imitating the strategies of others. The last probability, Inline graphic, denotes the fixation probability of the neutral process, which is defined as the dynamic in a population with vanishing selection, Inline graphic. In such a case the game payoffs do not matter and everyone has the same fitness. Based on these fixation probabilities two distinct and complementary criteria are traditionally used to measure evolutionary success [15], [20]:

  1. Type Inline graphic is said to have an evolutionary advantage or is favoured if
    graphic file with name pcbi.1003567.e194.jpg (11)
    holds. If mutations, or errors in imitation, are rare the mutant has disappeared or taken over the entire population before the next mutation occurs. We can then view the population dynamic as an embedded Markov chain transitioning between two states: all-Inline graphic and all-Inline graphic. Denote the proportion of time spent in the state all-Inline graphic (respectively, all-Inline graphic) by Inline graphic (resp. Inline graphic). Together, Inline graphic and Inline graphic are known as the stationary distribution of the Markov chain and satisfy the balance equation
    graphic file with name pcbi.1003567.e203.jpg (12)
    where Inline graphic (Inline graphic) is the probability an Inline graphic (Inline graphic) appears in the all-Inline graphic (all-Inline graphic) population. For homogeneous populations, or if mutations are not tied to reproduction or imitation events, Inline graphic and so Eq. (12) reads
    graphic file with name pcbi.1003567.e211.jpg (13)
    Hence, if Inline graphic then Inline graphic, which captures the notion of Inline graphic having an advantage over Inline graphic. If the inequality, Eq. (11), is reversed then type Inline graphic has the advantage.
  2. Type Inline graphic is a beneficial mutation if
    graphic file with name pcbi.1003567.e218.jpg (14a)
    holds. Similarly, if
    graphic file with name pcbi.1003567.e219.jpg (14b)
    holds, the Inline graphic type is a beneficial mutation. Note that, in general, Eqs. (14a) and (14b) are not mutually exclusive. Inline graphic and Inline graphic types may simultaneously be advantageous mutants – in co-existence games, Inline graphic, such as the snowdrift game – or both disadvantageous – in coordination games, Inline graphic, such as the stag-hunt game. However, for payoff matrices that satisfy equal-gains-from-switching, such as Table 2, Inline graphic implies Inline graphic and vice versa in unstructured populations or for weak selection [46].

The above conditions (11) and (14) are based on the implicit assumption of homogenous populations or averaged payoffs and randomly placed mutants. In the present context of heterogenous populations and with mutants explicitly arising through errors in reproduction or imitation, both conditions require further scrutiny and appropriate adjustments.

The first condition implicitly assumes that an Inline graphic mutant appears in a monomorphic Inline graphic population with the same probability as a Inline graphic mutant in a monomorphic Inline graphic population. However, in heterogenous populations with accumulated payoffs this is not necessarily the case. Even in monomorphic states hubs may have a higher fitness and hence are more readily imitated, or reproduce more frequently, than low degree vertices. This can result in a bias of the rates Inline graphic at which Inline graphic and Inline graphic mutants arise. Thus, the condition for evolutionary advantage, Eq. (11), must read

graphic file with name pcbi.1003567.e234.jpg (15)

In general, Inline graphic and Inline graphic depend on the population structure as well as the payoffs and their accounting. The star structure serves as an illustrative example in the next section.

Similarly, the second condition also needs to be made more explicit. In general, to determine whether a mutation is beneficial its fixation probability should exceed the probability that in the corresponding monomorphic population one particular individual eventually establishes as the common ancestor of the entire population. We denote these monomorphic fixation probabilities by Inline graphic, and Inline graphic, respectively. Thus, the second condition, Eq. (14), should be interpreted as

graphic file with name pcbi.1003567.e239.jpg (16a)
graphic file with name pcbi.1003567.e240.jpg (16b)

i.e. that the fixation probability of a single Inline graphic (or Inline graphic) mutant in a Inline graphic (Inline graphic) population exceeds that of one Inline graphic (Inline graphic) individual turning into the common ancestor of the entire population.

If mutations occur during an updating event, then in heterogeneous populations mutants occur more frequently in some vertices than in others. For our imitation process, high degree vertices serve more often as models than low degree vertices and hence the mutation is likely to occur in neighbours of high degree vertices. Note that this is different from placing a mutant on a vertex chosen uniformly at random from all vertices [47]. A randomly placed neutral mutant fixates, on average, with a probability corresponding to the inverse of the population size. This is not necessarily the case if neutral mutants arise in reproductive events or errors in imitating or adopting other strategies. In fact, the distinction between Inline graphic and Inline graphic is only required on heterogenous graphs with accumulated payoffs and non-random locations of mutants. In all other situations the (average) monomorphic fixation probabilities are the same and equal to Inline graphic, where Inline graphic is the population size.

In summary, due to the fitness differences in a monomorphic Inline graphic population with accumulated payoffs the turnover is accelerated and more strategy updates take place and hence more errors occur than in the corresponding monomorphic Inline graphic population. This means that, on average, mutant Inline graphics more frequently attempt to invade an Inline graphic population than vice versa. Overall, this leads to new conditions for evolutionary success in heterogeneous populations, summarized as follows. Type Inline graphic (i) has an evolutionary advantage or is favoured if Inline graphic where Inline graphic is the probability a mutant Inline graphic arises in an all-Inline graphic population (and vice-versa), and, is beneficial if Inline graphic, where Inline graphic is the probability a single Inline graphic individual goes on to become the common ancestor in an all-Inline graphic population. Analogous conditions hold for a mutant Inline graphic type. We apply these novel conditions to an example found in the literature [47], the star graph.

The star graph

The star graph represents the simplest, highly heterogenous structure. A star graph of size Inline graphic consist of a central vertex, the hub, which is connected to all Inline graphic leaf vertices. On the star graph the range of degrees is maximal – the hub has degree Inline graphic and all leaves have degree one.

In order to illustrate the differences arising from accumulating and averaging payoffs, consider a situation where each individual initiated, on average, one interaction. Thus, the hub has Inline graphic interactions while the leaves have only Inline graphic. Assume that Inline graphic vertices are of type Inline graphic and Inline graphic of type Inline graphic. The payoff to a hub of type Inline graphic is then Inline graphic for accumulated payoffs and Inline graphic if payoffs are averaged. In contrast, the payoff of an Inline graphic leaf is Inline graphic (accumulated) and Inline graphic (averaged). From each Inline graphic leaf the hub gains Inline graphic for accumulated payoffs, which is the same as the gain for the Inline graphic leaf. However, for averaged payoffs, the hub only gains Inline graphic from each Inline graphic leaf but each Inline graphic leaf still gains Inline graphic from the interaction with the hub. Thus, Inline graphic-Inline graphic-interactions are more profitable for vertices with a low degree and the payoff gets discounted for vertices with larger degrees. Although potential losses against Inline graphic leaves also get discounted: Inline graphic for Inline graphic leaves versus Inline graphic for an Inline graphic hub for averaged payoffs as opposed to Inline graphic for Inline graphic leaves versus Inline graphic for an Inline graphic hub for accumulated payoffs. For Inline graphic types it is less attractive to interact with Inline graphic types whenever Inline graphic and hence applies to all generalized social dilemmas [12].

Similarly, the payoffs to a type Inline graphic hub are Inline graphic (accumulated) and Inline graphic (averaged) versus Inline graphic for Inline graphic leaves (accumulated and averaged) or Inline graphic (accumulated) and Inline graphic (averaged) for Inline graphic leaves. In Inline graphic-Inline graphic-interactions both players get zero, regardless of the aggregation of payoffs, which is a consequence of our particular scaling of the payoff matrix in Table 1. Hence there is no discrimination between vertices of different degrees. An illustration of the differences arising from payoff accounting for the simpler and more intuitive case of the prisoner's dilemma in terms of costs and benefits (see Table 2), is given in Figure 4.

Figure 4. A star graph has the hub in the centre surrounded by Inline graphic leaf vertices.

Figure 4

Using the matrix in Table 2, an Inline graphic type individual (blue) on the hub provides a benefit Inline graphic to each leaf, regardless of whether the payoffs are a accumulated or b averaged. For each interaction, the costs to the hub amount to Inline graphic in the accumulated case whereas only Inline graphic in the averaged case. Conversely, the costs to a type Inline graphic leaf are always Inline graphic and it provides a benefit Inline graphic to the hub if payoffs are accumulated whereas only Inline graphic when averaged. Hence for averaged payoffs an Inline graphic type hub provides a benefit to each leaf at a fraction of the costs while Inline graphic type leaves provide a fraction of the benefits to the hub. This means that the leaves and the hub are playing different games. More specifically, the cost-to-benefit ratio of Inline graphic leaves is Inline graphic while it is Inline graphic for an Inline graphic hub. For most of the population (the leaves), this ratio is much larger than for accumulated payoffs where the cost-to-benefit ratio is Inline graphic. As a consequence cooperation is much more challenging if payoffs are averaged rather than accumulated.

In particular, on star graphs or, more generally, on scale-free networks, averaged payoffs result in higher and hence less favourable cost-to-benefit ratios for most individuals in the population, those with the lower degree vertices. Naturally these differences are also reflected in the evolutionary dynamics. We demonstrate this through the fixation probabilities of a single Inline graphic (Inline graphic) type in a population of Inline graphic (Inline graphic) types.

Let us first consider the fixation probability of a single Inline graphic type, Inline graphic. Because of the heterogenous population structure, Inline graphic depends on the location of the initial Inline graphic – for a star graph, whether the Inline graphic originated in the hub or one of the leaves. We denote the two fixation probabilities by Inline graphic and Inline graphic, respectively. With probability Inline graphic one of the leaves is chosen to update its strategy and the hub with probability Inline graphic. For averaged payoffs the fitnesses of everyone is the same in a monomorphic Inline graphic population and hence the hub is equally likely to adopt the strategy of a leaf, and make a mistake with probability Inline graphic, as are leaves that are adopting the hubs strategy. Hence the average fixation probability is given by

graphic file with name pcbi.1003567.e342.jpg (17)

In contrast, for accumulated payoffs even in a homogenous population the hub does not necessarily have the same payoffs as the leaves because of the larger number of interactions. However, for our payoff matrix in Table 1, this does not matter for homogenous Inline graphic populations as all Inline graphic-Inline graphic-interactions yield a payoff of zero. Consequently, Eq. (17) equally holds for averaged and accumulated payoffs and, incidentally, this is also the average fixation probability for a randomly placed Inline graphic mutant.

Similarly, we are interested in the average fixation probability, Inline graphic, of a single Inline graphic type in an otherwise homogenous Inline graphic population. Again we first need to determine with what probability the Inline graphic mutant arises in a leaf or in the hub. Interestingly, and in contrast to Inline graphic, this now depends on the accounting of payoffs. If payoffs are averaged then all individuals have the same payoff and, in analogy to Eq. (17), we obtain

graphic file with name pcbi.1003567.e352.jpg (18)

However, for accumulated payoffs, the hub achieves a payoff of Inline graphic as compared to an average payoff of merely Inline graphic for the leaves. In order to determine the average fixation probability of a single Inline graphic type, Inline graphic, we first consider the case where the mutant arises on a leaf. With probability Inline graphic a leaf is selected to update its strategy and adopts the hub's strategy with probability Inline graphic (c.f. Eq. (10a)). If the leaf adopts the strategy it makes an error with a small probability and instead of copying the Inline graphic strategy, the leaf becomes of type Inline graphic. Similarly, the hub reassesses its strategy with probability Inline graphic and switches to the leafs strategy with probability Inline graphic, which may then give rise to an Inline graphic type in the hub with a small probability. Based on these probabilities we can now determine the proportion of mutants that occur in the leaves and the hub, respectively. For the leaves we get

graphic file with name pcbi.1003567.e364.jpg

and similarly for the hub

graphic file with name pcbi.1003567.e365.jpg

Thus, the average fixation probability of a single Inline graphic mutant is

graphic file with name pcbi.1003567.e367.jpg (19)

In the weak selection limit, Inline graphic (or, more precisely, Inline graphic), Eq. (19) takes on the same form as for averaged payoffs, Eq. (18). Conversely, for large populations, Inline graphic, mutants almost surely arise in leaves and hence Inline graphic. Note that this is a good approximation as for Inline graphic and Inline graphic the probability that the mutant arises in the hub is already less than Inline graphic.

In order to determine the evolutionary advantage of Inline graphic and Inline graphic types we still need to determine the rates Inline graphic at which Inline graphic and Inline graphic mutants arise in monomorphic Inline graphic and Inline graphic populations, respectively. If payoffs are averaged all individuals in the population have the same fitness and hence with probability Inline graphic the focal individual imitates its neighbour (c.f. Eq. (10a)) and with a small probability Inline graphic an error (or mutation) occurs. This holds for monomorphic populations of either type and hence Inline graphic. For accumulated payoffs the same argument holds for monomorphic Inline graphic populations where all individuals have zero payoff. Consequently, Inline graphic mutants arise at a rate Inline graphic. In contrast, in a monomorphic Inline graphic population the hub has a much higher fitness and leaves will almost surely imitate the hub (whereas the hub almost surely will not imitate a leaf):

graphic file with name pcbi.1003567.e389.jpg (20)

For large Inline graphic every update essentially results in one of the leaves imitating the hub, so that Inline graphic.

Equations (17) through (19) yield the conditions under which type Inline graphic or Inline graphic has an evolutionary advantage. For star graphs, the fixation probabilities, Inline graphic and Inline graphic, can be derived based on the transition probabilities to increase or decrease the number of mutants by one and hence the results can be easily applied to any update rule [47]. For the imitation dynamics Inline graphic types are favoured under weak selection if and only if

graphic file with name pcbi.1003567.e397.jpg (21a)
graphic file with name pcbi.1003567.e398.jpg (21b)

and in the limit of infinite populations, Inline graphic, the conditions reduce to

graphic file with name pcbi.1003567.e400.jpg (21c)
graphic file with name pcbi.1003567.e401.jpg (21d)

A detailed derivation of the different fixation probabilities is provided in the Materials and Methods Section.

In order to determine whether a mutant is favoured or not (see Eq. (16)), we first need to determine the fixation probabilities Inline graphic and Inline graphic. Naturally, those fixation probabilities again depend on whether the ancestor is located in the hub or one of the leaves. Let us first consider a monomorphic Inline graphic population. The fixation probability of a Inline graphic located in the hub, Inline graphic, or in one particular leaf, Inline graphic, can be derived from the fixation probabilities Inline graphic and Inline graphic by setting Inline graphic (see Materials and Methods), which yields

graphic file with name pcbi.1003567.e411.jpg (22a)
graphic file with name pcbi.1003567.e412.jpg (22b)

Intuitively, the hub individual becomes the common ancestor with probability Inline graphic because any leaf individual updates its strategy to the hub's with a probability of Inline graphic and the hub keeps its strategy also with probability of Inline graphic but both probabilities are independent of the size of the population. Conversely, a leaf individual must first be imitated by the hub, which is Inline graphic times less likely than the reverse. On average we then obtain (insert into Eq. (17)):

graphic file with name pcbi.1003567.e417.jpg (23)

Note that in a monomorphic Inline graphic population the payoffs are zero regardless of the selection strength, Inline graphic, location (hub and leaves) or the payoff accounting. Again, this is a consequence of our particular choice of payoff matrix (Table 1), and thus, Eq. (23) holds for both averaged as well as accumulated payoffs and is, in fact, the same as the neutral fixation probability Inline graphic. Note that the fixation probabilities in (22a) through (23) corroborate the approximation results of [23] and the analytical results of [21].

Let us now turn to the monomorphic Inline graphic population and determine Inline graphic. If Inline graphic then everything is the same as in the monomorphic Inline graphic population above and Inline graphic. However, for any non-zero selection, Inline graphic, the situation becomes more interesting. If payoffs are averaged, all individuals have the same (non-zero) payoffs and a mutant is equally likely to appear in the hub as any particular leaf (c.f. Eq. (18)) and hence Inline graphic still holds. However, if payoffs are accumulated the hub has a higher fitness. The fixation probabilities that an Inline graphic on the hub or one of the leaves becomes the common ancestor are Inline graphic and Inline graphic (see Materials and Methods) and, on average we obtain

graphic file with name pcbi.1003567.e431.jpg (24)

Now we are able to derive the conditions under which an Inline graphic and/or Inline graphic mutant is beneficial, c.f. Eq. (16):

graphic file with name pcbi.1003567.e434.jpg (25a)
graphic file with name pcbi.1003567.e435.jpg (25b)

for averaged payoffs and, for accumulated payoffs,

graphic file with name pcbi.1003567.e436.jpg (25c)
graphic file with name pcbi.1003567.e437.jpg (25d)

The parameter region which delimits the region of evolutionary success of Inline graphic and Inline graphic types is illustrated in Figure 5.

Figure 5. Criteria for evolutionary success on the star graph for accumulated (left column) and averaged (right column) payoffs for weak selection, Inline graphic.

Figure 5

The range for which Inline graphic is advantageous (top row, c.f. Eq. (15)) depends on the population size, Inline graphic, and is shown in the limit Inline graphic (solid line) and for Inline graphic (dashed line). Below the respective lines Inline graphic is favoured. Similarly, the range for which Inline graphic and Inline graphic mutants are beneficial (c.f. Eq. (16) also depends on Inline graphic. Inline graphic mutants are beneficial above the red lines, while Inline graphic mutants are beneficial below the blue lines (solid for Inline graphic; dashed for Inline graphic). Additive games (or equal-gains-from-switching) satisfy Inline graphic (dotted line).

We can analyze Eqs. (21a)(21d) and (25a)(25d) in terms of the additive prisoner's dilemma game by substituting Inline graphic and Inline graphic. For simplicity, we restrict attention to the case Inline graphic and since in the additive prisoner's dilemma game a strategy is favoured if and only if it beneficial we need only consider Eqs. (21a)(21d). We have

graphic file with name pcbi.1003567.e457.jpg (26a)
graphic file with name pcbi.1003567.e458.jpg (26b)

If we suppose Inline graphic, then Eq. (26a) is never satisfied. That is, averaging rather than accumulating the payoffs altogether removes the ability of the star graph to support cooperation.

Note that for additive, or equal-gains-from-switching, games (games that satisfy Inline graphic) and for weak selection the condition Inline graphic implies both Inline graphic and Inline graphic, regardless of the accounting of payoffs. This extends results obtained for homogenous populations [8], [10].

Stochastic Interactions and Updates

As we have seen, when payoffs are averaged, members of a heterogeneous population are possibly playing different games, while if they are accumulated, all individuals play the same game. Therefore, only accumulating payoffs allows for meaningful comparisons of different heterogeneous population structures. A common simplifying assumption is that each individual interacts once with all its neighbours, see Figure 3. For heterogeneous populations this assumption means that those individuals residing on higher-degree vertices are interacting with their neighbours at a higher rate than those on lower-degree vertices. This leads to a separation of time scales, where interactions occur on a much faster time scale than strategy updates.

Realistically, all social interactions require a finite amount of time and hence the number of interactions per unit time is limited. This constraint already affects the evolutionary process in unstructured populations [48] but becomes particularly important in heterogenous networks where, for example, in scale-free networks some vertices entertain neighbourhood sizes that are orders of magnitude larger than that of other vertices. For those hubs it may not be possible to engage in interactions with all neighbours between subsequent updates of their strategy or the strategies of one of their neighbours. In order to investigate this we need to abandon the separation of the timescales for interactions and strategy updates.

A unified time scale on which interactions and strategy updates occur can be introduced as a stochastic process where a randomly chosen individual Inline graphic initiates an interaction with probability Inline graphic with a random neighbour Inline graphic and reassesses its strategy with probability Inline graphic by comparing its payoff to that of a random neighbour according to Eq. (9). Interactions alter the payoffs Inline graphic of both individuals (and hence their fitnesses, Inline graphic, see Eq. (1a)) according to the game matrix in Table 1. If individual Inline graphic adopts the strategy of its neighbour, then its payoff (and interaction count) is reset to zero, Inline graphic, regardless of whether the imitation had resulted in an actual change of strategy. Simulation results for various Inline graphic are shown in Figure 6.

Figure 6. Average fraction of strategy Inline graphic for different ratios between interactions and strategy updates in homogenous (top row) and heterogeneous (middle row) populations and the difference between them (bottom row) as a function of the game parameters Inline graphic and Inline graphic (c.f. Figure 3).

Figure 6

Interactions occur with probability Inline graphic and strategy updates with Inline graphic. For example, for Inline graphic each individual has, on average, initiated Inline graphic interactions between strategy updates but only an average of Inline graphic interactions for Inline graphic. For small Inline graphic effects of heterogenous population structures have little chance to manifest themselves and the results are closer to those for averaged payoffs (c.f. Figure 3). In contrast, for large Inline graphic heterogeneity plays an important role: for scale-free networks it is guided by the structural heterogeneity whereas in homogenous populations another form of heterogeneity spontaneously emerges in the number of interactions. Even on lattices, stochastic differences in the number of interactions get amplified by the dynamics because an increased number of interactions reduces the chances that an individual updates its strategy (c.f. Figure 7). As a consequence the results for lattices and scale-free networks become increasingly similar but scale-free networks keep promoting Inline graphic types to a greater extend. Parameters and averaging technique are as in the caption to Figure 3.

For small Inline graphic few interactions occur between strategy updates and in the limit Inline graphic neutral evolution is recovered because no interactions occur. Conversely, in the limit Inline graphic many interactions occur between strategy updates, which allows individuals to garner large payoffs as well as build up large payoff differences. The average number of interactions initiated by any individual between subsequent reassessments of the strategy is Inline graphic, the relative ratio of the time scales of game interactions versus strategy updates. However, the distribution of the number of interactions is biased: individuals with a large number of interactions tend to score high payoffs and hence are less likely to imitate a neighbours' strategy, which in turn results in a further increase of interactions. On heterogenous graphs and scale-free networks, in particular, this bias is built in by the underlying structure because highly connected hubs engage, on average, in a much larger number of interactions than vertices with few neighbours. Moreover, hubs are more likely to serve as models when neighbours are reassessing their strategy – simply because hubs have many neighbours. Thus, hubs are not only more resilient to change but also have a stronger influence on their neighbourhood. When Inline graphic this ratio begins to get large, interactions dominate strategy updates and the resulting game dynamics on heterogeneous and homogeneous graphs becomes indistinguishable.

Interestingly, a similar bias in interaction numbers spontaneously emerges on homogenous graphs, lattices in particular. Since all vertices have the same number of neighbours, no vertices are predisposed to achieve more interactions than others but some inequalities in interaction numbers occur simply based on stochastic fluctuations. As above, those vertices that happen to engage in more interactions tend to have higher payoffs and hence are less likely to imitate their neighbours and keep aggregating payoffs. This positive feedback between interaction count and resilience to change spontaneously introduces another form of heterogeneity, which becomes increasingly pronounced for larger Inline graphic. In fact, for large Inline graphic it rivals the structurally imposed heterogeneity of scale-free networks, see Figure 7.

Figure 7. Distributions of the number of interactions on lattices (black Inline graphic) and scale-free networks (blue Inline graphic) with a few interactions between updates (Inline graphic or, on average, Inline graphic interactions) and b many interactions between updates (Inline graphic or, on average, Inline graphic interactions).

Figure 7

For small Inline graphic the heterogeneity of scale-free networks results in a pronounced tail at higher numbers of interactions compared to the approximately exponential distribution for lattices. This tail is responsible for the reduction of cooperation in scale-free networks observed in Figure 6: as interactions dominate, some vertices almost never update their strategies. This “static network” emerges in both lattices and scale-free graphs and prevents the complete proliferation of the rare strategy. Nevertheless, most of the individuals in the population experience essentially the same number of interactions. The distributions look different for large Inline graphic but the main difference remains that scale-free networks produce a more pronounced tail. More importantly, however, for most of the population the distributions are actually very similar and hence the heterogeneities very similar. On lattices, the skewed distribution is caused by stochastic variations and the positive feedback between the number of interactions and the resilience to changing strategy.

Regardless of the structure, the positive feedback between payoff aggregation and the diminishing chances to change strategy (and hence reset payoffs) means that a small set of nodes forms an almost static backdrop of the dynamics and hence has a considerable effect on the evolutionary process. This set is a random selection on homogenous structures and consists of the hubs on heterogenous structures. As a consequence, the initial configuration of the population has long lasting effects on the abundance of strategies.

A more detailed view on the effects of Inline graphic on the evolutionary process is provided by restricting attention to the prisoner's dilemma and additive payoffs, c.f. Table 2. This can be accomplished by setting Inline graphic with Inline graphic. The equilibrium levels of cooperation in the Inline graphic-plane are shown in Figure 8 for lattices and scale-free networks.

Figure 8. Impact of the time scale relation Inline graphic on the equilibrium fractions of cooperators in the Inline graphic-plane for additive prisoner's dilemma games (Inline graphic): a lattices and b scale-free networks.

Figure 8

The limit Inline graphic recovers the neutral process (no interactions) whereas for Inline graphic individuals hardly update their strategies. Thus, in both of the two limiting cases the fraction of cooperators remains at the initial value of Inline graphic. For both types of population structures there exists an intermediate Inline graphic that leads to an optimal level of cooperation. On lattices the support for cooperation is strongest if interactions and strategy updates occur at equal rates, Inline graphic, but on scale-free networks more frequent updates than interactions are even more beneficial, Inline graphic. Parameters and averaging technique are as in the caption to Figure 3.

Altering the relative rates of interactions versus strategy updates has interesting effects on the evolutionary outcome. For lower rates of interaction (Inline graphic), scale-free networks outperform lattices in their ability to promote cooperation. As interaction rates increase and strategy updates become more rare (Inline graphic), scale-free networks and lattices become virtually indistinguishable in their ability to support cooperation. For both lattices and scale-free networks an optimal ratio between strategy updates and interactions exist: for lattices this is roughly Inline graphic, suggesting that lattices support the greatest amount of cooperators when interactions occur at the same rate as strategy updates, whereas for scale-free networks the optimum lies around Inline graphic, which suggests that scale-free networks provide the strongest support for cooperation if there are roughly three updates per interaction.

Discussion

Evolutionary dynamics in heterogenous populations, scale-free networks in particular, have attracted considerable attention over recent years. Somewhat surprisingly, the underlying microscopic processes and their implications for the macroscopic dynamics and the corresponding biological interpretations have received little attention.

Here we have shown that established criteria to measure success in evolutionary processes make different kinds of implicit assumptions that do not hold in general for heterogenous structures. Instead, for such structures it becomes imperative to reconsider, revise and generalize these criteria, which was done in the Criteria for Evolutionary Success section. If errors arise in imitating the strategic type of other individuals, or mutations occur during reproduction, then mutations are more likely to arise in some locations than in others. For example, on the star graph mutants likely occur in the leaf nodes for birth-death updating and imitation processes but in the hub for death-birth processes. Moreover, in heterogenous populations the fixation probabilities generally depend on the initial location of the mutant and hence even the fixation probability of a neutral mutant may no longer simply be the reciprocal of the population size but rather intricately depend on the population structure.

Another crucial determinant of the evolutionary dynamics in heterogenous populations is the aggregation of payoffs from interactions between individuals. Individuals on vertices with a higher (lower) degree expect to have more (fewer) interactions than on average. Even though the choice between averaging or accumulating payoffs may seem innocuous, it has far reaching consequences. Previous authors [29] have found that averaging payoffs in a prisoner's dilemma game on a scale-free network eliminates such a network's ability to promote cooperation as observed in earlier studies [24], [25], [27], [49]. We have extended this result to general Inline graphic games and provide a detailed rationale for this phenomenon which is summarized as follows. If payoffs are accumulated, some individuals are capable of accruing more payoffs than others strictly by virtue of them having more potential partners. Averaging payoffs removes the ability of hubs to accrue greater payoffs, but simultaneously makes it difficult to compare results for different population structures (e.g. lattices versus scale-free networks) even if their average degrees are the same because the type of game played depends on the location in the graph. Hence, accumulating payoffs seems a more natural choice to compare evolutionary outcomes based on different population structures because it ensures that everyone engages in the same game. However, if we assume all interactions are realised then those individuals with more neighbours interact at a much greater rate than those with less.

In order to investigate the disparity in the number of interactions on the success of strategies on heterogenous graphs we introduced the time-scale parameter Inline graphic, which determines the probability that an interaction or a strategy update occurs. When increasing the rate of strategy updates (small Inline graphic), heterogeneous graphs are able to support higher levels of cooperation than lattices. Conversely, increasing the rate of interactions (large Inline graphic) results in small differences between lattices and scale-free networks; both support roughly the same levels of cooperation. For imitation processes, individuals with high payoffs are unlikely to change their strategies and hence are likely to keep accumulating more payoffs. On scale-free networks, hubs are predestined to become such high performing individuals but on lattices they spontaneously emerge, triggered by stochastic fluctuation in the interaction count and driven by the positive feedback between increasing payoffs and increasing resilience to changing strategies (and hence to resetting payoffs).

For intermediate Inline graphic an optimum increase in the level of cooperation is found: lattices support cooperation most efficiently if a balance is struck between interactions and strategy updates (Inline graphic), whereas scale-free networks work most efficiently if slightly more updates occur (Inline graphic). For lattices a related observation was reported for noise in the updating process [50]. If the noise is large, updating is random but if it is small the game payoffs become essential. Interestingly, cooperation is most abundant for intermediate levels of noise – which is similar to having some but not too many interactions between strategy updates.

Previous work has found that heterogeneous graphs support coordination of strategies, where all individuals are inclined to adopt the same strategy, while homogeneous graphs support co-existence [51], [52]. The time scale parameter Inline graphic introduced in the Stochastic Interactions and Updates section seems to aid in promoting coexistence in both types of graphs, based on the large green region in Figures 3, 6, and 8. Exactly how the time scale parameter Inline graphic promotes coexistence is a topic worthy of further investigation.

Naturally there is no correct way of modelling the updating of the population or the aggregation of payoffs but, as so often, the devil is in the detail and implicit assumptions originating in traditional, homogenous models may be misleading or have unexpected consequences in more general, heterogenous populations.

Materials and Methods

In [47], the authors calculate expressions for the probability that a single mutant fixes on a star graph. These expressions are in terms of state transition probabilities. Denote by Inline graphic the transition probability from a state with Inline graphic Inline graphic individuals on the leaves and an Inline graphic individual on the hub to a state with Inline graphic Inline graphic individuals on the leaves and a Inline graphic on the hub. With this notation, the fixation probability of a single Inline graphic on a leaf vertex is

graphic file with name pcbi.1003567.e534.jpg (27)

and for a single Inline graphic on the hub,

graphic file with name pcbi.1003567.e536.jpg (28)

where, in both cases,

graphic file with name pcbi.1003567.e537.jpg (29)

For the imitation process defined by Eq. 9 and accumulated payoffs we have

graphic file with name pcbi.1003567.e538.jpg (30a)
graphic file with name pcbi.1003567.e539.jpg (30b)
graphic file with name pcbi.1003567.e540.jpg (30c)
graphic file with name pcbi.1003567.e541.jpg (30d)

and for averaged payoffs,

graphic file with name pcbi.1003567.e542.jpg (31a)
graphic file with name pcbi.1003567.e543.jpg (31b)
graphic file with name pcbi.1003567.e544.jpg (31c)
graphic file with name pcbi.1003567.e545.jpg (31d)

These are incorporated into the Eqs. (27) and (28) to yield the fixation probabilities Inline graphic and Inline graphic. The fixation probabilities Inline graphic and Inline graphic are obtained in a similar way. The averages Inline graphic and Inline graphic are then calculated using Eqs. (17), (18), and (19). Finally, a first-order approximation in Inline graphic is found for the above. For example,

graphic file with name pcbi.1003567.e553.jpg (32a)

The other fixation probabilities are found in a similar way:

graphic file with name pcbi.1003567.e554.jpg (32b)
graphic file with name pcbi.1003567.e555.jpg (32c)
graphic file with name pcbi.1003567.e556.jpg (32d)

Assuming Inline graphic, and employing the appropriate condition for evolutionary advantage, yields Eqs. (25a25d) in the main text.

Funding Statement

WM and CH are funded by NSERC. FF is funded by ETH Zurich. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1. Wright S (1931) Evolution in Mendelian populations. Genetics 16: 97–159. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Kimura M, Weiss G (1964) The stepping stone model of population structure and the decrease of genetic correlation with distance. Genetics 49: 561–575. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Levins R (1969) Some demographic and genetic consequences of environmental heterogeneity for biological control. Bulletin of the Entomological Society of America 15: 237–240. [Google Scholar]
  • 4. Nowak MA, May RM (1992) Evolutionary games and spatial chaos. Nature 359: 826–829. [Google Scholar]
  • 5. Lieberman E, Hauert C, Nowak MA (2005) Evolutionary dynamics on graphs. Nature 433: 312–316. [DOI] [PubMed] [Google Scholar]
  • 6. Frucht R (1949) Graphs of degree three with a given abstract group. Canadian Journal of Mathematics 1: 365–378. [Google Scholar]
  • 7. Ohtsuki H, Nowak MA (2006) Evolutionary games on cycles. Proceedings of the Royal Society B 273: 2249–2256. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Taylor PD, Day T, Wild G (2007) Evolution of cooperation in a finite homogeneous graph. Nature 447: 469–472. [DOI] [PubMed] [Google Scholar]
  • 9. Grafen A, Archetti M (2008) Natural selection of altruism in inelastic viscous homogeneous populations. Journal of Theoretical Biology 252: 694–710. [DOI] [PubMed] [Google Scholar]
  • 10. Ohtsuki H, Hauert C, Lieberman E, Nowak MA (2006) A simple rule for the evolution of cooperation on graphs. Nature 441: 502–505. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Dawes RM (1980) Social dilemmas. Annual Review of Psychology 31: 169–193. [Google Scholar]
  • 12. Hauert C, Michor F, Nowak MA, Doebeli M (2006) Synergy and discounting of cooperation in social dilemmas. Journal of Theoretical Biology 239: 195–202. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. Taylor PD, Jonker L (1978) Evolutionary stable strategies and game dynamics. Mathematical Biosciences 40: 145–156. [Google Scholar]
  • 14.Hofbauer J, Sigmund K (1998) Evolutionary Games and Population Dynamics. Cambridge University Press, Cambridge. [Google Scholar]
  • 15. Nowak MA, Sasaki A, Taylor C, Fudenberg D (2004) Emergence of cooperation and evolutionary stability in finite populations. Nature 428: 646–650. [DOI] [PubMed] [Google Scholar]
  • 16. Nowak MA, Sigmund K (1990) The evolution of stochastic strategies in the prisoner's dilemma. Acta Applicandae Mathematicae 20: 247–265. [Google Scholar]
  • 17. Van Baalen M, Rand DA (1998) The unit of selection in viscous populations and the evolution of altruism. Journal of Theoretical Biology 193: 631–648. [DOI] [PubMed] [Google Scholar]
  • 18. Hauert C (2001) Fundamental clusters in spatial 2×2 games. Proceedings of the Royal Society B 268: 761–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Fletcher JA, Doebeli M (2009) A simple and general explanation for the evolution of altruism. Proceedings of the Royal Society B 276: 13–19. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Zukewich J, Kurella V, Doebeli M, Hauert C (2013) Consolidating birth-death and death-birth processes in structured populations. PLoS One 8: e54639. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Maciejewski W (2014) Reproductive value in graph-structured populations. Journal of Theoretical Biology 340: 285–293. [DOI] [PubMed] [Google Scholar]
  • 22. Broom M, Rychtar J, Stadler B (2011) Evolutionary dynamics on graphs - the effect of graph structure and initial placement on mutant spread. Journal of Statistical Theory and Practice 5: 369–381. [Google Scholar]
  • 23. Li C, Zhang B, Cressman R, Tao Y (2013) Evolution of cooperation in a heterogeneous graph: Fixation probabilities under weak selection. PLoS One 8: e66560. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Santos FC, Pacheco JM (2005) Scale-free networks provide a unifying framework for the emergence of cooperation. Physical Review Letters 95: 098104. [DOI] [PubMed] [Google Scholar]
  • 25. Santos FC, Rodrigues JF, Pacheco JM (2006) Graph topology plays a determinant role in the evolution of cooperation. Proceedings of the Royal Society B 273: 51–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Santos FC, Pacheco JM, Lenaerts T (2006) Cooperation prevails when individuals adjust their social ties. PLoS Computational Biology 2: 1284–1291. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Santos FC, Santos MD, Pacheco JM (2008) Social diversity promotes the emergence of coope ration in public goods games. Nature 454: 213–216. [DOI] [PubMed] [Google Scholar]
  • 28. Tomassini M, Pestelacci E, Luthi L (2007) Social dilemmas and cooperation in complex networks. International Journal of Modern Physics C 18: 1173–1185. [Google Scholar]
  • 29. Szolnoki A, Perc M, Danku Z (2008) Towards effective payoffs in the prisoner's dilemma game on scale-free networks. Physica A 387: 2075–2082. [Google Scholar]
  • 30. Antonioni A, Tomassini M (2012) Cooperation on social networks and its robustness. Advances in Complex Systems 15: 1250046. [Google Scholar]
  • 31. Huberman BA, Glance NS (1993) Evolutionary games and computer simulations. Proceedings of the National Academy of Sciences USA 90: 7716–7718. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Masuda N (2007) Participation costs dismiss the advantage of heterogeneous networks in evolution of cooperation. Proceedings of the Royal Society B 274: 1815–1821. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Perc M, Szolnoki A (2008) Social diversity and promotion of cooperation in the spatial prisoner's dilemma game. Physical Review E 77: 0011904. [DOI] [PubMed] [Google Scholar]
  • 34. Pacheco J, Pinheiro FL, Santos FC (2009) Population structure induces a symmetry breaking favouring the emergence of cooperation. PLoS Computational Biology 5: e1000596. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Grilo C, Correia L (2011) Effects of asynchronism on evolutionary games. Journal of Theoretical Biology 269: 109–122. [DOI] [PubMed] [Google Scholar]
  • 36. Santos FC, Pinheiro FL, Lenaerts T, Pacheco JM (2012) The role of diversity in the evolution of cooperation. Journal of Theoretical Biology 299: 88–96. [DOI] [PubMed] [Google Scholar]
  • 37. Hauert C, Doebeli M (2004) Spatial structure often inhibits the evolution of cooperation in the snowdrift game. Nature 428: 643–646. [DOI] [PubMed] [Google Scholar]
  • 38. Hauert C (2002) Effects of space in 2×2 games. International Journal of Bifurcation and Chaos 12: 1531–1548. [Google Scholar]
  • 39. Szabó G, Fáth G (2007) Evolutionary games on graphs. Physics Reports 446: 97–216. [Google Scholar]
  • 40. Boguná M, Pastor-Satorras R (2002) Epidemic spreading in correlated complex networks. Physical Review E 66: 047104. [DOI] [PubMed] [Google Scholar]
  • 41.Bollobas B (1985) Random Graphs. Cambridge Studies in 628 Advanced Mathematics.
  • 42. Barabási A, Albert R (1999) Emergence of scaling in random networks. Science 286: 509–512. [DOI] [PubMed] [Google Scholar]
  • 43. Traulsen A, Claussen JC, Hauert C (2005) Coevolutionary dynamics: From finite to infinite populations. Physical Review Letters 95: 238701. [DOI] [PubMed] [Google Scholar]
  • 44. Traulsen A, Claussen JC, Hauert C (2012) Stochastic differential equations for evolutionary dynamics with demographic noise and mutations. Physical Review E 85: 041901. [DOI] [PubMed] [Google Scholar]
  • 45. Szabó G, Tőke C (1998) Evolutionary Prisoner's Dilemma game on a square lattice. Physical Review E 58: 69–73. [Google Scholar]
  • 46. Taylor PD, Day T, Wild G (2007) From inclusive fitness to fixation probability in homogeneous structured populations. Journal of Theoretical Biology 249: 101–110. [DOI] [PubMed] [Google Scholar]
  • 47. Hadjichrysanthou C, Broom M, Rychtář J (2011) Evolutionary games on star graphs under various updating rules. Dynamic Games and Applications 1: 386–407. [Google Scholar]
  • 48. Woelfing B, Traulsen A (2009) Stochastic sampling of interaction partners versus deterministic payoff assignment. Journal of Theoretical Biology 257: 689–695. [DOI] [PubMed] [Google Scholar]
  • 49. Santos FC, Pacheco JM, Lenaerts T (2006) Evolutionary dynamics of social dilemmas in structured heterogeneous populations. Proceedings of the National Academy of Sciences USA 103: 3490–3494. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50. Szabó G, Vukov J, Szolnoki A (2005) Phase diagrams for an evolutionary prisoner's dilemma game on two-dimensional lattices. Physical Review E 72: 047107. [DOI] [PubMed] [Google Scholar]
  • 51. Pinheiro F, Pacheco JM, Santos F (2012) From local to global dilemmas in social networks. PLoS One 7 2: e32114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52. Pinheiro FL, Santos FC, Pacheco JM (2012) How selection pressure changes the nature of social dilemmas in structured populations. New Journal of Physics 14: 073035. [Google Scholar]
  • 53. Erdős P, Rényi A (1960) On the evolution of random graphs. Publ Math Inst Hung Acad Sci 5: 17–61. [Google Scholar]
  • 54. Newman MEJ, Watts DJ (1999) Scaling and percolation in the small-world network model. Physical Review E 60: 7332. [DOI] [PubMed] [Google Scholar]
  • 55. Klemm K, Eguiluz VM (2002) Highly clustered scale-free networks. Physical Review E 65: 036123. [DOI] [PubMed] [Google Scholar]

Articles from PLoS Computational Biology are provided here courtesy of PLOS

RESOURCES