The effect of network topology on optimal exploration strategies and the evolution of cooperation in a mobile population

Abstract

We model a mobile population interacting over an underlying spatial structure using a Markov movement model. Interactions take the form of public goods games, and can feature an arbitrary group size. Individuals choose strategically to remain at their current location or to move to a neighbouring location, depending upon their exploration strategy and the current composition of their group. This builds upon previous work where the underlying structure was a complete graph (i.e. there was effectively no structure). Here, we consider alternative network structures and a wider variety of, mainly larger, populations. Previously, we had found when cooperation could evolve, depending upon the values of a range of population parameters. In our current work, we see that the complete graph considered before promotes stability, with populations of cooperators or defectors being relatively hard to replace. By contrast, the star graph promotes instability, and often neither type of population can resist replacement. We discuss potential reasons for this in terms of network topology.

1. Introduction

Early works [1,2] on evolutionary processes considered well-mixed populations of finite size, but did not include interactions between individuals (i.e. individuals were assumed to have fixed fitness). These models were later adapted to include interactions between individuals using evolutionary game theory [3,4], but only for well-mixed populations of infinite size, where individuals were assumed to interact in a pairwise fashion. Game-theoretical models of important features of animal populations were developed, such as the Hawk–Dove game [5] and the sex ratio game [6]. The evolutionary game theory was next extended to well-mixed and structured finite populations [7, ch. 6–9].

There are different ways to consider population structure; one important approach that has proved popular is the meta-population [8]. Here, individuals are divided into groups where interactions with group members are more likely than with non-members; there are different examples of this methodology: in the community-structured population described in [9] interactions take place at multiple levels depending upon community membership, while in the island model considered in [10] communities are largely isolated from one another but are connected through a low rate of migration between them.

The modelling framework of [11], evolutionary graph theory, has proved very popular as it provides a natural and simple way to include complex population structure in finite populations. Here, graph vertices represent individuals and pairwise interactions take place between connected individuals only. Many papers have been written within this framework; see for example [12–17] and [18,19] for reviews of the earlier work.

A limitation of this approach is that interactions still take place in a pairwise fashion, because they result from two-player games. Yet this is not necessarily the case in real populations [20,21], and there are examples of multi-player games used in the literature to model such situations [22–25]. One possible solution to this issue is to use a separate ‘interaction’ graph as done in [26], allowing for more complex behaviours. However, with this approach there is no obvious connection of the interaction graph to the evolutionary graph because the interaction graph does not necessarily naturally arise from the evolutionary process. An alternative approach [27–31] generates multi-player games by grouping individuals with their neighbours, but this generates a limited range of specific games between particular individuals.

A flexible approach was developed in the framework of [32], which maintains many useful features of evolutionary graph theory but allows multi-player interactions between groups of any size, including lone individuals, which could depend upon various factors like the population's history; see also [33,34]. Different types of models can be considered within this framework; for instance, a subdivided population based on a network, similar to the meta-population concept described above, was considered in [35].

In [36], we applied the framework for the first time to a model where the individuals moved around the network following the Markov property. The semi-analytic approach of [36] limited the analysis to the complete graph (effectively an unstructured population) and small population size (10 and 20). It turned out that lower movement costs and longer exploration times promoted cooperation, while higher movement costs and shorter exploration times inhibited cooperation.

In the current paper, we extend this model to consider two other network structures (circle and star graphs) and larger population sizes (up to 50). Our research was motivated by this question: to what extent does network topology affect the outcome of the evolution of cooperation on multi-player evolving networks? We find that it plays little role in the stability of the defector population, but it plays a crucial role in the stability of the cooperator population. We hypothesize that the clustering coefficient and degree centralization of a network are two of the network topology characteristics that are responsible for the outcomes we observed. The clustering coefficient measures how nodes of a network cluster together (are two neighbours of the same node also neighbours of each other?), and the degree centralization measures the heterogeneity of the distribution of the degrees of the nodes (it is equal to 0 when all nodes of a network have the same degree). We also propose a roadmap for further investigation of this phenomenon. An interested reader may benefit by consulting several recent reviews on the role of network topology in the evolution of cooperation [37,38].

2. The model

Here, we provide a brief overview of the framework of Broom & Rychtář [32]. We consider only sufficient detail to understand the current paper, and we use terminology as in [36], which was slightly different from [32] because it is more natural for the Markov models considered here. The model is broken down into four key components: population structure, movement, fitness and evolutionary dynamics.

(a). The population: structure and distribution

We consider a population of N individuals who can move around M places, which are connected by some underlying structure. In this paper, N = M and each individual has a (unique) home place that it can return to. The structure is described by a network such that each node represents a place and links represent connections between places.

We consider three interaction network structures: the complete, circle and star graphs. Our motivation for choosing these three networks was twofold. First, they possess a high degree of symmetry, which allows us to compute the replacement weights (see §2d) analytically. Second, they represent extreme cases of two network topology characteristics we consider: the clustering coefficient [39] and degree centralization [40] (table 1). The clustering coefficient measures how nodes of a network cluster together, and it is computed as the number of closed triplets divided by the number of all triplets. In other words, it corresponds to the probability that two neighbours of a node are also neighbours of each other (thus forming a triangle, or a closed triplet). The degree centralization measures the heterogeneity of the distribution of the degrees of the nodes of a network (the degree of a node is the number of connections it has to other nodes in the network). It is equal to 0 if all nodes have the same degree, and it takes the maximum value of 1 for the star graph, which has a hub-and-spoke topology with one highly connected node (the centre of the star) and many nodes of degree 1 (leaves of the star).

Table 1.

Network topology characteristics for the complete, circle and star graphs.

	complete	circle	star
clustering coefficient	1	0	0
degree centralization	0	0	1

The population distribution at time t is denoted by the matrix M_t = [M_n,t]_n=1,…,N, where M_n,t = m if individual I_n is in place P_m at time t. Movement is probabilistic, dependent upon the current location of individuals in the population. The transition probability function is $p(\mathbf{m}\mid {\mathbf{m}}_{t-1})=\mathbb{P}({\mathbf{M}}_t=\mathbf{m}\mid {\mathbf{M}}_{t-1}={\mathbf{m}}_{t-1})$.

The individual transition probability function p_n,t(m_n|m_t−1) gives the probability that the individual I_n moves to place m_n at time t given the current population distribution m_t−1, i.e. $p_{n,t}(m_n\mid {\mathbf{m}}_{t-1})=\mathbb{P}(M_{n,t}=m_n\mid {\mathbf{M}}_{t-1}={\mathbf{m}}_{t-1}).$ We then have

$$p(\mathbf{m}\mid {\mathbf{m}}_{t-1})=\prod _n{p}_{n,t}(m_n\mid {\mathbf{m}}_{t-1}).$$ 2.1

Following [36], we now consider the underlying population structure and movement as separate components.

(b). Individual movement

The set of individuals that are present with the individual I_n at the same location at time t − 1 is denoted ${\mathcal{G}}_n({\mathbf{m}}_{t-1})$ (i.e. ${\mathcal{G}}_n({\mathbf{m}}_{t-1})=\{i:m_{i,t-1}=m_{n,t-1}\}$). We call this set the group of the individual I_n. As in [36], the individual transition probabilities are time homogeneous but dependent upon the previous group and previous position of the individuals. We have the following transition probabilities:

$$p_{n,t}(m\mid m_{n,t-1},{\mathcal{G}}_n({\mathbf{m}}_{t-1}))=\{\begin{array}{ll}{h}_n({\mathcal{G}}_n({\mathbf{m}}_{t-1}))& m=m_{n,t-1}\\ \frac{1-h_n({\mathcal{G}}_n({\mathbf{m}}_{t-1}))}{d(m_{n,t-1})}& m\ne m_{n,t-1},\end{array}$$ 2.2

where $h_n({\mathcal{G}}_n({\mathbf{m}}_{t-1}))$ is the staying probability of individual I_n, d(m_n,t−1) is the degree of the node at which I_n was at time t − 1, and the place P_m is assumed to be directly connected to P_mn,t−1 in the underlying structure. In other words, if an individual decides not to stay at the current place, then it moves to one of the adjacent locations randomly with equal probability.

The staying probability $h_n({\mathcal{G}}_n({\mathbf{m}}_{t-1}))$, the probability that the individual stays at the current place, depends upon the staying propensity α_n of I_n and the attractiveness of the group ${\mathcal{G}}_n({\mathbf{m}}_{t-1})$ (as we will see in the following subsection, the payoff to an individual depends upon this group composition). The parameter α_n is an intrinsic measure, irrespective of group composition, of how much I_n wants to remain where it is; in particular, $h_n({\mathcal{G}}_n({\mathbf{m}}_{t-1}))={\alpha }_n$ when I_n is alone (i.e. when ${\mathcal{G}}_n({\mathbf{m}}_{t-1})=\{n\}$). In our model, the staying propensity is a strategy (the exploration strategy) of an individual; it is one of the two characteristics that constitutes an individual's type.

The other characteristic of an individual is its interactive strategy, cooperate (C) or defect (D), for the multi-player game. The attractiveness β_i of a group member I_i to others depends upon its interactive strategy, and it is defined as

$${\beta }_i=\{\begin{array}{ll}{\beta }_C& \mathrm{if}\hspace{0.17em}{I}_i\hspace{0.17em}\text{is a cooperator}\\ {\beta }_D& \mathrm{if}\hspace{0.17em}{I}_i\hspace{0.17em}\text{is a defector},\end{array}$$ 2.3

where β_C and β_D correspond to the attractiveness of being with a cooperator and defector, respectively. The attractiveness of the group ${\mathcal{G}}_n({\mathbf{m}}_{t-1})$ to the individual I_n is then equal to

$${\beta }_{{\mathcal{G}}_n({\mathbf{m}}_{t-1})\setminus \{n\}}=\sum _{i\in {\mathcal{G}}_n({\mathbf{m}}_{t-1})\setminus \{n\}}{\beta }_i.$$ 2.4

An individual's staying probability is a combination of its staying propensity and the attractiveness of its group. As in [36], we use the following sigmoid function:

$$h_n({\mathcal{G}}_n({\mathbf{m}}_{t-1}))=\frac{{\alpha }_n}{{\alpha }_n+(1-{\alpha }_n)S^{{\beta }_{{\mathcal{G}}_n({\mathbf{m}}_{t-1})\setminus \{n\}}}},$$ 2.5

where 0 < S < 1 is a sensitivity parameter. The smaller the value of S, the more sensitive individuals are to the group composition, i.e. the more likely they are to react to the composition of their group. In this paper, we assume that individuals are highly sensitive to the group composition (S = 0.03); this is the same value of the sensitivity parameter that was used in [36]. Figure 1 shows how the staying probability of an individual depends on the individual's staying propensity and the group attractiveness when S = 0.03.

Figure 1.

The staying probability as a function of the group attractiveness for various values of the individual's staying propensity. Dashed line, staying propensity 0.01; dotted line, staying propensity 0.5; solid line, staying propensity 0.99. Solid circles mark the data points corresponding to integer values of the group attractiveness, as in our model.

(c). Fitness

For our version of the model, the fitness of an individual at the point of reproduction depends upon a sequence of fitness contributions accumulated through time. Each contribution depends upon the current group that an individual finds itself in together with the place that it moved from. The fitness contribution of I_n at time t is denoted

$$f_{n,t}({\mathbf{m}}_{\mathbf{t}}\mid {\mathbf{m}}_{\mathbf{t}-\mathbf{1}})=f_{n,t}(m,{\mathcal{G}}_n({\mathbf{m}}_{\mathbf{t}})\mid m_{n,t-1}).$$ 2.6

The change in fitness of an individual above will depend upon direct group interactions and whether a movement has been made, since movements incur a cost.

The multi-player game we consider is a public goods game where the payoffs are determined by two interactive strategies: cooperate or defect. All individuals receive a base reward of 1 regardless of their interactive strategy. A cooperator always pays a cost 0≤c < 1, and every individual within its group receives an equal share of a reward v > 0. The direct group interaction payoffs are thus:

$$R_{n,t}({\mathcal{G}}_n({\mathbf{m}}_t))=\{\begin{array}{ll}1+\frac{|{\mathcal{G}}_n({\mathbf{m}}_t){|}_C-1}{|{\mathcal{G}}_n({\mathbf{m}}_t)|-1}v-c& \mathrm{if}\hspace{0.17em}{I}_n\hspace{0.17em}\text{is a cooperator and }\hspace{0.17em}|{\mathcal{G}}_n({\mathbf{m}}_t)|>1,\\ 1-c& \mathrm{if}\hspace{0.17em}{I}_n\hspace{0.17em}\text{is a cooperator and }\hspace{0.17em}|{\mathcal{G}}_n({\mathbf{m}}_t)|=1,\\ 1+\frac{|{\mathcal{G}}_n({\mathbf{m}}_t){|}_C}{|{\mathcal{G}}_n({\mathbf{m}}_t)|-1}v& \mathrm{if}\hspace{0.17em}{I}_n\hspace{0.17em}\text{is a defector and }\hspace{0.17em}|{\mathcal{G}}_n({\mathbf{m}}_t)|>1,\\ 1& \mathrm{if}\hspace{0.17em}{I}_n\hspace{0.17em}\text{is a defector and }\hspace{0.17em}|{\mathcal{G}}_n({\mathbf{m}}_t)|=1,\end{array}$$ 2.7

where $|\mathcal{G}{|}_C$ is the number of cooperators in a group $\mathcal{G}$. We note that the cooperators still pay a cost when they are alone, and thus the interaction payoff of a lone cooperator is less than that of a lone defector.

An individual will also pay a cost of λ for each movement that it makes. The fitness contribution to the individual I_n at time t is then given by

$$f_{n,t}(m,{\mathcal{G}}_n({\mathbf{m}}_t)\mid m_{n,t-1})=\{\begin{array}{ll}{R}_{n,t}({\mathcal{G}}_n({\mathbf{m}}_t))-\lambda & m\ne m_{n,t-1},\\ R_{n,t}({\mathcal{G}}_n({\mathbf{m}}_t))& m=m_{n,t-1}.\end{array}$$ 2.8

Individuals continue to move around the environment and accumulate fitness contributions for a fixed number of discrete steps, which is determined by an exploration time parameter T. Once the exploration time is over, all individuals instantaneously return to their home places. The fitness of all individuals is reset to 0 at the beginning of each exploration phase. The fitness of the individual I_n at time t at the end of an exploration phase of length T is then the total fitness contribution across the previous T time steps up to and including t, i.e.

$$F_{n,t}({\mathbf{m}}_t)=\sum _{k=t-T+1}^t{f}_{n,k}({\mathbf{m}}_k\mid {\mathbf{m}}_{k-1}).$$ 2.9

The effect of history-based (e.g. accumulated) payoffs has been studied elsewhere [41–43]. Our exploration-phase approach is similar to these investigations in a sense that accumulated payoffs over several rounds of the game are used to update the population. On the other hand, in our model individuals may move around the network, and hence they accumulate payoffs based on interactions with various other individuals, not just their immediate neighbours.

(d). Evolutionary dynamics

We follow the invasion process where an individual I_n is selected to give birth (generating a copy of itself) with probability proportional to its fitness F_n,t, and the copy of the individual replaces one of the existing members of the population. An individual to be replaced is selected at random, proportional to replacement weights. The replacement weight w_i,j,t corresponds to the propensity for the individual I_i to replace the individual I_j if the individual I_i is chosen for reproduction at time t. This evolutionary dynamics has been called the birth–death–birth (BDB) dynamics in the literature [44]. It means an individual for reproduction (birth) is chosen first, and an individual to be replaced (death) is chosen next; the fitness of individuals is used to weigh the birth process only.

Following [34,36], the replacement weight contribution depends upon the amount of time spent with each individual. If the individual is alone, then it effectively allocates all the time to itself. Otherwise, we assume that it divides its time equally between all other members of the group (i.e. not including itself). The replacement weight contribution function, which specifies the proportion of time the individual I_i spent interacting with individual I_j at the current unit of time, is then

$$u_{i,j}({\mathcal{G}}_i(\mathbf{m}))=\{\begin{array}{ll}\frac{1}{|{\mathcal{G}}_i(\mathbf{m})\setminus \{i\}|}& \mathrm{if}\hspace{0.17em}i\ne j\mathrm{and}j\in {\mathcal{G}}_i(\mathbf{m}),\\ 0& \mathrm{if}\hspace{0.17em}i\ne j\mathrm{and}j\notin {\mathcal{G}}_i(\mathbf{m}),\\ 1& \mathrm{if}\hspace{0.17em}i=j\mathrm{and}|{\mathcal{G}}_i(\mathbf{m})|=1,\\ 0& \mathrm{if}\hspace{0.17em}i=j\mathrm{and}|{\mathcal{G}}_i(\mathbf{m})|>1.\end{array}$$ 2.10

In this paper, replacement will only happen directly after individuals have returned to their home places (i.e. to their place at time t − T) at the end of an exploration phase. As in [36], we choose the replacement weight at such a time t, a multiple of T, as the mean replacement weight contribution, averaged over all possible moves of all individuals over a single step,

$$w_{i,j,t}=\sum _{\mathbf{m}}{u}_{i,j}({\mathcal{G}}_i(\mathbf{m}))p_{}(\mathbf{m}\mid {\mathbf{m}}_{t-T}).$$ 2.11

After replacement, the next exploration phase begins with the new population composition.

(e). The evolution of the population

The evolution of the population composition follows a Markov chain. We assume that at any time there are at most two types of individuals in the population: A and B. The two types of individuals are defined by their interactive strategies and staying propensities. One of the types could be a cooperator and the other a defector, so for example we could have A = D_0.9 and B = C_0.7, meaning that the type A individual is a defector with staying propensity 0.9 and the type B individual is a cooperator with staying propensity 0.7. Alternatively, both types could use the same interactive strategy but different staying propensities, so for example we could have A = C_0.6 and B = C_0.2. We evolve the population until the corresponding Markov chain reaches one of the two absorbing states: all individuals are of type A or all individuals are of type B.

We constructed and ran Monte Carlo simulations of the evolution process. At initialization, all individuals begin at their home places. Then the individuals move around the environment (according to the network structure) and accumulate fitness (by playing the public goods game) during an exploration phase, which consists of T time steps. At each time step of an exploration phase, every individual decides whether to stay at the current place or to move to one of the neighbouring places; the probability that an individual is going to stay is given by (2.5). At the end of the exploration phase, all individuals instantaneously return to their home places for reproduction; they will have accumulated total fitness (2.9) by the end of the exploration phase.

Next, a reproduction event occurs. An individual is chosen for reproduction randomly proportional to its total fitness accumulated during the last exploration phase. The offspring replaces one of the existing individuals to keep the population size constant. An individual to be replaced is drawn randomly proportional to the replacement weights (2.11). Because replacement events happen at home places only, we think of them as local events. We therefore assume, as in [36], that an individual may replace only those whom it can meet by making a single movement from its home place. To compute the replacement weights, we allow each individual to move once from its home location; in particular, the probability that an individual is going to stay at its home location is determined only by its staying propensity. We then take a snapshot of the environment and record the replacement weight contribution function (2.10).

We note that we did not simulate the replacement weights; they were computed analytically. The replacement weights depend on the composition of the population because they depend on the staying propensities of the individuals in the population. Thus, there is a unique set of replacement weights for each possible population composition. If the underlying network structure is the complete graph, then each population composition can be uniquely described by the number of type A individuals due to the inherent symmetry of the graph. If the network structure is the star graph, then each population composition can be uniquely described by the number of type A individuals and the type of the individual in the centre of the star. Finally, if the network structure is the circle graph, then any given individual may meet only four other individuals (if everyone makes a single movement from their home location), and hence it is sufficient to compute the replacement weights for all 32 possible combinations of the type distributions in a 5-tuple of individuals.

After a reproduction event, all individuals reset their fitness to 0, and a new exploration phase begins. We continue this process until only one type of individual remains in the population.

We would like to emphasize that our model effectively uses two different network structures: (i) the fixed underlying network, which is used for the exploration (and fitness accumulation) phase, and (ii) the evolving weighted network corresponding to replacement weights (the frequencies of local interactions), which is used for replacement events. We note that this decoupling of the replacement structure from the interaction structure has already been considered elsewhere; see [26]. The key feature of our approach is that the weighted (evolutionary) network arises naturally from the underlying (interaction) network and the current population composition. The weighted network based on the replacement weights evolves with the population because replacement weights depend on the population composition. The fixed underlying network (complete, circle or star graphs) determines the exploration strategies of the individuals and the nature of the multi-player–game interactions of the individuals through varying group compositions. It is the effect of the topology of the fixed underlying network that we investigate here.

3. Results

In this section, we shall consider the effect that the parameters of the model have on the evolution of the population. The parameters used in the simulations are summarized in table 2.

Table 2.

Model parameters.

notation	meaning	base value	range of values
N	population size	10 and 50	{10, 20, 30, 40, 50}
T	exploration time	10	{5, 10, 25}
λ	movement cost	none	{0, 0.1, 0.2, …, 0.8, 0.9}
γ	cooperator staying propensity	none	{0.01, 0.1, 0.2, …, 0.9, 0.99}
δ	defector staying propensity	none	{0.01, 0.1, 0.2, …, 0.9, 0.99}
c	cost of cooperation	0.04
v	reward of cooperation	0.40	{0.04, 0.2, 0.4, 0.8, 1.2, 1.6, 2}
S	sensitivity to group members	0.03
βC	cooperator attractiveness	1
βD	defector attractiveness	−1

With the exception of an individual's interactive strategy and staying propensity, all other parameters take a fixed value. An individual inherits the value of these two characteristics from its parent. Staying propensities can take any value from 0.01, 0.1, 0.2, …, 0.9, 0.99, as indicated in table 2. Thus, no individual moves all the time or never, and all will change their staying probability depending upon the group they are in.

Mutations occur in the population to introduce a new strategy (out of our allowable set). These mutations are sufficiently infrequent that the population is assumed to always consist of a maximum of two types, resident and mutant, whose competition results in the fixation of one of the types before a new mutant appears. Below we consider two different scenarios.

(a). Scenario 1: interactive strategy mutations are rare

A mutation can result in a change in the interactive strategy and/or the staying propensity. In scenario 1, the mutation rate of an individual's interactive strategy is much slower than that involving their staying propensity. Thus, once one of the interactive strategies (cooperate or defect) is removed from the population it will be a long time before a new mutant involving this interactive strategy appears again. During this period, there will be a sequence of mutations in the staying propensity, leading to contests among individuals with the same interactive strategy. The population will evolve a strategy which is a (strict) Nash equilibrium staying propensity, given the interactive strategy.

A mutant with the other interactive strategy will eventually appear; the resident population will have evolved to the Nash equilibrium staying propensity by that time. The staying propensity of the mutant can be different from the equilibrium staying propensity of the residents, and hence we consider mutants with all possible staying propensities. We are interested in the fixation probability ρ of a mutant; it is defined as the probability that a population of size N consisting of N − 1 type A individuals (residents) and a single type B individual (mutant) evolves to the absorbing state consisting of N type B individuals. We then select the mutant whose staying propensity yields the highest fixation probability, and compare the fixation probability of the fittest mutant with the neutral drift fixation probability ρ = 1/N. To describe the outcomes of the selection process in a population where one interactive-strategy mutant has been introduced into a resident population, we adopt the terminology from [45]. There are four possible outcomes, and they are summarized in table 3. We denote the fixation probabilities of cooperator and defector mutants by ρ^C and ρ^D, respectively.

Table 3.

Four possible outcomes of the selection process in scenario 1.

outcome	terminology
ρC > 1/N and ρD < 1/N	selection favours cooperators
ρC < 1/N and ρD > 1/N	selection favours defectors
ρC > 1/N and ρD > 1/N	selection favours change
ρC < 1/N and ρD < 1/N	selection opposes change

All data points we report in this scenario are based on the average of n = 10⁵ independent trials for each combination of parameters. Using the binomial distribution, we can estimate the standard deviation of the fixation probability of the mutants as $\sqrt{q(1-q)/n}$, where q is the actual fixation probability. We are mostly interested in the accuracy of the simulated mutant fixation probabilities close to the neutral drift fixation probability 1/N, and table 4 shows the values of the standard deviation of the simulated fixation probabilities assuming q = 1/N.

Table 4.

Accuracy of the simulated mutant fixation probabilities close to the neutral drift fixation probability.

population size	10	20	30	40	50
s.d.	0.00095	0.00069	0.00057	0.00049	0.00044

(i). Complete graph

Figure 2 shows the optimal staying propensities of residents and mutants and the fixation probabilities of cooperator and defector mutants in the population of size N = 50. Generally, the optimal staying propensities increase with the movement cost because high movement costs discourage frequent relocations. Resident defectors use the staying propensity δ = 0.99 for all movement costs. Mutant cooperators do best with the staying propensity γ = 0.7 for λ = 0, and their optimal staying propensity gradually increases with the movement cost to reach the maximum value γ = 0.99 for λ = 0.6. Resident cooperators evolve to γ = 0.2 for λ = 0, and their optimal staying propensity increases with the movement cost up to γ = 0.6. Mutant defectors do best with the minimum staying propensity δ = 0.01 for low movement costs, and then sharply increase their optimal staying propensity to δ = 0.99 for λ = 0.4. The fixation probability of the fittest mutant cooperators exceeds the neutral drift value 1/N for movement costs up to λ = 0.2, but the fixation probability of the fittest mutant defectors always stays below 1/N. According to the terminology in table 3, this means selection favours cooperators for sufficiently low movement cost, and selection opposes change otherwise.

Figure 2.

Complete graph, population size 50. Selection favours cooperators if the movement cost is at most λ = 0.2, and selection opposes change otherwise. (a) Optimal staying propensities for resident defectors and the fittest mutant cooperators; (b) optimal staying propensities for resident cooperators and the fittest mutant defectors; (c) the fixation probabilities of the fittest mutant cooperators and mutant defectors in the resident defector and cooperator populations, respectively. Defectors, red with star markers for data points; cooperators, blue with round markers for data points; residents, solid line; mutants, dashed line; neutral drift fixation probability, solid grey line. (Online version in colour.)

The main feature of the complete graph is that selection usually opposes change. The optimal staying propensities stay almost the same for different population sizes, and selection favours cooperators when the movement cost is at most λ = 0.2 regardless of the population size. Selection favours defectors only when the population is sufficiently small and the movement cost is sufficiently high (at least λ = 0.6 when N = 10 and at least λ = 0.9 when N = 20). The outcomes of the selection process for all values of the movement cost and population size are shown in figure 3. For full results, including equilibrium staying propensities and fixation probabilities, see electronic supplementary material, figure S1.

Figure 3.

Outcomes of the selection process in the complete graph for all values of the movement cost and population size. Selection usually opposes change. The exploration time and reward-to-cost ratio are at their base values. This figure summarizes the results which can be seen in detail in the electronic supplementary material. We note that the results are for population sizes in multiples of 10 and movement costs in multiples of 0.1; where the result changes, the dividing line between the regions is drawn halfway (for example, selection favours cooperators for 0.2, but opposes changes for 0.3, so the divider is drawn at 0.25; similarly for population size). (Online version in colour.)

Shortening the exploration time from the baseline value T = 10 to T = 5 helps defectors and hurts cooperators. In particular, selection now favours defectors in large populations for sufficiently high movement cost. Extending the exploration time to T = 25 helps cooperators and hurts defectors. Selection no longer favours defectors even in small populations, and selection favours cooperators for higher values of the movement cost (up to λ = 0.5). These outcomes are consistent with the results in [36] for small (N = 10 and N = 20) populations. For full results, see electronic supplementary material, figures S2 and S3.

Increasing the reward-to-cost ratio allows selection to favour cooperators for all movement costs. Conversely, decreasing the reward-to-cost ratio allows selection to favour defectors even in large populations. For full results, see electronic supplementary material, figures S4 and S5.

(ii). Circle graph

Figure 4 shows optimal staying propensities and fixation probabilities in the population of size N = 50. The optimal staying propensities of residents and mutants behave similar to the complete graph, but the fixation probabilities of the fittest mutants follow a different pattern. The fixation probability of the fittest mutant cooperators exceeds the neutral drift value 1/N for all movement costs up to λ = 0.4, and the fixation probability of the fittest mutant defectors exceeds 1/N for movement costs above λ = 0.4. In the terminology of table 3, selection favours cooperators for movement costs up to λ = 0.4, and selection favours defectors for movement costs above λ = 0.4.

Figure 4.

Circle graph, population size 50. Selection favours either cooperators (for low movement costs) or defectors (for high movement costs). For legend, see caption of figure 2. (Online version in colour.)

The main feature of the circle graph is that selection favours either cooperators (for lower movement costs) or defectors (for higher movement costs). There are a couple of exceptions for small populations: selection favours change when λ = 0 and N = 10 only, and selection opposes change for intermediate movement costs λ = 0.3 and 0.4 when N = 10 and for λ = 0.4 when N = 20. But the pattern stabilizes for larger populations. The outcomes of the selection process for all values of the movement cost and population size are shown in figure 5. For full results, see electronic supplementary material, figure S6.

Figure 5.

Outcomes of the selection process in the circle graph for all values of the movement cost and population size. Excepting a couple of outliers for small populations, there is a threshold for the movement cost which separates two outcomes: selection favours cooperators (for low movement costs) or selection favours defectors (for high movement costs). Region divisions are drawn as in figure 3. Exceptionally, this division occurred almost exactly at 0.4 for N = 30 and N = 40, and so the vertical line here is drawn at 0.4. (Online version in colour.)

Shortening the exploration time to T = 5 significantly helps defectors and hurts cooperators. Selection favours defectors for almost all movement costs, and selection favours cooperators only for movement costs up to λ = 0.1. Extending the exploration time to T = 25 significantly helps cooperators and hurts defectors. Selection no longer favours defectors for all but the highest movement costs, and selection favours cooperators for movement costs up to λ = 0.8 in large populations (N = 50). For full results, see electronic supplementary material, figures S7 and S8.

Increasing the reward-to-cost ratio allows selection to favour cooperators for all movement costs, and decreasing the reward-to-cost ratio allows selection to favour defectors for all movement costs. For full results, see electronic supplementary material, figures S9 and S10.

(iii). Star graph

Figure 6 shows the optimal staying propensities and fixation probabilities for the population of size N = 50. In the star graph, both optimal staying propensities and fixation probabilities exhibit different patterns from the complete and circle graph cases. The optimal staying propensity of resident cooperators is γ = 0.01 for all movement costs. The staying propensity of the fittest mutant cooperators starts at γ = 0.2 for λ = 0 and gradually increases to γ = 0.99 for λ = 0.6. However, the staying propensity of the fittest mutant defectors stays at the minimum value δ = 0.01 for all movement costs but λ = 0.9, where it is δ = 0.1. The fixation probability of the fittest mutant cooperators exceeds the neutral drift value 1/N for movement costs up to λ = 0.4. The fixation probability of the fittest mutant defectors, however, exceeds 1/N for all movement costs. In the terminology of table 3, selection favours change for movement costs up to λ = 0.4, and selection favours defectors for movement costs above λ = 0.4.

Figure 6.

Star graph, population size 50. Resident cooperators' optimal staying propensity is γ = 0.01 for all movement costs. Selection favours change for movement costs up to λ = 0.4, and selection favours defectors for all higher movement costs. For legend, see the caption of figure 2. (Online version in colour.)

The unique feature of the star graph is that the fixation probability of mutant defectors ρ^D exceeds the neutral drift fixation probability 1/N for all parameter combinations. The fixation probability of mutant cooperators exceeds the neutral drift one only for sufficiently low movement costs; the threshold for the movement cost is increasing with the population size. The optimal staying propensity for resident cooperators is always γ = 0.01 for all population sizes and movement costs, but both cooperator and defector mutants tend to have higher staying propensities in smaller populations. The outcomes of the selection process for all values of the movement cost and population size are shown in figure 7. For full results, see electronic supplementary material, figure S11.

Figure 7.

Outcomes of the selection process in the star graph for all values of the movement cost and population size. Selection either favours change (for sufficiently low movement costs depending on the population size) or favours defectors. Region divisions are drawn as in figure 3. (Online version in colour.)

Shortening the exploration time to T = 5 hurts cooperators; the region where selection favours change shrinks to include only the movement cost λ = 0 in a small population (N = 10) and the movement costs up to λ = 0.3 in a large population (N = 50). Extending the exploration time to T = 25 also hurts cooperators; the region where selection favours change shrinks to include only the movement cost λ = 0 in a small population (N = 10) and the movement costs up to λ = 0.2 in a large population (N = 50). This is different from the other cases, where cooperators do better with a longer exploration time. A longer exploration phase allows cooperators to spend more time together once they find each other and thus to accumulate more rewards of cooperation. Resident defectors do not move often enough to be able to effectively locate and exploit cooperators. In the star graph, however, the most likely location of a cooperating cluster is in the centre, and hence a longer exploration phase gives resident defectors more opportunities to move to the centre and to exploit the cooperators. In this case, cooperators want to spend just the ‘right’ amount of time together to accumulate sufficient rewards yet to avoid defectors which will eventually come to the centre. For full results, see electronic supplementary material, figures S12 and S13.

Increasing the reward-to-cost ratio results in selection favouring change for all movement costs, and decreasing the reward-to-cost ratio results in selection favouring defectors for all movement costs. For full results, see electronic supplementary material, figures S14 and S15.

(iv). Summary and analysis

The three network structures we considered resulted in similar outcomes for the selection of mutant cooperators. The fixation probability of mutant cooperators ρ^C exceeded the neutral drift fixation probability 1/N only for sufficiently low movement cost in all three types of networks. The threshold for the movement cost where ρ^C > 1/N varied with the network topology and/or the population size: it was somewhat higher for larger populations on the circle and star graphs. Cooperators need to aggregate in clusters in order to outcompete defectors, and hence they must explore the environment to find each other. Paying higher movement costs decreases the accumulated payoff and, consequently, results in lower fixation probabilities.

On the other hand, the fixation probability of the mutant defectors, ρ^D, exhibited contrasting behaviour relative to the neutral drift fixation probability for different network topologies. We observed three different outcomes for the selection process of the mutant defectors: (i) ρ^D < 1/N for all movement costs in the complete graph for a sufficiently large population; (ii) ρ^D > 1/N for sufficiently high movement costs in the circle graph; and (iii) ρ^D > 1/N for all movement costs in the star graph. In particular, mutant defectors found it easiest to replace resident cooperators in the star graph, which has a hub-and-spoke topology. Resident cooperators evolved to the lowest staying propensity γ = 0.01 for all movement costs in the star graph because paying any movement cost to quickly get to the centre of the star was amply compensated by the reward of staying in the resulting cluster of cooperators for the remaining exploration steps. The high sensitivity to group members ensured that the attractiveness of a cluster of cooperators outweighed the inherently low staying propensity (figure 1). Because the centre of the star was the most probable location for the cluster of resident cooperators, it could be easily exploited by mutant defectors. The fittest mutant defectors mirrored the exploration strategy of resident cooperators: it was best to get to the centre of the star as quickly as possible.

We thus see that network topology only has a relatively small effect on whether cooperators can replace a defector population or not. This is an interesting and, at first sight, surprising result. However, cooperators need to be able to find each other to enjoy the benefits of cooperation; hence, they need to move around the environment sufficiently often. When the movement cost is high, such an exploration strategy proves too costly. Yet having a high staying propensity might preclude cooperators (when there are few of them) from interacting with each other. So, mutant cooperators find themselves in a no-win scenario when the movement cost is high no matter what the network structure is. But they often prevail in a low-movement-cost environment. This is also partially due to the resident defectors employing suboptimal exploration strategies in such environments: their staying propensity is always the maximum possible regardless of the movement cost. If defectors were able to evolve to lower staying propensities in the absence of cooperators in low-movement-cost environments, then invading cooperators would have a harder time.

On the other hand, there is a very big effect on the stability of a population of cooperators once established. For a complete graph, which has a high clustering coefficient and low degree centralization (table 1), cooperators can always resist defectors for sufficiently large populations. For the circle, with a low clustering coefficient and low degree centralization, cooperators can sometimes resist defectors. For the star, with a low clustering coefficient and high degree centralization, cooperators can never resist defectors. This is logical, as when there are many cooperators a high clustering coefficient makes it easier for them to find each other (only small cooperative groups are required, and there will probably be many such small groups) but high degree centralization makes it easy for defectors to find cooperators and exploit them. We hypothesize that this may be a general phenomenon. The graphs that we have used are the extreme such cases, with the clustering coefficient and degree centralization taking the maximum or minimum values of 1 or 0 in each case; however, we would expect a similar pattern for other graphs with high or low values of these expressions, and this is something that we will explore in future work.

(b). Scenario 2: interactive strategy mutations are not rare

In scenario 2, the mutation rate of an individual's interactive strategy is of the same order as that of their staying propensity. Any successful strategy must repeatedly face invaders of both interactive-strategy types. We thus look for a (strict) Nash equilibrium staying propensity which will be determined in a mixed population, with individuals of both types.

Following [36], we consider a population where there are N/2 individuals of each interactive-strategy type, and find the Nash equilibrium staying propensity for each type within this population. We then record (i) the equilibrium staying propensities of cooperators and defectors in a mixed population and (ii) the probabilities that the mixed population evolves to all cooperators or all defectors when both types of individuals employ the equilibrium staying propensities. These probabilities add up to 1, and we call them fixation probabilities for consistency in terminology with scenario 1. All data points we report in the mixed population case are based on the average of n = 10⁴ independent trials. Using the binomial distribution as in scenario 1, with q = 1/2, we can estimate that the maximum standard deviation for the simulated fixation probabilities is 0.005.

We also consider the selection process when a single interactive-strategy mutant has been introduced into a resident population where both residents and mutants employ the mixed-population equilibrium staying propensities (we shall refer to this as a mutant-residents population). The data points in these simulations are based on the average of n = 10⁵ trials, as in scenario 1.

(i). Complete graph

Figure 8 shows the equilibrium staying propensities for cooperators and defectors in a mixed population and the fixation probabilities in (i) a mixed population and (ii) the case when a single interactive-strategy mutant has been introduced into a resident population; the graphs correspond to the population size N = 50. In a mixed population, cooperators dominate defectors for all movement costs except λ = 0.9. When both mutants and residents employ the equilibrium staying propensities from the mixed population, selection favours cooperators for movement costs 0.1–0.3 and opposes change otherwise.

Figure 8.

Complete graph, population size 50. Cooperators dominate defectors in a mixed population for all movement costs but the highest one. (a) Equilibrium staying propensities in a mixed population; (b) the fixation probabilities of cooperators and defectors with equilibrium staying propensities in a mixed population; (c) the fixation probabilities of mutant cooperators and mutant defectors in a resident defector and cooperator population, respectively, where both cooperators and defectors inherit equilibrium staying propensities from the mixed population. Defectors, red with star markers for data points; cooperators, blue with round markers for data points; mixed population, solid line; mutants, dashed line; neutral drift fixation probability, solid grey line. (Online version in colour.)

In a small population (N = 10), cooperators dominate defectors in a mixed population for movement costs 0.1–0.6; defectors prevail otherwise. In a mutant-residents small population, selection opposes change for all positive movement costs; selection favours defectors when λ = 0. Shortening the exploration time to T = 5 hurts cooperators (e.g. cooperators do better than defectors in a large mixed population only for movement costs 0.1–0.3, and the fixation probability of cooperator mutants in any resident defector population is always below the neutral drift one), while extending the exploration time to T = 25 helps cooperators (e.g. cooperators dominate defectors in a large mixed population for any movement cost, and selection favours cooperators in a small resident defector population for movement costs 0.2–0.5). Varying the reward-to-cost ratio has a similar effect to scenario 1. For full results, see electronic supplementary material, figures S16–S20.

(ii). Circle graph

Figure 9 shows the results of the simulations in the population of size N = 50. There are two threshold values of the movement cost: λ₁ = 0 and λ₂ = 0.5, which separate the outcomes for both mixed and mutant-residents populations. For movement costs between λ₁ and λ₂, cooperators dominate defectors in a mixed population, and selection favours cooperators in a mutant-residents population. For movement costs above λ₂, defectors dominate cooperators in a mixed population and selection favours defectors in a mutant-residents population. For the threshold movement cost values, cooperators tie with defectors in a mixed population, and selection opposes change in a mutant-residents population.

Figure 9.

Circle graph, population size 50. There are two threshold values of the movement cost: λ₁ = 0 and λ₂ = 0.5, which separate the outcomes of the evolution process in both mixed and mutant-residents populations. For legend, see caption of figure 8. (Online version in colour.)

In a small population (N = 10), cooperators do better than defectors in a mixed population for movement costs 0.1–0.4, and defectors do better than cooperators otherwise. In a mutant-residents small population, selection favours cooperators for movement cost 0.2, selection favours defectors for movement costs 0 and 0.6–0.9, and selection opposes change for movement costs 0.1 and 0.3–0.5. Shortening the exploration time to T = 5 hurts cooperators (e.g. cooperators never do better than defectors in a small mixed population, and the fixation probability of cooperator mutants in a small resident defector population is always below the neutral drift one), while extending the exploration time to T = 25 helps cooperators (e.g. cooperators do better than defectors for all positive movement costs in a small mixed population and for all movement costs in a large mixed population, and selection favours cooperators in a large mutant-residents population for all movement costs except 0 and 0.9). Varying the reward-to-cost ratio has the usual effect. For full results, see electronic supplementary material, figures S21–S25.

(iii). Star graph

The star graph presented a unique challenge in a mixed-population case: there were no (Nash) equilibrium staying propensities for cooperators and defectors. This was caused by the discontinuity of the best-response staying propensity of cooperators as a function of the staying propensity of defectors for all movement costs higher than λ = 0.1 in a large population (N = 50). Figure 10 shows an example of the phase transition of the system when the best-response staying propensity of cooperators exhibits a jump discontinuity from 0.99 at defectors' staying propensity 0.86 to 0.01 at defectors' staying propensity 0.88. The best response of defectors to cooperators with staying propensity 0.01 is the staying propensity 0.01, while the best response of defectors to cooperators with staying propensity 0.99 is 0.99. However, cooperators' best response to all defector staying propensities up to 0.86 is 0.99, while cooperators' best response to all defector staying propensities above 0.87 is between 0.01 and 0.4. Hence, there are no equilibrium staying propensities for cooperators and defectors in this case.

Figure 10.

Phase transition of the cooperators' best-response staying propensity on a star graph of size N = 50. Each panel shows the fixation probabilities of cooperators with all possible staying propensities in a mixed population with defectors using a fixed staying propensity value δ. (a) δ = 0.86, (b) δ = 0.87, (c) δ = 0.88; movement cost λ = 0.4 in all graphs. There is a jump discontinuity in the cooperators' best-response staying propensity at δ = 0.87 from γ = 0.99 to γ = 0.01. (Online version in colour.)

To overcome this problem, we made an additional assumption: staying propensity mutations are ‘local’. For example, if the current mixed population consists of cooperators with staying propensity γ = 0.8 and defectors with staying propensity δ = 0.9, then cooperators may adjust their staying propensity to either 0.7, 0.8 or 0.9 (and they choose the one which results in the highest fixation probability for cooperators), and defectors may adjust their staying propensity to either 0.8, 0.9 or 0.99. We start the process with both cooperators and defectors having staying propensity 0.99 (e.g. the initial colonization of the environment is performed by individuals who have not yet learned to explore the environment). Cooperators and defectors ‘take turns’ to locally adjust their staying propensity in response to the individual of the other type. This algorithm results in two possible outcomes: (i) the staying propensities of cooperators and defectors stabilize in a ‘local trap’: the resulting staying propensity of cooperators is the best response to the resulting staying propensity of defectors locally, but not necessarily globally; similarly for defectors, and (ii) the locally best-response staying propensities of cooperators and defectors oscillate in a loop; for example, the locally best response to δ = 0.9 is γ = 0.8, the locally best response to γ = 0.8 is δ = 0.99, the locally best response to δ = 0.99 is γ = 0.7, and the locally best response to γ = 0.7 is δ = 0.9, closing the loop. In the last case, we take the middle points of the loop (i.e. γ = 0.75 and δ = 0.95) for the staying propensities of cooperators and defectors in a mixed population.

Figure 11 shows the results of the simulations in the population of size N = 50. In a mixed population, defectors dominate cooperators for movement costs up to λ = 0.2. Cooperators tie with defectors for movement cost λ = 0.3, and cooperators do better than defectors for intermediate movement costs 0.4–0.7. Defectors prevail for movement costs starting from λ = 0.8 because local-loop traps force cooperators to adopt lower staying propensities than for movement costs 0.4–0.7. In a mutant-residents population, selection favours defectors for movement costs up to λ = 0.7, and selection opposes change for higher movement costs.

Figure 11.

Star graph, population size 50. In a mixed population, defectors do better than cooperators for either low or high movement costs; cooperators do better than defectors for intermediate values of the movement cost. For legend, see caption of figure 8. (Online version in colour.)

In a small population (N = 10), cooperators do better than defectors in a mixed population for movement costs 0.2–0.5. In a mutant-residents population, selection opposes change for movement costs 0.3–0.6, and selection favours defectors otherwise. For full results, see electronic supplementary material, figure S26.

Shortening the exploration time to T = 5 hurts cooperators (e.g. defectors dominate cooperators in both small and large mixed populations for all movement costs, and selection favours defectors in both small and large mutant-residents populations for all movement costs), while extending the exploration time to T = 25 helps cooperators (e.g. cooperators dominate defectors in a large mixed population for all positive movement costs, and the fixation probability of mutant cooperators exceeds the neutral drift one in a large mutant-residents population for movement costs 0.1–0.6). We note that this contrasts with the equivalent case from scenario 1. In a mixed population, cooperators may afford to employ a higher staying propensity when the exploration phase is long. Because there are sufficiently many cooperators in the environment, some of them will eventually make it to the centre of the star and form a successful cooperating cluster. It is harder for defectors to exploit cooperators in this case. If defectors move often (and pay the associated cost), the cooperators may not be in the centre yet, and the defectors will find the group unattractive, thus moving again on the next round to the leaves. But if they move rarely, then they might miss the moment the cooperating cluster is formed. In scenario 1, however, there are few mutant cooperators initially, and they cannot afford high staying propensities because they might never meet in this case. Defectors, on the other hand, are plentiful and some of them will make it to centre. For full results, see electronic supplementary material, figures S27 and S28.

Varying the reward-to-cost ratio has a standard effect except that increasing this ratio does not help cooperators in a small mixed population when the movement cost is equal to 0.1. For full results, see electronic supplementary material, figures S29 and S30.

(iv). Summary and analysis

The three network structures resulted in different outcomes for the evolution of both mixed and mutant-residents populations. In general, cooperators did best on the complete graph and worst on the star graph in a mixed population. Cooperators dominated defectors for all movement costs but the highest one (0.9) on the complete graph; there were two threshold values of the movement cost (0 and 0.5) that separated cooperator and defector dominance on the circle graph; and cooperators did slightly better than defectors for intermediate values of the movement cost on the star graph.

In the mutant-residents population, where both cooperators and defectors inherited equilibrium staying propensities from the mixed population, the fixation probabilities of mutant defectors, ρ^D, followed patterns similar to scenario 1: (i) ρ^D < 1/N for all movements costs in a large population on the complete graph; (ii) ρ^D > 1/N for sufficiently high movement cost on the circle graph; and (iii) ρ^D > 1/N for almost all movement costs (except the two highest ones) in a large population on the star graph. The fixation probabilities of mutant cooperators were above the neutral drift one for sufficiently small positive movement costs on the complete and circle graphs. The fixation probabilities of mutant cooperators were always below the neutral drift one on the star graph.

4. Discussion

In this paper, we build upon the work of [36], the first history-dependent model of the evolutionary framework of [32], which allows for general multi-player interactions and structured populations. Here (and in [36]) a mobile population moves around a territory in the form of a network, with groups of individuals interacting at the nodes. Movement is Markov, with the place an individual moves to next depending upon both current position and the composition of their current group. Individuals were of two (classic) types, cooperator or defector, and individuals of either type preferred to associate with cooperators and to avoid defectors. The main findings of [36] were that longer exploration times and lower movement costs promoted cooperation. Moreover, the evolutionary dynamics did not significantly affect the outcome of the evolution process. In particular, the BDB dynamics used in this paper allows cooperation to evolve even though typically selection does not favour cooperators with these dynamics [46].

In [36], the territory was in the form of a complete graph (i.e. it was essentially unstructured), and the semi-analytic approach restricted the analysis to small populations only (size 10 and 20). Our main motivation for the current paper was to extend the analysis of [36] to other networks (and larger populations). We were primarily interested in learning the effect of the network structure on the outcome of the evolutionary process, and we have considered three network structures here: the complete graph, the circle graph and the star graph. These networks represent all possible extreme cases for the values of the clustering coefficient and degree centralization. Additionally, they possess a high degree of symmetry, which allowed us to use analytically computed replacement weights in the evolutionary dynamics.

Our findings confirm those of [36] for a small population on the complete graph, but show some key differences for the other two network structures. For all networks, the fixation probability of mutant cooperators exceeded the neutral drift one only if the movement cost was below a certain threshold, which varied between 0.1 and 0.4 depending on the network and the population size. Cooperators need to cluster together to outcompete defectors, and hence they must explore the environment to meet each other. If the movement cost is high, then moving around is detrimental to one's payoff. We found that the network structures we considered here could not overcome this general phenomenon, and the outcomes of the cooperator invasion did not vary much among different networks.

But the outcomes of the defector invasion differed drastically for the three networks, and we saw three contrasting outcomes: the fixation probability of mutant defectors never exceeded the neutral drift one on the complete graph (for sufficiently large populations), it exceeded the neutral drift fixation probability for all sufficiently high movement costs on the circle graph, and it always exceeded the neutral drift fixation probability on the star graph. Therefore, the network topology plays a crucial role in the stability of a population of cooperators.

Cooperators resist replacement by mutant defectors on the complete graph, which corresponds to the highest clustering coefficient and lowest degree centralization. The uniform degree distribution of the vertices of the complete graph means there are no hubs, and hence a cluster of cooperators may form at any place. It could be difficult for defectors to locate such clusters. On the other hand, if defectors start exploiting a cooperating cluster, then cooperators will soon find the group unattractive. They will move away from that location, but they will still be neighbours of each other owing to the high clustering coefficient. Consequently, cooperators might form new clusters soon after leaving their previous location.

By contrast, cooperators cannot resist replacement by mutant defectors on the star graph, which corresponds to the lowest clustering coefficient and highest degree centralization. In the hub-and-spoke topology of the star graph, the centre of the star is the most likely location for a potential cooperating cluster, and hence it can be easily exploited by defectors.

To summarize, the stability of a population of defectors was determined by the movement cost; the network topology played a small (or no) role. But the stability of a population of cooperators was determined by the network topology; the movement cost played a secondary (or no) role. These observations constitute the main findings of our work.

We note also previous work on the evolution of cooperative behaviour which centred upon movement, such as in [47,48], where individuals left with probability 1 if their payoff was below a minimum threshold (the ‘walk away’ strategy), or in [49,50], where individuals on a lattice similarly moved when their interactions did not yield sufficient reward. In [51], individuals moved, again over a lattice, when the reputation of others at their place was lower than their own. In [52,53], individuals could see all locations within a certain neighbourhood, and would use this information to decide where to move to.

Although our work involves an underlying fixed structure, the replacement weights change with the population composition, and this can be thought of as analogous to a changing underlying structure. There has been significant research considering the explicit coevolution of population strategy and structure. In [54,55], there was an underlying graph where edges were formed or broken as a consequence of the types of individuals involved. An alternative more general set-centred model was developed in [56]; in [57] individual reputation drove the changes in the structure; and in [58] individual prosperity was the determining factor. An excellent review of work up to 2010 is given in [59]; a more general review is given by Allen & Nowak [60].

A general feature of models where cooperation can evolve is positive assortment, where cooperators have, on average, a higher degree of interaction with other cooperators than defectors have [61,62]. In our model, the movement of individuals allows this to occur since cooperators will move to be with other cooperators; defectors do this too, but this is less efficient, since cooperators will try to resist this association.

In this paper, we identified two network topology characteristics (clustering coefficient and degree centralization) that affected the outcome of the evolution of cooperation. But we considered extreme cases of these topologies only, and the exact nature of the effect of network topology on the evolution of cooperation warrants further investigation. For example, what happens for intermediate values of the clustering coefficient and degree centralization measures? Will we observe a smooth transition of the stability of a population of cooperators as these measures change, or are there threshold values which result in phase transitions? How far would we have to deviate from the two extremely opposite cases (complete graph and star graph) to obtain mixed behaviour? We will address these questions in subsequent work.

In particular, we can think of the complete graph as a random Erdős–Rényi graph with the probability of two random vertices being connected equal to 1. We may lower this probability, which would result in a lower clustering coefficient and a higher degree centralization, and consider a family of random Erdős–Rényi graphs. A natural generalization of the circle graph are random regular graphs and (to some extent) small-world Watts–Strogatz graphs. We also plan to consider metastructures built from several star graphs (star-like networks) and complete graphs (cliqued networks). Building such metastructures may drastically change the outcome of the evolution of cooperation that was observed on the original base structures [63].

Data accessibility

The electronic supplementary material contains full details of simulation results.

Authors' contributions

I.V.E. wrote and ran the computer simulations and prepared all figures. All authors contributed to the design and the analysis of the study, as well as the draft of the manuscript, and gave final approval for publication.

Competing interests

We declare we have no competing interests.

Funding

This work was supported by funding from the European Union's Horizon 2020 research and innovation programme under a Marie Sklodowska-Curie grant (agreement no. 690817). The research was also supported by an EPSRC grant (no. EP/N014499/1) to K.P.