Small bots, big impact: solving the conundrum of cooperation in optional Prisoner’s Dilemma game through simple strategies

Abstract

Cooperation plays a crucial role in both nature and human society, and the conundrum of cooperation attracts the attention from interdisciplinary research. In this study, we investigated the evolution of cooperation in optional Prisoner’s Dilemma games by introducing simple bots. We focused on one-shot and anonymous games, where the bots could be programmed to always cooperate, always defect, never participate or choose each action with equal probability. Our results show that cooperative bots facilitate the emergence of cooperation among ordinary players in both well-mixed populations and a regular lattice under weak imitation scenarios. Introducing loner bots has no impact on the emergence of cooperation in well-mixed populations, but it facilitates the dominance of cooperation in regular lattices under strong imitation scenarios. However, too many loner bots on a regular lattice inhibit the spread of cooperation and can eventually result in a breakdown of cooperation. Our findings emphasize the significance of bot design in promoting cooperation and offer useful insights for encouraging cooperation in real-world scenarios.

1. Introduction

Explaining the evolution of cooperation poses a scientific challenge because individuals who incur costs to benefit others may reduce their own fitness, which conflicts with the theory of ‘survival of the fittest’ [1]. However, this prediction contrasts with the reality of ubiquitous altruistic behaviour. Evolutionary game theory offers a powerful framework to address this issue [2]. Within this framework Prisoner’s Dilemma game, in particular, has been extensively used to study the evolution of cooperation in the context of social dilemma games, as it highlights the conflict between individual and collective interests [3]. It has been shown that cooperation can survive through several reciprocity mechanisms, including group and kin selection [4,5], direct and indirect reciprocity [3,6], as well as network reciprocity [7,8]. Although these reciprocity mechanisms may appear materially different, they share a common principle, which is the mutual recognition of cooperators, also referred to as positive assortativity [9]. If the game is one-shot and anonymous, where the above reciprocity mechanisms are fully excluded, the problem of cooperation can still be addressed by enabling players to punish wrongdoers [10], reward altruists [11,12] or exercise their freedom to choose whether to participate in the game or not [13,14]. Several recent theoretical studies have shown that introducing committed cooperators can enable the survival of cooperation in well-mixed populations, even under unfavourable conditions such as one-shot and anonymous games without any reciprocity mechanisms [15–17].

In the context of social dilemma games, committed cooperators refer to individuals who hold strong beliefs about altruistic behaviour and do not change their behaviour over time. While committed cooperators may not alter the advantage of defection over cooperation, their presence does increase the likelihood of interactions between cooperators. This ultimately leads to a greater propensity for cooperation in well-mixed situations when frequency-dependent updating rules are applied [15,17]. However, the cooperation-boosting effects of committed cooperators disappear when network reciprocity is considered [18]. Network reciprocity suggests that cooperators can survive by forming compact clusters to resist the invasion of defectors [7]. The establishment of network reciprocity can be better understood by examining how cooperative clusters can form during their enduring period and how they can expand their territories during the expanding period [19]. However, the presence of committed cooperators who consistently choose to cooperate can inhibit the formation of cooperative clusters and lower their spreading efficiency, resulting in a detrimental effect on cooperation rather than a beneficial one [18]. Committed minorities have also been studied in relation to various issues, including trust and trustworthiness [20], the development race in artificial intelligence [21], and opinion polarization in the voter model [22], among others [23].

Research has shown that committed minorities can play a crucial role in promoting cooperation, but in practical situations, such individuals are usually scarce. If a significant number of committed individuals are required to encourage cooperation, the impact of committed cooperators may be insignificant. With the advancement of artificial intelligence (AI), future populations will be more hybrid, consisting of both human and AI players. In the context of one-shot and anonymous games, where there are no reputation effects or repeated interactions and where information is limited, bots can be designed using straightforward algorithms based on the concept of committed individuals [24]. These bots can be programmed to consistently choose one action and never change it over time. Although their strategy is simple, they can help solve complex problems that human players may find challenging. For example, they can aid in resolving cooperation problems [24–26], tackling the collective risk dilemma [27,28] and addressing the conundrum of punishment [29] and others [30].

In this paper, we investigate the evolution of cooperation among human players in human–machine interactions by examining the impact of four types of bots on an optional Prisoner’s Dilemma game. The game involves three actions: cooperation to benefit others (C), defection for self-interest (D) and loner for risk aversion (L). The four types of pre-designed bots are: (i) always choosing to cooperate, (ii) always choosing to defect, (iii) always choosing loner, and (iv) choosing cooperation, defection and loner with equal probability. Additionally, we consider two representative population structures: a well-mixed population where players have an equal probability of interacting with others, and a regular lattice where players can only interact with their nearest neighbours. Our findings suggest that introducing cooperative bots can address the conundrum of cooperation under weak imitation scenarios in both well-mixed populations and regular lattices. Additionally, although introducing loner bots does not have any impact on the emergence of cooperation in well-mixed populations, it facilitates the dominance of cooperation under strong imitation scenarios in regular lattices. However, too many loner bots on a regular lattice inhibit the spread of cooperation and can eventually result in a breakdown of cooperation. Our results, taken together, indicate that the level of cooperation can be maximized by managing the behaviour of simple bots.

2. Model

Our model involves four essential components, which are: (i) the payoff matrix, (ii) population settings, (iii) game dynamics, and (iv) simulation settings. Each of these components is described in detail below.

2.1. Optional Prisoner’s Dilemma game

In the optional Prisoner’s Dilemma game, players choose between cooperation, defection, or acting as a loner simultaneously. Cooperators incur a cost of c but bring a benefit of b to their opponent, while defectors do nothing. Loners choose to quit the game and receive a small, but positive payoff of σ, generating the same payoff for their opponent. If two cooperators meet, they both receive a payoff of R = b − c. Two defectors receive a payoff of P = 0, while a cooperator meeting a defector incurs a cooperation cost of S = −c, and the defector receives a benefit of T = b. By rescaling the payoffs using r = c/(b − c) and substituting, the payoffs become T = 1 + r, R = 1, P = 0 and S = −r. The resulting payoff matrix is shown in table 1. We use the concept of universal dilemma strength [31] to quantify the dilemma extent, which is confirmed to be D_g′ = D_r′ = r [32]. This particular Prisoner’s Dilemma game exhibits characteristics of both chicken-type dilemmas (arising from greed) and stag-hunt-type dilemmas (arising from fear).

Table 1.

Payoff matrix for the optional Prisoner’s Dilemma game.

	C	D	L
C	1	−r	σ
D	1 + r	0	σ
L	σ	σ	σ

2.2. Population settings

There are typically two population structures considered in research: well-mixed and structured. In a well-mixed population, individuals can interact with any other individual. We represent the proportions of cooperators, defectors and loners among normal players as x, y and z, respectively, where x + y + z = 1. We assume that the fraction of bots is ρ, and therefore the total population density is 1 + ρ. By contrast, a structured population only allows for local interactions, where individuals can only interact with their immediate neighbours. We represent the structured population as a two-dimensional regular lattice with periodic conditions and a degree of four, indicating that each node has four closest neighbours in the up, down, left and right directions. The regular lattice has a size of L × L. The entire population is divided into two types at random in a regular lattice, with ρ proportion as bot players and the remaining proportion as normal players. In both well-mixed and regular lattice populations, normal players are presented with three options: cooperation, defection and loner, each with an equal likelihood. Bot players are pre-programmed with four strategies: (i) always choosing cooperation, (ii) always choosing defection, (iii) always choosing loner, and (iv) choosing cooperation, defection and loner with equal probability.

2.3. Game dynamics

The mean-field theory was used to analyse the human–machine game within an infinite and well-mixed population. We provide an example of the game dynamics for the cooperative bots, while additional details regarding the other three types of bots can be found in appendix A. As described in the population settings section, the expected payoff for each actor can be represented by

$$\begin{array}{rl}{\Pi }_C& =\frac{x-yr+z\sigma +\rho }{1+\rho },\\ {\Pi }_D& =\frac{(1+r)(x+\rho )+\sigma z}{1+\rho }\\ \mathrm{and}{\Pi }_L& =\sigma .\end{array}\}$$ 2.1

The expected payoff of cooperators in equation (2.1) can be understood as follows: the probability of a cooperator interacting with another cooperator from the entire population is (x + ρ)/(1 + ρ); similarly, the probability of a cooperator interacting with a defector from the entire population is y/(1 + ρ), and the probability of a cooperator interacting with a loner from the entire population is z/(1 + ρ). Thus, according to the payoff matrix defined in table 1, the expected payoff of a cooperator should be (x − yr + zσ + ρ)/(1 + ρ). The expected payoff of defectors and loners can be similarly understood in the same way.

We adopt the pairwise comparison rule, where the imitation probability depends on the payoff difference between two randomly selected players. If the randomly selected players choose the same action, nothing happens. Otherwise, the probability that action i replaces action j is

$$P_{\hspace{0.17em}j\to i}=\frac{1}{(1+e^{({\Pi }_i-{\Pi }_{\hspace{0.17em}j})/\kappa })},$$ 2.2

where $i\ne j\in \{C,D,L\}$, and κ⁻¹ > 0 represents the imitation strength, indicating the degree to which players base their decisions on comparing payoffs [33]. This concept is similar to the widely used notion of selection strength in the literature on finite populations [34–36], which emphasizes the crucial role of selection strength in determining the evolutionary outcomes. For the sake of simplicity, we adopt a fixed value of κ = 0.1 throughout the main text. Additionally, in appendix A, we explore the impact of varying levels of imitation strength on evolutionary outcomes. The evolutionary dynamics of the well-mixed and infinite population under the imitation rule are represented by

$$\begin{array}{rl}\dot{x}& =\frac{2}{1+\rho }((x+\rho )y{P}_{D\to C}+(x+\rho )z{P}_{L\to C}-xy{P}_{C\to D}-xz{P}_{C\to L}),\\ \dot{y}& =\frac{2}{1+\rho }(xy{P}_{C\to D}+yz{P}_{L\to D}-(x+\rho )y{P}_{D\to C}-yz{P}_{D\to L})\\ \mathrm{and}\dot{z}& =\frac{2}{1+\rho }(xz{P}_{C\to L}+yz{P}_{D\to L}-(x+\rho )z{P}_{L\to C}-yz{P}_{L\to D})\end{array}\}.$$ 2.3

We use the Runge–Kutta method to generate numerical solutions for equation (2.3).

To generate results for networked populations, we employed Monte Carlo simulations. These simulations involved the following steps: first, each node was designated as a bot with probability ρ, while the probability that each node was designated as a normal player was 1 − ρ. For the population of normal players, each player was randomly assigned as a cooperator (C), defector (D) or loner (L). Normal players and bots residing on networks received their payoffs by interacting with all direct neighbours. In each round of the game, a randomly selected normal player i updated its action by imitating the action of a randomly selected neighbour j, which could be either a bot or a normal player. The probability of imitation was determined by the pairwise Fermi function (PW-Fermi), as defined in equation (2.2).

2.4. Simulation settings

The game dynamics for a single Monte Carlo step were executed L * L times, allowing each player to modify their action at least once on average. We set the value of L at 100 for the entire study. In order to achieve an equilibrium state, all simulations were typically conducted for 50 000 steps. This ensured that there was no obvious increasing or decreasing trend for the fractions of each actor. The proportion of each action was then calculated by averaging the results of the last 5000 steps, providing a more stable and accurate representation of the equilibrium state. To reduce the impact of initial conditions, we calculated the data by averaging the results of 20 independent runs. Our focus is solely on the impact of bots on the evolution of cooperation among normal players. Therefore, when calculating the fractions of each action in the population, we disregard the fraction of bot players.

3. Results

3.1. Well-mixed populations

Loners receive a small, but positive payoff by opting out and generate the same payoff for their opponent. This definition allows for the coexistence of cooperators, defectors and loners through the effect of cyclic dominance in well-mixed populations [13]. However, the cooperation-sustaining effect by loners is only possible in the public goods game, and not in Prisoner’s Dilemma game [37]. In fact, Prisoner’s Dilemma game can lead to a pure loner state in well-mixed populations. Although the cyclic dominance pattern can still exist along the edges (i.e. C giving way to D, D giving way to L, who gives way to C), the invasion direction from L to C requires a certain level of cooperation [38]. The addition of simple bots to the optional Prisoner’s Dilemma game can significantly alter the fate of cooperators. Rather than being entirely eliminated by defectors, cooperators can now thrive and even establish dominance over the entire population with the assistance of cooperative bots (figure 1).

Figure 1.

In well-mixed populations, the introduction of cooperative bots can facilitate the emergence of cooperation among ordinary players, whereas in the absence of bots, loners tend to dominate the entire population. Shown are the fraction of cooperation (a), defection (b) and loner (c) at steady state, as a function of dilemma strength and density of bots in well-mixed situations. Here, bots were programmed to consistently choose cooperation and not change their action over time. The imitation strength was fixed at κ⁻¹ = 10.

Introducing cooperative bots does not change the payoff scheme for human players, but it does increase the chances of human players encountering other cooperators. This higher likelihood of encountering cooperators helps to reduce the advantage that loners have over cooperators and defectors. However, it also creates an opportunity for free-riding defectors to exploit the system. If human players place less emphasis on material gains when updating their actions (in a weak imitation scenario), natural selection is likely to favour the dominance of cooperation (figure 2). Conversely, in a strong imitation scenario, where players tend to copy the behaviour of those who are most successful, the entire population would be dominated by defectors (figure 2). The introduction of defective or loner bots cannot mitigate the advantage that loners hold over cooperators and defectors in the absence of bots. Consequently, these two types of bots do not affect the emergence of cooperation, as illustrated in the top and middle rows of figure 3. If the actions of bot players were programmed to choose cooperation, defection and loner with equal probability, then the diversity of bot actions reduces the cooperation-promoting effect by cooperative bots (refer to the bottom row of figure 3).

Figure 2.

The impact of cooperative bots on promoting cooperation in well-mixed populations depends on the strength of imitation. In weak imitation scenarios, cooperative bots can effectively increase cooperation rates, but in strong imitation scenarios, defection tends to dominate. Shown are the proportions of cooperation, defection and loners relative to imitation strength κ⁻¹, with fixed parameters of r = 0.2, and ρ-values of 0 (a), 0.2 (b), 0.6 (c) and 1 (d).

Figure 3.

Introducing defective or loner bots into the optional Prisoner’s Dilemma game does not have any effect on the emergence of cooperative behaviour in well-mixed populations, and the diversity of bot actions reduces the promotion effect of cooperation by cooperative bots. The three panels from top to bottom correspond to different bot behaviours: (i) always defect (marked as $Bots\mathrm{\_}\hspace{0.17em}D$), (ii) always loner (marked as $Bots\mathrm{\_}\hspace{0.17em}L$), and (iii) random choice among cooperation, defection and loner with equal probability (marked as $Bots\mathrm{\_}\hspace{0.17em}ALL$). The imitation strength was fixed at κ⁻¹ = 10.

While the pure loner state of the optional Prisoner’s Dilemma game is observed in well-mixed populations, regular lattices facilitate the stable coexistence of cooperators, defectors and loners through cyclic dominance, in which cooperation give way to defection, who give way to loner, who in turn give way to cooperation [37]. This difference prompts further investigation into how the addition of simple bots influences the evolution of cooperation in structured populations.

3.2. Structured populations

Consistent with the cooperation-promoting effect by cooperative bots in well-mixed populations, introducing cooperative bots to the optional Prisoner’s Dilemma game on a regular lattice still promotes cooperation among human players under the weak imitation scenarios (figure 4). However, the required imitation strength threshold to achieve cooperation prosperity decreases. For example, introducing cooperative bots can decrease cooperation when the imitation strength equals κ = 10⁻¹ (as shown in the top row of figure 5), whereas the same imitation strength value ensures increased cooperation in well-mixed populations. A similar cooperation-decreasing phenomenon can be also observed when using defective bots or when bots randomly choose among cooperation, defection and being a loner (as shown in the middle and bottom rows of figure 5). In contrast with the results on well-mixed population in which introducing loner bots does not have any impact on the emergence of cooperation, the cooperation-promoting phenomenon can be observed if bots are programmed to always act as loners and not change their behaviour over time, as demonstrated in figure 6. This figure shows the fraction of cooperation (a), defection (b) and loner (c) as a function of dilemma strength and the density of loner bots, obtained on a regular lattice using Monte Carlo simulations. Interestingly, introducing loner bots actually increases the level of cooperation over the entire range of dilemma strength. Moreover, there exists an optimal density of loner bots at which cooperation is maximized, and further increasing the density of loner bots results in the breakdown of cooperation. In addition, the cooperation-promoting effect by loner bots is restricted in strong imitation scenario, while loners tend to dominant the population in the weak imitation scenario (figure 7). To reveal the potential mechanism behind the effect of loner bots on cooperation, we examine the overall evolution of actor abundance along the temporal dimension, as well as the spatial and temporal dimensions by analysing the time-series of each actor and the evolutionary snapshots.

Figure 4.

The cooperation-promoting effect by cooperative bots in regular lattice still holds under weak imitation strength scenarios, while defectors tend to dominate the population under the strong imitation scenarios. Shown are the proportions of cooperation, defection, and loners relative to imitation strength κ⁻¹, with fixed parameters of r = 0.5, and ρ values of 0 (a), 0.2 (b), 0.4 (c) and 0.5 (d).

The dynamics of the three actors and their interactions can be seen in figure 8, where the fractions of cooperators, defectors and loners are plotted as a function of Monte Carlo steps for different values of $\rho \hspace{0.17em}:\hspace{0.17em}0,\hspace{0.17em}0.1,\hspace{0.17em}0.4\hspace{0.17em}\mathrm{and}\hspace{0.17em}0.5$. This figure illustrates how the behaviour of one actor affects the others, and provides insight into the system’s overall evolution. When ρ = 0, our model reduces to the traditional optional Prisoner’s Dilemma game where cooperation coexists with defection and loner through cyclic dominance. As shown in figure 8a, defectors initially reach their peak and then give way to loners, who in turn give way to cooperators. This process repeats until the system reaches its stable state, where the abundances of each actor fluctuate around their average value. Similar cyclic dominance occurs for a small density of loner bots (i.e. ρ = 0.1), but the stationary defection level decreases (figure 8b). However, for a large density of bots (i.e. ρ = 0.4 and ρ = 0.5), the cyclic dominance phenomenon disappears, and loners initially reach their peak followed by cooperators, while defectors decrease and go extinct directly (figure 8c,d). The evolutionary dynamics at ρ = 0.4 and ρ = 0.5 are similar, but the survival periods of defection in the former situation are longer than those in the latter case.

Figure 9 shows the evolutionary snapshots under the same parameter values as figure 8. In this figure, cooperators, defectors and loners are represented by red, blue, and green, respectively. Bot players are represented by dark green. When ρ = 0, our model is reduced to the traditional optional Prisoner’s Dilemma game, and we observe the expected travelling waves of cooperators, followed by defectors, then loners and finally cooperators again (top row of figure 9). Introducing a small density of loner bots does not significantly affect the travelling waves of cooperators but reduces the chasing ability of defectors, as shown in the second row of figure 9. Here, loner bots tend to be surrounded by compact cooperative clusters, and defective clusters are much smaller than those observed in the scenario without bots. However, when the density of loner bots is high (i.e. ρ = 0.4), defectors struggle to survive the invasion of loners (see the second column and third row of figure 9) and gradually go extinct. Without the threat of defectors, cooperators take over the territory of loners until some lucky loners are surrounded by loner bots. If the density of loner bots is too large (i.e. ρ = 0.5), although defectors can still be eliminated by loners, cooperation cannot spread further in the population since cooperative clusters can easily be fully surrounded by loner bots (bottom row in figure 9), leading to a decreased cooperation level compared with the situation with ρ = 0.4.

4. Discussion

In this paper, we investigated the impact of simple bots on the evolution of cooperation in the optional Prisoner’s Dilemma game, using four types of bots and two representative populations. Despite their simplicity and lack of intelligent algorithms, these bots play a crucial role in determining the evolution of cooperation. Our findings demonstrate that the introduction of cooperative bots can enhance the emergence of cooperation among ordinary players in both well-mixed populations and regular lattices under weak imitation scenarios. However, regular lattices require a weaker imitation strength scenario to achieve the same level of cooperation. In contrast to the results on well-mixed populations, where adding loner bots does not impact the emergence of cooperation, we observed that introducing loner bots promotes cooperation on a regular lattice under a strong imitation scenario. Specifically, we found that an optimal density of loner bots exists, at which the highest level of cooperation is achieved; otherwise, cooperation levels decrease. Introducing defective bots does not significantly impact the evolution of cooperation in well-mixed populations, and diversity in bot actions reduces the cooperation-promoting effect by cooperative bots. Similar cooperation-devastating results can be observed for defective bots or considering the diversity of bot actions. Our results, taken together, indicate that the level of cooperation can be maximized by managing the behaviour of simple bots.

Voluntary participation in public good games can be an effective way to promote cooperation in well-mixed populations, even in the absence of other reciprocity mechanisms like reputation, iterated interactions or group selection. However, it is important to note that cooperation is only likely to emerge if the group size is larger than two; otherwise, pairwise interactions lead to the homogeneous loner state [39]. While we focused on the simplest form of the game with only two players and did not explore more complex multiplayer scenarios in this paper, the results on regular lattices may still hold, as the cyclic dominance effect is valid for both pairwise and multiplayer games [14]. It is worth noting that the results on well-mixed populations may differ significantly, as periodic oscillations can only be observed for multiplayer games [39]. Therefore, future research should investigate how the existence of simple bots affects period oscillations and which types of bots can promote cooperation in well-mixed populations, particularly in the context of public good games.

The critical assumptions in our human-machine framework are the one-shot game and anonymous interaction. It is important to clarify that a one-shot game differs from a strict one-round setting game. In a one-round game, the game can occur only once for all players, but it can still be played multiple times in a one-shot game setting as long as the opponents are strangers who have never met before. Unlike a strict one-round game, where players cannot revise their actions, players in a one-shot game can revise their actions based on their past experiences. This feature of the one-shot setting makes it possible for bot players to improve the cooperation level among human players, as experimental evidence shows that cooperative behaviour can cascade among human players in social networks [40]. The existence of bots that always cooperate and do not change their actions over time may increase the probability that human players pass their obtained help to new strangers. This is possible under the weak imitation scenario, in which human players may still pass on this help even if they know they may endure some losses. However, our results are not limited to the weak imitation scenario since loner bots can realize high cooperation levels among human players in structured populations under the strong imitation scenario. These results indicate that cooperation can be improved through effective bot design management by considering specific population structures or the features of human learning moods.

The second anonymous interaction assumption posits that there are no reciprocity mechanisms such as indirect reciprocity, direct reciprocity, reputation or costly signals [3,6,41–43]. This assumption implies that both human and bot players are unable to use information about their opponents in their decision-making process. Additionally, human players are unaware of the presence of bot players. These assumptions ensure that human players update their actions similarly when playing against either human or bot players. However, these assumptions also limit the intelligence of bot strategies as they lack access to information about their opponents. Although the past experiences of bot players are available in our framework, we deliberately disregarded them during the design of the bots’ strategies. Our primary focus was to investigate the impact of simple bots on cooperation, and considering the past experiences of bots would have introduced unnecessary complexity to the analysis.

However, it is worth noting that future studies could explore the incorporation of reinforcement learning methods, as outlined in studies such as [44,45], to leverage the past experiences of bots and develop more sophisticated strategies. Additionally, relaxing the strict anonymity of the game scenario while still focusing on the one-shot game scenario by allowing the presence of information sources, such as reputation [42], prior commitment [46], emotion [47] and intention [48,49], would provide a more realistic and nuanced understanding of the role of bots in promoting cooperation.

While our assumptions may not hold in more complex and realistic game situations where human–machine interactions are not one-shot but repeated, or where the reputation of opponents is also available, our results offer valuable insights into the emergence of cooperation. Nonetheless, addressing these more complex scenarios would require the development of more intelligent machines. Our findings can serve as a valuable starting point for future studies in this direction.

Appendix A. Game dynamics for scenarios with defective, loner and randomized bots

Remark A.1 (Defective bots). —

Let x, y and z denote the fractions of cooperators (C), defectors (D) and loners (L) in a well-mixed and infinite population, such that x + y + z = 1. We assume the fraction of bots is ρ, and the bots are designed always to choose D and never change their action. The total population density is then 1 + ρ, and the expected payoffs of each of the actors are as follows:

$$\begin{array}{rl}{\Pi }_C& =\frac{x-(y+\rho )r+z\sigma }{1+\rho },\\ {\Pi }_D& =\frac{(1+r)x+z\sigma }{1+\rho }\\ \mathrm{and}{\Pi }_L& =\sigma .\end{array}\}$$ A 1

We adopt the pairwise comparison rule, where the imitation probability depends on the payoff difference between two randomly selected players. If the randomly selected players choose the same action, nothing happens. Otherwise, the probability that action i replaces action j is

$$P_{\hspace{0.17em}j\to i}=\frac{1}{(1+e^{({\Pi }_{\hspace{0.17em}j}-{\Pi }_i)/\kappa })}.$$ A 2

Where $i\ne j\in \{C,D,L\}$, and κ⁻¹ is the imitation strength such that κ⁻¹ > 0. The evolutionary dynamics of the well-mixed and infinite population under the imitation rule are represented by

$$\begin{array}{rl}\dot{x}& =\frac{2}{1+\rho }(xy{P}_{D\to C}+xz{P}_{L\to C}-x(y+\rho )P_{C\to D}-xz{P}_{C\to L}),\\ \dot{y}& =\frac{2}{1+\rho }(x(y+\rho )P_{C\to D}+(y+\rho )z{P}_{L\to D}-xy{P}_{D\to C}-yz{P}_{D\to L})\\ \mathrm{and}\dot{z}& =\frac{2}{1+\rho }(xz{P}_{C\to L}+yz{P}_{D\to L}-xz{P}_{L\to C}-(y+\rho )z{P}_{L\to D}).\end{array}\}$$ A 3

Figure 5.

Introducing cooperative or defective bots into the optional Prisoner’s Dilemma game bring little impact on the emergence of cooperation in a regular lattice, and the diversity of bot actions reduces the promotion effect of cooperation by loner bots. The three panels from top to bottom correspond to different bot behaviors: (i) always defect (marked as $Bots\mathrm{\_}D$), (ii) always loner (marked as $Bots\mathrm{\_}L$), and (iii) random choice among cooperation, defection, and loner with equal probability (marked as $Bots\mathrm{\_}ALL$). The imitation strength was fixed at κ⁻¹ = 10.

Figure 6.

In a regular lattice, the introduction of loner bots can facilitate the emergence of cooperation among ordinary players, whereas too high a density of loner bots results in a breakdown of cooperation. Shown are the fraction of cooperators, defectors and loners among ordinary players as a function of dilemma strength and density of bots in a regular lattice. Here bots were programmed to consistently choose loner action and not change their action over time. The imitation strength was fixed at κ⁻¹ = 10.

Figure 7.

The impact of loner bots on promoting cooperation in a regular lattice depends on the strength of imitation. In scenarios with weak imitation, loners tend to dominate the population, whereas in scenarios with strong imitation, loner bots can effectively increase cooperation rates. Shown are the proportions of cooperation, defection and loners relative to imitation strength κ⁻¹, with fixed parameters of r = 0.5, and ρ values of 0 (a), 0.2 (b), 0.4 (c) and 0.5 (d).

Figure 8.

The introduction of loner bots can disrupt the cyclic dominance that typically sustains cooperation among cooperators, defectors and loners in a regular lattice. However, adding loner bots can improve the level of cooperation among human players by reducing the number of defectors. Shown are the fractions of cooperators, defectors and loners as a function of simulation steps under different values of density of bots. The dilemma strength was fixed at r = 0.5, while the values of ρ were set to 0, 0.1, 0.4 and 0.5 for panels (a–d).

Figure 9.

Evolutionary snapshots reveal that a high density of loner bots can create a barrier for the spread of cooperation, leading to the breakdown of cooperative behaviour. Shown are the evolutionary snapshots taken at different values of bot intensity, with the dilemma strength fixed at r = 0.5 and the value of ρ fixed at 0, 0.1, 0.4 and 0.5 from top to bottom rows. The snapshots in the first column were taken at the initial step, where actions were distributed randomly for both human players and bot players. The snapshots in the remaining columns were taken at different time steps, making the figure as illustrative as possible.

Remark A.2 (loner bots). —

Let x, y and z denote the fractions of cooperators (C), defectors (D) and loners (L) in a well-mixed and infinite population, such that x + y + z = 1. We assume the fraction of bots is ρ, and the bots are designed always to choose L and never change their action. The total population density is then 1 + ρ, and the expected payoffs of each of the actors are as follows:

$$\begin{array}{rl}{\Pi }_C& =\frac{x-yr+(z+\rho )\sigma }{1+\rho },\\ {\Pi }_D& =\frac{(1+r)x+(z+\rho )\sigma }{1+\rho }\\ \mathrm{and}{\Pi }_L& =\sigma .\end{array}\}$$ A 4

We adopt the pairwise comparison rule, where the imitation probability depends on the payoff difference between two randomly selected players. If the randomly selected players choose the same action, nothing happens. Otherwise, the probability that action i replaces action j is

$$P_{\hspace{0.17em}j\to i}=\frac{1}{(1+e^{({\Pi }_{\hspace{0.17em}j}-{\Pi }_i)/\kappa })},$$ A 5

where $i\ne j\in \{C,D,L\}$, and κ⁻¹ is the imitation strength such that κ⁻¹ > 0. The evolutionary dynamics of the well-mixed and infinite population under the imitation rule are represented by

$$\begin{array}{rl}\dot{x}& =\frac{2}{1+\rho }(xy{P}_{D\to C}+xz{P}_{L\to C}-xy{P}_{C\to D}-x(z+\rho )P_{C\to L}),\\ \dot{y}& =\frac{2}{1+\rho }(xy{P}_{C\to D}+yz{P}_{L\to D}-xy{P}_{D\to C}-y(z+\rho )P_{D\to L})\\ \mathrm{and}\dot{z}& =\frac{2}{1+\rho }(x(z+\rho )P_{C\to L}+y(z+\rho )P_{D\to L}-xz{P}_{L\to C}-yz{P}_{L\to D}).\end{array}\}$$ A 6

Remark A.3 (randomized bots). —

Let x, y and z denote the fractions of cooperators (C), defectors (D) and loners (L) in a well-mixed and infinite population such that x + y + z = 1. We assume that bots have equal probabilities to be C, D and L. Thus, the fractions of bots with action C, with action D, and with action L are all ρ/3. The total population density is 1 + ρ. Therefore, the expected payoffs of each of the actors are as follows:

$$\begin{array}{rl}{\Pi }_C& =\frac{x+(\rho /3)-(y+(\rho /3))r+(z+(\rho /3))\sigma }{1+\rho },\\ {\Pi }_D& =\frac{(1+r)(x+(\rho /3))+(z+(\rho /3))\sigma }{1+\rho }\\ \mathrm{and}{\Pi }_L& =\sigma .\end{array}\}$$ A 7

We adopt the pairwise comparison rule, where the imitation probability depends on the payoff difference between two randomly selected players. If the randomly selected players choose the same action, nothing happens. Otherwise, the probability that action i replaces action j is

$$P_{\hspace{0.17em}j\to i}=\frac{1}{(1+e^{({\Pi }_{\hspace{0.17em}j}-{\Pi }_i)/\kappa })},$$ A 8

where $i\ne j\in \{C,D,L\}$, and κ⁻¹ is the imitation strength such that κ⁻¹ > 0. The evolutionary dynamics for a well-mixed and infinite population under the imitation rule are represented by

$$\begin{array}{rl}\dot{x}& =\frac{2}{1+\rho }((x+\frac{\rho }{3})y{P}_{D\to C}+(x+\frac{\rho }{3})z{P}_{L\to C}-x(y+\frac{\rho }{3})P_{C\to D}\\ & -x(z+\frac{\rho }{3})P_{C\to L}),\\ \dot{y}& =\frac{2}{1+\rho }(x(y+\frac{\rho }{3})P_{C\to D}+(y+\frac{\rho }{3})z{P}_{L\to D}-(x+\frac{\rho }{3})y{P}_{D\to C}\\ & -y(z+\frac{\rho }{3})P_{D\to L})\\ \mathrm{and}\dot{z}& =\frac{2}{1+\rho }(x(z+\frac{\rho }{3})P_{C\to L}+y(z+\frac{\rho }{3})P_{D\to L}-(x+\frac{\rho }{3})z{P}_{L\to C}\\ & -(y+\frac{\rho }{3})z{P}_{L\to D}).\end{array}\}$$ A 9

Data accessibility

All scripts used in this study are openly accessible through https://github.com/Steven-Shen91/CDL-model-with-simple-bots.

Authors' contributions

H.G.: formal analysis, methodology, validation, visualization, writing—original draft; G.S.: data curation, formal analysis, investigation, methodology, visualization, writing—original draft; C.S.: conceptualization, formal analysis, investigation, methodology, supervision, validation, visualization, writing—original draft, writing—review and editing; J.T.: conceptualization, supervision, validation, writing—review and editing.

All authors gave final approval for publication and agreed to be held accountable for the work performed therein.

Conflict of interest declaration

We declare we have no competing interests.

Funding

This research was supported by (i) the JSPS Postdoctoral Fellowship Program for Foreign Researchers (grant no. P21374), and an accompanying Grant-in-Aid for Scientific Research from JSPS KAKENHI (grant no. JP 22F31374), and the National Natural Science Foundation of China (grant no. 11931015) to C.S. as a co-investigator, (ii) MITSUI2207 from JEES awarded to G.S. as a Scholarship (iii) the grant-in-Aid for Scientific Research from JSPS, Japan, KAKENHI (grant no. JP 20H02314) awarded to J.T.