Heterogeneity in evolutionary games: an analysis of the risk perception

Abstract

In this study, we analyse the relationship between heterogeneity and cooperation. Previous investigations suggest that this relation is non-trivial, as some authors found that heterogeneity sustains cooperation, while others obtained different results. Among the possible forms of heterogeneity, we focus on the individual perception of risks and rewards related to a generic event, which can appear in a number of social and biological systems. The modelling approach is based on the framework of evolutionary game theory. To represent this kind of heterogeneity, we implement small and local perturbations on the pay-off matrix of simple two-strategy games, such as the Prisoner’s Dilemma. So, while usually the pay-off is considered to be a global and time-invariant structure, i.e. it is the same for all individuals of a population at any time, in our model its value is continuously affected by small variations, in both time and space (i.e. position on a lattice). We found that such perturbations can be beneficial or detrimental to cooperation, depending on their setting. Notably, cooperation is strongly supported when perturbations act on the main diagonal of the pay-off matrix, whereas when they act on the off-diagonal the resulting effect is more difficult to quantify. To conclude, the proposed model shows a rich spectrum of possible equilibria, whose interpretation might offer insights and enrich the description of several systems.

1. Introduction

The inherent heterogeneity of human beings, as well as that of other animals, reflects the variety of behaviours observable in groups and communities of individuals.

However, by using a well-defined set of categories to describe human behaviours [1], such as conformity, rationality, zealotry and so on, the complexity of a social system can be reduced and addressed by appropriate models. The modern field of sociophysics (also called social dynamics) [2–10] represents a class of models whose goal is studying social phenomena and human behaviours in mathematical terms.

Therefore, following an approach similar to that adopted in physics, investigations in sociophysics usually define simple models, based on a few degrees of freedom, and solve them by analytical calculations or numerical simulations. This approach is extremely successful, for instance, when studying the motion of particles and other classic physical systems.

On the other hand, the selection of relevant properties, and the granularity of the corresponding variables, might entail the loss of significant information. For instance, when studying human psychology, behaviours and general social phenomena, such loss can be so relevant that psychologists and sociologists might be reluctant to accept results obtained by a ‘too simple’ model, even if based on an elegant set of equations. In saying ‘too simple’, we refer to those models that oversimplify a real scenario, or phenomenon, becoming unable to represent it properly. It is worth mentioning the case of the voter model [11], whose limits and potentialities for describing real political scenarios have been reported in [12].

Let us proceed with our focus on heterogeneity, also called diversity. Despite being an almost ubiquitous property of social and biological systems [13], for the sake of simplicity many models can safely neglect it. On the other hand, sometimes further efforts to represent heterogeneity can lead to findings more valuable than expected (e.g. [14]). Also, since this property can refer to one or more aspects of a system, seeing it as beneficial or detrimental becomes a fairly relative matter. For instance, networks [15] can be heterogeneous or homogeneous in relation to their topology, like scale-free [16] and classic random graphs [17], respectively. Here, both the transmission of information and viral spreading are slower in networks with homogeneous topology. So, in the first case, heterogeneity is clearly beneficial, while in the other it is detrimental. Interesting observations arise also when considering animal cell replication, which can be realized by mitosis and meiosis. Notably, the latter increases the genetic diversity (i.e. heterogeneity) [18] of a population. In business and science, heterogeneity of skills can make a group more effective (e.g. [19]); in ecological systems, it could affect the size and the stability of a group [20]; and in social systems, heterogeneity (e.g. cultural differences) might constitute the cause of a conflict [21]. Recently, the existence of a deep relationship between cultural differences and cooperation in human societies has been experimentally confirmed [22]. It is worth mentioning that cooperation still represents an important scientific challenge, and, despite a huge literature focused on its dynamics (e.g. [23–27]), a lot of questions around it still require a proper answer. Hence, clarifying further whether and how some forms of heterogeneity can support cooperative behaviours might be relevant.

To this end, we concentrate on the heterogeneity of the individual perception of risk (or reward) associated with a generic event, i.e. we study an aspect of humans and, probably, also of other animals that can manifest in a variety of common scenarios. In addition, since our investigation aims to understand the relationship between this psychological aspect and cooperative behaviours, we use the framework of evolutionary game theory (hereinafter EGT) [28,29]. The latter allows the study of social dynamics, biological phenomena and many other evolutionary systems [30]. In addition, heterogeneity has already been analysed under its lens and a variety of results, sometimes controversial, have been obtained [31–34]. In our view, an increasing interest in the relationship between heterogeneity and cooperation reflects its potential relevance in many contexts and domains. This can also be observed in the influx of recent works trying to find a general framework to explain the effects of perception diversity [14,34–47].

Thus, motivated by a challenging debate, we analyse the heterogeneity resulting from varying the profits that rational individuals try to achieve in simple games. In saying rational [48], we assume that all individuals act to maximize their profit, so that any variation in the pay-off structure can actually influence their actions. Accordingly, in the proposed model, individuals play games such as the Prisoner’s Dilemma (PD), whose pay-off matrix is affected by small random perturbations. As one can note, here the pay-off is not a global and constant property of the system, since small random perturbations can last a single interaction, and can affect a single individual at a time. In doing so, we represent a population whose perception of risk/reward is not defined by a static and global parameter but, instead, by a value that fluctuates around an average.

We remark that many real scenarios involve people having a different risk perception of the same event. Notably, from financial markets to scientific organizations, the psychology of risk perception plays a fundamental role, with a spectrum of behaviours (e.g. group polarization) that can become even dangerous. A relevant case can be found in [49], which describes the failure of the NASA Challenger project as an example of a dangerously reduced perception of risk. Hence, beyond being a common aspect of the real world, heterogeneity of risk perception is also quite relevant and difficult to predict or quantify. We emphasize that the perturbation approach developed here finds connections also with new research on potential games [50] that use the operator formalism from quantum mechanics to relate the pay-off matrix to the Hamiltonian of a system [51].

The following sections describe in detail the proposed model, show the results of numerical simulations and, eventually, discuss our findings and future developments.

2. Model

In the proposed model, we consider two-strategy games whose agents can either cooperate (C) or defect (D). Mutual cooperation yields a pay-off R (reward), while mutual defection yields P (punishment). Also, a defector receives a pay-off T (temptation) when it interacts with a cooperator that, in turn, receives a pay-off S, known as the sucker’s pay-off [30]. The pay-off matrix combines, in a single structure, all the above-mentioned pay-offs, so, depending on their value, it relates to a specific game (e.g. the PD). Usually, this matrix is constant over time and it contains the same entries for all agents. In order to represent the heterogeneity of risk perception, we introduce small perturbations into this matrix.

Notably, such perturbations act at the local level. Hence, at the same instant, two individuals might have a slightly different pay-off matrix; also, one individual might find two slightly different pay-off matrices in two different game iterations. In particular, perturbations (i.e. ϵ) are implemented via a uniform distribution with range D (our control parameter) and they are not cumulative. Moreover, it is important to add that the resulting heterogeneity has zero average, i.e. perturbations should actually give a null effect in the long term.

We consider three different configurations: FP (full perturbation, i.e. acting on all pay-off entries), MDP (main diagonal perturbation, i.e. acting only on the main diagonal of the pay-off matrix) and ODP (off-diagonal perturbation). Note that, by doing so, there are no asymmetries between the strategies regarding the perturbations, i.e. for any configuration both strategies, C and D, will always have at least one of their possible pay-off values perturbed. The considered configurations do not cover the whole spectrum of possibilities as, for instance, another configuration could be based on perturbations acting only in the R and S entries, and would result in a population where only cooperators would be affected. The first configuration (FP) has the following pay-off matrix:

$$\begin{array}{cc}& \begin{array}{cc}C\hspace{0.17em}\hspace{0.17em}& \hspace{0.17em}\hspace{0.17em}D\end{array}\\ \begin{array}{c}C\\ D\end{array}& \left[\begin{array}{cc}R+{ϵ}_R& S+{ϵ}_S\\ T+{ϵ}_T& P+{ϵ}_P\end{array}\right]\end{array};$$ 2.1

the MDP configuration is defined as

$$\begin{array}{cc}& \begin{array}{cc}C\hspace{0.17em}\hspace{0.17em}& \hspace{0.17em}\hspace{0.17em}D\end{array}\\ \begin{array}{c}C\\ D\end{array}& \left[\begin{array}{cc}R+{ϵ}_R& S\\ T& P+{ϵ}_P\end{array}\right]\end{array};$$ 2.2

eventually, the ODP configuration reads

$$\begin{array}{cc}& \begin{array}{cc}C\hspace{0.17em}\hspace{0.17em}& \hspace{0.17em}\hspace{0.17em}D\end{array}\\ \begin{array}{c}C\\ D\end{array}& \left[\begin{array}{cc}R& S+{ϵ}_S\\ T+{ϵ}_T& P\end{array}\right]\end{array},$$ 2.3

where T ∈ [0, 2] and S ∈ [ − 1, 1]. Without loss of generality, we set R = 1 and P = 0 [52]. Although [R, P] are fixed, the perturbation can make said values fluctuate for each individual and at each time (i.e. iteration). The parametrization adopted in the above pay-off matrices spans four different classes of games in the [T, S] parameter space: the PD, snowdrift (SD), stag hunt (SH) and harmony (HG) games [7,30] (figure 1). We stress that the actual parameter space has four dimensions, and the perturbations will act on all four parameters [T, S, P, R] in an uncorrelated manner.

Figure 1.

T × S × R × P (four-dimensional, 4D) parameter space with R = 1, P = 0, spanning four classes of games. The P dimension is represented only in the Cartesian coordinate system (outside the parameter space, on the left-hand side). The pay-off fluctuation can act over [T, S, P, R] simultaneously and uncorrelated. This makes any given game able to fluctuate around a (4D) box with 2ϵ size edges, centred on the original game. Note that local fluctuations can lead agents to (locally) play different classes of games depending on the fluctuation strength. (Online version in colour.)

It is a well-known result that the Nash equilibrium of a game is not changed by the sum of a constant term in the pay-off matrix or by its multiplication by a constant [29,30]. However, it is important to remember that we are adding random numbers in each pay-off entry, and not global constants. So, the actual individual pay-off value will change in time and for each agent. Accordingly, although the average value of the perturbation is zero, locally the game class can change from time to time [33,53,54]. Even so, the average game will stay centred in the chosen initial [T, S] parameters. For the population dynamics, we implement the usual imitative update rule weighted by the Fermi distribution [48], in a spatially distributed population with the square lattice topology. At each time step, one agent, i, updates its strategy by comparing its pay-off with one randomly chosen first neighbour, j. Then, agent i adopts the strategy of agent j with probability

$$p(\mathrm{\Delta }{u}_{ij})=\frac{1}{1+{\text{e}}^{-(u_j-u_i)/k}},$$ 2.4

where k is the irrationality level [30] and u_i represents the pay-off of agent i. We set k = 0.1 for all simulations unless stated otherwise.

3. Results

We begin with a simplified analytical model of a mixed game setting and try to generalize it to the perturbation pay-off scenario. As shown in [55], the random mixing of two fixed different games (i.e. pay-off matrices) in the context of the well-mixed population should be identical to the evolution of a population using the average pay-off matrix in the mean-field approximation (note that this is not necessarily true for spatially distributed systems). We can generalize this result for the case of infinite discrete games randomly mixed, which represents our model of perturbed pay-offs. Suppose that, at time t, a system is characterized by the strategy configuration {s} = {s₁, s₂, …, s_N}, where s_i is the strategy of agent i, and by the pay-off matrix assignment configuration {g(t)} = {g₁₂(t), g₁₃(t), …}. Note that the variable g_ij(t) indicates which game agents i and j are playing at time t. In our specific case, g_ij(t) = G + ϵ_ij(t), where G is the fixed usual pay-off matrix, and ϵ_ij(t) is the perturbation that is randomly drawn and has zero average. Let P({s}, {g}, t) be the probability of finding the system in configuration {s} and {g} at time t. The time evolution of this system is given by the master equation,

$$\begin{array}{rl}\frac{\text{d}}{\text{d}t}P(\{s\},\{g\},t)& =\sum _{{\{s\}}^{\prime },{\{g\}}^{\prime }}P(s^{\prime },g^{\prime },t)W(s^{\prime },g^{\prime }\to s,g)\\ & -P(s,g,t)W(s,g\to s^{\prime },g^{\prime }),\end{array}$$ 3.1

where W(s′, g′ → s, g) is the transition rate from state {s}, {g}′ to state {s}, {g}.

Supposing that the strategy variables {s} are statistically independent of the game variables {g}, and that the transition rates are linear functions of the pay-offs, we can use the main result from [55] to obtain

$${⟨\frac{\text{d}}{\text{d}t}P(\{s\},\{g\},t)⟩}_g=\frac{\text{d}}{\text{d}t}P(\{s\},⟨g⟩,t).$$ 3.2

That is, the average evolution of a population playing a set {g} of randomly mixed games will undergo the usual evolution of a single population playing the equivalent average pay-off matrix 〈g〉.

We stress that this initial result only applies to the mean-field approximation. Nevertheless, it is useful for comparison with our model, since the average pay-off matrix of the perturbed game is equal to the unperturbed matrix. Analytical results predict that, on average, small random perturbations should not change the evolution as long as the population is interacting without spatial structure and the update rule is linear. This is also in line with recent results reported in [56]. Our simulations using a well-mixed topology presented some small changes in the perturbed population evolution, with the perturbations enhancing cooperation. By contrast, the outcomes for the square lattice show strong and diverse emergent effects caused by the perturbations. We proceed by presenting the main results for the square-lattice topology.

To implement the numerical simulations, we adopt an asynchronous Monte Carlo protocol with a square lattice, Von Neumann neighbourhood and periodic boundary conditions with N = 10⁴ agents. We wait for the system to reach a dynamic equilibrium (around 10⁴ Monte Carlo steps) and average the values over the final 1000 steps. This is repeated for 50 different samples with the same parameters.

We begin by focusing on the final fraction of cooperators (ρ) near the phase transition in T_c = 1.04. For the sake of simplicity, we present the outcomes related to the weak PD case (i.e. S = 0). Figure 2 shows results achieved by small (i.e. D = 0.1) and medium (D = 0.3) perturbations, compared with the unperturbed case (D = 0). The main differences between the perturbed configurations and the unperturbed one happen in the region around 0.9 < T < 1.15. From figure 2, we see that all three kinds of perturbation settings can enhance the range of T where cooperation survives. Even more, we see that for strong perturbations the MDP model seems to be the most efficient in doing so, followed by the FP model. The general effects of the three different perturbation configurations become more and more similar as D → 0. This effect was consistently observed for other D-values. It is also worth pointing out that the extinction point of cooperation is very similar for all three configurations.

Figure 2.

Average final cooperation level as a function of T for the three different configurations, full perturbation (FP), main diagonal perturbation (MDP) and off-diagonal perturbation (ODP). Dashed lines represent small perturbation (D = 0.1) and solid lines represent medium perturbation (D = 0.3). Even small perturbations can sustain cooperation past the extinction point and larger noise strength can greatly enhance cooperative behaviour. Note however that the effects of each configuration are slightly different. (Online version in colour.)

Figure 3 presents the difference between the final cooperation level of the perturbed (ρ) and the unperturbed (ρ₀) case. Again, for simplicity, we present only the cases for D = 0.3 and D = 0.1. We report that simulations with intermediate D-values also present curves that change continuously. We can observe that the overall effect of the perturbation is to increase cooperation. Nevertheless, the ODP model can hinder cooperation for the small region 0.9 < T < 1.0. In figure 3, it is clear that the MDP model can have a greater impact on cooperation. What is more, if we consider that the FP model is the sum of the MDP and ODP perturbations, it becomes clear that the small negative values of ρ − ρ₀ observed for the weak perturbations in the FP model (red dashed line) in the region 0.9 < T < 1.0 are due to the off-diagonal perturbations. We also note that, while the FP model is the direct sum of the MDP and ODP model in terms of perturbations, the final cooperation level in the FP model is not just the sum between the final cooperation level of the ODP and MDP models. That is, as expected, the final cooperation behaves nonlinearly in the types of perturbations implemented.

Figure 3.

Difference between the final cooperation level of the perturbed (ρ) and non-perturbed (ρ₀) cases as a function of T. By analysing (ρ − ρ₀) we can filter out the effect of increasing T, leaving just the perturbation effect. Dashed lines represent small perturbation (D = 0.1) and solid lines represent medium perturbation (D = 0.3). While the FP and MDP settings increase cooperation, the ODP has a non-trivial effect, decreasing cooperation if T < 1 (HG region). (Online version in colour.)

In figure 4, we present the final cooperation level as we vary the perturbation strength for T = 1.04 (solid lines) and T = 1.1 (dotted lines) in the three configurations. This allows us to continuously visualize the effect of the perturbation strength. Note that, for both T values, the FP model generates more cooperation for small perturbation effects. However, the MDP model quickly overcomes this difference, and becomes the more efficient type of perturbation regarding the promotion of cooperation. The specific turning point varies with T in a non-trivial manner. Also, note that, while the ODP configuration has a very shallow slope, the other two configurations keep increasing ρ at a fast rate as D increases. We see again that the perturbation in the off-diagonal elements of the pay-off matrix is of little effect when compared with the main diagonal terms.

Figure 4.

Average cooperation level as a function of D for the three configurations using T = 1.04 (solid lines) and T = 1.1 (dotted lines). The cooperation-enhancing effect increases monotonously with the perturbation strength. Such an effect is mostly pronounced in the cooperation extinction point, T_c = 1.04. (Online version in colour.)

Despite our main focus on the behaviour of rational individuals, for the sake of completeness, here we briefly present the results achieved by continuously varying the level of irrationality (controlled by the associated temperature k). The results can be seen in figure 5. Notably, increasing k (i.e. adding some irrationality) one observes random fluctuations in the imitation process; however, it is important to realize how differently that variation affects our population in comparison with the effects resulting from random perturbations in the pay-off. The pay-off perturbation can increase cooperation for all the studied values of k, although, after a peak value, this effect begins to diminish as irrationality keeps increasing. We can also see that this increase is proportional to the perturbation strength. On the other hand, it is more difficult to compare the effects of each perturbation configuration. We see that, for low k-values, the most influential is the MDP configuration, followed by the FP and lastly the ODP. Nevertheless, for high k, their effect is inverted. In the context of the evolution of cooperation, scenarios with low irrationality are more interesting, since they lead to more cooperation, and as k becomes much larger than 1 the system just behaves randomly. In the region of k ∈ [0, 1], we see again that the diagonal perturbations are the most efficient route to promote cooperation. For low perturbations, however, the FP is a little more efficient.

Figure 5.

Final cooperation level versus the irrationality level k for the three different configurations at D = 0.3 (solid lines) and D = 0.1 (dashed lines). The solid black curve shows the unperturbed model, D = 0. Notice that, while fluctuations originating from varying irrationality (k) can boost cooperation, they are not correlated with the fluctuations resulting from pay-off diversity (D). (Online version in colour.)

We stress that the random perturbation effects on pay-off entries lead to outcomes that are different from randomness arising from the irrationality level k. Specifically, we ran simulations for a wide range of k (i.e. k ∈ (0, 3]) and, while the perturbation effects remained qualitatively similar, the pure effects of varying k are very different from those of the pay-off perturbation. This can be more clearly seen by comparing figures 5 and 4. Further interesting analyses, on the effects of irrationality, have also been reported in [57,58].

It is worth mentioning that pay-off perturbation changes population dynamics for very short Monte Carlo times. Remarkably, even earlier than 100 Monte Carlo steps the perturbation can change the population evolution. This effect has been observed in all three configurations. We also observe that lattice snapshots were not useful in characterizing the population behaviour with regard to pay-off perturbations. That is, while perturbations increase the overall cooperation level, snapshots of the perturbed and unperturbed populations are largely similar, with the only difference being in a greater density of cooperators for all perturbed configurations.

4. Conclusion

In social and biological systems, and in many other contexts, heterogeneity often shows up in different forms. Understanding the way it contributes to a specific process is not always intuitive or immediate. Also, the inherent heterogeneity of a system is frequently neglected for practical modelling reasons, and often this choice turns out to be convenient. However, the relationship between heterogeneity and cooperation is still a matter of open debate.

Notably, previous investigations addressing other forms of heterogeneity (e.g. [59–63]) suggested that its relationship with cooperation is absolutely relevant, whereas other authors obtained results leading to opposite, or different, views [31,64]. Therefore, this variety of findings turned the relationship between heterogeneity and cooperation into a stimulating debate. At this point, we also deem it relevant to mention a very recent investigation [22], related to large-scale forms of cooperation, reporting a clear connection between human cooperation and cultural differences. Looking at this scenario, in our view it is definitely relevant to identify the system parameters (and conditions) whose heterogeneity supports the emergence of cooperative behaviours, and to try to compare models that lead to (perhaps only apparent) conflictual outcomes. Notably, even a small variation in a configuration can change an equilibrium drastically.

So a further, perhaps somehow speculative, generalization of our view can be read as the need to identify parameters whose small variations can affect the strategy equilibrium, in the same way as in physics one aims to find quantities whose variation does not change the equations of motion. For the sake of clarity, we specify that, while Noether’s theorem is focused on the relation between symmetry and conservation, in this context, the opposite seems to be definitely more interesting, i.e. the need to identify those variables whose heterogeneity leads to a different equilibrium.

Let us now go back to the main question, i.e. is heterogeneity a beneficial ingredient for stimulating cooperative behaviours? Notably, we considered the heterogeneity of reward and risk perception for two main reasons. First, risk perception is an individual aspect of humans, and probably also of other animals, that influences the way individuals act in the real world. Second, the results can find applications also in other domains, such as biology and clinical research [37]. Thus, before discussing the main achievements, we deem it important to begin with a sort of claim; that is, our findings corroborate the idea that heterogeneity might influence cooperation. Then, how such influence takes place is something to clarify properly. Let us recall that, to represent this property (i.e. heterogeneity), we added small and local perturbations on the pay-off matrix of two-strategy games. Then, we observed a non-trivial, and partially unexpected, spectrum of outcomes. In particular, we analysed three cases, i.e. perturbations acting on the whole pay-off matrix, acting on the main diagonal and acting on the off-diagonal, separately. The effect of these perturbations has to be evaluated in the proximity of the phase transition of the system, i.e. within a narrow set of values of the pay-off matrix since far from this critical area the evolution of strategies is dominated by average pay-off values and is poorly affected by small variations. It is also worth remarking that perturbations act independently on each single entry of the pay-off matrix, i.e. perturbations are generated by four independent uniform distributions. So, in the first case (i.e. perturbations acting on the whole matrix), cooperation is clearly supported by heterogeneity. Then, considering each diagonal individually, i.e. the main diagonal and the off-diagonal, we found some non-trivial outcomes. In particular, perturbations acting only on the main diagonal are more effective in fostering cooperation than those acting on the whole matrix. On the other hand, perturbations acting only on the off-diagonal support defection at the beginning of the phase transition, but support cooperation after a given value. Therefore, the former case shows an unexpected behaviour, and we deem it to also deserve further investigations in order to be fully clarified. According to the results of the proposed model, the role of heterogeneity in the dynamics of evolutionary games is definitely non-trivial and difficult predict. Also, as in other systems (e.g. [65]), individual perception can affect the dynamics of a population.

To conclude, we provide our personal intuitions on the achieved results. Abusing some of the terminology, we observe that small perturbations on the main diagonal act on ‘homogeneous groups’ (i.e. C − C and D − D), while those on the off-diagonal act on ‘heterogeneous groups’ (i.e. C − D and D − C). So, as also confirmed in previous studies (e.g. [40,44]), groups of cooperators are able to obtain positive feedback loops of system fluctuations. The same benefits cannot be obtained by groups of defectors as, by definition, the latter never contribute to their community. This observation can be related to the results reported in [42] and in [66], where it has been shown that a population of random walkers has a higher probability of cooperating if agents ‘move slowly’ in a continuous space. In particular, as in the proposed model, small perturbations have been more effective in group C − C: temporary groups of cooperators (i.e. whose topology entails C − C interactions) support each other and do that better than groups of defectors, which cannot benefit from such perturbations.

Finally, analysing the dynamics of evolutionary games through the lens of perturbations, in the pay-off matrix, could become useful both for modelling other kinds of scenarios and for making connections with recent works on potential games and their description based on Hamiltonian functions [50,51]. For instance, pay-off perturbations could be embedded into a Hamiltonian by following an approach similar to that used in perturbation theory, e.g. splitting the Hamiltonian function into two components, one unperturbed and the other containing the additional (perturbative) term.

Data accessibility

This article does not contain any additional data.

Authors' contributions

M.A.A. and M.A.J. designed and performed the research and wrote the paper.

Competing interests

We declare we have no competing interests.

Funding

This research was supported by the Brazilian Research AgencyCNPq (proc. 428653/2018-9).