Reputation-based partner choice promotes cooperation in social networks

Abstract

We investigate the cooperation dynamics attributed to the interplay between evolution of individual strategies and evolution of individual partnerships. We focus on the effect of reputation on individual's partner switching process. We assume individuals can either change their strategies by imitating their partners or adjust their partnerships based on local information about reputations. We manipulate the partner switching in two ways, that is, individuals can switch from the lowest reputation partners, either to their partners' partners who have the highest reputation (i.e., ordering in partnership) or to others randomly chosen from the entire population (i.e., randomness in partnership). We show that when individuals are able to alter their behavioral strategies and their social interaction partnerships on the basis of reputation, cooperation can prevail. We find that the larger temptation to defect and the denser the partner network, the more frequently individuals need shift their partnerships in order for cooperation to thrive. Furthermore, increasing tendency of switching to partners' partners is more likely to lead to higher level of cooperation. We show that when reputation is absent in such partner-switching process, cooperation is much less favored than that of reputation involved. Moreover, we investigate the effect of discounting individual's reputation on the evolution of cooperation. Our results highlight the importance of consideration of reputation (indirect reciprocity) on promotion of cooperation when individuals can adjust their partnerships.

I. Introduction

Cooperation is widespread in the real world, and can be observed at different scales of biological organization, ranging from genes to multicellular organisms [1]. Most importantly, human society is based upon cooperation. But cooperation is costly: a cooperator pays a cost to benefit others. These cooperative behaviors apparently contradict Darwinism [2]: selfish behaviors will be rewarded during fierce competition between individuals, but how can natural selection lead to cooperation? This puzzle of cooperation has fascinated evolutionary biologists over several decades.

Evolutionary game theory provides a framework for studying the evolution of cooperation among unrelated individuals [3]. As a metaphor, the Prisoner's Dilemma has been widely employed for investigating the origin of cooperation [4]. In this simple game, two players simultaneously decide whether to cooperate (C) or to defect (D). They both receive R upon mutual cooperation and P upon mutual defection. A defector exploiting a cooperator gets T, and the exploited cooperator receives S. The game is a Prisoner's Dilemma if T > R > P > S. As a result, it is best to defect regardless of the co-player's decision in one single round. Thus, defection is the Nash equilibrium, even though all individuals would be better off if they cooperated. The Prisoner's Dilemma depicts the conflict of interest between what is best for the individual and what is best for the group, and thus creates the social dilemma [5]. Therefore, specific mechanisms for the evolution of cooperation are needed [2]. Direct reciprocity [4, 6–8], indirect reciprocity [9–11], kin selection [12], group selection [13–15], and graph selection (or spatial reciprocity) [16, 17] have been proposed to promote cooperative behaviors in different contexts.

Understanding the evolution of cooperation also attracts increasing interests from physics community[18– 40] (see, for example, a recent review Ref. 41 and references therein). Inspired by papers on spatial games, much work pays attention to evolution of cooperation in structured populations. Especially, the development of evolutionary graph theory offers a convenient framework for describing population structures [42]: vertices denote players and edges represent links between players in terms of dynamical interaction. It has been well recognized that graph topologies play a crucial role in evolution of cooperation. A variety of networks have been explored [20–23, 25–28, 30, 31, 43, 44]. Most notably, scale-free networks provide a framework for emergence of cooperation, as reported in Refs. 32–34.

In this paper, we will study how cooperation can evolve by the interplay between graph selection and indirect reciprocity (leading to ‘partner-switching’). Some previous works have stressed the significance on investigating cooperation on adaptive networks, rather than on static networks (see Refs. 36–40, 45). Additionally, reputation is naturally involved in repeated games, if players can have cognitive abilities. Very recently, Ref. 46 found that gossip is an alternative for direct observation in games of indirect reciprocity. Reputation by itself has been shown to have a strong influence on cooperation dynamics in games of indirect reciprocity [9], and this helps to explain the observed high level of cooperation in human societies. In partner marketplace, individual's partner choice tends to make use of potential partners' reputations, and prefer to choose the ones having good reputation. In addition, individual's partner switching is inclined to avoid aversive partnerships, and prone to dump the ones having low reputations. Besides, individuals usually only have local information about reputations in a group. Here, we assume they know the reputation of their partners and partners' partners. Meanwhile, there exists evidence showing that partner switching causes cooperative behavior in cleaning fishes [47]. Motivated by these, we propose a computational model incorporating these factors. Interestingly, when reputation is in effect during partner switching process, even if individuals shift their partners at lower frequency than adjust their strategies, cooperators still have a good chance to wipe out defectors. Furthermore, increasing individual's tendency in partner choice based on reputation will result in higher level of cooperation.

The rest of this paper is organized as follows. Sec. II describes in detail the basic model. Sec III presents the numerical results, including time evolution of the entangled dynamics of strategy updating and reputation-based partner switching, the influences of varying model parameters, and the role of reputation in promoting cooperation. Sec. IV makes some discussions to the obtained results and draws the conclusion.

II. The Model

Let us first define the population structure and evolutionary dynamics. The vertices of a dynamical graph represent players and the edges denote the pairwise partnership (game interaction) between individuals. Initially, the coevolution of individual strategies and graph starts from a random and homogeneous state. Each of N individuals has the same number of interaction partners (i.e., network neighbors, where the total M edges uniformly pair them up at random) and an equal probability to be a cooperator (C, denoted by two-dimensional unit vector s = [1,0]^T) or defector (D, s = [0, 1]^T). Besides, we assume the numbers of individuals and edges remain constant during the individual strategy updating and partner switching processes, namely, the average degree 〈k〉 = 2M/N is invariant. This constraint implies a limited resource environment and introduces a limitation into the emergent network configuration. Each individual engages in pairwise interactions with his immediate neighbors defined by the partner network. That is, individual i plays Prisoner's Dilemma game with all his social partners and obtains an income as:

$$P_i=\sum _{j\in N_i}{\text{s}}_i^{T}M{s}_j,$$

where N_i represents the neighborhood set of i, and the 2 × 2 payoff matrix M has a simple rescaled form with single parameter b (1< b < 2) [16, 43]:

$$M=\left(\begin{array}{cc}1& 0\\ b& 0\end{array}\right).$$

In order to account for the reputation effect in partner choice, we define the reputation R_i(t) of individual i at time t as the times he cooperates with his neighbors in the past games, which reads,

$$R_i(t)=R_i(t-1)+{\Delta }_i(t),$$

where Δ_i(t) is 1 if individual i cooperates at time t, otherwise being 0. Notably, this definition of reputation is similar to the image scoring proposed by Nowak and Sigmund [9], which could be an affine transformation of the image score.

In our model, we consider a coupling between individual strategy and partner network structure. We assume individual strategy updating introduces a new time scale τ_e, not necessarily equal to the time scale associated with adaptive partner switching process (τ_a) [39]. Depending on the ratio W = τ_e/τ_a > 0, coevolution of strategy and network proceeds together under asynchronous updating: a strategy update event is chosen with probability (1 + W)⁻¹, a structural update event being selected otherwise. Now, the W-value governs the two competing updating moves: with W → 0, the evolution of cooperation on static graphs is recovered; with increasing W, individuals become prompt to adjust their social partners with increasing efficiency [39].

Strategy updating

an individual i is chosen at random and another individual j is chosen randomly from i's nearest neighbors. The individuals i and j interact with all their social partners (those directly connected to them by links) according to the specified Prisoner's Dilemma. As a result, they accumulate total payoffs P_i and P_j, respectively. The strategy of j replaces that of i with likelihood given by the Fermi function [48]:

$$\phi (s_i\leftarrow s_j)=\frac{1}{1+exp[\beta (P_i-P_j)]},$$ (1)

where β represents the intensity of selection (β → 0 leads to random drift while β → ∞ the deterministic imitation dynamics). Besides, the individual's reputation R_i(t) is also updated when he is picked up for strategy update. Thus, as aforementioned, R_i(t) = R_i(t − 1) + Δ_i(t), Δ_i(t) is 1 if individual i cooperates at time t, otherwise being 0.

Partner switching

We assume individuals have local information about his nearest and next-nearest neighbors, i.e., reputations of these individuals are known to the focal individual since he can witness his social partners' cooperation in past games and obtain the next-nearest neighbors' information from his nearest neighbors. In addition, individuals are assumed to know nothing about others (types and reputations) except his nearest and next-nearest neighbors. This assumption only requires individuals have local information and thus is reasonable. An individual i is picked at random to update his interaction partners on the basis of the reputations of his social partners. i dismisses the edge to the one who has the lowest reputation, namely, i switches from this partner, either to another one chosen among i's next-nearest neighbors preferentially according to their reputations or to a random member of the entire population (excluding nearest neighbors). Specifically, with probability p, the focal individual redirects the link from the one, who has the lowest reputation among his social partners, to the one chosen from his neighbors' neighbors, who has the highest reputation (“ordering” in partnership); otherwise with likelihood 1−p, he rewires the link from the lowest-reputation partner to the one randomly selected from the entire population except his nearest neighbors (randomness in partnership).

More explicitly, at time t, individual i terminates future interactions with player j, satisfying,

$$j=arg{min}_{l\in {\mathcal{N}}_i}{R}_l(t),$$

and then with probability p switches to individual k as future partner,

$$k=arg{max}_{{\cup }_{l\in {\mathcal{N}}_i}{\mathcal{N}}_l\backslash \{{\mathcal{N}}_i,i\}}{R}_l(t),$$

otherwise with likelihood 1−p, switches to individual m, randomly selected from the entire population except his nearest neighbors. Herein, individuals can unilaterally break up the aversive tie and are likely to choose a good-reputation partner, which can be advantageous in future interactions. For the sake of simplicity, we assume that, the chosen individual accepts new social partners without any choosiness. The Prisoner's Dilemma is a non-zero sum game, and the elements of payoff matrix are nonnegative here. If an individual is picked up by others as future partner in the partner switching process, he will gain a new partnership and thus a potential profitable interaction for himself. Therefore, he will not reject this offer. If some certain participation cost is considered in such partner-switching event, the scenario becomes very different: individuals will exhibit choosy behavior in regarding to his new partners (see, for example, Ref. [45]).

In what follows, we will demonstrate that such coevolution of strategy updating and partner switching will lead to stable cooperation in Prisoner's Dilemma. Furthermore, it is shown that reputation significantly enhances the cooperation level.

III. Simulation Results and Discussions

In this section, we will show in detail the results about stable cooperation as a result of reputation-based partner switching. We start from a homogeneous partner network [49], where all individuals have the same number of edges, randomly connected to arbitrary players. Initially, 50% of cooperators are randomly distributed in the population. The key quantity to characterize the cooperative behavior is the average fraction of cooperators in the population. To this end, for each parameter set, we compute the fraction of cooperators by averaging over the last 10³ generations of the total 10⁶ generations. Besides, we impose that nodes linked by a single edge cannot lose this connection, thus the evolving networks are connected at all times. The amount of heterogeneity of the networks is calculated as $h=N^{-1}{\sum }_k{k}^{2}N(k)-{\langle k\rangle }^2$ (the variance of the network degree sequences), where N(k) represents the number of vertices with k edges. Additionally, in order to investigate the degree-degree correlation pattern about the emerging social networks, we adopt the assortativity coefficient r suggested by Newman [50]

$$r=\frac{M^{-1}{\sum }_i{j}_i{k}_i-{\left[M^{-1}{\sum }_i\frac{1}{2}(j_i+k_i)\right]}^2}{M^{-1}{\sum }_i\frac{1}{2}(j_j^2+k_i^2)-[M^{-1}{\sum }_i\frac{1}{2}(j_i+k_i){]}^2},$$ (2)

here j_i, k_i are the degrees of the vertices at the ends of the ith edge, with i = 1,…,M. Networks with assortative mixing pattern, i.e., r > 0, are those in which nodes with large degree tend to be connected to other nodes with many connections and vice versa.

A. Time evolution

We first study the typical time evolution of the entangled dynamics of networks and strategies (see Fig. 1). Under such reputation-based partner switching mechanism, cooperation becomes evolutionarily competitive and the system moves towards full cooperation as shown in Fig. 1(a). Besides, the DD and CD pairs are significantly suppressed during the coevolutionary process of strategy updating and partner switching, resulting in positive assortment between cooperators [see Fig. 1(b)]. Noteworthy, such positive assortment between cooperators plays a crucial role in the evolution of cooperation, as reported in previous investigations on spatial games [19, 44, 51–53]. We should stress that such high level of cooperation is also attributed to the emergent network's heterogeneity (see Refs. [32–34, 39]). As plotted in Fig. 1(c), the social network readily tends to be heterogeneous as individuals start to adjust their social partners based upon their partners' reputations. The emergent network at final state exhibits high heterogeneity with a power-law tail [see Fig. 1(c)]. Moreover, it is found that nodes with large connectivity (i.e., hubs) are inclined to be occupied by cooperators [32–34, 39]. Hubs are thus more likely to be cooperators with high reputation. Due to the tendency of connecting to partners' partners (triadic closure) in the partner-switching process, hubs are more likely to be chosen by the focal individual. As a consequence, we find “the rich get richer”, namely, inhomogeneity in individual's partnership is induced. Further, such high-reputation partner seeking behavior also leads to the disassortative mixing pattern in the emerging social network since the large-degree nodes (having high reputation) are more likely to be attached by others. Clearly, these results indicate that reputation-based partner switching leads to stable cooperation in the Prisoner's Dilemma on graphs.

FIG. 1.

(color online). Dynamics of strategies and partner networks. Panel (a): fraction of cooperators; (b): fraction of CC/CD/DD links; (c): cumulative degree distribution of the resulting partner network at final state. Cooperation evolves under reputation-based partner switching. The emergent partner network is disassortative (assortativity coefficient r = −0.39373) and highly heterogeneous. The amount of heterogeneity of the partner network increases as individuals adjust their partners selectively according to reputation. Individuals with good reputation tend to attract new partnerships, while those with low reputation are confronted with the threat of losing existing partnerships. Thus under this social selection pressure, individuals are forced to be generous in order to obtain a good reputation. Parameter values are: N = 10³, 〈k〉 = 10, b = 1.2, W = 1, p = 0.5, and β = 0.01. Initially, cooperators and defectors are uniformly distributed in the partner network with equal probability.

B. Effects of the temptation to defect and the average degree

In Fig. 2(a), we show the effect of temptation to defect, b, on the evolution of cooperation. With other parameter fixed, the cooperation level monotonically increases with increasing W-value. With increasing b, the structural updating event (partner switching) must be sufficiently frequent to guarantee the survival of cooperators. In other words, the probability of individuals chosen for updating social ties should be accordingly increased to ensure the sustainability of cooperators. When the temptation to defect becomes larger, defectors become more favorable by natural selection. Nevertheless, with the aid of reputation-based partner switching, a small fraction of surviving cooperators promotes them into hubs, since they are attractive to neighborhood because of their high reputation. Such co-evolution dynamics of strategy and structure leads to highly heterogeneous networks in which the cooperators become evolutionarily competitive as demonstrated in Refs. [32–34, 39]. For fixed b, we observe a critical value W_c for W, above which the cooperator will wipe out defectors. For other parameters fixed, the critical value W_c monotonically increases with increasing b. Therefore the prompt partner switching deters cooperators from becoming extinct, further, resulting in an underlying heterogeneous social network which is the “green house” for cooperators to prevail under natural selection (i.e., strategy updating). Additionally, long-term profitable partnerships between high-reputation individuals are formed via the partner switching process. Consequently, the reputation-based partner switching (i.e., entangled co-evolution of strategy and structure) promotes cooperation among selfish individuals. Interestingly, under such reputation-based partner switching mechanism, W-values smaller than 1 are sufficient to guarantee the success of cooperation. That is, the adjustment of individual's partnership is slower than the imitation dynamics. This is consistent with our daily-life experience, especially in human society, that the evolution of social networks could not be faster than the dynamics on top of them since removing and creating edges are mostly costly in terms of time.

FIG. 2.

(color online). The effects of the temptation to defect [panel (a)] and the average degree [panel (b) ] on the evolution of cooperation. The fraction of cooperation is plotted as a function of the time scale ratio, w, between strategy updating and partner switching. Cooperation becomes dominant when individuals sufficiently often adjust their partnerships based on reputation. Furthermore, the larger the temptation to defect and the more number of interaction partners on average, the more promptly individuals need to change their partnerships in order for cooperation to thrive. Parameter values are: in panel (a) N = 10³, 〈k〉 = 10, p = 0.5, and β = 0.01; in panel (b) N = 10³, b = 1.5, p = 0.5, and β = 0.01. Each data point results from averaging over 100 independent runs.

We report the influence of average degree 〈k〉 on the evolution of cooperation in Fig. 2(b). With increasing 〈k〉 (the network becomes denser), individuals should more frequently carry out their partner switching (i.e., increasing W_c−values). On static networks, the fate of cooperators hinges on the sparseness of the networks [16, 43]. In previous study, it is found that when the social networks are highly connected, cooperation is greatly inhibited and does not have the chance to thrive. Thus, to explain the ubiquitous cooperative behavior in human society, where individuals have large numbers of neighbors, the relevant coevolution of network and strategy should be taken into account.

C. Influence of ordering in partner switching

We investigate the effect of ordering (i.e., triadic closure) in partner-switching process on the evolution of cooperation, as shown in Fig. 3. We can see that when individuals tend to switch partners with local reputation information, i.e., preferentially choose high-reputation potential partners from their neighbors' neighbors, cooperation is enhanced greatly. In other words, if individuals adjust their social partners by choosing strangers at random as future interaction partners, instead of depending on reputation about their new potential partners, cooperation would be suppressed. This result further pinpoints the importance of local information about reputation in determining new partners. When individuals prefer to switch to partners' partners, who have high reputation, it enhances the influence of high-reputation ones in their neighborhood, and thus facilitates the evolution of cooperation. Furthermore, if individuals tend to switch partners in a triadic way, it also increases the number of triangles presented in the partner network, i.e., clustering coefficient. This leads to the positive assortment between high-reputation individuals, and consolidates the maintenance and robustness of cooperation. Altogether, if individuals are likely to decide the new partnership on the basis of their knowledge about their potential partners' reputation (which could be obtained by inquiring their neighbors about reputation of their neighbors' neighbors), low-reputation individuals (defectors) are unlikely to have the opportunity to exploit the high-reputation ones. Therefore, cooperation is promoted in this situation.

FIG. 3.

The influence of “ordering” in partner switching on the evolution of cooperation. When individuals tend to switch to partners' partners with good reputation, cooperation is favored. Parameter values are: N = 10³, 〈k〉 = 10, β = 0.01, W = 0.1, and b = 1.2. Each data point results from averaging over 10³ independent runs.

D. Role of reputation

In order to examine the reputation effect on the evolution of cooperation in such partner-switching process, we conduct a comparative study: we explore the case in which no reputation is involved in partner-switching process. In addition, we introduce a memory decaying effect on individual's reputation updating (discounting of reputation), that is,

$$R_i(t)=\delta R_i(t-1)+{\Delta }_i(t),$$

where δ denotes the decaying rate. For δ → 0, the influence of previous game experience is vanishing and individuals judge the new partnership relying on the present game result. With δ → 1, it recovers our original model. Besides, we also consider the situation in which individuals switch their partners at random, and do not make use of any information about partners. The relevant result is provided in Fig. 4. Clearly, compared with the original model, if no reputation involved, the cooperation level is much lower. Hence, reputation could enhance the cooperation in a way. Additionally, discounting of reputation corresponds to higher cooperation level (see Fig. 4) provided that individuals are inclined to frequently switch partners. To be concrete, if individuals tend to promptly change partners exclusively relying on present interacting results (like δ = 0 in Fig. 4), rather than the partners' accumulated reputation (as δ=1 in Fig. 4), with a higher possibility, defectors will be dumped by the focal individual, and thus face the shrink of their partnerships. As a consequence, this to a large extent promotes cooperation through game dynamics. Interestingly, compared with the original model, when individuals slowly adjust their social partners (i.e., small time scale ratio W), decaying memory effect reduces the cooperation. On the other hand, decaying memory in reputation will result in high cooperation only in the situation where individual's partner switching occurs frequently. The interplay between discounting of reputation and propensity of partner switching can be understood as follows. Individuals' swift response to present defection of their partners (i.e., individuals tend to frequently dump their partners with low reputation) increases the risk cost of being a defector. Consequently, discounting of reputation leads to the situation in which individual's partner choice tends to rely on present game result. It thus enhances the efficiency of prompt partner switching, and also increases social selection pressure in such a marketplace, resulting in favorable condition for cooperation. In contrast, when individuals slowly change their partners, accumulated reputation indicates the exact information regarding cooperativity of partners in all past games, thus is helpful for individuals to successfully seek beneficial long-term partners. Overall, these results demonstrate that reputation-based partner-switching behavior can lead to stable cooperation. Otherwise, random partner switching can not induce high cooperation level as well, in comparison with the situation in which reputation of partners is involved.

FIG. 4.

(color online). The effect of reputation on the evolution of cooperation. When there is no reputation involved in partner switching, the cooperation level is much lower. Compared with original model without discounting of reputation (δ = 1), decaying memory in reputation (here, δ = 0.5) leads to lower cooperation level when individuals occasionally shift their partners; in contrast, discounting of reputation provides a more advantageous environment for cooperation when individuals frequently change their partners. Parameter values are: N = 10³, 〈k〉 = 10, β = 0.01, b = 1.4, and p = 0.5. Each data point results from averaging over 100 independent runs.

IV. Discussion

Here, we study, by numerical simulations, the promotion of cooperation induced by reputation-based partner switching. Prior to present work, Refs. [36–40] have pointed out that the entangled evolution of individual strategy and network structure constitutes a key mechanism for the sustainability of cooperation in social networks. Nevertheless, our results indicate that if individual's reputation is involved in partner-switching dynamics (co-evolutionary dynamics), this significantly reduces the critical value of time scale ratio W—above which cooperators wipe out defectors. Indeed, individual's reputation is accumulated in repeated games, and contains information about individual's cooperativity. Thus, when individuals switch their partners according to reputation, low-reputation ones will be confronted with the threat of losing the existing partnership as well as impede their ability to recruit new ones. In contrast, high-reputation individuals attract new partners, and their old partners are maintained. In such a social marketplace, where people can choose with whom to interact, there is competition to be more generous (to establish a high reputation) in order to be chosen as a partner. Therefore, reputation greatly enhances the cooperation in such coupled dynamics of partner switching and strategy updating.

Recently, indirect reciprocity is proposed to be an important mechanism for evolution of cooperation [2, 9, 10]. Reputation fuels the engines of indirect reciprocity. It is worth noting that our model for dynamic population structures incorporates indirect reciprocity: helping someone establishes a good reputation, which will be rewarded by others. Actually, those who are more helpful are more likely to receive new partnership, and new partnerships benefit these with high reputation. The interaction is observed by a subset of the population who might inform others. Variations in tendencies to cooperate or defect in discrete interactions with rapidly switching partners certainly create social selection forces [54], further stabilizing cooperation. Accordingly, reputation allows evolution of cooperation by partner switching.

Noteworthy, punishment is an important factor that can promote cooperative behavior in some situations [55– 57]. In general, punishment is manipulated by imposing some fine to defectors but at a cost to punishers. We say this kind of punishment is the “active” one. An alternative “passive” punishment is implemented in our model. Individuals tend to “punish” the low reputation ones in a way that they terminate future interactions between them. The severe case is ostracism [56], in which low-reputation individuals lose all their partners, excluded from a group. Furthermore, the good reputation individuals will be rewarded in a way that they are preferentially chosen as partners by others. Once such kind of selective punishment and reward is in effect, it can enhance the level of cooperation that is achieved in our model.

We have, numerically, studied the entangled evolution of individual strategy and partnership network structure, in which individuals are able to simultaneously alter their behavioral strategies and their social partners. We found that reputation-based partner switching can lead to stable cooperation in networked Prisoner's Dilemma. Besides, we explored the effects of varying model parameters on the evolution of cooperation. Our results suggest that, when individuals are faced with large temptation to defect (i.e., large b-values) and on average engage in dense interactions (i.e., highly-connected network), they must be able to promptly adjust their partners for cooperation to thrive. Owing to individual's preference in selecting good reputation ones as potential partners, the resulting partnership network is highly heterogenous. In addition, promotion of cooperation is attributable to such emergent heterogeneity. Finally, we examined the role of reputation in evolution of cooperation via comparative study. We found that discounting of reputation leads to higher level of cooperation when individuals promptly adjust their partnerships. Comparing with the original model, we also found that when individuals randomly adjust their partners, rather than relying on reputation, to a large extent, cooperation will be diminished even if individuals are able to rapidly switch their partners. Our results may offer some insight into understanding the occurrence of cooperative behaviors in the real world, especially in human societies.