Behavioural flexibility and reputation formation

Abstract

Limited flexibility in behaviour gives rise to behavioural consistency, so that past behaviour is partially predictive of current behaviour. The consequences of limits to flexibility are investigated in a population in which pairs of individuals play a game of trust. The game can either be observed by others or not. Reputation is based on trustworthiness when observed and acts as a signal of behaviour in future interactions with others. Individuals use the reputation of partner in deciding whether to trust them, both when observed by others and when not observed. We explore the effects of costs of exhibiting a difference in behaviour between when observed and when not observed (i.e. a cost of flexibility). When costs are low, individuals do not attempt to signal that they will later be trustworthy: their signal should not be believed since it will always pay them to be untrustworthy if trusted. When costs are high, their local optimal behaviour automatically acts as an honest signal. At intermediate costs, individuals are very trustworthy when observed in order to convince others of their trustworthiness when unobserved. It is hypothesized that this type of strong signalling might occur in other settings.

1. Introduction

Individuals often show behavioural consistency, for example, in levels of aggression, over related but distinct circumstances [1]. There may be situations in which there is selection pressure to be consistent [2]. However, in many cases, evidence suggests that this limited plasticity is maladaptive, so that its existence requires an explanation. For example, female spiders might require a different response to a prey item they should eat than to a male with whom they should mate. They do not always seem to be able to differentiate between these two circumstances, and may be left unmated having eaten a series of possible mates [3]. This suggests their lack of flexibility is producing suboptimal behaviour, although optimal behaviour under limited information cannot be completely ruled out [4].

The number of subtly distinct circumstances encountered in the real world is vast. Evolution shapes behavioural mechanisms that must deal with this vast range. It is not clear that it is even possible to define what we mean by ‘optimal’ in the real world [5–7]. Even if we could, we would not expect evolved behaviour strategies to be exactly optimal. An optimal strategy would necessarily be enormously complex as it would have to respond flexibly and appropriately to every possible circumstance. The psychological mechanism needed to implement such a strategy probably could not evolve in the first place, and even if it did it would involve so much neuronal machinery that the maintenance costs of this machinery [8] would outweigh the benefit. For this reason, we expect the evolution of strategies implemented by psychological mechanisms of limited complexity that perform well on average but may not be exactly optimal in any circumstance [9]. It is inevitable that such mechanisms show some consistency because of their limited complexity.

It is not straightforward to predict the form of the psychological mechanisms that will evolve in the real world [9], and we make no attempt to do so here. Instead, we are concerned with the implications of lack of flexibility on the predictions of social interactions. We focus on the consequences of a lack of flexibility when there is reputation: how does the previous behaviour of a social partner in one circumstance (circumstance O) predict their behaviour in a different circumstance (circumstance U).

Suppose that individuals were completely flexible in their behaviour, responding optimally to every circumstance. Then behaviour in the two circumstance would not be correlated; individuals in circumstance U would just do the best given the circumstance, irrespective of what they did before. Thus the reputation of an individual, which acts as a summary of past behaviour, is only predictive if there is limited flexibility. When reputation is predictive, others will respond to this reputation. This then exerts a selection pressure on individuals to change their reputation by changing their behaviour, so as to affect how others respond to them in the future. Thus the degree to which reputation is predictive is crucial to the evolution and ontogeny of social behaviour. We examine the effect of the degree of flexibility on model predictions in a simple model of a game of trust. In order to capture the idea of limited flexibility, we assume that flexibility has costs [10–12]. This approach may be thought of as a simple expedient (see Discussion). In our model, reputation formation acts as a signal of future behaviour. As we will explain, costs of flexibility mean that there is a cost of deviating from the behaviour that is ‘promised’ by the signal. For instance, studies suggest that lying, a form of deception, is cognitively more demanding than telling the truth [13,14].

In the trust model, individuals pair up to play a sequence of trust games with other population members. In each game, one of the pair is randomly designated as Player 1 and the other as Player 2. Player 1 decides whether to trust their partner based on this partner’s reputation for trustworthiness. If they do trust them, Player 2 then decides whether to be trustworthy or not. Each game occurs in one of the two circumstances, O and U. Both pair members are aware of the current circumstance. In circumstance O, the interaction is observed by all other population members. In circumstance U, the interaction is only observed by the two pair members, and others do not later learn about the interaction. Thus a Player 2’s reputation for trustworthiness is based solely on their behaviour in circumstance O, but this reputation must nevertheless be used to decide whether to trust this individual in circumstance U. Player 2 pays a fitness cost for being flexible that increases with the difference in the level of trustworthiness in circumstances O and U. We vary this cost, showing that intermediate costs can produce qualitatively different behaviour to the extreme cases of zero cost (perfect flexibility) and infinite cost (no flexibility). We also investigate the cost of flexibility to Player 1.

2. The model

We consider a very large (essentially infinite) asexual population with discrete, non-overlapping generations. In each generation, there is a series of many decision epochs. At each epoch population, members are paired at random and play a trust game. This trust game has been previously analysed by a number of authors [15–18] and has the following structure (figure 1). One player is randomly assigned to the role of Player 1 and the other to Player 2. In the first stage of the game, Player 1 can either trust or reject Player 2. If Player 2 is rejected, then both individuals receive payoff s and the game ends. If Player 2 is trusted, then the game proceeds to its second stage. At the second stage, Player 2 can either cooperate with Player 1 or defect, after which the game ends. If Player 2 cooperates, both players receive payoff r. If Player 2 defects Player 1 gets nothing while Player 2 receives payoff 1. We assume that 0 < s < r < 1. For this payoff structure, it is easily seen that for a single round of this game with no reputation effects, if Player 2 is trusted they should defect. Given this, Player 1 should never trust Player 2. Thus we would predict an equilibrium at which Player 1 rejects Player 2 (and Player 2 defects if ever trusted). (This is the subgame-perfect equilibrium.) Both players then receive a payoff of s, whereas they would have both received the large payoff r had Player 1 trusted Player 2 and Player 2 cooperated. The game can thus be seen as a sequential version of the prisoner’s dilemma game.

Figure 1.

The structure and payoffs for the trust game.

In this paper, each game is played in one of two circumstance: with probability α the game is open to observation (O) by all population members, with probability 1 − α it is not observed (U) by other population members. All population members are aware of their current circumstance. The behaviour of a Player 2 is described by the two genetically determined parameters p_o and p_u that specify the probabilities of cooperating if trusted in the two circumstances. We also refer to these quantities as the Player’s trustworthiness in the two circumstances. We assume that there have been many previous rounds of games, so that a Player 1 has been able to accurately estimate the trustworthiness of a Player 2 when observed. Thus, in a particular game Player 1 knows the Player 2’s p_o value, so that this value acts as the signal of Player 2’s behaviour if trusted. Since behaviour in circumstance U is not observed, Player 1 does not know p_u, and can only infer its value from the value of p_o. The behaviour of Player 1 is described by the two genetically determined parameters z_o and z_u. When the game is observed Player 2 is only trusted if p_o ≥ z_o. When the game is unobserved Player 2 is trusted only if p_o ≥ z_u, so that whether Player 2 is trusted when unobserved is based on the previous behaviour of Player 2 when in role 2 and observed.

An individual has four genes specifying z_o, z_u, p_o and p_u. The fitness of an individual is the long-term mean reward per game multiplied by the factor

$${\text{e}}^{-[k_z{(z_o-z_u)}^2+k_p{(p_o-p_u)}^2]}.$$ 2.1

Here, the constants k_z and k_p determine the cost of being flexible when in role 1 and 2, respectively. To consider evolutionary dynamics, we take the values of each of the parameters z_o, z_u, p_o, p_u to lie on a discrete grid. We then compute how the proportion of population members at each grid point combination changes over generations. The population is followed over many generations until proportions settle down to equilibrium values. The results presented are for this equilibrium. A detailed specification of the dynamics and computational procedure is given in the electronic supplementary material.

3. Results

When an interaction is observed, a Player 1 should trust a Player 2 if and only if their probability of cooperating satisfies p_o ≥ s/r [17]. Thus in order to be trusted in future when observed, the Player 2 should have a probability of cooperating when observed such that p_o ≥ s/r. We refer to s/r as the trustworthiness threshold. Since the payoff to Player 2 if trusted decreases with their trustworthiness (because r < 1), we would not expect Player 2’s probability of cooperating to exceed this threshold unless there are reputational reasons to do so. Thus we expect p_o = s/r unless Player 2s are behaving to impress future partners that they are trustworthy when not observed.

In our evolutionary simulations, there is a discrete grid of values. The probability of acceptance is also smoothed close to acceptance thresholds (electronic supplementary material). Thus instead of the precise values predicted, evolved mean values satisfy z_o < s/r < p_o (figure 2a,c). There is variation about these means maintained by mutation (electronic supplementary material): mean values reflect a mutation–selection balance. The selection pressure is, however, asymmetric. For example, if z_o is low, then the probability of wrongly trusting a Player 2 is small since few Player 2s have low p_o values. By contrast, if z_o > s/r then there is a high probability of wrongly rejecting a Player 2. As a consequence of these asymmetric selection pressures, the mean of z_o may be significantly below s/r and the mean of p_o may be significantly above this threshold. This results in almost all Player 2s being trusted when observed (figure 2e). These results hold for all values of the cost of flexibility.

Figure 2.

Evolved mean population characteristics as a function of the cost of flexibility for Player 2, k_p (shown on a logarithmic scale). There is no cost of flexibility for Player 1 (k_z = 0). (a) Acceptance threshold for Player 1 when observed. (b) Acceptance threshold for Player 1 when unobserved. (c) Probability Player 2 cooperates when observed. (d) Probability Player 2 cooperates when unobserved. (e) Probability Player 2 is trusted when observed. (f) Probability Player 2 is trusted when not observed. (g) Probability Player 2 cooperates when trusted (observed). (h) Probability Player 2 cooperates when trusted (unobserved). Solid line: α = 0.3. Dashed line: α = 0.7. Dotted line: critical acceptance threshold $\frac{s}{r}$. s = 0.24, r = 0.60.

Figure 2 considers the case in which flexibility by Player 2 is costly but flexibility by Player 1 is not. When the cost is small (k_p < 0.1), Player 2s are only sufficiently trustworthy to be trusted when observed (p_o just greater than the trustworthiness threshold $\frac{s}{r}$, figure 2c). When not observed they are rather untrustworthy (p_u less than $\frac{s}{r}$, figure 2d), so that Player 1s set such a high acceptance threshold (figure 2b) that Player 2s are rarely trusted (figure 2f).

When the cost to Player 2 is intermediate (0.1 < k_p < 1), a Player 2 with p_o just greater than s/r is liable to have p_u < s/r and should not be trusted when unobserved. By contrast, if p_o is much greater than s/r, then we might expect p_u > s/r since otherwise the Player 2 would incur high costs. Thus only a Player 2 with high p_o should be trusted when unobserved. This means that there is selection on Player 2s to be very trustworthy when observed in order to convince Player 1s that they are also trustworthy when unobserved. Our computations bear out this intuitive argument (figure 2c).

When the flexibility cost is large (k_p > 1) Player 2s are again only sufficiently trustworthy to be trusted when observed (p_o just greater than s/r, figure 2c). Since Player 2s pay a high cost from flexibility most have approximately the same probability of being trustworthy when not observed as when observed (p_o ≈ p_u, figure 2c,d). Thus Player 1s apply the same acceptance threshold in the two circumstances (z_u ≈ z_o, figure 2a,b), so that Player 2s tend to be trusted when not observed (figure 2f ), and are indeed reasonably trustworthy; i.e. those that are trusted unobserved have a probability of being trustworthy that exceeds s/r (figure 2h)

Our central result—that trustworthiness when observed should be highest at intermediate costs—appears to be robust under changes in parameter values. In particular, it still occurs when s and r are changed, holding the ratio s/r at 0.4 (electronic supplementary material, figure S1), and it occurs when s/r is changes (electronic supplementary material, figure S2). The result is also predicted by a simple analytic model (electronic supplementary material, figures S4, S5, S6).

Lowering α increases the proportion of interactions that are unobserved and hence the importance to Player 2s of being trusted when unobserved. Thus, as figure 2c and electronic supplementary material, figures S1(c) and S2(c) illustrate effects become stronger. The analytic model of electronic supplementary material also illustrates that effects are weaker for large α (electronic supplementary material, figures S4, S5, S6).

In order to highlight the effects of costs of flexibility for Player 2, in figure 2, we assumed no cost of flexibility for Player 1. It seems reasonable, however, that there will be similar flexibility costs in both roles. Figure 3 illustrates the case where the two costs are equal. When costs are small they have little effect, so that results are similar in figures 2 and 3. At intermediate cost, there is selection to make z_o and z_u similar. Since Player 1s need to be very choosy when unobserved, with flexibility costs they are more choosy when observed (compare figures 2a and 3a), so that Player 2s are even more trustworthy when observed (compare figures 2c and 3c). This also means that more of those that are trusted when observed are actually trustworthy (compare figures 2g and 3g), with this effect being particularly strong when most interactions are observed (α = 0.7). Finally, at high costs, since z_o and z_u were already similar in the k_z = 0 case, results are little changed when a cost to flexibility of Player 1 is introduced.

Figure 3.

Results corresponding to figure 2 when there are also costs of flexibility to Player 2, with k_z = k_p.

4. Discussion

In social interactions, the best action of an individual typically depends on that of other individuals. Consequently, when an individual must choose its action before the behaviour of others is known, any information that can help predict their behaviour will be advantageous. Here, we have investigated how past behaviour provides information. We have shown that limited flexibility in behaviour results in individuals having to exhibit high levels of cooperation (trustworthiness) when other can observe them so that they will be trusted in future interactions that do not have an audience, and so cannot affect their reputation. This conclusion might also apply to other phenomena. For example, the propensity to be angry or retaliate as a result of an infringement when observed might be correlated with the level of this behaviour when not observed. For intermediate levels of flexibility, one might then expect a very strong reaction when observed in order to credibly signal that the individual will retaliate if there is an infringement when not observed.

When the past behaviour of an individual partially predicts current behaviour a feedback loop is set up: population members evolve to pay attention to reputations and respond to them; which then selects for individuals to change their reputation through changing their behaviour in order to change how others respond to them [19,20]. When reputation is important, the action of an individual has two effects: (i) it gives an immediate payoff, and (ii) it affects future payoffs through its effect on the response of others to the individual in the future. There is often a trade-off in the choice of action, and it may be advantageous to reduce the current payoff if this is more than made up for in terms of increased future payoffs. In our example, the payoff to a Player 2 in a single round is maximized by the player being as untrustworthy as possible (always defecting). Behaving in this way when observed does not evolve because it results in a poor reputation, leading to rejection by future partners and hence a poor future payoff.

The previous behaviour of an individual will often have been observed in circumstances that are distinct, but related to, the current circumstances. The value of reputation then depends crucially on the consistency of behaviour across contexts. In our example, the behaviour of a Player 2 when observed acts as signal about their behaviour when not observed. When an individual is not observed their behaviour cannot affect their reputation and hence affect future payoffs. Thus when there are no costs of or constraints on flexibility, any signal is worthless: if trusted when not observed it is always best defect, maximizing the current payoff.

When there are sufficient costs to being flexible, the situation is different. The trustworthy behaviour of a Player 2 when observed can then signal that the trustworthiness of this player when unobserved exceeds the trustworthiness threshold, and should be trusted. It is costly to produce this signal, since the payoff to a Player 2 when trusted decreases with their trustworthiness, but this initial cost is not relevant to the honesty of the signal. Rather it is the cost of a future deviation from what is promised that is critical: if a Player 2 signals high trustworthiness then it would not be in the interests of the Player 2 to be too untrustworthy when unobserved. The situation is therefore what Kirmani & Rao [21] refer to as default-contingent signalling; that is signalling in which a cost is paid if the signaller deviates from the behaviour that is indicated by the signal. This signalling paradigm has been highlighted in a biological context by Enquist [22] and Hurd [23].

When the cost of flexibility is high in our trust model, a Player 2 that is only slightly more trustworthy than the trustworthiness threshold when observed is giving a reliable signal they will exceed this threshold when not observed. When costs are intermediate it requires a stronger signal when unobserved to convey the same message. Finally, for low costs, it would still be worth a Player 2 that was completely trustworthy when observed to be below the trustworthiness threshold when unobserved, so that Player 2s no longer attempt to signal their behaviour when unobserved.

The existence and strength of costs of flexibility in behaviour has been debated (e.g. [10–12]). Although costs plays a central role in the analysis of the specific model we have presented, we see lack of flexibility rather than the effect of costs as the key to the real biology of social reputations. In our model, costs are just a way of constraining flexibility in a model that assumes that evolution produces optimal behaviour. We have argued, however, that the real world is too complex to produce exactly optimal behaviour, and that a lack of flexibility is an inevitable consequence of the evolution of mechanisms. Thus a more realistic model would evolve some class of behavioural mechanism in a suitably complex environment [9]. A lack of flexibility would then automatically arise, even without costs. For example, it might arise through the internalization of social norms [24–26]. Such models should be investigated in the future. Detailed results might then depend on the class of mechanisms and environments considered, although we would expect our main conclusion that signals should be amplified to still hold.

Data accessibility

This article has no additional data.

Competing interests

We declare we have no competing interests.

Funding

Z.B. was financed by the Higher Education Institutional Excellence Programme (NKFIH-1150-6/2019) of the Ministry of Innovation and Technology in Hungary, within the framework of the DE-FIKP Behavioural Ecology Research Group thematic programme of the University of Debrecen.