When hawks give rise to doves: The evolution and transition of enforcement strategies

Abstract

The question of how altruism can evolve despite its local disadvantage to selfishness has produced a wealth of theoretical and empirical research capturing the attention of scientists across disciplines for decades. One feature that has remained consistent through this outpouring of knowledge has been that researchers have looked to the altruists themselves for mechanisms by which altruism can curtail selfishness. An alternative perspective may be that just as altruists want to limit selfishness in the population, so may the selfish individuals themselves. These alternative perspectives have been most evident in the fairly recent development of enforcement strategies. Punishment can effectively limit selfishness in the population, but it is not free. Thus when punishment evolves amongst altruists, the double costs of exploitation from cheaters and punishment make the evolution of punishment problematic. Here we show that punishment can more readily invade selfish populations when associated with selfishness, whereas altruistic punishers cannot. Thereafter, the establishment of altruism due to enforcement by selfish punishers provides the ideal invasion conditions for altruistic punishment, effectively creating a transition of punishment from selfishness to altruistic. Thus, from chaotic beginnings, a little hypocrisy may go a long way in the evolution and maintenance of altruism.

Introduction

The evolution of altruism has intrigued scientists for over a century, resulting in numerous theories and debates that have been both complimentary and competitive, often resembling the topic of study itself (eg, Nowak et al. 2010, Abbot et al. 2011). Despite the outward appearance of dissimilar, competing theoretical frameworks, in the end, altruism can evolve despite being at a local disadvantage to selfish strategies (i.e. multilevel selection, see Wilson and Wilson 2007). Generally, researchers look to the altruists themselves for mechanisms by which selfishness can be curtailed. This is not surprising as the competition between altruism and selfishness may be influenced by our own humanity, driving researchers to seek out ways the good prevail over the evil. Despite this way of thought, another possibility may exist, one by which the selfish limit selfishness, thus maintaining altruism in the population (Eldakar et al. 2007, Eldakar and Wilson 2008).

The concept of selfishness as a self-limiting strategy was first suggested to us by an empirical study on humans showing that individuals most likely to punish cheaters were also most tempted to cheat (Eldakar et al. 2006), and later supported by its observed presence across societies (Herrmann et al. 2008). Although the concept of the maintenance of altruism by selfishness may appear paradoxical, examples of this relationship are abundant in both human and non-human social groups. For example, in non-human primates leading members of dominance hierarchies receive proportionately greater benefits from the group, yet often pay proportionately greater costs in maintaining the social order (de Waal 1989, Boehm 1999, Flack et al. 2005, 2006) (here we consider dominants as selfish due to performing self-serving behaviors while restricting such behaviors in subordinates). Furthermore, in groups not confounded by power asymmetries associated with hierarchies, those that punish transgressors are also likely transgressors themselves. For example, in the tree wasp (Dolichovespula sylvestris), individuals that police worker laid eggs and reduce overall cheating are also those most likely to cheat by laying eggs, earning the term “corrupt policing” (Wenseleers et al. 2005). In addition, Scrub Jays (Aphelocoma coerulescens) most defensive of the theft of food caches are also those most likely to steal caches from others (Emery and Clayton 2001). A similar dynamic may even play out within multicellular organisms, as cancerous tumors have been shown to restrict the growth of other tumors (Camphausen et al. 2001, Guba et al. 2001).

Similar scenarios have also existed throughout the history of human societies. Castillians/knights of the middle ages were initially considered thugs that took resources from defenseless civilians, yet at the same time defended those very same civilians from the exploitation of others (Bison 1994). Modern examples of this relationship include the Mafia, by which crime rates are kept exceedingly low, increasing the local prosperity and benefiting the majority of those in their region (Fiorentini and Peltzman 1995, Lupo 2009). Furthermore, studies on resource control and aggression find similar strategies present even in preschoolers. Socially dominant children with high moral cognition (understanding of rules) were able to control resources through the manipulation of their peers, thus promoting overall moral behavior while also using it to their own advantage (see, Hawley and Geldhof 2012).

Enforcement does not always entail corruption and exploitation; indeed the theory of altruistic punishment (also sometimes known as moralistic punishment and strong reciprocity) posits that enforcement behavior is performed by altruists rather than cheaters (eg, Bowles and Gintis 2002, 2004, Boyd et al. 2003). This perspective is apparent in the vast majority of research on punishment in humans (see Fehr and Gachter 2000, Fehr and Fischbacher 2003, Gintis et al. 2003). Furthermore, punishers are often revered, such that punishers experience greater rewards and favor by those group members in which they protect (Barclay 2006), which leads to the inevitable question of whether some motivations of altruistic punishment are actually altruistic.

The evolutionary role of punishing selfishness is straightforward; it reduces the within group advantage of selfishness over altruism. Although punishment is effective, it is not free; it costs time, resources and generates risk. Non-punishers, on the other hand, benefit from punishment but do not share in the costs, qualifying punishment as a form of second-order altruism and non-punishment as a form of second-order free-riding. Thus, a major issue with the evolution of punishment is identical to the original problem of altruism; how punishers prevail when facing exploitation from non-punishers (Yamagishi 1986, Bowles and Gintis 2002, 2004, Boyd et al. 2003, Fehr 2004, Panchanathan and Boyd 2004, Eldakar and Wilson 2008). In the typical formulation, selfishness wins within groups and altruism wins between groups (see review, Wilson and Wilson 2007), and the same applies for selfish and altruistic punishers. Specifically, Eldakar et al. (2007) demonstrated that when propensity to cooperate and the propensity to punish noncooperators were modeled as continuous, independent traits, over time a negative correlation between these traits readily evolved. Furthermore, as the cost of punishment increased, essentially magnifying fitness differences and selection within groups, the more “selfish” punishers became. However, as the number of iterations increased (rounds spent in groups between mixing), thereby increasing the strength of group selection, the more “altruistic” punishers became. In general, selfish punishers offset the costs of punishment through the exploitation of altruists, maximizing their fitness within groups (Nakamaru and Iwasa 2006, Eldakar et al. 2007, Eldakar and Wilson 2008), while altruistic punishers benefit indirectly through maximizing group fitness.

Previous models have focused on the differences between the punishment strategies and considered them as exclusive to particular conditions (Nakamaru and Iwasa 2006, Eldakar et al. 2007, Eldakar and Wilson 2008). These differences in within- and between-group advantages however make selfish punishers better suited to invade selfish populations, while their altruistic counterparts thrive in populations with high frequencies of altruism (Eldakar and Wilson 2008). Thus, although these punishment strategies appeared mutually exclusive, here we propose another possibility; that they represent transitional stages in the evolution of enforcement.

In predominantly selfish populations, punishers bear immense costs to purge selfishness from groups. However, when punishment takes the form of cheaters targeting other cheaters, this competition amongst cheaters transforms selfishness to a self-limiting strategy, fostering an increase in altruism in the population. With altruism established and cheating curtailed, the need/cost of punishment is reduced causing selection to favor groups of high cooperation, so that selfish punishers ultimately reduce group fitness compared with groups enforced by altruistic punishment. Thus, from chaotic beginnings, a little hypocrisy may go a long way in producing large-scale cooperation.

Here, we test the hypothesis that the evolutionary success of selfish punishers in predominantly selfish populations establishes conditions that favor the subsequent evolution and transition of enforcement to altruistic punishment. We use agent-based simulations to determine the invasion potential of these punishment strategies in isolation as well as transitional forms under various conditions.

The model

Here we present both an analytical and agent based simulation model. The analytical model provides a guide to the construction of the agent based simulation, and is not intended to represent the mean experience of all individuals of a given strategy or population-level outcomes. The simulation is based on the behavior and interaction of individual agents, thus the purpose of the analytical model is to illustrate what occurs from the perspective of an individual within a given group (within group selection), not accounting for the average fitness of each strategy across groups given a set of population frequencies (within and between group selection). The simulation occurs in a large interacting multigroup population of agents. Therefore, the average fitness of each strategy is the result of the average experience of all individual agents of that strategy in the overall multigroup population.

The simulation model was implemented using the agent based modeling program Netlogo, which is freely available for download (for Netlogo program see http://ccl.northwestern.edu/netlogo/, and for code of the simulation model see Dryad repository: doi:10.5061/dryad.jg081). We composed an N-person evolutionary game theoretical model based on the standard public goods game in the experimental economics literature (Fehr and Gachter 2000). The model begins with a population (size = N) with individuals that are either altruistic or selfish, and punish or do not punish. Individuals associate in a large number of randomly formed groups (group size = n) and interact for a number of iterations (I) of a two-phase public goods game. During the initial phase, all individuals acquire a resource of the value (R), with altruists contributing this resource towards the group, whereas their selfish counterparts keep the resource for themselves. Contributions towards the group by the altruists (number of altruists = n_a) are then multiplied by a value (m>1) and then distributed equally amongst all group members (including the altruists). This is relevant to a biological situation involving social animals, as the combined effort of a group will yield a greater return than the sum of what individuals can accomplish alone.

After the cooperation phase, the fitness of altruists (w_s) and selfish individuals (w_s) are represented by equations 1 and 2 respectively. Individuals maximize their fitness within groups by selfishly withholding the resource, while overall group fitness is maximized by altruism, typifying the classic struggle between altruism and selfishness.

$$w_a=\frac{n_{a}Rm}{n}$$ (1)

$$w_s=R+\frac{n_{a}Rm}{n}$$ (2)

Each round, the cooperation phase is followed by the punishment phase in which individuals can pay a cost to expel selfish group members from social interactions. The overall cost of punishment paid by each punisher in each round is the product of the cost of punishment (C) and the number of selfish individuals in the group (n_s), although a selfish punisher do not punish him/herself. Therefore, when considering both phases of interaction, the overall fitness of altruistic (a_p) and selfish punishers (s_p) are represented by eq 3 and 4 respectively, with altruistic non-punishers and selfish non-punishers fitness represented by eq 1 and 2 since they do not bear the costs of punishment.

$$w_{ap}=\frac{n_{a}Rm}{n}-C{n}_s$$ (3)

$$w_{sp}=R+\frac{n_{a}Rm}{n}-C(n_s-1)$$ (4)

Punishment results in the expulsion of cheaters from the social interactions of the group based on the probability of not escaping detection by all punishers. Although punishment is modeled by ejecting individuals from groups, punishment can be considered as either complete removal from the group, or merely expulsion from only particular social interactions of the group (e.g. not interacting in group foraging while otherwise remaining in the group). The probability of a single punisher detecting and punishing a given cheater is represented by the term (D) which can vary from 0 – 1. Therefore the chance of escaping a given punisher is (1-D), while the chance of escaping detection by all punishers in the group (n_p) is (1-D)^np, leaving the overall chance of punishment represented by equation 5. If punished, individuals are excluded from all subsequent rounds of the game for that generation, and replaced by individuals drawn randomly from a pool based on the initial frequencies at the start of the current generation. The fact that the replacements play fewer rounds than the original members is immaterial because they still contribute to fitness differentials of the overall strategies in the total population. In addition, individuals die if possessing a fitness of ≤ 0 at any point, in which case they are replaced in the same manner as those expelled.

$$1-{(1-D)}^{n_p}$$ (5)

If expelled, punished individuals receive a payoff of (R) reduced by the costs of solitary behavior (S; 0 ≤ S ≤ 1), and receive this payoff for the remaining iterations of the game (e). Since punishment occurs each round, an individual excluded earlier in the generation will have a greater value of (e) than a different individual expelled in a later round, resulting in proportionately less of their payoff from solitary behavior. This is biologically reasonable as it is assumed an animal evolved for social living will fare worse than those benefiting from sociality. Thus, a selfish non-punisher’s and selfish punisher’s overall fitness for all iterations (I), is the sum of the payoff acquired during the iterations not expelled (I-e) and the payoff acquired during the rounds expelled (e), represented by equations 6 and 7, respectively.

$$w_s=\left(R+\frac{n_{a}Rm}{n}\right)(I-e)+RSe$$ (6)

$$w_{sp}=\left(R+\frac{n_{a}Rm}{n}-C(n_s-1)\right)(I-e)+RSe$$ (7)

Following the completion of all iterations, all individuals (including those expelled) reproduce asexually in proportion to their relative fitness (resource acquired) in the overall multigroup population (individual fitness – mean fitness) and immediately die. Offspring adopt the strategy of the parent with a chance of mutating to any strategies at a fixed rate (see table 1 for baseline conditions) and disperse to a random group to begin the next cycle of interactions (see fig 1 for pictorial model summary).

Table 1.

Definition of variables with baseline values. Simulations were terminated when stabilized by either fixing (>99% of the total population) for altruism or selfishness in the population, typically within 100 generations. Subsets of simulations were extended beyond the point of fixation for 1000 generation, assuring long-term stability. Populations of 10,000 individuals or greater produced equivalent results such that drift did not significantly influence outcomes.

Variable	Baseline	Definition
N	10,000	Total population size
n	5	Group size
I	10	Iterations played within each group per generation
R	10	Resources allocated each iteration
C	2	Cost of punishing each selfish individual
D	0.7	The probability of a single punisher detecting and punishing a given cheater
S	0.5	Cost of reduced fitness due to solitary behavior each round.
M	1−5	Mutation rate

Figure 1.

The population begins at set initial frequencies of strategies and individuals disperse randomly to groups (size n). Individuals interact within groups involving a cooperation phase and punishment for a set number of iterations (I). Following each iteration, individuals may remain in groups, die due to a fitness of ≤ 0, or be banished to the punishment patch (socially excluded) if the chance of being punished 1-(1-D^np) is greater than a random value (X). Once banished, the punished individuals acquire a fitness of R*S for the remaining iterations (e). Groups losing interacting members to either death and/or punishment are filled to the set group size randomly by individuals drawn from the population stock of the same frequencies from the beginning of the generation. At the conclusion of the iterations, individuals from both the interacting population and the punishment patch reproduce asexually (with a chance of mutation M) in proportion to their relative fitness in the overall population and die, thus resetting the population frequencies and beginning a new generation. For simulation runs, the frequency of punishment was varied every 0.01, from 0.02 to 0.05, and every 0.025 from 0.025 - 0.975 and the frequency of altruism was varied every 0.01 from 0.01 to the limits of the testable range as shown in figure 2.

Invasion of enforcement: altruistic punishers

Here we first present the analytical conditions in which altruistic punishment can maintain altruism within groups (with selfish punishment absent), then present multigroup simulations. Due to both first- and second-order costs of altruistic punishment, the conditions for the invasion of altruistic punishers are limited (Yamagishi 1986, Bowles and Gintis 2002, 2004, Fehr 2004). Furthermore, because altruistic punishers are altruists, as the initial frequency of altruistic punishers is increased, the corresponding initial frequencies of selfishness must decrease (i.e. frequency of altruistic punishers must be ≥ frequency of selfishness), thus reducing the testable range of predominantly selfish populations that altruistic punishment can invade compared to selfish punishers (fig. 2).

Figure 2.

Testable invasion ranges for altruistic and selfish punishment. The region above the solid line reveals the range of possible altruism and punishment combinations for the invasion of altruistic punishment. Points below the solid line comprise conditions by which the initial frequency of altruistic punishers is > the initial frequency of altruism, therefore an invalid combination. The testable range of selfish punishment resides below the dotted line, whereas points above the line are invalid as initial frequency of selfish punishers is > the initial frequency of selfishness (1 – frequency of altruism).

For altruistic punishment to outcompete selfish non-punishers within a single group, the costs of punishing cheaters must be less than the benefits lost by not remaining in the group for expelled cheaters (eq. 8).

$$C{n}_{s}I<e(R+\frac{n_{a}Rm}{n}-Rs)-RI$$ (8)

Although a within group perspective is insightful, this simplistic formulation does not consider the consequences of competition at the group level. Within groups, altruistic non-punishers undermine punishment, however groups policed by altruistic punishers fare better than those without punishers. This is readily observed when comparing the relative fitness differences over a single generation in scenarios when altruistic punishment effectively maintains cooperation in the mulitgroup model (fig 3a). Here we see that altruistic punishers initially have the lowest relative fitness due to exploitation by both selfish individuals as well as non-punishers. Over subsequent iterations, the relative fitness of both altruistic punishers and altruistic non-punishers increases and surpasses selfish non-punishers as these selfish individuals are expelled from the group (thus, obtaining a reduced payoff). Eventually altruistic punishers surpass altruistic non-punishers as the differences between groups become magnified through enforcement. However, as expected, under our baseline conditions, altruistic punishment requires either high levels of altruism or correspondingly low costs of punishment to successfully invade (fig 4).

Figure 3.

For simulations involving altruistic punishment (a), at the start of a generation, altruistic punishment has the lowest relative fitness due to first- and second-order costs. However, over time, altruistic punishers and altruistic non-punishers surpass the fitness of selfish non-punishers, with altruistic punishers ultimately having the greatest fitness due to their group-level advantage. Conversely, for simulations involving selfish punishment (b), selfish punishers have relative advantage over altruistic non-punishers at the start of the generation. However, over time the relative fitness of selfish punishers and altruistic non-punishers rises to surpass selfish non-punishers as they are expelled. The competition between selfish punishers ultimately limits their fitness relative to altruistic non-punishers due. Note that generations under baseline last 10 iterations, with effects here shown beyond 10 to reveal further consequences of group selection. Error bars indicate standard error of 50 replications of single generation runs.

Figure 4.

Invasion ranges for altruistic and selfish punishment in baseline conditions. The region above the altruistic punishment line indicates initial population frequencies altruistic punishment can successfully invade. The region above the selfish punishment line indicate initial population frequencies in which selfish punishment can invade. As expected, selfish punishment was able to invade a wider range of selfish populations than altruistic punishment. Invasion conditions were tested 10 times for combinations of punishment and altruism with the lines denoting the respective borders in which altruism was established in 100% of runs.

Invasion of enforcement: selfish punishers

In our previous models on the evolution of punishment, we demonstrated that punishers who cheat during the cooperation phase are more likely to invade than those that do not (Eldakar et al. 2007, Eldakar and Wilson 2008), as cheating (not cooperating) can directly offset the costs of punishment. For selfish punishment to outcompete selfishness within a group, the costs of punishment must be less than the difference in resources earned from remaining in the group and being expelled (eq. 9).

$$C(n_s-1)I<e\left(R+\frac{n_{a}Rm}{n}-Rs\right)$$ (9)

When comparing the relative fitness differences within multigroup populations policed by selfish punishment (with altruistic punishment absent) the relative fitness of selfish punishers as well as altruistic non-punishers begin low due to the costs of punishment and the costs of exploitation respectively (see fig. 3b). However, over time their relative fitness rises to surpass selfish non-punishers as they are expelled and overall exploitation is reduced. Selfish punishers, however, also recover the costs of punishment by exploiting the remaining altruists. As expected, under baseline conditions, selfish punishers invade predominantly selfish populations more readily than their altruistic counterparts (figure 4).

The transition of punishment

We predicted that selfish punishment and altruistic punishment can constitute transitional stages of enforcement over time. The conditions for which altruistic punishment outcompetes selfish punishment within groups resemble the similar conditions in which selfish punishment outcompetes selfish non-punishers, based on whether the costs of punishment is less than the fitness lost from expulsion (eq. 10).

$$C{n}_{s}I<e(R-RS+\frac{n_{a}Rm}{n})-RI+C(n_s-1)(I-e)$$ (10)

In a typical simulation run of multigroup populations under conditions in which selfish punishers can successfully invade and maintain altruism, plots tracking frequency over multiple generations demonstrates how the negative frequency dependence of selfish punishers ultimately limits their overall frequency in the population (fig 5). However, with cheating curtailed, altruistic punishment successfully invades from mutation and replaces selfish punishment. This transition of enforcement strategies ultimately drives altruism to fixation in all populations in which selfish punishers can initially invade, thus resolving the most limiting constraint of altruistic punishment theory, the ability to invade predominantly selfish populations. Here we now explore the parameter space of the model variables on the evolution of punishment in simulation of multigroup populations.

Figure 5.

Time series plots comparing the frequency of competing strategies over time. Under baseline conditions with 22% altruism and 10% punishment, (a) selfish punishment can successfully invade and establish altruism. Ultimately selfish punishers and altruistic non-punishers form a stable equilibrium with altruism maintained at ~75% of the total population. Under the same conditions (b) altruistic punishment cannot successfully invade and establish altruism. However, under the same conditions as (a), but allowing for mutation to altruistic punishment, altruistic punishment successfully emerge and replace selfish punishment. With the initial invasion of selfish punishment and subsequent invasion of altruistic punishment, altruism is ultimately established and maintained at >99% of the total population. Note that mutation to any strategy is permitted in all models, however in both (a) and (b) runs, the alternative punishment strategy is excluded. Error bars indicate standard error of 50 replications.

Cost of punishment

The cost of punishing transgressors is a crucial variable when considering the evolution of enforcement. Low cost punishment in the form of gossip (Wilson et al. 2000, Kniffin and Wilson 2005) and the potent negative consequences of having a poor reputation (Milinski et al. 2002) can be extremely effective; however, punishment may often take the form of more costly physical encounters as well (Clutton-Brock and Parker 1995). To account for the variation across social systems, we varied the costs of punishment from baseline values (20% of the resources allocated each iteration), to values of 40% and 5%, respectively.

When varying the costs of punishment, not surprisingly, increasing punishment costs reduced the effectiveness and evolvability of punishment, particularly hindering altruistic punishment as high punishment costs can plummet fitness to negative values/death (fig. 6). Reducing the costs of punishment, however, produced the opposite outcome and favored the evolvability of altruistic punishment over selfish punishment. This finding is not surprising when considering the different selection regimes that favor these alternative strategies. Reducing the costs of punishment favors all punishers by increasing their ability to purge selfishness from the group. Whereas the primary advantage of selfish punishment is the reduced relative costs of punishment compared to their altruistic counterparts, this advantage is minimized when costs of punishment are exceedingly low and the disadvantages of selfish punishment (reducing group productivity and vulnerability to expulsion) remain unchanged. This may also provide further insights to why gossip is most effective when aimed to benefit the group as opposed to when perceived as self-serving (Wilson et al. 2000, Hess and Hagen 2006).

Figure 6.

Invasion plots demonstrate how decreasing the costs of punishment (C) aids the invasion of both punishment strategies, while increasing the costs of punishment hinders the success of punishment. Interestingly, reducing the costs of punishment favors altruistic punishment (a) more so than selfish punishment (b). The lines indicate the border above which punishment can successfully invade and establish altruism in 100% of runs. The dotted lines indicate baseline conditions for comparison.

Costs of solitary behavior

Social animals depend on their social group for many fitness influencing behaviors such as foraging, protection, raising offspring, and mating opportunities (see review, Krause and Ruxton 2002), thus being expelled from social interactions can result in tremendous fitness costs for the excluded individual. Here we altered the costs of exile by varying the degree the individual’s resource allotment each round is reduced (S) from baseline values of 0.5, to values 0.3 and 0.7. To review, punished individuals do not take part in social interactions, thus once excluded receive a payoff of R*S for the remaining iterations of the generation. As expected, increasing the fitness of discrepancy between expulsion and remaining in a group increases the effectiveness/evolvability of both punishment strategies (fig. 7).

Figure 7.

Invasion plots demonstrate how varying fitness effects from expulsion from social interactions to solitary behavior (S) influence the effectiveness of punishment. When expulsion from group social interactions is more costly (reducing S), punishment overall is more effective for both strategies. When expulsion results in only a minor drop in fitness, punishment loses its effectiveness and therefore more difficult to evolve. Overall, across values of (S) selfish punishment remains more effective at establishing altruism in predominantly selfish populations (b) than altruistic punishment (a). The lines indicate the border above which punishment can successfully invade and establish altruism in 100% of runs. The dotted lines indicate baseline conditions for comparison.

Iterations

The length individuals stay within groups is a crucial variable to many social systems. The longer individuals stay within their social groups, the greater the potential strength of group selection. The increased duration of group life has two major effects on selection in the overall population: 1) the longer individuals associate in groups (between population mixing phases), the greater the ability of punishers to remove cheaters and reduce within-group variance relative to between-group variance and 2) the longer that variation is partitioned at the between-group versus within-group level, the greater the strength of selection at the level of the group in the overall population.

In randomly mixing populations, individuals can readily exploit groups and escape consequences as the shared fate of individuals within groups becomes less relevant. For example, in scenarios such as the tragedy of the commons, individuals selfishly exploit a common resource to the detriment of all group members. The longer individuals associate in groups with like strategies, the more the benefits of altruism are shared by altruists, and the consequences of selfishness are shared by other selfish individuals.

Here we varied the number of iterations (I) within a generation from the baseline value of 10, to 6 and 14. As expected, increasing the duration of group association increased the effectiveness of both punishment strategies to invade populations and establish altruism while reducing the duration of group association inhibited the invasion of punishment (fig. 8). Although increased group duration (and thereby group selection) is expected to favor altruistic punishment and potentially hinder selfish punishers, this would also dramatically aid selfish punishers when sufficiently rare as they benefit from the role of exclusive exploiter for even longer periods.

Figure 8.

Invasion plots demonstrate how varying iterations (I) influence the effectiveness of punishment. Increasing iterations, functionally increases group selection favoring both punishment strategies, while decreasing iterations favors purely selfish strategies. Overall, across values of (I) selfish punishment remains more effective at establishing altruism in predominantly selfish populations (b) than altruistic punishment (a). The lines indicate the border above which punishment can successfully invade and establish altruism in 100% of runs. The dotted lines indicate baseline conditions for comparison.

Group size

Group sizes vary within and between species (Krause and Ruxton 2002) as well as in evolutionary models from simple dyadic interactions (Trivers 1971) to large group scenarios (Boyd and Richerson 1992, Boyd et al. 2003). A prominent feature of group size is its effect on the partitioning of variance within and between groups (Aviles et al. 2002). As groups decrease in size, the probability that the variance within groups equals that of the overall population is reduced. Conversely, as groups increase in size, they are more likely to resemble the overall population, effectively reducing variance between groups. Thus all things considered, reducing group size is typically thought to favor group selected behaviors. Another feature of group size with respect to punishment strategies is that punishers can more readily influence within group variation as expelling the same percentage of cheaters from a small group requires less effort than in larger groups. Therefore as expected, reducing group size aided the evolution of punishment, while increasing group size reduced its effectiveness (fig. 9).

Figure 9.

Invasion plots demonstrate how varying group size (n) effects influences the effectiveness of punishment. Decreasing group size increases the effectiveness of punishment to reduce the phenotypic variance within groups, as smaller groups means less individuals to potentially punish. When group size is increased, it is more costly for punishers to effectively purge out cheaters. Overall, across values of (n) selfish punishment remains more effective at establishing altruism in predominantly selfish populations (b) than altruistic punishment (a). The lines indicate the border above which punishment can successfully invade and establish altruism in 100% of runs. The dotted lines indicate baseline conditions for comparison.

Discussion

Punishment is difficult to explain because, like altruism, those who benefit from the behavior do not necessarily share in the costs. This is especially true for altruistic punishment, as the two-fold relative fitness cost of altruism and punishment makes invasion of predominately selfish populations exceedingly problematic. Here we show that this evolutionary obstacle is readily resolved if punishment originates with selfishness. In general, selfish punishers are better adapted to invade selfish populations, as they benefit from the role of exclusive exploiter (and protector) of flocks of altruists. This benefit, however, is undercut once selfish punishers become sufficiently abundant, and competition between them ultimately limits their frequency in the population. Thus, once selfishness is effectively curtailed, and the necessity of punishment is reduced, the selective advantage of altruistic punishment is realized.

The stabilization of altruism from originally selfish populations by selfish punishment strategies has been quite successful in human societies. The origins of the mafia began in areas devoid of government policing (Lupo 2009). Where official policing was incapable of maintaining order, prosperous families often associated with criminal activity became the unofficial enforcers of order. Criminal organizations reduced crime and supplied public services in some communities in the United States before cooperative/less exploitative governmental policing were able to assume control (Fiorentini and Peltzman 1995, Lupo 2009). Furthermore, the aforementioned example of the Castillians provides an ideal example where the selfish punishers themselves transition to more altruistic defenders. These individuals, initially thought of as bands of thugs, were later revered as honorable defenders of the commons once the competition between other groups became paramount (Bison 1994).

Although our model suggests that selfish punishers are entirely replaced by their altruistic counterparts, it is likely that natural populations are not as stable and a mix of strategies may always remain present to some extent. Fluctuations in density, cost/benefit structure, group size, and group longevity, may preserve populations where all strategies are maintained. Furthermore, large scale trends at the level of governments may cycle independent of smaller scale enforcement at the local level. Thus while human societies may resemble an overall transition from totalitarian states to democratic republics, populations my nevertheless encompass diverse punishment strategies in economic games (considering studies that include selfish punishment Eldakar et al. 2006, O’Brien et al. in press), as well as natural variation in genetic predispositions to punish transgressors (Mc Dermott et al. 2009).

The variation of enforcement strategies in natural populations may also depend on species specific or context dependent parameters. One likely important parameter is the existence of power asymmetries. In theoretical models, individuals are often assumed equivalent with regards to their ability to carry out various strategies. However, considering natural selection requires variation in strategies for evolution to occur, individuals should also vary in their effectiveness of performing these various strategies. For example, in pigtailed macaques (Macaca nemestrina), third-party punishment which reduces within group conflict (Flack et al. 2005) and stabilizes social niches (Flack et al. 2006), is performed by the more powerful individuals in the group. In rhesus macaques (Macaca mulatta), individuals are punished for failing to announce their discovery of food. However, high ranking males were not punished for their own deception yet remained those most vigilant against other transgressors (Hauser and Marler 1993), further demonstrating how particular strategies are associated with certain individuals. This may be especially true for selfish punishment. Human examples of selfish punishment often associate punishment behavior with those most capable of performing the very strategy they are defending against. In other words, it may take a thief to catch a thief. Economists Russel Sobel and Brian Osoba (2009) have proposed that youth gangs form in response to violence, such that in regions where governmental enforcement is absent, violent gangs form to effectively reduce further violence. The mafia is thought to have also emerged due to a lack of government protection. Organized crime is just that, crime, however, the presence of the mafia functionally reduces other sources of crime and improves the quality of the region they both protect and exploit (Fiorentini and Peltzman 1995; Salvatore Lupo 2009). Thus while we have not included power asymmetries or increased efficacy of cheaters finding/punishing other cheaters in this model, it appears that these are important components of selfish punishment and may partially explain why selfish punishers are ultimately always replaced by altruistic punishment in our model.

Another feature of our model that may have also ushered in the eventual and complete replacement of punishment strategies was the use of discrete instead of continuous traits. Selfish punishers by definition are individuals that punish yet acquire exclusive benefits for this behavior that are otherwise unavailable to non punishers. In our previous model using continuous traits, the initially independent traits of altruism and punishment became negatively correlated over time (Eldakar et al. 2007). Although the strength of this negative correlation varied across conditions (group size, cost of punishment, and iterations), we considered the presence of a negative association between altruism and punishment as evidence of selfish punishment. Because this creates a very broad categorization of selfish punishment, here we opted to use discrete traits to eliminate any ambiguity regarding punishment types and observe which conditions favor particular strategies in their simplest form. Therefore, it may be predicted that a continuous trait model would have stabilized at a middle point between the two strategies. However, using discrete strategies we were able to show the fundamental differences between the two punishment types and demonstrate their transitional relationship in producing altruistic populations.

Since the inclusion of punishment in social evolutionary models, there has been an explosion of punishment strategies and discussion on how these strategies interact. This is most evident in recent models (e.g. Rand and Nowak 2011, Garcia and Traulsen 2012) that include all punishment strategies such as altruistic punishment, selfish punishment, antisocial (punishers target altruists), and even loner strategies that do not partake in the social exchanges (cooperation/selfishness) yet can punish and be punished by others. These models result in various conclusions that may confuse as to how punishment influences social evolution, and what it means when models involving the same strategies produce very different outcomes. Altruism can evolve in population structures with stronger selection between groups than within groups (strategies cluster in groups compared with a randomly mixing population). This is because a continuously random mixing population is effectively a single large group, favoring selfish individuals based on their within group advantage. If individuals stay in groups for longer periods, selfish individuals suffer from the consequences of decreased group fitness (productivity), compared to individuals within more altruistic groups. The longer individuals stay in groups, the greater the differences between groups matters. Punishment of selfishness augments altruism in the middle ground, because it reduces the within group advantage of selfishness over altruism (weakens competition within groups). Strategies that undermine punishment of selfishness (such as antisocial punishment) will have the opposite effect. This is because the punishment of altruists magnifies the competitive disadvantage of altruism within groups. While altruistic punishment and antisocial punishment are still considered “punishment”, they are very different in their influence on within- and between-group competition, and the balance of within- and between-group selection.

Population structures that increase the relative strength of selection between groups will favor altruistic punishment (e.g. Boyd et al. 2003) or even altruism without the need for punishment (Wilson 1975, Wilson and Wilson 2007). Conversely, population structures that increase the relative strength of selection within groups will favor strategies such as selfishness, antisocial punishment (Hauert et al. 2007, Garcia and Traulsen 2012), and even behaviors produced by extreme within-group selection (e.g. spite, see Hamilton 1964, Gardener and West 2004, West and Gardener 2010). Anything in between may result in a mixture of all strategies. Thus, a single model can produce pure altruism (strategies that benefit the group at a cost to self), or pure spite (strategies that harm the group at a cost to self, but actor has a reduced fitness loss compared to recipients) by only altering the population structure. And a model can produce altruistic punishment, or destroy altruistic punishment through antisocial punishment in the same manner. To understand why particular models favor alternative forms of punishment, one must first understand how selection acts within and between groups, and how punishment mediates this balance in general.

Summary

In this model we set out to demonstrate how enforcement can invade and ultimately stabilize altruism. Previous models have shown the effectiveness of altruistic punishment and have also addressed its evolutionary pitfalls of initial invasion. For instance, features such as conformance bias transmission Henrich and Boyd (2001), coalitional enforcement (Bingham 1999), coordination (Boyd et al. 2010, Perc 2012), opting out of social interactions (see Hauert et al. 2007, Garcia and Traulsen 2012), and reputation (Brandt et al. 2003) have all dramatically improved the ability of altruistic punishment to both avoid second-order costs and invade more selfish populations. However, these added features often carry additional evolutionary requirements such as cognition, or other behavioral features such as the willingness to coordinate and reputation networks to already be available when altruistic punishment emerges. Although these are all probable features present in humans, here we took the opposite approach to resolving the dilemma of punishment, steering away from adding complexity. Instead, we looked to the most parsimonious yet ignored starting point, the initially selfish population, and have shown how hawks can give rise to doves.