Correlated pay-offs are key to cooperation

Abstract

The general belief that cooperation and altruism in social groups result primarily from kin selection has recently been challenged, not least because results from cooperatively breeding insects and vertebrates have shown that groups may be composed mainly of non-relatives. This allows testing predictions of reciprocity theory without the confounding effect of relatedness. Here, we review complementary and alternative evolutionary mechanisms to kin selection theory and provide empirical examples of cooperative behaviour among unrelated individuals in a wide range of taxa. In particular, we focus on the different forms of reciprocity and on their underlying decision rules, asking about evolutionary stability, the conditions selecting for reciprocity and the factors constraining reciprocal cooperation. We find that neither the cognitive requirements of reciprocal cooperation nor the often sequential nature of interactions are insuperable stumbling blocks for the evolution of reciprocity. We argue that simple decision rules such as ‘help anyone if helped by someone’ should get more attention in future research, because empirical studies show that animals apply such rules, and theoretical models find that they can create stable levels of cooperation under a wide range of conditions. Owing to its simplicity, behaviour based on such a heuristic may in fact be ubiquitous. Finally, we argue that the evolution of exchange and trading of service and commodities among social partners needs greater scientific focus.

1. Introduction

Current scientific knowledge about the evolutionary stability of cooperation results to a great extent from the study of families and closed societies, where interactions mainly involve relatives and can hence be explained by kin selection [1]. However, the general belief that kin selection is necessary to maintain cooperation and altruism in social groups has recently been challenged by results from cooperatively breeding insects and vertebrates, in which groups are composed mainly of non-relatives (e.g. [2–5]; cf. [6,7]), and by the realization that cooperative interactions among non-relatives are widespread in contexts outside of cooperative breeding (e.g. [8–11]). This observation allows testing predictions of alternative concepts to kin selection theory without the confounding effect of relatedness, which may provide a more comprehensive understanding of evolutionary mechanisms promoting cooperation.

Here, we discuss the evolution of cooperation based on mutual fitness benefits and correlated pay-offs. Our primary focus will be on reciprocity, which we regard as a deplorably underestimated mechanism underlying many of the most intriguing forms of cooperation in nature. We shall explain the different forms of reciprocity, the prerequisites for its evolutionary stability, favourable and detrimental conditions for its evolution, and the importance of context for the evolution of cooperation in general. However, first we shall briefly introduce the different concepts that have been developed to explain the evolution of cooperation.

2. Cooperation for mutual fitness benefits

If a cooperative behaviour results in net fitness benefits to the actor, irrespective of the behaviour of an interaction partner, i.e. if the benefits of acting outweigh the costs, the behaviour will be selected irrespective of its potential fitness effects on the partner. If two or more individuals interact in this way, we usually call this mutualistic. Such mutualism is fundamentally distinct from altruism (see box 1 for definition), where an individual benefiting an interaction partner bears fitness costs by its behaviour. A mutualistic behaviour yields beneficial effects to the direct fitness of the actor, and hence cheating is not favoured by selection [13,14]. For instance, if improving the survival chances of group members by protecting them from external threats also enhances the survival prospects of the actor, this behaviour will be selected as long as its direct fitness costs do not exceed the actor's own fitness benefits (group augmentation benefits; [15–17]). Clearly, cooperation based on mutualistic interactions does not pose the same evolutionary paradox identified by Darwin [18]. Hence, the greater challenge for evolutionary biologists is to explain cases in which the immediate costs of an act that benefits another individual outweigh the immediate benefits to the actor, i.e. altruistic help or cooperation.

Box 1. Glossary.

Social semantics is a contentious issue. Much controversy about the importance of different evolutionary mechanisms of cooperation results just from divergent understanding of terms. Therefore, to minimize the danger of creating misunderstandings, we use everyday language terms in the way most closely resembling the connotation they have in daily life. In addition, our intention is to simplify the terminology in this field wherever fragmentation of terms in myriad subcategories seems superfluous. Cooperative behaviour involves a limited number of basic principles, which need to be represented by appropriate terms.

Many terms used in everyday language in the context of cooperation imply costs and benefits measured in some currency. In evolutionary biology, the ultimate currency is Darwinian fitness, which may be subject to some specification, such as direct, indirect or inclusive fitness. If not specified otherwise, costs and benefits refer to direct fitness in the definitions given below.

Altruism: This is defined by the immediate consequences of an action. It refers to a behaviour by which an individual (actor) benefits someone else (receiver(s)) at some immediate cost to itself. This does not make assumptions about whether and how these costs may be compensated by, for example, future benefits (cf. reciprocal altruism) or fitness benefits to relatives (indirect fitness benefits).

By-product mutualism: Synonymous with ‘mutualism’ (see below). As behaviours or other traits must have beneficial fitness effects to each actor by themselves to classify as mutualism, the benefits to interaction partners are of secondary importance for the trait to be selected.

Coaction: Concurrent acting of two or more individuals.

Cooperation: Simultaneous or consecutive acting together of two or more individuals by same or different behaviours. Costs and benefits to either partner are not implied (i.e. net fitness benefits of cooperation may or may not result to one or all involved parties).

Correlated pay-offs: This refers to the fitness effects of a trait on two or more individuals. Altruistic behaviour can be favoured by natural selection if its fitness effects on donor and recipient are positively correlated.

Help, helping: Action of an individual to the apparent benefit of one or several receivers. This term is devoid of assumptions about costs to the actor.

Mutualism: A cooperative trait or behaviour enhancing the (inclusive) fitness of each involved party. The behaviour by itself has beneficial fitness effects, irrespective of the behaviour of the interaction partner. Therefore, it cannot be cheated.

Pay-to-stay: Tolerance of subordinates by dominants contingent on the delivery of some service or commodity (such as paying ‘rent’; cf. ‘trading of service and commodities').

Reciprocal altruism (sensu Trivers [12]): A mechanism by which the costs of an altruistic act are compensated in the future by some form of reciprocity.

Reciprocity: This is essentially a proximate (i.e. mechanistic) concept implying certain cost/benefit relationships. At the ultimate (i.e. evolutionary) level, this term refers to an apparently helpful behaviour that benefits a receiver of the act at immediate costs to the actor. At the same time, it increases the probability of receiving benefits in return, from the same or different partners. Reciprocation is hence intrinsically altruistic and prone to cheating. At the proximate level, there are three forms of reciprocity implying different decision rules:

Generalized reciprocity: Help anyone if helped by someone.

Direct reciprocity: Help someone who has helped you before.

Indirect reciprocity: Help someone who is helpful.

Trading of service and commodities: Concurrent or consecutive exchange of work or goods between individuals, either in the same or different currencies.

3. Correlated pay-offs

If a cooperative act itself causes net fitness costs, i.e. the costs of acting are exceeded by its benefits, such altruistic behaviour can only be favoured by natural selection if the fitness effects of actor and receiver are somehow positively correlated. There are three principal ways in which such a correlation can come about [19]. (i) Genealogy, which means that interaction partners share genes with above-random likelihood by common descent (kin selection; [1,20]). (ii) A genetic correlation between genes coding for cooperation and phenotypic traits that can be used to identify the bearers of such altruism genes (‘green-beard effect’; [21–24]). (iii) An above-random chance that help provided by an actor will increase the likelihood that the costs of this act will be outweighed by benefits accrued from receipt of help in the future (reciprocity; [12,25]).

Intraspecific cooperation and altruism have been studied most extensively in the context of correlated pay-offs generated by genetic relatedness, which has revealed great explanatory power [26–30]. In contrast, the general importance of green-beard effects for the evolution of cooperation has been questioned on theoretical grounds [26] and its prevalence is currently unclear, partly because of the difficulty in detecting such mechanisms outside of microorganisms [24,31]. Reciprocity, the third potential cause of correlated pay-offs, is arguably the most contended possibility to generate cooperation and altruism [12,32]. It is the theme of this article and a focus of many contributions to this special issue of the Philosophical Transactions [7,33–37].

4. What is reciprocity, and when should it be evolutionarily stable?

What is so special about the possibility that a cooperative act bears net costs to the actor without benefitting individuals sharing the genes responsible for this altruism? Obviously, such behaviour will be counter-selected if other individuals in the population abstain from such costly behaviour while still obtaining potential benefits from others' help [19,26]. In other words, cooperation based on such a mechanism can be exploited by cheating [13]. There are various ways in which exploitation of cooperative input can be avoided. Generally, evolutionarily stable cooperation can result if a helpful act increases the likelihood of obtaining fitness benefits in return that outweigh the costs of the cooperative investment. Reciprocity involves a better than random guess about whether the help provided to somebody else increases the likelihood of obtaining fitness benefits in return. Therefore, reciprocal help will much more easily establish stable cooperation between social partners if the exchange is concurrent (‘coaction’) than if there is a considerable time lag between actions [38]. A concurrent exchange of services and commodities allows for immediate responses to the behaviour of the interaction partner, which eliminates the uncertainty about the recompense [36]. In contrast, if there is a time delay between successive interactions, a prudent decision about whether or not to help an interaction partner is contingent on previous experiences and may take one of three forms: generalized, direct or indirect reciprocity [39,40]. In the following, we shall discuss these different possibilities and both their underlying assumptions and empirical evidence.

(a). Generalized reciprocity

If an individual experiences help from someone in the population, this may increase its propensity to behave cooperatively towards any other individual, thereby applying the decision rule ‘help anyone if helped by someone’. This possibility has been referred to variously as ‘upstream tit-for-tat’ [41], ‘upstream indirect reciprocity’ [42], ‘serial reciprocity’ [43], ‘upstream reciprocity’ [44], ‘pay-it-forward reciprocity’ [45–47] or ‘generalized reciprocity’ [48–54]. We shall stick to the latter term in this article. Generalized reciprocity has been found to generate evolutionarily stable levels of cooperation in theoretical models assuming a wide range of conditions, such as small group size, population viscosity, the existence of interaction networks, the possibility to opt out contingent on social experience, or the application of simple updating rules in response to previous interactions [41,42,44,46,48,49,52–54]. Several animals have been shown to apply this simple rule in experiments controlling for alternative mechanisms (Norway rats: [51]; domestic dogs: [55]; capuchin monkeys: [56]), and it has been shown to be readily used by humans (e.g. [50,57,58]). It is currently unknown how widespread the application of this simple heuristic is by animals in nature. It should be noted that all types of reciprocity mechanisms can hardly be shown conclusively in wild animals, without the possibility of controlling for possible alternative explanations by stringent experimentation. However, there are many contexts in which generalized reciprocity rules may be employed in social groups, including mutual vigilance, cooperative hunting and territory defence, alternation between leading and following positions in group locomotion, mutual grooming or sharing of limited resources such as food and shelter. For example, in guppies, Poecilia reticulata, experimental subjects interacted more cooperatively with unfamiliar partners after receiving cooperative experience with others, depending on environmental conditions and sex [59], and in the field, guppy social networks are positively assorted by cooperative predator inspection behaviour [60]. So even if generalized reciprocity has not been unequivocally demonstrated in wild guppies, the crucial preconditions for this behaviour seem to occur in this species.

The behavioural decision mechanism employed in generalized reciprocity resembles rules applied by many animals in an agonistic context. When individuals have won or lost an encounter with some (known or anonymous) member of the population, they behave differently in a subsequent encounter with someone else, resulting in the renowned ‘winner and loser effects' [61,62]. These effects are among the most widespread social phenomena known in animals, and anyone who has ever observed sequences of aggressive encounters between animals has probably recognized significant carry-over effects between subsequent contests. To explain the behavioural changes underlying winner and loser effects of course requires the use of highly controlled experimental conditions [62]. For example, male rats have been shown to attack unknown competitors more immediately after winning a contest with someone else, whereas they behaved more submissively towards an unfamiliar conspecific after losing a previous contest. This contingent behaviour reduces contest costs significantly [63]. Hence, social experience with anonymous partners may help individuals to make prudent decisions in future interactions with others. Humans, for instance, have been shown to ‘pay forward’ both received and withheld help (or ‘greed’; [47]) from anonymous interaction partners, confirming that social experience can affect behaviour towards others in both sociopositive and socionegative contexts.

(b). Direct reciprocity

Receiving help from a social partner may increase one's propensity to help this individual in return at a subsequent occasion [12]. The underlying decision rule ‘help someone who has helped you before’ is most commonly referred to as ‘direct reciprocity’, which can generate evolutionarily stable levels of cooperation as suggested by a large number of theoretical models [25,64–67]. Numerous examples from a wide range of taxa reveal that animals do reciprocate help, both under natural conditions and in controlled laboratory experiments (table 1). In primates, for instance, comparative studies revealed significant effects of reciprocity on the exchange of goods and services. A meta-analysis of 25 social groups belonging to 14 species revealed a significantly greater effect of reciprocity than kinship on allogrooming between social partners [151]. Another meta-analysis involving 32 study populations of six species (including humans) revealed significant effects of reciprocity on food sharing and trading food against other commodities, independent of relatedness. In both studies, when controlling for kinship, roughly 20% of the variance in the exchange of hygiene, food and other commodities were explained by direct reciprocity.

Table 1.

Examples of reciprocal cooperation in animals. This table indicates the great diversity of reciprocal exchanges observed in vertebrates, including examples for different mechanisms of reciprocity and commodity trading. It is by no means exhaustive. The evidence for reciprocity as reported in the peer-reviewed literature referred to here is varied, including studies performed under field, seminatural and laboratory conditions, and ranging from correlation analyses of reciprocal exchange observations (obs) to results from stringently controlled experiments (exp) testing for application of specific decision rules. The majority, but not all of the studies provide evidence that relatedness has been excluded as a major explanatory variable, either by the nature of group structure, experimental design or statistical analysis. Reciprocal cooperation in reproduction between unrelated male competitors has been reviewed already in previous publications ([68]: 33 fish species additional to examples contained in this table; [69]: 87 species including invertebrates, fishes, birds and mammals) and is therefore not represented here.

species	common name	order, family	form of reciprocity	obs/exp	field/lab	references
Gasterosteus aculeatus	threespine stickleback	Gasterosteiformes, Gasterosteidae	cooperative predator inspection	E	L	[70,71]
Poecilia reticulata	guppy	Cyprinodontiformes, Poeciliidae	cooperative predator inspection	E/O	L/F	[72,73]
Siganus corallinus S. doliatus S. puellus S. vulpinus	rabbitfish	Perciformes, Siganidae	reciprocal vigilance among long-term dyads	O	F	[74]
Neolamprologus pulcher	princess of Lake Tanganyika	Perciformes, Cichlidae	commodity trading based on pay-to-stay negotiations among breeders and helpers	O/E	F/L	[75–86]
Pelvicachromis pulcher	Kribensis cichlid	Perciformes, Cichlidae	satellite males trading fertilization access against help in brood care and defence	O	L	[87]
Symphodus ocellatus	Ocellated wrasse	Perciformes, Labridae	satellite males trading fertilization access against help in defence and female attraction	O/E	F	[88]
Geronticus eremita	northern bald ibis	Ciconiiformes, Threskiornithidae	turn-taking in leading flight position	O	F	[89]
Uria aalge	common guillemot	Charadriiformes, Alcidae	allopreening	O	F	[90]
Tyto alba	barn owl	Strigiformes, Tytonidae	reciprocal allopreening, food exchange, and trading preening for food, among nestlings	E	F	[91]
Phoeniculus purpureus	green woodhoopoe	Bucerotiformes, Phoeniculidae	allopreening	O	F	[92]
Ficedula hypoleuca	pied flycatcher	Passeriformes, Muscicapidae	reciprocal defence of neighbours against predators	E	F	[93–96]
Parus major	great tit	Passeriformes, Paridae	parental turn-taking in chick provisioning	O	F	[97]
Sitta pusilla	brown-headed nuthatch	Passeriformes, Sittidae	allopreening	O	F	[98]
Cantorchilus leucotis	buff-breasted wren	Passeriformes, Troglodytidae	allopreening	O	F	[99]
Taeniopygia guttata	zebra finch	Passeriformes, Estrildidae	reciprocal food provisioning	E	L	[100]
Malurus cyaneus	superb fairy wren	Passeriformes, Maluridae	pay-to-stay interactions among male breeders and male helpers	E	F	[101]
Cyanocitta cristata	blue jay	Passeriformes, Corvidae	reciprocal food exchange under reduced discounting effects	E	L	[102]
Corvus corax	raven	Passeriformes, Corvidae	cooperative loose-string pulling task	E	L	[103]
Desmodus rotundus	common vampire bat	Chiroptera, Phyllostomidae	reciprocal food provisioning and allogrooming	O/E	F/L	[7,104–107]
Artibeus jamaicensis	Jamaican fruit-eating bat	Chiroptera, Phyllostomidae	subordinate males trade territory defence against reproduction and territory inheritance	O/E	F	[108,109]
Apodemus microps	herb-field mouse	Rodentia, Muridae	allogrooming	O	L	[110]
Rattus norvegicus	Norway rat	Rodentia, Muridae	generalized and direct reciprocity in food exchange task	E	L	[39,51,111–113]
Eulemur fulvus	red-fronted lemur	Primates, Lemuridae	allogrooming	O	F	[114]
Saguinus oedipus	cotton-top tamarin	Primates, Callitrichidae	direct reciprocity in food exchange task; trading of grooming against alloparental infant carrying	E/O	L	[115,116]
Cebus apella	tufted capuchin	Primates, Cebidae	reciprocal food exchange; allogrooming; grooming for infant handling opportunities	E/O	F/L	[117–120]
Cebus capucinus	white-faced capuchin	Primates, Cebidae	allogrooming	O	F	[121]
Cercocebus atys	sooty mangabey	Primates, Cercopithecidae	trading grooming for infant handling opportunities	O	F	[122]
Lophocebus albigena	grey-cheeked mangabey	Primates, Cercopithecidae	allogrooming	O	F	[123]
Cercopithecus aethiops	vervet monkey	Primates, Cercopithecidae	trading grooming for social support, food provisioning, or infant handling opportunities	E/O	F	[122,124–126]
Macaca fascicularis	long-tailed macaque	Primates, Cercopithecidae	trading grooming for social support; allogrooming	E/O	L	[127,128]
Macaca sylvanus	barbary macaque	Primates, Cercopithecidae	allogrooming and trading grooming for agonistic support	O	F	[129]
Macaca fuscata	Japanese macaque	Primates, Cercopithecidae	allogrooming; covariation of allogrooming and social support	O	L	[130,131]
Macaca thibetana	Tibetan macaque	Primates, Cercopithecidae	allogrooming	O	F	[132,133]
Papio ursinus	chacma baboon	Primates, Cercopithecidae	allogrooming	O	F	[134,135]
Papio hamadryas	hamadryas baboon	Primates, Cercopithecidae	allogrooming; trading grooming for social support	O/E	L/F	[136,137]
Papio anubis	olive baboon	Primates, Cercopithecidae	role exchange in male coalitions against opponents; females trading grooming for infant handling opportunities	O	F	[138,139]
Pongo pygmaeus	orangutan	Primates, Hominidae	reciprocal token exchange	E	L	[140]
Pan troglodytes	chimpanzee	Primates, Hominidae	reciprocal food exchange; trading grooming for social support and food; allogrooming; reciprocal exchange of food, and trading of food for social support; direct reciprocity in food exchange task	O/E	L/F	[141–149]
Pan paniscus	bonobo	Primates, Hominidae	allogrooming	O	F	[150]
14 species	nine genera	Primates	allogrooming (meta-analysis)	O	L/F	[151,152]
14 species	eight genera	Primates	trading grooming for social support (meta-analysis)	O	L/F	[153]
six species (incl. Homo sapiens)	four genera	Primates	reciprocal food sharing and trading other commodities (meta-analysis)	O	L/F	[154]
Rangifer tarandus	reindeer	Artiodactyla, Cervidae	allonursing	O	F	[155]
Aepyceros melampus	impala	Artiodactyla, Bovidae	allogrooming	O	F	[156,157]
Tursiops aduncus	Indo-Pacific bottlenose dolphin	Artiodactyla, Delphinidae	reciprocal flipper rubbing	O	F	[158]

Nevertheless, some researchers have expressed doubts about the prevalence of reciprocal cooperation in natural systems [13,159,160], but this seems to result from semantic disagreements rather than from disputes about observed behaviours [32,161]. Alternative positions in this debate depend mainly on the way in which direct reciprocity is defined. If only ‘tit-for-tat-like’ exchanges of help within narrowly defined rules and pay-off relations of the iterated prisoner's dilemma are considered, scepticism seems to be indicated. Such conditions are unlikely to be widespread in nature [34]. However, the rule ‘help someone who has helped you before’ can be selected under a much wider range of conditions, for instance involving individual relationships such as partnerships and friendships [7,162–164], and contingency may be based on the integration of several interactions over longer timespans (attitudinal reciprocity [165]). If exchanges of different commodities are considered as well [136], the power of this mechanism emerges even more clearly (see [166] for a discussion of underlying mechanisms). It is an unfortunate misunderstanding that the functionality of direct reciprocity is often tied to a narrowly defined set of exchange rules used to model stable solutions to the prisoner's dilemma, such as ‘tit-for-tat’ exchanges or ‘calculated reciprocity’ [165,167]. Nevertheless, the prisoner's dilemma pay-off matrix is crucial to considerations of direct reciprocity: helping a social partner must at the moment involve higher costs than benefits, and if the partner does not reciprocate in any way in the future to an extent that the costs of the initial helpful act are fully compensated, the cooperator will receive the ‘sucker's pay-off’, i.e. fare worse than if it had not performed the helpful act in the first place.

(c). Indirect reciprocity

Individuals may decide to help generally cooperative social partners, even if they have not interacted with them before. This cooperation mechanism applying the decision rule ‘help someone who is helpful’, which has been named ‘indirect reciprocity’ [41,168], can generate stable levels of cooperation in a population if individuals can improve their reputation by being helpful ([169–175]; see [35] for review). The behavioural mechanism involved depends on public information obtained for instance by eavesdropping [176]. Cooperation based on indirect reciprocity has been shown to occur in humans [35,177]. Cooperation may be used as a signal, which can lead to competition among group members about ‘showing off’ by exerting altruistic help [178]. Such reputation-based partner choice can increase pay-offs obtained by cooperation above those from indirect reciprocity [179].

5. Which conditions select for reciprocity?

In analogy to Hamilton's rule for the evolution of cooperation by pay-offs correlated through relatedness [1], the profitability of behaving reciprocally depends on the relationship between the fitness costs (c) of helping to the actor, the fitness benefits (b) of helping to the receiver, and the probability (w) of receiving help of corresponding value back in the future as a consequence of helping [180]. Helping is favoured where w > c/b, which means that reciprocity is more likely to evolve if (1) the actor's costs of helping are low, (2) the benefits of help to receivers are high and (3) occasions where help can be exchanged are sufficiently frequent. Do these conditions concord with observed instances of reciprocal cooperation?

• (1) Costs of reciprocal cooperation have been shown to constitute an important trigger for the decision of Norway rats to help a social partner to obtain food. If the expense to a donor was experimentally raised by increasing the resistance of a mechanism by which the actor could pull food towards a receiver, the rats distinguished much more precisely between prospective receivers that had helped them before from those that had not [111].

  The assumption that costs of cooperation should be low for reciprocity to evolve seems to accord with the frequent occurrence of reciprocal grooming in mammals and allopreening in birds (table 1). The temporary fitness costs of mutual hygiene have been assumed to be low [13,153,181], but the involved time effort, energy expenditure and risk enhancement owing to reduced vigilance should not be underestimated [182]. Time costs of grooming and preening have been experimentally demonstrated for instance in great tits [183], bats [184] and gerbilline rodents [182,185], and energetic costs were demonstrated for grooming among bats [184] and preening among penguins [186]. Vigilance costs of grooming were found in antelopes [187,188] and gerbils [185]. Furthermore, allogrooming may facilitate parasite transmission [189] and involve various physiological costs (see [182] for review). Even though many of the above-mentioned costs were measured in autogrooming, extrapolation to costs of allogrooming seems justified (cf. [187]). This casts some doubt on the assumption that grooming is cheap with regard to potential fitness effects.

• (2) The prediction that high benefits must be gained by receivers in reciprocal cooperation has been confirmed by the mutual blood donations provided by vampire bats, which face a high starvation risk owing to their small body size [7,104]. Blood is donated to related and unrelated individuals, but experiments have revealed that food received from a social partner in a previous interaction predicts food donations by roughly an order of magnitude better than relatedness [105]. Nearly two-thirds of all blood sharing dyads in the study involved unrelated social partners. Moreover, the food-sharing network correlates with mutual allogrooming in this species [105], indicating service reciprocity based on an interaction network, as suggested also by field observations [106].

  Reciprocal food provisioning has been experimentally demonstrated also in Norway rats [39], where the propensity to pay back received help depends on the quality of help they received [112]. Donors that provided (attractive) banana to their interaction partner receive more food back than donors of (less attractive) carrot. Similar to vampire bats, the potential benefit to the receiving rat is apparently considered as well in the helping decisions of donors [111], as prospective receivers in need, i.e. when hungry, were serviced more readily if their body condition was low. Reciprocal food sharing occurs also in other animals and is particularly common in primates, including humans [154]. A meta-analysis on 32 independent study populations of six primate species revealed similar effect sizes in humans and primates [154], which contradicts the frequently expressed apprehension that cognitive constraints prevent animals from showing direct reciprocity [190].

  Regarding fitness consequences of reciprocal food sharing, important effects have been suggested for vampire bats [7,104,105], where the benefit of receiving a blood donation are typically much higher than the costs of providing it owing to the inevitable threat of starving to death when not obtaining a blood meal within a maximum period of 70 h [104,191]. Such asymmetries between costs of provided and benefits of received help can immensely enhance the evolutionary stability of reciprocity, which may explain also the condition-dependent help rats provide to hungry partners [111]. Regarding the potential fitness effects of other reciprocal exchanges, significant fitness benefits were demonstrated for reciprocal allopreening in common guillemot [90], both in the short- and long-term, even if mutual hygiene has been assumed to have little fitness value [192]. A positive fitness effect is likely also if social support is provided in return to same or different commodities or services received from a social partner in the past, as often observed in primates (male olive baboons: [138]; vervet monkeys: [124]; long-tailed macaques: [127]; female hamadryas baboons: [136]; snub-nosed monkeys: [193]). Significant fitness effects of the trading of different commodities and services among social partners can be demonstrated by cooperative breeders exchanging help in different functional contexts. In the cooperative breeder Neolamprologus pulcher, for instance, subordinates gain essential protection by the defence behaviour of dominant group members [194,195], while in return they boost the reproductive success of the dominants by their help in direct brood care and defence against predators of eggs and larvae [75,76,194,196].

• (3) The third condition favouring the occurrence of reciprocal cooperation is that the probability of receiving compensation for provided help in the future is high. This depends on several factors, including:

  • (a) Group size. In small groups, (i) the simple decision rule ‘help anyone, if helped by someone’ (generalized reciprocity) can create evolutionarily stable levels of cooperation [49]; (ii) social interactions and the history of cooperation of social partners can be much more easily and cheaply monitored than in large groups, which can favour both direct and indirect reciprocity [170].
  • (b) Interaction frequency. The more often individuals interact with each other, the larger is the probability that evolutionarily stable levels of cooperation will be established in a population, regardless whether by generalized, direct or indirect reciprocity [53,170]. A condition favouring frequent reciprocal interactions is the existence of stable group structures. The exchange of blood meals in vampire bats depends primarily on demand and previously received help [105], but a long-term relationship between social partners and frequent interactions may be prerequisites of such successful reciprocation. This has been suggested also by experimental results in zebra finches, which maintained high levels of cooperation in an iterated prisoner's dilemma game involving a time delay between the reciprocal exchange of help only when interacting with a long-term social partner [100]. Uncertainty caused by the delay of returned benefits can be reduced by a high interaction frequency, which is a common feature of close social relations and partnerships. Frequent reciprocal exchange of services such as allopreening, for instance, may in turn also stabilize social relationships, as suggested in family-living buff-breasted wrens, Cantorchilus leucotis [99].
  • (c) Short time delays. The more immediately partners can respond to each other's behaviours, the easier it is to stabilize reciprocal exchange. The condition favouring stable reciprocal cooperation most powerfully is characterized by the simultaneous exchange of goods and services among social partners (simultaneous trading and coaction; [38]). In this case, there is no temporal discounting because individuals can immediately respond to the level of cooperation exhibited by the partner. Hence, the level of uncertainty about the probability of receiving help in return to own helping effort is minimal. This is reminiscent of short-term versus long-term reciprocity in group augmentation [17], or pay-to-stay negotiations versus social queuing in cooperative breeders (cf. [6,197], where also the delayed benefits (e.g. territory inheritance) for current costs (e.g. help in brood care) bear a greater risk of unrealized returns than the concurrent interchanges.
  • (d) Pay-offs correlated by relatedness. Reciprocity can stabilize cooperation among relatives as well as unrelated individuals, since the costs to the donor of a helpful act to a relative are devalued by the degree of relatedness (r). For the same reason that cooperation can evolve among relatives by kin selection, reciprocity should be particularly common among relatives [198]: genetic relatedness between donor and receiver reduces the cost and increases the benefit of reciprocal cooperation for both. It is conceivable that reciprocal exchange among kin is even paving the way for the evolution of reciprocal cooperation among non-kin, as discussed as a possible evolutionary pathway for direct reciprocity in vampire bats [7]. There are additional reasons why reciprocity should occur particularly often between relatives (cf. [199]). Closely related individuals interact particularly frequently with each other owing to spatial viscosity [200,201], and groups of relatives are often small (e.g. in cooperative breeders; [202]). In addition, the danger of being exploited by free riders is relatively low if individuals perform reciprocal cooperation in groups of relatives, because via the sharing of genes, betrayal causes costs not only to the exploited, but also to the culprit.
    Surprisingly, there has been little theoretical or empirical research on evolutionary mechanisms underlying reciprocal cooperation in groups of relatives. As a result, the observation of cooperative behaviour occurring among relatives is usually attributed primarily or exclusively to the action of kin selection [203,204], even if this has rarely been tested [205] and other mechanisms such as reciprocity have not been considered. This is an unfortunate shortcoming, because relatives are subject to the same kinds of severe resource competition as unrelated social partners are [206–208], which can completely offset the benefits of cooperating with kin [206]. It is important to note that the concept of reciprocity refers to proximate mechanisms describing decision rules, just as the recognition of kin (or equivalent mechanisms ensuring that help is allocated preferentially to kin) is required to avoid exploitation by non-related individuals when cooperation evolves by kin selection. A recent study of the cooperative predator inspection behaviour of West African cichlid fish (Pelvicachromis taeniatus) illustrates this point. Experiments involving pairs of unfamiliar fish that were either full siblings or unrelated revealed that relatedness enhanced the probability of joining the social partner to collectively inspect a dangerous predator [198]. This indicates that correlated pay-offs based on both relatedness and reciprocity may positively interact [209], even if negative interactions have been demonstrated as well in a pay-to-stay scenario [36,76].

6. What factors constrain different types of reciprocity?

(a). Cognitive ability

Cooperative acts may be costly not just in terms of time and energy, as outlined above, but also in terms of the acquisition, maintenance, processing and retrieval of information that allows beneficial decisions regarding appropriate cooperation partners. The costs associated with information acquisition and processing vary greatly among the three major forms of reciprocity. We should expect that individuals deciding about reciprocal help should base their decision on ‘economically’ available information: if information on the individual identity of the helper is not available or too expensive to be obtained, it may be better to use unspecific information such as ‘I received help’ (irrespective of the identity of the helper) than to use no information at all. This has been referred to as the ‘hierarchical information hypothesis’ of reciprocal cooperation [39].

According to this hypothesis, if individuals can acquire information about the general cooperativeness of a social partner and are able to process this information appropriately, they can decide to help ‘cooperators' and refrain from helping ‘non-cooperators', thereby applying the decision rule of indirect reciprocity. This mechanism has been demonstrated in experiments with human subjects [35]. Indirect reciprocity requires individual recognition of social partners as well as a reputation mechanism based on the performance of social partners when interacting with others [170,171,177], and hence complex social memory [190,210,211]. Non-human animals might lack the ability to employ this mechanism owing to its high cognitive demands. However, preconditions of indirect reciprocity, such as information gathering by eavesdropping and contingent ‘image scoring’ expressed by preferential association with non-inimical social partners, have been demonstrated in interspecific cleaner fish–host interactions [212,213].

Direct reciprocity, in contrast, requires individual recognition and the ability to remember the outcomes of specific previous interactions with a social partner [190,210,214,215]. The memory capacity required for this type of reciprocity is not as demanding as in indirect reciprocity, but it may constrain the number of interacting group members for the functioning of this mechanism [216]. Careful experimentation revealed that representatives from different taxa, including songbirds, rodents and primates, are capable of applying direct reciprocity [39,93,100,115,128,217–219], and it seems to be widespread in nature in the context of allogrooming in birds and mammals in general (table 1). Still, limitations in the capacity to remember a previous interaction with a social partner may affect the functionality of this mechanism, as suggested by a study of zebra finches, Taeniopygia guttata, in which memory capacity was experimentally impaired [220]. Also in humans, memory capacity might not always suffice to fulfil the requirements for direct reciprocity [210,215], especially when groups are large [221].

Within social groups, interactions are typically structured according to different factors such as sex, age, kinship, familiarity, personality and sociability [60,222,223]. As a result, the quality and quantity of interactions among individuals differ, which may generate heterogeneous networks of social interactions within a population [224,225]. This may be conducive to reciprocal cooperation, because it facilitates memory. Humans tested for reciprocal cooperation, for instance, have been shown to remember rare partner types better in an experimental population containing cooperators and defectors [221].

In contrast to the cognitive demands involved in decision rules applied in direct and indirect reciprocity, generalized reciprocity requires only the ability to remember whether one received help or not in a previous social interaction, without having to identify or remember the partner [51,53]. The identity of the individual providing or withholding help is irrelevant, as is the identity of future interaction partners. Hitherto, this mechanism has been tested experimentally in humans [50,57], including small children [56], in Norway rats [51,111], domestic dogs [55], capuchin monkeys [56] and long-tailed macaques [128]. While humans, dogs, capuchin monkeys and Norway rats have been found to apply this mechanism when enabled to help an anonymous social partner, long-tailed macaques were not shown to act upon this decision rule. However, only five subjects were tested in this latter study of reciprocal grooming, and the monkeys could apply both direct and generalized reciprocity in a correlative, i.e. non-experimental study.

The cognitive requirements for generalized reciprocity resemble the rather undemanding information processing involved in winner and loser effects [62]. If the costs of information acquisition and management impose significant constraints on the evolution of different types of reciprocity, clearly generalized reciprocity should be applied in a broader set of circumstances than the other two reciprocity mechanisms. All biological organisms are likely to be capable of the essential mechanism required for generalized reciprocity: a behavioural response that is contingent on past social experience. Bacteria, for instance, have been shown to respond to the secretion of exoproducts such as iron-sequestering siderophores by secreting these substances themselves ([226]; reviewed in [227,228]), even though ‘cheaters' could potentially exploit their siderophore production [229–232], showing the essential contingency inherent in the decision process involved in generalized reciprocity. Therefore, if there is no mechanism selecting against this simple contingency, generalized reciprocity can be expected to be a ubiquitous phenomenon. As outlined above, several studies have in fact suggested that the conditions for generalized reciprocity to evolve are very widespread [41,42,48,49,52–54].

(b). Time lag between cooperative acts

The time structure of reciprocity may take two forms: simultaneous and successive exchange of service and commodities. This difference is significant, since reciprocal exchanges separated by a time delay are far more vulnerable to exploitation by cheaters.

• (i) Simultaneous exchange of services, trading of commodities and cooperative coaction: As outlined above, when a cooperative act is contingent on an immediate response of the receiver, exploitation is easily avoided [38]. The decision to withhold effort if help is not reciprocated can be made ‘in real time’, i.e. the trading of commodities is transparent to all partners involved. This kind of simultaneous collaboration is apparently widespread, but sometimes interpreted as (by-product) mutualism [13]. However, according to our definition, a mutualistic behaviour directly enhances the fitness of the actor, which is why it cannot be cheated; in other words, the behaviour should be shown anyway, irrespective of the behaviour of interaction partners. In contrast, if a cooperative act only pays when a social partner behaves cooperatively in return, i.e. in the case of reciprocity, the partner's response is of the essence. Many cooperative behaviours, such as predator inspection [70,72,233], cooperative hunting [234] or joint territory defence [93], typically rely on concurrent social information exchange. In these instances, individuals can survey the behaviour of their partners and respond immediately to their actions [235]. Cheating of partners can therefore be immediately penalized by withdrawing one's cooperation. Furthermore, concurrently acting partners may actively communicate with each other while establishing or maintaining cooperation [236]. This can generate coaction and advanced forms of behavioural coordination [234,237,238]. In humans, for instance, it has been shown experimentally that synchronous action can foster cooperation, which is a powerful mechanism as it may help to mitigate the free-rider problem [239].

• (ii) Successive exchange of services involving a time delay between reciprocal actions: If helping others is contingent on reciprocal exchange but there is some delay between actions of involved partners, there is uncertainty whether a helpful act may be paid back [12]. Here, a lack of information exchange between players facilitates exploitation of help by cheaters [13,240]. As a result, animals may be hesitant to pay in advance for an uncertain future benefit [102,241].

7. Trading and the reciprocal exchange of commodities

It is important to note that even if there is a time delay between reciprocal help, the basic principles of ‘trading’ may apply [160,242–244]. Often, different services are traded against each other among long-term social partners or members of stable groups [36]. For instance, wild vervet monkeys, Cercopithecus aethiops, were shown to pay back help in getting food by grooming the provider in turn [125]. In vampire bats, allogrooming received by a social partner prompts individuals to pay back with food donations [105]. In many primates, grooming is regularly exchanged against the same and other commodities, such as access to food, mating opportunities, social and agonistic support, or help in offspring care (table 1). A meta-analysis of the relationship between allogrooming and agonistic support including 36 studies of 14 primate species revealed an overall significant positive correlation, suggesting that allogrooming and social support might reflect a prototype of reciprocal cooperation in primates [153]. Of course, other commodities may also be traded against each other, such as mating opportunities for agonistic support (chimpanzees, Pan troglodytes: [245]).

It should be noted that in contrast to the exchange of commodities of one's own accord, negotiations between social partners may involve a component of force (cf. [36]), irrespective of whether the exchange involves a time delay or not. For instance, allogrooming may be demanded as a service by aggressively challenging another group member, as suggested by positive correlations between aggression given and grooming received for example in meerkats, Suricata suricatta [246] and Barbary macaques, Macaca sylvanus [129]. Aggression and threats of eviction from the group may help dominants to obtain a large variety of cooperative behaviours from subordinate group members (‘pay-to-stay’; [77–79,194,247]). But also in such asymmetric interactions, the subordinates must gain a benefit, e.g. by obtaining resource access or by being protected, as an incentive to stay and pay the demanded rent [36]. This shows that power asymmetries can be an important component of reciprocal exchanges, which, at first glance, may look like mere enforcement of cooperation.

In conclusion, the types of reciprocity that we should expect to find most often in nature may (i) either involve an exchange of commodities that allows for immediate mutual adjustment of benevolence; this adjustment is facilitated if the exchange of services is either concurrent (e.g. coaction; [38]), or if it occurs among long-term social partnerships (e.g. attitudinal reciprocity, [165]; altruism as a signal, [178,248]). Alternatively, (ii) reciprocal cooperation may be based on simple decision rules such as ‘help anyone if helped by someone’ (generalized reciprocity), which are not limited by constraints and costs of information acquisition and processing. There is ample evidence for the prevalence of the former, including also interspecific, apparently mutualistic interactions (e.g. coral–zooxanthellae endosymbiosis). However, it has not yet been scrutinized how widespread the latter mechanism is in biological systems, despite experimental demonstration of its functionality.

8. The importance of context for the evolution of cooperation

Cooperation typically generates non-additive positive fitness effects, i.e. synergism; the total effect of individual actions is greater than the sum of the individual effects. It has been suggested that ‘functional synergy’ can be the underlying cause of cooperation [14,249,250]. Synergies may be important in any type of cooperation, either involving mutualism or correlated pay-offs; they will always enhance the probability that cooperative behaviour is positively selected. This is one reason why the prisoner's dilemma paradigm is such an important test case for evolutionary mechanisms of cooperation: by definition, to obtain synergistic benefits of cooperation is intricate in this situation. The pay-off matrix specifies that abstaining from cooperation (i.e. defecting) yields on average higher benefits than helping the partner. Only if both players somehow manage to coordinate their behaviour towards cooperation will the pay-off be greater than if both partners act uncooperatively. However, coordination is virtually prevented by the decision process in this game, which excludes concurrent information exchange between players. Despite this difficulty, the reciprocity mechanisms discussed in this article can generate stable levels of cooperation, as demonstrated by both theoretical and empirical research (see above).

Social structure is another parameter crucial for the evolution of cooperation. The relationships of individuals in stable groups are inherently different from those characteristic for transient open groups. Groups of individuals jointly defending a territory, raising offspring and sharing resources, such as those typical for cooperative breeders, differ in many ways from aggregations that are more or less reflecting the distribution of resources and threats, but which are not based on individual relationships; the latter pattern characterizes shoals, schools, flocks and herds. The former type of groups typically coincides with viscous gene flow and assortment of individuals by genetic similarity, which enables the emergence and stabilization of cooperation by kin selection. However, also in these groups, reciprocal cooperation may be important, as we have outlined above. In addition, such groups often contain unrelated individuals [2,4,6,197,251,252], meaning that helping each other crucially depends on the give-and-take characterizing reciprocity. Close personal relationships between individuals in such groups will promote cooperation because the involved social partners will interact with each other more likely and more often. Especially in large groups, this may be important because interaction networks reduce functional group size, which is conducive to the evolution of cooperation by very simple, cognitively undemanding reciprocity mechanisms (generalized reciprocity; [49,53,54]). Such partnerships may be characterized by mutual dependence (‘interdependence’; [163]), ‘since the loss of the other means the loss of a guaranteed helper’ [162, p. 457]. In other words, such long-term partners may cooperate with each other because each has an interest or ‘stake’ in the partner. Cooperation between them does not follow the simplistic ‘tit-for-tat’ exchange predicted by direct reciprocity models, but may integrate actions and responses over longer time scales (‘attitudinal reciprocity’; [165]). Such relationships may be widespread for instance in pairs of breeders raising joint offspring. Reciprocal cooperation and involved negotiations among mates sharing parental investment is a promising area for future research (cf. [235,253]; see also [254]).

9. Conclusion

We have discussed here how (i) correlated pay-offs may come about and how (ii) the different causes of pay-off correlations may influence the evolution of cooperative behaviour. We have focused on the mechanism generating stable levels of cooperation that arguably is most difficult to understand, i.e. reciprocity, because it involves investment of one partner in favour of another, which may be vulnerable to cheating. Nevertheless, this focus should not give the wrong impression that this mechanism is more important than the others, i.e. kin selection and green-beard effects. Also, cooperation that is not necessarily based on correlated pay-offs, i.e. mutualism of any sort (intraspecific or interspecific), in our view is an important cause of cooperative behaviour, as has been discussed in recent reviews [13,255]), even though some definitions would not include such behaviour within the concept of cooperation [256].¹ One motivation for our focus on the evolutionary mechanisms underlying reciprocal cooperation has been that in the past, the existence and importance of reciprocity have met with scepticism, mainly because the evidence for reciprocal cooperation in nature seems to be scant. However, we argue that this is more apparent than real. Not only have a wide range of animals have been shown to apply decision rules characterizing direct and generalized reciprocity under stringent experimental conditions excluding alternative explanations, which likely indicates evolved responses, but reciprocal help among social partners that may reflect the application of such rules is also prevalent in nature (table 1). In addition, many observations of cooperative behaviour among kin are uncritically explained by the action of kin selection, but often this may not be justified or only partly explain the observed behaviours. Reciprocity among non-kin is more readily accepted as an explanation for cooperative behaviour, but there is no reason to believe that the application of such rules is not selected for also among relatives. We propose that this should get more attention in both theoretical and empirical research in the future.

Competing interests

We declare we have no competing interests.

Funding

M.T. was financed by the Swiss National Science Foundation (Project 31003A_156152).