Hormonal mechanisms of cooperative behaviour

Abstract

Research on the diversity, evolution and stability of cooperative behaviour has generated a considerable body of work. As concepts simplify the real world, theoretical solutions are typically also simple. Real behaviour, in contrast, is often much more diverse. Such diversity, which is increasingly acknowledged to help in stabilizing cooperative outcomes, warrants detailed research about the proximate mechanisms underlying decision-making. Our aim here is to focus on the potential role of neuroendocrine mechanisms on the regulation of the expression of cooperative behaviour in vertebrates. We first provide a brief introduction into the neuroendocrine basis of social behaviour. We then evaluate how hormones may influence known cognitive modules that are involved in decision-making processes that may lead to cooperative behaviour. Based on this evaluation, we will discuss specific examples of how hormones may contribute to the variability of cooperative behaviour at three different levels: (i) within an individual; (ii) between individuals and (iii) between species. We hope that these ideas spur increased research on the behavioural endocrinology of cooperation.

1. The challenge of cooperation: a brief introduction to its main ideas

Nature is full of examples of individuals helping others, or increasing the direct fitness of other individuals. The concept of an ‘evolutionarily stable strategy’ (Maynard Smith 1982) has been instrumental in evolutionary game theory to identify conditions that allow stable cooperative solutions to the problem of investments (Axelrod & Hamilton 1981; Nowak & Sigmund 1992; Clutton-Brock & Parker 1995). Game-theoretic approaches typically yield a single strategy as solution, while a mixture of strategies owing to negative frequency selection is rare (McNamara et al. 2004).

The simplicity and precision of theoretical solutions often contrast with reality in two ways. First, many individuals establish privileged relationships with specific partners, which are treated differently from other conspecifics (e.g. pair bonding, ‘friendships’; Silk 2003). These privileged relationships contribute to the individual variance in behaviour and are then difficult to properly describe within the counting strategies prominent in evolutionary game theory scenarios (Silk 2003; de Waal & Suchak 2010). Second, even if one excludes these privileged relationships from the general picture, empiricists still often observe strong variation within the behaviour of individuals. Theoreticians have started to incorporate this into their models as they have realized that variation between individuals does help to explain the persistence of conditional cooperative strategies, and hence the persistence of cooperation (Sherratt & Roberts 2001; McNamara & Leimar 2010).

Thus, behavioural variation is indeed an important variable to take into consideration; however, its underlying source still remains largely obscure. This statement is as similarly true for the ultimate level as for the proximate level. Variation might simply be due to noise around an optimum, or it could be owing to frequency-dependent selection on a distribution of strategies. To make things more complex, seemingly non-adaptive behaviour might result from correlations across contexts that may lead to behavioural syndromes (for example ‘be aggressive and don't help’ or ‘be non-aggressive and help’, Bergmüller et al. 2010). Variation could result from genetic differences, but ontogenetic effects could also be important (‘phenotypic defectors’ in Sherratt & Roberts 2001) because they can affect behaviour via body condition or via individual and/or social learning. Moreover, as mentioned above, bonding mechanisms allow for the possibility to show certain behaviours exclusively towards individualized partners. The uncertainty about the causes of variation makes it imperative to gain a better understanding of the proximate causes of decision-making. Proximate factors may be responsible for the variation or just modulate the variation. Only if we know how individuals decide will we be able to determine trade-offs or possible constraints that can be incorporated in generalized models exploring the evolution and stability of cooperation.

Relatively little research has been done with respect to the physiological mechanisms that underlie individual tendencies to help others. We predict that getting a better understanding of the matter will be a major task because of the diversity of behaviours that have to be studied. In a nutshell, helping behaviours can be aggressive (joint territorial defence), related to sexual behaviour (egg trading), related to parental behaviour (food provisioning), or related to foraging (cooperative hunting). Moreover, these behaviours can be directed towards a partner (grooming, food provisioning) or third parties (territorial defence, alliances). Finally, individuals may act independently of partners (vigilance, alarm calling) or in a coordinated manner (cooperative hunting, predator inspection). Thus, we predict that what appears to be a uniform phenomenon at the ultimate level—the increase of the direct fitness of a recipient—will be based on a great diversity of physiological and neural processes.

In this review, we discuss the possible influence of hormones and neurohormones in regulating the expression of helping behaviour. We will not specifically distinguish between cooperative behaviour and altruistic behaviour, though we will focus on examples of cooperation. Hormones are likely to affect levels of helping in various ways as they have both activational and organizational influence on general social behaviour. However, only a few studies have tried to specifically establish a link between hormones and cooperative behaviour. Therefore, this review will remain somewhat speculative when it comes to factual data while conveying the need for future studies. We will also focus on vertebrates, a restriction that largely reflects the limits of our own expertise. In §2, we provide an introduction to the endocrine basis of social behaviour, aimed at non-specialists. In §3, we identify the cognitive building blocks of cooperative behaviour and suggest how hormones may modulate these parameters. Finally, in the last two sections (§§4 and 5), we discuss in which ways hormones may explain variation in cooperative behaviour within individuals, between individuals and between-species and then briefly round-up (in §5) how general endocrine mechanisms may or may not be implicated in the modulation of cooperative behaviour.

2. A primer to the neuroendocrine regulation of social behaviour

(a). Hormonal modulation of social behaviour

Aeons before hormones themselves were discovered, people were already aware that some physiological characteristics (hormones) could directly influence behaviour. Consider farmers, for instance, who for thousands of years removed testicles to turn uncooperative and aggressive bulls into docile oxen; or emperors who kept eunuch servants. The main hormone responsible for those changes in behaviour is now known as testosterone, the principal steroid secreted by most vertebrate testes, which plays a pivotal role in the regulation of morphology, physiology and behaviour of male vertebrates (for review, e.g. Wingfield et al. 2006).

Countless studies have convincingly demonstrated that behaviour is influenced by hormones. The reverse idea, that behaviour also influences hormone levels, is more recent. The relationship between androgens and aggressive and sexual behaviour in male vertebrates might also be one of the best examples for this: androgens are behavioural facilitators by acting as modulators of neural pathways of social behaviour, while in turn their concentrations may respond to social circumstances (see Wingfield et al. 1990; Oliveira 2004, 2009; Goymann et al. 2007; Goymann 2009). This reciprocal relationship between hormones and behaviour is important: the influence of hormones is not simply unidirectional, but includes intrinsic feedbacks of social context and social behaviour on hormones. Moreover, within a social network, an individual's androgen level will modulate perceptive, motivational and cognitive mechanisms, that will in turn influence future social behavioural efficiency (Oliveira 2009). In short, hormones help animals to deal with complex real-life problems by acting as coordinators of behavioural, morphological and physiological outputs at both short- and long-term life-history scales (Adkins-Regan 2005). A single hormone may have effects on many different aspects of behaviour, which may depend on social context and the life-history stage (e.g. Wingfield 2008); a hormone may also have pleiotropic effects by affecting many different traits at the same time, thereby acting as the proximate mediator of functional trade-offs (for review, e.g. Lessells 2008).

(b). How do hormones affect the expression of behaviour?

Hormones may modulate the expression of behaviour, but are not causes of behaviour. Behaviour is mainly driven by internal and environmental stimuli, with different stimuli eliciting different behaviours. For example, courtship behaviour occurs when a male and a female in reproductive condition meet each other. Three functional components are involved: (i) sensory systems that translate environmental cues into neural signals; (ii) the central nervous system that integrates sensory input with endogenous activity and (iii) effector systems (e.g. neuromuscular system) that perform the response. Thus, in order to modulate the expression of behaviour, hormones have to modulate one or more of these components (Nelson 2005; Oliveira 2005). Therefore, hormones should not be seen as deterministic factors but instead as modulators of behaviour that may increase or decrease the probability of the expression of a given behaviour by acting on the neural mechanisms underlying behaviour (Oliveira 2005).

The modulatory action of hormones on the nervous system can occur at a functional level, by changing the activity of a given neural circuit, or at a structural level, by changing the architecture and/or connectivity of different nodes of the neural circuit. Functional effects are rapid and short-lived and can either result from a direct effect of hormones on neural excitability and neurotransmission (Remage-Healey & Bass 2006a) or an indirect effect via neuromodulators (e.g. serotonin and dopamine; Di Paolo 1994; Bethea et al. 2002). Structural effects are slow and long-lasting and can involve the recruitment of new cells to the circuit (neurogenesis; e.g. Galea 2008), the removal of pre-existing cells from the circuit (apoptosis; e.g. Maclusky et al. 2003) or changes in the connectivity of the circuit (synaptic plasticity; e.g. Parducz et al. 2006). The rapid and transient effects of hormones on behaviour are called activational effects, in contrast to long-term and usually not reversible effects referred to as organizational effects. These latter effects are usually only effective at an early stage of life within strict time windows termed sensitive or critical periods, and are expected to last for the entire lifespan of the individual. Hormones can permanently affect (differentiate or ‘organize’) an individual's phenotype during development either directly, via hormones transmitted from the mother or from litter mates to the offspring during prenatal development, or indirectly, by effects on the offspring's hormonal profile either pre- or post-natally via maternal behaviour, such as grooming (for instance Liu et al. 1997; Meaney 2001), physical activity (Bick-Sander et al. 2006) or nutritional provisioning. These behavioural traits are also likely to determine the tendency of an individual to show cooperative behaviour.

(c). Chemical neuromodulation: neurotransmitters, neuromodulators, neurohormones and hormones

Of the many different hormones present in vertebrates, there are three groups that have received most attention in terms of their role in the expression of social behaviour: sex hormones, stress hormones and neuropeptides from the vasopressin (AVP)/oxytocin (OT) family (table 1). The reason for this has to do with the fact that social behaviour is naturally related with reproduction and to responses to emergencies or challenges (e.g. social stressors).

Table 1.

Major hormones acting on social behaviour in vertebrates (adapted from Adkins-Regan 2005). AR, androgen receptor; ER, oestrogen receptor; PR, progesterone receptor; GR, glucocorticoid receptor; MR, mineralocortcoid receptor; OTR, oxytocin receptor; BNST, bed nucleus of the stria terminalis.

hormone family	hormone	receptor	major source
sex steroids	testosterone	AR1, AR2	testis
	oestradiol	ERα, ERβ1, ERβ2	ovaries
	progesterone	PR-A, PR-B	ovaries
stress steroids	cortisol (humans, fish)	GR1,GR2, MR	adrenal glands (tetrapods)
	corticosterone (rodents)		inter-renal glands (fish)
neurohormones	arginine–vasopressin (AVP; mammals)	V1a, V1b, V2	hypothalamus
	arginine vasotocin (AVT; non-mammals)		BNST
	oxytocin (OT; mammals), mesotocin (birds, reptiles, amphibians), isotocin (IT; fish)	OTR	hypothalamus

Both sex and stress hormones are organized in specific neuroendocrine axes, the hypothalamic–pituitary–gonadal (HPG) and the hypothalamic–pituitary–adrenal (HPA) axes, both of which are organized in a hierarchical fashion. At the top of the hierarchy are the hypothalamic peptides, i.e. gonadotropin-releasing hormone in the HPG and corticotropin-releasing hormone in the HPA, which control the synthesis and release of specific tropic hormones in the anterior pituitary, gonadotropins (LH and FSH, HPG axis) and adrenocorticotrophic hormones (ACTH, HPA axis), which in turn regulate the production of specific steroids in the gonads (i.e. testosterone in males; oestradiol and progesterone in females) and in the adrenals (i.e. cortisol or corticosterone depending on the species). In contrast to the hierarchical organization of the HPG and the HPA axes, the posterior pituitary receives neural projections from the hypothalamic neurons, which end in a capillary network, where the neurohormones produced by these neurons are released into the bloodstream. These neurohormones are nonapeptides that belong to the AVP/OT family: AVP and OT in mammals and their non-mammalian homologues, arginine–vasotocin (AVT), mesotocin (birds, reptiles, amphibians) and isotocin (fish). Apart from their peripheral (hormonal) actions, these neuropeptides also have central (neuromodulator) actions in the brain that regulate social behaviour (Caldwell et al. 2008a).

Hormones are not the only modulators of neural circuits that underlie behaviour (table 2). Neurotransmitters and neuromodulators are also known to modulate behavioural expression. In contrast to the fixed behavioural responses to environmental stimuli (i.e. reflexes), flexible behavioural responses require neural plasticity, which can be achieved by the chemical modulation of neural circuits at various levels. Chemical synapses, with neurotransmitters that allow focal modulation of signal transmission (in contrast to cellular coupling in electrical synapses) represent a first step of modulation at the level of cell–cell signalling. Neuromodulators are released from neurons in a non-synaptic fashion and may then interact with receptors at multiple sites within significant distances from their site of release. Therefore, neuromodulators have a peculiarly diffuse modulator action in the brain. In short, there is a continuum from neurocrine to endocrine communication, and hormonal and neural mechanisms are interconnected with multiple reciprocal effects. However, in this review, we shall only focus on endocrine and neuroendocrine factors affecting cooperative behaviour; the effects of neurotransmitter systems and central neuromodulators fall outside the scope of this manuscript.

Table 2.

Chemical/hormonal terminology (adapted from Nelson (2005) and Norris (2007)).

agents	description
hormone	an organic chemical messenger released from endocrine cells that travels through the blood system to interact with cells some distance away and cause a bio-response. Secreted by specialized non-neural cells into the blood
neurohormone	substance secreted by neurons into the blood that may be stored in neurohemal organ prior to release
neuropeptide	a peptide hormone produced by a neuron
neurosteroid	a steroid hormone produced by a neuron
neuromodulator	substances that do not directly activate post-synaptic receptors but that enhance the excitatory or inhibitory responses of these receptors
neurotransmitter	chemical messenger that acts across the synaptic space
chemical messenger	substance produced by a cell that will in turn affect the function of another cell

(d). The social brain

A set of brain areas in the basal forebrain and midbrain have been identified as being involved in the regulation of multiple forms of social behaviour (aggression, affiliation, bonding, parental behaviour, social stress) and to have bidirectional connections between each pair (Newman 1999; Goodson 2005). These areas include the extended medial amygdala, namely the medial amygdala and the bed nucleus of the stria terminalis, the lateral septum, the preoptic area, the anterior hypothalamus, the ventromedial hypothalamus and the periaqueductal grey in mammals and homologous structures in other vertebrates (Goodson 2005). This network is seen as the core of the social brain but it is certainly not its whole, since there are other brain areas that are quite relevant for social behaviour, such as the basal forebrain rewarding system (see §3b(iv) below) and the cortical areas for executive functions in mammals. Newman (1999) originally proposed the existence of this social behaviour neural network in mammals, and Goodson (2005) confirmed its presence across different vertebrate classes and identified the putative homologous areas for each node in the different classes and/or taxa.

It is important to note that the social behaviour neural network encodes information in a distributed and dynamic fashion, such that the expression of a given behaviour is better reflected by the overall profile of activation across the different loci in the network rather than by the activity of a single node (Goodson et al. 2009). Therefore, it is conceivable that different combinations of activations across nodes will be able to produce a wide variety of social behaviour as the weights of each node in the network may change at different levels: at the individual level, if node weights change temporally; at the intraspecific level, if weights have a genetic and epigenetic component giving rise to different social phenotypes; and at the interspecific level, if weighting is changing with evolution (figure 1).

Figure 1.

Hypothetical representation of transitions in the functional state (i.e. relative activity at each node and strength of connectivity) of the brain social behaviour network (BSBN) corresponding to transitions on behavioural phenotype at different levels (POA, preoptic area; Mid, midbrain; VMH, ventromedial hypothalamus; AH, anterior hypothalamus; LS, lateral septum; eMA, extended medial amygdala). Changes in the weight of each node and in the strength of the connection between each pair of them correlate with behavioural changes within and between species.

Steroid and neuropeptide (AVP, OT and their non-mammalian homologues) receptors are present in different nodes of this network, suggesting that plasticity in social behaviour is likely to be modulated by these hormones (Goodson 2005). In fact, steroid hormones are also known to modulate the synthesis of neuropeptides and their receptors and therefore may also exert their effects indirectly through the AVP/OT neuromodulatory system. AVP, OT and their non-mammalian homologues (table 1) are synthesized in two different cells groups: (i) magnocellular cells of the supraoptic and paraventricular hypothalamus; (ii) parvocellular neurons within the paraventricular nuclei of the hypothalamus (for both AVP and OT) and in the bed nucleus of the stria terminalis, the medial amygdala and the suprachiasmatic nucleus, which project to the limbic system (specifically for AVP). The two cell types have different projections associated with differential function: the OT and AVP hypothalamic magnocellular cells project to the posterior pituitary and these pathways are responsible for peripheral systemic effects of these neuropeptides (e.g. AVP: anti-diuretic; OT: parturition and lactation); whereas the AVP and OT parvocellular cells project to limbic areas, including several of the nodes of the social behaviour neural network described above, and this system is responsible for central effects on the brain (De Vries & Panzica 2006; Caldwell 2008b).

Finally, AVP and OT may also affect the expression of behaviour through reciprocal interactions with the dopaminergic rewarding system and with the serotonergic system (Skuse & Gallagher 2009). Therefore, the hormonal modulation of social behaviour may occur at multiple points and at several levels of integration: direct effects of peripheral hormones (i.e. sex and stress hormones), effects of peripheral hormones on neuropeptides (AVP/OT), direct effects of neuropeptides, effects of neuropeptides on neuromodulators (dopamine/serotonin) and direct effects of peripheral hormones on central neuromodulators (figure 2). All of these may occur in different compartments of the neural circuits underlying behaviour (input, central processing, output), therefore affecting the perception, the valence and the salience of social interactions.

Figure 2.

Hormones modulate the BSBN via central effects of neuropeptides (AVT, OT) and via peripheral effects of sex and stress steroids that may act either directly at the different nodes of the network or on the inputs or outputs of the network.

3. Hormones and the cognitive modules underlying cooperative behaviour

In this section we will try to identify the major cognitive parameters or building blocks of cooperative behaviour and explore how hormones might influence these. Once we identify these building blocks of cooperative behaviour, we can discuss how hormones may affect them and subsequently the expression of cooperative behaviour itself.

(a). The building blocks of cooperative behaviour

Considering the vast diversity of examples for cooperation, there likely exists an equal diversity of mechanisms that regulate cooperation. For instance, proximate mechanisms are required that make an individual tolerate the presence of conspecifics, coordinate its actions to produce behaviour that reduces immediate pay-offs, recognize partners and assess their behaviour, and choose the appropriate responses from the available behavioural repertoire. Many of these problems may be solved with genetically determined rules (see Brosnan et al. 2010). However, in many vertebrate animals, the brain mechanisms described above allow individuals to more flexibly respond to environmental stimuli. The perception, evaluation and behavioural output can be adapted to specific situations in a complex and variable world. Hormones are known to affect and be affected by the following components of cooperative behaviour.

(i). Prosocial behaviour

A first condition for the occurrence of cooperative behaviour is that individuals show a predisposition to approach conspecifics or other potential partners, and tolerate their presence. Therefore, the tendency to approach a partner in a cooperative context overcome the tendency for social withdrawal.

(ii). Social recognition

Social recognition is necessary to distinguish between cooperators and cheaters or between partners and opponents. Individual recognition may even be necessary within any group of possible partners. This becomes important when the behavioural response depends on the specific value of a partner. For example, cleaner fish (Labroides dimidiatus) are able to distinguish between familiar and unfamiliar clients from the same client species, and prefer to interact with familiar ones (Tebbich et al. 2002).

(iii). Social bonding

Bonding mechanisms may be necessary to avoid aggression against partners. Bonding may also be crucial for any individual's willingness to invest: by creating affection towards another individual, bonding may help to overcome the problem of objective reduction in immediate gains through cooperative behaviour. Bonding may modulate the perception of pay-offs and thus facilitate cooperative behaviour.

(iv). Assessment of the social environment

To be able to respond in an appropriate manner, individuals first need to evaluate their partner's behaviour. Is this partner cooperating or rather holding back its efforts? Are its efforts good enough to elicit cooperation? Both the perception of the partner's effort and the critical threshold that may cause a switch in behavioural responses can be affected by hormones. In some cases, individuals respond directly to the behaviour of the partner. For instance, clients of the cleaner wrasse L. dimidiatus flee or chase the cleaner if it takes a bite of mucus instead of eating parasites (Bshary & Grutter 2002). Another kind of assessment occurs when cooperative benefits arise owing to the behaviour of a third party. In these cases, interactions take place in a communication network. Bystanders may gain information about interacting partners and may adjust their future behaviour with these partners depending on their observations. Bystanders must choose appropriate behaviours without having had personal experiences with potential interaction partners. Similarly, the interaction partners may adjust their current behaviour depending on the presence or absence of bystanders (‘audience effects’; see also Earley 2010). For example, humans are more cooperative if they are observed by third parties than if they remain anonymous (Milinski et al. 2002).

(v). Social memory and learning

Individuals may adjust their behaviour depending on how a partner has behaved in the past, especially if these previous interactions were risky or involved cheating. Under such circumstances, memory becomes important.

(vi). Temporal discounting

In a cooperative context, individuals sometimes choose a lower immediate reward to maintain future benefits. For example, the cleaner wrasse L. dimidiatus feeds against its preference when interacting with a client reef fish (Grutter & Bshary 2003). If instead, the cleaner chooses the larger immediate benefit (a bite of mucus), it may risk losing the chance for numerous future interactions. The cleaner thus must forsake the larger immediate benefit in order to gain future rewards.

(vii). Partner choice

Individuals may often choose among potential partners for an exchange of goods or services. This biological market paradigm (Noë 2001) is linked to communication and negotiation. Simple (i.e. genetically coded) solutions may exist, for example if partners are sessile (Brosnan et al. 2010). To choose an appropriate partner, most animals need to develop what we refer to as ‘cooperative behavioural competence’. This means they have to: (i) analyse partner quality and judge honesty/deception signals; (ii) recognize, memorize and categorize features of former interaction partners to build long-lasting relationships; (iii) apply different strategies depending on the context (levels of investment).

(b). How hormones may affect cooperative behaviour by acting on its building blocks

(i). Hormones and prosocial behaviour

It has been hypothesized that AVP and OT may influence basic emotional mechanisms that regulate social approach and social aversion (Porges 2001). According to this hypothesis, OT acting on hindbrain parasympathetic systems should stimulate prosocial behaviour, whereas AVP acting on sympathetic pathways should be associated with social withdrawal and/or aggression. Some empirical results give support to this theory. For example, in the goldfish (Carassius auratus), central infusions of AVT inhibit social approach, whereas the administration of an AVT antagonist or of IT stimulated social approach (Thompson & Walton 2004).

(ii). Hormones and social recognition

Social animals may take advantage of living in groups which are organized in complex social networks of related and unrelated individuals. To ensure successful operation within the group, individual recognition and cooperation between individuals (e.g. pair partners or similar social alliances) may be essential. Individual recognition or recognition of different classes of social partners (e.g. cooperators versus non-cooperators) is both the basis and the consequence of interactions with others, and requires consolidation of memory of past interactions and outcomes.

The role played by AVP and OT in the formation of social memories, that is, the process of learning to distinguish familiar from non-familiar individuals, has been investigated in detail in laboratory rodents (reviewed by Lim & Young 2006). In rats and mice, social recognition can be evaluated by measuring the duration of olfactory investigation of other individuals. Typically, olfactory investigation is longer for unfamiliar individuals than for familiar ones. An involvement of OT in the formation of social memories was first shown by Dantzer et al. (1987) and was recently confirmed by studies with transgenic mice lacking the OT genes. OT-knockout mice are not olfactory impaired and show no generalized deficits in learning and memory, and yet they fail to show behaviours which would indicate that they recognize familiar individuals, even after repeated encounters (Ferguson et al. 2000). The demonstration that this type of social learning is in fact depending on OT is quite straightforward: a single OT treatment of OT-deficient mice before a social interaction is sufficient to restore the later recognition of that interaction partner (Ferguson et al. 2001, 2002). Besides laboratory rodents, the involvement of OT in social recognition has been shown in ewes and in monogamous prairie voles. In monogamous voles, OT is important for mate recognition and pair bond formation (Young & Wang 2004). More recently, it has been shown that OT in particular can have important effects on social recognition in humans. Intranasal administration of OT improves the capacity to recognize faces, but has no effect on the memorization of non-social stimuli (Rimmele et al. 2009).

The involvement of AVP in the formation of social memories was originally illustrated by the finding of the AVP-deficient Brattelboro rat, which has a totally disrupted social recognition (Feifel et al. 2009). Thus, AVP seems also to be involved in olfactory social recognition in mice. Moreover, the AVP V1a receptor (V1aR) has also been implicated in individual recognition. Males with a null mutation for the vasopressin V1aR exhibit a profound impairment of social recognition (Bielsky et al. 2004), while injections of a V1aR-specific antagonist into the lateral septum (but not the medial amygdala) impair social recognition (Bielsky et al. 2005). In fact, the over-expression of this receptor in the lateral septum of wild-type mice potentiates social recognition, while in V1aR knockout mice (V1aRKO), the re-expression of V1aR (using a viral vector) in the lateral septum rescues the deficit in social recognition typical of V1aRKO mice (Bielsky et al. 2005). Together, these results suggest that AVP acting on the lateral septum plays a critical role in olfactory social recognition in mice.

(iii). Hormones and social bonding

Social and/or individual recognition allows animals to establish preferential relationships with chosen individuals within their social groups. The emergence of social bonds depends on the expression of preferences associated with specific individuals (and not others), which underlies the emergence of different types of social behaviour, such as parental care and pair-bonding (i.e. strong bonds between mating pairs). The formation and maintenance of pairs is a good example of cooperation between two unrelated individuals, as both may benefit from tolerating and supporting each other. Pair bonding is also a behavioural trait that may easily be assessed in laboratory conditions by, for instance, standardized partner preference tests (e.g. Williams et al. 1992). Different vole species have been used in comparative studies of pair-bonding mechanisms which show marked interspecific differences in terms of social attachment (which is reflected in their mating system) despite their close phylogenetic relationship.

Cooperation between pair partners is particularly relevant for maintaining the pairbond, at least in long-term monogamous species. The maintenance of a pairbond is the result of ‘bilateral’ interactions between pair partners and of ‘multilateral’ interactions between the pair and its social environment. Therefore, when searching for behavioural and physiological correlates of successful reproduction, it may be revealing to shift the unit of analysis from the individual to the pair. For example, in greylag goose (Anser anser) pairs, a compatible timing of peaks and troughs in the seasonal androgen patterns of the male and the female pair partner predicts individual fitness of both pair partners, as well as the pair's breeding success (Hirschenhauser et al. 1999; Weiß et al. 2010). Yet, testosterone co-variation may equally be the cause or consequence of pair synchronization. In birds and mammals, female testosterone is related to sexual motivation and fecundity. In males of all vertebrate taxa, testosterone regulates and responds to sexual and/or agonistic interactions. It may be viewed as a physiological mediator of the trade-off between investing into male–male aggressiveness or paternal care, although this is not a ubiquitous phenomenon. Thus, a coordinated pair may be more successful even beyond the immediate benefit of reproduction, e.g. during conflicts with its social environment for access to food or in coping with unexpected disturbances.

(iv). Hormones and the assessment of valence and salience of social information

Animals must assess the valence (positive versus negative) and salience (high versus low) of social stimuli in their environment. Two brain systems are critical for the attribution of valence to social stimuli: the amygdaloid complex and the basal forebrain rewarding system. The amygdala has been viewed as a danger detection centre in the brain, which is activated when a potential threat is detected in the environment (LeDoux 2007). The activation of the amygdala in a social context leads to social anxiety and withdrawal, which happens for example in response to unfamiliar conspecifics or threatening adversaries (Stein et al. 2002). For example, in humans fearful faces elicit the activation of the amygdala (Whalen et al. 1998) and patients with lesions of the amygdala fail to recognize fearful faces and display behavioural disinhibition (Adolphs et al. 2005). On the other hand, decreased activation of the amygdala has been associated with hypersociability (Meyer-Lindenberg et al. 2005).

OT and AVP seem to have opposite effects on the modulation of the amygdala and the concomitant assessment of the valence of social stimuli. In humans, the intranasal infusion of OT reduces the activation of the amygdala in response to threatening faces (Kirsch et al. 2005), whereas AVP increases the subjective perception of threat to emotionally neutral faces, evoking responses similar to those elicited by angry facial expressions (Thompson et al. 2004, 2006). These opposite effects have a parallel at the cellular level, since these two neuropeptides excite different neuronal populations in the central amygdala: OT has excitatory effects on GABAergic neurons that inhibit neurons which can be excited by AVP (Huber et al. 2005; Viviani & Stoop 2008). Since the central amygdala is a major source of projections from the amygdaloid complex to the autonomic nervous system, the opposite effects of OT and AVP on its activity may be a way of regulating the expression of autonomic signals of fear, which may affect the motivation to cooperate, in other words, the trust in a social partner. In line with this hypothesis two independent studies have suggested a role for OT in promoting trust in a partner in a social context. In one study, intranasal OT administration increased money transfer by investors in a trust game, but had no effect in risk-taking in a non-social context (Kosfeld et al. 2005). In another study, OT increased trusting behaviour and decreased amygdala activation during a trust game (Baumgartner et al. 2008). Thus, OT seems to increase trust by reducing amygdala activation and concomitantly anxiety states associated with the possibility of non-reciprocation.

Social interactions can be rewarding and lead to further interactions with the same partner. The rewarding value of social interactions suggests that their valence and salience might be coded by the circuitry involved in reward-learning, namely the mesolimbic dopaminergic pathway. Brain imaging studies of humans showed that this may be a more general mammalian or vertebrate phenomenon (Bartels & Zeki 2004; Zeki 2007). The mesolimbic dopaminergic pathway also seems to be involved in decision-making in the context of reciprocal exchange (Rilling et al. 2008) and thus may be of particular interest for the study of proximate mechanisms of cooperation.

Most importantly, these dopaminergic reward pathways in the brain are also under the influence of AVP and OT. Neuropeptide receptors for OT and AVP interact with dopamine receptors in the reward centres of prairie voles (Young & Wang 2004) and potentially other mammals, including humans and birds (reviewed in Fisher et al. 2006). OT- and AVP-releasing neurons project into the midbrain and exert an influence on the mesolimbic dopaminergic reward pathway (reviewed in Skuse & Gallagher 2009). Thus, if central OT and AVP influence midbrain dopaminergic reward centres, these neuropeptides could potentially modulate cooperative behaviour in vertebrates.

(v). Hormones, memory and social learning

Hormones can influence learning processes. Because cooperation and conflict are often based on individual recognition and memory of the partner or opponent, hormone-modulated learning may play an important role in establishing social relationships. Hormones may also be involved in other learning and memory tasks relevant for the expression of cooperative behaviour. For example, the ability to remember specific aspects of past social encounters, such as a relative place in time and space, associated emotions and other contextual knowledge (i.e. episodic memory) would allow individuals to reciprocate cooperative behaviour and to punish non-cooperators. The existence of episodic memory in animals is still a matter of debate (Clayton et al. 2003; Suddendorf & Busby 2003; Roberts et al. 2008). Nevertheless, some criteria related to episodic memory have been investigated and confirmed for mammals and birds, including ‘what–where–when memory’: whether an animal remembers information about a specific event experienced in a given spatial location and at a particular time in the past (e.g. Babb & Crystal 2005).

The presence of steroid hormone receptors in the hippocampus (McEwen 2001; Hajszan et al. 2007), a brain area involved in relational memory processing and episodic memory (Squire 1992; Eichenbaum et al. 1999), indicates a potential role of steroids as modulators of higher brain functions. Moreover, both sex steroids (in particular oestrogens) and glucocorticoids modulate learning and memory processes. For example, the decrease in circulating oestrogen in the menopause brings about cognitive deficits, which include reduced capacity for spatial learning. There is also evidence that oestrogens are involved in social learning (Markham & Juraska 2007).

Some recent evidence suggests a possible involvement of AVP (through its V1b receptor) in episodic-like memory in terms of social interactions. Apart from deficits in aggressive behaviour and social motivation, AVP V1b receptor knockout mice also display impaired social memory, despite having normal olfactory ability and other memory functions (e.g. spatial memory; Wersinger et al. 2004). Since the V1b receptor is expressed in pyramidal cells of the CA2 hippocampal area (Young et al. 2006), which are similar to place cells involved in spatial memory formation, it has been hypothesized that this particular population of pyramidal cells would be involved in either the formation or recall of memories of past social encounters, and that the V1b receptor would be relevant in the modulation of these memories (Caldwell et al. 2008b).

4. Variation in cooperative behaviour: integrating hormones and cooperation

From what has been outlined in the previous sections, it should be clear that most hormones cannot be expected to directly cause behavioural decisions. Unlike neuronal signals, hormones target areas of the brain rather than specific post-synaptic cells, or may reach the entire body, thus influencing systemic adjustments. Hormones may affect the basic mood of an individual, which in turn may influence neuronal decisions. With this in mind, we can now ask under which conditions hormones may influence levels of cooperation.

Under natural conditions, hormones may affect levels of cooperation in three different ways. First, hormones may work through organizational effects during ontogeny, which may cause differentiation between individuals with respect to growth, strength and personality, and which in turn may lead to consistent differences between individuals with respect to behaviour; including levels of cooperation (see Bergmuller et al. 2010). Second, activational effects, in which hormones prepare individuals for specific life-history stages, may promote or hinder the expression of cooperative behaviour in different life stages. Third, species with different life histories will differ both with respect to organizational and activational effects of hormones, which may also promote or hinder the expression of cooperative behaviour. We will now discuss these three issues in more detail.

(a). Organizational effects of hormones and levels of cooperation

There is plenty of evidence to suggest that the early environment causes long-term effects in individual organization and behaviour, including levels of cooperation. A large body of work in litter-bearing rodents (especially mice, rats and gerbils) has demonstrated that siblings influence each other's hormonal milieu in utero, which in turn affects their morphology and social behaviour later in life (Clark & Galef 1998). For example, male Mongolian gerbils (Meriones unguiculatus) situated during gestation between two female siblings show up to a 50 per cent increase in alloparental care (to offspring that are not their own) and lower rates of aggressive behaviour when compared with males situated between two male siblings during gestation (Clark & Galef 1998, 2000).

Elegant studies in rats have shown that the mother's degree of maternal care (e.g. her licking and grooming) to newborn offspring can profoundly affect its behavioural phenotype during adulthood, in particular its stress responsiveness, brain organization and social behaviour (including her own maternal behaviour as an adult, e.g. Meaney 2001). These effects are mediated by stress hormones, the secretion of which is increased after the separation of the mother from her offspring and decreased again when she is allowed to lick and groom her offspring intensively after the separation. The organizational effect of maternal behaviour on her offspring has been pinpointed to epigenetic effects, specifically a demethylation of the glucocorticoid receptor gene in the hippocampus, leading to a permanent increase in the expression of this receptor with subsequent effects on stress and social behaviour in adult life (e.g. Weaver et al. 2004). Mothering behaviour also has strong effects on other aspects of hippocampal organization, such as dendrite and spine lengths, affecting memory and cognitive abilities of her offspring in adulthood (Champagne et al. 2008). Finally, the degree of maternal care can also determine the density of OT and oestrogen receptors in the medial preoptic area in females, and of AVP receptors in males (Champagne et al. 2003).

Not only steroid hormones, but also neuropeptides can impact an individual's social phenotype during development (reviewed in Cushing & Kramer 2005). Both OT and AVP have organizational effects on the brain both during the neonatal as well as postnatal periods in rodents, with sex-specific effects (Cushing & Kramer 2005). In prairie voles, a single injection of OT on the day of birth affects aggressive behaviour (Carter 2003) and male alloparenting rates (Bales et al. 2004). Likewise, administration of AVP soon after birth increases aggressive behaviour in adult male prairie voles (Stribley & Carter 1999) and changes risk-taking and social behaviour in rats (Boer et al. 1994). Furthermore, in California mice (Peromyscus californicus), social experiences during the pre-weaning period (being raised by either monogamous conspecifics or by closely related polygynous P. leucopus) changed the aggressive behaviour expressed by these mice as adults with corresponding changes in brain AVP expression (Bester-Meredith & Marler 2001).

(b). Activational effects of hormones to prepare individuals for life-history stages

In long-lived species and in particular those inhabiting arctic and temperate climates, environmental conditions change in a predictable fashion across the seasons. Species have adapted to these changes by adjusting their life history: specific times of the year are used for specific activities. Most notably, spring and summer are typically used for reproduction. Hormones play a key role in the transitions from one life-history stage to the next (e.g. Wingfield et al. 2006). When different life-history stages are linked to different cooperative tasks or to different levels of cooperation and competition, hormones may be part of the mechanistic explanation of individual variation in levels of cooperation throughout the year.

In males, an increase in testosterone prepares the individual for breeding: the size of the gonads increases and sperm production starts, and secondary sexual traits may develop. At the behavioural level, the increase in testosterone levels prepares males to display territorial behaviour, courtship and mating. Consequently, males can be expected to be particularly self-centred during this life-history stage and hence less cooperative. However, testosterone may play a positive role in pair-bonding during this life-history stage in long-term monogamous species (see §3b(iii) above), and in group-living species testosterone could play a role in cooperative tasks of males that involve aggressive behaviour, i.e. the defence of a common territory.

Corticosteroids prepare animals for energy-demanding periods. In many vertebrates, baseline levels of corticosteroids are higher during the breeding season than during the rest of the year. These changes in baseline levels are known to affect social behaviour in various ways, i.e. by enhancing foraging, food intake (Koch et al. 2002), attention levels and alertness (Chapotot et al. 1998), and also affectionate interactions with infants (Fleming et al. 1997). Parents are prepared to work hard to raise their young. In species with biparental care, glucocorticoids may have a positive effect on cooperation between parents raising offspring (Goymann & Wingfield 2004). Glucocorticoids also have a positive effect on pup feeding by helpers in cooperatively breeding meerkats (Carlson et al. 2006a). Helpers with higher glucocorticoid levels provided more food to pups than helpers with low glucocorticoid levels. However, it is important to emphasize that the effects of glucocorticoid concentrations are usually nonlinear. Above a critical threshold, glucocorticoids cause an emergency life-history response, which may result in abandoning current activities, including the desertion of young or maintaining social relationships (Adkins-Regan 2005).

In male meerkat helpers, Carlson et al. (2006b) also found an effect of prolactin on helping behaviour. Elevated prolactin levels immediately preceded decisions to babysit. Furthermore, it appears that subordinate females downregulate oestrogen levels to a point that their fertility is greatly reduced (Young et al. 2008a,b). They are more likely to remain in the helper role than to become a breeder. The downregulation is most likely an adaptation to the risks that dominant females will kill offspring of subordinates or that they will evict pregnant subordinates (Young et al. 2008a,b). Thus, at least three types of hormones—prolactin, oestrogen and glucocorticoids, seem to affect helping behaviour in one general context, namely the contribution of helpers in cooperatively breeding species.

(c). Hormonal effects that explain differences between species

Species differ in life histories, and hormones are involved in the regulation of and transition between life-history stages. Because the effects of hormones differ between species with different life-history stages, one could predict that they also play a role in modulating species differences in cooperation. In particular, personality differences between species and differences in social tolerance may be key features modulated by hormones that could have implications on cooperative behaviour. For example, the behavioural differences between cooperatively breeding primate species and others, as discussed by Jaeggi et al. (2010) and by de Waal & Suchak (2010), may well be due in part to divergent organizational effects of hormones. In estrildid finches, species differences in sociality are associated with the differential activation of AVT neurons in the bed nucleus of the stria terminalis (BNST), a brain nucleus known to regulate social approach and withdrawal (Goodson & Wang 2006). In response to the presence of a same sex individual, the expression of immediate early genes in BNST AVT neurons is higher in individuals from gregarious and colonial species than in individuals from territorial species (Goodson & Wang 2006). The number of AVP neurons in the BNST and the density of the AVP V1a receptor in the lateral septum, an area that receives projections from BNST neurons, is also higher in gregarious than in asocial species (Goodson & Wang 2006; Goodson et al. 2006). These results reveal a mechanism that allows gregarious species to accentuate the positive valence of social stimuli during social interactions, therefore promoting prosocial behaviour. If we generalize these findings, we could predict similar results between phylogenetically related species that express or do not express cooperative behaviour.

Comparative studies of central AVP and OT systems in the vole brain have revealed a conserved pattern of distribution for AVP and OT, but an otherwise divergent pattern between monogamous and promiscuous vole species in terms of distribution of their receptors (see Young et al. (2008a,b) for a recent review). These differences are associated with the type of social organization and are not necessarily species-specific, since both monogamous prairie and pine voles share a similar pattern of AVP V1a receptor and OT receptor labelling in the brain, whereas promiscuous montane and meadow voles show a different pattern (Young et al. 2008a,b). Neuropharmacological studies have confirmed the close involvement of AVP and OT in pair bond formation across vole species. Brain administration of a V1a receptor or an OT receptor antagonist prevented mating, but induced partner preference formation, whereas the administration of AVP induced partner preference even without mating (e.g. Winslow et al. 1993; Williams et al. 1994; Lim et al. 2004). Although it was initially thought that AVP and OT would have sex-specific roles in pair-bonding, recent evidence demonstrates that both neuropeptides are involved in pair-bond formation in both sexes, with males being more sensitive to AVP, and females to OT (Young et al. 2008a,b). The involvement of AVP and OT in social attachment in a non-mating context remains to be investigated. In short, both comparative data in estrildid finches presented above, on the association between the degree of sociality and the socially driven activation of AVT neurons in the BNST, together with data on the association among pair-bonding, mating systems and OT in voles, suggest that the evolution of social behaviour, including cooperative behaviour, may emerge by selection on discrete nuclei in the brain (Goodson et al. 2005), which are receptive to hormonal or neurohormonal modulation.

5. Concluding remarks

In this review, we have established a potential role for hormones as mechanistic levers of cooperative behaviour. Because there is very little, if any, research on these hormonal mechanisms in a cooperative context, most of what we have developed was based on knowledge about the role of hormones in general social behaviour. Although much of what we have stated above remains speculative for a cooperative scenario, this reflection is basically intended to outline the first mechanistic research approach to cooperative behaviour. In the ‘finite state machine theory’, Jacobs & Wingfield (2000) explain why different hormones may be linked to the same behaviour during different life-history stages. We think that this should be kept in mind; just as territorial aggression may be regulated by different hormones during different life-history stages (Canoine & Gwinner 2002; Soma et al. 2008), different hormones may modulate cooperation in different life-history contexts. Indeed, hormones may play a more prominent role in the modulation of behaviour when this is restricted to a particular life-history stage or context (Adkins-Regan 2005), contrary to a permanent or year-round behaviour, in which activation is less likely to be influenced by hormones.

Thus, year-round cooperative behaviour, such as cooperative territoriality or hunting, may be less modulated by hormones than seasonal cooperative behaviour, such as cooperative breeding. Year-round cooperative behaviours may be hard-wired by neuronal pathways instead or influenced by hormones only during ontogeny (i.e. organizational effects). Furthermore, the time-course of cooperative behaviour may be important: most hormones act within minutes. Thus, if cooperative behaviour changes within seconds, it is unlikely to be hormone-dependent. For example, increased cooperative behaviour induced by a previous interaction with a predator client in cleaner fish (Bshary et al. in press) is more likely to be modulated by changes in neurotransmitters and central neuromodulators than by hormones produced in peripheral glands. For example, in rainbow trout (Oncorhynchus mykiss), individuals defeated by larger aggressive fish display subsequent displaced aggression towards smaller individuals. The possibility of redirecting aggression towards a subordinate fish is associated with increased serotonergic activity in the forebrain rather than changes in circulating cortisol (Øverli et al. 2004). This does not mean that steroids may not mediate rapid actions on behaviour, but the delay between the stimulus that elicited the steroid response and the subsequent effects of this steroid surge on behaviour will occur in the order of minutes rather than seconds (Remage-Healey & Bass 2006b). Thus, the temporal scale of variation in cooperative behaviour may be used as a cue to whether hormones may be involved in the modulation of this behaviour.

In conclusion, this review is generally meant to serve as a starting point in integrating the fields of endocrinology and cooperative behaviour. We hope this overview will generate interest and unlock new research avenues that will unravel the complexity of cooperative behaviour on a more mechanistic level.