Oxytocin modulates mate-guarding behavior in marmoset monkeys

Abstract

In socially-monogamous species, intolerance of interactions between a pairmate and a sexual rival (i.e., mate-guarding) promotes the preservation of long-lasting partnerships. One promising neurobiological candidate for the regulation of mate-guarding behavior in monogamous primates is the oxytocin (OT) system, given its established role in both the development of monogamous bonds and the behavioral processes that facilitate the preservation of those bonds. In this study, male and female marmosets were exposed to a same-sex intruder in their home environment during conditions when their pairmate was present and absent, and across three treatment conditions (OT receptor agonist; saline control; OT receptor antagonist). Saline-treated marmosets spent significantly more time in proximity to the intruder, relative to the empty pairmate enclosure, when their pairmate was absent. However, when marmosets received OT they spent less time in proximity to the intruder, indicating that OT may reduce interest in a same-sex stranger in a territorial context. When their pairmate was present, saline-treated marmosets spent equal time in proximity to both intruder and pairmate; yet when they received OT they spent significantly more time in proximity to the intruder, indicating that OT may increase interest in a same-sex stranger in a mate-guarding context. While OT treatment did not directly influence the expression of aggression, OT system manipulations impacted the expression of selective social interest during an intruder challenge, suggesting that OT may enhance adaptive responses to social challenges. Moreover, these findings add to the converging evidence that the OT system regulates behavioral processes that underlie the preservation of established relationships.

The formation and preservation of a high quality monogamous bond is crucial for physiological and psychological well-being (Holt-Lunstad et al., 2010; Tay et al., 2013), as well as the survival and reproductive success of both parents and offspring in socially monogamous species (Rasmussen, 1981). While the most conspicuous features of a monogamous relationship are the behavioral manifestations of a ‘pairbond’ between two individuals, which include pairliving, high levels of mate-directed affiliation, and a pervasive and reciprocal partner preference (French et al., 2017), intolerance and active discouragement of extra-pair sociosexual encounters between a long-term pairmate and a sexual rival (i.e., mate guarding) is essential for bond preservation. Mate guarding behavior serves two purposes: (1) to prevent genetic cuckoldry by same-sex strangers (aka ‘mate poaching’; Schmitt and Buss, 2001), and (2) to maintain access to a pairmate by preventing defection; these are accomplished via selective aggression toward a same-sex stranger, as well as maintaining proximity with a mate.

Mate-guarding behavior is part of a fundamental suite of social behaviors among monogamous and non-monogamous primates. Since the majority of primates engage in polygynous or polygynandrous mating, males are principally responsible for engaging in mate-guarding behavior, as they must maintain access to multiple females (Alberts et al., 2006; Arlet et al., 2008; Boesch et al., 2006; Setchell et al., 2005; Watts, 1998; Weingrill et al., 2003). However, in socially monogamous primates both sexes often express mate-guarding behavior. Some socially monogamous primates (e.g., marmosets, tamarins) express context-specific aggression toward same-sex rivals (Buss, 2002; French et al., 1995; French and Inglett, 1989; French and Snowdon, 1981; Ross et al., 2004; Ross and French, 2011), while others (e.g., titi monkeys, owl monkeys) engage in mate exclusivity behavior, including a reluctance to approach and interact with both same-sex and opposite-sex strangers, and may even engage in agonistic displays toward opposite-sex strangers during an intruder challenge (Anzenberger et al., 1986; Fernandez-Duque, 2004; Fernandez-Duque and Huck, 2013; Fisher-Phelps et al., 2015, 2015; Mendoza and Mason, 1986; Wolovich et al., 2010). While mate guarding serves as an important behavioral mechanism to maintain access to mate(s) in both monogamous and non-monogamous primates (Brotherton and Komers, 2013), the consequences of mate defection and genetic cuckoldry on reproductive success are arguably more disadvantageous for monogamous species, including humans, due to the potential loss of their only mate. Thus, it is essential to improve our understanding of the behavioral and neurobiological mechanisms underlying this fidelity-promoting behavior in monogamous primates.

One promising neurobiological candidate for the regulation of mate-guarding behavior in monogamous primates is the oxytocin (OT) system, given its established role in both the development of long-term monogamous bonds (Lieberwirth and Wang, 2016; Young et al., 2011) and the behavioral processes that facilitate the preservation of those bonds (French et al., 2017). OT has also attracted interest as a neuromodulator a host of social, emotional, and cognitive process (Caldwell, 2017; Caldwell and Albers, 2015; de Jong and Neumann, 2017; Rilling and Young, 2014), modifying contextually-specific responses to social agents (Bartz et al., 2011), magnifying selective attention to social stimuli (Chang and Platt, 2014), increasing negative social judgments of others (Shamay-Tsoory et al., 2009), and amplifying in-group/out-group biases (De Dreu, 2012). Social conflict and same-sex aggression have been directly linked to oxytocinergic activity in a variety of social contexts in several rodent studies. Following aggressive encounters with same-sex intruders, OT levels were increased in the paraventricular nucleus (PVN) of the hypothalamus in monogamous female California mice (Trainor et al., 2010) and in the PVN and central amygdala of female rats (Bosch et al., 2004; Bosch, 2005). Non-monogamous female rats that were ‘conditioned’ to express mate-guarding behavior had higher levels of double-labeled Fos/OT neurons in the supraoptic nucleus (SON) of the hypothalamus and PVN, as well as increased Fos-expression in the nucleus accumbens (NAcc), than non-conditioned females following exposure to a same-sex stranger (Holley et al., 2015, 2014). Moreover, OT exposure during early sensitive periods increased intrasexual aggression and decreased affiliation in females, but not males, in monogamous prairie voles during adulthood, suggesting that developmental exposure to OT has persistent and sex-dependent effects (Bales and Carter, 2003). While there is converging evidence that the OT system regulates selective aggression and territoriality in rodents, there remains a massive gap in our understanding of the role of the OT system in monogamous mate-guarding behavior in a translational primate model.

Mate-guarding behavior is studied in lab populations via the use of a standardized intruder challenge, where a same-sex stranger is placed within the home environment of an established monogamous pair. During an intruder challenge, it is often difficult to attribute intruder-directed aggression to the protection of any one resource since males and females often engage in aggressive behavior to protect multiple resources simultaneously (e.g., mates, offspring, food, territory). Thus, in order to examine the relative contributions of the OT system to both mate-guarding behavior and territoriality, we experimentally manipulated the absence/presence of a pairmate during territorial intrusions in marmoset monkeys (Callithrix jacchus). We reasoned that if marmosets primarily utilize selective aggression to promote pair exclusivity, then expression of aggressive behavior would be greatest during intrusions from a same-sex stranger when their pairmate is present, relative to conditions when their pairmate is absent. To the contrary, if marmosets primarily utilize selective aggression to protect territorial resources, then the expression of aggressive behavior during territorial intrusions from a same-sex stranger would be stable across conditions when their pairmate is present or absent. To the extent that the OT system modulates territoriality, but not mate-guarding behavior, we predicted that marmosets treated with an OT receptor (OTR) agonist would display increased selective aggression toward a same-sex intruder regardless of the presence or absence of their pairmate, relative to a saline control. Alternatively, if the OT system modulates mate-guarding behavior, then marmosets treated with an OTR agonist will selectively increase their expression of stranger-directed aggression and mate-directed affiliation when their pairmate is present compared to when their pairmate is absent. Finally, we explored links between OT-system manipulations and hypothalamic-pituitary-adrenal (HPA) and hypothalamic-pituitary-gonadal (HPG) function. We predicted that to the extent that the OT system enhances the social salience of same-sex territorial intruders, then marmosets receiving an OTR agonist will display increased levels of cortisol and testosterone post-intruder exposure, while marmosets that receive an OTR antagonist will display decreased levels of cortisol and testosterone post-intruder exposure.

Method

Subjects

Eight (four male and four female) nulliparous adult white-tufted ear marmosets (Callithrix jacchus), housed at the Callitrichid Research Center (CRC) at the University of Nebraska at Omaha, served as subjects in this experiment. Additionally, twelve Callithrix spp. (six jacchus, six penicillata) were used as intruders; intruders did not serve as subjects and each intruder was only used as a stimulus animal once for each subject. Subjects were 5.69 ± 0.24 (mean ± SEM) years of age at the start of the study and had cohabitated with the same partner for at least 2.5 years in large indoor wire-mesh enclosures (1.0 × 2.5 × 2.0 m), equipped with a sleeping hammock, natural branches for climbing and various enrichment materials. Visual access was restricted between enclosures, but auditory and olfactory was possible among enclosures. Colony rooms at the CRC were maintained on a 12 h: 12h light: dark cycle and at a temperature range between 19 °C and 22 °C. For all dietary and husbandry protocols please refer to Schaffner et al. (1995). All procedures were approved by the University of Nebraska at Omaha/University of Nebraska Medical Center Institutional Animal Care and Use Committee (#12–099-12; #13–048-07).

Drug Treatments

In a within-subjects design, marmosets were administered 150 µg/kg (50 µg/100µL saline solution) of an OTR agonist native to marmosets (Pro⁸-OT, synthesized by Anaspec Corp, California) (French et al., 2016), a saline control, or 20 mg/kg OTR antagonist (OTA; L368,899®; provided by Dr. Peter Williams, Merck & Co., Inc.). Due to the availability of peripherally-administered pharmacological compounds, marmosets received both an intranasal treatment and an oral treatment during each condition (Table 1). Oral administration of OTA or saline control was administered in a preferred food item in their home enclosure 90 minutes prior to the intruder challenge; subjects remained in their home enclosures for 60 minutes following oral administration. The non-peptide OTA (L-368,899®) is readily absorbed by the bloodstream after passage through the digestive system (Thompson et al., 1997), reaches peak concentration in the cerebrospinal fluid (CSF) 90 minutes after administration, is maintained at 50% peak concentration 4+ hours after administration, accumulates in areas of limbic system (Boccia et al., 2007), and has a high affinity for the OTR (and a moderate affinity for V1aR; Manning et al., 2012). During OTR antagonist conditions, marmosets also received an intranasal saline control treatment.

Table 1.

Experimental Design

Condition	Pairmate Presence	Intranasal treatment	Oral treatment
Oxytocin receptor agonist	Present	OT	SAL
	Absent	OT	SAL
Oxytocin receptor antagonist	Present	SAL	OTA
	Absent	SAL	OTA
Saline control	Present	SAL	SAL
	Absent	SAL	SAL
Intruder-absent control	Present	SAL	SAL
	Absent	SAL	SAL

Note. Marmosets experienced same-sex intruder challenges over a series of six counterbalanced conditions derived from a factorial design, and an additional two intruder-absent control trials. Marmosets received an intranasal treatment and an oral treatment during each condition (Oxytocin receptor agonist = OT, Oxytocin receptor antagonist = OTA, Saline control = SAL).

Intranasal OTR agonist or saline control was administered 30 minutes prior to intruder challenge during a brief (~3 min.) manual restraint using a 100-µL Eppendorf pipette to administer 50 µL of solution to each nostril drop-wise (30 s between each nostril). Doses were determined based on previous primate literature (Boccia et al., 2007; Cavanaugh et al., 2014; Heinrichs et al., 2003; Parker et al., 2005; Smith et al., 2010). Peptides administered intranasally are quickly absorbed into the bloodstream via the nasal passage (Pires, Fortuna, Alves, & Falcão, 2009), and some fraction of the peptides appear to bypass the blood-brain barrier to access the central nervous system (CNS) via the olfactory bulb and the maxillary branch of the trigeminal nerve (MacDonald & Feifel, 2013; Quintana, Alvares, Hickie, & Guastella, 2015). Intranasal neuropeptides are transported to the CNS and accumulate in the CSF in humans (Striepens et al., 2013) and macaques (Dal Monte, Noble, Turchi, Cummins, & Averbeck, 2014). In rats and mice, OT levels were increased in microdialysates from the hippocampus and amygdala, and in plasma, 30–60 minutes after intranasal administration (Neumann, Maloumby, Beiderbeck, Lukas, & Landgraf, 2013). Peripheral levels of OT after intranasal administration peak at 60 minutes and persist for up to 7 hours in humans (van IJzendoorn, Bhandari, van der Veen, Grewen, & Bakermans-Kranenburg, 2012). The OTR agonist (Pro⁸-OT) also has a high affinity for marmoset OTRs (Mustoe et al., 2018). During OTR agonist conditions, marmosets also received an oral saline control treatment. During the 30-minute uptake period after intranasal treatment, subjects were moved to a large transport cage (50 × 50 × 50 cm) located in a room some distance from their home enclosure. After the drug uptake period, marmosets were returned to their home enclosures 60 seconds before the intruder challenge.

Behavioral Paradigm

Marmosets experienced same-sex intruder challenges over a series of six counterbalanced conditions derived from a factorial design [pairmate presence (present; absent) * treatment condition (OTR agonist; saline control; OTR antagonist)], and an additional two intruder-absent control trials (see description below). An intruder was housed in a clear PVC enclosure (50 × 50 × 50 cm), which was suspended at branch-height in the subject’s home enclosure. This procedure allowed for the full expression of agonistic behavior between the subject and same-sex intruder, while eliminating the risk of injury to all animals. Subjects were exposed and habituated to the empty intruder enclosure for several weeks prior to testing. For each subject, a different same-sex stranger was used for each condition. In the pairmate-present condition, the subject’s long-term pairmate was restricted to an enclosure 1.5 m from the intruder enclosure. Since marmosets are sensitive and responsive to the behavioral state of their pairmate (Caselli et al., 2018) the pairmate was restricted from interacting with the intruder to limit behavioral non-independence. The untreated pairmate was removed from the home-enclosure during the pairmate-absent condition and transferred to a transport cage located in a room some distance from its home-enclosure.

For each OT-treatment condition, the subject experienced the intruder challenge once with, and once without, their pairmate present in counterbalanced fashion. Social behavior (i.e., proximity, attention, aggression, affiliation, vocalization) directed toward the intruder and pairmate was observed during the intruder challenge (Table 2). Locomotion was measured by dividing the home-enclosure into four equal zones (top-back, top-front, bottom-back, bottom-front). After the intruder challenge, the same-sex stranger was returned to their home-enclosure. The subject’s pairmate was returned (during pairmate-absent conditions) or released from the pairmate enclosure within the home-enclosure, and a 10-minute reunion observation was conducted. There was a 4–6 day period between exposure to intruder challenge as a subject and exposure to intruder challenge as a partner, and a 6–8 day washout period between each drug treatment. A schematic of the behavioral paradigm is presented in Figure 1.

Table 2.

Ethogram

Behavior	Operational definition
Approach	Moving to zone containing pairmate or intruder enclosure
Proximity	Duration within zone containing pairmate or intruder enclosure
Interaction	Duration in contact with pairmate or intruder enclosure
Locomotion	Sum of zone crosses. The home-enclosure consisted of four zones of equal size, split down the middle horizontally and vertically
Leap	Jump onto substrate of intruder enclosure with aggressive posturing
Attack	Rapidly charge the intruder enclosure with aggressive posturing
Erh-Erh vocalization	Low guttural call, often accompanied by aggressive posturing
Alarm vocalization	Short sharp call, often accompanied by startle response from family
Phee vocalization	Contact call, long in duration and high in pitch
Piloerection	Hair extended from body
Enclosure manipulation	Grasp or bite the wire of enclosure
Scent marking	Anogenital rub on substrate
Eating/drinking	Consuming food or water

Note. Behaviors observed during the intruder challenge.

Figure 1.

Design of the intruder challenge is displayed [subject=black; pairmate=dark grey; intruder=light grey]. In the intruder trials, subjects received three drug treatments [oxytocin receptor agonist Pro⁸-OT; oxytocin receptor antagonist (OTA: L368,899); saline control] across two pairmate presence conditions [pairmate present; pairmate absent]. In the two intruder-absent control trials, the procedure was the same as above except the intruder (light grey) was not present for either pairmate present or pairmate absent conditions and subjects received saline.

Intruder-absent control trials.

We also performed two counterbalanced intruder-absent trials (pairmate present; pairmate absent) that served as important controls for exposure to novel stimuli. The procedure for intruder-absent control trials was the same as above except subjects received saline for both pairmate-present and pairmate-absent conditions, and an empty intruder enclosure was placed within the subject’s home-enclosure instead of an intruder enclosure with a same-sex stranger. Subjects did not receive either OTR agonist or OTR antagonist for intruder-absent control trials.

Hormone Analysis

Urine samples were collected to measure excreted cortisol and testosterone prior to and following the intruder challenge using non-invasive techniques described by (French et al., 1996). After collection, urine samples were centrifuged at 2,000 rpm for five minutes to separate sediment from the sample. The supernatant was then transferred to a clean vial and stored at – 20°C pending assay. Testosterone levels, from hydrolyzed and extracted samples, were measured via enzyme immunoassay (EIA; for details see Nunes et al. (2000). The mean of three extraction recovery estimates was 90.0%; hormone levels were adjusted accordingly to account for procedural losses of steroid.

To quantify urinary testosterone, microtitre plates were coated with testosterone antibody (Ab; 5/98), diluted 1:15000 in bicarbonate coating buffer, and incubated for 12 h. Testosterone standards were diluted in phosphate-buffered saline (PBS) and ranged from 1000 to 7.8 pg/well. Labeled testosterone conjugate (horseradish peroxidase; HRP, 12/03) was diluted 1:12,000 in PBS. After the 12 h incubation, 50 μl of PBS was added to each well, followed by 50 μl of the extracted urine samples (1:100 in PBS) or testosterone standards. After 50 μl of HRP was added, the plates were set to incubate for 2 h. Free and bound hormones were separated, after which an EIA substrate (ABTS, H₂O₂) was added. Absorbance at 405 nm was measured in a microplate reader. Intra-assay coefficients of variation (CV) for high and low concentration pools were 3.64% and 3.20%, respectively. Inter-assay CVs for the same high and low concentration pools were 11.58% and 10.79%, respectively.

To quantify urinary cortisol, microtitre plates were coated with cortisol Ab (3.6.07), diluted to 1:25,000 in bicarbonate coating buffer, and incubated for 12 h. Cortisol standards were diluted in PBS and ranged from 1000 to 7.8 pg/well. Labeled cortisol HRP (R4866) was diluted 1:35,000 in PBS. After the 12 h incubation, 50 µl of PBS was added to each well, followed by 50 µl of the diluted urine samples (1:6400 in distilled water) or cortisol standards. After 50 µl of HRP was added, the plates were set to incubate for 2 h. Free and bound hormones were separated, after which an EIA substrate (ABTS, H₂O₂) was added. Absorbance at 405 nm was measured in a microplate reader. Intra-assay CVs for high and low concentration pools were 4.62% and 6.08%, respectively. Inter-assay CVs for the same high and low concentration pools were 9.56% and 10.37%, respectively. Full details on the validation of this assay for marmosets can be found in (Smith and French, 1997). The mass of cortisol and testosterone is expressed in µg/mg of creatinine (Cr; measured using a standard Jaffé reaction colorimetric assay; French et al. (1996) to account for variable fluid intake.

Data Analysis

To assess differential expression of social behavior during the intruder challenge we calculated several derived variables. Locomotion was measured by summing the number of zone crosses. An interaction index was calculated to assess the % of time marmosets spent in direct contact with the enclosures of the pairmate and the intruder. This was accomplished by taking value A (proportion of time marmosets spent directly interacting with the intruder enclosure, relative to the time spent interacting with the enclosures of both the pairmate and intruder) and subtracting it from value B (the proportion of time marmosets spent directly interacting with the pairmate enclosure, relative to the time spent interacting with the enclosures of both the pairmate and intruder) [i.e., ((Pairmate) / (Pairmate + Intruder)) − ((Intruder) / (Pairmate + Intruder))]. This measure yields a score from +1.0 to −1.0, where positive values indicate a greater proportion of time spent on the pairmate’s enclosure than the intruders, and negative scores reflect the converse. Aggression was calculated by summing the number of attacks and the number of leaps. Additional territorial behaviors (e.g., scent marking, erh-erh vocalization) were scored individually. For both cortisol and testosterone, pre- and post-territorial intrusion concentrations were measured in samples collected on the morning of behavioral testing (immediately prior to intruder tests) and on the morning after behavioral testing, respectively. We evaluated the effect of drug treatment and pairmate presence on hormonal and behavioral measures using several-mixed model ANOVAs, with OT-treatment (OTR agonist; saline control; OTR antagonist; saline intruder-absent) and pairmate presence (pairmate present; pairmate absent) as within-subject factors, and sex as a between-subject factor (4 × 2 × 2). If main effects or interactions were significant, post-hoc comparisons were made using Fisher’s least significant difference. All alpha levels were set at p < 0.05. Effect sizes were calculated and reported as follows: Eta-squared was calculated using η² = SS_effect/SS_total for repeated-measures F tests. Cohen’s D was calculated using d = M₁ − M₂ / σ_pooled for paired-samples t-tests and d = t/√n for one-sample t-tests.

Results

Across all conditions, marmosets approached the intruder enclosure significantly more quickly [F(1,7) = 26.9, p = 0.001, η² = 0.8] and frequently [F(1,7) = 8.8, p = 0.02, η² = 0.6], and spent a longer duration in proximity to the intruder enclosure [F(1,7) = 16.4, p = 0.005, η² = 0.7] than they did with the pairmate enclosure. Marmosets also interacted directly with the intruder closure significantly more quickly [F(1,7) = 18.7, p = 0.003, η² = 0.7] and frequently [F(1,7) = 12.2, p = 0.01, η² = 0.6], and spent a longer duration interacting with the intruder enclosure [F(1,7) = 19.7, p = 0.003, η² = 0.7] than they spent interacting with the pairmate enclosure, across all conditions (Table 3). Neuropeptide treatment and pairmate presence did not significantly interact to influence proximity and interaction measures [p’s > 0.05].

Table 3.

Differential expression of social behavior across all conditions

	Pairmate enclosure		Intruder enclosure		F (1,6)	p	η2
Approach Latency (min)	3.9 (±0.8)		0.4 (±0.2)		26.9*	.001	0.8
Approach Frequency (per hr)	30.0 (±6.1)		40.1 (±8.3)		8.8*	.02	0.6
Proximity Duration (min/hr)	13.2 (±1.9)		26.1 (±1.5)		16.4*	.005	0.7
Interaction Latency (min)	12.2 (±2.5)		1.8 (±0.9)		18.7*	.003	0.7
Interaction Frequency (per hr)	11.5 (±3.0)		28.4 (±6.2)		12.2*	.01	0.6
Interaction Duration (min/hr)	2.5 (±0.6)		10.7 (±1.8)		19.7*	.003	0.7
Differential expression of social behavior across OT treatment conditions
	OT	OTA	SAL	SAL (intruder absent)	F (3,21)	p	η2
Latency to first aggression (min)	10.3 (±3.8)	7.0 (±2.5)	8.6 (±2.9)	15.1 (±4.2)	1.7	0.2	0.2
Aggression frequency (per hr)	24.5 (±11.7)	13.8 (±5.4)	17.8 (±6.8)	3.6 (±1.3)	2.4	0.09	0.3

Note. Means [± SEMs] for selected behaviors expressed during the intruder challenge. Statistics for main effect of stimulus and OT treatment are included.

^*F values [p < 0.05].

Effects of OT and partner presence on proximity to pairmate and intruder enclosures.

Oxytocin treatment and pairmate presence differentially influenced the amount of time marmosets spent in proximity to the enclosures of the same-sex intruder and the pairmate [F(3,21) = 4.0, p = 0.02, η² = 0.4]. During conditions when their pairmate was absent [Figure 2A], saline-treated marmosets spent significantly more time in proximity to the same-sex intruder compared to the empty pairmate enclosure [t(7) = 10.7, p < 0.001, d = 4.93]; this pattern was unchanged when they received an OTA [t(7) = 4.2, p = 0.004, d = 2.45]. However, when marmosets were treated with an OTR agonist they spent an equal amount of time in proximity to the same-sex intruder compared to the empty pairmate enclosure [t(7) = 0.6, p > 0.05, d = 0.37]. When their pairmate was absent, saline-treated marmosets also spent more time in proximity to the intruder enclosure than the pairmate enclosure during control trials when the intruder was absent [t(7) = 6.1, p < 0.001, d = 2.68]. During conditions when their pairmate was present [Figure 2B], saline-treated marmosets spent equal time in proximity to both the same-sex intruder and their long-term pairmate [t(7) = 0.4, p < 0.05, d = 0.19]. However, OTR agonist-treated marmosets spent significantly more time in proximity to the intruder compared to their pairmate [t(7) = 2.8, p = 0.02, d = 1.67]. This effect was largely driven by an increase in time spent in proximity to the intruder enclosure [t(7) = 2.0, p = 0.09, d = 0.83], as opposed to a decrease in time spent in proximity to their pairmate [t(7) = 1.3, p > 0.05, d = 0.42]. When marmosets received an OTA, they spent moderately more time spent in proximity with the same-sex intruder compared to their pairmate [t(7) = 2.2, p = 0.06, d = 1.45]. When their pairmate was present, saline-treated marmosets exhibited no differences in the time spent between intruder and pairmate enclosures [t(7) = 0.5, p > 0.05, d = 0.19].

Figure 2.

Mean (± SEM) duration of time spent in proximity to the pairmate enclosure and intruder enclosure, during (A) pairmate absent and (B) pairmate present conditions, are expressed as a function of OT treatment. Marmosets were treated with OT receptor agonist (OT), OT receptor antagonist (OTA), and saline control (SAL) during conditions when the intruder was present, and saline control (SAL – hatched bars) when the intruder was absent. Letters and asterisks indicate significant differences (a > b > c > d at p < 0.05, * at p < 0.05, ^# at p < 0.1). Data points from individual marmosets are displayed for each condition (♦; n=8).

Effects of OT on direct interactions with pairmate and intruder.

OT treatment, but not pairmate presence or the interaction between pairmate presence and neuropeptide treatment, modulated the frequency [F(3,21) = 3.7, p = 0.02, η² = 0.3; Figure 3A] and duration [F(3,21) = 4.9, p = 0.01, η² = 0.4; Figure 3B] that marmosets directly interacted with the enclosures of the same-sex intruder and the pairmate. Saline-treated marmosets interacted with the intruder enclosure significantly more frequently [t(7) = 2.7, p = 0.03, d = 1.32] than they interacted with the pairmate enclosure; this pattern was unchanged when they received an OTR agonist [t(7) = 3.6, p = 0.01, d = 1.26] or an OTA [t(7) = 2.6, p = 0.03, d = 0.60]. However, there was no difference in how frequently marmosets interacted with the enclosures of the intruder or their pairmate during control trials when the intruder was absent [t(7) = 1.3, p > 0.05, d = 0.61]. This effect was largely driven by a decrease in the frequency of interactions with both the intruder enclosure [t(7) = 1.9, p < 0.1, d = 1.30] and the pairmate enclosure [t(7) = 1.9, p < 0.1, d = 0.08]. Saline-treated marmosets also directly interacted with the intruder enclosure for a greater duration than they interacted with the pairmate enclosure [t(7) = 3.3, p = 0.01, d = 1.25]; this pattern was unchanged when they received an OTR agonist [t(7) = 4.1, p = 0.004, d = 2.26] or an OTA [t(7) = 3.7, p = 0.007, d = 2.05]. This effect was driven by a significant decrease in the duration of time OT-treated marmosets spent interacting with the pairmate enclosure [t(7) = 2.4, p = 0.04, d = 1.01] and a moderate increase in the duration of time OT-treated marmosets spent interacting with the intruder enclosure [t(7) = 2.2, p = 0.06, d = 0.59]. Saline-treated marmosets also spent a longer duration of time interacting with the intruder enclosure than the pairmate enclosure during control trials when the intruder was absent [t(7) = 2.7, p = 0.03, d = 1.56]. OT treatment also moderately impacted the frequency of aggressive interactions [F(3,21) = 2.4, p = 0.09, η² = 0.3; Table 3]. Marmosets tended to display aggression least frequently during control trials when the intruder was absent [p’s < 0.1].

Figure 3.

Mean (± SEM) (A) frequency and (B) and duration of time spent interacting with the pairmate enclosure and intruder enclosure are expressed as a function of OT treatment. Marmosets were treated with OT receptor agonist (OT), OT receptor antagonist (OTA), and saline control (SAL) during conditions when the intruder was present, and saline control (SAL – hatched bars) when the intruder was absent. Letters and asterisks indicate significant differences (a > b > c > d at p < 0.05, * at p < 0.05, ^# at p < 0.1). Data points from individual marmosets are displayed for each condition (♦; n=8).

Effects of pairmate presence on measures of sociality.

Pairmate presence, but not OT treatment or the interaction between pairmate presence and neuropeptide treatment, modulated additional measures of proximity, direct interaction, aggression, and contact vocalizations. Marmosets spent significantly more time per bout in proximity with the intruder enclosure than the pairmate enclosure in conditions when their pairmate was absent [t(7) = 2.4, p = 0.04, d = 1.01], but not when their pairmate was present [t(7) = 0.9, p > 0.05, d = 0.21; F(1,7) = 4.7, p = 0.06, η²= 0.4; Figure 4A]. Marmosets interacted with the intruder enclosure significantly more often than they interacted with the pairmate enclosure both when their pairmate was absent [t(7) 5.2, p = 0.001, d = 1.84] and when their pairmate was present [t(7) = 3.3, p = 0.01, d = 1.17], as indicated by interaction index scores. However, marmosets spent more time interacting with the intruder enclosure than the pairmate enclosure when their pairmate was absent, relative to when their pairmate was present [t(7) = 2.3, p = 0.05, d = 0.75; Figure 4B], as indicated by interaction index scores. Marmosets were also moderately quicker to display aggression, [F(1,7) = 4.6, p =0.06, η² = 0.4; Figure 4C], but did not display aggression more frequently [F(1,7) = 0.03, p > 0.05; Figure 4D], toward a same-sex intruder when their pairmate was present compared to when their pairmate was absent. Males and females did not differ in the expression of affiliative or aggressive behavior during the intruder challenge [p’s > 0.05]. Neuropeptide treatment, pairmate presence, or sex did not significantly change overall locomotion, piloerection, erh-erh vocalizations, alarm vocalizations, phee vocalizations, enclosure manipulation, scent marking behavior, or eating/drinking behavior, [p’s > 0.05].

Figure 4.

Mean (± SEM) values for social behaviors directed at the pairmate enclosure and the intruder enclosure are expressed as a function of pairmate presence. (A) Time spent per proximity bout, (B) interaction index, (C) latency to first aggression, and (D) aggression frequency, are presented. Interaction index (B) scores range from −1 to +1; where a positive score indicates that marmosets spent a greater proportion of their time directly interacting with the pairmate enclosure compared to the intruder enclosure and a negative score indicates that marmosets spent a greater proportion of their time directly interacting with the intruder enclosure compared to the pairmate enclosure. Letters indicate significant differences (a > b at p < 0.05, ^# at p < 0.1). Data points from individual marmosets are displayed for each condition (♦; n=8).

Effects of OT and pairmate presence on HPA-axis and HPG-axis activity.

Pairmate presence significantly influenced change in cortisol excretion from pre-to-post intruder exposure [F(1,6) = 6.4, p = 0.04, η² = 0.5; Figure 5A]. Marmosets that experienced an intruder challenge with their pairmate absent did not experience a significant increase in cortisol [t(7) = 4.1, p > 0.05, d = 0.41]. However, marmosets that experienced an intruder challenge with their pairmate present experienced a significant increase in urinary cortisol [t(7) = 4.6, p = 0.003, d = 0.90]. OT treatment also influenced change in cortisol excretion from pre-to-post intruder exposure [F(3,18) = 2.5, p > 0.05, η² = 0.3; Figure 5B]. Saline-treated marmosets did not exhibit a significant increase in cortisol from pre-to-post intruder exposure [t(7) = 2.7, p = 0.03, d = 0.20]. However, marmosets experienced a significant increase in cortisol excretion pre-to-post intruder exposure when they received an OTR agonist [t(7) = 3.6, p = 0.008, d = 0.75], an OTA [t(7) = 3.2, p = 0.02, d = 0.80], and during control trials when the intruder was absent [t(7) = 4.7, p = 0.002, d = 0.48].

Figure 5.

Mean (± SEM) cortisol and testosterone levels the morning of the intruder challenge (pre-exposure) and the morning after the intruder challenge (post-exposure). Data are expressed as a function of pairmate presence (A and C) and OT-treatment (B and D). Marmosets were treated with OT receptor agonist (OT), OT receptor antagonist (OTA), and saline control (SAL) during conditions when the intruder was present, and saline control (SAL – hatched bars) when the intruder was absent. Asterisks indicate significant differences at p < 0.05. Data points from individual marmosets are displayed for each condition (♦; n=8).

While pairmate presence did not affect change in testosterone excretion from pre-to-post intruder exposure [F(1,6) = 0.04, p > 0.05, η² = 0.01; Figure 5C], OT treatment significantly influenced change in testosterone excretion from pre-to-post intruder exposure [F(3,18) = 3.3, p = 0.04, η² = 0.4; Figure 5D]. Saline-treated marmosets exhibited a significant increase in testosterone from pre-to-post intruder exposure [t(7) = 2.7, p = 0.03, d = 0.51]. However, marmosets did not experience a significant increase in testosterone excretion pre-to-post intruder exposure when they received an OTR agonist [t(7) = 1.5, p > 0.05, d = 0.26], an OTA [t(7) = 1.7, p > 0.05, d = 0.30], or during control trials when the intruder was absent [t(7) = 0.05, p > 0.05, d = 0.01]. The interaction between pairmate presence and neuropeptide treatment did no significantly influence cortisol or testosterone excretion across the intruder challenges [p’s > 0.05].

Discussion

The behavioral response to a same-sex rival has important consequences for the integrity of an established relationship with a long-term pairmate in socially monogamous species. Here, we have shown for the first time that OT has an important role in regulating mate-guarding behavior in the socially monogamous marmoset, and appears to differentially modulate the expression of sociality in territorial and mate-guarding contexts. In conditions when their pairmate was absent, marmosets that received intranasal OT decreased time spent in proximity to a same-sex intruder and increased time spent in proximity to an empty pairmate enclosure, indicating that OT may enhance intruder avoidance in a territorial context. In conditions when their pairmate was present, marmosets that received intranasal OT increased time spent in proximity to a same-sex intruder and decreased time spent in proximity to their pairmate, indicating that OT may enhance intruder interest in a mate-guarding context. These results suggest that the OT system is sensitive to social context, and may alter adaptive responses to complex social situations. We also demonstrated that both the HPA- and HPG-axes are sensitive to pairmate presence and OT-system manipulation. Overall, these results indicate that the OT system and social context jointly influence behavioral and physiological responses to an intruder challenge in marmoset monkeys.

While the OT system did not directly influence the expression of aggressive behavior in either a mate-guarding or territorial context, pharmacological manipulations of the OT system influenced the expression of selective social interest during the intruder challenge. In conditions when their pairmate was absent, saline-treated marmosets spent more time in proximity with an intruder compared to an empty pairmate enclosure, which was anticipated since the intruder was the only social agent present. In conditions when both the intruder and pairmate were absent (non-intruder control trials), marmosets spent more time interacting with an empty intruder enclosure compared to an empty pairmate enclosure, which is also not necessarily surprising since: (1) the empty intruder closure was simply a new object that was in their environment, and (2) marmosets may have associated the intruder enclosure with previous intruder encounters as they expressed a non-zero amount of aggression toward the empty intruder enclosure, but less than following other treatment conditions. However, OT treatment eliminated the proximity preference to the intruder enclosure, suggesting that OT increases intruder avoidance and reduces territoriality by inhibiting selective social interest in the same-sex intruder in conditions when the pairmate was absent.

In contrast to an OT-enhanced intruder-avoidance response when their pairmate was absent, changes at the OTR facilitated intruder interest when their pairmate was present. While saline-treated marmosets spent equal time between the intruder and their pairmate when their pairmate was present, they spent more time in proximity to the same-sex stranger when treated with OT or OTA, suggesting that modifying endogenous activity levels at the OTR decreases intruder avoidance and increases selective social interest in and vigilance toward the same-sex intruder in conditions when their pairmate was present. While we expected that the OTR antagonist would have opposing effects to the OTR agonist, we found that both increasing bioavailable OT and antagonizing the OTR with an antagonist resulted in similar changes in behavioral expression, suggesting that increasing or decreasing normative oxytocinergic tone may result in similar behavioral outcomes. Thus, there appears to be a U-shaped relationship between oxytocinergic activity and vigilant responses toward a same-sex stranger during intruder challenges when a pairmate is present.

While the current results suggest that OTR antagonism impacts same-sex stranger-directed behavior, antagonism of the OTR has been previously shown to impact context-specific pairmate-directed behavior. Blocking endogenous OT activity with an OTA increased the magnitude of the physiological stress response and reduced proximity to a long-term mate both during a stressor and upon reunion (Cavanaugh et al., 2018, 2016), suggesting that the OT system may be an important regulator of partner-seeking behavior during stressful life events, as well as the reestablishment of normative levels of affiliation with a mate following a stressor as a means to buffer against stress. However, OTA administration does not alter pairmate- or stranger-directed behaviors during partner/stranger preference tests or prosocial food-sharing tasks (Cavanaugh et al., 2018, 2014; Mustoe et al., 2016, 2015). The current study is the first to examine whether OT-system modulation via OTR agonism/antagonism alters the complex behavioral repertoire of marmosets during encounters with same-sex strangers, both in the presence and absence of their long-term mate. Overall, the findings from the current study indicate that the OT system is at the intersection of context-specific selective social interest during intruder challenges and facilitates adaptive behavioral strategies to avoid potentially costly interactions with a same-sex stranger when alone, while encouraging selective social interest in a same-sex stranger when a long-term pairmate is present.

Much of the research on mate-guarding behavior has focused primarily on the role of aggression during conflicts with sexual or social competition. However, mate guarding is multifaceted and encompasses affiliative and reconciliation-like behaviors both during and following intruder encounters. Mate-guarding behavior is accomplished via two adaptive behavioral strategies: (1) proximity to and selective interest/aggression toward an intruder and (2) proximity and selective affiliation with a mate. During a laboratory simulation of an intruder in monogamous titi monkeys, OT and AVP were positively correlated with both agonistic displays and mate-directed affiliation (Fisher-Phelps et al., 2015). In the current study, OT treatment increased time spent in proximity to an intruder, suggesting that OT may increase selective interest in a same-sex intruder at the expense of interest in a pairmate. OT treatment not only facilitated increased time spent in the intruder zone, it also enhanced direct interactions with the intruder enclosure. Regardless of pairmate presence, OT-treated marmosets spent more time interacting with the same-sex intruder and less time interacting with their pairmate, compared to when they received saline. This provides further support for OT’s role in differentially shifting marmosets’ attention and interest toward a same-sex intruder during these encounters. In marmosets, OT is an important modulator of sociality across multiple social contexts, including offspring care and food provisioning (Finkenwirth et al., 2016; Saito and Nakamura, 2011; Taylor and French, 2015), interactions between bonded individuals (Cavanaugh et al., 2015; Smith et al., 2010; Snowdon et al., 2010), and interactions with opposite-sex strangers in the presence of a pairmate (Cavanaugh et al., 2014; Mustoe et al., 2015; c.f. Mustoe et al., 2016), suggesting that the OT system reflexively responds to social stimuli to promote appropriate behavioral responses contingent upon the milieu of the social context. The results of the current study suggest that OT increases selective social interest in a same-sex intruder during a mate-guarding context and decreases selective social interest in a same-sex intruder in a territorial context, thereby facilitating a behavioral strategy to promote vigilant responses during threats to the established bond and avoidance responses during territorial intrusions.

Marmosets’ differential behavioral responsiveness to an intruder challenge across social contexts and OT treatments lends itself to several possible interpretations: [1] OT enhances social vigilance by modifying selective social gaze and attention toward an intruder. OT generally promotes the expression of social gaze and attention in rhesus macaques and humans (Bartz et al., 2011; Chang et al., 2012; Guastella et al., 2008; Putnam et al., 2016), and attenuates vigilant responses to aversive facial expressions and dominant individuals in macaques (Ebitz et al., 2013; Parr et al., 2013). While the reproductive and social systems of marmosets and macaques are not analogous, both species selectively engage in social vigilance to reduce the occurrence of costly aggressive encounters (Treves, 2000). The results from the current study suggest that the intersection of social context and OT-system activity jointly regulate vigilant responses to a same-sex intruder in marmosets. [2] OT amplifies inherent in-group/out-group biases by modifying social recognition and selective aggression. Since OT is involved in the regulation of social recognition (Domes et al., 2007; Ferguson et al., 2001; Petrovic et al., 2008; Samuelsen and Meredith, 2011), OT may regulate the specific social distinction between a pairmate and a same-sex stranger. More likely, OT influences not only the distinction between a familiar in-group member (pairmate) and an unfamiliar out-group member (sexual rival), but also the behavioral response to compete. This effect supports and extends the research that suggests that OT strengthens inherent in-group/out-group biases (De Dreu, 2012; De Dreu et al., 2011; De Dreu and Kret, 2015). [3] OT decreases the expression of intruder interest when it is disadvantageous and increases the expression intruder interest when it is advantageous. OT may shift marmosets’ resource-protection priorities across social context. It may be disadvantageous for marmosets to aggressively engage a same-sex intruder alone due to the risk of injury/death when their most important ‘resource’ (i.e., pairmate) is absent, and advantageous for marmosets to aggressively engage a same-sex intruder when their pairmate is present as a means to prevent mate defection and genetic cuckoldry. Thus, in sum, OT appears to facilitate a behavioral strategy to avoid interactions with a sexual rival when a pairmate is absent and enhance vigilant responses toward a sexual rival when a pairmate is present.

In the current study, pairmate presence alone was a powerful modulator of social and physiological responses to an intrasexual intruder. While marmosets did not differ in the overall amount of aggression they exhibited across pairmate presence conditions, they were quicker to display aggression toward an intruder and displayed increased cortisol post-intruder exposure when their pairmate was present. Across all conditions, marmosets expressed a strong preference to spend time in the intruder zone and interact with the intruder compared to the pairmate zone, yet there were a few notable differences in these preferences across social context. During conditions when their pairmate was absent, marmosets also expressed a preference to spend time in proximity with an intruder compared to an empty pairmate enclosure; yet during conditions when their pairmate was present, marmosets showed no proximity preference. Marmosets spent more time interacting with the intruder regardless of pairmate presence; however, this preference was amplified during conditions when their pairmate was absent. Thus, marmosets have distinct behavioral repertoires to combat social challenges across territorial and mate-guarding contexts, as they displayed more social interest in the intruder when their pairmate was absent, but they were quicker to display aggression toward the intruder when their pairmate was present.

Much of the research on neuroendocrine responses during an intruder challenge has focused on the role of testosterone in the protection of resources and responsiveness to social challenges (e.g., intrasexual competition, territory establishment, mate guarding; aka ‘the challenge hypotheses’; Wingfield et al., 1990). In marmosets, the intensity of aggressive interactions with an intruder is positively associated with the magnitude of the post-encounter testosterone response in both males (Ross et al., 2004) and females (Ross and French, 2011) up to 24 hours later, which suggests that the degree of HPG responsiveness is conditional upon the intensity of the aggressive encounter. The current study found that testosterone levels were higher post-intruder exposure in control conditions. However, testosterone levels did not change following intruder exposure when marmosets received an OT treatment or following exposure to a control intrusion without an intruder present. While the challenge hypothesis suggests that increased testosterone levels facilitate future aggression, these results suggest that elevated testosterone is likely a consequence of aggressive responses during exposure of a same-sex intruder.

Some socially monogamous primates (e.g., marmosets, tamarins) engage in both monogamous and polyandrous mating strategies depending on a variety of environmental factors (Baker et al., 1993; Dietz and Baker, 1993; Digby, 1995). As a result, their behavioral repertoires and HPG-axis need to be flexible to adapt to different types of group composition and variable probability of intra- and extra-group competition. Specifically, dominant male tamarins in a polyandrous group had significantly higher androgen levels compared to unrelated subordinate males, but not related subordinate males (Bales et al., 2006). These differences in androgens based on group relatedness and mating strategy suggest that changes in androgens may regulate social cohesion among the group and competition toward non-kin. There are not observed differences in testosterone levels between males in monogamous groups and polyandrous groups in marmosets, although males in polyandrous groups copulate with the female significantly more than males in monogamous groups (Schaffner and French, 2004). Androgens also play an important role in mate-guarding behavior in primates that do not live in family groups. Male howler monkeys that live in unimale groups and have exclusive access to a female are much more likely to be challenged by extra-group males, and thus, are more likely to engage in mate-guarding behavior. Unlike in marmosets and tamarins, male howler monkeys living in male-female pairs have higher testosterone levels than males living in multi-male groups (Rangel-Negrín et al., 2011), and transiently increase their testosterone levels when solitary males are in close vicinity (Cristóbal-Azkarate et al., 2006). This suggests that testosterone secretion in males may be an anticipatory response to guard against reproductive conflict. Titi monkeys, which have a more rigid monogamous mating structure, that viewed their pairmate in close proximity to a sexual rival had higher levels of plasma testosterone compared to conditions when their viewed an opposite-sex stranger next to a sexual rival (Maninger et al., 2017), suggesting that HPG-axis activation is specific to challenges to the established bond. Overall, changes in androgen levels and behavior depend on both social status and group composition in species that engage varying mating strategies.

The HPA-axis appears to be particularly sensitive to encounters with a same-sex intruder in a mate-guarding context, as cortisol levels were higher following an intruder challenge only when their pairmate was present. Previous reports found no change in cortisol levels following intruder exposure in a mate-guarding context for male (Ross et al., 2004) or female (Ross and French, 2011) marmosets, however, these studies did not directly contrast pairmate presence with pairmate absence. The current results suggest that marmosets may perceive an intruder challenge as stressful or arousing in a mate-guarding context, but not necessarily so in a territorial context. In the socially monogamous titi monkey, subjects that gazed longer at their pairmate in close proximity to a sexual rival experienced greater HPA-axis activity compared to conditions when they viewed a stranger male-female pair. Further, titi monkeys showed increased neural activity in central areas of the social behavior network with abundant OT receptors (Freeman et al., 2014; Maninger et al., 2017), which indicates that OT activity within these social network hubs may be sensitive and responsive to threats to the established bond. There is burgeoning evidence that the OT system regulates the anxiolytic and stress-reducing effects of social buffering in primates (Cavanaugh et al., 2018, 2016; Crockford et al., 2017; Parker et al., 2005). Yet, in the current study, we found that OT treatment enhanced HPA-axis responsively to an intruder challenge. HPA-axis activity increased in response to an aggressive encounter with a same-sex intruder only when a pairmate was present. Since the amygdala is known to play a crucial role in fear and anxiety (Davis, 1992) and has a high density of OTRs in rodents and primates (Freeman and Young, 2016), exogenous OT administration may have enhanced arousal and/or a fearful response to novel social stimuli (i.e., same-sex intruder). Accordingly, treatment with OT may have eliminated the preference to interact with the same-sex intruder in favor of seeking out the pairmate during pairmate-absent conditions. During conditions where there was a challenge to the established bond (i.e., pairmate present during intruder challenge), treatment with OT enhanced interest in a same-sex intruder. Thus, exogenous administration of OT facilitated fidelity during challenges to the established bond by increasing social attention and interest in a sexual rival.

The intolerance and active discouragement of extra-pair sociosexual encounters between a pairmate and an intrasexual rival is essential to prevent genetic cuckoldry and maintain a monogamous bond with a pairmate. While OT treatment did not directly influence the expression of aggressive behavior, OT system manipulations influenced the expression of mate-guarding behavior by modulating selective social interest during an intruder challenge. The current study showed for the first time that OT treatment regulates this fidelity-promoting behavior by differentially shifting marmosets’ interest toward a same-sex intruder when their pairmate was present, and away from the intruder when their pairmate was absent. These results suggest that marmosets’ neuroendocrine systems are sensitive to the social milieu of challenges to their pairbond, that they utilize their flexible behavioral repertoires to respond to unexpected and varied challenges, and that the OT system appears to enhance adaptive responses to these social challenges. Moreover, these findings add to the converging evidence that the OT system regulates behavioral processes that underlie the preservation of established relationships.

Highlights.

• Mate-guarding behavior promotes the preservation of established relationships

• Marmosets were quicker to aggress toward a same-sex intruder when their mate was present

• OT reduced interest in a same-sex stranger in a territorial context

• OT increased interest in a same-sex stranger in a mate-guarding context

• OT system regulates adaptive responses to challenges to established bonds