The role of motivation and reward neural systems in vocal communication in songbirds

Abstract

Many vertebrates are highly motivated to communicate, suggesting that the consequences of communication may be rewarding. Past studies show that dopamine and opioids in the medial preoptic nucleus (mPOA) and ventral tegmental area (VTA) play distinct roles in motivation and reward. In songbirds, multiple lines of recent evidence indicate that the roles of dopamine and opioid activity in mPOA and VTA in male birdsong differ depending upon whether song is used to attract females (sexually-motivated) or is produced spontaneously (undirected). Evidence is reviewed supporting the hypotheses that 1) mPOA and VTA interact to influence the context in which a male sings, 2) distinct patterns of dopamine activity underlie the motivation to produce sexually-motivated and undirected song, 3) sexually-motivated communication is externally reinforced by opioids released as part of social interactions, and 4) undirected communication is facilitated and rewarded by immediate opioid release linked to the act of singing.

Vocal communication is central to successful behavioral interactions in multiple vertebrate species, including humans. To be socially successful an individual must adjust vocal behavior to match a particular social context, and what is appropriate in one context or for one individual may not be appropriate for another. For example, in humans cheering loudly at a sporting event is considered socially appropriate, but the same behavior at a funeral would be considered remarkably inappropriate. Behaviors considered socially appropriate can even differ between individuals in the same context, depending on characteristics of the individuals producing the behavior. For example, it is considered socially appropriate for an invited presenter to assume the podium and deliver a speech. The same behavior in the same context in an uninvited audience member would be considered highly inappropriate. Indeed, deficits in social communication are a hallmark of several mental disorders in humans, including depression, anxiety, and autism spectrum disorders, which are characterized by social withdrawal, inappropriate responses to social stimuli, and deficits in communication skills that can be context specific. Although vocal behavior is a fundamental part of social behavior in multiple vertebrate species (e.g., [35,46,47,130,172]), little is known about the neural mechanisms ensuring that individuals produce vocal behavior that is appropriate within a given social context.

Songbirds are well known among vertebrates for vocalizing at high rates within multiple distinct social contexts. This suggests that they are highly motivated to communicate and raises the possibility that the consequences of vocal production are rewarding. Motivation, (i.e., a state of “wanting” or “anticipation” leading to the pursuit of a particular stimulus [28,141]) is closely linked yet distinct from reward. If a behavior such as vocal communication is associated with reward, an individual is more likely to repeat this behavior. If the reward associated with vocal communication is high, this may be reflected in high rates of vocal production even in the face of obstacles (e.g., predators), consistent with a high level of motivation [28,161,183]. Reward associated with the production of particular behaviors strongly motivates animals to engage in activities that are highly adaptive and critical for reproductive success, such as feeding, social or sexual behavior (e.g., [3,30,56,141,193]). Recent studies in songbirds indicate that highly evolutionarily conserved neural systems involved in motivation and reward also serve to influence vocal communication in a context-specific manner. In this review, I will 1) introduce songbirds as powerful study species in which to address questions on the neural basis of the motivation to communicate, 2) provide a brief history of research on the motivation to communicate in songbirds, 3) review data highlighting distinct roles for select brain regions implicated in motivation and reward in song produced in distinct social contexts, 4) describe data highlighting possible context-specific roles for dopamine and opioids in the motivation to communicate, and 5) build the case that the social factors and neural substrates rewarding vocal production differ depending upon the social context in which an animal is communicating.

The motivation to sing differs across individuals and social contexts

Songbirds produce song at specific points during development and within multiple distinct social contexts as adults, including during song learning and practice, when in large affiliative flocks, when singing to attract a mate or to repel a competitor [47] and even, in some species, in response to predators [80]. The factors motivating a bird to sing can differ depending upon the function of song, the social context in which singing occurs, and factors such as endocrine state, social status and the resources available to an individual.

A primary function of song in male songbirds is mate attraction. Seasonally and opportunistically breeding males with elevated testosterone (T) concentrations often respond to the introduction of a female conspecific with high rates of singing behavior [16,47,202]. In some species, a general decrease is observed in song output after a male attracts and forms a breeding pair with a female (e.g., European starlings [51,61]). Males often resume singing at high rates immediately prior to each copulation [59,62,150] or upon mate removal [51], indicating that singing behavior in this context is a reflection of male sexual interest. In this review I consider song directed towards or produced in response to females by males with high T concentrations to be highly sexually motivated.

In many male songbirds with elevated T concentrations another primary function of song is territorial defense [47]. When presented with male conspecifics, many male songbirds also respond with high rates of song production [14,81,182]. Nesting locations broadcasting male song also tend to be avoided by male conspecifics [132,136]. These observations suggest that male-directed song in some contexts can be considered agonistically motivated.

Additional factors such as an individual’s social status and resource possession also strongly influence the production of sexually- and agonistically-motivated song. For example, male starlings that successfully acquire and defend nesting sites or territories (i.e., socially dominant individuals) respond to female conspecifics with substantially higher rates of song than males that fail to acquire nesting territories [81,155,166]. Thus, even within a breeding context vocal responses that are socially appropriate for one individual may differ from responses that are appropriate for another. Specifically, in contrast to a socially-dominant male with a nesting territory, it can be considered socially inappropriate for a low status male without a territory to produce high rates of sexually-motivated song, if without a nest site he cannot effectively initiate breeding.

Male songbirds also sing high rates of spontaneous, non-sexually motivated song that is described as undirected in studies of zebra finches [58,98,207]. I use this term here although some of the specific details related to undirected song described for zebra finches may differ in other species. Undirected song in adults is not clearly produced in response to or directed towards a specific individual. It is generally ignored by potential recipients and has no obvious, immediate effect on the behavior of individual conspecifics (i.e., it does not immediately function to attract female mates or repel male competitors [58,207]). In seasonally breeding birds, undirected song is common outside of the breeding season when T concentrations are low and males are singing as part of large communal flocks (e.g., [155,207]). Opportunistically breeding songbirds, such as male zebra finches, also sing high rates of undirected song while in large affiliative flocks [207]. In seasonal breeders, males in such flocks outside of the breeding season do not sing at high rates in response to the introduction of a female (e.g., European starlings [155]), and song does not immediately attract a female, although it is certainly possible that females evaluate males at this time. Undirected song may be important for the maintenance of social contact, dominance rank or song practice [63,184,207], yet exactly what motivates birds to produced undirected song is not clear.

One point worth addressing is that song, in some instances, can be difficult to categorize as undirected or directed. For example, a male that loses his mate may increase singing behavior even when alone, a type of singing behavior that has been referred to as undirected in zebra finches [58]; however here I would categorize this type of singing as directed given that it is designed to immediately influence conspecific behavior (i.e., attract a female) and would be reduced should the male successfully attract a female. In this review, a key feature of undirected song is that it is not strongly influenced in an immediate sense by the behavior of individual conspecifics. For example, while undirected singing may attract conspecifics to a flock, the addition of another bird to a flock does not rapidly result in increased or decreased rates of individual song production [86,155]. Therefore this type of song in this review would be categorized as undirected.

The mechanisms that reward song may also differ context-dependently

Differences in the social functions of conspecific-directed (sexually- and agonistically- motivated song) and undirected song raise the possibility that the factors rewarding song in these contexts differ as well. Given that directed singing behavior functions to attract female mates and to repel male competitors, directed song may primarily be externally-reinforced by the behavioral responses of conspecifics. For example, sexually-motivated singing behavior may be positively reinforced (i.e., rewarded) through the successful attraction of and subsequent copulation with a female. In contrast, song used to repel a male (agonistically-motivated song) may be negatively reinforced (i.e. strengthened by the removal of a negative stimulus) through the successful repulsion of a rival. Unlike sexually- or agonistically-motivated song, undirected song is not immediately followed by any obvious, immediate form of externally-mediated reinforcement, yet males continue to sing at high rates. It is thus possible that rather than being reinforced by conspecific behavioral responses, undirected communication is strongly linked to intrinsic reward (a possibility supported by place preference data reviewed later in this review). Specifically a positive affective state may facilitate undirected singing behavior and/or the act of song production itself may be intrinsically rewarding. Recent studies (reviewed next) linking motivational neural systems to brain regions devoted to song production offer strong support for the hypothesis that the role of motivation and reward systems differs depending upon whether song is directed or undirected.

Studies on the neural control of song set the stage for understanding the motivation to sing

In the early 1970’s the neural centers controlling song production had yet to be discovered, and the neural regulation of motivation was a burgeoning field of research. Early findings in songbirds indicated that vocal behavior was elicited by electrical stimulation of midbrain, tegmental, and integrative diencephalic brain regions. A few researchers suggested that these highly evolutionarily conserved systems were influencing vocal behavior through effects on motivation [38,65]. The early suggestion that motivation neural centers might act to influence vocal output in songbirds was overshadowed by the excitement generated by the discovery of the song control system in the mid 1970’s by Fernando Nottebohm and colleagues [135]. Nottebohm found that songbirds possess a specialized group of interconnected brain nuclei devoted to song, referred to as the song control system ([135]; Fig. 1). Since this discovery a great deal of progress has been made in elucidating the specific functions of nuclei of the song control system in aspects of song such as learning and sensorimotor processing (for recent reviews see [17,36,37,72,134,186,200]); however, until recently almost nothing was known about the mechanisms that activate this system or motivate birds to sing at a neurobiological level. Activity within the song control system can differ depending upon the motivational context in which vocalizations are produced [84,91,98,157]. For example, neuronal firing in area X and the lateral portions of the magnocellular nucleus of the anterior nidopallium (LMAN), and gene expression in area X, LMAN, and the robust nucleus of the arcopallium (RA) are higher when a male sings alone (undirected song) compared to when a male directs song towards a conspecific [91,98]. However there is no evidence to implicate this system in reward or motivational aspects of singing behavior. For example, male songbirds with lesions to the song control nucleus HVC (used as a name) continue to assume a singing posture and display motor aspects of song production but fail to produce audible song [135]. Males with lesions to area X or LMAN (regions involved in song learning, maintenance, and structural adjustment [32,121,135]), similarly continue to sing at high rates post lesioning [32,135]. These examples indicate that even with portions of the song control system damaged, birds are motivated to sing. Thus, the neural control of the motivation to sing must lie outside of the song control system. Now after more than 30 years, renewed interest in the motivation to sing has led researchers back to the idea that highly evolutionarily conserved brain areas involved in motivation and reward interact with the song control system to regulate the motivation to communicate.

Figure 1.

Sagittal schematic highlighting select neuroanatomical routes by which mPOA and VTA interact and access regions involved in song production. VTA projects directly to HVC and the robust nucleus of the arcopallium (RA), song control regions that regulate structural aspects of song [67] and to Area X, which is involved in song learning and song variability [121,180]. mPOA sends one known direct projection to the song system (to the dorsomedial portion of nucleus intercollicularis; DM an area involved in vocal output [170]) and can influence the song system indirectly through VTA as well as through the locus coeruleus (LoC) and the midbrain periaqueductal gray (PAG), which also project directly to HVC, RA, and area X. Other abbreviations: NIf = nucleus interfacialis; Uva = nucleus uvaeformis; nXIIts = 12^th cranial nerve. See text for additional details and references.

mPOA and VTA underlie motivation, reward and sexually-motivated song

Since the late 1990’s, when interest in the neural basis of the motivation to sing was rekindled, two brain areas have been a primary focus of research; the medial preoptic nucleus (mPOA) and ventral tegmental area (VTA; Fig. 1). mPOA and VTA are core components of distinct yet interconnected neural systems underlying motivation and reward. VTA is part of the mesolimbic system, which is implicated in motivation associated with multiple behaviors such as feeding, sexual behavior, and the use of drugs of abuse [29,95,102,141,162,191]. mPOA is part of the incertohypothalamic system and has been well studied for its role in male sexual motivation [20,21,50,97,109,111,153,190]. These two systems are interconnected, including via a reciprocal connection between mPOA and VTA ([19,156,174,176]; Fig. 1). The nucleus accumbens is another component of the mesolimbic system that is strongly implicated in reward in studies of mammals (e.g., [43,92,103]). The role of this region in singing behavior however has not been extensively investigated, but recently, at least one study in zebra finches implicated the avian nucleus accumbens in the production of affiliative behaviors during interactions with opposite sex conspecifics [8]. Its role in singing behavior however has yet to be determined. Thus this review will focus primarily on studies of mPOA and VTA.

Several studies implicate mPOA and VTA in the regulation of sexually-motivated male song. For example, attributes of mPOA change seasonally in association with changes in the production of sexually-motivated song. Specifically, in male starlings mPOA is largest and densely innervated with aromatase (the enzyme converting testosterone into estradiol) during compared to outside the breeding season [149,155]. In the breeding season mPOA was also largest in males with nesting sites who produced the highest rates of sexually-motivated song, compared to those without nest sites [155], and song bout length (a feature of song attractive to females [60,71]) correlated positively with the volume of mPOA [155]. Immunolabeling for immediate early gene products (used as indirect markers of neuronal activity) revealed positive correlations between the numbers of labeled cells in mPOA and sexually-motivated but not non-sexually motivated song in both male starlings (Fig. 2A,B) and house sparrows [84,157]. This link between activity in mPOA and sexually-motivated singing behavior was verified by a series of lesion studies. Male starlings with lesions to mPOA consistently sang fewer and shorter songs in response to females compared to mPOA intact males [6,7,154]. Female starlings prefer males singing long songs [60,71]. Thus these findings suggest that an intact mPOA is required to instill song with features attractive to females, a possibility supported by the finding that males with mPOA lesions were approached less frequently by females [7]. The role of mPOA in sexually-motivated vocal production appears to be well conserved across species. For example mPOA has been linked to courtship vocalizations in fish [68,77], hamsters [69], and mice [177].

Figure 2.

Composite figure highlighting evidence for context-specific roles for mPOA and VTA in song in studies of male starlings. A–D) correlations between undirected (A and C) and sexually-motivated (B and D) singing behavior and numbers of immediate early gene immunolabeled cells in mPOA (A and B) and VTA (C and D). Each point represents data from a single male. Red regression lines indicate significant correlations (p < 0.05). E–G) Bar graphs illustrating that the effects of mPOA lesions on song differ depending on whether song is undirected in the nonbreeding season (E), nonsexually-motivated within the breeding season (i.e., produced away from a nest site; F) or sexually-motivated (produced from a nest site within the breeding season); G). Open bars indicate data from control-lesioned males. Filled bars indicate data from mPOA-lesioned males. Sample sizes are indicated above standard error bars). * indicates p < 0.05. H–I) Plots illustrating the finding that lesions to mPOA disrupt statistical clustering of activity in song control and social brain regions. Illustration of the results of a principle components analysis on effects of mPOA lesions on patterns of activity (identified indirectly through counts of immediate early gene [IEG] labeled cells, FOS and ZENK) among brain regions implicated in song (light blue points) and areas involved in motivation and social behavior (red points) in male starlings presented with a female under breeding conditions. H) Projection of IEG-labeling on the factor plane for PC1 and PC2 for control animals. I) Projection of IEG-labeling on the factor plane for PC1 and PC2 for mPOA-lesioned animals. Abbreviations: LHy = lateral hypothalamus; VMH = ventromedial nucleus of the hypothalamus; see Fig. 1 for additional abbreviations. Figures redrawn from Heimovics and Riters, 2005; Alger and Riters, 2006; Alger et al., 2009.

Several studies also link activity in the ventral tegmental area (VTA) to sexually-motivated male song production. Specifically, studies using labeling for immediate early genes to indirectly identify neuronal activity revealed positive correlations between immunolabeling in VTA and female-directed but not undirected song in male starlings ([84]; Fig. 2C,D), and numbers of cells double-labeled for Fos and tyrosine hydroxylase correlated with courtship singing in male zebra finches [79]. Electrophysiological recordings and immediate early gene expression data also link activity in VTA in male zebra finches to production of sexually-motivated, female-directed song [82,93,205].

Data link activity in VTA to highly motivated, directed vocal behavior across social contexts and species. For example, labeling for the immediate early gene Fos in VTA correlated positively with agonistically-motivated song directed towards another male conspecific in song sparrows [125]. Furthermore in rats lesions to VTA selectively reduced vocalizations produced in anticipation of play behavior (which may be considered to be directed); whereas electrical stimulation of VTA triggered these vocalizations [42].

mPOA and VTA may both contribute to undirected song

Although mPOA is well known for its role in male sexual motivation [20,21,50,97,109,111,153,190], several studies indicate that role of this region in song extends beyond sexually-motivated contexts. For example, the densities of labeling for the opioid peptide met-enkephalin, as well as D1 dopamine receptors in mPOA correlated positively with undirected song produced by male starlings singing in overwintering flocks (reviewed in more detail below [87,158]). Furthermore, although lesions to mPOA nearly abolish sexually-motivated song [6,7,154], lesioned males produce excessive rates of song in non-sexually motivated contexts (e.g., when singing away from nest territories [6]; Fig. 2E–G). These findings indicate that mPOA stimulates sexually-motivated song but plays an inhibitory role in undirected or non-sexually-motivated song. These findings suggest that the POM may provide contextual input to song control regions and this interaction could be what ensures that song occurs in an appropriate context and in response to appropriate stimuli.

Data across species also implicate mPOA in vocal behavior in non-sexually-motivated contexts. For example, electrical stimulation of mPOA stimulates vocal behaviors not typically observed in sexual contexts in non-songbird species, such as domestic chicks (distress peeps) and Japanese quail (vocalizations not similar to crowing) [10,171]. In rats and Guinea pigs mPOA also contributes to the generation of alarm or distress calls [39,90], supporting the idea that the role of mPOA in vocal production extends beyond the context of sexual behavior.

Few studies have examined VTA involvement in undirected song. In studies using electrophysiological recording and dopamine receptor/immediate early gene double labeling to examine neuronal activity in VTA associated with directed and undirected singing, a small increase in dopamine neuronal activity is associated with the production of undirected song [118,205]. Additionally, in one of the major dopaminergic efferent projections of VTA (area X; Fig. 1) a small increase in levels of the neurotransmitter dopamine was observed in association with undirected song [167], perhaps reflecting dopamine input from VTA.

In sum, overall activity in both mPOA and VTA tracks the extent to which song is triggered by an external motivating factor of high incentive value (e.g., a female). The data suggest the hypothesis that these regions provide information about an individual’s motivational state to song control regions to trigger a song that is structurally appropriate within a given social context.

mPOA and VTA are well-positioned to play integrative roles in song production

Results of tract tracing studies reveal that mPOA and VTA are well-positioned neuroanatomically to integrate information about an individual’s internal state with external factors and to pass this information along to the song control system ([156]; Fig. 1). In songbirds, VTA sends direct projections to HVC, RA, and area X ([11,12,33,45,122]; Fig. 1). mPOA projects directly to at least one song control region, the dorsomedial portion of the nucleus intercollicularis (Fig. 1), which is involved in vocal output in a number of species [38,170,199]. mPOA can additionally influence the song control system indirectly through VTA or other areas found to project directly to the song control system, including the locus coeruleus and mesencephalic periaqueductal gray; [11,12,33,45,122]; Fig. 1).

Both mPOA and VTA are reciprocally connected to multiple steroid-sensitive brain regions implicated in social and sexual behavior, including regions proposed to be components of an interconnected, steroid-sensitive “social behavior network”, underlying all social behaviors [78,133]. These regions include the lateral septum, bed nucleus of the stria terminalis, nucleus taeniae of the amygdala [homologue of medial amygdala in mammals], ventromedial nucleus of the hypothalamus, and mesencephalic periaqueductal gray [1,18,19,108,129,156]. These anatomical connections suggest central, integrative roles for both mPOA and VTA in singing behavior. This idea is supported by a study in male starlings showing that lesions to mPOA disrupt relationships between sexually-motivated song and patterns of immediate early gene activity within song control, limbic, hypothalamic, and midbrain regions (Fig. 2H, I [7]).

Dopamine and opioids play distinct roles in the regulation of motivated behavior

Motivation and reward are distinct components of behavior, and studies demonstrate that they are regulated by distinct neurochemicals acting within mesolimbic and incertohypothalamic brain regions. Mesolimbic dopaminergic projections originating in VTA play a central role in motivation and reward associated with multiple behaviors [29,141,169,203]. Recently studies indicate that rather than underlying the experience of hedonic pleasure or reward, dopamine underlies the anticipatory, motivated state that energizes animals to engage in incentive/reward-directed behaviors such as the pursuit of food, a mate, or a drug [26,27,29–31,96,102,113,131,139,141,145,164,190]. Dopamine in mPOA of the incertohypothalamic system also selectively underlies the motivational components of male sexual behavior, as opposed to the rewarding properties [3,96,109,111].

Other neurochemicals, including opioid neuropeptides, are proposed to act within both mesolimbic and incertohypothalamic brain regions to regulate the reward or pleasure that occurs once an animal comes into contact with a rewarding stimulus [3,41,101,143]. Activation of both mu and delta opioid receptors by endorphin and enkephalin opioids in specific brain regions induces reward; whereas activation of kappa receptors triggers a negative affective state [25,119,195]. Rats will work (i.e., press a lever) to receive injections of mu and delta opioids into VTA, indicating that opioid release in this region is rewarding [34,55,196,206]. Infusion of the opioid met-enkephalin into mPOA in rats results in a preference for a place associated with this treatment (a conditioned place preference), indicating that opioid release in mPOA is also rewarding [4,5].

The working hypothesis proposed here based on these studies is that the factors motivating and rewarding song differ depending upon the context in which an individual is singing. Dopamine may regulate the motivation to sing within any context, however given that sexually-motivated song is highly incentive/reward-directed, dopamine acting within mPOA, VTA or its projection regions is expected play a central role in this type of singing behavior. Opioids are expected to influence both directed and undirected song; however if the hypothesis that undirected song is more highly dependent upon immediate intrinsic reward than directed song is correct (as proposed in the introduction and further supported in a study on place preference described below), I predict that undirected song will be facilitated and rewarded by opioid release in mPOA and VTA occurring either just prior to or as an immediate consequence of the act of singing. In contrast, I predict that song directed towards a female will be rewarded by opioids released as a consequence of mate attraction and copulation. I next review studies that support these predictions.

Dopamine and opioids play distinct roles in sexually-motivated singing behavior

Several pharmacology studies indicate that dopamine can act to facilitate sexually-motivated song. Peripheral administration of a dopamine uptake inhibitor (GBR-12909) stimulated, whereas dopamine receptor antagonists (a D1/D2 antagonist cis-flupenthixol and a D1 antagonist SCH-23390) inhibited sexually-motivated, female-directed song in male starlings (Fig. 3A, B) and zebra finches [151,168]. In contrast to dopamine manipulations, data suggest that opioids can inhibit sexually-motivated song. Specifically, peripheral injections of the mu opioid receptor agonist fentanyl dramatically reduced sexually-motivated singing behavior in male starlings, whereas the non-selective antagonist naloxone reliably enhanced sexually-motivated male singing behavior (Fig. 3C, D [158,168]). These manipulations did not similarly affect feeding or drinking behaviors, indicating that they were not likely caused by general motor effects on behavior. These findings were interpreted as consistent with past data showing that opioid release in mPOA leads to reward [4,5] along with a temporary inhibition of sexually-motivated behavior [94,114,189,190] (i.e., sexual satiety) [128,144,148,163]. However, in contrast to these data a study in male zebra finches found that at a low dose (but not at two additional higher doses) peripheral injections of the opioid receptor antagonist naloxone reduced female-directed song [105]. The differences in the effects of naloxone on singing behavior in male starlings and zebra finches may in part reflect the inability to control the sites at which peripherally administered opioid pharmacological agents act. In male rats, opioids in mPOA inhibit male sexual behavior, but opioids in VTA have been shown to increase sexually motivated behavior [191], presumably by removing GABA inhibition of VTA neurons [54,100,191]. Individual and environmental factors also influence the impact of opioid receptor manipulations on sexually-motivated behaviors. For example, in male starlings, the same dose of naloxone uniformly increased singing behavior in males initially singing at low rates but had little to no effect on song production in males singing at high rates [159]. Compared to male starlings singing relatively high rates of sexually-motivated song, males singing at low rates have been found to have higher densities of mu opioid receptor labeling in mPOA, VTA and additional regions [104]. These findings suggest that differences in behavioral responses to opioids may reflect individual differences in substrate sensitivity to opioids. Furthermore, effects of peripherally administered opioids on sexual behavior in a novel environment in male rats have been found to vary depending upon an individual’s past sexual experience [146]. Although peripheral pharmacological manipulations of opioid receptors in songbirds demonstrate that opioids influence sexually-motivated singing behavior, differences observed in individuals and across studies indicate that future work must be conducted to identify the relevant opioid neural targets, receptor subtype densities and distributions, as well as individual and environmental factors that contribute to sexually-motivated song production.

Figure 3.

Data illustrating a stimulatory role for dopamine and an inhibitory role for opioids in sexually-motivated song. Mean measures of song produced by male starlings after peripherally administered vehicle treatment (open bars) and drug treatment (dark bars) with A) the D1 dopamine receptor antagonist SCH 23390, B) the dopamine reuptake inhibitor GBR-12909, C) the opioid receptor antagonist naloxone, and D) the mu opioid receptor agonist fentanyl. *Indicates p < 0.05. See text for additional detail. Figures redrawn from Riters et al., 2005 and Schroeder and Riters, 2006.

Dopamine markers in VTA are tightly coupled to sexually-motivated song

Although site-specific pharmacological manipulations have yet to be performed, several lines of research highlight dopamine activity in VTA as central for sexually-motivated male singing behavior. Specifically, studies in starlings and zebra finches show positive correlations between dopamine markers in VTA and sexually-motivated but not undirected song [9,79,86,87,89] (These measures include concentrations of dopamine and dopamine metabolites measured using high pressure liquid chromatography [HPLC], labeling for tyrosine hydroxylase [TH] and immediate early gene/TH double labeling; [e.g., Fig. 4A–D].). Synaptic potentiation of dopamine neurons in VTA was also identified in association with sexually-motivated but not undirected song in zebra finches [93]. Furthermore, several measures of dopamine in area X indicate that dopamine in this VTA projection region is tightly linked to sexually-motivated singing behavior. For example, dopamine concentrations in area X measured using HPLC correlate positively with sexually-motivated but not non-sexually motivated song in male starlings [89]. Dopamine levels in area X also increase to a greater extent during female-directed, sexually-motivated song compared to undirected song in male zebra finches [167]. Patterns of D1 and D2 receptor colocalization within neurons expressing the immediate early gene egr-1 are consistent with the findings showing dopamine release to increase in area X in association with female-directed song in zebra finches [118]. Finally, pharmacological blockade of dopamine D1 receptors in area X disrupts context-dependent changes in song variability in male zebra finches singing female-directed compared to undirected song [121].

Figure 4.

Evidence that dopamine in mPOA and VTA is tightly linked to female-directed song. Plots showing correlations between non-sexually-motivated (left) and sexually-motivated (right) singing behavior and measures of A – B) dopamine and C – D) tyrosine hydroxylase in VTA; and E – F) tyrosine hydroxylase and G – H) D1 dopamine receptor densities in mPOA. Each point represents one individual. Presence of regression line indicates significant relationships, p < 0.05. See text for interpretation. Figures redrawn from Heimovics and Riters, 2008; Heimovics et al., 2009; 2011.

Although data to date support a facilitating for dopamine acting within VTA and area X in sexually-motivated male song, multiple studies demonstrate that dopamine can have diverse (even opposing) effects on motivated behavior that depend upon the target, the specific receptor subtypes activated, and interactions with other neurochemicals, such as glutamate or GABA (e.g., [66,100]). This functional diversity must be considered in future studies examining dopamine activity in specific regions, at specific receptor subtypes, and the contributions of additional neurochemical systems to dopamine function. Based on studies in songbirds to date however, it is clear that dopamine activity at least at some specific receptor subtypes in VTA and in some of VTA’s projection regions facilitates highly-motivated, directed vocal behavior.

The findings in songbirds are supported by research in rats, in which dopamine-neuron specific lesions of VTA and dopamine receptor D1/D2 antagonist injections in VTA selectively reduced 50 kHz vocalizations [42], vocalizations that reflect the anticipation or seeking of reward, including social reward [112]. Furthermore, dopamine agonist administration into the nucleus accumbens (another region receiving dopaminergic input from VTA, well studied for its involvement in reward) in rats triggered the same vocalizations [40]. Taken together these data highlight dopamine activity in VTA and some of its projections to the striatum (i.e., area X and nucleus accumbens) as central to directed, incentive-motivated vocal communication.

Dopamine markers in mPOA are also tightly coupled to sexually-motivated song

Dopamine markers in mPOA are also linked to sexually-motivated male singing behavior. Specifically, labeling for TH fibers and cells in mPOA correlated negatively with sexually-motivated but not undirected song in male starlings (Fig. 4E, F [86]). Densities of autoradiographically labeled D1 dopamine receptors in mPOA also correlated negatively with sexually-motivated singing behavior in male starlings (Fig. 4H [87]). Past studies in rats and Japanese quail indicate that dopamine in mPOA is elevated in sexually-motivated males [96,110] and that activity of D1 and D2 receptors in mPOA facilitates sexually-motivated behavior [109,131]. Based on these findings we speculated that the low densities of TH and D1 receptors found in association with the production of sexually-motivated song may reflect high TH use (resulting in depletion) and down regulation of D1 receptors (due to high dopamine release). However, studies do not uniformly demonstrate a facilitating role for dopamine in male sexually-motivated behavior. For example, in male Japanese quail central administration of dopamine itself (rather than D1 or D2 receptor subtype selective agonists) inhibits copulatory behavior [49]. Furthermore, a strong body of work demonstrates that dopamine release in mPOA acts primarily at noradrenergic receptors, rather than through D1 or D2 receptors [48]. Dopamine very well may act at dopamine receptors in mPOA to facilitate sexually-motivated singing behavior; however, this must be tested in future studies involving detailed site-directed measures and manipulations of dopamine, dopamine receptors, and exploration of dopamine effects at noradrenergic receptors.

Opioids are not tightly coupled to sexually-motivated singing behavior

In the few studies examining links between opioid markers in mPOA and VTA and sexually-motivated singing behavior, it does not appear that opioids are linked linearly to rates of song production in this context. Specifically, in male starlings immunolabeling measures for the opioid met-enkephalin in neither mPOA nor VTA correlated with sexually-motivated singing behavior (although a trend was observed for a positive relationship in VTA; Fig. 5B, E [158]).

Figure 5.

Evidence that opioids in mPOA and possibly VTA are most closely related to undirected song. Scatterplots illustrating relationships between male song production and mENK fiber density in mPOA (A, B) and VTA (D, E). Each dot represents data from a single male. The red regression line indicates a significant relationship in mPOA for males singing undirected song, p < 0.05. For VTA this relationship was shy of significance (p = 0.06). See text for additional detail. C, F) Darkfield photomicrographs illustrating mENK fiber density in mPOA and VTA of a male that did not sing and a male that sang at high rates. Arrows point to the boundaries of mPOA and VTA. TSM = tractus septomesencephalicus, v = ventricle, nIII = 3rd cranial nerve. G, H) Bar graph showing densities of immunolabeled mu opioid receptors in male starlings that sang high rates of sexually-motivated song (those with nest boxes indicated by dark bars) and males who did not (those without nest boxes indicated in white bars). Asterisk indicates significant difference, p < 0.05. Figures adapted from Riters et al., 2005 and Kelm et al., 2011.

We take these findings to be consistent with the working hypothesis that undirected but not directed song is facilitated and maintained by intrinsic reward mediated by immediate opioid release. However we do not take these findings to suggest that opioids do not play a role in rewarding sexually-motivated singing behavior. Rather we propose that opioid release in mPOA rewards both types of singing behavior, but that it is the social consequences of sexually-motivated singing behavior (i.e., female proximity or copulation) that result in a large surge of opioid release, which rewards and temporarily inhibits sexually-motivated song. Indeed, in male starlings measures of immunolabeled mu opioid receptor densities in both mPOA and VTA were significantly lower in males with nest boxes (singing sexually-motivated song at high rates) compared to those without nest boxes (singing sexually-motivated song at low rates; [104]; Fig. 5G, H). Given that opioids in mPOA inhibit sexually-motivated behavior (e.g., [94,114,189,190]), the lower densities of mu opioid receptors in mPOA in males with nest boxes may facilitate sexually-motivated singing behavior by reducing sensitivity of this region to the inhibitory effects of opioids. Interpretation of the same finding for VTA is difficult (given that opioids in VTA can lead to stimulation of dopamine release and male sexual behavior, reviewed above).

Altogether the studies reviewed above suggest that dopamine is tightly linked to sexually-motivated song and highlight VTA and mPOA as sites of dopamine action. Data suggest that opioids are not as linearly linked to sexually-motivated song as dopamine and that opioids may inhibit this type of singing behavior (although there is at least one exception in zebra finches). These findings generally map onto findings for other male sexual behaviors and suggest the hypothesis that dopamine acting in part within mPOA and VTA motivates males to produce sexually-motivated song. In contrast, opioids in mPOA may serve to reward and temporarily inhibit male song, once a male successfully attracts and copulates with a female.

Dopamine and opioids in mPOA and VTA may both facilitate undirected singing behavior

The neural mechanisms underlying undirected singing behavior have not been well characterized. Effects of peripheral pharmacological manipulations of dopamine receptors on undirected singing behavior have not been examined in songbirds; however treatment of domestic chicks with the non-selective dopamine agonist apomorphine increased production of general vocal behaviors [53], suggesting a stimulatory role for dopamine in undirected vocal behavior. With respect to opioids, treating male zebra finches with the non-specific antagonist naloxone resulted in a linear dose-dependent suppression of undirected male song [105]. This suggests that opioids may act to stimulate undirected singing behavior.

Results of a few studies suggest a role for dopamine, possibly acting in mPOA or in the striatal VTA projection region, area X in undirected singing behavior. For example, in male starlings a strong positive correlation was identified between undirected singing behavior and the densities of D1 dopamine receptors in mPOA ([87]; Fig. 4G). Although several studies fail to identify correlations between dopamine markers in VTA and undirected song (e.g., [86,87,89]; Fig. 4A,C), in male zebra finches immediate early gene expression occurs in dopamine receptor containing neurons in association with both undirected as well as directed song in area X [118], and dopamine measured using microdialysis in area X rises in association with the initiation of undirected song, albeit not to the same levels observed in association with female-directed song [167].

The regions in which opioids act to influence the motivation to sing are not known; however, past work suggests a role for opioids in mPOA and possibly VTA in undirected male song. Specifically, in male starlings densities of immunolabeled met-enkephalin opioid fibers in mPOA correlated positively with undirected song (but not sexually-motivated song), with a similar trend observed for VTA (Fig. 5A–F [158]). Measures of immunolabeled mu-opioid receptors were also higher in mPOA and VTA of males without nest boxes compared to those with nest boxes ([104] Fig. 5G,H). Males without nest boxes do not sing in response to females but they do sing [155], suggesting song in these males may be undirected. If this assumption is correct, then these data indicate a link between high mu opioid receptor densities and undirected song. Given that opioids in mPOA generally act to inhibit sexual behaviors [94,114,127,189,190], it may be that the high opioid receptor density in males without nest boxes renders mPOA more sensitive to opioids, thereby inhibiting sexually-motivated song at a time when this behavior may be inappropriate (i.e., in the absence of a nesting location). Additionally, given that opioid release in both mPOA and VTA is rewarding (at least in mammals, [4,5,34,55,196,206]), it is also possible that opioid-mediated reward in these areas promotes the production of undirected singing behavior.

Measuring song-associated reward

This review is written with the underlying assumption that the motivation to sing is strongly shaped by reward. Specifically it is proposed that singing behavior can be either externally reinforced through its consequences on the behavioral responses of others (e.g., sexually-motivated, female-directed song) or that the act of singing itself is intrinsically rewarding or strongly facilitated by a positive affective state (i.e., undirected song). The studies reviewed above certainly demonstrate that neural systems implicated in reward are differentially active during directed and undirected singing behavior; however, these studies do not directly test whether these patterns of activity actually relate to reward or pleasure associated with singing behavior. In fact, the link between singing behavior and reward until recently had not been explicitly tested.

To fill this knowledge gap, in a recent study we evaluated the potential use of a modified condition place preference (CPP) paradigm to measure song-associated reward. CPP paradigms are commonly used to assess rewarding properties of sexual behavior, food consumption or the use of drugs of abuse [24,44,142,147,188]; however, because for singing behavior the bird itself decides whether, when, and how much to sing, unlike drugs or food “the act of singing” cannot be experimentally administered (i.e., one cannot inject the act of singing or place the number of songs produced in a cup as one could a drug or food reward). Thus we modified our CPP paradigm to measure song-associated reward based on studies of copulation-associated reward in male rats [3,185], a behavioral act that also cannot be administered in the same way as food or drug rewards. In separate studies, we observed male starlings or zebra finches singing for 30 minutes in an aviary. Birds were then individually transferred to one of two distinctive compartments of a CPP apparatus for 30 minutes. The next day birds were allowed to move freely between the two compartments for 30 minutes. If the song-paired compartment was preferred, we considered song-associated CPP to have occurred. The idea is that the affective state triggering or resulting from singing is the unconditioned stimulus. When this affective state is paired with a distinct compartment of the CPP apparatus, this compartment becomes associated with the state coupled with singing behavior. Thus, if song is facilitated or maintained by a pleasurable or rewarding affective state, then males should develop a CPP [160].

In this study both male zebra finches and starlings developed a preference for a chamber associated with production of undirected song but not directed song (sexually-motivated song in starlings [Fig. 6] and male-directed song in zebra finches). This is consistent with the hypothesis that undirected song is more tightly coupled to intrinsic reward than directed song. The lack of a place preference associated with directed song suggests that this type of song may not be as dependent on intrinsic reward as undirected song. Instead, data from this study offer at least some support for the hypothesis that female-directed song may be externally reinforced through female responses to song. Specifically, some starlings successfully attracted females to nest sites from which they were singing; others did not. Males whose song production successfully attracted females did not develop a place preference (possibly because no copulation occurred); however, males that failed to attract a female displayed an aversion to the song-paired side of the apparatus. This suggests that the failure to attract a female induced a negative affective state, consistent with the possibility that the external responses of conspecifics to directed singing behavior provide at least one source of reinforcement for this type of song.

Figure 6.

Evidence that undirected song is tightly linked to reward. A and B) Correlations between song and CPP in male starlings. Y-axis represents the proportion of vocal behaviors (calls plus undirected songs) that were undirected songs (left panel) and sexually-motivated songs (right panel) produced by males during and just prior to being placed in one side of the CPP apparatus (song-paired side). The X-axis represents the proportion of time males spent on the previously song-paired side of the apparatus the following day (CPP, considered a reflection of song-associated reward). Each point represents data from a single male. Significant results of multiple regression analysis presented in A. C and D) Mean time spent on the previously song-paired side of the CPP apparatus for males that produced low (open bars) or high (filled bars) rates of undirected (left) or sexually-motivated (right) song. * = p < 0.05. Figure adapted from Riters and Stevenson, 2012.

Hormones may contribute to the regulation of song-associated reward

Altogether the studies reviewed here suggest that the factors rewarding vocal communication differ depending upon whether song is testosterone dependent (i.e., sexually-motivated and female directed) or not (i.e., undirected). Several examples exist of hormones altering neural reward systems to shape behavior so that it is appropriate within a particular context. For example, hormones of pregnancy shape reward systems so that interactions with offspring are rewarding at birth [126]; in female rats proceptive behaviors alter hormonal profiles so that copulation is rewarding when a female is most ready to conceive [76]; and reward associated with feeding is rapidly adjusted by nutrient-induced actions of peptide hormones on reward circuitry [52]. Treatments combining estrogens and androgens modulate dopamine turnover in both the song control system (area X and RA) and mPOA in male zebra finches [22,23]. Androgen and estrogen manipulations also influence tyrosine hydroxylase (the rate limiting enzyme for the synthesis of both dopamine and norepinephrine) labeling density in birds and mammals in a complex time and region specific manner, indicating that steroid hormones can affect dopamine synthesis [2,13,106,107,115–117,120,173]. Enkephalin opioid densities are affected by testosterone or its metabolites in mammals, specifically within the preoptic area (e.g., [175,194]), and androgens increase beta-endorphin levels in VTA [99]. In male dark-eyed juncos opioid receptor densities in mPOA and VTA shift seasonally in association with gonad volume [204], suggesting seasonal changes in testosterone in seasonally breeding birds such as starlings may alter opioid receptor activity in mPOA and VTA. Data also demonstrate rapid estrogen receptor effects on opioid receptor activity in the spinal cord [123], suggesting that rapid changes in testosterone or its metabolites in opportunistic breeders such as male zebra finches may alter opioid activity in the brain to regulate directed and undirected singing behavior within a short time frame. Future studies are needed to provide insight into whether and how hormonal modification of reward circuitry acts to shape vocal communication.

Synthesis and conclusions

A growing body of data implicates dopamine and opioids in the regulation of song, and supports the idea that the role of these neurochemicals differs depending upon whether song is female-directed or undirected. The working hypothesis proposed here is that the factors motivating song differ depending upon the context in which song is produced, with undirected song relying more heavily on intrinsic reward (in part through opioids released in mPOA and VTA by the act of singing) than song directed towards a female (which may be reinforced through opioids released as part of social interactions). Dopamine is proposed to regulate the motivation to sing within any context, however given the role of dopamine in sexual motivation and reward-seeking behavior that dopamine is expected to play a predominant role in highly incentive-motivated (e.g., female- or male-directed) singing behavior.

One can imagine that if opioid concentrations are excessively low a bird will not sing in any context (i.e., if a bird is experiencing an extremely negative affective state). Thus it may be that at least some level of opioid release is necessary to facilitate singing (or that singing behavior may induce a low level of opioid release) in any context. In the case of undirected song, opioid release acting in part at mu receptors in mPOA and possibly VTA may reward song production and trigger additional singing behavior. In contrast, in the case of sexually-motivated song this release may not be sufficient to induce a reward state, given the relatively lower densities of mu opioid receptors found in mPOA and VTA in birds singing highly sexually motivated song ([104]; Fig. 5G, H). This idea is supported by the results of the CPP study showing that males develop a place preference associated with undirected but not directed song ([160]; Fig. 6).

It has been proposed that neurochemical deviations from a homeostatic equilibrium induce a negative affective state, which can be alleviated by a return to equilibrium [138,140]. For opportunistic or seasonally breeding songbirds, it is possible that in the absence of sexual contact, a reduction in opioid levels in mPOA results in an unpleasant affective state that can be ameliorated through copulation-induced opioid release. Song in this case would be used to attract a female so that her proximity or copulation would elevate opioid levels and associated affect. In this scenario, song is a component of a system that coordinates the dynamic equilibrium of opioid-mediated affective state.

In studies of feeding behavior, homeostatic feeding (feeding to meet nutritional needs) is contrasted with hedonic feeding (feeding for pleasure beyond nutritional needs) [165]. In contrast to the idea that sexually-motivated song coordinates opioid homeostatic equilibrium, it may be that undirected singing behavior is strongly mediated by its hedonic value, as supported by the results of the CPP test [160]. Undirected song is common in large affiliative flocks [98,155,178,207]. It is possible that positive environmental factors such as the presence of food, the absence of predators or the safety of a flock results in a rise in dopamine as well as opioids in mPOA and possibly VTA which triggers singing behavior. Singing behavior may then trigger further opioid release and reward which may promote further singing behavior. It is possible that there is a threshold at which singing-induced opioid release may inhibit song (inducing satiety); however I propose that undirected singing generally does not result in surge of opioid release comparable to levels released by copulation. This somewhat positive feedback loop, could facilitate and maintain undirected singing behavior indefinitely; however there are situations in which it is important for flocks to rapidly inhibit song production, for example in the presence of a predator. There is evidence that opioids are released in mPOA in response to stressors [137,152]. Thus it may be that a surge in opioid release in response to an environmental stressor, such as a predator serves to rapidly inhibit male singing behavior. After the removal of a stressor, opioid levels may drop to a point at which members of a flock reinitiate singing behavior. Here I propose that opioid reward triggers and maintains singing independent of conspecific behavioral responses to singing behavior. I do not propose however that undirected singing is completely divorced from social processes given that flock membership strongly promotes singing in this context.

Future directions

The data reviewed here link enkephalins and mu opioid receptors in mPOA and VTA most closely to undirected singing and dopamine in mPOA and VTA to sexually-motivated song. The CPP data also support the idea that undirected song production is closely linked to immediate reward [160]. It is now necessary to combine site- and receptor subtype-specific opioid and dopamine pharmacological manipulations with paradigms designed to evaluate the rewarding properties of song (e.g., the CPP reward paradigm reviewed above [160]) to determine directly whether, where, and at what receptors subtypes dopamine and opioids contribute to song-associated motivation and reward.

This review is limited to studies on mPOA and VTA; however, there are several additional dopamine- and opioid-rich brain regions implicated in motivation, reward, and social behavior that are differentially active during sexually-motivated, agonistically-motivated, and undirected singing behavior. These areas include among others the mesencephalic periaqueductal gray, ventromedial nucleus of the hypothalamus, bed nucleus of the stria terminalis, and lateral septum [79,82,85,87,88]. Furthermore, additional neurochemicals known to strongly influence reward and reward-seeking behaviors are also likely to contribute to differential regulation of the motivation to sing within distinct social contexts. These include endocannabinoids, hypocretin/orexin, GABA, glutamate, serotonin and norepinephrine [66,83,100,124,179,181,187,192].

Future work is also needed to explore the neural regulation of the motivation to communicate within additional distinct social contexts. Data already suggest that dopamine D1 receptors in the lateral septum may be closely coupled to singing in the presence of other males rather than sexually-motivated song in male starlings [87]. Furthermore, distinct patterns of immediate early gene labeling in mPOA and the lateral septum are linked to individual differences in alarm call production in chickadees [64]. These data support the hypothesis that distinct patterns of neuronal activity in brain systems implicated in social motivation underlie motivationally-distinct forms of vocal behavior. Future studies of additional functionally/motivationally distinct forms of communication are needed, including studies of territorial vocalizations, food begging, social contact, and alarm calling.

Although it is proposed here that undirected singing in adult songbirds is not predominately influenced by conspecific reactions to song, rapid behavioral responses of females to male songs alter the development of male song in cowbirds [197,198]. Similarly, responses of caregivers to infant vocal behavior in humans influence language development [74,75]. Research on reward associated with song learning in young birds and the extent to which potentially subtle social responses of conspecifics to undirected song during learning influence song in this context is also needed [73].

Finally, the findings reviewed here contribute to the understanding of mechanisms underlying consistent individual differences in behavior (sometimes referred to as “personality”) [15,57,70,201]. Specifically, data reviewed here suggest that individual differences in the distribution and densities of dopamine and opioid receptors or proteins may in part underlie consistent individual differences in the motivation to communicate.