The Behavioral Neuroscience of Anuran Social Signal Processing

Abstract

Acoustic communication is the major component of social behavior in anuran amphibians (frogs and toads) and has served as a neuroethological model for the nervous system’s processing of social signals related to mate choice decisions. The male’s advertisement or mating call is its most conspicuous social signal, and the nervous system’s analysis of the call is a progressive process. As processing proceeds through neural systems, response properties become more specific to the signal and, in addition, neural activity gradually shifts from representing sensory (auditory periphery and brainstem) to sensorimotor (diencephalon) to motor (forebrain) components of a behavioral response.

A comparative analysis of many anuran species shows that the first stage in biasing responses toward conspecific signals over heterospecific signals, and toward particular features of conspecific signals, lies in the tuning of the peripheral auditory system. Biases in processing signals are apparent through the brainstem auditory system, where additional feature detection neurons are added by the time processing reaches the level of the midbrain. Recent work using immediate early gene expression as a marker of neural activity suggests that by the level of the midbrain and forebrain, the differential neural representation of conspecific and heterospecific signals involves both changes in mean activity levels across multiple subnuclei, and in the functional correlations among acoustically active areas. Our data show that in frogs the auditory midbrain appears to play an important role in controlling behavioral responses to acoustic social signals by acting as a regulatory gateway between the stimulus analysis of the brainstem and the behavioral and physiological control centers of the forebrain. We predict that this will hold true for other vertebrate groups such as birds and fish that produce acoustic social signals, and perhaps also in fish where electroreception or vibratory sensing through the lateral line systems plays a role in social signaling, as in all these cases ascending sensory information converges onto midbrain nuclei which relay information to higher brain centers.

Introduction: Acoustic communication in anuran amphibians

Social behavior emerges from neural processing that transforms sensory representations of social signals into the expression of a behavioral response. The acoustic communication system of anuran amphibians (frogs and toads) has served as a valuable neuroethological model for this process[1–4]. Anurans are relatively uniform in the basic communication strategy employed for reproductive social interactions. Almost universally, males aggregate at breeding sites where they produce an advertisement, or mating, call that serves to attract females. This acoustic communication system plays an important role in the evolutionary processes of speciation and sexual selection: the conspecific call and the female’s preferential response to it can restrict genetic exchange to members of the same species, and female preferences for calls among conspecifics can lead to the further evolutionary elaboration of calls[5,6] (for perspectives on other taxa, see [7,8]). Males also respond to male advertisement calls, usually by counter-calling, and like females they respond preferentially to conspecific calls over heterospecific ones.

A major focus of neuroethological work on anuran acoustic communication has been on interspecific variation in vocal signals and the relationship of that signal variation to interspecific variation in auditory processing. This work is driven by the question of how species-specific signals are encoded by the nervous system in a manner that guides receivers to respond preferentially to conspecific over heterospecific signals. Work by many labs suggests that the analysis of advertisement calls leading to response decisions is a progressive process from the periphery to the forebrain. Recent work has shown that the process is more complex than a simple processing hierarchy, moving from less to more complex neural responses, although that certainly is an important component. Changes also occur in the way neural activity patterns in different brain regions are correlated with each other; that is, changes in functional connectivity are a critical component underlying decision making. Figure 1 outlines the various processes that take place as acoustic social signals are processed by the frog brain, processes which we cover in this review.

Figure 1.

Aspects of social signal processing in the nervous system of anuran amphibians, superimposed on schematic dorsal view of a frog brain showing the basic ascending auditory pathways (not all connections are shown). Rostral is at the top, caudal at the bottom. List of abbreviations: Components of the VIIIth cranial nerve complex: G VIII vestibulocochlear nerve ganglion; N VIII vestibulocochlear nerve; AP amphibian papilla; BP basilar papilla; AVC anterior vertical canal; PVC posterior vertical canal; HC horizontal canal; S sacculus; L lagena. Lower brainstem auditory areas: CN caudal nucleus; DLN dorsal lateral nucleus; SON superior olivary nucleus; NLL nucleus of the lateral lemniscus. Midbrain torus semicircularis subdivisions: Tl laminar nucleus; Tm magnocellular nucleus; Tp principal nucleus. Diencephalon: A anterior thalamic nucleus; C central thalamic nucleus; P posterior thalamic nucleus; VM ventromedial thalamic nucleus; SC suprachiasmatic nucleus. Telencephalon: Pm medial pallium; Sep septal complex; Str striato-pallidal complex. Illustration of frog brain and connections from[11].

A. Processes involved in the analysis of conspecific social signals as information ascends from the ear though the brain. 1. Neural responses in the ear generated by two auditory end organs, the amphibian and basilar papillae, are biased toward the spectral features of the conspecific call. 2. As information ascends from the lower brainstem to the midbrain, responses properties of cells become more specialized for coding elemental spectral and temporal features of the conspecific call. 3. In the midbrain, many neurons act as specialized feature detectors for either spectral or temporal call characteristics, which in turn relay this information to forebrain areas regulating motor and physiological responses to the call. However, the call is best represented there by activity distributed in multiple midbrain neurons across its subnuclei, that is, by a process of population coding. 4. In addition to changes in the firing rates of neurons within brain nuclei, behavioral salient signals like conspecific social signals induce changes in the correlated patterns of activity locally and across brain divisions, that is, a change in the functional connectivity of brain areas.

B. Coding of acoustic social signals at different levels of the frog nervous system. In the brainstem (medulla and midbrain), neural activity is driven by sensory features of the signal. Activity in thalamic nuclei is correlated with sensory features and movement during the stimulus period. Telencephalic activity in most areas is more strongly correlated with movement than with stimulus features. The torus semicricularis of the midbrain (circled in red) is a key gateway for transferring sensory information from lower levels to the motor and endocrine control areas of the forebrain and its properties are important for generating sex differences in responding as well as changes in responding with physiological state.

Species differences exist at the level of the ear in differences in band-pass characteristics and tuning that generally match the frequency composition of important components of the advertisement call. Robert R. Capranica articulated the behavioral relevance of species specializations in peripheral tuning in his classic papers in the 1960’s, where he proposed the “match filter hypothesis” [9,10]. Capranica’s idea was that tuning of the ear generally matches the spectral frequency peaks of a species’ call, thus biasing the organism’s sensory system to detecting their species’ call among the noise and irrelevant heterospecific signals that threaten to obscure it. An individual frog could use the match between call and tuning to determine whether a signal was in fact that of a conspecific particularly when there are multiple frequency peaks in the call. In that case, behavioral responses depend on the activation of those sensitive portions of the ear simultaneously or in some characteristic temporal sequence (depending on the frequency structure of the call).

Capranica noted that the central nervous system would need to extract information from the ear and further process it. In fact, the frog central auditory system does gradually construct neurons with feature-detector characteristics that respond to particular frequency combinations or temporal characteristics that typify the species advertisement call as processing proceeds sequentially from caudal to rostral auditory nuclei in the brain. This species-typical tuning bias is carried through lower brainstem auditory nuclei. Additional frequency specializations may occur, and responses to temporal features of the call begin to appear in this part of the brain. It is not until the midbrain, however, that a significant integration of auditory information into complex feature detectors first occurs.

The midbrain auditory center as an analyzer of social signals

The anuran auditory midbrain is the torus semicircularis, a homolog of the inferior colliculus[11]. It is composed of several subnuclei, the most significant of which is the laminar nucleus. Laminar nucleus neurons provide a significant portion of the torus’ output to both forebrain and brainstem areas, and are rich in receptors for a variety of hormones and neuromodulators. Walkowiak and Luksch [12] and others [11,13–15] proposed the torus to be a sensory-motor interface, that is, a key neural area that links sensory input to behavioral responses.

Throughout the anuran torus, neurons show complex response properties with special sensitivity to behavioral important call features[11,16–18]. Some cells respond to combinations of tones characteristic of calls. Others are temporally tuned to features such as pulse or amplitude modulation rate that are important for call recognition. Edwards et al. [19,20] (see also [21]) reported a remarkable example of this specialization: toral neurons that “count” pulses and are extremely sensitive to the interpulse interval. These neurons respond only after a certain number of pulses have occurred and only if the interpulse intervals are consistent during that counting phase. Such specialized neurons provide a mechanism for the torus to ascertain critical temporal information needed to identify species-specific advertisement calls in those species that employ amplitude modulated trills. Rose and Brenowitz[22], in fact, found that male Hyla regilla use interpulse interval information of the kind important for these toral cells to discriminate advertisement from aggressive calls of neighboring males and mount the appropriate behavioral response.

The spectral and temporal tuning characteristics of toral neurons show clear links between midbrain auditory processing and behavioral expression. However, the accumulated electrophysiological studies show that the individual midbrain cells each code only a portion of the information necessary for true call representation[16,17]. Spectrally tuned cells, for example, may respond best to particular combinations of tones, but none is so selective that it only responds to a conspecific call. Moreover, spectrally tuned cells do not have a particularly strong bias toward temporal properties. Similarly, the temporally tuned neurons have broad spectral bandpass characteristics, so have no preference for the frequency characteristics of the signals carrying the temporal information. If frogs attend to both the spectral composition and the temporal patterns that define their calls, it must be with at least two populations of neurons, as no individual midbrain cells are capable of doing both and in fact none of either type capture more than one or two call features. Therefore, if information necessary to recognize conspecific calls reliably is present at the level of the midbrain, it must be encoded in the population response across some area of the torus. Population coding has been invoked to explain the representation of complex stimuli in many mammalian sensory systems[23–29], often in regards to cortical processing. In the mammalian auditory system, population coding in the cortex and midbrain have provided insights into processing amplitude, location, and spectro-temporal features of complex signals.

Investigations of anuran population coding and call processing have not reached the level of sophistication represented by these studies. However, recent work using immediate early gene (IEG) expression[30,31] as a marker of stimulus-evoked neural activity has suggested that population coding is an important component of call representation in the frog auditory midbrain. Using IEGs to denote neural responses does have its limitations[32], but it allows one to overcome some significant technical limitations encountered in traditional electrophysiological studies in non-mammalian vertebrates such as amphibians, reptiles and fish. Single cell recording in such small animals generally provide a limited sample of neurons in any one individual, constraining the statistical approaches one might use to extract population codes. Moreover, their very small brains and the general lack of stereotaxic methods and maps make it difficult to mark multiple recording sites in a single individual with enough precision to reliably delineate subarea locations. IEGs allow precise anatomical measurements of activity (gene expression) levels across a brain area and between multiple nuclei throughout the central nervous system. Hoke et al.[31] used this approach to assess IEG expression (using the immediate early gene egr-1 (also called ZENK, zif268, NGFI-A, and krox-24) in the torus of the túngara frog (Physalaemus pustulosus) in response to a variety of conspecific, heterospecific, and biologically neutral acoustic stimuli. A simple analysis of mean expression levels showed that IEG expression was higher in response to conspecific calls in most toral subnuclei, with the laminar toral nucleus showing the clearest difference to behaviorally salient conspecific calls compared to other stimuli. These results in themselves are not surprising given the sensory bias toward conspecific calls apparent even in the auditory periphery. What did emerge, however, from a discriminant function analysis was that information in the activity levels across all four toral subnuclei responding to calls was the best predictor of stimulus type, correctly identifying different types of calls 85% of the time. Even more interesting, the results were not explainable by any simple acoustic features. Rather, the activity pattern across the torus appears to be an emergent property related to overall characteristics and behaviorally salience of complex acoustic signals.

Functional network changes as codes for social signals

Population coding is a process whereby responses across different neurons (or regions) represent information about a signal. Signals may also differentially change the relationship among brain regions in their responses; that is, how tightly activity in one brain area correlates with activity in other areas. The activity correlations among brain regions define their ‘functional connectivity’, that is, how activity in one brain area predicts activity in another, a process that can be the result of direct connections, indirect connections, or common inputs. Such correlated activity patterns emerge rapidly and transiently and are tied to a particular context or input. Functional connectivity has emerged as an important concept explaining the emergence of cortical patterns of activity underlying higher cognitive processes examined in human imaging studies[33–36]. Although rarely used this way, IEG studies can form the basis of functional connectivity studies[37] by providing a record of neural activation during a stimulus. They can be valuable when functional imaging is not feasible, as is the case for many small animals and non-mammals. IEG mapping in túngara frogs has revealed that shifts in functional connectivity accompany changes in activity levels when frogs are hearing conspecific signals. Hoke et al.[38] compared IEG patterns in túngara frogs hearing a conspecific advertisement call (the “whine”), a similar but ecologically meaningless heterospecific call, a conspecific aggressive “mew” call, and a silent control. A correlation analysis showed that hearing the conspecific advertisement call induced much great coupling of activity (in the sense of correlated patterns of IEG expression) across the brain, linking more closely activity levels in the brainstem, midbrain, and forebrain, than did hearing of the other signals (Figure 2). An investigation[39] of functional networks showed that different patterns of functional connectivity within the hypothalamus were elicited by hearing conspecific calls, heterospecific calls, and silence. Correlation changes were apparent even among hypothalamic nuclei that did not show statistically significant mean activity differences to different signals.

Figure 2.

Matrix indicating patterns of correlated activity, measured by level of egr-1 expression, across regions of the telencephalon, hypothalamus, thalamus, and midbrain (coded by gray scale) when Physalaemus pustulosus frogs heard a heterospecific Physalaemus enesefae advertisement call (upper left), a conspecific aggressive call (the mew, upper right), a conspecific advertisement calls (the whine, lower left), or silence. Positive correlations are in yellow, negative in blue, and no significant correlation is indicated by black. When hearing a conspecific advertisement call, neural activation within and across brain regions becomes significantly more correlated indicating a change in functional connectivity. From[38].

The correlation analyses made possible by IEG studies show that the behavioral relevance of a signal is represented by interactions among both closely and distantly related brain areas as much as it is represented by changes in mean activity within any one area. What these changes in functional connectivity actually mean for perception or behavior is still unresolved and is very difficult to test experimentally. Nevertheless, these results and those related to population coding provide a new way to consider the relationship between brain and behavior. This different view depends less on the idea that there are discrete brain “centers” for particular behaviors activated by stimuli, as opposed to considering behavior as emerging from patterns of neural activity tying sets of neurons or brain areas together in different ways depending on the context or meaning of stimuli.

Sex differences in responding to signals

All of the work on frogs reviewed above treats the behavioral neuroscience of social communication as a species-specific problem, that is, how does the typical individual of that species recognize its social signals? But of course, there are considerable differences among individuals within a species in terms of what a particular social signals means to them and how they respond to those signals. Sex differences are a clear example of this. Males and females respond very differently to male signals. For the acoustic communication behavior of frogs, evidence points to the midbrain auditory center, the torus semicircularis, as a playing a key role in mediating the transformation of signal perception into a sexually differentiated response.

One of the most common features of animal social communication is that, as a general rule, females respond to a more restricted range of potential male signals than do males to either female signals or male signals. This sex difference is predicted by the different costs to the sexes in responding inappropriately to sexual solicitation signals (higher in females than males) compared to the costs of not responding to a potential sexual signal (higher in males than females) [40,41]. In the case of túngara frogs, males and females differ in terms of how they respond to calls: males most often respond vocally, and much less often by moving toward the call, whereas females move toward the call and do not vocalize. In addition, the sexes differ in their selectivity: males vocalize in response to a wide range of audible signals whereas females are much more choosy when responding phonotactically [42–45]. In principle, the neural correlates of sex differences could be found in sensory areas, in brain areas serving as sensorimotor links, or in motor areas (or any combination). By identifying the processing levels at which these stages take place, one can then ascertain where male and female neural responses differ.

This task can be accomplished through a covariance approach that determines whether IEG activation in different areas of the brain correlate best with either conspecific signal presentation, or with movement that occurred when a signal was present, or varied with both signal presentation and movement[38]. Within the brainstem auditory nuclei up to the midbrain, IEG expression predicts that signal reception is on average higher for conspecific calls than other sounds or silence, but does not covary with the individual’s movement during the stimulus period. This would be expected based on neurophysiological studies of frog auditory processing and based on the fact that the areas assessed are standard auditory nuclei increasingly dominated by specializations to detect conspecific calls at progressively more rostral levels. In the diencephalic targets of the midbrain, however, IEG activity covaries with both stimulus type and with movement. Multiple forebrain areas also have increased IEG expression to conspecific calls[39,46–48], but a covariate analysis reveals that telencephalic limbic and basal ganglia areas are either correlated to movement alone or exhibited a movement by stimulus interaction. These results indicate that ascending pathways in the frog brain mediate a transition from primarily sensory, to sensorimotor, to motor processing as the information makes it way from brainstem, to thalamus, to telencephalon.

Where, then, do sex differences in activation arise? Expression patterns in lower brainstem areas do not differ in males and females (Figure 3). However, the laminar nucleus of the torus does show a sex difference related directly to the sex difference in the range of signals generating a response[49] (Figure 3). In males, midbrain IEG expression to conspecific Physalaemus pustulosus calls and to heterospecific P. petersi calls are not significantly different, which is consistent with the male’s behavior—males will vocalize in response to both. Female toral IEG expression is significantly greater in response to the P. pustulosus call than to the P. petersi call, and females make phonotaxis responses to the former, but not the latter; again, the analyses of brain and behavior are congruent. These results suggest that the midbrain is a key component of the neural mechanisms generating sex differences in responding to social signals, and it acts by regulating the range of signals passing from the auditory system to the forebrain. Examining the functional connectivity between midbrain and forebrain also implicates the torus as a gateway contributing to the sex differences in behavioral responses to signals[47]. In females, there is a significant linear correlation between laminar nucleus IEG expression and forebrain expression; in males, the correlation is not significant. Assuming that the correlation patterns indicate something about the transfer of information from midbrain to forebrain, one could interpret this result as showing much tighter control of the information flow in females.

Figure 3.

Relative expression of egr-1 in the superior olivary nucleus (A) and laminar nucleus of the torus semicircularis (B) in male (gray bars) and female (black bars) túngara frogs hearing a conspecific Physalaemus pustulosus advertisement call, a heterospecific Physalaemus petersi advertisement call, or silence. The results show that the superior olivary nucleus, a lower brainstem auditory center, responds best to the conspecific call in both sexes and there is no sex difference. In the midbrain, however, male neural response is equally strong to the conspecific and heterospecific calls, whereas female responses are significantly lower to the heterospecific call. This matches the behavioral response: males will call in response to either call, whereas females express phonotaxis only to the conspecific call. Modified from[49].

Somewhat surprisingly, the patterns of correlation among forebrain nuclei do not differ in males and females despite sex differences in IEG expression in several forebrain areas[47]. This suggests that once activated, forebrain areas work together similarly in males and females, again indicating that the midbrain link between auditory processing in the brainstem, and forebrain operations underlying behavioral responses, is the key to generating sex differences in behavior. It is important to note that these results do not reveal the reason for the more major sex difference in what behavior males (vocal response) and females (phonotaxis) most predictably express in response to the same social signal (the male’s advertisement call). Anatomical sex dimorphisms in descending vocal control pathways have been identified[50,51], and such anatomical differences could play a role. Furthermore, neurophysiological or within-area circuits may exist at any level of processing, something that IEG studies can not identify. Nevertheless, the idea that some type of a gateway regulates the passage of information from sensory to motor areas, and that differences in that choke point reveal themselves in sex and possibly other differences in behavior has been advanced before. Kimchi et al.[52] suggested something similar for sex differences in response to olfactory social signals.

Neural correlates of the túungara frog’s intrasexual differences in responding to conspecific signals also point to the laminar nucleus of the torus as a key regulatory gateway. Female frogs (like many female vertebrates) vary in their receptivity toward male advertisement signals depending on their hormonal state[53,54]. Female túngara frogs also vary in the range of signals to which they will respond: when estrogen is high, females are more receptive and more permissive in their responses[54]. Toral activity levels, whether measured electrophysiologically or by IEG expression, vary in a way that predicts the behavioral pattern[55–57]. Neural responses in the torus are high when estrogen levels and female behavioral responses are high, and decrease significantly after females release their eggs and their receptivity drops. Thus the auditory midbrain might serve to control intrasexual differences in social responding as well as intersexual differences, in both cases by regulating the passage of sensory information to effector regions of the forebrain.

The results of IEG studies on sex differences and changes with hormonal state help to distinguish from among three possibilities for the neural systems variation underlying behavioral differences (Figure 4). Rather than arising from differences in basic sensory processing, or in the basic organization of forebrain motor or physiological control areas, the most important mechanism may be employing a strategically located gateway that regulates the flow of information from sensory to motor areas.

Figure 4.

Three possible mechanisms of could account for differences in the responses to male social signals between sexes, among individuals with in a sex, or within an individual depending on physiological state or external conditions. A. Sensory systems may be different, thereby biasing responses at higher levels. B. A neural locus acting as a ‘gateway’ controls access to behavioral control centers so that information transfer to higher behavioral or physiological response areas is different. C. Both the sensory processing and the transfer of information is constant, but the behavioral control centers have different thresholds, filter functions, or other characteristics leading to the behavioral differences. Immediate early gene results in túngara frogs suggest that the most likely possible mechanism is B, with the gateway being the midbrain auditory center relaying information about calls to forebrain areas controlling motor and endocrine functions. Figure courtesy of K. Hoke.

Anuran communication as a model for behavioral neuroscience

Acoustic communication in anuran amphibians may be a specialized trait in that vertebrate group, but the principles of neural processing underlying it can be applied to the behavioral neuroscience of social communication more generally. As in any other social behavioral context, signals produced by conspecifics must be received, represented in the central nervous system, and transformed into a behavioral response. The anuran brain shows a pattern in which sensory biases starting in the periphery are honed into feature detectors representing elemental aspects of species-typical signals as the information is gradually transmitted through the sensory pathway. These feature detectors are not sufficient to explain either the overall preference for conspecific over heterospecific signals, or the variation in responding in different physiological or environmental contexts. Instead, more complete information about a social signal is contained in a population code as well as in the pattern of functional connectivity that emerges depending on salience and context. It should be noted that not all behavioral responses need depend on a full analysis of information emerging from processes at all levels of the brain. Some responses may occur based solely on processing at lower sensory levels, much as happens for simple motor reflexes. Some male frogs can respond vocally to conspecific calls so rapidly that it is unlikely that the forebrain is involved[58]. It is very likely that this complex, multidimensional mode of representation is important in many aspects of social behavior, and the work in amphibians suggests that it is not a process restricted to higher areas of the mammalian cortex.

In anurans, signal analysis culminates at the level of the midbrain rather than in a sensory cortex (in fact, based on its functional anatomy of the telencephalon, a functional equivalent of primary sensory cortex may be lacking in amphibians[11]). Nevertheless, the idea that processing proceeds to sensorimotor representation, then to motor representation in forebrain centers is an idea that can be applied to many functional systems. Also important is the idea that a gateway regulating the transfer of sensory information to effector centers. Our data show that in amphibians the midbrain acts as the gateway, as it is located in a strategic position between the purely sensory areas of the lower brainstem and the motor and endocrine regulatory areas of the forebrain. We believe that the same may hold true for other vertebrates such as birds and sound-producing fish that use acoustic social signals. The auditory midbrain in fish, for example, occupies such a position anatomically[59], has neurons that code features of conspecific vocal signals[60–62], and it contains steroid hormone receptors that may regulate auditory processing there[63]. The processing of social signals using other sensory systems, such as the electrosensory system that, like the auditory system, depend on a midbrain relay, might similarly employ the midbrain as a gateway. The location of the gateway is likely different depending on the sensory system being used and the behavior being mediated. One might not expect the midbrain to act as the sensory gate for olfactory or visual signaling, for example, because these sensory systems do not pass through an obligatory midbrain relay en route to the forebrain. But if the amphibian acoustic communication paradigm can be generalized to communication in other modalities, it would predict a gatekeeper function at some critical node between sensory and motor systems. Finally, this concept implies that both the perceptual analysis features and the functional circuits defining different male and female behaviors are present in both sexes; what generates the sex difference in behavior is the way in which the perceptual process is connected to those functional circuits. Testing the limits of this idea would be important for understanding the behavioral neuroscience of social behavior and the true nature of sex differences.