Neural map formation and sensory coding in the vomeronasal system

Abstract

Sensory systems enable us to encode a clear representation of our environment in the nervous system by spatially organizing sensory stimuli being received. The organization of neural circuitry to form a map of sensory activation is critical for the interpretation of these sensory stimuli. In rodents, social communication relies strongly on the detection of chemosignals by the vomeronasal system, which regulates a wide array of behaviours, including mate recognition, reproduction, and aggression. The binding of these chemosignals to receptors on vomeronasal sensory neurons leads to activation of second-order neurons within glomeruli of the accessory olfactory bulb. Here, vomeronasal receptor activation by a stimulus is organized into maps of glomerular activation that represent phenotypic qualities of the stimuli detected. Genetic, electrophysiological and imaging studies have shed light on the principles underlying cell connectivity and sensory map formation in the vomeronasal system, and have revealed important differences in sensory coding between the vomeronasal and main olfactory system. In this review, we summarize the key factors and mechanisms that dictate circuit formation and sensory coding logic in the vomeronasal system, emphasizing differences with the main olfactory system. Furthermore, we discuss how detection of chemosignals by the vomeronasal system regulates social behaviour in mice, specifically aggression.

Introduction

The ability of mammals to interact with their environment relies on the interpretation of a wide variety of signals detected through various sensory modalities. An accurate representation of these sensory inputs is obtained through the formation of neural maps that allow for the organized relay of information from sensory neurons in the periphery to brain structures that control behavioural responses. Several sensory maps, such as the retinotopic map of the visual system, relay spatial information through the formation of stereotypic connections that preserve spatial order between sensory neurons in the periphery and their targets in the central nervous system. Maps can also provide information about the discrete qualities of the signals detected, such as in the glomerular map of the olfactory system. The development of neural maps that provide both spatial and qualitative information relies on genetic and activity-dependent mechanisms that ensure the formation of accurate synaptic connections between the peripheral and central nervous system (CNS).

The establishment of the glomerular map in the olfactory systems is crucial for the regulation of a wide variety of innate and social behaviours in animals. In most mammals, the detection of olfactory cues is mainly mediated by two anatomically distinct chemosensory systems, the main and accessory (or vomeronasal) olfactory systems. While these two systems differ by the types of chemosensory receptors they use and the organization of their neuronal circuitry within the central nervous system, there is convincing evidence that they have complementary functions in the regulation of multiple social behaviours in mice, including reproduction and aggression [1–8]. The olfactory systems are useful models to study the mechanisms that underlie sensory map formation because of the existence of multiple lines of genetically modified mice that allow precise tracing of specific populations of sensory neuron axons. Furthermore, the innate nature of multiple behaviours they control facilitates the examination of the relationship between precise circuit formation and the social behaviours that these circuits regulate.

Although the glomerular maps formed in both the main olfactory system (MOS) and vomeronasal system provide information about the discrete qualities of the chemosensory signals detected, sensory information processing appears to differ between the two systems. In the mouse MOS, olfactory sensory neurons (OSNs) express one of approximately 1200 odorant receptors (OR), and all OSNs expressing the same OR innervate on average two glomeruli per olfactory bulb (OB) [9–11]. Dendrites of second-order neurons in these glomeruli, therefore, receive input from a single OR. Most ORs are not specifically tuned to an odorant but instead bind odorants with varying affinities based on their molecular features [12]. This property enables ORs to bind to multiple ligands and allows a given ligand to activate multiple ORs on different OSNs, depending on the molecular features of the ligand [12]. Although the majority of ligands tested to date bind multiple ORs, recent evidence suggest that some ligands may bind a single OR [13]. Since most odorants bind multiple ORs, the MOS must rely on a combinatorial code of OR activation to differentiate between particular odorant molecules [12]. Because OSNs cannot discriminate specific odours, a glomerular map of OR activation is relayed to the brain where further processing is needed to encode the desired response to the stimulus. In the vomeronasal system, a subset of vomeronasal sensory neurons (VSNs) appears to be tuned to specific cognate ligands, suggesting that the integration of at least some stimulus information may happen at the level of the accessory olfactory bulb (AOB) [14–16]. In this review, we will discuss the mechanisms underlying the formation of the glomerular map in the vomeronasal system and the importance of this map in regulating aggression in rodents.

Organization of the vomeronasal system

The accessory olfactory system begins with the vomeronasal organ (VNO), a chemoreceptive structure located in the base of the nasal septum, and of the AOB, located in the dorso-caudal region of the OB [17, 18]. The VNO neuroepithelium houses vomeronasal sensory neurons (VSN) that express receptors capable of detecting chemosignals, including proteins and small organic molecules (Fig. 1a). While OSNs express one of over 1000 OR genes, each VSNs expresses one or a restricted few VRs from a repertoire of close to 400 functional genes [19–24]. The sensory epithelium can be subdivided into two non-overlapping regions based on the type of vomeronasal receptors expressed by the VSNs. VSNs located in the apical and basal regions of the VNO selectively express members of the vomeronasal receptor 1 (V1R) and V2R families of G-protein coupled receptors (GPCRs) that signal through the Gαi and Gαo proteins, respectively (Fig. 1b) [19, 25–31]. While most VSNs express V1R or V2R genes, a subset of VSNs exclusively expresses another family of chemosensory receptors known as formyl peptide receptors (FPR) [32, 33]. The segregated localization of V1R- and V2R-expressing VSN cell bodies in the VNO is maintained at the level of their axonal projections to the AOB. V1R-expressing VSNs that have their cell bodies in the apical layer of the VNO project axons to the anterior portion of the AOB whereas basally located V2R-expressing VSNs innervate the posterior region of the AOB [34, 35] (Fig. 1b). In the AOB, VSN axons synapse onto dendrites of mitral cells in neuropil structures termed glomeruli. While the implications of segregating V1R and V2R VSN axonal populations into different regions of the AOB remain unknown, this wiring pattern is not maintained at the level of mitral cell projections, which directly innervate multiple nuclei of the limbic system, bypassing cortical structures [36–40]. Indeed, stereotaxic injections of tracers into the glomerular and mitral cell layers of the AOB, as well as in the amygdala, have revealed significant overlap in the innervation of the amygdala by mitral cells located in the anterior and posterior regions of the AOB [41–43]. Neurons located in nuclei innervated by mitral cells project to multiple areas of the hypothalamus that are linked to aggression, parental behaviour, and reproduction, including the ventromedial hypothalamus (VMH) [44–46]. Thus, the AOB can be viewed as a structure that consolidates chemosensory information from the environment and in turn relays this integrated information to higher brain centres that can direct behavioural outputs.

Fig. 1.

Circuitry of the mouse vomeronasal system. a The vomeronasal system consists of the VNO and AOB. The VNO located in the base of the nasal septum contains VSNs that express chemosensory receptors. These VSNs project axons to the AOB located on the caudal aspect of the OB. The neuroepithelium of the VNO is divided into apical (green), and basal (purple) regions. The AOB is also subdivided into two regions, the anterior (green) and posterior (purple) sections. b Segregated projection of VSN axons in the AOB. VSNs that have their cell bodies in the apical region of the VNO express V1R receptors and project their axons to the anterior region of the AOB (green). In contrast, basally located VSNs express V2R receptors and innervate the posterior region of the AOB (purple). A third population of VSNs located in the basal region of the VNO expresses both V2Rs and H2-Mvs (blue). This population of VSNs projects its axons to a posterior subdomain within the posterior region of the AOB. Thus, a tripartite organization of VSN projections exists in the AOB. c VSN axons expressing the same VR coalesce to form homogenous glomeruli. VSNs expressing the same VR project their axons to the same glomeruli within the AOB. Therefore, each glomerulus is a homogeneous structure innervated by a single set of VSN axons. Each population of VSNs expressing the same VR can innervate as many as 30 different glomeruli in the AOB. d Mitral cell dendrite projections in the AOB. VSN axons entering the AOB synapse onto dendrites of mitral cells whose cells bodies are located within the external plexiform layer (EPL) of the AOB. Mitral cells project their dendrites to multiple glomeruli in the AOB to form both homotypic and heterotypic connections with VSN axons. Some mitral cells project dendrites to multiple glomeruli innervated by populations of VSN axons expressing different VRs, thereby forming connections referred to as heterotypic (I). While the majority of mitral cells that have their cell bodies in the anterior or posterior halves of the AOB project their apical dendrites to the homonymous half of the glomerular layer (I, III), some mitral cells located near the anterior–posterior border of the AOB can project their dendrites to the opposite half of the glomerular layer (II). Mitral cells can also project dendrites to multiple glomeruli innervated by populations of axons expressing the same VR, thereby forming connections referred to as homotypic (III). AOB accessory olfactory bulb, OB olfactory bulb, OE olfactory epithelium, VNO vomeronasal organ

Circuitry of the vomeronasal system

In the MOS, the olfactory bulb (OB) provides a spatial map of olfactory receptor (OR) activation. Here, olfactory sensory neurons (OSNs) expressing the same OR converge onto two main glomeruli at fixed locations within the glomerular layer of the OB where they innervate mitral cells [10, 47, 48]. Each mitral cell projects one apical dendrite to a single glomerulus innervated by axons expressing a given OR. In turn, mitral cells project axons to higher cortical areas where sensory information can be further processed. This one OSN type (one OR) to one mitral cell connectivity is described as a “labelled line” of sensory information processing in the MOS and allows for spatial recognition of a given OR activation to be relayed to higher brain areas [49].

A combination of genetic, electrophysiological, and imaging approaches has revealed that the glomerular map in the vomeronasal system differs from the MOS. Genetic labelling of VSN axonal projections has shown that VSNs expressing the same VR innervate as many as thirty different glomeruli within broad but spatially conserved regions of the AOB, with each of these glomeruli containing a single population of VR-expressing axons (Fig. 1c) [50, 51]. Ablating expression of the VR in VSNs leads to improper targeting of axons in glomeruli of the AOB indicating that VRs are required for axonal coalescence [51]. The use of a multireporter transgenic mouse line to label multiple VRs that belong to the same or distinct phylogenetic clade of VRs revealed that VSNs expressing closely related VRs innervate nearby and spatially conserved glomeruli within the AOB [52].

While the use of genetically modified mouse lines to label VSN axons has provided important information regarding VSN connectivity to the AOB, the principles underlying mitral cell connectivity remain to be fully established. Analyses of mitral cell dendritic morphology revealed that the majority of mitral cells located in either the anterior or posterior half of the external plexiform layer (EPL) of the AOB project their apical dendrites to glomeruli located in the homonymous half of the AOB (Fig. 1d) [53, 54]. These results would suggest that the spatial segregation of V1R and V2R glomerular inputs along the anterior–posterior axis of the AOB may be maintained at the level of the mitral cell layer. However, a subset of mitral cells located at the anterior–posterior border of the AOB has been reported to project their apical dendrites to the opposite half of the AOB [53]. Furthermore, mitral cells of the AOB possess long lateral dendritic branches that cross the anterior–posterior border of the AOB, which could convey signals across the two subdivisions and provide lateral modulation of incoming signals within the AOB [54].

A series of cell tracing experiments have revealed significant heterogeneity among mitral cell dendritic projections to AOB glomeruli. In contrast to the MOS, where a mitral cell projects its apical dendrite to a single glomerulus, mitral cells in the AOB project their dendrites to multiple glomeruli [55]. Interestingly, while some mitral cells project their dendrites to multiple glomeruli that are innervated by axons expressing the same VR, other mitral cells project their dendrites to several glomeruli with different VR identities (Fig. 1d) [56]. These observations show that mitral cells can exhibit both homotypic and heterotypic connections with VSNs, suggesting that multiple types of mitral cells may exist in the AOB. Furthermore, electrophysiological studies examining the response profiles of mitral cells to different stimuli have revealed functional diversity between mitral cells. In a study by Meeks and colleagues, stimulation of VSNs with sulfated steroids and urine mixes combined with ex vivo recordings from mitral cells in the AOB led to the understanding that mitral cells are mostly activated by a single stream of information whereas some mitral cells respond to multiple streams of sensory input [57]. Since mitral cells can respond to multiple streams of sensory input, these results provide further evidence that sensory stimuli processing and integration could happen at the level of the AOB.

Mechanisms of glomerular map formation in the AOB

Vomeronasal sensory neurons project their axons in large tightly fasciculated bundles along the medial aspect of the olfactory bulb and turn upon reaching the caudal part of the bulb to innervate the AOB. These axons then segregate into the anterior and posterior regions of the AOB to maintain the spatial separation of apical and basal VSN cell bodies at the level of axonal inputs into the AOB. VSNs expressing the same VR then coalesce to form glomeruli innervated by a single population of VR-expressing axons. The segregation of VSN axons within the AOB and their coalescence into glomeruli appear to be regulated through different mechanisms. A combination of attractive and repulsive forces promotes the segregation of axons along the anterior–posterior axis while the sorting and coalescence of axons into specific glomeruli are at least in part dependent on cell adhesion molecules expressed at their surface (Fig. 2a).

Fig. 2.

Molecular mechanisms underlying vomeronasal circuit formation. a General overview of the mechanisms and key factors that control the targeting of VSN axons to the AOB and their coalescence within the AOB. The fasciculation of the vomeronasal nerve is maintained by Nrp-2-mediated Sema3F repulsion. Receptors for Ephrin, class 3 semaphorins, and Slits regulate the segregation of axons into the anterior and posterior regions of the AOB. Expression of a VR and of cell adhesion molecules, such as Kirrel-3, controls the coalescence of VSN axons into glomeruli. b Axon guidance molecules that regulate the targeting and coalescence of VSNs. I Nrp-2 expression in apical VSNs is required for the proper targeting of apical VSN axons to the anterior AOB. While secreted class 3 semaphorins are expressed by mitral cells and can contribute to preventing Nrp-2-positive axons from entering the posterior AOB, additional Nrp-2 ligand(s) are likely to contribute as well. II EphrinA5 expression in apical VSNs is needed for the targeting of their axons to the anterior portion of the AOB in response to the high levels of EphA6 receptors expressed on mitral cells in this region. EphrinA5–EphA6 interactions promote attraction of apical VSN axons to the anterior AOB. III Robo-2 expression in basally located VSNs promotes repulsion of these axons to the posterior region of the AOB in response to a high anterior to low posterior Slit1 gradient created by Slit1 expressing cells in the anterior tip of the AOB. IV Kirrel-3 is required for the coalescence of VSN axons into glomeruli. The differential expression of Kirrel molecules on VSN axons creates a molecular code, which dictates their proper coalescence into glomeruli. It is suggested that homophilic interactions between axons expressing similar Kirrel codes drive the coalescence of like axons together into target glomeruli. AOB accessory olfactory bulb, VNO vomeronasal organ

In the main olfactory system, there is strong evidence that fasciculation and sorting of OSN axons within the olfactory nerves impinge on the targeting accuracy of these axons in the olfactory bulb [58, 59]. Similarly, the pre-sorting of VSN axons within the vomeronasal tract that project to the AOB may be one mechanism through which segregation of apical and basal VSN inputs occurs at the AOB. However, severe defasciculation of the vomeronasal tract does not significantly affect the anterior–posterior segregation of VSN axons in the AOB, and a clear segregation of basal VSN axons within the vomeronasal tract in the mouse has not been observed [60, 61]. The tight fasciculation of the vomeronasal tract may instead be required to prevent premature innervation of the OB by VSN axons projecting to the AOB. Indeed, ectopic innervation of the OB by VSN axons has been reported in multiple mouse models where defasciculation of the vomeronasal tract is observed [60, 62–64].

Rather than being pre-sorted within the vomeronasal tract, apical and basal VSN axons appear to differentially respond to a combination of repulsive and attractive cues in the AOB that dictate their segregation within the two regions. The semaphorin–neuropilin complex is one of the ligand–receptor pairs that contribute to this process. Semaphorins are a large family of secreted and membrane-associated proteins that mediate a wide variety of biological processes during development [65]. In some contexts, binding of the class 3 secreted semaphorins to neuropilins, located on growing axons, results in growth cone collapse and axon repulsion [65–68]. The class 3 semaphorin receptor Nrp-2 is selectively expressed in apical VSNs axons, which are repelled in vitro by explants of the posterior half of the AOB, suggesting that this region secretes a chemorepellent capable of preventing entry of apical axons into the posterior AOB (Fig. 2b) [62]. Ablation of Nrp-2 leads to mistargeting of apical VSN axons to the posterior region of the AOB, but does not affect the targeting of basal VSN axons [25, 62]. In situ hybridization experiments have shown that multiple members of the class 3 semaphorin family of secreted chemorepellents are expressed by mitral cells of the AOB but no gradient of expression across the anterior–posterior axis was detected [62]. Removal of either Sema3C or Sema3B expression did not affect the targeting of VSN axons [69]. Sema3F is so far the only member of this family that has been associated with the targeting of VSN axons. Indeed, mistargeting of apical VSN axons in the posterior AOB has been reported in a small subset of Sema3F mutant mice analysed [60]. The mild defects observed in Sema3F mutant mice, combined with the lack of a gradient of expression for secreted semaphorins in the AOB, suggest that additional ligands may also contribute to the Nrp-2-dependent segregation of apical VSN axons to the anterior AOB.

In addition to apical VSN axons being repelled from the posterior AOB due to Nrp-2 activity, apical VSN axons also respond to an attractive signal in the anterior region of the AOB. The ephrin-Eph signalling system contributes to directing the growth of axons in the developing nervous system [70]. Activation of Eph receptors at the cell surface of the growing axon by ephrin ligands located in a target field induces a mode of signalling referred to as “forward signalling”, which often leads to axonal repulsion [71]. In contrast, binding of Eph molecules to ephrin ligands at the surface of growing axons induces “reverse signalling” downstream of ephrin, which can lead to axonal attraction [72]. In the vomeronasal system, apical axons express high levels of the glycosylphosphatidylinositol (GPI)-anchored ephrin-A5 at their surface, while the EphA6 receptor is more highly expressed in the anterior region of the AOB (Fig. 2b) [73]. A subset of EphrinA5 knockout mice exhibits apical VSN axon mistargeting to the posterior AOB defining a requirement for ephrinA5 in VSN axon targeting [73]. Furthermore, evidence from in vitro stripe assays suggests that ephrin-A5-expressing VSN axons prefer to grow on cells expressing EphA6, suggesting that ephrinA5-EphA6 interactions promote attraction [73]. The interaction of the ephrin ligand (ephrin-A5) on axons with the receptor (EphA6) in the target region represents a classic example of ephrin reverse signalling. Considering that ephrin-A5 is a GPI-anchored protein, reverse signalling in VSN axons is likely to require a co-receptor for ephrin-A5. Potential co-receptor candidates include the receptor tyrosine kinase RET, and the neurotrophin receptors p75 and TrkB, which have been shown to act as ephrin-A co-receptors in the motor and visual systems, respectively [74–76].

In contrast to apical VSN axons, which express Nrp-2, basal VSN axons express Robo-2, a receptor for the Slit family of secreted chemorepulsive axon guidance cues (Fig. 2b) [60, 77–79]. Slit-1 mRNA is highly expressed in cells located at the anterior tip of the AOB, suggesting that a high anterior to low posterior gradient of Slit-1 protein is generated in the AOB [60, 77–79]. Furthermore, Slits can repel VSN axons in vitro, suggesting that high levels of Slit-1 present in the anterior AOB prevent Robo-2-expressing basal VSN axons from entering this region [77]. Indeed, ablating Robo-2 expression in VSNs resulted in the improper targeting of basal VSN axons to the anterior region of the AOB, without affecting the segregation of apical VSN axons [79]. In addition, Slit-1, but not Slit-2 or Slit-3, is required for the projection of VSN axons into the posterior region of the AOB [60, 79].

An additional layer of axonal organization has been proposed to exist within the posterior region of the AOB. A subset of basal VSNs expresses members of a family of non-classical class I major histocompatibility Mhc genes, known as H2-Mv genes, which have been shown to regulate VR cell surface expression [80–82]. One member of this family, M10.2, is highly expressed in the basal region of the VNO, and M10.2-positive axons project to the most posterior edge of the posterior half of the AOB. This pattern of expression has been suggested to establish a so-called “tripartite organization” of the AOB with V1R-expressing axons in the anterior AOB, V2R-expressing axons in the posterior AOB, and V2R/H2-Mv-positive axons restricted to the most posterior part of the AOB (Figs. 1b, 2a) [81]. The molecular cues that are involved in specifically targeting axons of V2R/H2-Mv-positive VSNs to the posterior edge of the AOB remain to be identified.

While classical axon guidance molecules play a critical role in the segregation of apical and basal VSN axons within the anterior and posterior regions of the AOB, they are not required for the coalescence of VSN axons within glomeruli, suggesting that other mechanisms regulate this process. VR expression is required for the formation of discrete glomeruli, and deletion of a specific V1R led to the broad dispersion of VSN axons normally expressing this V1R in the anterior AOB [50, 51]. Early after birth, sensory activity can regulate the coalescence and refinement of glomeruli in the AOB. An abnormal increase in VSN activity during this time period results in a delay in coalescence of axons into defined glomerular structures and in exuberant VSN axon projections [83]. Interestingly, in the main olfactory system, the expression of several cell adhesion molecules that affect axonal coalescence is regulated by neuronal activity [84, 85]. Some members of one of these cell adhesion families, the Kirrels, are expressed in VSNs, and their expression is altered in VSNs that lack the TRP2 ion channel [61]. A detailed analysis of the expression of Kirrel family members in VSNs revealed that subsets of VSNs express varying levels of Kirrel-2 and Kirrel-3. The differential pattern of expression creates a molecular code by which axons can identify one another and facilitate like axons to coalesce (Fig. 2b). Ablation of Kirrel-3 led to the formation of larger glomeruli receiving inputs from multiple types of VR in the posterior AOB, demonstrating a role for this family of molecules in VSN axonal coalescence [61]. Considering the complexity of the glomerular map formed in the AOB, it is very likely that additional families of cell adhesion molecules contribute to the formation of a diverse molecular code among VSN axons coalescing into the AOB.

Sensory coding in the vomeronasal system

While genetic and cell labelling studies have provided significant insight into the wiring of the vomeronasal system, an understanding of VSN responses to specific ligands to activate a stereotypic glomerular map in the AOB is critical in revealing how sensory information is encoded in this system. The identification of chemosignals that activate VSNs combined with the ability to monitor electrophysiological responses or changes in calcium levels in either VSNs or mitral cells has provided insight into sensory processing in the AOB. To date, several ligands capable of activating VSNs have been identified including urine-derived small organic molecules [15], sulfated steroids [16, 57, 86], major urinary proteins (MUPs) [87–90], MHC class 1 peptides [91], exocrine gland-secreting peptide 1 (ESP1) [92], ESP22 [93], and N-formylated peptides [32, 33, 94–96].

Some observations support the idea that VSNs have highly tuned response profiles, where individual ligands tend to activate specific VSN subtypes, and individual VSN populations respond to only specific ligands. Firstly, V1R VSNs are activated differently than V2R VSNs. For example, three-dimensional imaging of VSNs in the VNO in response to urine stimulation using objective-coupled planar illumination microscopy revealed that urine selectively activates V1R VSNs [97]. Interestingly, while whole urine samples specifically activate V1R VSNs, a study by Isogai and colleagues exposed differential roles for apical and basal VSNs. They investigated the response profiles of V1R and V2R VSNs to a wide range of animal cues using egr1 as a marker of VSN activation [98]. Results from this study emphasized the difference in activation between V1R VSNs and V2R VSNs and concluded that V2Rs encode information about the identity of the donor animal, such as a conspecific or predator, whereas V1Rs serve to detect its physiological status [98]. Secondly, and more particularly, some individual VSNs have response profiles that are highly tuned to specific ligands. Simultaneous electrophysiological recordings of a wide range of VSNs in response to mouse urine revealed that different subsets of VSNs respond discriminatorily to either male or female urine [99]. Calcium imaging in organotypic slices of the VNO revealed that low concentrations of small organic molecules from urine activate non-overlapping subsets of VSNs, suggesting that VSNs show a high degree of sensitivity and selectivity to a specific ligand [15]. Finally, the male specific peptide, ESP1, is recognized by a single V2R whose deletion abolishes VNO stimulation and ESP1-induced lordosis behavioural responses [14].

While early studies suggested that individual VSNs show a high degree of selectivity in their response to chemosignals, more recent evidence has revealed that VSNs can also display combinatorial responses to chemosignals. A specific V2R, V2r1b, can respond to multiple MHC peptides, while a single MHC peptide can activate at least two different V2Rs [100]. The ESP family peptide ESP5 is recognized by two different V2Rs, and a single V2R can bind to both ESP5 and ESP6 [101]. Sulfated steroids also show significant overlap in activation of V1R family receptors [86]. More specifically, a single sulfated steroid can activate VSNs expressing at least two different V1Rs, while a single V1R can recognize multiple sulfated steroids [102]. The detection of cues that encode individual and strain specific information relies on the combinatorial activation of VSNs [103]. Interestingly, the detection of MUP ligands that can both elicit male aggression and allow discrimination of self from non-self employs different sensory coding strategies to promote these two behaviours. While male–male aggression is induced through a tuned activation of VSNs by specific MUPs, discrimination of self from non-self relies on the detection of a blend of MUP ligands through combinatorial sensory coding [90].

While the identification of individual chemosignals has provided us with a means to begin understanding how VSNs are tuned to these ligands, much remains to be learned about the integration of these signals at the level of the AOB. Electrophysiological studies examining mitral cell responses in awake behaving male mice demonstrated that mitral cells are selectively activated when these mice are in contact with a mouse of a given sex or strain [104]. Urine and saliva from donor animals with different sexual and genetic statuses activate different subsets of mitral cells in the AOB [105]. In addition, activation of a small population of AOB mitral cells is sufficient to reliably encode both sex and strain of a mouse urine donor [106]. Taken together, these results indicate that identification of an animal’s sex and genetic status could be encoded at the level of the AOB. More recently, an elegant study by Hammen et al. using calcium imaging to visualize glomeruli activation in the AOB in response to a wide range of stimuli has provided valuable insight into the organization of the glomerular map in the AOB [107]. Genetic labelling experiments have shown that VSNs expressing closely related VRs innervate nearby and spatially conserved glomeruli within the AOB, suggesting that the glomerular map may be organized to juxtapose inputs of VSNs that recognize ligands with similar structures [52]. However, examination of the pattern of activation of glomeruli in response to specific stimuli revealed that the organization of the AOB glomerular map appears to be based on phenotypic rather than molecular similarity between chemosignals [107]. Indeed, while glomeruli selectively activated by urine of sexually mature male mice are preferentially located near the posterior border of the anterior AOB, glomeruli selective for urine from sexually mature female mice are preferentially clustered at the anterior edge of the anterior AOB (Fig. 3a). In contrast, exposure of VSNs to a variety of sulfated steroids showing a range of molecular similarity revealed that juxtaposed glomeruli do not systematically have similar receptive fields. Furthermore, glomeruli activated by molecularly similar steroid structures do not juxtapose one another [107] (Fig. 3b). These results suggest that glomeruli in the AOB are not organized in a manner that correlates with the similar receptive fields of ligands, but instead by similar phenotypic qualities of the signal donor, such as sex and sexual maturity. Although VRs with similar amino acid sequence homology target closely positioned glomeruli within the AOB, amino acid sequence of a receptor may, therefore, be more important for axonal targeting than for receptor–ligand tuning. Assessing global activation of VSNs and mitral cells in response to a wide range of natural stimuli has helped reveal the logic of sensory coding in the vomeronasal system and suggests that, in contrast to the MOS where sensory input processing takes place at the level of the olfactory cortex, encoding of sensory information takes place at the level of the AOB and is then relayed to the limbic system.

Fig. 3.

Organization of the sensory glomerular map in the AOB. a Chemosignals that share similar phenotypic qualities activate glomeruli that are juxtaposed to each other. For example, urine from male mice activates multiple glomeruli close to the anterior–posterior border of the AOB (green) while female urine activates glomeruli closer to the anterior tip of the AOB (blue). b Ligands that share similar molecular features activate glomeruli that are not necessarily located in close proximity to one another. For example, two sulfated steroids with similar structures belonging to the same chemical family (light and dark pink) activate glomeruli that are far apart. In contrast, sulfated steroids with several structural differences and belonging to different chemical families (light pink and orange) can activate glomeruli that are juxtaposed

Regulation of aggression by the vomeronasal system

The VNO detects chemosignals that regulate a variety of social behaviours, including mating, social dominance, and maternal care [108]. In addition, the vomeronasal system detects conspecific cues which inform the sex and genetic strain of mice, which are important stimuli initiating instinctive behaviours, such as aggression [103–105, 109]. Several studies employing gene inactivation approaches to disrupt VNO function have revealed a critical role for this organ in regulating both male and female aggression. Ablation of the TRP2 ion channel blocks sensory activation of VSNs in response to urine and leads to decreased male-to-male and maternal aggression, along with an inability to discriminate male from female mice [110, 111]. Several studies support a role for both V1R and V2R-expressing VSNs in the control of aggression by the VNO. Deletion of a cluster of V1R genes leads to selective defects in male reproductive behaviours and maternal aggression, suggesting that subsets of VRs can mediate specific behaviours [112]. Blocking V1R signalling through ablation of the trimeric G protein subunit Gαi leads to a reduction in male-to-male aggression in a resident intruder assay [113]. Interestingly, ablation of the V2R-specific Gαo subunit also disrupts male territorial behaviour and maternal aggression [87]. Furthermore, ablation of the G-protein subunit γ8, which is preferentially expressed in V2R VSNs, displays defects in both male and female aggressive behaviours [114]. Although the effects of V2R gene ablation on aggression remain to be assessed, the absence of V2R localization to the dendritic tip of VSNs in β2m mutants is associated with reduced male–male aggression in these mice [82]. Aggressive behaviours in mice have, therefore, been associated with both V1R and V2R functions.

Although the vomeronasal system is thought to mediate pheromone detection and the regulation of aggressive behaviours in mice, there is evidence that the MOS can also contribute to these processes. Loss of the OR signal transduction molecule type 3 adenylyl cyclase expression in mice led to an inability to detect pheromonal cues and to a lack of male–male aggression and male sexual behaviours [115]. Furthermore, Cnga2 mutant mice defective in odour-evoked OSN signalling displayed defects in aggression [116]. Genetic approaches to block the function of the dorsal aspect of the OB have also revealed that this region plays a role in regulating multiple social behaviours in mice, including aggression [8]. It is, therefore, likely that the control of such a vital innate behaviour implicates the detection of a large array of chemosignals that require both the vomeronasal and MOS.

Our understanding of the principles underlying the control of aggression by the vomeronasal system has been greatly improved through the discovery of chemosignals that can induce aggressive behaviours. The detection of urine represents an important means by which mice communicate with one another and can trigger innate behaviours, including aggression. Urine contains organic volatile compounds, such as 2-s-butyl-4,5-dihydrothiazole (SBT), that can promote inter-male aggression [117]. The MUPs are a family of proteins abundantly excreted in mouse urine that exclusively activate V2R-expressing VSNs and that induce male–male aggressive behaviour in mice [87]. The effect of MUPs is dependent on V2R signalling as ablation of Gαo expression leads to a loss of MUP-induced aggression [88]. Two members of this family of proteins, MUP20 (also known as Darcin) and MUP3, are individually sufficient to promote inter-male aggression [90].

Although these studies revealed that isolated MUPs can activate V2R-expressing VSNs, previous observations have shown that low concentrations of urine mainly activate glomeruli located in the anterior region of the AOB, innervated by V1R-expressing VSNs [97, 107]. It remains possible that varying concentrations of MUPs can differentially activate V1R and V2R expressing VSNs. Alternatively, the presence of additional compounds in urine that can bind to MUPs could be responsible for the activation of V1R-expressing neurons by urine. In keeping with this possibility, the MUP binding molecule SBT activates VSNs located in the apical region of the VNO that express V1R family receptors [15]. In addition to acting as a pheromone for intra-species communication, MUPs can act as kairomones to elicit defensive behaviours to predators in inter-species communication [89]. A recent study has also demonstrated that MUPs are sufficient for inducing territorial urine countermarking in mice, which suggests a role for these proteins in discriminating between self and non-self [90].

While the identification of ligands that modulate aggression through the VNO has provided important insight into the regulation of this behaviour, it remains to be determined whether the activation of specific glomerular maps by these ligands is necessary for a behavioural response to take place. In the main olfactory system, disrupting the glomerular map results in a loss of innate avoidance of specific aversive odorants [118]. Interestingly, the improper formation of glomeruli in the posterior region of the AOB in Kirrel-3 mutant mice is associated with a loss of male–male aggression in a resident intruder male assay [61]. Furthermore, Ephrin-A5 mutant mice, which display improper segregation of apical VSN axons in the AOB, also exhibit a severe reduction in conspecific aggression [119]. While these results support an important role for the formation of the AOB glomerular map in the regulation of aggression, further studies combining VNO-specific ablation of these axon guidance proteins and imaging of glomerular activation maps will be needed to conclusively demonstrate that the spatial arrangement of glomeruli in the AOB is necessary to regulate VNO-specific behaviours.

Conclusion

In this review, we discussed our current understanding of the molecular mechanisms that dictate the formation of the glomerular map in the AOB, and we have highlighted the general principles of sensory coding that underlie vomeronasal system regulation of behaviours in rodents. While the field has made enormous headway in understanding sensory coding in the vomeronasal system, the complex blend of chemosignals excreted in mouse urine has hampered obtaining a comprehensive view of the coding logic that regulates social behaviours in mice. For example, the differences observed in the activation of V1R and V2R families of receptors when exposed to urine versus isolated components of urine have made it difficult to assign specific functions to these two families of receptors in encoding sensory information in the vomeronasal system [15, 87, 97–99, 107]. Large-scale identification of the full repertoire of molecular cues excreted by rodents and of the receptors they activate will be necessary to assign definite functions to VR families and to reconcile some of the inconsistencies that exist in the literature. For example, the observations that MHC peptides activate V2r1b-expressing VSNs and that this receptor is necessary for the peptide response conclusively identify a specific ligand–receptor pair in these neurons [100]. The systematic identification of ligand–receptor pairs, combined with receptor loss of function studies, will be essential to identify causative links between chemosignals, their receptors, and altered behavioural outputs in the absence of these receptors. Furthermore, the identification of specific ligands that activate VSNs, in association with imaging approaches to visualize the glomerular map these ligands activate, and with axon guidance receptor gene loss-of-function studies, will help further our understanding of the association that exists between the stereotyped glomerular map that is formed in the AOB, the encoding of information in the vomeronasal system, and the control of social behaviours in rodents.