Signal Detection and Coding in the Accessory Olfactory System

Abstract

In many mammalian species, the accessory olfactory system plays a central role in guiding behavioral and physiological responses to social and reproductive interactions. Because of its relatively compact structure and its direct access to amygdalar and hypothalamic nuclei, the accessory olfactory pathway provides an ideal system to study sensory control of complex mammalian behavior. During the last several years, many studies employing molecular, behavioral, and physiological approaches have significantly expanded and enhanced our understanding of this system. The purpose of the current review is to integrate older and newer studies to present an updated and comprehensive picture of vomeronasal signaling and coding with an emphasis on early accessory olfactory system processing stages. These include vomeronasal sensory neurons in the vomeronasal organ, and the circuitry of the accessory olfactory bulb. Because the overwhelming majority of studies on accessory olfactory system function employ rodents, this review is largely focused on this phylogenetic order, and on mice in particular. Taken together, the emerging view from both older literature and more recent studies is that the molecular, cellular, and circuit properties of chemosensory signaling along the accessory olfactory pathway are in many ways unique. Yet, it has also become evident that, like the main olfactory system, the accessory olfactory system also has the capacity for adaptive learning, experience, and state-dependent plasticity. In addition to describing what is currently known about accessory olfactory system function and physiology, we highlight what we believe are important gaps in our knowledge, which thus define exciting directions for future investigation.

Introduction

Social communication among conspecifics is a crucial prerequisite for evolutionary success. In most mammals, chemical cues have emerged as the predominant “language” for communicating information about individuality, endocrine state, social hierarchy, sexual maturity, and receptivity (Wyatt 2014). Yet, much remains unknown about the underlying chemical “alphabet”, the sensory mechanisms that detect it, and the neural circuits that interpret the information and react upon it to elicit specific behaviors and physiological processes.

Rodents, and mice in particular, have become the model system of choice to study vertebrate chemical communication (Chamero et al. 2012). These species display an exquisite sense of smell and heavily rely on this sensory modality for social communication. In addition, the ever-expanding toolbox available for monitoring and manipulating neuronal activity has made the mouse a particularly attractive model for chemosensory research. Accordingly, this review focuses on chemosensory signaling in rodents, with an emphasis on recent advances that emerged from genetically modified mouse models. We note that although the general features of accessory olfactory system (AOS) function are likely to be common across many vertebrate species, there are undoubtedly aspects that are species-specific, adapted to different ethological niches and lifestyles. Our focus on the rodent AOS does not imply that we consider these differences unimportant. Indeed, the identification of species-specific AOS features can be highly revealing, and a comparative analysis of AOS structure and function across species, orders, and classes will certainly lead to a more complete understanding of AOS function (see Future directions).

Most mammals have evolved several sensory subsystems to detect environmental chemostimuli (Munger et al. 2009). The gustatory system samples the chemical makeup of food for nutrient content, palatability, and toxicity (Roper and Chaudhari 2017), but is not known to play a role in social signaling. The mammalian nose, in contrast, harbors several chemosensory structures that include the main olfactory epithelium, the septal organ of Masera (Rodolfo-Masera 1943), the vomeronasal organ (VNO; Jacobson et al. 1998), and the Grueneberg ganglion (Grüneberg 1973). Together, these structures serve various olfactory functions including social communication.

The VNO plays a central, though not exclusive, role in semiochemical detection and social communication. It was first described in 1813 (more than 200 years ago), by the Danish anatomist Ludwig L. Jacobson, and is thus also known as Jacobson’s organ. From a comparative analysis in several mammalian species, Jacobson concluded that the organ “may be of assistance to the sense of smell” (Jacobson et al. 1998). With the notable exception of humans and some apes, a functional organ is likely present in all mammalian and many nonmammalian species (Silva and Antunes 2017). Today, it is clear that the VNO constitutes the peripheral sensory structure of the AOS. Jacobson’s original hypothesis that the VNO serves a sensory function gained critical support in the early 1970s when parallel, but segregated projections from the MOS and the AOS were first described (Winans and Scalia 1970; Raisman 1972). The observation that bulbar structures in both the MOS and the AOS target distinct telen- and diencephalic regions gave rise to the “dual olfactory hypothesis” (Scalia and Winans 1975). In light of this view, the main and accessory olfactory pathways have been traditionally considered as anatomically and functionally distinct entities, which detect different sets of chemical cues and affect different behaviors. In the past two decades, however, it has become increasingly clear that these systems serve parallel, partly overlapping, and even synergistic functions (Spehr et al. 2006). Accordingly, the AOS should not be regarded as the only chemosensory system involved in processing of social signals. In fact, various MOS divisions have been implicated in the processing of social cues or other signals with innate significance. Several neuron populations residing in the main olfactory epithelium (e.g., sensory neurons expressing either members of the trace amine-associated receptor [TAAR] gene family (Liberles and Buck 2006; Ferrero et al. 2011) or guanylate cyclase-d in conjunction with MS4A proteins [Fülle et al. 1995; Munger et al. 2010; Greer et al. 2016]) detect conspecific or predator-derived chemosignals and mediate robust behavioral responses. Anatomically, there are various sites of potential interaction between the MOS and the AOS, including the olfactory bulb (Vargas-Barroso et al. 2016), the amygdala (Kang et al. 2009; Baum 2012), and the hypothalamus as an integration hub for internal state and external stimuli. A comprehensive description of this issue is beyond the scope of this review, and thus, we refer the reader to several recent articles specifically addressing potential MOS–AOS interactions (Baum 2012; Mucignat-Caretta et al. 2012; Suárez et al. 2012).

Although much remains to be explored, we now have a relatively clear understanding of peripheral and early central processing in the MOS. By contrast, our mechanistic understanding of AOS function is still fragmentary (Box 1). In this review article, we provide an update on current knowledge of the rodent AOS and discuss some of the major challenges lying ahead. The main emphasis of this review concerns the nature of the computations performed by the initial stages of the AOS, namely sensory neurons of the VNO and circuits in the accessory olfactory bulb (AOB).

Box 1 The AOS: an emerging multi-scale model to study how sensory stimuli drive behavior

A key goal in neuroscience is to understand how sensory stimuli are detected and processed to ultimately drive behavior. Given the inherent complexity of the task, attempts to gain a holistic (i.e., multi-scale) analytical perspective on sensory coding have frequently resorted to reductionist approaches in invertebrate model organisms such as nematodes or fruit flies. In such models, the “from-gene-to-behavior” strategy has proven extremely powerful and, accordingly, has led to numerous breakthroughs. In mammals, however, sensory processing pathways are typically more complex, comprising multiple subcortical stages, thalamocortical relays, and hierarchical flow of information along uni- and multimodal cortices. Although MOS inputs also reach the cortex without thalamic relays, the route of sensory inputs to behavioral output is particularly direct in the AOS (Figure 1). Specifically, peripheral stimuli can reach central neuroendocrine or motor output via a series of only four stages. In addition to this apparent simplicity of the accessory olfactory circuitry, many behavioral responses to AOS activation are considered stereotypic and genetically predetermined (i.e., innate), thus, rendering the AOS an ideal “reductionist” model system to study the molecular, cellular, and network mechanisms that link sensory coding and behavioral outputs in mammals.

To fully exploit the benefits that the AOS offers as a multi-scale model, it is necessary to gain an understanding of the basic physiological properties that characterize each stage of sensory processing. With the advent of genetic manipulation techniques in mice, tremendous progress has been made in the past few decades. Although we are still far from a complete and universally accepted understanding of AOS physiology, several aspects of chemosensory signaling along the system’s different processing stages have recently been elucidated. In this article, we aim to provide an overview of the state of the art in AOS stimulus detection and processing. Because much of our current mechanistic understanding of AOS physiology is derived from work in mice, and because substantial morphological and functional diversity limits the ability to extrapolate findings from one species to another (Salazar et al. 2006, 2007), this review is admittedly “mouse-centric.” Thus, some concepts may not directly apply to other mammalian species. Moreover, as we attempt to cover a broad range of AOS-specific topics, the description of some aspects of AOS signaling inevitably lacks in detail. The interested reader is referred to a number of excellent recent reviews that either delve into the AOS from a less mouse-centric perspective (Salazar and Sánchez-Quinteiro 2009; Tirindelli et al. 2009; Touhara and Vosshall 2009; Ubeda-Bañon et al. 2011) and/or address more specific issues in AOS biology in more depth (Wu and Shah 2011; Chamero et al. 2012; Beynon et al. 2014; Duvarci and Pare 2014; Liberles 2014; Griffiths and Brennan 2015; Logan 2015; Stowers and Kuo 2015; Stowers and Liberles 2016; Wyatt 2017; Holy 2018).

The vomeronasal organ

The rodent VNO is a paired cylindrical structure at the base of the anterior nasal septum (Meredith 1991; Halpern and Martinez-Marcos 2003). Just above the palate, the blind-ended tubular organ, enclosed in a cartilaginous capsule, opens anteriorly to the nasal cavity via the vomeronasal duct (Figure 1). Whether the organ is functional at birth or gains functionality during a later developmental stage is still subject to debate (Box 2). In the adult mouse, each VNO harbors approximately 100000 to 200000 vomeronasal sensory neurons (VSNs; Wilson and Raisman 1980), which gain both structural and metabolic support from a band of sustentacular cells in the most superficial layer of a crescent-shaped pseudostratified neuroepithelium. VSNs display a characteristic morphology: as bipolar neurons, they extend a single unbranched dendrite from the apical pole of a small elliptical soma (~5 µm in diameter). The apical dendrites terminate in a paddle-shaped swelling that harbors numerous microvilli at its tip (knob). These microvilli are immersed in a viscous mucus that is secreted by lateral glands and fills the entire VNO lumen. Thus, the microvillar arrangement provides a massive extension of the neuroepithelium’s interface with the external environment. From their basal pole, VSNs project a long unmyelinated axon. At the basal lamina, hundreds of these VSN axons fasciculate into vomeronasal nerve bundles that run in dorsal direction below the septal respiratory and olfactory epithelia. Together with olfactory nerve fibers, VSN axon bundles enter the brain through small fenestrations in the ethmoid bone’s cribriform plate. The vomeronasal nerve then projects along the medial olfactory bulbs and targets the glomerular layer of the AOB (Meredith 1991; Belluscio et al. 1999; Rodriguez et al. 1999).

Figure 1.

 Schematic overview of the mouse AOS. Shown is a sagittal view of a mouse head indicating the locations of the two major olfactory subsystems, including 1) main olfactory epithelium (MOE) and main olfactory bulb (MOB), as well as 2) the vomeronasal organ (VNO) and accessory olfactory bulb (AOB). Not shown are the septal organ and Grueneberg ganglion. The MOE lines the dorsolateral surface of the endoturbinates inside the nasal cavity. The VNO is built of two bilaterally symmetrical blind-ended tubes at the anterior base of the nasal septum, which are connected to the nasal cavity by the vomeronasal duct. Apical (red) and basal (green) VSNs project their axons to glomeruli located in the anterior (red) or posterior (green) aspect of the AOB, respectively. AOB output neurons (mitral cells) project to the vomeronasal amygdala (blue), from which connections exist to hypothalamic neuroendocrine centers (orange). The VNO resides inside a cartilaginous capsule that also encloses a large lateral blood vessel (BV), which acts as a pump to allow stimulus entry into the VNO lumen following vascular contractions (see main text). In the diagram of a coronal VNO section, the organizational dichotomy of the crescent-shaped sensory epithelium into an “apical” layer (AL) and a “basal” layer (BL) becomes apparent.

Box 2 VNO ontogeny

The mouse vomeronasal neuroepithelium is derived from an evagination of the olfactory placode that occurs between embryonic days 12 and 13 (Cuschieri and Bannister 1975). As a marker for VSN maturation, expression of the olfactory marker protein is first observed by embryonic day 14 (Tarozzo et al. 1998). In general, all structural components of the VNO appear present at birth, including lateral vascularization (Szabó and Mendoza 1988) and vomeronasal nerve formation. However, it is unclear whether the organ is already functional in neonates. Although previous observations suggested that it is not (Coppola and O’Connell 1989), others recently reported stimulus access to the VNO via an open vomeronasal duct at birth (Hovis et al. 2012). Moreover, formation of VSN microvilli is complete by the first postnatal week (Mucignat-Caretta 2010), and the presynaptic vesicle release machinery in VSN axon terminals also appears to be fully functional in newborn mice (Hovis et al. 2012). Thus, the rodent AOS might already fulfill at least some chemosensory functions in juveniles (Mucignat-Caretta 2010).

At the molecular level, regulation of VSN development is still poorly understood. Bcl11b/Ctip2 and Mash1 are transcription factors that have been recently implicated as crucial for VSN differentiation (Murray et al. 2003; Enomoto et al. 2011). In Mash1-deficient mice, profoundly reduced VSN proliferation is observed during both late embryonic and early postnatal stages (Murray et al. 2003). By contrast, Bcl11b/Ctip2 function appears to be restricted to postmitotic VSNs, regulating cell fate among newly differentiated VSN subtypes (Enomoto et al. 2011).

On its lateral side, the VNO is composed of highly vascularized cavernous tissue. A prominent large blood vessel provides a characteristic anatomical landmark (Figure 1). In his original publication, Jacobson already noted the rich innervation of the organ’s lateral aspects (Jacobson et al. 1998). Most of these sympathetic fibers originate from the superior cervical ganglion, enter the posterior VNO along the nasopalatine nerve, and innervate the large lateral vessel (Meredith and O’Connell, 1979; Eccles, 1982; Ben-Shaul et al., 2010). Although in several species vomeronasal stimulus uptake is presumably accompanied by the Flehmen response, in rodents, vomeronasal activation is not readily apparent to an external observer. Indeed, due to its anatomical location, it has been extremely challenging to determine the precise conditions that trigger vomeronasal stimulus uptake. The most direct observations stem from recordings in behaving hamsters, which suggest that vomeronasal uptake occurs during periods of arousal. The prevailing view is that, when the animal is stressed or aroused, the resulting surge of adrenalin triggers massive vascular vasoconstriction and, consequently, negative intraluminal pressure. This mechanism effectively generates a vascular pump that mediates fluid entry into the VNO lumen (Meredith et al. 1980; Meredith 1994). In this manner, low-volatility chemostimuli such as peptides or proteins gain access to the VNO lumen following direct investigation of urinary and fecal excretions, vaginal or facial gland secretions (Wysocki et al. 1980; Luo et al. 2003), or other stimulus sources. Interestingly, solitary chemosensory cells have been identified near the opening of the VNO duct, suggesting that they could play a role in regulating VNO function (Ogura et al. 2010). However, our knowledge regarding the behavioral contexts that trigger activation, and whether it is entirely reflex or rather accessible to voluntary control, is still limited and certainly warrants further investigation (see Future directions).

Similar to gustatory and olfactory neurons, which are also constantly exposed to the external chemical environment (including a variety of potentially harmful xenobiotics), VSNs are short lived and thus continuously replenished from a local stem cell reservoir. This life-long regenerative capacity (Brann and Firestein 2010) is maintained by basal cells, a group of pluripotent neural stem cells predominantly located in the marginal proliferation zone (Halpern and Martinez-Marcos 2003).

Vomeronasal stimuli

The physiological function of the VNO has been frequently described as a specialized detector for “pheromones.” The term for this somewhat enigmatic class of chemical cues (in Greek, “pherin” is “to transfer” and “hormάn” is “to excite”) was originally coined by Karlson and Lüscher almost 60 years ago. According to their definition, “pheromones are substances that are secreted by one individual and received by a second individual of the same species, in which they release a specific reaction, for example, a definite behavior or a developmental process” (Karlson and Lüscher 1959). Although this definition properly applies to many insect chemostimuli, it often falls short when applied to mammalian social chemosignals. Indeed, this issue has sparked some intense debate in the past (Doty 2010; Wyatt 2014).

Today, it is clear that the VNO is not exclusively dedicated to “pheromone detection.” For one, the VNO is critical for detection of predator odors, which are formally distinct from pheromones, and rather defined as “kairomones” (see below). Similarly, in snakes the VNO is important for prey detection (Halpern and Frumin 1979). Furthermore, contrary to the original definition of pheromones, many of the social chemosignals that robustly activate the AOS are not single compounds, but rather species-specific or individual-specific combinations of molecules in precise ratios (Wyatt 2009). Indeed, whereas pheromones are defined as intraspecies social signals that are “anonymous” with respect to the sender, many of the signals detected by the VNO serve to convey information about individuality (Hurst et al. 2001; Leinders-Zufall et al. 2004; Kaur et al. 2014; Ben-Shaul 2015). These include signature mixtures, which allow individuals or other social groups (e.g., families or colonies) to be recognized and distinguished. Finally, although pheromones, by strict definition, elicit a fixed and well-defined response, behavioral changes in response to many AOS signals can require learning and plasticity (Kaur et al. 2014; Xu et al. 2016), concepts that were long considered inapplicable to the AOS.

One fundamental question concerns the distinction between the AOS and MOS, and specifically in this context, the difference between stimuli that each of these systems has evolved to detect. Indeed, this was recently suggested as one of the key distinctions between the two systems (Holy 2018). Although obviously the MOS is more suitable for volatile airborne stimuli, whereas the AOS is suitable for the detection of larger nonvolatile yet soluble ligands, this is by no means a strict division of labor, as some stimuli are clearly detected by both systems. In fact, any chemical stimulus presented to the nasal cavity might also be detected by the MOS, complicating the identification of effective AOS ligands via behavioral assays alone. Thus, the most direct approach to identity AOS stimuli involves measurement of neuronal responses in various structures along the vomeronasal pathway. Because it is difficult to achieve full control of stimulus delivery in behaving animals, especially in the case of the AOS, most knowledge about effective AOS stimuli emerges from physiological studies using reduced preparations or recordings from intact anesthetized animals.

In search of AOS ligands, it is important to distinguish responses to natural stimuli (which normally contain numerous components) from signals evoked by identified molecular components of such natural stimuli. Clearly, identification of the latter is considerably more challenging. Effective stimuli are often associated with excretions, for example, urine and feces, as well as fluids emanating from skin, or specialized glands (e.g., lacrimal, Harderian, submaxillary, vaginal, preputial, and salivary) (Albone 1984). It thus comes as no surprise that social investigatory behavior in mice primarily involves periods of intense licking and sniffing of both facial and anogenital regions (Luo et al. 2003). By far, the most frequently studied bodily secretion in animal chemosensory research is urine (Krieger et al. 1999; Pankevich et al. 2004; Brann and Fadool 2006; Chamero et al. 2007; Zhang et al. 2007; He et al. 2008; Martel and Baum 2008; Nodari et al. 2008; Ben-Shaul et al. 2010; Meeks and Holy, 2010; Yang and Delay 2010; Li et al. 2013; Tolokh et al. 2013; Kaur et al. 2014; Cichy et al. 2015). This is due to the well-established role of urinary signals in social communication, but also to the ease of collecting large quantities of this rich source of semiochemicals.

In vivo recordings from the AOB and the medial amygdala revealed that a large proportion of neurons respond to predator cues, many of them exclusively, and in a species-specific manner (Ben-Shaul et al. 2010; Bergan et al. 2014). Furthermore, predator cues robustly activate VSNs and, consistently, vomeronasal lesions impair murine responses to predator cues (Papes et al. 2010; Isogai et al. 2011). Indeed, the proportion of VSNs apparently dedicated to heterospecific cues is substantial. Presenting soiled bedding from several different species including mammalian, avian, and reptile predators, it was shown, somewhat surprisingly, that approximately one-third of male mouse VSNs were activated by a mixture of heterospecific cues, whereas only ~7% of all neurons responded to bedding from female conspecifics (Isogai et al. 2011). One implication of these studies is that the AOS cannot be considered as a system exclusively designed for processing “pheromones” (even when the most permissive definition is applied), because cues from other organisms do not fall under this definition. For example, semiochemicals that mediate interspecific interactions by benefitting the receiver, while providing a behavioral disadvantage to the emitter, are defined as “kairomones” (Wyatt 2017).

Chemically, semiochemicals cover many structural groups and dimensions (Wyatt 2017). Prominent chemosignals in the low- and high-molecular weight fractions of mouse urine are sulfated steroids (Nodari et al. 2008), which could reflect the dynamic endocrine state of an individual, and members of the major urinary protein (MUP) family (Hurst et al. 2001), respectively. In addition, several other small volatiles (Novotny 2003; Röck et al. 2006) and a plethora of peptides, including those that function as major histocompatibility complex (MHC) class I peptide ligands (Sturm et al. 2013; Overath et al. 2014), are found in urine.

Recently, it was shown that members of the exocrine gland–secreting peptide (ESP) family serve as semiochemicals in tear fluid (Kimoto et al. 2005; Haga et al. 2010). Like MUPs, the 38 rodent ESPs have undergone species-specific gene duplications (Kimoto et al. 2007; Logan et al. 2008). The founding family member, ESP1, is a striking example of a sex-specific male pheromone. In an experimental tour de force that lasted more than a decade, the Touhara laboratory has revealed the complete ESP1-dependent sensory pathway. This pathway begins with the molecule (Kimoto et al. 2005) and its cognate vomeronasal receptor (Haga et al. 2007); continues with the first, second, and third stages of AOS central processing (Ishii et al. 2017); and ends with a stereotyped response in female mice: lordosis (Haga et al. 2010). Although ESP1 is clearly effective in the context of other sensory cues associated with mating behaviors, it remains unclear whether it is sufficient by itself to trigger lordosis (Woodson et al. 2017).

Expression of another member of the ESP family, ESP22, is dramatically age-dependent. The concentration of ESP22 in tear fluid increases in juvenile mice during the first postnatal weeks but drops sharply with puberty. By activating VSNs, ESP22 is sufficient to inhibit sexual displays from adult males (Ferrero et al. 2013). Presumably, this inhibitory signaling system has evolved to suppress male sexual behavior toward reproductively futile targets such as juvenile conspecifics (Yang and Shah 2016).

As mentioned earlier, one important class of AOS ligands is the MUPs, which are encoded by 21 polymorphic loci in the mouse genome (Logan et al. 2008; Mudge et al. 2008). Following their synthesis in the liver, MUPs are excreted in urine. Notably, expression of these lipocalin proteins has been observed in several secretory tissues and fluids (Finlayson et al. 1965; Stopka et al. 2016). Given their β-barrel structure that forms an internal ligand-binding pocket, MUPs efficiently bind small urinary molecules. Accordingly, they might not only function as genuine VSN stimuli (Chamero et al. 2007), but also could serve as storage sites or carrier proteins for otherwise short-lived volatile signals (Hurst and Beynon 2004). Individual males express a discrete subset of 4–12 of the MUPs that remain stable throughout their lifetime (Robertson et al. 1997) and provide a unique chemosensory signature. MUPs regulate diverse behaviors with different sensory-coding strategies. Some dedicated ligands, including MUP20 (also known as Darcin [Roberts et al. 2010]), promote male-specific territorial aggression in a “hard-wired” (i.e., experience-independent) but context-dependent manner (Chamero et al. 2007; Kaur et al. 2014). By contrast, another behavior, male countermarking, depends on a specific blend of MUP molecules (Kaur et al. 2014). This blend provides a chemosensory signature of “self” that serves as a combinatorial code, which depends on previous sensory experience. Darcin is arguably the most prominent member of the MUP family. It is highly attractive to females, facilitates conditioned place preference, and thus acts as a potent stimulus for single-trial social learning (Roberts et al. 2012). Interestingly, Darcin has recently been shown to also stimulate female hippocampal neurogenesis and cell proliferation in the subventricular zone (Hoffman et al. 2015). Given its dual function as 1) an aggression-promoting stimulus to males and 2) an attractant to females, Darcin is ideally suited to shed light on sex-specific differences in AOS signaling.

Subtractive gas chromatography–mass spectrometry of samples from intact versus castrated males identified several volatile androgen-dependent urinary cues (Novotny et al. 1999). Many of these compounds, including 3,4-dehydro-exo-brevicomin, 6-hydroxy-6-methyl-3-heptanone (HMH), 2-sec-butyl-4,5-dihydrothiazole (SBT), and α/β-farnesene, act as potent VSN stimuli in vitro (Leinders-Zufall et al. 2000). Although HMH, SBT, and α/β-farnesene were reported to promote female puberty acceleration (Jemiolo and Novotny 1994; Novotny et al. 1999), more recent analysis failed to reproduce these findings (Flanagan et al. 2011). Of several other small molecules found in urine (Schwende et al. 1984; Jemiolo and Novotny 1994), two (2,5-dimethylpyrazine and 2-heptanone) were shown to activate chemosensory neurons (Leinders-Zufall et al. 2000; Boschat et al. 2002; Mamasuew et al. 2011) and to be involved in puberty onset regulation and in signaling estrus, respectively. Several of these and other (putative) semiochemicals are metabolic by-products of common biochemical pathways. For example, 2-heptanone and α/β-farnesene also direct social behavior in several evolutionarily diverse species, including insects (Stowers and Spehr 2014). To achieve species-specific bioactivity, these molecules are likely to function as components of chemical blends.

Our present understanding of the vomeronasal stimulus space is far from complete. Even if each of the ~300 types of vomeronasal receptors (see Vomeronasal chemoreceptors) evolved to detect only one type of molecule (a scenario that, given several recent reports (He et al. 2008, 2010; Kaur et al. 2014), seems highly unlikely), the aforementioned small molecules, peptides and proteins, would still represent just the tip of the iceberg. Some promising candidates for additional VNO stimuli include cues associated with an individual’s pathogenic state (Boillat et al. 2015), such as formylated peptides and other inflammation-related ligands (Rivière et al. 2009; Bufe et al. 2015), or unconjugated bile acids recently identified from mouse fecal extracts (Doyle et al. 2016; Doyle and Meeks 2018).

Vomeronasal chemoreceptors

At least in rodents, the AOS shows a structural, and hence likely a functional, division (Dulac and Torello 2003; Halpern and Martinez-Marcos 2003; Mucignat-Caretta 2010), with a clear non-homogeneous distribution of signal transduction pathways. Specifically, probes for the G protein α-subunit G_αi2 and for the phosphodiesterase isoform PDE4A preferentially label VSNs in the more apical layer of the epithelium (Shinohara et al. 1992; Berghard and Buck 1996; Lau and Cherry 2000). By contrast, cells in the basal layer of the epithelium are G_αo-positive and thus molecularly distinct (Berghard and Buck 1996; Tanaka et al. 1999). Although these descriptors imply a clear topographic segregation, the spatial distinction between apical and basal neurons is by no means absolute (Leinders-Zufall et al. 2000). Thus, a VSN’s cellular identity cannot be determined merely by its position in the epithelium, its dendritic length, or any other obvious anatomic hallmark. This reservation notwithstanding, for simplicity, we use the terms “basal” and “apical” to refer to the G_αo- and G_αi2-expressing cell populations, respectively.

Currently known vomeronasal receptors belong to one of three gene families, Vmn1r, Vmn2r, and Fpr-rs (Bear et al. 2016; Silva and Antunes 2017) (Figure 2). Members of all three families are predicted to share a seven-transmembrane domain topology and, accordingly, encode G protein–coupled receptors (GPCRs). Notably, expression of all members of each of the three vomeronasal receptor families—the V1Rs, V2Rs, and FPR-rs proteins—is restricted to either the G_αi2- or the G_αo-expressing cell populations (the one known exception being FPR-rs1; see below). Thus, receptor expression profiles support the notion of distinct VSN populations.

Figure 2.

 Diagram illustrating the current model of VSN primary signal transduction. Known vomeronasal chemoreceptors—formyl peptide receptor-like (FPR-rs) proteins, V1R, and V2R receptors—initiate G protein–coupled phospholipase C type β (PLCβ) signaling that results in phosphoinositide turnover and elevations in both inositol 1,4,5-trisphosphate (IP₃) and diacylglycerol (DAG). Notably, a given VSN only expresses one member of either receptor family and, accordingly, either G_αi2 or G_αo. DAG-mediated Ca²⁺ entry via transient receptor potential canonical type 2 (TRPC2) channels underlies initial depolarization as well as gating of a Ca²⁺-activated Cl⁻ channel (anoctamin1 [ANO1]). Bound to calmodulin (CaM), Ca²⁺ also triggers negative feedback inhibition of TRPC2.

With the notable exception of seven highly homologous V2R proteins that constitute a distinct small subgroup (family-C) among the Vmn2r phylogenetic tree, all other putative vomeronasal chemoreceptors are expressed in monogenic, in fact, monoallelic fashion (Belluscio et al. 1999; Rodriguez et al. 1999; Liberles et al. 2009; Rivière et al. 2009). For those few receptors that, to date, allow immunolabeling, protein enrichment in VSN dendritic tips—that is, at the site of ligand interaction—strongly supports a role in VNO sensory signaling. Consistent with this, Vmn1r, Vmn2r, and Fpr-rs gene expression is VSN specific (Dulac and Axel 1995; Herrada and Dulac 1997; Matsunami and Buck 1997; Ryba and Tirindelli 1997; Liberles et al. 2009; Rivière et al. 2009).

The number of members in each of the three families of vomeronasal receptors varies considerably. Although the V1R and V2R receptor families each include more than 100 potentially functional members (Rodriguez et al. 2002; Roppolo et al. 2007; Young and Trask 2007), there are only five identified vomeronasal FPR-rs receptors (Liberles et al. 2009; Rivière et al. 2009). Like odorant receptors, TAARs, and T2R bitter-taste receptors, V1Rs and formyl peptide receptors (FPRs) are class-A, rhodopsin-like GPCRs. By contrast, V2Rs are typical class-C receptors (i.e., glutamate receptor like), which have a large hydrophobic amino (N)-terminus, frequently referred to as a “Venus flytrap” module. This module likely forms the extracellular ligand–binding domain (Mombaerts 2004; Spehr and Munger 2009).

One of the major goals in olfactory research in general, and in the context of the AOS particularly, is to identify ligands for specific receptors. However, attempts to express recombinant V1R and V2R receptor proteins in heterologous systems have largely failed. Thus, a simple preparation for systematic screening for potential ligands and corresponding structure–function data are lacking for both V1Rs and V2Rs, creating a major bottleneck in VNO-signaling research. Notably, it was shown that a chaperone common to many heterologous cell lines negatively regulates functional V2R expression (Dey and Matsunami 2011). This finding raises hope that inhibition or replacement of this chaperone will enable efficient trafficking of recombinant V2Rs, making the determination of cognate receptor–ligand pairs and structure–function relationships feasible. Alternative approaches to identify receptor–ligand interactions include combining immediate early gene expression with in situ expression to identify receptor clades (Isogai et al. 2011), or Ca²⁺ imaging followed by reverse transcription polymerase chain reaction to identify individual receptors (Haga-Yamanaka et al. 2014). The latter approach has been cleverly used to ectopically express individual vomeronasal receptors, allowing characterization of stimulus-induced responses in VSNs expressing identified receptors (Haga-Yamanaka et al. 2015).

Vomeronasal type-1 receptors

Initial searches for the elusive vomeronasal chemoreceptors were based on the assumption of homology to odorant receptors. However, these attempts failed until Dulac and Axel generated cDNA libraries from single rat VSNs and identified VNO-specific receptors by differential screening (Dulac and Axel 1995). This strategy uncovered the Vmn1r gene family, which, in mice, contains more than 150 potentially functional members, as well as a comparable number of predicted pseudogenes (Rodriguez et al. 2002; Roppolo et al. 2007). In situ hybridization revealed punctate, nonoverlapping patterns of Vmn1r transcripts that were confined to the apical G_αi2-/PDE4A-positive layer of the neuroepithelium (Dulac and Axel 1995). Vmn1r genes are unusually divergent and polymorphic, giving rise to 12 relatively isolated gene families, each containing between just one and up to 30 members (Rodriguez et al. 2002; Zhang et al. 2004). Typically organized in small clusters found on most chromosomes, Vmn1r genes share intron-free coding regions (Roppolo et al. 2007; Capello et al. 2009).

Vmn1r gene expression adheres to the “one neuron–one receptor” rule (Serizawa et al. 2004) and is therefore tightly controlled. Monoallelic expression ensures that each VSN displays a single V1R receptor type (Rodriguez et al. 1999), thus achieving a distinct functional identity. Although the molecular mechanisms that ensure strict monoallelic expression of most chemoreceptors have yet to be unraveled, considerable progress in understanding odorant receptor gene choice has recently been made in the MOS (Magklara et al. 2011; Vassalli et al. 2011; Clowney et al. 2012; Plessy et al. 2012; Fuss et al. 2013; Lyons et al. 2013; Colquitt et al. 2014; Markenscoff-Papadimitriou et al. 2014; Abdus-Saboor et al. 2016; Movahedi et al. 2016; Sharma et al. 2017). It remains to be determined whether similar mechanisms regulate VSN expression. Some insight into the underlying mechanisms was provided by studying the regulation of Vmn1r expression (Roppolo et al. 2007). On the basis of the typically uninterrupted sequence of Vmn1r genes within a given cluster, it was hypothesized that this arrangement could allow gene choice regulation at the cluster level. As previously observed for odorant receptors (Serizawa et al. 2003; Lewcock and Reed 2004), transcription of a mutant Vmn1r allele allows coexpression of a second, functional Vmn1r gene. Once a functional Vmn1r transcript is chosen, however, an unknown negative feedback signal maintains monoallelic expression. Remarkably, the initial loss-of-function transcript silences the entire Vmn1r gene cluster in cis (i.e., from the same chromosome). The gene exclusion mechanism, however, is permissive to alternative Vmn1r choice in trans (Roppolo et al. 2007), even if the locus in trans had been mutated to encode an odorant receptor gene (Capello et al. 2009). These findings indicate that the mechanisms underlying monogenic/monoallelic transcription in chemosensory neurons might follow a common molecular logic in both the MOS and the AOS.

The first causal link between the expression of specific V1Rs and VSN chemoreceptivity was demonstrated by deleting a single 600 kb Vmn1r gene cluster on mouse chromosome 6 (Del Punta et al. 2002a), which contains all but one member of the V1ra and V1rb gene families. Comparison of field potential recordings from the VNO surface from wild type and cluster-deleted mice revealed that receptor deletion abolished responses to three (HMH, n-pentylacetate, and isobutylamine) of eight compounds that elicited robust signals in wild type mice. More recently, establishing a high-throughput method for VNO activity mapping by expression profiling of the immediate early gene Egr1, the responses of 56 individual V1Rs to a range of ethologically relevant complex cues were determined (Isogai et al. 2011). This study revealed that nearly half of all V1Rs respond to cues with apparently conflicting ethological significance. Such response patterns can be due to selective responses to compounds that are widely represented in different natural stimuli, or to a broader response profile of individual VSNs.

Presently, direct matching of V1Rs to specific molecules was only accomplished in a few of cases. Vmn1r49 (also known as V1rb2) is activated by 2-heptanone (Boschat et al. 2002). Vmn1r89 (V1rj2) and Vmn1r85 (V1rj3) were repeatedly identified in VSNs activated by two distinct sulfated estrogens: 1,3,5(10)-estratrien-3,17β-diol disulfate, and 1,3,5(10)-estratrien-3,17β-diol 17-sulfate (Isogai et al. 2011; Haga-Yamanaka et al. 2014, 2015). Vmn1r89 apparently also responds to 5-androstene-3β,17β-diol disulfate, whereas Vmn1r226 (V1re2) has been matched to corticosterone-21 sulfate (Isogai et al. 2011). Interestingly, VSNs expressing either Vmn1r85 or Vmn1r89 were also sensitive to urine from female mice in estrus, suggesting that release of sulfated estrogens in urine could signal receptivity. Substantial recent advances in odorant receptor–ligand matching in vivo (McClintock et al. 2014; Jiang et al. 2015; von der Weid et al. 2015) hold great promise for more rapid future progress in identifying Vmn1r–ligand pairs.

Vomeronasal type-2 receptors

Two years after the discovery of V1Rs, three groups concomitantly identified a second multigene family that encodes GPCRs selectively expressed in the VNO (Herrada and Dulac 1997; Matsunami and Buck 1997; Ryba and Tirindelli 1997). Designated as V2Rs, these receptors are expressed in the basal G_αo-positive layer of the VNO sensory epithelium. Given their large putative extracellular ligand–binding site, V2Rs are predicted to preferentially detect large nonvolatile peptides and proteins.

The mouse genome harbors about 280 Vmn2r loci distributed over most chromosomes. Bioinformatic analysis indicates that approximately 120 of these include intact coding regions, whereas the remaining loci are pseudogenes (Munger et al. 2009; Young and Trask 2007). The Vmn2r genes do not share significant sequence homology with the Vmn1r family, but do show a distant phylogenetic relation to metabotropic glutamate receptors, Ca²⁺-sensing receptors, and T1r taste receptor genes (Dulac and Torello 2003; Mombaerts 2004). Unlike the many isolated Vmn1r subfamilies, individual Vmn2r genes group into only four families, designated as A, B, C, and D (Silvotti et al. 2007, 2011; Young and Trask 2007). The vast majority of Vmn2r genes (more than 100) belong to family-A, whereas only four genes constitute family-D.

The proteins encoded by family-C Vmn2r genes (also known as the V2r2 family) are a notable exception to the “one neuron–one receptor” rule. With seven highly homologous members (>80% sequence identity), at least one representative of this group is constitutively coexpressed in most, if not all, G_αo-positive basal VSNs (Martini et al. 2001). Reminiscent of the atypical Orco protein that functions as a mandatory co-receptor in insect olfactory neurons (Larsson et al. 2004; Trible et al. 2017; Yan et al. 2017), coexpression of family-C Vmn2r genes effectively allows for combinatorial V2R expression patterns. Whether family-C receptors serve as chaperoning dimerization partners for a ligand-specific V2R subunit (as postulated for Orco) remains to be determined.

The V2R-positive layer of basal VSNs is further subdivided into two populations according to the absence or presence of nonclassical class Ib MHC genes, known as H2-Mv or M10 (Ishii et al. 2003; Loconto et al. 2003). Although H2-Mv proteins were initially proposed to serve a chaperone function for V2R trafficking (Dulac and Torello 2003; Loconto et al. 2003), later studies showed that 1) a substantial fraction of V2R-expressing neurons lack H2-Mv transcripts (Ishii and Mombaerts 2008) and that 2) basal VSNs retained chemoresponsivity, albeit reduced, after H2-Mv gene cluster deletion (Leinders-Zufall et al. 2014). Nonetheless, the nonrandom combinatorial coexpression of one family-A/B/D V2r gene with a single family-C gene and either none or one of the nine H2-Mv genes is likely to bestow a unique functional phenotype on any given basal VSN (Chamero et al. 2012).

Presently, only few V2Rs were directly shown to confer VSN chemoreceptivity to specific ligands. Loss-of-function mutations in the Vmn2r26 (V2r1b) or Vmn2r116 (V2rp5) genes result in severely reduced sensitivity to two behaviorally relevant peptide ligands, which in wild type mice elicit robust responses at the low nanomolar to high picomolar range (Kimoto et al. 2005; Leinders-Zufall et al. 2009). Specifically, Vmn2r26 deficiency diminishes VSN responses to MHC class I peptide stimuli (Leinders-Zufall et al. 2009), whereas knockout of Vmn2r116 disrupts responses to the male-specific pheromone ESP1 (Haga et al. 2010).

Formyl peptide receptor–like proteins

Following the discovery of the Vmn1r and Vmn2r chemoreceptor genes, 12 years passed before a third family of putative VNO receptors was identified. In parallel large-scale GPCR transcript screenings, two groups independently uncovered a small family, comprising five VNO-specific genes (Fpr-rs1, rs3, rs4, rs6, and rs7) that encode members of the FPR-like protein family (Liberles et al. 2009; Rivière et al. 2009). The founding family member, FPR1, and its close relative FPR2 (also known as FPR-rs2) are expressed by neutrophils, monocytes, and other phagocytic leukocytes of the innate immune system (Le et al. 2002). Vomeronasal Fpr loci form a single gene cluster that is located adjacent to a region encoding Vmn1r and Vmn2r genes. Neither immune nor vomeronasal Fpr genes share significant sequence similarity with other chemosensory GPCRs.

The immune Fpr gene products, FPR1 and FPR2, function as sensors for various chemoattractants that guide the phagocytic immune cells to sites of pathogen invasion and inflammation (Soehnlein and Lindbom 2010). Strikingly, immune FPRs are highly promiscuous, responding to an unusually broad range of bacterial metabolites, mitochondrial peptides, and a variety of antimicrobial/inflammatory modulators (Kolaczkowska and Kubes 2013). Neither of the two immune FPRs is expressed by VSNs (Liberles et al. 2009; Rivière et al. 2009), but FPR3 (i.e., FPR-rs1) is found in both immune cells and VSNs, suggesting that it may play a distinct role in each system (Stempel et al. 2016). The Fpr-rs3, 4, 6, and 7 genes are selectively found in VNO neurons and may be thus designated as vomeronasal FPRs. Indeed, they fulfill all criteria for chemosensory GPCRs: putative seven-transmembrane topology, monogenic and punctate transcription patterns, and at least for FPR-rs3, enriched localization at VSN dendritic tips (Rivière et al. 2009). With the exception of FPR3, which is coexpressed with G_αo in “basal” VSNs, vomeronasal Fpr-rs transcripts are confined to the G_αi2-positive apical epithelial layer (Munger 2009).

Recombinant FPR3 is activated by W-peptide, a synthetic ligand for the known immune FPRs (Bufe et al. 2012). Although two studies somewhat disagreed on the general issue of ligand selectivity, both find that FPR3, when expressed in heterologous cells, is essentially insensitive to the prototypical immune FPR agonist N-formyl-methionyl-leucyl-phenylalanine (fMLF) or to the inflammatory lipid mediator lipoxin A4 (Rivière et al. 2009; Bufe et al. 2012).

Activation profiles of FPR-rs3, 4, 6, and 7 are far less clear. On one hand, recombinant receptors were reported to respond to fMLF (FPR-rs4, 6, 7), lipoxin A4 (FPR-rs4), the antimicrobial peptide CRAMP (FPR-rs3, 4, 6, 7), and an immunomodulatory peptide derived from the urokinase-type plasminogen activator receptor (FPR-rs6) (Rivière et al. 2009). Furthermore, VSNs are activated in situ by fMLF and mitochondria-derived formylated peptides (Chamero et al. 2011) as well as by other agonists of immune system FPRs (Rivière et al. 2009). Also consistent with a role for the AOS in pathogen detection (Stempel et al. 2016), avoidance of sick conspecifics in mice is mediated by the vomeronasal pathway (Boillat et al. 2015). Yet, other studies failed to detect activation of vomeronasal FPRs (FPR-rs3, 4, 6, 7) by peptide agonists of immune FPRs, suggesting that these receptors adopted entirely new functions in VSNs (Bufe et al. 2012). Clearly, further research is required to fully reveal the biological functions of vomeronasal FPRs.

VSN transduction

How is receptor activation transformed into VSN activity? Following stimulus binding to V1R, V2R, or FPR receptors at the luminal interface of the sensory epithelium, G-protein activation triggers complex biochemical cascades that ultimately result in ion channel gating and a depolarizing transduction current. If above threshold, the resulting receptor potential leads to the generation of action potentials, which are propagated along the vomeronasal nerve to the AOB.

Given their extraordinarily high input resistance of several gigaohms (Liman and Corey 1996; Shimazaki et al. 2006; Ukhanov et al. 2007; Hagendorf et al. 2009), VSNs are exquisitely sensitive to electrical stimulation, with only a few picoamperes of transduction current sufficing to generate repetitive discharge. Accordingly, electrophysiological examinations of VSN responses to natural chemostimuli frequently record rather small currents (Yang and Delay 2010; Kim et al. 2011, 2012). In olfactory sensory neurons, input resistance is similarly high. Paradoxically, however, these neurons often generate transduction currents of several hundred picoamperes (Ma et al. 1999; Fluegge et al. 2012; Bubnell et al. 2015), which effectively inhibit action potential firing because voltage-gated Na⁺ channels remain locked in an inactivated state (Catterall 2000). To date, the physiological significance of this discrepancy in transduction current magnitude between the two types of chemosensory neurons, if any, remains elusive. Interestingly, there is a wide range of recorded VSN resting membrane potentials with values ranging from −60 to −75 mV (Liman and Corey 1996; Ukhanov et al. 2007; Cichy et al. 2015). It is presently not clear whether this diversity is due to differences in experimental conditions, to heterogeneity between different VSN subpopulations, or to inherent variation between VSNs.

Primary transduction cascade

From the strictly layer-specific and mutually exclusive coexpression of G_αi2 and G_αo in V1R- and V2R-expressing VSNs, respectively (Halpern et al. 1995), a functional role of both G-protein α-subunits was taken for granted. However, direct proof of this postulation has only emerged recently, and so far only for G_αo (Chamero et al. 2011). Previous constitutive knockout of either G_αi2 (Norlin et al. 2003) or G_αo (Tanaka et al. 1999) provided inconclusive results because global deletion of these abundant and relatively promiscuous signaling proteins is likely to induce a variety of developmental and/or behavioral defects (Chamero et al. 2011) that cannot be specifically attributed to deficits in vomeronasal signaling. However, specific G_αo deletion in vomeronasal neurons demonstrated this α-subunit’s critical role in basal VSN chemosensitivity. Specifically, VSNs from G_αo-deficient animals failed to respond to antigenic MHC class I peptides, MUPs, ESP1, and FPR3 ligands, while responses to fMLF remained unaltered (Chamero et al. 2011). By contrast, comparable evidence for the proposed role of G_αi2 in V1R-mediated signaling is still lacking.

Although they do not catalyze GDP–GTP exchange, the β- and γ-subunits of heterotrimeric G proteins also serve essential signaling functions (Figure 2). Adding another layer of complexity, transcripts of multiple G_β/γ isoforms were found in the developing VNO (Sathyanesan et al. 2013). G_αi2-positive VSNs express the γ₂, γ₃, γ₈, and γ₁₃ isoforms, whereas G_αo-positive VSNs expressed only the G_γ8 subunit (Ryba and Tirindelli 1995; Tirindelli and Ryba 1996; Rünnenburger et al. 2002; Sathyanesan et al. 2013). Mice with a homozygous deletion of Gng8, the gene encoding G_γ8, displayed reduced maternal and intermale aggression during resident–intruder assays, whereas, notably, other sociosexual behaviors remained essentially unchanged (Montani et al. 2013).

The primary effector enzyme downstream to G protein activation in VSNs appears to be a β-isoform of phospholipase C (PLCβ) (Holy et al. 2000; Spehr et al. 2002; Lucas et al. 2003). Accordingly, VSN activation leads to hydrolysis of phosphatidylinositol-4,5-bisphosphate, elevating the local concentrations of two second messenger molecules: the membrane-bound lipid diacylglycerol (DAG) and the cytosolic messenger inositol-1,4,5-trisphosphate (IP₃) (Figure 2). PLCβ stimulation is most likely triggered by the G_β/γ complex after dissociation from the activated α-subunit upon receptor–ligand interaction (Rünnenburger et al. 2002). Although it has been commonly assumed that PLCβ2 governs phosphoinositide turnover in VSNs (Lucas et al. 2003; Montani et al. 2013), it was recently revealed that this isoform only serves as the primary transduction element in MUP-sensitive VSNs, whereas PLCβ4 is the dominant isoform in all other (non-MUP sensitive) neurons (Dey et al. 2015).

Downstream to PLC-dependent lipid turnover, two distinct ion channels—TRPC2 and anoctamin1 (ANO1)—are implicated in completing the transformation of a chemical cue detection into an electrical signal (Figure 2). TRPC2, a member of the transient receptor potential (TRP) channel family (Liman et al. 1999), is enriched in VSN microvilli and activated by DAG (Lucas et al. 2003; Spehr et al. 2009; Leinders-Zufall et al. 2018). In VSNs, DAG analogues activate a nonspecific (i.e., mono- and divalent) cation conductance (Lucas et al. 2003). Channel gating thus entails both membrane depolarization and a biochemical signal in the form of a Ca²⁺ elevation (Box 3). The TRPC2 channel serves an important, though likely not exclusive function (Kelliher et al. 2006; Yu 2015). Thus, although Trpc2^−/− mice have severe deficits in a number of both sexual and social behaviors (Leypold et al. 2002; Stowers et al. 2002; Kimchi et al. 2007), some phenotypic discrepancies have been observed between Trpc2^−/− mice and animals in which the complete VNO was surgically removed (Pankevich et al. 2004; Kelliher et al. 2006; Yu 2015). Moreover, some (residual) VSN activity has been recorded from TRPC2-deficient VSNs in response to natural stimuli (Kelliher et al. 2006; Zhang et al. 2008; Yang and Delay 2010; Kim et al. 2011, 2012). So far, all known attempts to express recombinant TRPC2 in heterologous cells have failed. Thus, our knowledge of this specific TRP channel isoform is, at best, limited.

Box 3 Ca²⁺ signaling in vomeronasal neurons

In addition to the electrical events associated with vomeronasal signal transduction, VSN signaling involves a significant biochemical component, that is, the dynamic mobilization of cytosolic Ca²⁺ across broad spatial and temporal scales. Coupled to stimulus-evoked action potential discharge, Ca²⁺ entry via voltage-gated channels has frequently been used as a proxy for VSN activity (Inamura et al. 1997, 1999, Holy et al. 2000; Inamura and Kashiwayanagi 2000; Leinders-Zufall et al. 2000, 2004; Spehr et al. 2002; Del Punta et al. 2002a; Lucas et al. 2003; Chamero et al. 2007; Kimoto et al. 2007,Nodari et al. 2008; Haga et al. 2010; Papes et al. 2010; Arnson and Holy 2011; Chamero et al. 2011; Kim et al. 2011; Turaga and Holy 2012). By virtue of being a signaling molecule with many roles, however, stimulus-induced Ca²⁺ elevations will affect multiple aspects of VSN signaling. The exact physiological effects are largely determined by the unique spatiotemporal profile of any given Ca²⁺ signal. Its reliability, specificity, and speed depend on 1) Ca²⁺ release and influx mechanisms, 2) cytoplasmic buffers that limit Ca²⁺ diffusion, and 3) extrusion and storage processes that restore resting conditions, which, in “textbook” neurons, are maintained at levels of ∼100–150 nM (Berridge et al. 2003; Clapham 2007).

The molecular mediators that orchestrate discrete Ca²⁺ response profiles have collectively been designated as the Ca²⁺ signaling “toolkit” (Berridge et al. 2003) (Figure 3). Key members include Na⁺/Ca²⁺ exchangers, plasma membrane Ca²⁺ ATPases, the mitochondrial Ca²⁺ uniporter, and the sarco/endoplasmic reticulum Ca²⁺ pump as well as several cytosolic buffer/effector proteins such as calmodulin (Kirichok et al. 2004; Clapham 2007; Brini and Carafoli 2009; Baughman et al. 2011; Veitinger et al. 2011; Stephan et al. 2012). The coordinated and spatially controlled activity of these proteins results in a cell type–specific Ca²⁺ fingerprint that affects both primary and secondary signaling events and exerts positive and negative feedback regulation (Chamero et al. 2012).

In VSN dendritic tips, cytosolic Ca²⁺ elevations mainly result from TRPC2-mediated influx (Lucas et al. 2003) and IP₃-dependent internal-store depletion (Yang and Delay 2010; Kim et al. 2011) though the latter mechanism might be dispensable for primary chemoelectrical transduction (Chamero et al. 2017). Both routes, however, could mediate VSN adaptation and gain control by Ca²⁺/calmodulin-dependent inhibition of TRPC2 (Spehr et al. 2009; Figures 2 and 3), a mechanism that displays striking similarities to CNG channel modulation in canonical olfactory sensory neurons (Bradley et al. 2004). Another property shared with olfactory sensory neurons is Ca²⁺-dependent signal amplification via the ANO1 channel (Yang and Delay 2010; Kim et al. 2011; Dibattista et al. 2012; Amjad et al. 2015; Münch et al. 2018). Moreover, a nonselective Ca²⁺-activated cation current (I_CAN) has been identified in both hamster (Liman 2003) and mouse (Spehr et al. 2009) VSNs. To date, the physiological role of this current remains obscure. Likewise, it has not been systematically investigated whether Ca²⁺-dependent regulation of transcription plays a role in VSN homeostatic plasticity (Hagendorf et al. 2009; Li et al. 2016). Ultimately identifying the various roles that Ca²⁺ elevations play in vomeronasal signaling will require a much better quantitative picture of the VSN-specific Ca²⁺ fingerprint.

Early notions of strict VNO specificity of TRPC2 have recently been challenged after it was reported that a subpopulation of neurons in the olfactory epithelium is TRPC2-positive (Omura and Mombaerts 2014, 2015). These cells are categorized as either type A (Gucy1b2-negative) or type B (Gucy1b2-positive) cells, according to the expression of the soluble guanylate cyclase Gucy1b2. For the latter, a role as sensors for low environmental oxygen concentrations has recently been described (Bleymehl et al. 2016). Notably, both Gucy1b2 and Trpc2 are required for type B cell responses to low oxygen levels.

Increased cytosolic Ca²⁺, either resulting from TRPC2-dependent influx (Lucas et al. 2003) and/or IP₃-mediated release from internal stores (Yang and Delay 2010; Kim et al. 2011), triggers a Ca²⁺-activated Cl⁻ current (Yang and Delay 2010; Kim et al. 2011; Dibattista et al. 2012). Similar to the Ca²⁺-gated Cl⁻ conductance that supplements signal transduction in olfactory sensory neurons (Pifferi et al. 2009; Stephan et al. 2009; Sagheddu et al. 2010; Billig et al. 2011; Dauner et al 2012; Ponissery Saidu et al. 2013; Henkel et al. 2015), the Ca²⁺-dependent Cl⁻ current in VSNs appears to be mediated by a member of the recently identified ANO channel family (Caputo et al 2008; Schroeder et al. 2008). Specifically, conditional knockout of TMEM16A/ANO1 abolished the Ca²⁺-activated Cl⁻ currents in mature VSNs, establishing ANO1 as the primary mediator of this transduction current (Amjad et al 2015). This finding was recently confirmed in VSN recordings from ANO1/2 conditional double knockout mice, which show diminished spontaneous and pheromone-evoked action potential firing (Münch et al. 2018). It therefore came as a surprise that these double knockout mice did not display profound changes in resident–intruder paradigm-induced male territorial aggression (Münch et al. 2018).

Notably, whether Cl⁻ channels lead to a depolarizing current (as they do in olfactory neurons) depends solely on the chloride equilibrium potential established in vivo at the microvillar VSN membrane. Two recent studies have investigated this important physiological parameter. Although differing in methodology and quantitative results, both studies support the presence of a substantially elevated Cl⁻ level in VSNs that can provide the electrochemical driving force necessary for boosting sensory responses via a depolarizing Cl⁻ efflux (Kim et al. 2015; Untiet et al. 2016).

Figure 3.

 General and VSN-specific (top left) members of the cellular Ca²⁺ signaling “toolkit.” Low cytoplasmic Ca²⁺ levels at rest (~100 nM) are maintained by 1) extrusion via active transport across either the plasma membrane (plasma membrane Ca²⁺ ATPase [PMCA]) or the endoplasmic reticulum (smooth endoplasmic reticular Ca²⁺ ATPase [SERCA]), 2) facilitated transport via the electrogenic Na⁺/Ca²⁺ exchanger (NCX) in the plasma membrane, and 3) mitochondrial uptake by the mitochondrial Ca²⁺ “uniporter” (mCU), a high affinity–low capacity ion channel. Both in the extracellular medium and inside storage organelles (ER and mitochondria), Ca²⁺ concentrations reach millimolar levels. The resulting steep gradient underlies the massive, but transient cytoplasmic Ca²⁺ increase upon opening of voltage- and/or ligand-gated ion channels, including voltage-activated Ca²⁺ (Ca_V) channels, transient receptor potential canonical type 2 (TRPC2) channels as well as endoplasmic reticulum IP₃ receptors (IP3R) and ryanodine receptors (RyR). Note that, in VSNs, TRPC2 and the Ca²⁺-activated Cl⁻ channel (anoctamin1 [ANO1]) are highly enriched in the plasma membrane of the microvillar compartment. By contrast, VSN storage organelles (endoplasmic reticulum and mitochondria) are likely restricted to other subcellular areas, creating functionally distinct Ca²⁺ signaling compartments. The precise location of the many diverse “toolkit” components in VSNs, however, is still missing.

Secondary events

A rich repertoire of “non-standard” ion channels complements the “conventional” Hodgkin–Huxley type voltage-activated conductances in VSNs. Once a receptor potential is generated, the VSN input–output relationship is shaped by several such channels, including voltage-gated Ca²⁺ channels, Ca²⁺-sensitive K⁺ channels (SK3), ether-à-go-go-related (ERG) channels, and hyperpolarization-activated cyclic nucleotide-gated (HCN) channels.

Both low voltage–activated T-type and high voltage–activated L-type Ca²⁺ channels (Liman and Corey 1996) generate low-threshold Ca²⁺ spikes that modulate VSN firing (Ukhanov et al. 2007). Although these two specific Ca²⁺ currents are present in both FPR-rs3 expressing and non-expressing VSNs, FPR-rs3 positive neurons apparently express N- and P/Q-type Ca²⁺ currents with unique properties (Ackels et al. 2014).

In addition to Ca²⁺ channels, several K⁺ channels have been implicated in vomeronasal signaling, either as primary or as secondary pathway components. For example, coupling of Ca²⁺-sensitive large-conductance K⁺ (BK) channels with L-type Ca²⁺ channels in VSN somata is apparently required for persistent VSN firing (Ukhanov et al. 2007). By contrast, others suggested that BK channels play a role in arachidonic acid–dependent sensory adaptation (Zhang et al. 2008). Both mechanisms, however, could function in parallel, though in different subcellular compartments (i.e., soma vs. knob).

Recently, the small-conductance SK3 and a G protein–activated K⁺ channel (GIRK1) were proposed to serve as an alternative route for VSN activation (Kim et al. 2012). Mice with global deletions of the corresponding genes (Kcnn3 and Kcnj3) display altered mating behaviors and aggression phenotypes. Although these results are intriguing, the global nature of the deletion complicates the interpretation of the behavioral effects.

One form of VSN homeostatic plasticity is maintained by activity-dependent expression of the ERG channel (Hagendorf et al. 2009). In VSNs, these K⁺ channels control the sensory output of V2R-expressing basal neurons by adjusting the dynamic range of their stimulus–response function. Thus, regulation of ERG channel expression, as a function of stimulus exposure, enables calibration of the target output range of basal VSNs, in a use-dependent manner (Hagendorf et al. 2009).

In addition to the aforementioned Ca²⁺ and K⁺ channels, two members of the HCN channel family, HCN2 and HCN4, are involved in controlling VSN excitability (Dibattista et al. 2008). Notably, HCN channels also appear to play a role in vomeronasal gain control during semiochemical detection (Cichy et al. 2015). On the basis of the surprising observation that the estrus cycle dictates stage-correlated changes in urinary pH among female mice, extracellular acidification was identified as a potent activator of the vomeronasal hyperpolarization-activated current I_h (which is mediated by HCN channels). Whether vomeronasal sensation of a female’s estrus stage involves pH-dependent changes in VSN excitability is still unknown, but regardless, these findings reveal a potential mechanistic basis for detection of stimulus pH in rodent chemosensory communication (Cichy et al. 2015).

Signaling plasticity

An emerging and somewhat unexpected theme from several recent studies is that AOS responses can be modulated by physiological status or prior experience already at early processing stages (Yang and Shah 2016). For example, at the VSN level, identification of “self” and “non-self” by individual MUP “bar codes” results from learning and, accordingly, can be manipulated experimentally (Kaur et al. 2014). Similarly, individual differences in the abundance of specific functional VSN types result from experience-dependent plasticity (Xu et al. 2016). A striking example of endocrine state–dependent vomeronasal plasticity is selective VSN silencing in females during the diestrus phase of the reproductive cycle (Dey et al. 2015). Apparently, vomeronasal PLCβ2 expression (and hence MUP sensitivity) is controlled by progesterone, linking estrous cycle stage and sensory processing in female mice. Thus, increased progesterone levels during diestrus act directly on a subset of VSNs that, prior to ovulation, mediate female attraction behavior in response to male pheromones. Another mechanism for experience-driven feedback in the AOS is peripheral sensory adaptation in VSNs. Although the existence of such peripheral adaptation has long remained subject of some debate (Holy et al. 2000; Nodari et al. 2008; Spehr et al. 2009), recent evidence shows both short- and long-term adaptation upon repeated VSN stimulation (Wong et al. 2018).

VSN projections and axon targeting

Although a distinct AOB primordium is morphologically discernible in rodents around E16 (Marchand and Bélanger 1991; Knöll et al. 2001), the critical period for AOB wiring and glomeruli formation occurs during postnatal days 4–6 (Salazar et al. 2006; Hovis et al. 2012). VSN axons give rise to large, tightly fasciculated bundles that pass through the cribriform plate, project along the medial aspect of the olfactory bulb, and then turn upon reaching the olfactory bulb’s caudal part to target a specialized region at its dorsal/caudal end, the AOB. The AOB appears to retain the structural dichotomy observed in the VNO: the two main subsets of either V1R- or V2R-expressing neurons target two segregated regions in the glomerular layer along the AOB rostro-caudal axis. V1R-positive neurons coexpress olfactory axon cell adhesion molecule (OCAM) and synapse on OCAM-negative mitral cells in the rostral region of the AOB, forming multiple glomeruli (Belluscio et al. 1999; Rodriguez et al. 1999). The few published receptor-specific VSN-to-AOB tracing studies (Belluscio et al. 1999; Rodriguez et al. 1999; Wagner et al. 2006) report target ensembles of 4–30 individual glomeruli. For several reasons, however, caution should be exerted when interpreting/generalizing those numbers: 1) few VSN populations of defined receptor identity have been analyzed so far, 2) given their variable morphology and the reduced number of periglomerular cells, individual glomeruli are far less discernable in the AOB than in the main bulb, and 3) the extent to which individual glomeruli receive input from several VSN populations (Belluscio et al. 1999) is still unclear. Notably, V2R-expressing cells lack discernible OCAM expression and synapse with OCAM-positive second-order neurons. This interaction forms a physically separated projection site in the caudal part of the AOB (Jia and Halpern 1997; Mori et al. 2000; Ishii and Mombaerts 2008). Axons of FPR-rs3-expressing neurons also converge onto multiple (~8) glomeruli in the rostral AOB. Notably, glomeruli innervated by converging FPR-rs3 fibers are linked and located deep within a spatially restricted region of the AOB (Dietschi et al. 2013).

At least in rats, the division between V1R and V2R domains is also apparent at the AOB glomerular layer, as a region devoid of glomeruli, separating the rostral and caudal AOB halves (Larriva-Sahd 2008). The distinction is even clearer following staining with various lectins that bind to carbohydrate moieties expressed on specific classes of sensory neurons (Takami et al. 1992; Ichikawa et al. 1994; Shapiro et al. 1995). In mice, the pattern of lectin staining actually suggests a tripartite organization, with the posterior subdivision further divided into two parts (Salazar et al. 2001). This division is consistent with a differential pattern of AOB innervation by VSNs expressing or, alternatively, lacking H2-Mv genes (Ishii and Mombaerts 2008).

A notable property of VSN axons, distinguishing them from their MOS counterparts, is that upon reaching the AOB, individual axons can divide to terminate in multiple glomeruli (Larriva-Sahd 2008), rather than targeting a single glomerulus as typically observed in the main olfactory bulb (MOB). In rats, it has been estimated that ~20% of VSNs project to multiple glomeruli (Larriva-Sahd 2008). These findings are consistent with the observation that axons of sensory neurons expressing a given receptor form multiple glomeruli in the AOB (Belluscio et al. 1999; Rodriguez et al. 1999) and, as described later, with the spatial patterns of glomerular responses (Hammen et al. 2014).

Adding to this lack of organization, the finer-scale spatial patterns of sensory axon innervation to the AOB are also highly variable, with a given VSN population exhibiting diverse projection patterns, between individuals and even “within” individuals (i.e., between the two AOBs) (Belluscio et al. 1999; Rodriguez et al. 1999; Wagner et al. 2006). This situation markedly contrasts with the more stereotypical spatial innervation patterns observed in the MOB (Mombaerts et al. 1996), which on a functional level can be observed within and across individuals (Belluscio and Katz 2001), and even across species (Soucy et al. 2009). Nevertheless, the spatial distribution of VSN axons is not entirely random, as axons associated with different receptor types display stereotypical termination sites (Wagner et al. 2006). In addition to such divergence of processing channels (from a single receptor type to different glomeruli), there is also some evidence for convergence, in which single glomeruli (particularly large ones) gather inputs from more than a single receptor type (Belluscio et al. 1999).

The mechanisms underlying both homotypic fiber coalescence and VSN axonal pathfinding to select AOB glomeruli are far from understood. Similar to the MOS (Wang et al. 1998; Feinstein and Mombaerts 2004; Feinstein et al. 2004), vomeronasal chemoreceptors, which are found on both vomeronasal dendrites and axonal fibers, clearly play an instructive role during the final steps of the coalescence process (Belluscio et al. 1999). In addition, three prominent families of axon guidance cues, that is, semaphorins, ephrins, and slits (Bashaw and Klein 2010), have been implicated in VSN axon navigation (Cloutier et al. 2002; Prince et al. 2009, 2013). Both attractive and repulsive interactions play a critical role in axonal segregation of apical and basal VSN within the anterior versus posterior AOB regions. However, such mechanisms appear of minor importance for the sorting and coalescence of axons into specific glomeruli (Brignall and Cloutier 2015). Intriguingly, coalescence and refinement of AOB glomeruli is, at least to some extent, regulated by postnatal sensory activity (Hovis et al. 2012).

AOB—structure and functional circuitry

The AOB is the first brain relay of the AOS and is thus analogous to the MOB, the first processing stage of MOS. To a first approximation, the AOB, located at the posterior dorsal aspect of the olfactory bulb (Figure 1), shares many similarities with the larger MOB. These similarities include the broad classes of neuronal populations, their layered organization, and their connectivity. Yet, the AOB and MOB also show notable differences with respect to each of these aspects, and these differences may have important functional implications. Thus, one should be cautious about extrapolation of organizational and physiological principles from the main to the accessory bulb (Dulac and Wagner 2006; Stowers and Spehr 2014). Several studies have examined the anatomy of the AOB at the cellular level (Mori 1987; Takami and Graziadei 1991; Takami et al. 1992; Larriva-Sahd 2008). Here, we highlight the main features of AOB circuitry, particularly in comparison to those of the MOB.

The AOB glomerular layer, which (as described above) is divided into anterior and posterior regions, includes tightly clustered glomeruli that are sparsely surrounded by periglomerular cells (Figures 4 and 5). This sparseness implies that AOB glomerular boundaries are less well defined than those in the MOB. In addition, AOB glomeruli, which do not form a single layer, are often confluent and markedly variable in size (10‒30 µm diameter) (Tirindelli et al. 2009).

Figure 4.

 Visualization of the intact mouse AOB. In cleared brains from adult mice (CLARITY technique [Chung and Deisseroth 2013; Chung et al. 2013]), AMCs are specifically labeled with the fluorescent protein tdTomato (offspring from crossing Tbet-Cre [Haddad et al. 2013] and Ai9 reporter mice [Madisen et al. 2010]). (A and B) 3D rendering in which fluorescent cells that reside inside the mitral cell layer (MCL) are shown in green, whereas the lateral olfactory tract (LOT) and putative mitral cells adjacent to the AOB are shown in red. Perspectives implement a sagittal lateral-to-medial view (A) as well as the view from deep in the granule cell layer (B). Scale bars indicate 150 µm. A total of 21 203 nuclei were identified within the MCL. Of these, 6842 nuclei were also tdTomato-positive. (C) Single confocal section through the AOB from six stitched z-stacks. Nuclei are stained using DRAQ5 (blue; C_i); putative AMCs and LOT fibers are shown in red (C_ii). GL = glomerular layer; GCL = granule cell layer.

Figure 5.

 Simplified circuit diagram of the AOB. VSN axon bundles comprise the vomeronasal nerve layer (VNL) and innervate relatively small, loosely defined glomeruli (dashed circles) in the glomerular layer (GL). AOB periglomerular cells (PGCs) are sparser than in the MOB. The large mitral cell layer (MCL) contains juxtaglomerular neurons (JGNs) in an apical subglomerular zone as well as widely distributed projection neurons (i.e., AOB mitral cells [AMCs]) that each innervate several glomeruli. In the MCL, some external granule cells (eGCs) are also found. The LOT, a complex fiber tract that pierces the AOB between its external and internal cellular layers, receives efferent axons from both main bulb projection neurons and AMCs. The internal cellular layer mainly harbors axonless GABAergic internal granule cells (iGCs) and is thus designated as the granule cell layer (GCL).

The distinctions between the AOB and MOB also apply to their projection neurons. Although often named mitral cells, in analogy with the projection neurons of the MOB, the somata of AOB projection neurons rarely resemble those of MOB mitral cells (Larriva-Sahd 2008). In fact, most cellular components of these neurons, including cell bodies, dendritic arborizations, and axonal projections are highly variable from neuron to neuron, making it difficult to identify two anatomically similar projection neurons. Like their shapes, the locations of AOB projection neurons are also variable. Consequently, unlike the MOB, the AOB does not comprise well-defined “mitral cell” and “external plexiform” layers (Salazar et al. 2006) (Figures 4 and 5). Instead, the term “external cell layer” was suggested to describe the AOB layer that includes the somata and dendritic processes of projection neurons (as well as several classes of interneurons [Larriva-Sahd 2008]). These fuzzy boundaries also preclude a distinction between mitral and tufted cells in the AOB. Thus, AOB projection neurons are often collectively designated as mitral cells and will be denoted here as AMCs (AOB mitral cells). When crossing Tbet-Cre (Haddad et al. 2013) and Ai9 reporter mice (Madisen et al. 2010), AMCs are fluorescently labeled and readily identified. After whole brain tissue clearing using the CLARITY method (Chung and Deisseroth 2013; Chung et al. 2013), we imaged the intact AOB and counted fluorescently labeled nuclei within the external cell layer (Figure 4). A single AOB harbored 6842 putative AMCs, which corresponds to approximately one-third (0.32%) of all nuclei (21 203) registered in the external cell layer (Supplementary Movie).

The most striking differences between AOB and MOB projection neurons probably concerns their dendrites (Figure 5), which can be broadly divided into two classes: glomerular and secondary dendrites. Each AMC elaborates multiple thick glomerular (or primary) dendrites toward multiple glomeruli (with reported numbers ranging between one and ten) (Takami and Graziadei 1991; Urban and Castro 2005; Yonekura and Yokoi 2008). This unique organization is markedly distinct from that in the MOB where each mitral cell contacts a single glomerulus. This is significant because such an arrangement provides the obvious potential for extensive integration of information across multiple sensory channels, already at the level of the projection neurons (Box 4). Although clearly suggestive of integration, the anatomy itself does not reveal the fundamental nature of the computations performed by individual AMCs. Among other factors, these computations depend on the molecular identity of the sampled glomeruli, and on the physiological interactions between the different inputs. The few detailed studies addressing the molecular identity of glomerular inputs suggest that different AMCs can realize distinct modes of integration. Thus, it was shown that AMCs that send dendrites to glomeruli receiving V2R1b axons, target their dendrites exclusively to such glomeruli (Del Punta et al. 2002b; Hovis et al. 2012). In this so called “homotypic” scheme, AOB connectivity funnels the divergent glomerular patterns so that each projection neuron receives information from a single receptor type, a situation akin to that found in the MOB. Yet, in other cases, AMCs gather information from distinct receptor types (Wagner et al. 2006), a connectivity scheme that was designated as “heterotypic.” In at least some cases, connections are made with glomeruli sampling molecularly related receptor inputs (Wagner et al. 2006). This suggests that cross-channel sampling is not random, but rather follows a “selective heterotypical” scheme. Note, however, that molecular similarity between receptors does not necessarily imply similarity in stimulus selectivity. Thus, even in these cases, it cannot be determined that a given AMC necessarily samples inputs from receptors with similar response properties.

Box 4 The essence of computations performed by the AOB

Given the wiring scheme described earlier, is it possible to predict the “receptive fields” of AOB output neurons, namely AMCs? For example, in the MOB, where the wiring diagram is more regular, one may expect responses of output cells, at least to a first approximation, to resemble those of the sensory neurons reaching the corresponding glomerulus. This prediction has been confirmed experimentally, showing that at least in terms of general tuning profiles, MOB mitral cells inherit the tuning curves of their respective receptors (Tan et al. 2010). Likewise, sister mitral cells share similar odor tuning profiles (Dhawale et al. 2010), at least to the strongest ligands of their corresponding receptors (Arneodo et al. 2018).

In the wiring diagram of the AOB (Figure 5), the key theme is “integration” across multiple input channels (i.e., receptor types). Such integration can take place at several levels. Thus, in each AOB glomerulus, a few hundred VSN axons terminate and, upon vomeronasal stimulation, release the excitatory neurotransmitter glutamate (Dudley and Moss 1995). Integration across channels may already occur at this level, because, in at least some cases, a single glomerulus collects information from several receptors. In a subset of these cases, the axons of two receptors occupy distinct domains within the glomerulus, but in others, they intermingle, suggesting that a single mitral cell dendrite may sample information from multiple receptor types (Belluscio et al. 1999).

Although integration at the glomerular layer is still speculative, access to multiple glomeruli via the apical dendrites of individual AMCs is a prominent feature of AOB circuitry. However, the connectivity itself is not sufficient to determine the mode of integration. At one extreme, AMCs receiving inputs from multiple glomeruli could be activated by any single input (implementing an “OR” operation). At the other extreme, projection neurons could elicit a response “only” if all inputs are active (an “AND” operation). More likely than either of these two extremes is that responses are graded, depending on which inputs channels are active, and to what extent. In this context, a crucial physiological property of AMC glomerular dendrites is their ability to actively propagate signals both from and toward the cell soma. Indeed, signals can propagate from the cell body to apical dendritic tufts via Na⁺ action potentials (Ma and Lowe 2004), as well as from the dendritic tufts. These Ca²⁺-dependent regenerative events (tuft spikes) may cause subthreshold somatic EPSPs or, if sufficiently strong, somatic spiking, leading to active backpropagation of Na⁺ spikes from the soma to glomerular tufts (Urban and Castro 2005). These properties, together with the ability to silence specific apical dendrites (via dendrodendritic synapses) provide a rich substrate for nonlinear synaptic input integration by AMCs. One may speculate that the back-propagating somatic action potentials could also play a role in spike time-dependent plasticity, and thus strengthen or weaken specific input paths. Interestingly, AMC dendrites can also release neurotransmitters following subthreshold activation (Castro and Urban 2009). This finding adds a further level of complexity to the computations that AMCs could realize. One implication of this mechanism is that AMCs can shape the output of the AOB (i.e., of other AMCs) without firing action potentials themselves.

In this context, it should also be noted that AMCs respect the two major AOB subdivisions described earlier. Thus, any given AMC samples from only one of the two major vomeronasal receptor subdivisions (Belluscio et al. 1999; Del Punta et al. 2002b; Wagner et al. 2006). Incidentally, AMC somata are not necessarily located in the same divisions as their glomerular dendrites (Yonekura and Yokoi 2008).

Although the homotypic scheme has obvious functional appeal, the anatomical arrangement of the AOB actually appears well suited for cross-channel integration, whereas it seems rather inefficient for exclusively realizing the homotypic scheme. Therefore, although molecular level information on functional connectivity is still based on a small number of cases, it is likely that future studies will show that different AMCs vary in the extent of cross-channel integration.

In addition to their thick glomerular dendrites, AMCs elaborate secondary (accessory) dendrites that emanate from the cell body. Although some of these secondary dendrites remain within the same division as the cell body, others cross the boundary between the two AOB subdivisions. These secondary dendrites are considerably thinner than the glomerular dendrites (<1 µm as compared with >8 µm) and are reminiscent of the secondary lateral dendrites of MOB mitral cells, which can extend to considerable distances (~1 mm). By contrast, AMC secondary dendrites are shorter and fewer in number (Mori 1987).

A key circuit element, in both the AOB and MOB, involves the interaction between principal neuron dendrites and inhibitory neurons (Figure 5). At the glomerular level, principal neuron dendrites form reciprocal dendrodendritic synapses with inhibitory periglomerular (or juxtaglomerular) interneurons (Brennan and Kendrick 2006), which can extend dendrites into one or more glomeruli. As mentioned earlier, compared with the MOB, these neurons comprise a sparse population, which results in less distinct glomerular boundaries (Mugnaini et al. 1984). As in the MOB, most of these periglomerular neurons are GABAergic (Quaglino et al. 1999). However, unlike the case in the MOB, only a small number of AOB periglomerular neurons are dopaminergic (Mugnaini et al. 1984). Interestingly, following mating, expression of tyrosine hydroxylase—a marker of dopaminergic neurons—increases in the AOB of female rats (Matthews et al. 2013). Although these periglomerular neurons can serve to attenuate (e.g., to normalize) the strength of inputs into apical dendrites, their actual physiological function remains unknown.

A prominent feature of olfactory bulb circuits, both in the MOB (Price and Powell 1970; Schoppa and Urban 2003) and the AOB (Hayashi et al. 1993; Jia et al. 1999; Taniguchi and Kaba 2001), is the reciprocal dendrodendritic synapse between mitral and granule cell dendrites (Figure 5). Because the mitral-to-granule cell synapse is glutamatergic/excitatory, and the granule-to-mitral synapse is GABAergic/inhibitory, mitral cell activation leads to consequent inhibition, which could lead to self-inhibition and/or to lateral inhibition of other mitral cells. On the basis of anatomical evidence, it was suggested that in the MOB, the balance is shifted toward lateral inhibition, whereas self-inhibition is more dominant in the AOB (Mori 1987). While lateral inhibition is suitable for “sharpening” odor representations (Price and Powell 1970; Schoppa and Urban 2003), self-inhibition is appropriate for silencing particular mitral cells. The latter has been proposed to serve a sensory “gating” function that mediates olfactory memory formation upon one-trial learning (Hayashi et al. 1993; Kaba et al. 1994; Brennan and Keverne 1997; Castro et al. 2007), particularly in the context of the pregnancy block (Bruce) effect (Bruce 1960). According to this theory, synaptic events that occur during mating strengthen inhibitory synapses and silence stud-responsive AMCs (Brennan and Keverne 1997). As a result, stud male odors lose their responsivity and hence can no longer induce pregnancy block. Although this compelling theory is supported by several lines of evidence (Kaba et al. 1989; Brennan et al. 1995; Otsuka et al. 2001; Matsuoka et al. 2004; Keller et al. 2009), two recent studies suggest that experience-dependent plasticity is actually associated with intrinsic changes in excitability of the elements of these synapses. Specifically, it was shown that olfactory imprinting in the context of mating is associated with pronounced intrinsic excitability changes in a subset of mating activated AMCs (Gao et al. 2017). Similarly, another study showed that following male–male social interactions, many responsive inhibitory granule cells displayed increased excitability (Cansler et al. 2017). These findings reveal that, in addition to mating-associated plasticity as observed in the context of the Bruce effect, non-mating behaviors can also drive AOB inhibitory plasticity. More generally, these studies suggest a novel cellular basis for encoding sensory memories in the AOB, using intrinsic excitability changes.

The notion that lateral inhibition is more widespread in the MOB, whereas self-inhibition is stronger in the AOB is based on the observation that, in the AOB, reciprocal dendrodendritic synapses are formed by the larger glomerular dendrites (Mori 1987; Moriya-Ito et al. 2013), whereas in the MOB they are formed on the lateral dendrites. However, it is premature to discount a role for lateral inhibition in the AOB, as AMC secondary dendrites certainly do form dendrodendritic synapses (Mori 1987; Larriva-Sahd 2008). More directly, it was shown that blocking inhibition modifies stimulus response properties of AOB projection neurons (Hendrickson et al. 2008), supporting a role for lateral inhibition, presumably mediated through granule cells, in shaping stimulus-evoked responses. In the context of the pregnancy block, the location of the inhibitory dendrodendritic synapses (see later) implies that silencing will be selective to inputs from “particular” glomeruli. For the Bruce effect, this implies that learning should not lead to overall silencing of particular AMCs, but rather to changes in their tuning profiles.

Two major classes of granule cells have been described in the AOB (Larriva-Sahd 2008). One class includes the internal granule cells, whose cell bodies are located below the lateral olfactory tract (LOT) and thus resemble the granule cells of the MOB. The second class includes the so-called external granule cells, whose somata lie in the external cell layer (Figure 5). Notably, while the external granule cells form synapses with the soma and the proximal regions of AMCs, the internal granule cells form synapses at more distal dendritic sites. This implies that, while the former are suitable for self-inhibition, the latter are more likely to mediate lateral inhibition. The sources of inputs into these two cell classes of granule cells also differ, supporting the notion that they play distinct roles in AOB physiology (Larriva-Sahd 2008).

Another factor that affects the balance between self and lateral inhibition is the distribution of glutamate receptors, and particularly the metabotropic receptor subtypes on granule cell dendrites. It has been shown that activation of mGluR2 receptors suppresses granule cell inhibition (Hayashi et al. 1993), whereas activation of mGluR1 is required for reciprocal inhibition (Castro et al. 2007). Thus, the ratios between these two types of receptors may be another factor determining the functional effects of individual dendrodendritic synapses.

Although glomerular dendrites provide the most obvious mechanism for cross-channel integration, another possibility for direct AMC interaction involves their axons, many of which ramify in the external cell layer before joining the LOT (Figures 4 and 5). Unlike glomerular dendrites, axons and their collaterals may cross the border separating the two AOB halves, and reach other cells, including AMCs. Although the physiological significance of these pathways, if any, is unclear, a recent study provided physiological evidence for a functional link between the anterior and posterior AOB, which could be mediated by such axonal projections (Vargas-Barroso et al. 2016).

AOB centrifugal inputs

The AOB is richly innervated by centrifugal fibers that originate from diverse brain regions such as the corticomedial amygdala, the bed nucleus of the stria terminalis, and well-known top-down control centers including the locus coeruleus, the horizontal limb of the diagonal band of Broca, and the raphe nuclei (Broadwell and Jacobowitz 1976; Fan and Luo 2009; Smith and Araneda 2010; Oboti et al. 2018). Feedback afferents, which play a critical role in olfactory memory formation (Keverne and Brennan 1996), enter the AOB either via the LOT or through the bulbar core white matter (Larriva-Sahd 2008). Early research concentrated on both noradrenergic and glutamatergic feedback from the locus coeruleus and amygdala, respectively. During mating, vaginocervical stimulation triggers lasting noradrenaline elevations in the AOB that remain for ~4 h (Brennan et al. 1995). This time window defines a critical period during which noradrenaline causes plastic changes in dendrodendritic synaptic strength (Brennan and Keverne 1997, 2004). Mechanistically, initial findings indicated noradrenaline-dependent mitral cell disinhibition via α2-receptor-mediated granule cell suppression (Otsuka et al. 2001; Brennan 2004). More recent results, however, suggest α1-dependent increase in granule cell GABA release that inhibits AMC firing (Araneda and Firestein 2006; Smith et al. 2009). Toward a reconciliation of these seemingly contradictory models of chemosensory plasticity, it was recently found that noradrenaline sculpts mitral responses in a cell- and stimulus-specific manner (Doyle and Meeks 2017).

Interest in AOB neuromodulation has also focused on cholinergic centrifugal input from neurons in the horizontal limb of the diagonal band of Broca. Two studies investigated activation of muscarinic acetylcholine receptors in the rodent AOB (Smith and Araneda 2010; Takahashi and Kaba 2010). Both studies showed muscarinic receptor-dependent increase in granule cell excitability by direct (long-lasting depolarization) and indirect (increase in excitatory glutamatergic input from AMCs) mechanisms. More recently, serotonin was added to the list of potential top-down neuromodulators in the AOB (Huang et al. 2017). Similar to the proposed cholinergic functions (Smith and Araneda 2010; Takahashi and Kaba 2010), serotonergic projections appear to increase the inhibitory tone of AOB granule cells, stimulating GABA release via 5-HT₂ metabotropic receptors. Furthermore, serotonergic afferents may also inhibit AMCs more directly by activation of 5-HT₁ receptor isoforms (Huang et al. 2017).

Interestingly, tracing studies revealed that feedback projections to the AOB from the bed nucleus of the stria terminalis and the amygdala are topographically organized and use different neurotransmitters (Fan and Luo 2009). Specifically, GABAergic projections from the bed nucleus terminate in the external cell layer, whereas glutamatergic projections from the amygdala target the inner granule cell layer. In addition, a substantial number of such feedback neurons in both brain areas express ER-α estrogen receptors, potentially explaining how AOB computations can be regulated by endocrine state (Fan and Luo 2009).

Although presently the jury is still out with respect to the exact functional consequences of feedback projections, it seems safe to conclude that afferent centrifugal modulation of AOB processing plays an important physiological role in AOS function (Stowers and Spehr 2014).

AOS response profile

Vomeronasal sensory neurons

VSN selectivity

Various secretions and bodily fluids elicit vomeronasal activity. So far, VSN responses have been recorded upon exposure to tear fluid (from the extraorbital lacrimal gland), vaginal secretions, saliva, fecal extracts, and other gland secretions (Macrides et al. 1984; Singer et al. 1987; Briand et al. 2004; Doyle et al. 2016). Experimentally, the most widely used “broadband” stimulus source is diluted urine, either from conspecifics or from predators (Inamura et al. 1999; Sasaki et al. 1999;Holy et al. 2000; Inamura and Kashiwayanagi 2000; Leinders-Zufall et al. 2000; Spehr et al. 2002; Stowers et al. 2002; Brann and Fadool 2006; Sugai et al. 2006; Chamero et al. 2007; Zhang et al. 2007, 2008; He et al. 2008; Nodari et al. 2008; Ben-Shaul et al. 2010; Meeks and Holy 2010; Yang and Delay 2010; Kim et al. 2012; Cherian et al. 2014; Cichy et al. 2015; Kunkhyen et al. 2017). For urine, reports of vomeronasal activity are highly consistent across laboratories and preparations, with robust urine-induced signals generally observed in 30–40% of the VSN population (Holy et al. 2000, 2010; Kim et al. 2011, 2012; Chamero et al. 2017).

The molecular identity of the active components in urine and other secretions is far less clear. Initially, several small molecules, which were identified as bioactive constituents of rodent urine (Novotny 2003), were found to activate VSNs in acute slices of the mouse VNO (Leinders-Zufall et al. 2000). These compounds, including 2,5-dimethylpyrazine, SBT, 2,3-dehydro-exo-brevicomin, α-farnesene, β-farnesene, 2-heptanone, and HMH, had previously been associated with diverse functions such as induction or synchronization of estrus as well as delay or acceleration of puberty (Schwende et al. 1984; Jemiolo and Novotny 1994; Novotny et al. 1999; Sam et al. 2001). Later, when separating urine fractions according to molecular mass, Chamero and coworkers reported that a distinct VSN population is activated by molecules of high molecular weight (>10 kDa) (Chamero et al. 2007). A prominent fraction of these macromolecules is represented by the MUPs) (Berger and Szoka 1981; Shaw et al. 1983), which also activate a unique neuronal subpopulation (Chamero et al. 2011; Kaur et al. 2014; Dey et al. 2015). Other molecularly identified VSN stimuli include various sulfated steroids (Nodari et al. 2008; Celsi et al. 2012; Turaga and Holy 2012; Haga-Yamanaka et al. 2015), MHC class I peptide ligands (Leinders-Zufall et al. 2004, 2009; Kelliher et al. 2006; Hovis et al. 2012), fecal bile acids such as cholic and deoxycholic acid (Doyle et al. 2016), and the exocrine gland–secreted peptides ESP1 and ESP22 (Kimoto et al. 2005, 2007; Haga et al. 2010; Ferrero et al. 2013).

When single molecules are tested, each compound generally activates a small subset of VSNs. Small bioactive molecules (Leinders-Zufall et al. 2000), MHC peptides (Leinders-Zufall et al. 2004), MUPs (Chamero et al. 2007; Kaur et al. 2014; Dey et al. 2015), ESP1 (Kimoto et al. 2007), and ESP22 (Ferrero et al. 2013) each activate approximately 1% of VSNs. Sulfated steroids, however, are a notable exception. A mix of 12 members of this ligand family was reported to activate ~50% of all apical VSNs (Turaga and Holy 2012). Assuming similar potency and nonoverlapping VSN response profiles, each steroid would be expected to stimulate ~2% of all VNO neurons. In addition, just two sulfated estrogens—1,3,5(10)-estratrien-3,17β-diol disulfate and 1,3,5(10)-estratrien-3,17β-diol17-sulfate—were found to activate ~15% of VSNs (Haga-Yamanaka et al. 2015) when presented at relatively high concentrations. Moreover, a single female steroid metabolite, that is, 16-hydroxycorticosterone-20-hydroxy–21-acid, was recently found to account for ~25% of all VSN responses to urine from C57BL/6J females (Fu et al. 2015). Unraveling the physiological basis and coding logic behind this surprisingly broad potency range of individual stimuli will, no doubt, prove highly informative.

In sharp contrast to the relatively broad tuning and marked ligand promiscuity of odorant receptors that underlies the notion of combinatorial coding in the MOS, early studies proposed extraordinarily high stimulus selectivity in VSNs (Leinders-Zufall et al. 2000). Confocal Ca²⁺ imaging studies revealed that each of six small molecule ligands activates a unique, nonoverlapping subset of apical VSNs. Supported by extracellular recordings of electrical activity, these experiments established the notion of extremely selective VSN tuning, relatively independent of stimulus concentration, and small linear dynamic ranges of VSN responses (Leinders-Zufall et al. 2000). At least for some stimuli, however, these concepts appear not applicable. A large fraction (60%) of neurons responding to sulfated estrogens, for instance, were found to display bell-shaped dose-response curves with peak responses at intermediate concentrations (Haga-Yamanaka et al. 2015). In this study, a few VSNs even displayed tuning properties that did not fit either sigmoidal or bell-shaped profiles. Similarly, population Ca²⁺ imaging identified a VSN population that, when challenged with urine, is only activated by low concentrations (He et al. 2010). Given the molecular heterogeneity of urine, the authors explained these somewhat unusual response profiles by antagonistic interactions in natural secretions. Unexpectedly, responses of VSNs to MUPs were shown to follow a combinatorial coding logic, with some MUP-detecting VSNs functioning as broadly tuned “generalists” (Kaur et al. 2014). Further complicating the picture, some steroid ligands appear to recruit an increasing number of neurons over a rather broad range of concentrations (Haga-Yamanaka et al. 2015).

Likely, the information content of bodily secretions is more than the sum of their individual components. The mixture (or blend) itself might function as a semiochemical. An example is provided by the concept of “signature mixtures,” which are thought to form the basis of individual recognition (Wyatt 2017). Examining VSN population responses to individual mouse urine samples from both sexes and across strains (He et al. 2008), a small population of sensory neurons that appeared to respond to sex-specific cues shared across strains and individuals was identified. However, in contrast to sex coding, strain and individual information appeared encoded by combinatorial VSN activation, such that urine from different individuals activated overlapping, but distinct cell populations (He et al. 2008).

VSN sensitivity

VSNs are exquisitely sensitive chemosensors. Threshold responses are routinely recorded upon exposure to ligand concentrations in the picomolar to low nanomolar range. This holds true for small molecules (Leinders-Zufall et al. 2000), MHC peptides (Leinders-Zufall et al. 2004), sulfated steroids (Haga-Yamanaka et al. 2015; Chamero et al. 2017), and ESPs (Kimoto et al. 2005; Ferrero et al. 2013).

Our knowledge about the electrophysiological properties of a “typical” VSN response is still fairly limited. Given the electrically tight nature of these neurons, it might not be surprising that sensory stimulation sometimes evokes inward receptor currents of only a few picoamperes (Kim et al. 2011, 2012). In other cases, substantially larger receptor currents were reported (Zhang et al. 2008; Spehr et al. 2009; Yang and Delay 2010), particularly in response to sulfated steroids (Chamero et al. 2017). Paradoxically, the huge input resistance of VSNs would likely lock these neurons in an inactive depolarized state when challenged with stimuli that induce such strong inward currents.

This heterogeneity in primary transduction current amplitude might underlie the broad range of maximal firing rate changes observed across VSNs. Extracellular recordings of discharge frequency reported “typical” stimulus-dependent spike frequency modulations ranging from ≤8 Hz (Kim et al. 2012; Chamero et al. 2017) up to 25–30 Hz (Stowers et al. 2002; Haga-Yamanaka et al. 2015) and even up to ~80 Hz (Nodari et al. 2008). These higher values are remarkable because VSNs firing rates typically saturate at frequencies ≤25 Hz upon whole-cell current injections (Liman and Corey 1996; Shimazaki et al. 2006; Ukhanov et al. 2007; Hagendorf et al. 2009; Kim et al. 2011).

Recently, the topographical mapping of response profiles to sulfated steroids across the anterior AOB was examined (Hammen et al. 2014). Imaging presynaptic Ca²⁺ signals in vomeronasal axon terminals using light sheet microscopy, the authors revealed a complicated organization involving selective juxtaposition and dispersal of functionally grouped glomerular classes. Although similar tuning to urine often resulted in close glomerular association, testing a panel of sulfated steroids revealed tightly juxtaposed groups that were disparately tuned, and reciprocally, spatially dispersed groups that were similarly tuned (Hammen et al. 2014). Overall, these results indicate a modular, nonchemotopic spatial organization in the AOB.

AOB mitral cells

Virtually all published in vivo electrophysiological recordings from the AOB involve extracellular recordings targeted to AMCs (i.e., to the mitral cell layer). Although cell type identity is never entirely certain with conventional extracellular recordings, it is likely that AOB projection neurons are by far the dominant cell type in these multiple studies of AOB in vivo physiology. Thus, our discussion is focused on this cell type. It should also be noted that, at present, there are no studies clearly distinguishing the physiological properties of AMCs sampling from anterior or posterior AOB divisions.

AMC spontaneous activity

Initial recordings from intact behaving mice (Luo et al. 2003), and later recordings from anesthetized mice (Hendrickson et al. 2008; Ben-Shaul et al. 2010), highlighted the low baseline firing rates of AOB neurons, with some neurons being virtually silent until an appropriate stimulus is applied. Mean firing rate estimates of AMCs are on the order of 1–2 Hz (Luo et al. 2003; Hendrickson et al. 2008; Ben-Shaul et al. 2010). Unlike MOB mitral cells, AMC firing does not follow the breathing rhythm, but most typically corresponds to a popcorn like (i.e., Poisson) firing pattern. More recent work, initially in vitro, has provided novel insights into the discharge patterns that characterize AMCs. Some of these patterns are rather unusual. In an “idle” state, several groups have shown that some AMCs display slow and periodic bursts of activity (Gorin et al. 2016; Vargas-Barroso et al. 2016; Zylbertal et al. 2017). This oscillatory resting state has been observed both in vitro and in vivo and some neurons intrinsically generate these oscillations independent of fast GABAergic and glutamatergic synaptic input (Gorin et al. 2016). As AMC axon collaterals contact both adjacent projection neurons as well as interneurons in both the anterior and posterior AOB (Larriva-Sahd 2008), periodic bursts will be transmitted throughout the AOB. How such slow oscillations shape AOB activity and what role they play for chemosensory processing will be an exciting avenue for future research.

AMC stimulus-induced activity: general features

As a generalization from multiple studies, stimulus-induced responses of AMCs are low in rates, slow in onset, and prolonged in duration. Maximal rates reported for single units are on the order of 20 Hz, and for many neurons are lower (<10 Hz). Stimulus delivery can induce both firing rate elevations and suppression (Luo et al. 2003; Hendrickson et al. 2008; Ben-Shaul et al. 2010; Yoles-Frenkel et al. 2018). However, the former are far more distinct from baseline firing rates and, at least in anesthetized mice, considerably more common (Yoles-Frenkel et al. 2018). In behaving mice, where baseline rates tend to be higher (Luo et al. 2003), rate suppressions following stimulus sampling appear more prevalent than in anesthetized mice (Hendrickson et al. 2008; Ben-Shaul et al. 2010). Notably, it has also been shown in vitro that the maximal rates to which AMCs can be driven is <50 Hz (Zibman et al. 2011). In comparison, most MOB projection neurons can be driven to rates >50 Hz and often also above 100 Hz (Zibman et al. 2011)

The low maximal rates of individual AOB neurons limits their ability to convey fast temporal changes. Indeed, the emerging picture from a systematic analysis of AOB responses (Yoles-Frenkel et al. 2018) is that AOB responses are very slow, in terms of both their onset time and their duration. Thus, in both freely exploring mice and in anesthetized preparations with intact VNO pumping, rate elevations begin several seconds following the start of exploration (Luo et al. 2003; Yoles-Frenkel et al. 2018), with peak rates appearing on the order of ~5 s following sympathetic trunk stimulation (Ben-Shaul et al. 2010; Yoles-Frenkel et al. 2018). Notably, in preparations with direct stimulus delivery to the VNO, response onsets and peak response times generally occur earlier than in preparations requiring VNO pumping (Hendrickson et al. 2008). Yet, as with VSNs (Holy et al. 2000), even with direct stimulus delivery, delays were larger for urine than for a high-potassium stimulus that circumvents the need for the normal signal transduction cascade. Taken together, these multiple studies suggest that temporal delays of vomeronasal responses are due to the pumping action, but also to the intrinsic time constants of VSNs and AMCs. Along the same lines, AMCs are intrinsically adapted to produce prolonged responses (Zibman et al. 2011), accommodating both transient and persistent firing responses upon stimulation (Shpak et al. 2012). Mechanistically, persistent mitral cell activity in response to brief sensory stimulation appears to depend on rather slow Na⁺ removal and a resulting reverse mode of dendritic Na⁺/Ca²⁺ exchangers (Zylbertal et al. 2015). The slow neuronal dynamics in the AOB are matched with the slow pumping action of the VNO, which itself is consistent with the prolonged (~ seconds) time course of social investigation for which the AOS is often used for. Recently, we have suggested that the slow dynamics of AOS neurons can be regarded as an adaptation to the intrinsically variable, and hence unreliable, temporal aspects of stimulus delivery (Yoles-Frenkel et al. 2018).

AMC stimulus-induced activity: tuning properties

In vivo recordings have shown that AOB neurons respond to investigation of other species, in both the anogenital and facial region (Luo et al. 2003), but such studies cannot reveal the sources of the effective stimuli. By far, the most widely investigated bodily source of semiochemicals is urine, and several studies showed that it is a highly effective stimulus for AOB neurons (Hendrickson et al. 2008; Ben-Shaul et al. 2010). More specifically, it was shown that AOB neurons not only respond to urine, but are also sensitive to features of the urine donor. Thus, there are many examples of neurons that appear to be selective for specific traits, such as sex, physiological status, and strain (often regarded as a model for individuality). We note that caution should be exercised when designating a neuron as selective for one trait or another, as natural secretions are complex and can vary in ways that are not controlled by the experimenters. For example, it is clearly not justified to designate a neuron that responds to urine from one male individual, but not from one female individual, as “male specific,” because the neuron may be sensitive to some other aspect, which distinguishes the two samples but is not specifically related to sex. To convincingly demonstrate that a neuron is sensitive to a particular trait (e.g., sex), it is required to show that it responds to that feature across a large number of samples, which vary in other traits. For obvious technical limitation of feasible stimulus sets, this has only been partially done. Such neurons with genuine “high level” receptive fields have yet to be convincingly identified in the AOB. At least for some features, it seems that reliable determination of traits from AOB activity requires polling information from multiple neurons (Tolokh et al. 2013; Kahan and Ben-Shaul 2016).

Despite its dominance as a stimulus source, urine is by no means the only effective stimulus for AOB neurons. Other effective stimulus sources include saliva, vaginal secretions (Kahan and Ben-Shaul 2016), and feces (Doyle et al. 2016). Although not tested directly in real-time in vivo preparations, it is more than likely that other bodily sources such as tears (Kimoto et al. 2005; Ferrero et al. 2013) will also induce activity in AOB neurons. Interestingly, information about both genetic background and receptivity can be obtained from various stimulus sources, including urine, vaginal secretions, and saliva. However, particular secretions may be optimized for conveying information about specific traits. For example, detection of receptivity is more accurate with vaginal secretions than with urine (Kahan and Ben-Shaul 2016).

As mentioned earlier, the AOS is also sensitive to predator odors, and indeed, AOB neurons show strong responses to stimuli from predators, and can often respond in a predator-specific manner (Ben-Shaul et al. 2010). In this context, the rationale for a combinatorial code is even more apparent, because individual AOB neurons often respond to multiple stimuli with very distinct ethological significance (e.g., female urine and predator urine) (Bergan et al. 2014). Taken together, AOB neurons appear to be responsive to a wide range of bodily secretions from multiple sources and species. Whether, and to what extent, AOB neurons respond to “non-social” stimuli remains largely unexplored.

A distinct question concerns the compounds that actually activate AOB neurons. Although all individual compounds shown to activate VSNs are justifiably expected to also influence AOB neurons, they will not necessarily suffice to elicit AOB activity. This is particularly true if AOB neurons, as would be consistent with their dendritic organization, require inputs from multiple channels to elicit action potentials. Thus far, the only individual compounds shown to activate AOB neurons in direct physiological measurements are sulfated steroids and bile acids (Nodari et al. 2008; Doyle et al. 2016). As noted earlier for VSNs, these two classes of compounds activate a remarkably large fraction of neurons, comparable to that activated by whole urine.

The robust responses to sulfated steroids allowed analysis of an important and still unresolved issue related to AOB physiology, namely the functional computations implemented by AOB neurons. Comparing responses of VSNs and AMCs to a panel of sulfated steroids, it was concluded that chemical receptive fields of almost half of all responsive AOB neurons (termed “functional relays”) mirror the responses of single VSN types (Meeks et al. 2010). Responses of the rest of the neurons could not be accounted for by a single VSN type and thus likely involved inputs from multiple channels. Although highly informative, it should be emphasized that this approach is limited to reveal the extent of integration applied to ligands in the tested set. Thus, the analysis of the important, but limited class of sulfated steroids, provides a lower limit to the extent of integration performed by individual AOB neurons. One other study that touched upon the issue of integration identified a predominance of mixture suppression, suggesting the importance of inhibitory interactions among distinct channels (Hendrickson et al. 2008). Another study, using a similar approach, mostly found synergistic responses to mixtures (Ben-Shaul et al. 2010). Overall, determining the essence of the information conveyed by AOB projection neurons—in molecular/computational terms and in ethological terms—remains an important issue for future investigation (Box 4).

Beyond the AOB

For a sensory system, the neural circuitry of the AOS is rather unusual and often portrayed as relatively simple. After AOB processing, vomeronasal signals bypass the thalamocortical axis. Instead, they are directly relayed to third-order limbic system nuclei including the medial amygdala and posteromedial cortical nucleus (which together comprise the vomeronasal amygdala (Martinez-Marcos 2009; Gutiérrez-Castellanos et al. 2014; Stowers and Liberles 2016)). The last major processing relay between sensory input and output command is the hypothalamus (Lo and Anderson 2011). In an elegant recent study, the entire information pathway underlying ESP1 signaling in female mice—from the peripheral receptive organ to the motor-regulating midbrain via the amygdala–hypothalamus axis—was identified (Ishii et al. 2017). Accordingly, ESP1-dependent enhanced sexual receptivity (lordosis) results from information processing along a labeled line.

Although the limbic system is traditionally considered to utilize more hardwired mechanisms (Meredith 1991) than do cortical structures, accumulating evidence suggests considerable flexibility and state-dependent modulation along the accessory olfactory processing stream (Yang and Shah 2014). In fact, it is becoming increasingly clear that both modulatory mechanisms and feedback loops enable dynamically adaptive, state-specific responses to vomeronasal stimuli. It is the combination of a relatively “simple” circuit anatomy with both complex processing mechanisms and a direct link to neuroendocrine alterations, emotional changes, and social/sexual behavior that renders the AOS an ideal model system for chemosensory coding (Box 1).

A comprehensive description of AOS circuitry and information processing beyond the VNO and AOB is clearly beyond the scope of this review. For more detailed insight into these fascinating brain areas, the interested reader is referred to several excellent recent reviews (Swanson 2000; Dong et al. 2001; Martinez-Marcos 2009; Duvarci and Pare 2014; Gutiérrez-Castellanos et al. 2014; Yang and Shah 2014, 2016; Bains et al. 2015; Janak and Tye 2015; Anderson 2016).

Future directions

As this review shows, much still remains to be explored about AOS function. Here, we highlight some important topics that in our opinion present particularly important directions for future research.

Revealing the limitations/capacities of AOS-mediated learning

That the AOS is involved in social behaviors, which are often innately encoded, does not mean that it rigidly maps inputs to outputs. As described here, there are several examples of response plasticity in the AOS, whereby the efficacy of a particular stimulus is modulated as a function of internal state or experience (Beny and Kimchi 2014; Kaur et al. 2014; Dey et al. 2015; Xu et al. 2016; Cansler et al. 2017; Gao et al. 2017). Thus, there is no doubt that the AOS can display plasticity. However, a distinct question is whether the AOS can flexibly and readily pair arbitrary activation patterns with behavioral responses. In the case of the MOS, it is well known that the system can mediate fixed responses to defined stimuli (Lin et al. 2005; Kobayakawa et al. 2007; Ferrero et al. 2011), as well as flexibly pair responses to arbitrary stimuli (Choi et al. 2011). In the AOS, it is known that particular stimuli can elicit well-defined behaviors or physiological processes (Brennan 2009; Flanagan et al. 2011; Ferrero et al. 2013; Ishii et al. 2017), but it is not known to what extent it can flexibly link arbitrary stimuli (or neuronal activation patterns) with behavioral, or even physiological responses. This is a crucial question because the AOS, by virtue of its association with social and defensive behaviors, which include substantial innate elements, is often regarded as a hardwired rigid system, at least in comparison to the MOS.

A deeper understanding of stimulus sampling

One of the unique aspects of vomeronasal sensing is the potential gating of chemical cues to the VNO, and hence to the sensory neurons of the AOS. When the organ is activated is a question of great physiological and behavioral importance. Although we do know that sympathetic activation can induce pumping (Meredith and O’Connell 1979; Eccles 1982; Ben-Shaul et al. 2010), a basic understanding of the exact circumstances that trigger VNO pumping is still lacking. For example, is vomeronasal sensing automatically initiated during periods of arousal? Is it triggered via sensory neurons that are not part of the AOS, including MOS neurons (Ogura et al. 2010; Slotnick et al. 2010)? One possibility is that odor detection by the MOS could trigger automatic or perhaps even voluntary VNO sampling. Notably, it is not known if the VNO can at all be controlled voluntarily (i.e., like skeletal muscle). These are difficult questions to address because there is at present no straightforward way to monitor vomeronasal uptake directly. One important study in hamsters (Meredith 1994) showed that VNO activity occurs during periods of arousal, but the precise relationship between the recorded signals and pumping activity was not known. Observation of species that exhibit an overt Flehmen response, often regarded as an indicator of vomeronasal sampling, indicates that vomeronasal uptake is associated with processing of socially relevant stimuli (Houpt et al. 1989; Stahlbaum and Houpt 1989; Sankar and Archunan 2004). Consistent with this, single unit (Luo et al. 2003) and local field potentials recordings from the AOB (Tendler and Wagner 2015; Pardo-Bellver et al. 2017) of behaving rodents provide real-time evidence that social investigation is indeed associated with AOS activation. Thus, although it is well established that vomeronasal function is associated with social investigation (and likely with risk assessment behaviors), a good understanding of AOS stimulus uptake dynamics is still missing. In particular, how do external stimuli, behavioral context, and physiological state dictate VNO pumping? And, in turn, how do the details of VNO pumping affect neuronal activity in recipient structures? Because the AOS probably serves different functions in different species, the circumstances of vomeronasal uptake are also likely to differ across species. Understanding these circumstances, especially in mice and rats—the most common model for chemosensory research—will clearly enhance our understanding of AOS function. How this can be achieved is not obvious. Potential approaches, none of them trivial, include noninvasive imaging of VNO movements, or physiological measurements in the VNO itself.

Role of oscillatory activity in AOS function

Oscillatory activity is a hallmark of brain activity, and it plays a role across many sensory and motor systems (Buzsáki 2006). In olfaction, oscillations play a central role, most basically through its dependence on the breathing cycle (Kepecs et al. 2006; Wachowiak 2011). One important consequence of this dependence is that the timing of neuronal activity with respect to the phase of the sniffing cycle can be informative with respect to the stimulus that elicited the response (Cury and Uchida 2010; Shusterman et al. 2011). Breathing-related activity is strongly linked to theta (2–12 Hz) oscillations in neuronal activity or local field potentials, but oscillatory activity in the olfactory system is not limited to the theta band. Other prominent frequency bands are the beta (20–30 Hz) and gamma bands (40–100 Hz), both of which have been suggested to play a role in the computational function of the olfactory system (Kay 2014).

In comparison, much less is known about the phenomenology and roles, if any, of oscillatory events in the AOS, where stimulus uptake is uncoupled from the breathing cycle. Indeed, in vivo recordings did not reveal robust sniffing locked activity in single AOB neurons (Luo et al. 2003; Hendrickson et al. 2008; Ben-Shaul et al. 2010), and until recently, there were very few reports about oscillatory activity of individual neurons in the AOS. One notable observation concerns the ultraslow oscillations described in the AOB, recently observed in in vitro preparations by several groups (Gorin et al. 2016; Vargas-Barroso et al. 2016; Zylbertal et al. 2017). Taken together, these studies indicate that although some AMCs are intrinsically oscillatory, network interactions also play a role in their generation. Although these oscillations are also present in spontaneous activity in vivo, they are more prominent in vitro (Gorin et al. 2016). Unlike stereotyped oscillations in the MOS, which fall into relatively discrete bands, these ultraslow oscillations are much more heterogeneous, raising the possibility that they may functionally bind particular subsets of AOB neurons (Gorin et al. 2016; Zylbertal et al. 2017). A better understanding of the mechanisms and the functional implications of these oscillations is one important topic for future investigation.

Another related aspect concerns the role of local field potentials. Several in vivo studies in behaving animals have shown signatures of local field potential fluctuations, in bands similar to those observed in the MOS, during stimulus processing (Binns and Brennan 2005; Leszkowicz et al. 2012; Tendler and Wagner 2015; Pardo-Bellver et al. 2017). Specific remaining questions include the following: How are these oscillations generated? Are they correlated with stimulus sampling? Do they play a role in vomeronasal information processing? Do they provide a temporal reference, according to which timing of neuronal activity can be evaluated? These challenging questions are only partly resolved in the MOS (Kay 2015). In the AOS, our understanding of these issues lags behind considerably, providing yet another set of important topics to explore.

A better understanding of the heterogeneity of AOB neurons

Unlike primary chemosensory neurons, which are distinguished by the receptors that they express, AOB neurons cannot easily be separated into distinct molecular classes and are therefore often regarded as one single population. One obvious classification of AOB neurons concerns the subdivision from which they receive inputs (i.e., basal vs. apical). However, apart from some cell adhesion molecules (von Campenhausen et al. 1997; Cloutier et al. 2002), it is not known if this difference is somewhat reflected by molecular features. As described earlier, from a morphological point of view, individual AMCs are highly heterogeneous, with each exhibiting a unique dendritic pattern. The fact that AMC morphologies do not appear to fall into clear clusters, but rather to a continuum of morphologies, raises the possibility that there may not be any well-defined molecular subtypes. Regardless, the question of functional heterogeneity (and their correlation with morphological properties) remains. For example, are AMCs that sample from many glomeruli more or less selective than those that sample from a single glomerulus? The answer to this question depends on how individual neurons integrate information from different channels. More generally, another important future goal is to understand how the range of physiological properties is related, if at all, to the molecular and/or morphological characteristics of AMCs.

Monitoring the activity of AOB populations

Although there are several reports of large-scale VSN recordings, both in the epithelium (Rivière et al. 2009; Turaga and Holy 2012) and in their axonal termination sites in AOB glomeruli (Hammen et al. 2014), little is known about how ensembles of AOB neurons encode sensory information. Although in vivo recordings from the AOB have employed multisite electrodes (Tolokh et al. 2013; Kahan and Ben-Shaul 2016), there are at present no systematic in vivo analyses of correlated activity of AOB neuron ensembles. Knowledge about the joint activity of multiple neurons would be particularly important given that 1) in vitro recordings suggest the existence of such ensembles (Gorin et al. 2016; Zylbertal et al. 2017), that 2) AOB neurons are heterogeneous in their structural (Larriva-Sahd 2008) and functional (Yoles-Frenkel et al. 2018) properties, and that 3) information about relevant stimuli is likely distributed over populations of neurons (Kahan and Ben-Shaul 2016). Large-scale monitoring would allow identifying the spatial and temporal patterns of stimulus-induced neuronal activity. Although conducting large-scale recordings is not trivial in any neural structure, the AOB presents a particular challenge due to its anatomical location. One approach is large-scale electrophysiological recordings, but these are usually limited to one plane and do not provide definite determination of cell body location. A more appropriate approach is Ca²⁺ imaging. Until recently, this approach was not readily applicable to structures such as the AOB, but recent technical developments for deep brain imaging—for example, insertion of gradient-index lenses (Yang and Yuste 2017) or microprisms (Andermann et al. 2013; Low et al. 2014)—promise to overcome this hurdle and reveal the response dynamics of large AOB ensembles.

Expanding the range of animal models—and examining variability among subjects

As we stated in the Introduction, our current emphasis on the rodent AOS, and the murine system in particular, results from the fact that most recent studies on the AOS involve this animal order. However, perhaps even more than other sensory systems, the AOS, which is dedicated to processing signals from other organisms, is likely to exhibit species-specific properties. Most obviously, particular lifestyles could affect vomeronasal receptor repertoires. Merely examining the numbers (rather than sequences and structures) of distinct vomeronasal receptors, and the relative prevalence of V1R and V2R receptors, reveals prominent differences across species (Ibarra-Soria et al. 2014a; Silva and Antunes 2017). For example, among mammals, rodents exhibit particularly high numbers of V2Rs, which are entirely absent from many other species (e.g., dogs, cats). By contrast, reptiles and amphibians express more V2Rs than V1Rs (Silva and Antunes 2017). Another factor that was examined comparatively is VNO size (Dawley 1998), and perhaps more importantly, the relationship of the VNO duct to the nasal and oral cavities (Bertmar 1981; Wöhrmann-Repenning 1984). This aspect too varies across species and is likely to reflect different adaptations of the AOS to sample stimuli from different sources. Beyond these molecular and anatomical aspects, which are relatively easy to identify, there may be more subtle differences involving the control of VNO sampling, processing of semiochemical information by local circuits, and interactions between early and central AOS structures. Thus, detailed studies of AOS structure and function in other species, with different social structure, predator pressures, nutritional demands, and diurnal cycles, will certainly provide a more complete and less biased understanding of AOS function.

In the same context, like many other studies that use mice as model organisms, most physiological analyses of the AOS have focused on a small number of inbred mouse strains. This applies both to the source of natural secretions and, to a larger extent, to the strains used as subjects. Although the effects of inbreeding and artificial selection in laboratory conditions can be significant for any physiological system, they are particularly likely to affect a system with a central role in social communication. Indeed, it is not hard to appreciate that laboratory breeding conditions can alter both the signals emitted by individuals and the sensory systems used to detect them. For example, mice that emit high concentrations of aggression-eliciting compounds may be artificially selected against, because they are either likely to be injured by other mice, or to injure them. Likewise, females with acute sensory systems may be more susceptible to pregnancy block and thus may be less productive and hence undergo negative selection. Furthermore, although the experimental benefits of working with a genetically amenable inbred model organism are obvious, important insights could be gained from comparison of responses to chemical cues from various inbred strains, and comparison of responses to the same stimuli in different strains. The rationale for the former is to identify the range of cues emitted by different strains (Robertson et al. 1997; Kimoto et al. 2007; Ferrero et al. 2013). The rationale for the latter is that distinct strains may differ in their sensory machinery. Indeed, across commonly used laboratory strains, the Vmn1r/Vmn2r gene repertoire displays unusually high levels of genetic variation, including several putative loss-of-function mutations (Logan 2015) as well as variation at the level of gene expression (Duyck et al. 2017). Moreover, at the individual level, RNA_seq studies revealed that distinct receptors are present in the VNO at widely ranging abundances (Ibarra-Soria et al. 2014b). Whether such proportional differences reflect the biological relevance of the receptors is currently unclear. In addition to studying different inbred strains, it is perhaps no less important to examine wilder mouse populations as both stimulus donors and subject animals. Such mice likely represent a more diverse source of semiochemicals (Beynon et al. 2002; Sheehan et al. 2016; Stopka et al. 2016) and a more ethologically relevant instance of chemosensory processing and behavior (Chalfin et al. 2014), thus allowing a better assessment of the “native” function of the AOS.

Supplementary material

Supplementary material can be found at http://www.chemse.oxfordjournals.org/

Conflict of interest

The authors declare that they have no competing interests.