Evolutionarily conserved mechanisms for the selection and maintenance of behavioural activity

Abstract

Survival and reproduction entail the selection of adaptive behavioural repertoires. This selection manifests as phylogenetically acquired activities that depend on evolved nervous system circuitries. Lorenz and Tinbergen already postulated that heritable behaviours and their reliable performance are specified by genetically determined programs. Here we compare the functional anatomy of the insect central complex and vertebrate basal ganglia to illustrate their role in mediating selection and maintenance of adaptive behaviours. Comparative analyses reveal that central complex and basal ganglia circuitries share comparable lineage relationships within clusters of functionally integrated neurons. These clusters are specified by genetic mechanisms that link birth time and order to their neuronal identities and functions. Their subsequent connections and associated functions are characterized by similar mechanisms that implement dimensionality reduction and transition through attractor states, whereby spatially organized parallel-projecting loops integrate and convey sensorimotor representations that select and maintain behavioural activity. In both taxa, these neural systems are modulated by dopamine signalling that also mediates memory-like processes. The multiplicity of similarities between central complex and basal ganglia suggests evolutionarily conserved computational mechanisms for action selection. We speculate that these may have originated from ancestral ground pattern circuitries present in the brain of the last common ancestor of insects and vertebrates.

1. Introduction

Brains of roving animals have evolved to make decisions in response to change in their internal environment based on cues indicative of, for example, nutritional status, and to change in their external environment. Decisions of the first kind involve relatively simple and ancient autonomic circuits that sense and regulate expected variations. Decisions of the second kind involve more complex circuits that serve to detect external events, weigh their saliency and relevance, and decide when and how to act on them [1]. Although different species have evolved elaborate and specific circuits that match their specific ecologies, with respect to the selection of appropriate behaviours brains nevertheless have deep commonalities. A honeybee and a nectar eating bat undertake comparable foraging tasks with central nervous systems of vastly different size and complexity. However, their brains share two fundamental properties. Both can recollect prior actions and select appropriate actions on the basis of present stimuli and recalled associations.

Here we consider evidence that in arthropods and vertebrates, parts of the brain mediating these properties derive from genealogically corresponding circuits, and that without those parts, behavioural activity would be restricted to reflex-like actions. In a cockroach, for example, forward walking can be triggered by touching its abdomen, even after its brain has been disconnected from its ventral nerve cord. But the animal is unable to respond to novel external cues and change its direction [2–5]. A decerebrate cat will walk with a normal gait on a treadmill, but it is unable to respond to novel stimuli that would elicit a change in its gait [6]. In arthropods and vertebrates, selection and maintenance of adaptive motor actions involve more than reflexes. Cerebral ganglia are required for adaptive motor actions, and in both taxa specific regions of the forebrain are required for the selection of such actions. In insects, these regions comprise interconnected centres collectively known as the central complex (CX), which when injected with venoms containing inhibitory transmitter substances cause ataxia and an inability to initiate and mediate voluntary movement [7]. In vertebrates, interconnected centres in the forebrain, known as the basal ganglia (BG), are required for the selection of voluntary behaviours, while their pathological disruption results in ataxia, paralysis and other behavioural deficits as seen in Parkinson's and Huntington's disease [8].

The insect CX is composed of five discrete interconnected midline neuropils in the anterior-most segment of the brain (protocerebrum). These are the protocerebral bridge (PB), fan-shaped body (FB), ellipsoid body (EB), the paired noduli (NO) and the paired lateral accessory lobes (LAL) [9,10] (figure 1a). The vertebrate BG consists of an arrangement of basal forebrain nuclei that includes the striatum (which in primates consists of caudate, putamen and ventral striatum, including nucleus accumbens), the internal and external domains of the globus pallidus (GPi and GPe, respectively), the subthalamic nucleus (STN), and the substantia nigra pars reticulata (SNr) (figure 1b,c) [11].

Figure 1.

Principal arrangements of the insect central complex, the lamprey and primate basal ganglia and their associated loops. (a) (i) Simplified schematic of the central complex (CX) showing connections between the protocerebral bridge (PB), fan-shaped body (FB) and ellipsoid body (EB), along with two satellite neuropils, the Gall and lateral accessory lobe (LAL). The noduli have been omitted. The PB is divided into synaptic modules which, depending on the species, vary between symmetrical arrangements of 9 + 9 (in Drosophila, Mantis religiosa) and as few as 5 + 5 (Notonecta) units. PB modules encode sensory representations (visual/tactile) from eight sectors on each side of the animal's long axis. Left and right representations of this ‘where’ code are relayed across eight modules of the FB such that left and right maps from the PB are compared. Outputs from the FB project into 16 modules of the EB such that modules representing opposite sectors are adjacent. The PB receives numerous inputs, including neurons entering laterally (arrows) carrying high-level information about visual motion direction. Columnar module in the FB are intersect by many dendritic trees and terminals (two shown) that likewise carry synthesized sensory information about complex parameters (‘what’ inputs) as well as a broad palette of modulatory peptides. The EB also receives a variety of inputs (also ‘what’ afferents), such as from the Gall and other satellite neuropils. Different combinations of where and what inputs result in different levels of activity in EB modules. Competing strengths of activity among EB modules result in a few achieving a stable output to the LAL. Further connections link the LAL system to pre-motor descending neurons (not shown). (ii) Comparison between sensorimotor and associative loops in insects (using the anatomy of the Drosophila as a general model) and sensorimotor, associative and ventral loops in mammals (using primates and humans in particular as a general model). Spatial organization is highlighted by the presence of numerically ordered modules in PB, FB, EB and LAL. The grey and bold black fonts indicate modules on the left and right side, respectively. (b) Anatomical representation of the re-entrant neural circuits characterizing sensorimotor and associative selections in the lamprey, which diverged already 560 Ma from the vertebrate lineages. (i) The first re-entrant neural circuit involves BG, thalamus and areas such as the optic tectum or the MLR, which provide sensorimotor inputs to the striatum via thalamus and receive direct inhibitory output from the BG. (ii) The circuit involves BG, thalamus and pallium, which projects directly towards the striatum and in turn receives mediated (via thalamus) inhibitory output from the BG. (c) Sensorimotor, associative and ventral (limbic) loops in mammals, here shown for primates. In the left hemisphere in humans, the different colours highlight the connectivity between separate areas in the cortex and their specific targets in the striatum and thalamus. This parallel partial segregation is maintained throughout the basal ganglia in the GP (globus pallidus), STN and SNr. Abbreviations: PB, protocerebral bridge; FB, fan shape body; EB, ellipsoid body; LAL, lateral accessory lobes; Pal, pallium; Str, striatum; MLR, mesencephalic locomotor region; Thal, thalamus; GPe, globus pallidus external segment, GPi, globus pallidus internal segment; STN, subthalamic nucleus, SNr, substantia nigra pars reticulata; Cau, caudate; NAcc, nucleus accumbens; Put, putamen.

Both the CX and the BG share extensive similarities in their heritable ontogeny and behavioural performance, and previous analyses have identified multiple correspondences between them. These include their embryological derivation and orthologous genetic specification, neural architectures, neurochemical attributes and physiological properties as well as behavioural outcomes of neuronal activity, including pathologies [12,13]. Together these correspondences imply a common ancestral origin of circuits that have diverged over a timespan of more than 540 Myr to provide the insect CX and the vertebrate BG. Here we extend this comparative analysis and identify common principles underlying the functional anatomy of the CX, the BG and their associated circuits. Our analysis suggests that evolutionarily corresponding computational mechanisms underlie the selection and maintenance of adaptive behaviour in insects and vertebrates.

2. Clonal unit architecture and functional compartmentalization

Both the CX and BG substructures derive from neural stem cells of the basal forebrain [12] that generate lineage-related sister cells. Here we first consider the BG. Earlier studies in rats showed that injection of [³H] thymidine, as well as retroviral-mediated gene transfer, marked the progeny of individual progenitors at different times of embryonic development that in turn identified the lineage relationship of striatal neurons [14,15]. These studies also revealed that neuronal birth dates define the segregation of striatal neurons, with clonal units either populating striosomes or matrix [14], thus contributing to the formation of functionally distinct compartments in the striatum [16,17]. More recent studies using lineage analyses of genetically modified mice expressing a traceable marker, identified that Nkx2.1-expressing progenitor cells of the embryonic subpallium generate distinct subpopulations of interneurons of the striatum as well as projection neurons of the globus pallidus [18], including cholinergic and GABAergic interneurons [19]. Moreover, analysis of mouse mutants revealed a Dlx1&2-dependent sequence of transcription factor activity required for the specification of striatal neurons [20], suggesting that neuronal cell fate within BG substructures is determined by lineage relationship and their birth time/order, which are mediated by combinatorial codes of selector gene functions.

Comparable regulatory programs for the spatio-temporal specification of neuronal fates have been identified for the insect CX. Using genetically marked neural stem cells, called neuroblasts (NBs), studies of Drosophila reveal that the CX derives from a limited number of NBs whose lineage-specific progeny constitute specific columns, layers and modules of CX substructures [21–25], thus mediating the formation of functionally distinct compartments comparable to the neural organization in the vertebrate BG. Such studies, which use mosaic analysis with a repressible cell marker (MARCM), also showed that lineage-specific progeny acquire a neuronal identity owing to their birth time and order directed by the spatio-temporal activity of selector genes. For example, a detailed analysis of a group of six neurons innervating layers of the EB and LAL in a cell type-specific manner demonstrated that they are generated in an invariant contiguous order from one single progenitor cell [26]. This process requires Chinmo, a broad complex, tramtrack, bric-a-brac (BTB)-zinc finger nuclear protein, which selectively regulates the third temporal identity among the six neurons [26]. These data provide evidence that CX substructures share common lineages in functionally organized groups, mediated by genetic mechanisms that link birth time and order to neuronal identity and function. Given that comparable mechanisms underlie the formation of BG nuclei, together these findings suggest that genetically encoded clonal units, also referred to as ontogenetic clones [27], are evolutionarily conserved cytoarchitectonic modules underlying the heritable ontogeny and reliable performance of BG and CX circuits.

3. Parallel-projecting, partially segregated circuits for sensorimotor and associative representations

Previous comparative analyses revealed that the CX and BG have executive control over comparable behaviours, and that their development or disease-related dysfunction can lead to homologous pathologies, including movement disorders, such as Parkinsonism, as well as neuropsychiatric disorders like schizophrenia, essentially affecting goal-directed behaviour and habitual control [12]. To identify the underlying computational mechanisms, we further examined the functional anatomy of CX and BG circuitry.

(a). Central complex circuitry

Histological, immunocytochemical and clonal analyses reveal that the functional anatomy of the CX is built on three structural principles: columns, modules and layers [21,23–25,28,29]. Columnar projection neurons are those that extend through the depths of three successive centres (neuropils): the PB, FB and EB, which are themselves further divided into discrete domains (figure 1a(i)). Columnar neurons are reiterated across the lateral extent of these neuropils, which are anatomically subdivided into modules. Each module usually contains the same set of columnar neurons, certain of which encode spatial information about sensory events surrounding the organism. Such events include tactile [30] and visual cues [5,31–34]. Modules are intersected by layered arrangements of wide-field dendrites and terminals, in particular, neurons carrying neuroactive peptides or that relay information indirectly from learning and memory processing regions, such as the mushroom bodies [34]. Their terminals contribute to the stratification of the FB and EB where they intersect columnar neurons disposed in the repeated arrangement of modules across the CX [21–25,28,29]. In general, this arrangement provides a substrate for multisensory space to be mapped across the CX, whereby a representation of a defined segment of sensory space is functionally represented in each module.

In the Drosophila PB, there are 18 such modules. Nine each side of the midline, represent one-half of the sensory hemisphere [21–25]. Axons from each half of the PB distribute across the entire width of the FB, which is the next level of the CX. This arrangement ensures that, within the FB, corresponding loci from the two halves of the sensory hemisphere interact [28,29]. The layered organization of efferents to the FB provide high-level information about sensory events irrespective of location [35]. CX modules are thus disposed to compare encoded events across the entire representation of sensory space thereby permitting, in principle, global assessment of stimuli. Interactions between modules and strata, together with modulatory aminergic inputs extending across modules (mainly dopamine, and also serotonin (5-HT) and octopamine), ultimately shape the output from the CX to neuropils that interact with descending channels carrying information from the brain to sensorimotor circuits in thoracic and abdominal ganglia [5]. Recordings and experiments, in which lesions or local stimulation result in specific behavioural defects, suggest that it is the interactions among modules that determine the nature of the expression of motor actions [36]. However, for this to happen they must provide outputs to at least one computational layer that determines which of many concomitant sensory signals are the most relevant in terms of whatever behaviour in which the insect is engaged. Such outputs from the FB supply the EB, the deepest level of the CX, and it is at this level that arrays of tangential neurons provide reciprocal connexions among modules allowing such decisions to be made [37] and relayed to the bilateral arranged LALs and their connections to descending channels (figure 1a).

(b). Central complex circuitry in sensorimotor transformation

At least three sensory modalities (visual, mechanosensory, temperature) are relayed to the PB and FB in the form of highly structured codes [10]. Each represents one of many presumably simultaneous sensory events as well as recollections that must be assessed for their most probable adaptive value. The computational role of the EB is to determine what within this incoming stream of data is best translated as possible motor actions. It is proposed that outputs from the EB serve to gate those parts of the LAL that have executive control of the activity of descending channels, the role of which is to appropriately modify local sensory-motor circuits to accomplish complex motor actions. It is assumed that reafferent copy from the motor output, and the LAL to the CX, closes the feedback loop (see below).

This conceptual framework [12,38] is supported by a number of studies using targeted inactivation of the CX that identify specific regions and neuronal subtypes as essential anatomical substrates for the selection and maintenance of behavioural activity, ranging from courtship and orientation behaviours, to visual memory and place learning, as well as attention, arousal and decision-making [39–51]. These data suggest that specific sensorimotor and associative representations are integrated and processed in partially segregated neural loops of the CX, which is best illustrated by the functional properties of the EB.

The EB is composed of populations of nerve cells that have been described in several insect species [52]. In Drosophila, these so-called ring neurons (R-neurons) are classified into at least four subtypes (R1–R4) based on their morphology, synaptic organization and terminal arborizations that define concentric layers of the EB neuropil [24,25,29]. Previous studies showed that EB ring neurons of the R3 subtype mediate selection between opposing visual cues for orientation [53] and spatial memory formation [42]; whereas EB R1 neurons were shown to process place learning in a heat maze where flies had to find a hidden cool place in an otherwise noxious 36°C environment [48]. EB R2/R4 neurons were found to mediate visual pattern memory in a flight simulator where a fly could choose its flight direction relative to visual patterns [43]. The same EB R-neuron subtype proved to be involved in ethanol sensitivity and tolerance [54], and ethanol-induced locomotion [46], where flies were given the choice for ethanol intake. The same R2/R4 neurons have been found to regulate repetitive startle-induced arousal when flies were exposed to mechanical stimulation by repeated air puffs [44]. A comparable organization into partially segregated functional units has been observed for different layers of the FB involved in locomotion, visual orientation and memory (e.g. [35,39,43,51]). Moreover, recent connectomics-based information flow analysis suggests that this partially segregated organization not only applies within each CX substructure but also to columnar projections across them [55]. Given its structural organization into columns, layers and modules, these findings suggest a functional anatomy of the CX whereby parallel-projecting, partially segregated loops integrate and convey sensorimotor and associative representations for the coordination and control of adaptive behaviour (figure 1a).

(c). Basal ganglia circuitry

A comparable structural and functional organization in the basal ganglia also characterizes two overlapping classes of recurrent loops, so-called re-entrant circuits, that have been identified across vertebrate species [11,56,57]. Studies on lamprey (figure 1b), a jawless fish closer to the base of the vertebrate lineage than any other extant vertebrate species, highlight the presence of both classes of circuits and their extensive similarities with the equivalent neural structures in mammals (cf. figure 1c).

In the first class of these circuits, the BG establish a series of partially segregated re-entrant circuitries involving distinct parts of the thalamus and brainstem motor centres such as the optic tectum (superior colliculus in mammals), or the mesencephalic locomotor region (MLR) [57–62]. Each sensorimotor input source projects, via the thalamus, towards separate parts of the input nuclei of the BG (striatum and STN) and each in turn is directly inhibited by specific sub-domains of the BG output nuclei (GPi/SNr). This partial segregation enables the re-entrant circuits to process information separately, displaying high functional specialization in controlling motor programs associated with specific sensory stimuli (e.g. the optic tectum conjunctly controls visual stimuli and oculomotor activity) [60].

For the second class of circuits, a similar re-entrant loop involves BG, thalamus and pallium (cortex in mammals) [63]. Compared with the first class of re-entrant circuits, the pallium directly projects towards the striatum and the STN, and is indirectly inhibited by the output of the BG via the thalamus [64]. The partial overlap characterizing these two classes of loops, jointly with dopaminergic innervation [65], is thought to allow the pallium to function as both an associative system and a bridge for the exchange of sensorimotor information among parallel loops. In mammals, this function is exemplified by the extensive development of the cortex, which takes the place of the pallium in the re-entrant circuit formed with the BG and thalamus, enabling more elaborated motor control for voluntary movements. Studies in primates and rodents have identified the presence, alongside the subcortical re-entrant circuits [60], of three major re-entrant striato-thalamo-cortical circuits to control and select sensorimotor, associative and limbic information (figure 1c) [66,67].

(d). Basal ganglia circuitry in sensorimotor transformation

Both classes of thalamo-striatal loops exploit the structure and information processing of the BG to realize the same function of selection via gating [60,68–71], adjusting for different input and output nuclei. The input reaching the BG is spatially and somatotopically organized to preserve information about the input stimuli [67]. The presence of channels within the BG allows these nuclei to process separately each element in the input, maintaining, amplifying or suppressing them in their path towards the output nuclei of the BG [69]. Selection and maintenance of behaviour in the BG relies on the converging signal conveyed by three pathways into the gating systems of the GPi and SNr (figure 1b,c). These are called the direct, indirect and hyperdirect pathways, the first two of which originate in the striatum and the third originating in the cortex [66]. Direct and indirect pathways are characterized by two distinct populations of GABAergic ‘medium spiny neurons (MSN)’ that are distinguished by their morphologies and the expression of distinct dopamine receptor subtypes [8,72].

The direct pathway consists of direct parallel inhibitory circuits originating from striatal MSNs characterized by dopamine D1 (excitatory) receptors and projecting towards the GABAergic output nuclei of the BG (either GPi or SNr). The indirect pathway originates from striatal MSNs characterized by dopamine D2 (inhibitory) receptors and reaches the BG output nuclei via GPe. The GPe itself projects inhibitory connections to both GPi and SNr. Finally, the hyperdirect pathway bypasses the striatum and connects the cortex directly to the STN, which then sends glutamatergic projections to the GPi (figure 1c) [73].

4. Dimensionality reduction via sensory integration

The connectivity within PB–FB–EB–LAL and thalamo-striatal loops realizes another shared and essential feature of the CX and BG, termed dimensionality reduction. Dimensionality reduction describes the processing of inputs from a high-dimensional data space to a lower dimensional space, which in the brain entails the compression of information encoded by a large neuronal population to a smaller one (as is the case, for instance, for visual and olfactory inputs). Its efficiency is measured by the ability to preserve information from the original data space, which can be the combined activity of a population of neurons that encode specific variables such as the angle of a looming object, or the angle of limb movement [74].

In both CX and BG, dimensionality reduction is achieved by sensorimotor and associative loops, which receive information from basically all regions of the brain. None of the CX and BG neuropil/input nuclei are directly connected to brain regions that encode sensory data. Thus, processed input is integrated and conveyed within loops whereby dimensional space is significantly reduced. In the case of the rat BG, the cortico-striatal reduction is 10 : 1, followed by a 1000 : 1 reduction between striatum and the GPi and SNr output nuclei [74].

(a). Dimensionality reduction in the basal ganglia

In mammals, the ventral part of the striatum acts as a nexus for the integration of information received from the amygdala (object-related values), hippocampus (spatial-related value and novelty) and prefrontal and orbitrofrontal cortices (future outcomes) [75,76]. Processed information in the ventral loop converges also on the sensorimotor loop such that this information flow is spatially organized and is added to information conveyed from sensory areas of the motor cortex (feed-forward control). Here the process of selection is repeated, integrating sensory signals from different modalities [77], thereby triggering an appropriate motor response that is once again provided to the ventral loop as a reafferent (feedback) signal. Comparable feed-forward and feedback systems involved in motor control occur among the subpallial thalamo-striatal loops where this bidirectional information flow enables, for instance, coordination of visuo-motor information of head orientation and body postures [78]. The significance of dimensionality reduction [74] is nicely illustrated in freely moving rats. While exploring a cage, their striatal neurons’ firing rates encode both spatial and behavioural features at the same time [79], including head movement velocity [80], whereas the firing rates of specific classes of SNr neurons encode a cartesian x or y coordinate of the position vector [81].

(b). Dimensionality reduction in the central complex

Similar to its vertebrate counterpart, the input layers of the EB function as a nexus for heterogeneous sensory information (visual, tactile, haptic, gravitational). In Drosophila, incoming processed sensory inputs covering large areas of sensory space are integrated and conveyed by a population of columnar neurons and EB neurons, as has been shown for polarized light information that is assumed for navigation [32,41,45], for features and orientations of moving objects [34,51] and for the position of an object within the visual field in relation to the animal's own body position [34]. Thus, high-dimensional data space (e.g. a moving object across the visual field) is reduced to a lower dimensional space (e.g. the activity of columnar neurons), which is the case for both BG and CX. Accordingly, the stimulus-related neural activity of channels (for the BG) or columns (for the CX) has been shown to encode, for instance, a specific action or goal [82], movement velocity [80], space coordinates or body orientation [34,81]. Dimensionality reduction thus integrates sensory stimuli and behavioural repertoires, and therefore aids in the selection of motor actions, and the coordination and control of behavioural activity.

In addition to dimensionality reduction and sensorimotor integration, stimulus-related neural activities of channels (for the BG) or columns (for the CX) are thought to become reference signals that are coupled with the transformation of sensory input. Coupling enables the association of a behavioural selection with its immediate perceivable consequences in the environment [83]. Such feedback loops establish a neural gain [84] that can either amplify or suppress the signals represented in the channels or columns and thus corroborate or diminish a behavioural selection.

(c). Nonlinear dynamics and attractor states

In computational terms, re-entrant neural networks exhibit nonlinear input–output transformations. These dynamics can be described as complex energy landscapes, characterized by the presence of attractor states (figure 2) [85,86]. The spatially organized loops generate a series of parallel feedback microcircuits that compete against one another for the control of the activation pattern of the system [37,87–89]. Each channel is characterized by a specific neural gain, which generates an attractor in the energy landscape. Here, the stronger the gain characterizing the self-sustaining channel, the steeper and larger the attractor. When a sensory input perturbs the system, thereby activating a pattern of activity, it triggers a transition towards the closest stable point (figure 2). Once stable, the system maintains its activity until further perturbed.

Figure 2.

Energy landscape illustrating nonlinear dynamics of a hypothetical neural network. The nonlinear dynamics of neural networks can be described in energy landscapes, where any starting condition of the system will cause the network to evolve and change its activity towards the closest low energy state, the ‘attractor’. (a) In this first example, two different starting conditions (A and B), initially located in a high energy position, reach the bottom of the basin offered by a single vast attractor state. The initial considerable differences in the input are ignored by the system which, via state transitions, reach the same stable point. (b) In this second example, the same starting conditions (A and B) in a similar high energy starting position are characterized by very different state transitions. State transition for A results in reaching the closest low energy state of a small attractor state, if compared with the attractor represented in (a). Conversely, state transition for B causes movement towards a very shallow area, where minor perturbation (e.g. by noise) can result in reaching any of three different unstable points and further perturbation can trigger frequent switch among these weak attractors. The same illustrative heat maps of the energy landscapes of an arbitrary network are depicted twice using different perspectives to highlight its three-dimensional features.

In general, depending on the steepness of an attractor, a transition (and hence the appraisal of the sensory input) may take a very short or long time. At the bottom of the attractor basin (low energy state), the transition phase ends, competition with other potential attractors terminates and a behavioural selection is triggered so that each attractor state in the sensorimotor loop is associated with a unique motor response [34,81,90,91].

5. Neural mechanisms and computations for action selection

Adaptive behaviour can be parsed into a number of sub-functions, including the integration of multiple sensory stimuli (dimensionality reduction), the suppression of noise and irrelevant competing signals, and the detection and selection of the most salient stimulus in reference to the internal state of the agent. These operations must show a certain degree of context specificity. First, they enable switching between a configuration that needs quick adaptation to changes in the sensory input and another configuration that affords maintenance of a selection (persistent behaviour) [88,92]. Second, the motor selections that are eventually triggered as a response to the combined effect of the internal state condition and perturbation by sensory stimuli have to be updated on the basis of previous experience that exploits knowledge about the environment and predictive cues.

(a). Basal ganglia direct and indirect pathway activity in action selection and maintenance

Within the respective loops, EB and BG are ideally positioned for such selections, with their location being downstream of sensory inputs and upstream of motor outputs. For the vertebrate BG, a wealth of studies suggest that an interaction among the three putative pathways (direct, indirect and hyperdirect) determines the selection process among the different channels and attractor states [8,93]. For instance, optogenetic manipulations show that direct pathway stimulation in the sensorimotor loop facilitates behavioural activity, whereas indirect pathway stimulation decreases motor action [94], both of which are cooperatively active during voluntary movement [95,96]. These data support computational hypotheses suggesting that direct pathway activity is coherent with the gain of the loops, strengthening attractor states and facilitating the attainment of a stable point, when carrying out a selection. By contrast, indirect and hyperdirect pathway activities interfere with gain, resulting either in increased instability and longer transition phases, owing to shallow attractors, thereby diminishing the ability to carry out a selection [68–70,97] or in the generation of cyclic attractors [90]. Indeed, BG oscillations have been considered to emerge from pathological dysfunctions under low dopaminergic conditions, causing motor disabilities such as tremor (e.g. in Parkinson's disease) [97]. More recently, optogenetic manipulation and recordings in mice show that the indirect pathway actively contributes to action initiation and the control of contraversive movement, [95,98], a type of movement involving turns to the left or to the right that are controlled by the opposite hemisphere of the brain. This new understanding calls for further amendments to the standard computational model of BG motor control [8], suggesting that the indirect pathway may also play a fundamental role in controlling the frequency of contraversive oscillatory motor selections [90].

(b). The central complex ellipsoid body circuitry in action selection and maintenance

In the insect CX, the selection of behavioural activity is mediated by the EB, which largely consists of a GABAergic structure receiving spatially organized sensory information from the PB and FB conveyed towards the EB via columnar projections [21–25,29]. The LAL also receives parallel and spatially organized sensory information from both the PB and FB and is directly connected to motor areas, therefore concatenating functional elements that in vertebrates are represented by the thalamus and the motor area subjected to the gating process (figure 3a). As for the subcortical thalamo-striatal loops, whereby the BG directly gates several motor command regions [57], the neural architecture of the columnar organization involving EB and LAL points to the existence of two sensorimotor circuits replicating the same connectome [25] that putatively subserves the selection of different types of motor responses and sensorimotor associations. Finally, the LAL also projects segregated information towards the EB, realizing the feedback required for the reference signal and the input–output coupling (figure 3a).

Figure 3.

Comparison of action selection circuitries in the insect central complex and vertebrate basal ganglia. Schematic representation of sensorimotor loop, underlying action selection in the (a) insect central complex and (b) vertebrate basal ganglia, together with modulatory dopaminergic loop. In both insects and vertebrates, the sensorimotor loop is responsible for processing multiple sensory stimuli that are somatotopically organized. Information about the selections performed is projected backwards via feedback excitatory parallel connections. The loop enables the creation of attractor states, which in turn result in noise cancellation, the detection and selection of the most salient stimulus (winner-take-all functionality) and rapid switching among attractor states thereby adapting to changes in the environment. The dopamine loop, via differential dopamine release, modulates gain of the sensorimotor loop, which in turn amplifies or suppresses the information entering the EB in insects and the BG in vertebrates. The dopaminergic loop is responsible for long-term memory formation (via reinforcement learning) and dynamic alteration of attractor states resulting via maintenance of sensory-driven flexibility (short-term memory). The internal wiring of CX and BG only allows a partial comparison as the information in insects is as yet incomplete. In particular, (*) directionality and the presence of parallel connectivity within the EB is a likely explanation of recent behavioural data, but the exact composition of these plausible internal pathways is not yet known. (**) The Gall, connected between PB, EB and LAL, projects towards the EB, but the specific targets in terms of EB layers have to be defined (e.g. the Gall might project towards the output layers only, as for STN in vertebrates). Finally, (***) the gating function performed by the EB may be realized via either parallel or centre-off inhibitory connections: the presence of a directionality in the EB strongly suggests the presence of parallel gating as it would match the computational requirements for the system. Abbreviations: PB, protocerebral bridge; FB, fan shape body; EB, ellipsoid body; interm. layers, intermediate layers; LAL, lateral accessory lobes; MLR, mesencephalic locomotor region; DLR, diencephalic locomotor region; GPe, globus pallidus external segment, GPi, globus pallidus internal segment; STN, subthalamic nucleus, SNr, substantia nigra pars reticulata.

Consistent with this interpretation, we identified lateral inhibition within EB layers and observed that over-activation or inhibition of specific EB layers results in significantly different effects on motor behaviour [37]. Based on these observations, we hypothesize that the EB expresses a certain degree of directionality, which in turn points to the presence of specialized input and output layers likely to be found among EB R1–R4 and the recently identified posterior (P) layer [24]. This hypothesis is supported by the presence of a small accessory neuropil to the CX-LAL called the ‘Gall’ [22,25]. Based on its connectivity between PB, EB and projections towards the EB (figures 1a and 3a), the Gall might computationally play a role comparable to that of the STN, which exerts regulating tonic activity on the inner and output nuclei of the GP and SNr. These internal nuclei require a strong, tonic excitatory input to balance the lateral and afferent inhibitions derived from the striatum (for the BG) and from the other layers of the EB (in the CX).

(c). Action selections as transitions through attractor states

Computationally, selections are achieved by means of state transitions among attractors, whose shapes (e.g. steepness and surface dimension) are directly determined by the gain of each channel or column in sensorimotor and associative loops. The presence of multiple attractors in both insects and vertebrates is the key requirement for a winner-take-all mechanism that suppresses competing behavioural responses and avoids the risk of triggering multiple incompatible action selections. Multiple attractors guarantee the selection of the most appropriate response to a given salience. Conversely, oscillatory selections require the presence of cyclic attractors, which allow a single sensory input to be processed in a periodic transition between states, in a pendulum-like activity.

The BG neural architecture is compatible with both types of attractors, with a key role of the direct pathway for stable attractors and their maintenance, and a key role for the indirect pathway (involving the direct connection between the GPe and either the GPi or the SNr) for oscillatory attractors [90]. It is reasonable to assume that the EB may play a similar role, sharing with the BG the ability to control both tonic steady action selections and patterns of alternating (slow oscillatory) motor selections to regulate rhythmic movements. In support of this notion, Drosophila mutants with structural defects of the FB and EB are unable to walk straight, with their motor behaviour characterized by circling and severely reduced speed [39,99], suggesting that alternating motor selections required for straight walking are impaired.

In both vertebrates and insects, the presence of such differential activity requires an internal mediator to modulate the switch from a condition favouring stable selections and another favouring instability or patterns of selections. In vertebrates, this role is played by dopamine through its differential effects on direct and indirect pathway activities in the BG [95,96,100].

6. Dopaminergic control: short- and long-term alteration of action selections

The significance of a stimulus is weighed with reference to previous (stored) experience. Dopamine has been identified as the principal modulator for this computational task, which applies to both CX and thalamo-striatal loops [12,65]. Both EB in the CX and GPi/SNr in the BG convey inhibitory signals towards a dopaminergic area. In turn, functioning of the loops is thought to be highly affected by the fast (phasic) and slow (tonic) fluctuations of dopamine release. The presence, causes and dynamics of dopaminergic bursts match those required by prediction error signals, supporting the hypothesis that phasic dopamine controls learning and memory formation in vertebrates [101,102] and insects [103,104]. In terms of neural connectivity, phasic dopamine bursts allow the system to change the strength of the connections conveying sensory information towards the EB or the BG. In turn, this alteration changes the way a sensory stimulus will be weighed in the future [105], affecting the probability that the motor action that resulted in the dopamine bursts will be selected again, thereby realizing reinforcement learning (e.g. [87,89]). By a long-lasting process that strengthens or weakens the gain established in each channel or column of the sensorimotor loops, the energy landscape associated with the neural system is constantly updated. After a few trials causing dopamine bursts, sensory inputs and reference action signals are thought to be coupled with a learning process that results in stronger gain in the respective channels or columns and therefore wider and steeper attractors. Thus, the shape and the steepness of the attractors are ‘storing’ information about the most positive or negative motor responses (i.e. causing phasic dopamine bursts and learning) to a perceived input [106]. For instance, over-trained neural systems exhibiting habits [107] are characterized by vast and steep attractor states in their energy landscapes. In these systems, even a partial identification of the over-trained stimulus makes the system fall into the associated attractor, triggering the learned action selection.

Slow tonic dopamine influences also impact on the attractor state landscape of control loops, though their effect is limited in time and results in phenomena resembling short-term memory [42,108]. Fluctuations in tonic dopamine do not alter the strength of the projections conveying sensory input information towards the EB or BG. On the contrary, tonic dopamine temporarily amplifies (in the case of D1 receptors) or compresses (in case of D2 receptors) the strength of the signal conveyed. In the sensorimotor loop, this modulation alters the gain of all channels affected by such fluctuations in dopamine parameters, contrary to channel-specific alterations characterizing long-term reinforcement learning. As a consequence, the effect is not to favour a single sensorimotor response, but rather to make the whole system generally more stable (in case of amplification), or unstable (in case of compression). The first condition allows for maintenance of a configuration pattern and a selection despite significant alteration of the sensory input, whereas the second condition drives the system to more frequent changes in selections, which can now be caused by even minor changes in the sensory input [88].

7. Summary and conclusion

Our comparative analysis identifies extensive correspondences of the functional anatomy of the CX and BG and their associated loop architecture (figure 3). The CX and BG share lineage relationships in functionally organized groups (ontogenetic clones), mediated by genetic mechanisms that link birth time and order to neuronal identities and functions. Similarly, the connectivity of these circuitries and the associated functionality are characterized by dimensionality reduction and attractor states whereby spatially organized parallel-projecting, partially segregated and yet interconnected loops integrate and convey sensorimotor representations for the selection and maintenance of behavioural activity. The underlying action selection mechanisms include integration of multiple sensory stimuli, the suppression of noise and less relevant competing signals, and the detection and selection of the most salient stimulus, while simultaneously suppressing competing behavioural responses in order to prevent multiple, incompatible, selections and motor outputs. The resulting selection of a single sensorimotor response is modulated by differential dopamine signalling that can mediate short- and longer-term maintenance, and thus short and longer-term memory. Given the extensive similarities in their origin, genetic specification, circuit architecture and behavioural output [12], the described sensorimotor circuits epitomize Lorenz & Tinbergen's postulate [109] that phylogenetically acquired behavioural activity relies on the physiological function of CNS substructures, whose heritable ontogeny and reliable performance depend on a genetically determined program. Thus, the multiplicity of similarities described here identifies conserved computational mechanisms underlying action selection, suggesting a shared evolutionary origin of the CX and BG.

Authors' contributions

V.G.F. and F.H. conceived and wrote the paper with contributions from N.J.S. and R.J.D.

Competing interests

The authors declare no competing financial interest.

Funding

The Wellcome Trust Centre for Neuroimaging is supported by core funding from the Wellcome Trust (091593/Z/10/Z). This work was supported by a Wellcome Trust Senior Investigator Award to R.J.D. (098362/Z/12/Z); the UK Medical Research Council (MR/L010666/1), the Royal Society (Hirth2007/R2), the MND Association (Hirth/Mar12/6085; Hirth/Oct07/6233) and the Alzheimer Research UK (Hirth/ARUK/2012) to F.H.