Neuromodulatory control of complex adaptive dynamics in the brain

Abstract

How is the massive dimensionality and complexity of the microscopic constituents of the nervous system brought under sufficiently tight control so as to coordinate adaptive behaviour? A powerful means for striking this balance is to poise neurons close to the critical point of a phase transition, at which a small change in neuronal excitability can manifest a nonlinear augmentation in neuronal activity. How the brain could mediate this critical transition is a key open question in neuroscience. Here, I propose that the different arms of the ascending arousal system provide the brain with a diverse set of heterogeneous control parameters that can be used to modulate the excitability and receptivity of target neurons—in other words, to act as control parameters for mediating critical neuronal order. Through a series of worked examples, I demonstrate how the neuromodulatory arousal system can interact with the inherent topological complexity of neuronal subsystems in the brain to mediate complex adaptive behaviour.

1. Introduction

The physical principles that govern the interactions of the billions of neurons that make up the adult human brain remain poorly understood. Any system with such a vast set of interacting parts is capable of remarkably complex and heterogeneous behaviour, due in large part to the enormous degrees of freedom available to the system. The individual elements of the system—namely, neurons and glia—are also themselves inherently complex and functionally nonlinear, which further augments the complexity of nervous system dynamics. There is also robust evidence to suggest that the brain contains scale-free order—i.e. is organized across multiple spatial and temporal scales. For these reasons, it is no wonder that the dynamics emergent in the activity of the nervous system are highly complex and flexible, yet distributed and low-dimensional.

A framework that captures each of these key features is the critical point of a phase transition, which arises in complex systems that exhibit an abrupt transition between distinct states (e.g. order and disorder) [1,2]. Systems poised at (or near) the critical point exhibit a number of beneficial features consistent with a well-organized nervous system. First, activity at the critical point is self-perpetuating, poised between the stillness of quiescence and the blooming and buzzing confusion of runaway activity. Second, systems poised at the critical point are susceptible to changes in inputs across multiple orders of magnitude and they can organize their responses to inputs across a similarly broad diversity of scales. This neural flexibility is thought to form the basis of cognitively flexible yet robust motor plans that aim to take advantage of internal and external affordances [3,4]. Finally, systems poised near criticality also display diverging autocorrelations (a process known as ‘critical slowing down’) that further imbue the system with susceptibility to a wide range of different signals.

Over the recent decades, a wide range of approaches have been used to detect signatures of criticality in the brain. By tracking the distribution of action-potentials (spiking) activity or thresholded local field potentials over consecutive time bins in neural data [3,5–9], it has been clearly demonstrated that neural populations typically follow a characteristics power-law distribution, such that the probability of observing ever larger (fluctuations/spikes) and longer avalanches scale equivalently across many different recording techniques [2,6,10]. Other methods have looked at temporal activity and used detrended fluctuation analysis [11,12], the Hurst exponent [13,14], or the power spectrum [15] to provide further evidence for criticality in neural recordings. Using different approaches to assess autocorrelation in neural dynamics has led to the suggestion that the brain might actually be poised in a slightly subcritical regime [16], which would confer the benefits of criticality while providing robustness to noisy fluctuations in the system that may otherwise push a critical system over into the dangerous supercritical regime. Finally, there are elegant descriptions of how the hierarchical, modular organization of the structural connections between regions of the cerebral cortex may act to ‘stretch’ the critical-point into a broader quasi-critical regime [17], further imbuing the system with a susceptibility across multiple orders of spatio-temporal activity patterns while protective of crossing into the supercritical regime. In sum, there are an array of different approaches that each provide complimentary evidence that the brain uses features of criticality.

Despite substantial evidence for criticality in the brain, the search for the biological mechanisms that facilitate the critical transition (known as control parameters) has remained more challenging. Here, I advance the thesis that the ascending arousal system (AAS)—a set of subcortical structures connecting the brainstem, thalamus, hypothalamus, and basal ganglia to the cerebral cortex—is a plausible means for controlling critical brain dynamics. By modulating the excitability and receptivity of targeted neurons, the AAS is capable of shifting distributed neural activity so as to maximize the adaptive capacity of the emergent dynamics, simultaneously imbuing the brain with flexibility, robustness and efficiency. In this manuscript, I will first overview the basic principles of the AAS, and provide evidence suggestive of their role in mediating criticality in the brain. I will then elaborate on this position by highlighting idiosyncratic features of the different arms of the arousal system as differential control parameters that further augment and distribute signatures of criticality in ways that help to explain the functional signatures of different neurochemical pathways in the brain. In this way, I hope to provide a unifying bridge between neurobiology, systems neuroscience and statistical physics.

2. A complex, adaptive system under the control of the ascending arousal system

In dynamical systems, a control parameter is a property of a system mediating a phase transition that typically impacts the state or order of the system's components. A classic example comes from the Ising model of magnetism (figure 1b)—here, temperature is a control parameter that regulates the emergent magnetic strength of a ferromagnetic material. If the temperature is high, the ferromagnet is non-magnetic—all of the individual magnetic domains orient in random directions stemming from stochastic thermal fluctuations, leading to a lack of global order (and hence, no magnetism). At cooler temperatures, the fluctuations of individual magnetic domains are decreased, and as such, the magnet remains locked in a highly ordered (and hence, magnetic) state. At the critical temperature, a value in between these two extremes, the magnetic domains are coordinated across multiple scales—i.e. there are magnetic islands within magnetic islands inside (and so on). Crucially, at the critical point, the magnet is maximally susceptible to an external magnetic field—i.e. an imposed magnetic field can rapidly permeate through the magnet aligning domains. In the brain, critical processes are often couched in terms of chains of spikes that either dissipate (subcritical), exhibit runaway activity (super-critical) or sit in a zone in which they can be self-perpetuating facilitating communication (figure 1c).

Figure 1.

Control over criticality. (a) Second-order phase transitions, the amount of order (y-axis) is nonlinearly related to small changes in a control parameter (x-axis). Below the critical point (green dot), the system has low order (subcritical; blue), whereas above the critical point, the system has increasing order (super-critical; red). (b) The classic Ising model of magnetism: in the subcritical regime, there is no magnetism, as individual domains are misaligned; near the critical point, the individual domains are now more susceptible to an external magnetic field, causing fluctuations in magnetism; and in the super-critical regime, the domains align into an ordered magnetic state. (c) Analogously, in a neuronal population, a subcritical regime is associated with an absorbing quiescent state that arises from ineffective interactions between neurons, leading to stimuli that effectively dissipate; near criticality, there is maximal percolation among the network; whereas in the super-critical regime, heightened interactions are such that the network is over-excited by external inputs leading to runaway activity.

What features must a control parameter possess to mimic these signatures in the brain? First, like temperature in the Ising model, the control parameter should have (relatively) global influence over large portions of the system—the greater the coverage of the signal, the more likely it is that the system will respond coherently to changes in the control parameter. Secondly, the neural system responsible for instantiating system-wide control should be relatively low-dimensional, relative to the controlled population, such that signals can be broadcast to multiple different regions distributed across the system [18,19]. Finally, the control parameter should retain the capacity to alter the susceptibility of the individual elements of the system. In the case of the ferromagnetic material, the increased susceptibility of the magnet is mediated by a diverging correlation length between spatially separated domains. With these simple characteristics, subtle changes in the control parameter (at the critical point) can have large effects on the order (and hence, the magnetism) of the system. The key question then becomes—where do we see evidence of these features in the brain?

Although there are numerous regions in the brain that share some (but not all) of these features, such as the diffusely projecting matrix thalamic nuclei [20], there is one system that aligns remarkably well—namely, the ascending arousal system (AAS; figure 2). The AAS is a collection of highly conserved nuclei in the brainstem and forebrain that are autonomously active during wake (and various states of sleep), send unmyelinated axons widely across the brain, contain substantially smaller numbers of neurons than their projection targets, and alter the excitability (and hence, susceptibility) of neurons and glia in efferent targeted regions [21–24]. As outlined above, these features align precisely with the requirements of an effective control parameter for a complex dynamical system like the brain. As is the case with biological systems, the specific mechanisms by which the AAS alters the susceptibility is highly informative of their computational role in the brain. Hence, I will outline these features, and (where possible) link the elements of the AAS to their computational signatures through empirical and theoretical investigations.

Figure 2.

The AAS alters neuronal gain. (a) The AAS comprises a set of autonomously active nuclei in the brainstem and forebrain that project diffusely around the brain, albeit with idiosyncratic projection patterns. Key: nBM, the cholinergic nucleus basalis of Meynert (nBM; light green); the orexinergic lateral hypothalamus (LH; light orange); the histaminergic tuberomammillary nucleus (TMN; dark orange); the dopaminergic ventral tegmental area (VTA; blue); the cholinergic pedunculopontine nucleus (PPN; dark green); the noradrenergic locus coeruleus (LC; red); and the serotonergic dorsal raphe (DR; purple). (b) Upon release, neuromodulatory ligands predominantly interact with G-protein-coupled receptors (G_q [green] and G_i [green]); (c) this causes a change in the relative spiking output (ΔQout) of a neuron for a given change in current (ΔI), which is also referred to a change in gain (dQ/dI). Key: Q, firing rate. Figure adapted from [21].

The neurons that comprise the AAS spike a relatively constant rate during wake [23–25]. This pervasive tonic spiking is actually a well-known characteristic that differentiates the waking brain from the same system during the deep stages of sleep (i.e. NREM; [23,26]), anaesthesia [13,26,27] and coma [28]. Fascinatingly, electrical stimulation of different regions within the AAS can wake an anaesthetized animal [29,30]. Given that these same processes can be mimicked through modelling the impact of the AAS, this thus provides evidence for critical processes mediated by the arousal system in the brain. Importantly, intermediate patterns of activity are also linked to specific functional signatures—for instance, the capacity to perform complex cognitive tasks has been linked to intermediate levels of the neuromodulator, noradrenaline, in the prefrontal cortex [31,32]. In the case of noradrenaline, this inverted U-shaped relationship between noradrenaline levels (i.e. arousal) and cognitive function is known as the Yerkes–Dodson relationship [33].

The majority of nuclei that comprise the AAS project relatively diffusely throughout the brain. The noradrenergic locus coeruleus, serotonergic dorsal raphe and histaminergic tuberomammillary nuclei are the most diffuse [24,34–36], whereas the dopaminergic substantia nigra/ventral tegmental area [37] and cholinergic basal nucleus, pedunculopontine nucleus/laterodorsal tegmentum and medial septum [38,39] projecting in a much more targeted, segregated fashion. Through these widespread contacts, the local activity of neuromodulatory nuclei can be distributed across large regions of the brain, allowing for relatively low-dimensional, coordinated responses across the nervous system [21,40,41]. Much like in the example of the magnet, small changes in neuromodulatory tone, particularly when they occur rapidly in time (i.e. in phasic bursts), can enact widespread change across the brain that mimics known features of critical systems.

Upon stimulation, the axonal fibres of the AAS release neuromodulatory ligands from stored vesicles in their terminal fibres. These ligands are released into the synapse or extracellular space, where they interact with trans-membrane G-protein-coupled receptors. These receptors in turn enact major conformational change to the internal milieu within neurons and glia in ways that alter the excitability (or gain) of targeted regions [21]. There are two major classes of neuromodulatory receptor that are used by the different neuromodulatory systems [42]: high-affinity G_s/i receptors_, which alter the refractory period and resting potential of targeted neurons; and low-affinity G_q receptors, which release intracellular Ca²⁺ ions from internal stores, and hence shift the membrane potential of the cell closer to its reversal potential (thus augmenting the likelihood of action potential formation). Depending on the density of receptors, their precise location on the neuron (i.e. either pre- or post-synaptic) and the identity of the cell on which they are located (i.e. an excitatory or inhibitory neuron), the activation (or silencing) of these pathways can have substantial impacts on emergent activity patterns.

By changing the gain of excitatory and inhibitory neurons, neuromodulatory ligands can act as control parameters for specific neural subpopulations, and in turn alter the manner in which they process information. Considering the arousal system in this light has specific implications for a system-level understating of decision making. So how might this process play out in a population of neurons distributed across the brain? An emerging consensus describes decision-making in the brain as the accumulation of information distributed across multiple sites in parallel [43,44]. Within this framework, decisions reflect the presence of neuronal ‘quora’ (figure 3), each signalling for a particular option, such as an outcome, an action plan or the presence (or absence) of a particular feature. While the subtleties of this process are still being investigated, there are numerous examples of distributed decision-making processes in biology that can provide intuition for how the same processes may play out in the human brain [45]. For instance, bacteria use chemical signals to track population decisions [46], and both ants [47] and bees [48] appear to rely on positive feedback patterns to determine the appropriate time to move to a new site, and in which direction to travel. In each of these cases, a key variable that tracks the ability to form quora is the balance between the amount of information stored in individuals and the amount shared among the population (figure 3; [49]).

Figure 3.

Controlling the systems that define adaptive decision making. During an information accumulation process, it has been suggested that decisions emerge through a process of information distributed across multiple sites in parallel. For instance, take an abstract scenario, such as an animal making a decision as to whether to commit to option A [red] or option B [blue])—this could stand in for a movement, a choice or a strategy (or any number of other scenarios). In this case, control parameters can provide useful catalysation of different aspects of the decision-making process, depending on which sub-process the control parameter was influencing. As I have argued above, neuromodulators (NM) are well-placed to play this role in the brain, and hence could signal: (a) an information processing signal, which could reflect the rate at which information is accumulated (NM₁; green); (b) an urgency signal, which could stand in for whether a quorum had not yet been obtained (NM₂; orange), often necessitating greater effort or flexibility; and/or (c) a satiation signal, which could identify whether the accumulation phase has been successful (NM₃; purple), which in turn would necessitate inhibition of the accumulation process.

Importantly, this balance between segregated information storage among individuals and shared information transfer is precisely the variable modulated by the neuromodulatory ligands of the AAS. Indeed, evidence from computational modelling suggests that the gain-altering mechanism of neuromodulatory systems can facilitate precisely this informational transfer [50], wherein changes in neural gain can alter the systems-level in the topological configuration of macroscale brain networks, as measured by techniques such as fMRI [40,41]. These computational links lead us to predict that, during cognitive processing, low-levels of neuromodulatory tone will mediate a self-dominated mode that is associated with relatively high local information storage [50] and a segregated network architecture [40,41], whereas at higher levels of neuromodulatory tone, heightened feedback between regions will lead to increased information transfer [50] and a more integrated network topology [40,41]. Although not interrogated through the lens of the AAS, there is evidence to suggest that this quorum-formation framework provides a parsimonious account of cortical activity during a two-alternative forced-choice [51].

These features of the AAS are all consistent with their role as control parameters in the brain. By augmenting (or diminishing) the signals propagating through trains of glutamatergic-mediated action potentials [52], the AAS is ideally placed to shape critical dynamics across the whole brain. But if this is indeed the case, one wonders why the brain would retain so many different classes of neuromodulatory chemicals to perform the role that could be maintained by, say, a single control parameter that is differentially received by a more nuanced receptor profile in distinct regions of the brain. One putative benefit for the diversity of neuromodulatory control that we observe in the brain is that it can allow the system to shift into different processing modes according to an animal's given set of needs [53–55]. Another (non-mutually exclusive) benefit of retaining multiple control parameters is that different neuromodulatory nuclei can recruit (and hence, either augment or diminish) the topological features inherent within the unique connectional architectures that characterize different regions of the brain [55,56] (figure 4). Although there are billions of neurons in the brain, the manner in which they are connected with one another is strikingly different depending on where the neurons reside. Nowhere is this difference more striking than in the differences inherent within the organization of the cerebral cortex (which is thought to be loosely organized into columns of cells that are connected in particular ways to one another; [57,58]; figure 4) and the cerebellum (which is the paradigmatic example of organized, modular architecture, with a precise cellular motif repeated across the entire cerebellar mantle; [59]; figure 4). Given that distinct computational benefits are presumed to emerge from the machinations of these different topological organizations [55,56], any neuromodulatory chemical that could augment one or the other could thus represent a difference that makes a difference in the nervous system.

Figure 4.

Neuromodulatory control over neural sub-system topology. Different arms of the ascending arousal system (AAS)—serotonin (5-HT; purple); acetylcholine (ACh; green); noradrenaline (NAd; red) and dopamine (DA; blue)—send widespread projections to different sub-systems in the brain (e.g. the cerebral cortex, thalamus, cerebellum, and basal ganglia), each of which have their own unique topological characteristics. By controlling critical dynamics in each sub-population in characteristic ways, the different arms of the AAS are capable of mediating a range of different systems-level processing modes within the same underlying structural architecture.

3. Shaping signal-to-noise properties in the thalamus

Neuromodulatory neurotransmitters play a crucial role in shaping evolving activity patterns in the thalamus. In particular, the addition of both acetylcholine and noradrenaline [60,61] has been shown to shift the thalamus from a relatively sparse ‘burst’ mode (typically elevated during slow-wave sleep and quiescence) to a more active ‘tonic’ mode (which tends to occur more frequently during wake [62]; though see [63]). By contrast, the two neuromodulators also have opposing effects on the reticular nucleus: acetylcholine hyperpolarizes, whereas noradrenaline depolarizes the inhibitory nucleus [60]. The systems-level implications for these effects present interesting conundrums: although the heightened signal-to-noise in the noradrenergic state [64] is consistent with our augmented sensory discrimination during a relatively vigilant state [65], the noisier background signal in the cholinergically modulated thalamus is more difficult to unify with the focused, normalization [66] and stabilized brain state trajectories [22,67] typically attributed to the cholinergic state. One interesting possibility (that remains to be confirmed empirically) is that the extremes of cholinergic modulation on neuronal variability only emerge in purely cholinergic states (such as rapid eye movement sleep; [68]), but are otherwise kept in check by other neuromodulatory systems (such as the noradrenergic, serotonergic and dopaminergic systems) that are active during the waking state.

By contrast to noradrenaline and acetylcholine, the predominant action of serotonin in the thalamus is to hyperpolarize the thalamic relay nuclei and thus, inhibit the tonic firing mode [69]. Interestingly, the magnitude of the hyperpolarizing response is typically more pronounced in diffusely projecting, higher-order nuclei [69]. The effect of increased thalamic serotonin would thus likely bias the system toward a deterministic, feedforward mode of processing by diminishing the role that feedback processes could play in the dynamic evolution of the global brain state. The associative nuclei of the primate thalamus also receive substantial dopaminergic inputs [70], though the precise role of these projections is currently poorly understood [71,72]. Given the crucial interactions between the thalamus and cortex, which provide important constraints over the balance between excitation and inhibition [73] and shape an animal's conscious awareness of the world around them [74–76], I propose that neuromodulatory inputs modulate the excitability of the thalamus in a way that directly effects the criticality propagation of activity in the cerebral cortex (figure 1c).

4. Shifting between feedforward and feedback modes in the cerebral cortex

The massive diversity and heterogeneity of both excitatory and inhibitory cells within the cerebral cortex allows it to process in a number of distinct modes, which in turn are modulated by the presence of different neurochemicals. For instance, inputs to the cortex from the thalamus (figure 5; dark red; [20]) or lower (more granular) cortical regions [77,78] can cause the pyramidal cells within a cortical region to send sparse spikes, and hence to function in a relatively deterministic, feedforward mode. By contrast, feedback projections arise from regions higher in the cortical hierarchy [77], and project to supragranular layers of lower cortical regions, wherein they innervate both interneurons and the apical dendrites of pyramidal neurons with cell bodies in L2, L3 and L5 (figure 5). These two streams of activity are presumed to combine together to instantiate a form of predict processing in the brain, albeit with distinct implementation-level predictions depending the particular cells under investigation [79–81]. For instance, one proposed implementation of predictive processing suggests that, when feedforward and feedback signals coincide within a precise (approx. 30 ms) temporal window, there is evidence that thick-tufted, layer 5 pyramidal-tract cells (L5_PT; figure 5; grey) undergo an ‘apical amplification’ and transition into an NMDA-receptor mediated burst mode of firing, that strongly increases their signal-to-noise properties [82] in a way that integrates their activity into an evolving state of conscious awareness [56,79,83].

Figure 5.

Neuromodulating the cortical μcircuit. (a) A selection of different excitatory and inhibitory cells types in the cortical μcircuit, along with some of the typical connections between cell types (note: this depiction is non-exhaustive). Key: Thal, thalamus; CBM, cerebellum; SC, superior colliculus; Str, striatum; Ctx, cerebral cortex; VIP, vasoactive intestinal peptide (pink); SST, somatostatin (light blue); PV, parvalbumin (red/orange); NGF, neurogliaform (dark red); CT, corticothalamic (light green); IT, intra-telencephalic (light orange and kelly green); PT, pyramidal-tract (black). (b) Examples of how different sub-populations of these cells are augmented by the excitatory (G_q or G_s) mechanisms associated with the four main classes of neuromodulatory nuclei: α1A (G_q) for noradrenaline (red); D1 (G_s) for dopamine (blue); M1 (G_q) for acetylcholine (green); and 5-HT_2A (G_q) for serotonin (purple).

There is evidence to suggest that the cholinergic system may favour feedforward modes of processing [84], either by exciting intra-telencephalic (IT) pyramidal cells [85] or by recruiting the parvalbumin-staining, fast-spiking GABAergic interneurons [86,87] that utilize feedforward inhibition [88,89] to facilitate high-frequency gamma rhythms in the cortex [90]. These neurobiological details may help to explain the links between cholinergic tone in the brain and leading computational accounts of focussed attention—namely, divisive normalization [66]. Other neuromodulatory classes can instead promote increases in feedback processing. There is evidence to suggest that neuromodulatory neurotransmitters can directly facilitate the interaction between the apical and basal dendritic trees L5_PT pyramidal cells [91–94]. Hyperpolarization-activated cyclic nucleotide-gated (HCN) Ih channels typically ensure that the apical and basal dendritic compartments of the L5_PT remain electrically isolated from one another [82,95,96]. Importantly, noradrenaline has been shown to inhibit these channels via the activation of α_2A receptors [31,93], which are the same receptors in the prefrontal cortex known to (somewhat paradoxically) augment cognitive function [32,97]. How might this work? When HCN Ih channels are blocked, the resting leak current reverses within the apical dendrites of L2, L3 and L5 pyramidal cells, such that the context-laden feedback signals from higher regions of the cerebral cortex that innervate the supragranular layers of cortex (i.e. the location of the apical dendrites) are able to temporally coincide with feedforward channels of information flow from infragranular layers (likely through the extensive horizontal projections of L2/3_IT pyramidal cells that make contact with L6 cells [98]) and transition the cell into burst-firing mode [99]. Apical amplification of L5_PT pyramidal cells in the somatosensory cortex of mice has recently been tied to supra-threshold perceptual episodes [100] and differentiates waking and anaesthesia [94], suggesting that this mechanism may be crucial for the mediation of our conscious awareness.

Other classes of neuromodulatory transmitters can also shape information flow in the cerebral cortex, particularly when they differentially effect unique cell classes in the brain. This is particularly true for inhibitory interneurons, which are of major importance for a number of cognitive, motoric and attention capacities [101–103]. For instance, the disinhibitory capacity of VIP+ interneurons [89,101], which synapse exclusively onto, and hence disinhibit [104] other (predominantly SST+) interneurons [105], is under the fast, ionotropic influence of both serotonin and acetylcholine [106,107], as well as the slow, metabotropic influence of noradrenaline and acetylcholine (figure 5b). The fast timescale of the ionotropic effects (on the order of microseconds) means that these cells can briefly shift a pyramidal cell from a feedforward into a feedback mode with high temporal precision [102]. This may help to explain why VIP+ interneurons in the barrel cortex of a mouse have been shown to be crucial for the deliberate movement of the whiskers [103] and active, wakeful behaviours [108], possibly through the augmentation of gamma band coherence [109]. By contrast, late-spiking neurogliaform cells [110] (figure 5a; red) are under predominantly metabotropic neuromodulatory control [107], suggesting that their late, hyperpolarizing effects within the cerebral cortex are only modulated at relatively slow timescales [105]. Far from being limited to these specific examples, we can expect that the coming years will bring a much finer appreciation of the relationships between neuromodulation and interneuron recruitment [90,111], particularly given the importance of inhibitory stabilization for mediating effective computation in the cerebral cortex [112–114].

The laminar topography of different neurotransmitter receptor families in the cerebral cortex is also of great importance [115–117]. In recent work that used quantitative in vitro receptor autoradiography to scan the post-mortem human brain at micrometre resolution, it has been shown that inhibitory 5HT₁ receptors, which are known to facilitate the balance between impulsive and goal-directed behaviour [118], are predominantly expressed in supragranular layers across the cortical mantle [115]. Similarly, the α₂ adrenergic receptors that facilitate pyramidal cell burst firing [31,93] are selectively enriched in supragranular layers in prefrontal cortex [115]. By contrast, nicotinic cholinergic receptors are typically enriched in granular layers, particularly in primary sensory cortices [115,119]. Activation of nicotinic receptors has been shown to inhibit both IT- and PT-type while activating CT-type pyramidal cells in frontal cortex of mice [120,121]. These mechanisms suggest that the balance between noradrenaline, serotonin and acetylcholine (among others) in the cerebral cortex places important constraints on whole-brain functional dynamics [122] in ways that can shift the critical dynamics within corticothalamic circuits in order to modulate patterns of adaptive information processing in the brain.

5. The distributed impact of dopamine on corticostriatal circuits

The diffuse nature of neuromodulatory projections implies that the effects of neuromodulatory ligands result from their multiple circuits in tandem, however much of the existing literature has (for good reason) focussed on individual regions one at a time. Perhaps the paradigmatic example of this is for the neurochemical dopamine. There is extensive literature linking the function of the basal ganglia with the relative levels of the neurochemical dopamine [123] that arise from the axons of the substantia nigra pars compacta. The prevailing notion is that dopamine acts to excite G_s-mediated D1 receptors of the ‘direct’ pathway in the striatum (which has the effect of releasing the tonic inhibitory effects of the globus pallidus internus) and inhibit G_i-mediated D2 receptors of the ‘indirect’ pathway (which has the opposite effect). Together, the balance between these two pathways is presumed to play an important role in the preparation and selection of behaviour [124], with the precise function depending on which cortical and thalamic regions are innervating the structure.

It is less well appreciated that these dopamine-sensitive systems also interconnect with precise circuits in the cerebral cortex that are differentially related to dopaminergic tone. Specifically, there is evidence that intra-telencephalic (L5_IT; dark green in figure 5a) cells with cell bodies in layer 5 predominantly express excitatory D1 receptors [125] and project to D1-expressing direct pathway spiny projection neurons in the striatum [126], whereas pyramidal-tract (PT-type [L5_PT]; grey in figure 5) express inhibitory D2 receptors [127] and project to D2-expressing indirect pathway striatal neurons [126]. Importantly, these two cell populations are differentially involved in action planning (L5_IT) and execution (L5_PT) [128–130]. This suggests that dopamine release from the ventral tegmental area and substantia nigra pars compacta does not just influence critical dynamics in the striatum, but also in the distributed excitatory populations in the cerebral cortex that project to these pathways as well.

These distributed circuits help to explain the role of dopamine in action selection (figure 6). When there is an action to be selected, the presence (or absence) of dopamine will innervate populations of L5_IT cells (e.g. red versus green) that then compete for dominance over one another. This competition is presumed to occur through circuits that send action potentials to L5_IT, which in turn are augmented by the presence of dopamine (i.e. they express excitatory D₁Rs). These same L5_IT cells innervate the direct pathway of the striatum (D₁R-rich cells in figure 6)—the activation of this circuit leads to disinhibition of the diffusely projecting matrix thalamic cells that send diffuse re-entrant projections to the supragranular layers of the cerebral cortex [56,131]. These thalamic projections (figure 6; orange) are precisely those responsible for transitioning L5_PT cells into a burst-firing mode (see above) that triggers action (in this case, choose ‘red’ or ‘green’), in part through the recruitment of cells in the colliculus, thalamus, pontine nuclei (and hence, cerebellum) and spinal cord [56,79,132]. By innervating the indirect pathway (i.e. D₂R-expressing cells), L5_PT cells are ideally placed to ‘switch off’ the striatum whenever a decision has been made to act (i.e. when a consensus is reached). This mechanism which is consistent with the fact that the striatum decreases its firing rate when an animal performs an action [133].

Figure 6.

Dopaminergic control over information accumulation and action. Different classes of dopaminergic receptors—the excitatory (G_s) D₁R and the inhibitory (G_i) D₂R—are preferentially expressed on different cells in both the striatum (Str) and cerebral cortex (the large trapezoidal structure). The distribution of these receptors is such that the presence (or absence) of dopamine—the putative control parameter of this process—likely facilitates a demarcation between two distinct information processing stages: at high levels of dopamine, such as when an animal is motivated, D₁R augment firing in L5_IT pyramidal cells (red and green neurons, depicting a hypothetical choice between two options: red and green) and direct pathway striatal neurons. The dopaminergically excited striatal neurons ultimately disinhibit (via the globus pallidus internus; GPi) diffusely projecting, matrix thalamic nuclei in the ventral tier of the thalamus (Thal; orange; [56,131]), which in turn innervate the supragranular regions of the cerebral cortex and help transition D₂R-enriched L5_PT pyramidal cells into a burst-firing mode (via apical Ca²⁺ waves; [79]) that triggers actions (via subcortical projections), while also innervating the indirect pathway of the basal ganglia (which is typically inhibited by the presence of dopamine via D₂Rs). This action ultimately cancels the thalamic disinhibition mediated by the direct pathway via inhibitory connections in the globus pallidus externus (GPe) and hence effectively signals the end of an information accumulation epoch.

How does this align with other neuromodulatory systems? Acetylcholine also has a major influence over the processing mode of the basal ganglia [134,135]. By contrast to dopamine, cholinergic input is actually more heavily enriched in the ‘indirect’ pathway, which is typically associated with the GABAergic suppression of thalamic and brainstem targets. Cholinergic ligands also act to increase the activity of cholinergic interneurons in the striatum, which have the net effect of inhibiting the direct pathway [135]. This suggests that a given circuit's ability to either facilitate or suppress a diffusely projecting matrix thalamic projection back to supragranular regions of cortex depends on the balance between dopamine and acetylcholine, rather than the absolute concentration of either neurotransmitter alone. It is also important to note that one of the major GABAergic interneuron classes in the striatum, the main input nucleus of the basal ganglia, is also enriched with cholinergic receptors [134]. Together, these facts have important implications for understanding the biological basis of the known relationship between cholinergic function and selective attention, which likely relates to cholinergic mechanisms in the cerebral cortex [86,87], thalamus [85] as well as the indirect pathway of the basal ganglia [135]. It is also interesting to consider the complex, nonlinear impact of serotonin on the basal ganglia [136], particularly in light of the strong inter-connections between the serotonergic dorsal raphe and the cholinergic system [137], as well as serotonin's role in facilitating different information processing modes in the brain, to which I will turn next.

6. Switching the balance of activity between the cerebellum and the cerebral cortex

While a great deal is known about the role of different neuromodulatory chemicals in the basal ganglia, less is known about the impact of modulation on the cerebellum [138]. There is evidence however to suggest that diverse regions within the extended cerebellar circuit are substantially modulated by serotonin [139]. For instance, the pontine nuclei [140], inferior olive [141] and cerebellar cortex [139] all increase their excitability in the presence of serotonin and shift many of these nuclei into a relatively high-frequency firing mode [139]. In addition, serotonin is known to boost the excitability of Lugaro interneurons in the cerebellar cortex [142], which play a role analogous to cortical VIP+ interneurons and inhibit other cerebellar interneurons. As in the cortex, the activation of these cells has the effect of disinhibiting the granule cells of the cerebellum. Combined with the knowledge that inhibitory (G_i/o) 5-HT₁Rs are highly expressed in the axon initial segment of cortical pyramidal cells, it is likely that serotonin is used to switch the balance of activity between the cerebellum and cerebral cortex, albeit likely in an inverted U-shaped relationship [143]: relatively low levels of serotonin likely inhibit the cerebral cortex and recruit the cerebellum, whereas higher levels likely recruit VIP+ interneurons (via 5-HT₃R) and PT-type pyramidal cells (via 5-HT_2AR) [144,145].

The cholinergic system also has an important influence over cerebellar function [146]. For instance, acetylcholine administration increases the firing rate of cortical pyramidal cells that contact the pontine nuclei [147], one of the two input structures to the cerebellum. In addition, there is also evidence that the granule cell layer of the cerebellar cortex [148], which is the most numerous population of cells in the human brain [149], and the deep cerebellar nuclei [150], the major output of the cerebellum, are both excited by increases in cholinergic tone. These processes are by no means selective to specific neurotransmitters [151], but they do suggest that elevated serotonin and acetylcholine have a preferential impact on the influence that the cerebellum has on shaping core thalamic activity, and hence feedforward activity patterns within the cerebral cortex.

7. Critical modulation of complex, adaptive dynamics

How might these processes play out across the complexity of individual sub-circuits within the brain, each of which can be augmented (diminished) by different combinations of neuromodulatory ligands (figure 3)? One parsimonious way in which to frame these ideas is in the context of an attractor landscape framework, in which state-based neural dynamics are framed as trajectories across an attractor landscape whose topography is shaped by opportunities for action (i.e. affordances; [55,56,152]). Within the attractor landscape framework, neuromodulators can be thought to manipulate the topography of the attractor landscape [22,153]. For instance, the addition of acetylcholine has been shown to accentuate the topography (i.e. make the landscape more rugged and wells deeper [67], likely through the ionotropic augmentation of granular layers [154]). By contrast, the addition of noradrenaline and serotonin would lead to a flattening of the landscape (i.e. which is equivalent integrating the brain [22,40,41,153], likely through the recruitment of supragranular feedback connections). This would have the effect of ‘resetting’ the network's landscape [22,155], allowing the brain to reach states that were otherwise hidden by the topography of the landscape. Others have argued that high levels of serotonin, particularly when the ligand agonizes 5-HT_2A receptors, can cause a similar flattening of the energy landscape [156]. In this way, the release of neuromodulatory neurotransmitters in response to sensory, cognitive or affective events would facilitate the dynamic reorganization of the brain so as to maximize adaptive behaviour. Although still in its nascent stages, I envisage the framework that outlines the logical organization of neurobiology in the language of dynamical systems as one of the crucial pillars of a foundational language for understanding the multi-scale adaptive dynamics of the nervous system in action.

When viewed through this lens, a key question becomes how can the nervous system provide control over its neuromodulatory control system? The entire AAS is under the further neuromodulatory influence of master controller neurons in the hypothalamus [157] and more ventral integrative regions, such as gigantic cells that together form an allodendritic core in the brainstem [158]. Numerous other subcortical structures are able to both excite and inhibit the projection nuclei of the AAS [159], often through the targeting of GABAergic neurons that predominate in the regions surrounding projection nuclei [53,160]. These peri-AAS cells also receive substantial inputs from a subset of L5_PT pyramidal cells [159,161]. Through these connections, the cerebral cortex retains closed-loop control over the activity of the neuromodulatory nuclei, which can in turn be tuned to particular cognitive scenarios, such as uncertainty [162] or Bayesian surprise [163,164]. Importantly, the impact of increasing the activity within these nuclei can have important effects on network topology [22,40] and information processing [50] in the brain. In addition, distinct locations in the frontal cortex have been shown to project to unique nuclei in the brainstem, suggesting nuanced control over ascending neuromodulatory tone in the brain (figure 6). Indeed, it has been shown that the capacity to control the temporal evolution of the brain state is related to heterogeneous patterns of neurotransmitter receptor profiles around the brain [122,165], allowing the closed-loop control over neuromodulatory chemicals to control the flow of the brain state around a hypothetical attractor landscape [55,56,152,166]. Interestingly, many of the cortical projections to neuromodulatory nuclei also heavily innervate local inhibitory (i.e. GABAergic) structures in the vicinity of the AAS hubs [159,167,168], suggesting that a key feature of the descending projections may be to control the excitatory/inhibitory balance in local neuromodulatory hubs (figure 7).

Figure 7.

Closed-loop frontal cortical control over ascending neuromodulatory activity. Distinct (though likely overlapping) regions of the frontal cortex send excitatory projections to distinct brainstem and forebrain nuclei: the cholinergic (ACh) basal nucleus of Meynert (BnM), the dopaminergic (DA) ventral tegmental area (VTA), the serotonergic (5HT) dorsal raphe (DR) and the noradrenergic (NAd) locus coeruleus (LC). The recruitment of these nuclei will then lead to an increase in neuromodulatory tone in the cortex, which in turn will shape network topology and information processing mode of local neural populations. In this way, microcolumns in the frontal cortex are able to effectively ‘steer’ the brain network into different information processing modes that befit the current challenges imposed on the system.

Through this lens, the relative evolutionary expansion of the lateral frontal cortex in Homo sapiens [169,170] takes on an interesting new perspective. Through an augmentation of the descending connections from the frontal cortex to brainstem and forebrain arousal nuclei, the human brain may have vastly increased the dimensionality of the control mechanism that it uses to shape the topography of the attractor landscape. This would afford the human brain greater flexibility for finding distant local minima in the complex sociocultural landscape in which we are embedded. By way of analogy, contrast flying in the cockpit of a modern airline with its myriad controls and gauges with the limited controls available to the Wright brothers. This increased fine-grained control over state-space dynamics suggests a refined ability to balance the competing trade-offs of exploration and exploitation across multiple hierarchical levels, which in turn may help to explain the nuanced cognitive architectures known to characterize human behaviour.

8. Conclusion

Over the years, there have been a number of compelling, computational accounts of higher-brain function that attempt to describe the brain according to its emergent functionality. For instance, much of human cognition can be successfully conceptualized ‘as if’ behaviour emerges from an idealized, Bayesian model [171]. Others have argued that serialized models of cognitive function betray intuitions developed from biological sciences, and instead that our higher neural capacities somehow arise from the dynamic synergy of activity across a diversity of autonomous agents [172–174]. Yet, while these approaches have had a great deal of impact, it has remained challenging to link computational accounts to process models that work within the framework imposed by neurobiology. Indeed, the conceptual limitations enforced by biology are often far more rigid than those imposed by computational accounts of the brain. Simply put, the precise interconnections of neurons and glia limit the flexibility of their organization in ways that place constraints on the manner in which higher brain functions emerge from the brain.

A major limitation over our working models of whole brain function has been an insufficient appreciation of how the brain is organized across multiple different spatial and temporal scales. In essence, we have known the constituent parts of the central nervous system for some time, but we have lacked a true appreciation of the manner in which the different parts interconnect and interact. Many research programmes focus on a particular location within the brain (such as the visual cortex, to take a popular example), and make an attempt to understand the function of that specific circuit. The idea is to understand a sub-component of the system in detail, and to then combine the different units together in an effort to understand the system as a whole. By any measure, this approach has been extremely successful. However, we also know that individual circuits within the brain did not evolve on their own, but embedded within the broader context of the other many and varied elements of the central nervous system [175]. For instance, there is now ample evidence that complex behaviours, such as working memory [176,177] and decision making [43,178], involve not just the cerebral cortex, but circuits broadly distributed across the central nervous system, such as the thalamus, basal ganglia, cerebellum, colliculus and brainstem [55,56]. In addition, many of the connections between sub-components are highly precise, conserved and crucial for system-wide function.

Recent large-scale, multi-centre studies, such as the Allen Brain Atlas [179], BlueBrain [180] and the Human Connectome Project [181], have changed the way that modern research in neuroscience is being conducted. These approaches offer access to blueprints of neural architecture, with high-resolution, whole-brain maps of cell types and their connections [182], as well as detailed reconstructions of microscopic cellular complexity [180,183]. Together, these data reveal intricate spatial patterns in the cellular makeup and organization of the brain's microcircuits. Similar patterns are observed at the macroscopic level [122,184], suggesting the potential for a mesoscopic level of organization that mediates the two extremes. Yet to truly appreciate how the emergent dynamics from these systems give rise to cognition [185], we need to refine our understanding of how the different sub-components of the central nervous system work together at the systems level. Advances made in this space will undoubtedly help us to better understand the factors that control neurological disorders, such as dementia, which has been linked to pathological damage within key structures in the AAS [186,187], psychiatric disorders, which are often treated through manipulation of the AAS [188], and disorders such as epilepsy, which have been linked in the past to neural activity within supercritical dynamical regimens [189].

In this manuscript, I have attempted to sketch the beginnings of an approach that begins with key features of neurobiology and then aims to determine what (if any) computational benefits these implementation-level details may imbue upon the large-scale dynamic patterns of neural activity emergent in the billions of cells that comprise the human brain. Specifically, I have argued that the AAS is ideally placed to act as a diverse set of control parameters for mediating critical dynamics in the brain, with the heterogeneity of the arousal system in turn augmenting and diversifying the capacity for criticality across the idiosyncratic circuits that characterize the mature nervous system. Crucially, there are other (relatively) diffusely projecting systems in the brain, such as the matrix thalamic nuclei [20,55,56,153,190], that can provide similar types of control over critical dynamics, albeit at faster timescales (due to their primary glutamatergic signalling) and over spatially extensive domains (due to their more targeted projection patterns). As we gain more insight into the detailed neurobiology of the brain and the manner in which the elements interact, I imagine that this picture will only appreciate in complexity and nuance. For instance, while I have not covered them in this manuscript, there are crucial interactions between the AAS and non-neuronal glia, such as astrocytes, which are of major importance for energy metabolism [191,192]. Important gains will be made if we can continue to discover novel means for testing the implications of these ideas using whole-brain modelling [153,193,194] and neuroimaging [22,122,195–197] approaches that embrace the dynamical systems language in which these hypotheses are addressed [14,196,198].

Data accessibility

There are no data associated with this manuscript.

Conflict of interest declaration

I declare I have no competing interests.

Funding

J.M.S. was supported by the National Health and Medical Research Council (grant no. GNT1193857).