The log-dynamic brain: how skewed distributions affect network operations

Abstract

We often assume that the variables of functional and structural brain parameters — such as synaptic weights, the firing rates of individual neurons, the synchronous discharge of neural populations, the number of synaptic contacts between neurons and the size of dendritic boutons — have a bell-shaped distribution. However, at many physiological and anatomical levels in the brain, the distribution of numerous parameters is in fact strongly skewed with a heavy tail, suggesting that skewed (typically lognormal) distributions are fundamental to structural and functional brain organization. This insight not only has implications for how we should collect and analyse data, it may also help us to understand how the different levels of skewed distributions — from synapses to cognition — are related to each other.

“We use only 10 percent of our brain.” Do we?

Balance, symmetry and normality dominate our thinking and culture, perhaps because of their simplicity. In neuroscience, we seek for and tend to present ‘typical’ or ‘representative’ neurons, dendritic arbors, spines, axon calibres and connectivity in our communications. For example, population activity is typically summarized by the mean firing rate of the contributing neurons or by the average number of neurons participating in various network patterns. Similarly, changes in synaptic strength, spike transfer function and other evoked and induced features are condensed to a single mean to characterize drug- or plasticity-induced effects or changes brought about by learning. Likewise, the power of the electro encephalogram (EEG) or the strength of the blood-oxygen-level-dependent (BOLD) signal is typically reported by a single mean value for comparison across experimental groups. The differences between mean values across groups or experimental conditions are then quantified by statistics based on symmetrical, bell-shaped Gaussian (normal) probability distributions.

However, recent advances, which are summarized in this Review, suggest that such simplification is no longer tenable because the majority of interactions in highly interconnected systems, especially in biological systems, are multiplicative and synergistic rather than additive. Most anatomical and physiological features of the brain are characterized by strongly skewed distributions with heavy tails and asymmetric variations that cannot be compressed into a single arithmetic mean or a typical example. Skewed distributions can take many forms. In the field of complex systems, power laws have attracted attention owing to their scale-free properties^1,2. Another frequently observed, related form — especially in biological systems — is the lognormal distribution (BOX 1). This is perhaps not surprising, as biological mechanisms possess emergent and collective properties as a result of many interactive processes, and multiplication of a large number of variables, each of which is positive, gives rise to lognormal distributions³ (BOX 1).

Perhaps the best-known skewed distribution in neuro science is the Weber–Fechner law, which postulates that the response to a sensory stimulus, such as light or sound, is proportional to the logarithm of the stimulus amplitude^4,5. Distance perception, time perception and reaction time also vary logarithmically with the length of distance and the time interval, respectively. Decision making and short-term memory error accumulation also obey the Weber–Fechner law^6,7. Word usage and sentence lengths in most languages, and even the lengths of e-mail messages, follow a lognormal form, irrespective of the language of the sender^8,9. Mental organization of numbers is also described by a logarithmic scale (log scale)¹⁰.

Despite the extensive evidence for skewed distributions of perceptual and other mental phenomena, very little is known about the brain mechanisms that give rise to such distributions. The goal of this Review is to show that skewed distributions of anatomical and physiological features permeate nearly every level of brain organization. An important consequence of the skewed distribution of brain resources is that a selected minority (say “10 percent”) of neurons can effectively deal with most situations (by which we mean brain states and environmental (place) changes but also context and other manipulations, such as novelty). However, to achieve 100% accuracy 100% of the time requires a very large fraction of brain networks to cooperate.

Macroscopic and mesoscopic activity

Brain oscillations as expressed by scalp EEG at the macroscopic level, and the activity of neuronal networks as reflected by the local field potential (LFP)¹¹ at the mesoscopic level form a system that spans several orders of magnitude of time^12,13 (FIG. 1a). They are coupled in a hierarchical manner, in which the power of a faster oscillation is modulated by the phase of a slow oscillator^12,14–16. As a result, cross-frequency phase–amplitude coupling in the cerebral cortex is characterized by temporal nesting of multiplexed processes on a log scale and the power dynamics observed within and across LFP frequency bands are typically expressed in the decibel (log) scale^12,13,17 (FIG. 1).

Figure 1. Logarithmic distributions at macroscales.

a A power spectrum of subdurally recorded local field potentials from the right temporal lobe in a human patient (mean is shown in blue and confidence interval in red). There is a near-linear decrease of power in the logarithmic scale (log power) with increasing frequency in the logarithmic scale (log frequency), except at lower frequencies. b A local field potential trace from layer 5 of the rat neocortex (1 Hz–3 kHz) is shown at the top and a filtered (140–240 Hz) and rectified derivative of a trace from the hippocampal CA1 pyramidal layer is shown at the bottom, illustrating the emergence of ‘ripples’. One ripple event is shown at an expanded timescale. The peak of a delta wave and the troughs of a sleep spindle are marked by asterisks. c A hippocampal ripple-triggered power spectrogram of neocortical activity centred on hippocampal ripples. Ripple activity is modulated by the sleep spindles (as shown by the power in the 10–18 Hz band), both events are modulated by the slow oscillation (the strong red band at 0–3 Hz), and all three oscillations are biased by the phase of the ultraslow rhythm (approximately 0.1 Hz, indicated by asterisks). Such cross-frequency modulation of rhythms, driven from lower to higher frequencies, as shown in panels b and c, is found throughout the brain and forms the basis of the hierarchical organization of multiple timescales. AU, arbitrary units. Data in part a from REF. 13. Parts b and c are reproduced, with permission, from REF. 151 © (2003) National Academy of Sciences USA.

How are these log scale patterns related to synaptic activity and spiking of neuronal assemblies? Below, we show that the statistical features of population synchrony, firing rate and synaptic strength distributions of cortical neurons may support the ‘log rules’ observed at mesoscopic and macroscopic scales.

Network synchrony

Temporal synchrony of neurons can be obtained by examining population firing patterns through large-scale recording of spiking neurons. As an example, sharp-wave ripples in the hippocampus are self-organized patterns that emerge from the extensive recurrent excitatory collaterals of the CA3 region¹⁸. An assessment of these ripple events during sleep and waking immobility reveals that population synchrony does not vary in a Gaussian manner around a typical mean. Rather, the magnitude of synchrony, measured as the spiking fraction of all recorded neurons during each network burst, follows a lognormal distribution: strongly synchronized (that is, large) events are interspersed irregularly among many medium- and small-sized events¹⁹ (FIG. 2a,b). The skewed nature of the distribution implies that there is no characteristic size of synchronous event that can faithfully describe the process. Such skewed distribution is not constrained to the super-synchronous sharp-wave ripples but prevails during hippocampal theta oscillations as well¹⁹ (FIG. 2b) and is also reflected by the distribution of the magnitude of the correlation coefficient between neuron pairs²⁰ (FIG. 2c), suggesting the existence of a general rule. The mechanism underlying the skewed and heavy-tail distribution of population synchrony is unknown, but clues may be obtained by examining the firing patterns of the contributing individual neurons.

Figure 2. Skewed distribution of the magnitude of population synchrony.

a Wide band and ripple-band (140–230 Hz) filtered local field potential (trace) and spiking activity (coloured dots) of 75 simultaneously recorded CA1 pyramidal cells. The shaded areas indicate two ripple events during which a relatively low (0.09) and high (0.16) fraction of neurons fire synchronously. b The probability distribution of the synchrony of CA1 pyramidal cells’ firing. The x axis shows the proportion of cells that fired during sharp-wave ripples (SPW-Rs) or in 100 ms time windows during theta periods in behavioural tasks (RUN) and slow-wave sleep (SWS), including ripple events. c Probability distribution of the magnitude of the pairwise correlation coefficient among pairs of neurons during theta periods in a behavioural task (RUN) and during SWS, recorded at 50 ms time resolution²⁰; only significantly correlated (P < 0.05) cell pairs are shown. Insets show cross-correlograms of poorly synchronized (left) and highly synchronous (right) neuron pairs during SWS. The graphs in panels b and c show that the probability distributions can be characterized as lognormal. Panel a is reproduced, with permission, from REF. 19 © (2013) Elsevier. Data in parts b and c from REF. 20.

Firing rates and bursts

Recent quantifications of firing patterns of cortical pyramidal neurons in the intact brain have shown that the mean spontaneous and evoked firing rates of individual neurons span at least four orders of magnitude and that the distribution of both stimulus-evoked and spontaneous activity in cortical neurons obeys a long-tailed, typically lognormal, pattern^19,21–26. Such a distribution creates a rate spectrum with a wide dynamic range, spanning from vast numbers of very slow-firing neurons to a small fraction of fast-firing ‘champion’ cells. Although cataloguing the rate distributions in multiple neuronal types in various cortical layers and regions will require further data collection, the existing data clearly indicate a substantial deviation from a Gaussian rate distribution. The firing rates of principal cells of every region in the cerebral cortex investigated to date have a lognormal or lognormal-like rate distribution (FIG. 3).

Figure 3. Lognormal distribution of firing rates in the cortex.

a Siliocon probe recordings in the rat brain showing the firing-rate distribution of principal cells in the hippocampus (CA1, CA3 and dentate gyrus (DG)) and the entorhinal cortex (EC; specifically, in layers 2, 3 and 5) during slow-wave sleep (SWS; left panel) and exploration (RUN; right panel). b Whole-cell patch recordings showing the firing-rate distribution of neurons in the auditory cortex of awake rats. The two distributions show all cells and a subset, from which seven neurons with narrow spikes (that is, putative fast-firing interneurons (FFIs)) are excluded. c Silicon probe recordings in the rat brain showing the firing-rate distribution of superficial (layers 2/3) and layer 5 neurons in the prefrontal cortex of an exploring rat. Neurons with a peak firing rate in the maze (<1 Hz) were excluded from the analysis. d | Recordings (using sharp metal electrodes) showing firing-rate distribution of neurons from lateral intraparietal and parietal reach region areas of the macaque cortex during a baseline condition and during performance of a reaching task. Data from principal cells and interneurons are not separated. e Utah array recordings showing firing-rate distribution of neurons in the human middle temporal gyrus during sleep. Principal cells and putative interneurons are plotted separately. f Firing-rate distribution of neurons in multiple cortical areas of human patients recorded with metal electrodes during various tasks. g Distribution of spatial features of dorsal hippocampal CA1 pyramidal neurons of rats exploring an open field, showing a lognormal distribution of spike information for place field representation. Spatial information is an information-theoretical measurement of place field sharpness¹⁵². The proportion of cells plotted against the spatial information content per spike (bits per spike) are shown in dark blue; the proportion of cells plotted against the spatial information rate (bits per second) are shown in light blue; and the proportion of place fields plotted against place field size (cm²) are shown in red. AU, arbitrary units. Data in part a from REF. 19. Data in part b from REF. 24. Data in part c from REF. 30. Data in part d courtesy of A. Berardino, New York University, USA, and B. Pesaran, New York University, USA. Part e is reproduced from REF. 26. Data in part f courtesy of M. Kahana, University of Pennsylvania, USA. Data in part g from REF. 51.

Such lognormal distributions of firing rates have been found using various recording methods. However, it may be argued that such distributions are an artefact due to methodological issues; indeed, each recording technique has some caveat. For example, patch-clamping of neurons may affect the firing patterns of neurons^24,25. Cell-attached methods are less invasive, but here the identity of the recorded cell often remains unknown and one might argue that the skewed distribution simply reflects the recording of large numbers of slow-firing pyramidal cells and a smaller number of faster-discharging interneurons. Furthermore, long-term recordings are technically difficult to obtain, and this may result in biased sampling of more-active neurons. Extracellular recording of spikes with sharp metal electrodes typically offers reliable single-neuron isolation (FIG. 3d–f); however, as in cell-attached recordings, sampling of single neurons is often biased towards selecting fast-firing cells because neurons with low firing rates are often not detected during short recording sessions. Moreover, in many cases, only evoked firing patterns in very short time windows are examined^24,25. Chronic recordings with tetrodes and silicon probes (FIG. 3a,c) can reduce such bias towards cells with a high firing rate, as the electrodes are moved infrequently and large numbers of neurons can be monitored from hours to days^27,28. In addition, one can separate the recorded population into excitatory and inhibitory neuron types in vivo through physio logical characterization^26,29–32 or by using optogenetic methods^33,34. Caveats of the extracellular probe methods include the lack of objective quantification of spike contamination and omission, the difficulty in isolating exceedingly slow-firing neurons and the lack of objective segregation of different neuron types³⁵.

The left tail of the firing-rate distribution can especially vary across studies because neurons with low firing rates are often not detected during short recording sessions¹⁹ or because an arbitrary cut-off rate eliminates slow-firing cells. The differences in the right tail of the distribution across studies and species are probably the result of in adequate segregation of principal cells and interneurons. In a recent study²⁴, the firing-rate distribution of all recorded neurons in rats (FIG. 3b) was very similar to the distribution observed in studies performed in monkeys and humans (FIG. 3d,f) in which no attempt was made to separate the two populations. However, when the fast-firing ‘thin’ spikes (corresponding to putative interneurons) were removed, the distribution became virtually identical with that in studies in which putative interneurons and principal cells had been separated, including in the rat entorhinal cortex, hippocampus and prefrontal cortex, and in the human cortex (compare the blue distribution curve in FIG. 3b with those in FIG. 3a,c,e).

Despite the above technical caveats, lognormal distributions seem to be a pervasive phenomenon across multiple neural scales. Importantly, long-term firing rates in logarithm scale (from here on referred to as ‘log firing rates’) also correlate with the log firing rates of induced responses, such as the peak and average firing rates within the place field of hippocampal pyramidal cells¹⁹. In addition, the overall firing rates of neurons correlate with their bursting probability so that the burst propensity also shows a lognormal-like distribution, with a handful of super-bursters and the majority of neurons bursting only occasionally¹⁹.

There are at least two possible explanations for the presence of lognormal distributions of firing rates in neuronal populations³⁶. The first possibility is that the neuronal population is relatively homogenous, but in different situations different subsets of neurons are activated by relevant inputs from the environment, body or other upstream networks. In this scenario, the cause of the skewed distribution is best explained by input selectivity. Another possibility is that the same subset of neurons tends to be highly active under different conditions and in different situations, perhaps because of the strongly skewed distribution of excitability of individual neurons and/or their pre-existing connectivity. In such a relatively ‘fixed’ firing-rate scenario, the discharge patterns of any neuron can change momentarily in response to afferent activation but the longer-term firing rates remain relatively stable. Large-scale recordings of neuron spikes in multiple situations can differentiate between these two possibilities, as discussed below.

Preserved log rates across environments

A comparison of the firing rates of the same individual principal cells recorded across different behaviours — including active exploration, quiet wakefulness, non-rapid eye movement (REM) sleep and REM sleep — shows that the firing rates remain robustly correlated in all brain states^19,21,37,38 (FIG. 4a–c). Because the firing rates of neurons are often used to discriminate between situations³⁹, it is important to examine how firing rates are correlated in different environments and conditions.

Figure 4. Firing rates of principal neurons are preserved across brain states and environments.

a The firing rates of the same CA1 pyramidal neurons in the rat during slow-wave sleep (SWS) and rapid eye movement (REM) sleep are correlated. b The firing rates of the same CA1 pyramidal neurons during exploration (RUN) and REM sleep are also correlated. c The firing rates of the same neurons in different mazes are correlated. d Firing-rate correlation of neurons during RUN in a novel maze and SWS in the home cage (before the maze session). e Firing-rate correlation in the mouse barrel cortex during background and object localization by whiskers. f Correlations of background and evoked firing rates in the lateral intraparietal (LIP) and parietal reach region (PRR) cortical areas of the macaque monkey. Part a is reproduced, with permission, from REF. 21 © (2001) National Academy of Sciences USA. Data in panels b–d from REF. 19. Data in part e from REF. 25. Data in part f courtesy of A. Berardino, New York University, USA, and B. Pesaran, New York University, USA.

Hippocampal place cells are known to remap when an animal is tested in different situations⁴⁰. Remapping of place cells can take two forms. When an animal is placed in a different maze that is in the same location in the same room as the original maze, this causes a change in the firing rate of place cells but not in their spatial location of firing (their ‘place fields’). Alternatively, when an animal explores the same maze in different rooms (in other words, in different environmental contexts)⁴¹, the firing fields of the neurons may appear entirely different (‘global remapping’)³⁹. During global remapping, there seems to be a relatively ‘orthogonal’ or random-sample relationship between population firing-rate vectors in the different contexts, such that a minority of neurons discharge at comparable rates in both contexts, whereas the remaining neurons are relatively silent in one context and active in another^39,42. It has been suggested that moderate rate changes enable the generation of representations of unique episodes, whereas larger changes serve to distinguish between different contexts³⁹. Despite these apparent changes in the population discharge patterns, the log firing rates of individual neurons in the different situations remain correlated; in other words, the same subset of neurons tends to be active in different environmental contexts and mazes when firing rates are plotted on a log scale. The preserved correlation is most striking in the fast-firing minority of neurons (FIG. 4c), perhaps because slow-firing neurons, which potentially have weaker synapses, are more sensitive to plastic changes^43,44. Even when an animal is moved from a familiar maze to a maze it has never previously visited⁴⁵, the log firing rates of individual hippocampal CA1 neurons remain significantly correlated between the two situations¹⁹. Furthermore, the log firing rates of individual neurons during slow-wave sleep in the home cage and those during subsequent exploration in a novel environment show a reliable positive correlation (FIG. 4d), demonstrating that log firing rates remain relatively stable across situations that involve changes in both brain state and environmental input. Firing rates and patterns in several cortical regions, including visual, somatosensory and auditory cortices, and lateral intraparietal and parietal reach regions in all species examined are also remarkably correlated between ‘background’ and stimulus conditions (FIG. 4e,f). This is also the case under anaesthesia^46–48.

The preserved rate correlations across states, testing environments and even novel experiences, suggest that the firing-rate distribution in cortical populations is relatively ‘fixed’ when longer timescales are considered and are more strongly controlled by factors intrinsic to internal network dynamics than by transient external factors. The relatively stable rate dynamics of cortical neurons raise several questions: how do such stable networks encode information? Do neurons at the opposite ends of the distribution have different intrinsic biophysical properties and wiring patterns (discussed in the next section)? And can slow-firing neurons become fast ones, or vice versa? Afferent activity can of course have a large effect on the firing rates and timing of neurons in any situation without fundamentally altering the stability of the network; a tenfold change in evoked firing of individual neurons is usually considered to be a large effect but it is only a single unit on the log scale on which firing rates of neuronal populations span several orders of magnitude.

Both electrophysiological and immediate-early gene expression studies indicate that when rats are tested in two different environments, a significantly smaller fraction of CA3 pyramidal cells are active in both environments relative to CA1 neurons^49–51. It seems that neurons that are active in both situations belong to the fast-firing minority rather than being randomly drawn from the population. However, even when exposure to a novel environment induces dramatic firing-rate changes in individual neurons, the population rate distribution is only modestly affected once the system relaxes back to its ‘default’ or ‘offline’ state, as has been shown in experiments comparing firing rates during pre- and post-experience sleep episodes^21,52. Creating persistent large shifts in the firing-rate structure of the network may require repeated and prolonged exposure to the same environment or situation^53–55.

Slow-firing and fast-firing neurons

Numerous indices of hippocampal place cell firing — such as within-session stability, place field coherence and information rate — are positively correlated with firing rates¹⁹, suggesting that fast-firing principal neurons are more stable and convey information more reliably than slow-firing neurons. Long-term potentiation and depression can induce new place cells and make place cells disappear⁴³, and this effect is most strongly expressed in slow-firing place cells. Imaging place cells over several weeks and months has demonstrated that the place fields of ~20% of the population remain stable over that period, whereas the remaining ~80% may lose their place features⁵⁶. Although firing rates were not reported in these long-term experiments, one can assume that the ‘place-stable’ neurons belong mostly to the fast-firing minority as there is a reliable relationship between rate, bursting and calcium influx⁵⁷.

Fast-firing- and slow-firing-rate neurons — alone or in alliance with their partners — may have different effects on their targets even if their spiking qualitatively ‘encodes’ similar stimulus features. Even if we assume that the axon arbors and synaptic contact probabilities with downstream neurons are similar for fast- firing and slow-firing neurons, the fast-firing minority (which also bursts more) may have a larger effect than the slow-firing majority because faster-firing cells can contribute more spikes in a given time window than can slow-firing neurons. Moreover, convergence of a large group of slow-firing cells will affect distributed spines on numerous dendrites, whereas the fast-firing minority of neurons can repeatedly activate the same synapses. Both in vitro and in vivo experiments have shown that bursts of spikes affecting the same synapse within <10 ms are often more effective than larger numbers of inputs on different dendrites of the same neuron^30,58.

Expression of the immediate-early gene Fos is often used to assess the firing-rate history of neurons⁵⁹. Indeed, targeted in vivo recordings from transgenic mice that express green fluorescent protein (GFP) under control of the Fos promoter (Fos-GFP mice) showed that FOS–GFP-expressing neurons fired faster than non-labelled neurons in layer 2/3 of the primary sensory cortex. Importantly, the highly active FOS–GFP-expressing neurons were more likely to be connected to each other than were non-FOS–GFP-expressing neurons⁶⁰, suggesting that the fast-firing minority may form a special, highly active subnetwork or ‘hub’. Similar subnetworks may exist in the hippocampus. Pyramidal cells that reside in the deep part of the pyramidal CA1 cell layer fire at higher rates and burst more frequently than their peers in the superficial sublayer⁶¹. Similarly, early- and late-born dentate granule cells possess different excitabilities and different dendritic and axonal morphologies⁶².

It is important to note that although stronger connections among small groups of neurons may be associated with faster firing rates, model networks of either random or complex synaptic connectivity can give rise to a skewed, long-tailed distribution of firing rates^63–68. Thus, the network structures that give rise to a lognormal firing rate distribution in the intact brain remain to be discovered.

Distribution of synaptic strengths

Several in vitro studies have found that synaptic strengths between pairs of connected cortical cells have a skewed, typically lognormal, distribution^68–75 (FIG. 5). Among the frequent small-amplitude excitatory postsynaptic potentials (EPSPs), a single-spike discharge of a presynaptic neuron occasionally evokes a giant depolarization and even a spike in the postsynaptic target in both the hippocampus and neocortex^68,74 (FIG. 5a–d). The trial-to-trial variability of giant unitary EPSPs is smaller than that of the small-amplitude unitary EPSPs, and the distribution of the EPSP size-dependent variability follows a skewed form^68,74. Thus, stronger synapses are not only stronger on average but are also more reliably strong from event to event. Recent computational studies suggest that a skewed distribution of synaptic strengths between participating neurons may be a critical factor in sustaining lognormal firing rate distribution^68,76. In these studies, a few large-amplitude and stable synaptic connections seemed to dominate the entire model network because removal of the weak synapses had little impact on population firing, and when all synapses were equalized to a ‘mean’ strength, a much larger number of neurons had to be stimulated by external inputs to maintain stable activity⁶⁸. Thus, the bulk of the information flow may be routed primarily through a web of strongly connected neurons. However, the variability of weaker synapses may also be an important factor, possibly by boosting transmission at times of high coincidence of larger EPSPs.

Figure 5. Lognormal distribution of synaptic weights and spike transfer probability.

a Six intracellularly recorded and biocytin-filled neurons in the barrel cortex of the mouse in vitro. b A colour-coded synaptic connectivity diagram of the six neurons in part a. c Corresponding colour-coded membrane potential traces showing presynaptic action potentials (top trace in each set) and unitary excitatory postsynaptic potentials (uEPSPs; bottom trace (or traces) in each set) in the synaptically connected neurons. d The distribution of synaptic weights in the mouse barrel cortex and rat visual cortex. e The graph in the lower panel shows the distribution of spike transmission probability values between CA1 pyramidal cells and putative interneurons during exploration (RUN) and slow-wave sleep (SWS). The top panel shows superimposed filtered waveforms (800 Hz–5 kHz) of a pyramidal cell (pyr) and an interneuron (int) triggered by spiking of the pyramidal cell. f The correlation of spike transmission probability during RUN and spike transmission probability during SWS. Each dot represents a single cell pair. g Examples of spines imaged along the dendrites of one neuron. The numbers indicate the size of the spine in arbitrary units (AU). The graph in the lower panel shows the lognormal distribution of sizes of all spines. h The top panel shows spines in the auditory cortex of the mouse in which calcium signals (indicating activity) were recorded (red). The middle panel shows activity indicating a spontaneous up state. The bottom panel shows a comparison of the response rate of spine calcium responses during sound stimulation (evoked (E)) and spontaneous (S) up states. Note that individual spines showed similar calcium response rates during spontaneous and evoked activities. Parts a–c are reproduced, with permission, from REF. 74 © (2009) Elsevier. Data in part d from REF. 72 (blue line) and REF. 74 (red line). Parts e–f are reproduced, with permission, from REF. 19 © (2013) Elsevier. Part g is reproduced, with permission, from REF. 91 © (2011) Society for Neuroscience. Part h is reproduced, with permission, from REF. 78 © (2013) Elsevier.

Even if the above in vitro and modelling findings are compelling, it is impossible to speculate about the mechanisms and trajectories of information flow without knowing the distribution of synaptic weights in behaving animals. An indirect estimate of synaptic strength can be obtained by measuring the short-timescale cross-correlation between pairs of neurons^29,77. The efficacy of spike transmission is highly variable across pairs of cortical neurons and has a skewed, lognormal-like distribution — the majority of pairs are weakly coupled and a small minority are strongly coupled¹⁹ (FIG. 5e,f).

Thus, both in vitro and in vivo findings demonstrate that the distribution of synaptic weights in the cortex is lognormal. Interestingly, synaptic weight strength of individual synapses remains correlated across brain states¹⁹ and, potentially, across situations and environments (FIG. 5e,f). In addition, imaging experiments in anaesthetized mice showed that the frequency of calcium transients in spines of auditory cortical neurons in response to sound stimulation is correlated with the frequency of calcium transients during the up states of slow oscillations⁷⁸ (FIG. 5h). Similarly, correlated spatiotemporal calcium patterns were observed between spontaneous and thalamic stimulation-evoked up states in vitro⁷⁹. These experimental results are probably related to the mesoscopic-level observations demonstrating that the variability of stimulus-evoked responses largely reflects the variability of the ongoing, spontaneous local voltage distributions in otherwise stable cortical networks^80,81.

A candidate mechanism for the generation of unequal synaptic strengths is spike-timing-dependent plasticity (STDP). This process can shift an initially homogeneous weight distribution to unique patterns. STDP can segregate synaptic weights into a bimodal distribution of weak and strong synapses, but the network can destabilize without additional constraints^82,83. However, when STDP works in a synaptic weight-dependent manner (called log-STDP⁸³), it can produce a skewed weight distribution and sustain stable network dynamics⁸³. In such a log-STDP regime, long-term depression exhibits linear weight dependence for weak synapses but much less so for strong synapses. The long-term depression of strong synapses enables synaptic weights to grow, generating a long tail in the synaptic weight distribution⁸³. In turn, the skewed distribution of rates contributes to the maintenance of the skewed distribution of synaptic strength. Whether the postulated log-STDP rule is operative in the brain remains to be demonstrated.

Substrates of skewed synaptic efficacy

The physiological efficacy of synaptic transmission may depend on the anatomical features of the synapse. Indeed, the strength of excitatory transmission, as measured by the probability and amplitude of excitatory postsynaptic currents (EPSCs), correlates with the number of post-synaptic AMPA receptors⁸⁴. In turn, the density of AMPA receptors is a function of the number of synapses and release sites established between neurons⁸⁴. Considering the lognormal distribution of synaptic weights, it is therefore not surprising that the number of synapses formed between pyramidal cells and between pyramidal cells and interneurons is strongly skewed^85–87. Because the strength of the synapse between excitatory neurons is positively correlated with the spine size^88,89, spine size can be used to quantify the distribution of synaptic strengths in imaging studies. Such studies have demonstrated that, in adult animals, most spines persist for extended periods of time and that only a small fraction of spines appear and disappear^44,90. Importantly, the distribution of spine sizes in individual layer 5 cortical neurons is well described by a lognormal rule⁹¹ (FIG. 5g). Spine size can change over time, and the magnitude of change is proportional to its initial size, which is indicative of a log-STDP rule and multiplicative dynamics⁹¹ (BOX 1). Such long-tail spine size distributions have also been observed in hippocampal CA1 pyramidal cells⁹². It is noteworthy that these structural–functional features may not be exclusive properties of cortical circuits: motor unit sizes in the spinal cord also show a lognormal pattern⁹³.

The skewed distribution of synaptic weights alone may not explain the lognormal firing patterns of neurons⁹¹; variations in their inhibitory inputs may also contribute⁶⁰. Particular interneuron types are known to innervate particular somatic, axonal and dendritic domains of pyramidal cells in the cortex^94,95. However, this does not necessarily mean that each pyramidal neuron is innervated by all interneuron types. Thus, it is possible that specific constellations of interneuron innervation of single principal cells may also contribute to variations in the firing rate of pyramidal cells.

Intrinsic biophysical properties and channel density variations across neurons^96,97 are additional factors that could contribute to the skewed distribution of firing rates. The observed lognormal-like distribution of spike burst rates supports the idea that membrane channel properties have a role, considering that the likelihood of burst occurrence depends largely on intrinsic properties of neurons^98,99.

Although we have identified several factors that may (partly) underlie the skewed distribution of firing rates in cortical neurons, exploring these and other biophysical, synaptic and circuit mechanisms that can affect firing rates remains an important goal for future research.

Corticocortical connections

So far, we have reviewed physiological observations of the lognormal-like distribution of synaptic weights, firing rates, population synchrony and the power–frequency relationship of the LFP. Such dynamics correlate well (at the single-neuron level) with the similarly skewed distribution of release sites and spine sizes. Are there also circuit-level features that could support lognormal network dynamics? Although it has long been known that anatomical connections among cortical areas are clustered, the quantitative exploration of the exact form of such connectivity has only recently become possible. Recent retrograde labelling studies in macaques have shown that the range of connection strengths between brain areas spans five orders of magnitude and that, in a given cortical area, a minority of strong afferents are mixed with large numbers of weak inputs, so that the connectivity profile of the area is best described by a log-normal distribution¹⁰⁰. These findings in monkeys also apply to the mouse brain^101,102.

The above findings suggest that in addition to a few strong connections, virtually any cortical seed site sends inputs to, and receives inputs from, numerous distant cortical areas through weak links, which thus provide conduits for global communication. Furthermore, the findings support the view that lognormal rules of connectivity can combine the advantages of local computation and global (top-down) control, synchronization, wiring economy, information storage and communication speed^2,68,103,104. These new anatomical observations, therefore, are highly relevant to the often-discussed multimodal and multisensory neurons in various parts of the brain and to the computation of multisensory integration¹⁰⁵. The macroscopic connectivity scheme can also provide clues about the relationship between the skewed firing-rate distributions and local–global communication: although fast-firing neurons with strong connections in a given circuit can respond effectively in most situations, the slow-firing neurons of the global connectome could be equally important. The global or ‘contextual’ activity provided by slow-firing neurons may constrain the effectiveness of ‘specific’ communication.

Distribution of axon diameters

Communication among neurons depends not only on firing rates and synaptic efficacy but also on the speed at which information can be communicated to targets. Speed of communication depends in large part on the speed of axonal conduction, which correlates with the diameter and myelination of the axons^106,107. An increased axon calibre enables signals to travel longer distances within approximately the same time window and ensures that signals from various sources can be delivered to the same target at approximately the same time. Axon calibres in the brain vary by several orders of magnitude, and their distribution is strongly skewed^108,109 (FIG. 6). In humans, the great majority of callosal axons have diameters <0.8 μm, but the thickest 0.1% of axons can have diameters as large as 10 μm¹¹⁰. The large-calibre axons are typically found in the cross-hemisphere paths of sensory modalities, whereas in frontal cortical areas (where the speed of communication is slower), the small-calibre axons dominate. Importantly, axons emanating from the same neuron but targeting different brain regions can vary substantially in calibre, suggesting that a single neuron can communicate with its partners at different speeds¹¹¹.

Figure 6. Skewed distribution of axon calibres.

a Representative micrographs of callosal tissue in different species. Note the various sizes of axon diameters. b The distribution of diameters of unmyelinated and myelinated axons in the corpus callosum of the macaque monkey and mouse. Note the lognormal-like distribution of axon diameters. Data from REF. 109.

In conclusion, it seems that the lognormal rule applies to various functions, from local circuits to systems of the brain connectome. Larger-diameter axons can provide faster communication, have larger terminal arbors¹¹² and typically originate from neurons with high firing rates¹¹³. It is thus tempting to speculate that the skewed distribution of cortical connections forms the morphological substrate for the log rule of macroscopic brain dynamics, as reflected by the LFP and EEG.

Log rules at multiple levels

The experimental data reviewed above demonstrate that synaptic strength, neuronal firing rate, population synchrony and information content of spikes show a skewed distribution, and this is associated with a lognormal distribution of spine size and connections at the large-scale level of the brain connectome. What makes these distributions potentially important is not so much the fact that skewed distributions are found at multiple levels but that these skewed distributions might be linked across the multiple levels. Neuronal firing patterns in numerous cortical areas in various species show a common theme: a small minority of strong synapses and fast-firing neurons provide a substantial portion of the activity in all brain states and situations, and information may spread along the trajectories of these strongly connected and highly active neurons. However, the rest of the brain is not silent; the remaining part of neuronal activity in any given time window is supplied by very large numbers of weak synapses and slow-firing cells. The divisions ‘strong’ and ‘weak’, and ‘fast’ and ‘slow’ are not absolute, as a continuous — rather than bimodal — distribution characterizes both brain activity and its supporting anatomical structure.

A lognormal distribution may be the result of a combination of genetically defined ‘scaffolding’ and experience-dependent modification of synaptic strengths, biophysical properties and anatomical wiring; and the quantitative features of the lognormal distributions of several measures of neural activity and connectivity may differ across cortical areas, layers and neuron types. The skewed, log-normal-like distributions point to a relatively hard-wired backbone in neural-network activity, as illustrated by the reliable correlations between various physiological log scale parameters across situations. Importantly, despite a degree of rigidity in terms of network statistics and structure, members of the slow-firing majority of neurons, which have more-plastic properties, can provide sufficient flexibility for dynamic neural coding.

A theoretical implication of the skewed distribution pattern is that the distinction between signal and noise inevitably becomes fuzzy. Synapses and neurons at the left and right tails of the distributions may seem to serve different functions, but there is no clear boundary between their postulated functions. A practical implication of the absence of a clear boundary is that comparing morphological and dynamic variables (for example, firing rate) with Gaussian distribution-based statistics is not justifiable. Indeed, the wide range of spike rates in cortical neurons creates challenges for calcium imaging-based assessment of neuronal activity. For example, methods with limited sensitivity may ignore large fractions of very slow-firing neurons and favour mainly bursting cells. At the other extreme, fast-firing and bursting neurons may saturate the indicator and obscure any modulation of the firing rate²⁵. Proper assessment of spiking activity by other indirect methods, such as measuring the expression levels of the immediate-early gene Fos⁵⁹, is also challenging because the level of Fos expression in the slow-spiking majority of cells may be too close to the detection ‘noise’ level, and therefore the method may be strongly biased towards revealing mainly the highly active minority of neurons¹¹⁴. The problem of proper sampling of the weak majority (that is, the left tail of the lognormal distribution) is a general issue and may contribute to the appearance of power laws in many models^2,103,115 (BOX 1). Thus, instead of simply reporting the mean changes of the parameters, a quantitative evaluation of the entire distribution is desirable in future experiments. Only through the quantification of the entire distribution can one gain accurate insights into experience-dependent changes in structure and function.

Fast decision–slow precision continuum

What is the relationship between the two ends of a continuous distribution that can span several orders of magnitude? A highly active and bursting minority of neurons, potentially supported by strong synapses in a preferentially connected subnetwork, may be responsible for nearly half of the spikes in any time window. This end of the distribution may serve to provide fast, economic and highly efficient but perhaps less-precise communication^12,104. Although this is currently a speculative idea, such high-firing and strongly connected neurons with faster-conducting axons may support a ‘rich-club’ organization in the brain, in which a minority of neurons, with stronger connectivity, have access to more information than the majority of neurons and share such information among themselves¹¹⁶. However, equally important is the observation that the remaining portion of spiking activity is supplied by a very large fraction of slow-discharging neurons, which are perhaps connected with weak synapses in a more loosely formed giant network. As there is no definable boundary between the two ends of the distribution, the boundary between local and global connectivity^117,118 is vague. In networks with such local–global interactions, optimal performance is characterized by a trade-off between computation speed and accuracy.

The importance of a highly active minority of neurons has often been discussed in the framework of ‘cell assemblies’ — a hypothetical set of neurons carrying the actual information^119,120. A common observation is that the activity of just a dozen or so strongly firing neurons is often more informative about an animal’s behaviour than that of the remaining majority of neurons that is being recorded simultaneously^{28,30,121–123}. Similarly, in brain–machine interface applications, just 10–20% of task-related neurons can predict as much as 60–80% accuracy of limb position or gripping force, and adding further information from the remaining majority of task-related neurons improves the prediction only by a modest 10–15%^124,125. Thus, it seems that a ‘backbone’ consisting of the fast-firing minority of neurons in a strongly connected network provides a ‘best guess’ for matching behaviour to environmental situations. However, perfect performance may require the participation of the more flexible majority of neurons, which can be fine-tuned with experience. In other words, ‘10%’ of neurons and synapses in the brain can do a decent job in most situations, but accurate and reliable performance requires the involvement and cooperation of a very large brain circuit. Similar mechanisms may be at work in the hippocampus, where the spatial information content (bits per spike or bits per second) carried by single neurons shows a skewed distribution (FIG. 3g).

Although the right end of the synaptic strength distribution may produce disproportionally strong excitation through strong synapses, the additional activity of the weak and flexible majority of synapses may be crucial for enhancing the fidelity of spike transmission through strong, sparse connections^68,126. The large quantity of weak synapses may also represent a reservoir for memory storage. In addition, the slow-firing majority of neurons may serve to provide excitation to interneurons and thereby secure a sufficient level of inhibition to counter the effects of the fast-firing minority of excitatory neurons¹¹⁵ and maintain self-organized, sustained activity in the cerebral cortex. Networks with skewed statistical properties have been shown both to provide a broad dynamic range and to offer stability, resilience and tolerance to component failures^68,75,127. Another major advantage of logarithmically linear computation in both electronic circuits and biochemical systems is its energy efficiency^128,129, and this efficacy probably applies to brain circuits as well.

Many features of cortical circuits, such as recurrent excitatory loops coupled with feedback and feedforward inhibition, redundancy and degeneracy, create substrates for wide-dynamic range, log-linear computation. Log scaling can compress a large input range into a smaller, more manageable output range, and after logarithm transformation, multiplication and division become simpler addition and subtraction problems, respectively (BOX 1). Positive feedback, which is needed to widen the log-linear dynamic range of operations, can be implemented by recurrent excitation in cortical circuits, and such circuits have been shown to obey the Weber– Fechner law¹²⁹. Another fundamental feature of cortical circuits is feedforward inhibition¹³⁰, and numerous experiments suggest that division is accomplished by dendritic subtraction of excitatory and inhibitory inputs that are encoded logarithmically, followed by exponentiation through active membrane conductances^{67,76,131–136}. These properties enable single neurons and neuronal circuits to perform Boolean logic operations and detect fold changes rather than absolute shifts, similarly to energy-efficient analogue circuits^128,134.

The preconfigured, log-dynamic brain

Rich brain dynamics can emerge from the wiring and biophysical properties of neurons during development even without any sensory input or experience; we refer to this as the preconfigured brain. It is experience that attaches ‘meaning’ to the firing patterns in a preconfigured brain^{12,19,22,45,48,80,119,137–139}. Even a non-experienced brain can have numerous high-probability states (often referred to as attractors¹⁴⁰) that can give rise to distinct constellations of neuronal firing (so-called ‘trajectories’ (REFS 121,141)), each of which is available to represent a unique event or situation¹¹⁹. Each situation — novel or familiar — activates the appropriate highest-probability state, and this matching process reflects the ‘best guess’ of the brain. From this perspective, there is no ‘unknown’, as every situation is able to activate one of the pre-existing trajectories on the basis of generalization from past experience. Such preconfigured connectivity and firing patterns, which are initially driven by the strongly connected and fast-firing minority of neurons, may explain why the spatiotemporal structures of spontaneous and evoked activity are often similar^{46–48,78–81} and why spontaneous activity can be predictive of activity evoked by an experience even when the experience is novel (a phenomenon known as preplay)^19,45. However, the large reservoir of the weakly connected majority of neurons lends itself to modifications, and as a result of these modifications, initially similar trajectories may become distinct from each other. From this perspective, learning and experience are largely a fast-matching and slower refinement process. For example, in a newborn, all human faces may give rise to the same activation pattern in the relevant parts of the brain and it may take months before he or she can differentiate among the faces and voices of family members while still grouping together all other people as ‘strangers’ (REF. 142). Similarly, place cells are present during the first exploratory trip of a weanling rodent from the nest^143,144, and in the adult a place map is ‘formed’ almost instantaneously in any novel environment. Nonetheless, it may take several repeated exposures to the same situation to fully develop a reliable representation of a novel object or a unique and specific map of a new environment^53,145. Likewise, a ‘representation’ of body parts in the somatosensory cortex is already present at birth, but it takes weeks to fully learn the relationships between the rapidly growing body parts and their distances from each other¹⁴⁶. Experiments in developing ferrets show that the similarity between spontaneous and evoked activity increases progressively with age and is specific to responses evoked by natural scenes, suggesting that internal models result from the adaptation of brain states to the statistics of the surrounding world¹³⁹. In summary, the brain can generate a large number of states that can determine the spatiotemporal evolution of neuronal activity. Experience becomes embedded into these pre-existing states (attractors) and modifies them with repetition so that the attractor states acquire ‘meaning’ by representing external inputs and events^81,145.

An important implication of the preconfigured brain with statistically preserved firing rates and synaptic weights across brain states and testing conditions is the ‘replay’ of waking neuronal sequences during sleep⁵². Neuronal replay refers to the higher-than-chance similarity of sequential spiking patterns during sleep and waking. Under the framework of preconfigured cortical circuits, it is inevitable that some correlation between firing rates and temporal co-activation of neuronal pairs persists. The preconfigured dynamics can account for the correlations between firing rate and co-activation of neurons during sleep and during post-sleep exploration of a novel environment^19,45. A re-activation of existing patterns in a different brain state thus cannot in and of itself be taken as evidence for learning or synaptic plasticity, as experience only slightly modifies the pre-wired configurations; it is precisely these minor deviations (rather than exact re-activation of existing patterns) that reflect the learning, and these deviations need to be identified by statistical methods. Considering that the strongly firing minority of neurons is the main driver of the correlations, recognizing the relatively small but important contribution of experience-dependent changes of slow-firing neurons and their altered relationships is a daunting task^22,48,147.

In summary, although ‘10%’ of highly active neurons can provide remarkable solutions in most encounters, optimal performance requires the commitment of a very large fraction of the brain through a very low level of engagement. A full understanding of the origin and biological utility of the multiscale, log-dynamic organization of brain function requires systematic work at the experimental, modelling and theoretical fronts. Perhaps the biggest challenge is to reveal how the lognormal form of connectivity and firing dynamics relates to the log scaling of the hierarchical nesting of multiplexed brain rhythms¹⁴⁸ and to the Weber–Fechner law of our senses.

Box 1 Normal and lognormal distributions.

Normal (Gaussian) distribution is a continuous probability distribution and is non-zero over the entire real line. It is characterized by a bell-shaped curve that is symmetrical around the mean, and it can be quantified by two parameters — the mean and the SD. Lognormal distribution is a probability distribution of a random variable whose logarithm is normally distributed; in other words, X is lognormally distributed if log(X) has a normal distribution. (See the figure, part a, which shows the firing rate of hippocampal CA1 pyramidal neurons during slow-wave sleep (SWS) on the x axis, plotted in the linear scale (left) and the logarithmic scale (log scale; right); the median (with the first and third quartiles), the arithmetic mean (± SD) and the geometric mean (± SD) in the log scale are shown above the histograms.) This is true regardless of the base of the logarithm function. If log_a(X) is normally distributed, log_b(X) is also normally distributed (a,b ≠ 1, a > 0 and b > 0). If X is distributed lognormally with location and scale parameters μ and σ, then log(X) is distributed normally with mean μ and SD σ. Lognormal distribution can be characterized by the geometric mean and geometric SD; namely, the geometric mean is equal to e^μ and the geometric SD is equal to e^σ, as lognormal distribution is unimodal on the log scale. The geometric mean of a lognormal random variable is equal to its median.

A random variable that is the sum of many independent variables has an approximate normal distribution (as stated in the central limit theorem). Likewise, a random variable that is the multiplicative product of many independent variables has an approximate lognormal distribution (this is justified by the central limit theorem in the log domain).

The sum of many independent normal variables is itself a normal random variable, whereas products and quotients of lognormal random variables are themselves lognormal random variables; that is, they are self-similar. Multiplication or division of two positive variables can be calculated by adding or subtracting the logarithms and taking the antilog of that sum or difference, respectively^3,149. For the ability of neurons to perform multiplicative interactions (antilog or exponentiation operations), see REFS 131,133,134.

In the normal distribution, the probability of a value several SDs above the mean is practically zero. Therefore in practice, the extreme ends of the distributions are often truncated, deeming those values as ‘outliers’, to make the distribution look more Gaussian. When values greater than three times the SD above the mean are present in the data, plotting the distribution on a log scale is advisable. Although log-transformed plots appear as simple as the normal distribution, their intuitive understanding is difficult. Studies analysing the distribution of population bursts in vitro have suggested that they reflect a power law (‘avalanche’)¹¹⁵. It is notable that the right tail of lognormal distribution often follows a power law (see the figure, part b, which shows the same data as part a in a log–log plot of firing-rate probability density and firing rate; black dots indicate the bins outside the mean + 1 SD in log transforms, the regression line (red) is based on the black dots), although a strict requirement of power law requires log–log distributions over several orders of magnitude¹. To be fair, the distinction between lognormal and power law distributions is not trivial. Power law and lognormal distributions connect naturally, and similar generative models can lead to one or the other distribution, depending on minor variations¹⁵⁰. If the variance at the left tail is large, or ‘noisy’, because of limited data samples, a lognormal distribution may appear as a line in a log–log plot. In addition, if the data are thresholded at an arbitrary value — as is often the case in practice — the bounded minimum may yield a power law instead of a lognormal distribution. However, a lognormal distribution has a finite mean and variance, in contrast to power low distribution of scale-free systems. Distinguishing lognormal and power law distributions is important for understanding their biological origin.

Glossary

Spike transfer

The fraction of spikes in the postsynaptic neuron relative to the number of spikes in the presynaptic, driver neuron (or neurons). It is an indirect way of measuring synaptic strength.

Power laws

A term to describe a relationship between two variables, where one varies as a power of the other. The indication of a power law is a distribution of values on a straight line on a double log plot. The right tail of the lognormal distribution often follows a power law distribution.

Scale-free properties

Properties that characterize networks with a degree distribution that follows a power law, characterized by a heavy tail (‘Pareto tail’).

Cross-frequency phase–amplitude coupling

This is perhaps the most prominent ‘law’ underlying the hierarchy of the system of brain oscillators. The phase of the slower oscillation modulates the power of the faster rhythm (or rhythms).

Decibel

A logarithmic unit used to express the ratio between two values of a variable. It is often used to describe gain or attenuation: for example, the ratio of input and output.

Sharp-wave ripples

Patterns of activity in the hippocampus, consisting of a sharp wave reflecting the strong depolarization of the apical dendrites of pyramidal cells and a short-lived, fast oscillation (‘ripple’) as a result of the interaction between bursting pyramidal cells and perisomatic interneurons.

Theta oscillations

A prominent 4–10 Hz collective rhythm of the hippocampus. Other brain regions can also generate oscillations in this band.

Remap

This term refers to the observation that place cell representations can abruptly change.

Immediate-early gene

A gene that is rapidly and transiently activated in response to relevant stimuli.

Fos

A prominent immediate-early gene in the brain; it is often used as an indicator of neuronal activity.

Synaptic weights

A measure of the strength of the synapse, which determines the amplitude of the postsynaptic neuron’s response to a presynaptic spike.

Up states

The active phases of the slow oscillation. Intracellularly, an up state corresponds to latching the membrane potential to a more depolarized, near-spiking threshold value.

Spike-timing-dependent plasticity, (STDP)

A plasticity-inducing paradigm in which the relative timing of spikes between the pre- and postsynaptic neurons determines the direction and magnitude of the change in synaptic strength.

Wiring economy

The idea that connections among multiple brain regions and neurons are arranged to reduce energy cost and volume demand.

Feedforward inhibition

Excitatory afferents to the various domains of pyramidal cells are matched by parallel sets of inhibitory interneurons to filter, attenuate or route excitation. It can perform division operation.

Redundancy

This term refers to the observation that multiple replicas of input representations exist.

Degeneracy

In biology, this term refers to the idea that different solutions evolved to carry out the same functions.

Preconfigured brain

This term refers to an idea that connections and dynamics in the brain are largely self-generated and that experience matches the pre-existing patterns to the external world, thereby giving rise to ‘meaning’.

Attractors

Hypothetical effectors that move elements of a system to more stable states over time. Inhibition-based brain rhythms often show properties of an attractor.

Internal models

A term derived from the hypothesis that the perceived world is not simply a reflection of the objective reality but depends on previous experience and brain state. In this hypothesis, internal models reflect the source of our individual views.