Evolutionary aspects of reservoir computing

Abstract

Reservoir computing (RC) is a powerful computational paradigm that allows high versatility with cheap learning. While other artificial intelligence approaches need exhaustive resources to specify their inner workings, RC is based on a reservoir with highly nonlinear dynamics that does not require a fine tuning of its parts. These dynamics project input signals into high-dimensional spaces, where training linear readouts to extract input features is vastly simplified. Thus, inexpensive learning provides very powerful tools for decision-making, controlling dynamical systems, classification, etc. RC also facilitates solving multiple tasks in parallel, resulting in a high throughput. Existing literature focuses on applications in artificial intelligence and neuroscience. We review this literature from an evolutionary perspective. RC’s versatility makes it a great candidate to solve outstanding problems in biology, which raises relevant questions. Is RC as abundant in nature as its advantages should imply? Has it evolved? Once evolved, can it be easily sustained? Under what circumstances? (In other words, is RC an evolutionarily stable computing paradigm?) To tackle these issues, we introduce a conceptual morphospace that would map computational selective pressures that could select for or against RC and other computing paradigms. This guides a speculative discussion about the questions above and allows us to propose a solid research line that brings together computation and evolution with RC as test model of the proposed hypotheses.

This article is part of the theme issue ‘Liquid brains, solid brains: How distributed cognitive architectures process information’.

1. Introduction

Somewhere between pre-biotic chemistry and the first complex replicators, information assumed a paramount role in our planet’s fate [1–3]. From then onwards, Darwinian evolution explored multiple ways to organize the information flows that shape the biosphere [4–11]. As Hopfield argues, ‘biology looks so different’ because it is ‘physics plus information’ [12]. Central in this view is the ability of living systems to capitalize on available external information and forecast regularities from their environment [13,14], a driving force behind life’s progression towards more complex computing capabilities [15].

We can trace computation in biology from pattern recognition in RNA and DNA [16,17] (figure 1a), through the Boolean logic implemented by interactions in gene regulatory networks (GRNs) [22–24] (figure 1a), to the diverse and versatile circuitry implemented by nervous systems of increasing complexity [25,26] (figure 1c–e). Computer science, often inspired by biology, has reinvented some of these computing paradigms, usually from simplest to most complex, or guided by their saliency in natural systems. It is no surprise that we find some fine-tuned, sequential circuits for motor control (figure 1c) that resemble the wiring of electrical installations. Such pipelined circuitry gets assembled to perform parallel and more coarse-grained operations, e.g. in assemblies of ganglion retinal cells that implement edge detection [27,28] (figure 1d), similar to filters used in image processing [29–31]. Systems at large often present familiar design philosophies or overall architectures, as illustrated by the resemblance between much of our visual cortex (figure 1e) and deep convolutional neural networks for computer vision [19–21,32] (figure 1e).

Figure 1.

Some computing devices in biology. (a) DNA and RNA perform multiple pattern recognition, starting with the simplest matching of codons in ribosomes to synthesize proteins. More complex pattern matching (here, Cas9 matches a longer string of RNA to DNA, drawing from Marius Walter available at https://commons.wikimedia.org/w/index.php?curid=62766587) allows cleavage and insertion of DNA snippets. (b) Boolean logic determines gene expression or silencing. In the example, a receptor gene that expresses green fluorescent protein (GFP) is only active if genes A and B are expressed and their corresponding products dimerize. This implements a Boolean AND gate. (Redrawn from [18].) (c) The knee jerk reflex is implemented by sequential circuits that remind us of the simple wiring of an electrical installation. Sensory neurons (A) gather a sudden input. Information follows the arrows towards ganglia at the entrance of the spinal cord (B) and grey matter within. There, information splits in two: it is partially sent upstream towards the brain stem (not shown) and partially used to excite motor neurons (C), which right away pull from muscles at their end (D). (d) Retinal ganglion cells (red) pool information from rods and cones (grey) through horizontal, bipolar (black and blue) and amacrine cells. This pooling involves excitatory (blue) and inhibitory (black) mediating connections, which make a ganglion cell responsive to light and its absence in the so-called on- and off-centres and surround (red and blue discs, corresponding to dark and light grey cones). Receptive fields of several ganglia are consequently pooled as well, resulting in edge detectors. (e) Specific neurons implementing average and pooling exist throughout the visual cortices [19]. These are the building blocks of deep convolutional neural networks [20]. The overall information flow, architecture and various receptive fields of such artificial networks resembles several processing steps found in mammal visual cortices [21], including edge detection in V1 thanks to retinal ganglia. (This image was drawn with elements from [21] and https://commons.wikimedia.org/w/index.php?curid=1679336.) (Online version in colour.)

Such convergences suggest that chosen computational strategies might be partly dictated by universal pressures. We expect that specific computational tricks are readily available for natural selection to exploit them (e.g. convolving signals with a filter is faster in Fourier space, and the visual system could take advantage of it). Such universalities could constrain network structure in specific ways. We also expect that the substrate chosen for implementing those computations is guided by what is needed and available. This is, at large, one of the topics discussed in this issue. Different authors explore specific properties of computation as implemented, on the one hand, by liquid substrates with moving components such as ants or T cells; and, on the other hand, by solid brains such as cortical or integrated circuits. Rather than this ‘thermodynamic state’ of the hardware substrate, this paper reviews the reservoir computing (RC) framework [33–37], which somehow deals with a ‘solid’ or ‘liquid’ quality of the signals involved, hence rather focusing on the ‘state’ of the software. As with other computing architectures, tricks and paradigms, we expect that the use of RC by nature responds to evolutionary pressures and contingent availability of resources.

RC matters within a broader historical context because it has helped bypass a huge problem in machine learning. The first widely successful artificial intelligence architectures were layered, feed-forward networks (figure 1e and [20] are modern examples). These get a static input (e.g. a picture) whose basic features (intensity of light in each pixel) are read and combined by artificial neurons or units. A neuron’s reaction, or activation, is determined by a collection of weights that measure how much each of the features matters to that unit. These activations are conveyed forward to newer neurons that use their own weights to combine features nonlinearly, and thus extract complex structures (edges, shapes, faces, etc.). Eventually, the whole network settles into a fixed state. A set of output units returns the static result of a computation (e.g. whether Einstein is present in the picture). Training a network consists of adjusting the weights for every neuron such that the system at large implements a desired computation (e.g. automatic face classification). Solving this problem in feed-forward networks with several layers was a challenge for decades until a widespread solution (back-propagation [38]) was adopted. Recurrent neural networks (RNN) brought in more computational power to the field. RNN contemplate feedback from more forward to earlier processing neurons. These networks do not necessarily settle in static output states, allowing them to produce dynamic patterns, e.g. for system control. They are also apt to process spatiotemporal inputs (videos, voice recordings, temporal data series, etc.), finding dynamical patterns often with long-term dependencies. Echoing the early challenges in feed-forward networks, full RNN training (i.e. adjusting every weight optimally for a desired task) presents important problems still not fully tamed [39,40].

RC is an approach that vastly simplifies the training of RNN, thus making more viable the application of this powerful technology. Instead of attempting to adjust every weight in the network, RC considers a fixed reservoir that does not need training (figure 2a), which works as if multiple, parallel spatiotemporal filters were simultaneously applied onto the input signal. This effectively projects nonlinear input features onto a huge-dimensional space. There, separating these features becomes a simple, linear task. Despite the simplicity of this method, RC-trained RNN have been robustly used for a plethora of tasks including data classification [42–44], systems control [43,45–47], time-series prediction [48,49], uncovering grammar and other linguistic and speech features [43,50–53], etc.

Figure 2.

Computational basis of reservoir computing. (a) Reservoir dynamics (black nodes and arrows) are driven by external inputs (grey arrows). For RC, weights within the reservoir are not modified. (b) A set of N^out output nodes can be appended. Each of them computes a linear combination of the reservoir state and solves a single task. In RC training, only the weights of these output units are modified. This allows us to solve several (in this case N^out) different tasks simultaneously without interference. (c) The key to RC computing is that the reservoir filters, and thus projects, low-dimensional input signals into a huge-dimensional space. Here we observe three example trajectories (elicited, e.g. by three different inputs) through a representation of this abstract space. (d) Appropriate reservoir dynamics will make it possible to separate the projection of different inputs using simple linear classifiers (implemented by the readout units from b). If we sort in a matrix the different dynamical states reached after different inputs, the rank of this matrix tells us how linearly independent the dynamics have remained. This gives us an idea of how many different hyperplanes (i.e. readouts and different tasks) we can draw in this abstract space. (e–g) r^S, r^G and r^S − r^G. Plot reproduced from [41]. (e) r^S, which captures this separability property, is plotted for reservoirs made of simulated cortical columns with varying correlation length between neurons (λ) and scale of synaptic strength (ω_scale). We find large r^S (hence large separability) for chaotic dynamics achieved by large λ and ω_scale. (f) When noise and redundant information have been provided, we expect the rank of a similar matrix (r^G, which measures a generalization capability) to score low. The same microcolumns as before present good generalization property when the dynamics are ordered and similar inputs are consistently mapped into a same region of the space of reservoir dynamics. (g) Subtracting r^G from r^S measures a balance between the two desired reservoir properties. Good performance also correlates with a measure of criticality (dot marked ‘2’, where the Lyapunov exponent vanishes). Performance is bad for too ordered (dot ‘1’, negative Lyapunov exponent) or too chaotic dynamics (dot ‘3’, positive Lyapunov exponent). Black curves on top indicate the limits where Lyapunov exponents are fairly close to 0, thus marking the edge of chaos. (Online version in colour.)

Again, we expect that nature has taken advantage of any computing approaches available, including RC, and that important design choices are affected by evolutionary constraints. These will be major topics through the paper: how could RC be exapted by living systems and how might evolutionary forces have shaped its implementation? Section 2 provides a brief introduction to RC and reviews what its operating principles (notably optimal reservoir design) imply for biological systems. Section 3 shows inspiring examples from biology and engineering. Comments on selective forces abound throughout the paper, but §4 wraps up the most important messages. This last part is largely speculative in an attempt to pose relevant research questions and strategies around RC, evolution and computation. We will hypothesize about two important topics: (i) what explicit evolutionary conditions might RC demand and (ii) what evolutionary paths can transform a system into a reservoir.

2. Computational aspects of reservoir computing

(a). Reservoir computing in a nutshell

RC was simultaneously introduced by Jaeger [33] and Maass et al. [34]. Jaeger arrived at Echo State Networks from a machine learning approach, while Maass et al. developed Liquid State Machines with neuroscientifically realistic spiking neurons. The powerful operating principle is the same behind both approaches, later unified under the RC label [35–37].

Consider a RNN consisting of N units, all connected to each other, which receive an external input ${\overline{y}}^{\mathrm{in}}(t)\equiv \{y_i^{\mathrm{in}}(t),i=1,\dots ,N\}$ (with $y_i^{\mathrm{in}}(t)=0$ if the ith unit receives no input). Each unit has an internal state x_i(t) that evolves following:

$$x_i(t+\mathrm{\Delta }t)=\sigma (\sum _{\hspace{0.17em}j=1}^N{\omega }_{ij}{x}_{\hspace{0.17em}j}(t)+y_i^{\mathrm{in}}(t)),$$ 2.1

where $\sigma (\cdot )$ represents some nonlinear function (e.g. an hyperbolic tangent). Variations of this basic theme appear in the literature. For example, continuous dynamics based on differential equations could be used; or inputs could consist of weighted linear combinations of N^feat more fundamental features $\{u_k(t),k=1,\dots ,N^{\mathrm{feat}}\}$ such that $y_i^{\mathrm{in}}(t)=\sum _{k=1}^{N^{\mathrm{feat}}}{\omega }_{ik}^{\mathrm{in}}{u}_k(t)$. This would allow us to trade off importance of external stimuli versus internal dynamics.

Such RNN can be trained so that N^out designated output units produce a desired response ${\overline{y}}^{\mathrm{out}}(t)\equiv \{y_i^{\mathrm{out}}(t),i=1,\dots ,N^{\mathrm{out}}\}$ when ${\overline{y}}^{\mathrm{in}}(t)$ is fed into the network. The training consists in varying the ω_ij until the state of the output units given the input ${\overline{x}}^{\mathrm{out}}(t|{\overline{y}}^{\mathrm{in}}(t))\equiv \{x_i^{\mathrm{out}}(t|{\overline{y}}^{\mathrm{in}}(t)),i=1,\dots ,N^{\mathrm{out}}\}$ matches the desired behaviour ${\overline{y}}^{\mathrm{out}}(t)$. A naive approach is to initialize the ω_ij randomly and modify all weights, e.g. using gradient descent, to minimize some error function $ϵ({\overline{y}}^{\mathrm{out}}(t),{\overline{x}}^{\mathrm{out}}(t|{\overline{y}}^{\mathrm{in}}(t)))$ that measures how much the network activity deviates from the target. Such a training procedure is often useless because the RNN’s recurrent dynamics introduce insurmountable numerical problems [39,40].

The RC approach to RNN training still uses random ω_ij but does not attempt to modify them. We say then that the units described by equation (2.1) constitute a reservoir (figure 2a). Upon it, we append a set of N^out readout units (figure 2b) whose activity ${\overline{x}}^{\mathrm{out}}(t)\equiv \{x_i^{\mathrm{out}}(t),i=1,\dots ,N^{\mathrm{out}}\}$ is just a linear combination of the reservoir activity:

$$x_i^{\mathrm{out}}(t)=\sum _{\hspace{0.17em}j=1}^N{\omega }_{ij}^{\mathrm{out}}{x}_{\hspace{0.17em}j}(t).$$ 2.2

Training proceeds on these output units alone. Only the ${\omega }_{ij}^{\mathrm{out}}$ are modified; these do not feed back into the reservoir, which remains unchanged. The absence of this feedback during learning dissolves the grave numerical problems that affect other RNN. Finding the right ${\omega }_{ij}^{\mathrm{out}}$ for a task becomes as simple as a linear regression between reservoir activity over time given an input $x_{\hspace{0.17em}j}(t|{\overline{y}}^{\mathrm{in}}(t))$ and the desired target ${\overline{y}}^{\mathrm{out}}(t)$ [54]. For a review on RC training (including issues of reservoir design discussed in §2b), see [55] and other works [56,57]. Also, the computational capacity of RC can be boosted if a (trainable) feedback is added from the linear readouts to the reservoir [58–62]. This allows for more agile context-dependent computations and longer-term memory [58]. Controlling the network attractors (thus the learning) that such feedback induces is still not fully understood. These and other variations upon the RC theme are good steps towards full RNN training.

Back to the most naive version of RC, the only task of the reservoir is to have its dynamic internal state perturbed by the input. In doing so, through its nonlinear, convoluted dynamics, the reservoir is picking up the external signal and projecting it into the huge-dimensional space that consists of all possible dynamic configurations of the reservoir (figure 2c). This high-dimensional space hopefully renders relevant features from the input more easily separable. Ideally, such features could be separated with a simple hyperplane that bisects this abstract space. This is precisely what the ${\omega }_{ij}^{\mathrm{out}}$ implement. RC training consists in finding the right hyperplane solving each task given the reservoir.

Of course, very poor dynamics could project inputs into boring reservoir configurations so that prominent features cannot be picked up. The next section discusses issues of reservoir design so that it produces optimal dynamics. However, the most important part of the training falls upon the readouts, whose dynamics do not affect the reservoir. This brings in the two most important advantages of RC: (i) learning is extremely easy, as just discussed and (ii) multiple readouts can be appended to a same reservoir (thus solving different, parallel tasks) without interference.

(b). Good reservoir design: separability, generalization, criticality and chaos

When RC was first introduced, the authors proved in a series of theorems what conditions (e.g. having ‘echo state’, ‘fading memory’, the ‘separation property’, etc.) a reservoir needed to fulfill to implement the paradigm properly [33,34]. While the theorems demanded these requirements in principle, practical implementations turned out more lenient and a wide variety of systems can trivially work as sufficiently good reservoirs. In practice, a wide variety of systems can trivially work as reservoirs. As proofs of concept, among many others, in silico implementations have used realistic theoretical models of neural networks [34,41,63], networks of springs [64,65] or cellular automata [66]; and in hardware, analogue circuits [67,68], a bucket of water [69] and diverse photonic devices [70–72] have been tried. All those theoretical conditions on reservoir dynamics boil down to two desired properties: (i) reservoir dynamics must be able to separate different input features that are meaningful for a variety of tasks while (ii) these same dynamics must be able to generalize to unseen examples, thus projecting reasonably similar inputs into a reasonable neighbourhood within the dynamical space of the reservoir. To fulfill these conditions, we want our reservoirs to behave somehow in between chaotic and simple dynamics, thus resonating with ideas about criticality in complex systems, as discussed below.

In a series of papers [41,63,73], Maass et al. explored two elegant measures that quantify the ability of a reservoir to separate relevant features and to generalize to unseen examples.

Regarding separability, since RC works with simple linear readouts, we can quantify straightforwardly how many different binary features can be extracted by hyperplanes that bisect the space of dynamic configurations of the reservoir (figure 2d). Suppose a collection of inputs $Y^S\equiv \{y_k^S(t),k=1,\dots ,N^S\}$ is fed to the reservoir. At time t₀ + T after input onset (which happened at t₀), the reservoir activity $\overline{x}(t_0+T|y_k^S)\equiv \{x_i(t_0+T|y_k^S),i=1,\dots ,N\}$ is recorded in an N-sized array. The collection $X^S(t_0+T|Y^S)\equiv \{\overline{x}(t_0+T|y_k^S),k=1,\dots ,N^S\}$ consists of N^S such arrays sorted in a matrix. The rank r^S ≡ rank{X^S} conveys an idea of how linearly independent the driven activity of the reservoir is—i.e. of how many binary-classifiable features the reservoir can pick apart. Given Y^S, there are $2^{N^S}$ different possible binary classification problems. If r^S = N^S, we ensure that the dynamics of the reservoir can pick apart the features relevant to all these problems. Even if r^S < N^S, in general, the larger r^S the better a reservoir would be, since it would make more degrees of freedom available to set up the linear readouts.

As for a reservoir’s ability to generalize, we approach the problem similarly, but assuming now that our input contains some redundant or tangential information (e.g. some of its variability comes from noise). Good reservoirs should be able to smooth this out. Assume that a larger collection of inputs $Y^G\equiv \{y_k^G(t),k=1,\dots ,N^G>N^S\}$ is fed to the reservoir. Its dynamics are similarly captured by the matrix $X^G(t_0+T|Y^G)\equiv \{\overline{x}(t_0+T|y_k^G),k=1,\dots ,N^G\}$. If the reservoir is capable of classifying noisy versions of the input under a same class (i.e. of generalizing), we expect now the rank r^G ≡ rank{X^G} to be as small as possible. (More rigorous analysis relates r^G to the VC-dimension of the system. This is, to the volume of input space that can be shattered by the reservoir dynamics—see [41,63,73–75] for further explanations.)

These measures rely on the expectation that the rank of arbitrary activity matrices (such as X^S and X^G) will increase if new meaningful examples are provided, but will stall if examples add spurious or redundant variability. What constitutes meaning and noise in each case is an open question, which still depends on the eventual task. We expect to find more spurious information the more examples we have—hence N^G > N^S. Final values of r^S and r^G will still depend on N^S and N^G—which could stand, e.g. for test and train set sizes.

Despite these caveats, naive applications of r^S and r^G seem to capture good reservoir design that works for a variety of problems. In [41,63,73], realistic models of cortical columns were used as reservoirs. These models mimic the three-dimensional geometric disposition of neurons in the neocortex. Given two neurons located at x₁ and x₂, the likelihood that they are connected decays as exp (−D²(x₁, x₂)/λ²) with the Euclidean distance D(x₁, x₂) between them. The parameter λ introduces an average geometric length of connections, resulting in sparse or dense circuits if λ is, respectively, small or large. This in turn leads to short-lived or more sustained dynamics. Individual neurons were simulated with equations for realistic leaky membranes [76], including proportions of inhibitory neurons within the circuit. Synaptic strengths were drawn randomly to reflect biological data [34], and they all were scaled by a common factor ω_scale. Small ω_scale led to weakly coupled neurons in which activity faded quickly; while large ω_scale implied strong coupling between units, resulting in more active dynamics.

The authors produce morphologically diverse reservoirs by varying λ and ω_scale. Too sparse a connection (owing to low likelihood of connection or weak synapses—respectively low λ and ω_scale, lower-left corner in figure 2e–g) leads to poor dynamics. This results in undesired low separability (low r^S, figure 2e) but brings about the expected large generalization (small r^G, figure 2f) because large classes of noisy input result in converging reservoir dynamics. Meanwhile strong synapses and a very dense network (large λ and ω_scale, upper-right corner in figures 2e–g) easily become chaotic. These have the desired large separability (large r^S, figure 2e) but very low generalization capabilities (large r^G, figure 2f). This is so because of the high sensibility of chaotic systems to initial conditions. Performance in arbitrary tasks is best when r^S and r^G are relatively balanced (figure 2g). Reservoirs in that region of the λ − ω_scale plane are capable of separating relevant behaviours while recognizing redundancy and noise in the input data.

The quantities r^S and r^G capture desirable properties of reservoir dynamics. They are also easy to measure empirically (see §3a) and hence determine if those given dynamics are good for RC. These convenient reservoir properties (separability and generalization) are conflicting traits—changing a circuit to improve one often degrades the performance in the other. The authors in [41,63,73] acknowledge that they do not have a principled way to compare r^S versus r^G—figure 2e just shows the difference between them. We propose that Pareto optimality [77–79] might be a well-grounded framework for this problem. Pareto, or multiobjective, optimization is the most parsimonious way to bring together quantifiable traits that cannot be directly compared (e.g. because they have disparate units and dimensions), and to do so without introducing undesired biases. For the current problem, Pareto optimality would give us the best r^S − r^G trade-off as embodied by the subset of neural circuits that cannot be changed to improve both quantities simultaneously. This offers a guideline to select systems that perform somehow optimally towards both targets. This approach has helped identify salient designs in biological systems evolving under conflicting forces [80–83]. It can also link the computation capabilities of reservoirs to phase transitions [79,84–86] and, more relevant for us, criticality [87].

Indeed, criticality has long been a good candidate as a governing principle of brain connectivity and dynamics [73,88–94]. In statistical mechanics, critical systems are rare configurations of matter poised between order and disorder. Such states present long-range correlations between parts of the system in space and time, with arguably optimal sensitivity to external perturbations. A similar phenomenon was noted in computer science studying cellular automata and random Boolean networks [23,73,95–98]. Such systems usually present ordered and disordered phases. In the former (analogous, e.g. to solid matter in thermodynamics), activity fades away quickly into a featureless attractor. No memory is preserved about the initial state (which acts as computational input to the system) or its internal structure. The later, disordered phase presents chaotic dynamics with large sensitivity to the input and its inner vagaries. Slightly similar initial conditions differ quickly, thus erasing any correlation between potentially related inputs and resulting in trajectories without computational significance. The so-called edge of chaos separates both behaviours, balancing the stability of the ordered phase (thus building relatively lasting steady states) and the versatile dynamics of the disordered phase (which enables the mixing of relevant input features). These properties allow systems at the edge of chaos to optimally combine input parts and compute.

This depiction of critical systems reminds us of the desirable design captured by r^S and r^G. Several authors have used hallmark indicators of criticality to contrive optimal reservoirs with enhanced performance. The measures employed include Lyapunov exponents [41,63,98–102] and Fisher information [103,104] of reservoir dynamics. The former estimates how rapidly slight perturbations get amplified (thus diverge) as the reservoir dynamics unfold. This divergence never happens if the dynamics are too ordered (perturbations get dumped, Lyapunov exponents are negative) and happens too quickly in chaotic regimes (positive exponents). Only at the edge of chaos (Lyapunov exponents tend to zero) can a small perturbation make a lasting yet meaningful difference that does not fade away or explode over time. Similar principles lead to diverging Fisher information as criticality is approached [105].

All these works report notable correlations between such traces of criticality and enhanced reservoir performance—in line with the evidence that a balanced r^S − r^G is an indicator of good reservoir design [41,63,73]. Figure 2g also shows how this r^S − r^G compromise degrades faster in the chaotic than in the more ordered region. Meanwhile, Toyozumi & Abbott [100] use Lyapunov exponents to suggest that reservoir performance should degrade faster in the ordered phase than in the disordered one. In the emerging picture, the advantages of criticality are clear and suggest powerful evolutionary constraints to bring naturally occurring RC-based systems towards the edge of chaos. But a critical state is often difficult to reach and sustain, so often we settle for getting as close as possible. Both [41,63,73,100] predict that it makes a difference from which side we approach the edge of chaos. They disagree on what side is computationally preferred. Empirical studies (see [94] for an up-to-date review) remain inconclusive too. Further research is needed.

3. Inspiring examples in biology and engineering

(a). Reservoirs in the brain

When conceiving liquid state machines, Maass et al. drew inspiration from the cortical microcolumn (figure 3a). Through RC, they proposed a plausible computing strategy that could be the basic operating principle of these circuits [34,111–113]. These neural motifs are roughly cylindrical structures that convey information mostly inwards from and perpendicularly to the neocortex surface. Contiguous columns are saliently distinguished from each other, but sparse lateral connections do exist. An important part of the internal structure, as well as external connections to and from other columns and other parts of the brain (notably the thalamus), appears to be stereotypical [114]. Specific connections and circuit motifs correlate with the different cortical layers within each column (figure 3b). Maass et al. built upon this known average structure when designing realistic reservoirs. Still, columns vary morphologically and functionally throughout the cortex. Most of them can hardly be associated with exclusive functionality, while others can be linked down to specific computations. For example, the receptive field of columns in V1 in cats was early identified by Hubel and Wiesel as responding to visual gratings with specific inclination [115]; the barrel cortex in mice consists of cortical columns that have grown in size through evolution and have specialized in processing the sensory stimuli from individual whiskers [116]. It has been proposed that the cortical microcolumn constitutes the operational unit of the neocortex, and that the advanced cognitive success of mammal brains is a consequence of the exhaustive use of this versatile circuit [106]. Evidence about this remains inconclusive. In any case, discerning the computational basis of cortical circuits is a relevant, open question in neuroscience to which RC can contribute greatly.

Figure 3.

Reservoirs in the brain and the body. (a) Cortical microcolumns have been proposed as the basic operating unit of the neocortex [106]. Liquid state networks [34] were largely introduced in an attempt to clarify the computational basis of these circuits. Figure credit to Oberlaender et al. reconstruction methods in [107]. (b) Schematic of connections between layers within a single cortical microcolumn. Figure reconstructed from [108–110]. Black node contour and black arrows represent, respectively, excitatory neurons and connections, red indicates inhibitory ones. Arrow width is proportional to connection strength as estimated from empirical data. Dashed connections indicate data are less significant. Grey-filled nodes indicate that they receive input from other areas—mostly cortex and thalamus. Green-filled nodes indicate they act as output—mostly to other cortical areas (both output nodes) and thalamus (layer 5 output node). (c) Networks of springs can work very efficiently as reservoirs, as long as they display heterogeneous dynamics as a response to inputs. This opens a huge potential in robotics and elsewhere, since multiple mechanoelastical systems can be modelled as networked spring-mass structures. (Online version in colour.)

In [110], Maass notes the heterogeneity of neuron types in the brain (also within cortical columns). They display great morphological diversity, varying numbers of presynaptic neurons and different physiological constituency that results in diversified time and spacial integration scales. The recursive nature of most neural circuits is also highlighted. This diversity and recursivity poses a challenge to the mostly feed-forward computing paradigms engineered into our computers, which also rely on a relative homogeneity of its components to make hardware modular and reprogramable. What could then be the computational basis of neural circuits? RC comes in to take advantage of these features, and steps forward as a likely computational foundation of the brain. Looking for empirical evidence, three hallmarks have been sought in cortical circuits that would be indicative of RC: (i) records of this morphological heterogeneity, which in turn result in the dynamical diversity proper of a reservoir; (ii) existence of parallel neurons that gather information from a same processing centre (akin to a repository) and project their output into distinct areas that solve different tasks, just as RC readouts use a same reservoir without interference; and (iii) the ability to retrieve relevant (including highly nonlinear) input features by training simple linear classifiers on neural recordings. Note that all these would constitute rather circumstantial evidence for RC since each of these features could be exploited by other computing paradigms too. As of today, this is the best that we can get regarding signs of RC in the brain. We review a series of studies showing such indirect evidence in the next paragraphs.

Neural heterogeneity resulting in desirable, reservoir-like properties has been reported in cortical neurons, the retina and the primary visual cortex, and in networks grown in vitro from dissociated neurons [109,117–121]. The existing diversity of neural components results in very heterogeneous dynamics [122]—which can be exploited by RC-like computation. The emergence of such dynamical heterogeneity in dissociated neurons reveals that this property follows spontaneously from the wiring mechanisms. Indeed, computational studies show how input-driven spike-timing dependent plasticity can generate heterogeneous circuitry that then displays good reservoir behaviour [123].

As argued above, by restricting training to the output units, RC facilitates the use of a same reservoir to solve different tasks by just plugging parallel readouts that do not interfere with each other. This principle seems to be exploited by the brain. In [124], neurons are recorded that project from the mouse barrel cortex to other sensory and motor centres. Both kinds of neurons retrieve the same information, but they respond differently, with task specificity depending on whether further sensory processing is required or whether the motor system needs to be involved. From the RC framework, the barrel cortex would be working as a reservoir whose activity is not enough to determine which neurons will respond and how, suggesting independent wiring for each individual task based on a same, shared input.

All these works casually suggest that advanced neural structures are indeed using some of the RC principles. More compelling evidence comes from the computational analysis of neural populations as they relate to input signals and more precisely, from the ability to retrieve nonlinear input features by using just linear combinations of sparse, recorded neural activity [125–127]. In [125], it is shown how sparse recordings from the primary auditory cortex of ferrets (involving just between 4 and 10 neurons) are enough to retrieve nonlinear features of auditory stimulus in a task that involves tones that increase or decrease randomly by octaves. This information can be extracted by simple linear classifiers trained upon the recorded neural activity. This is possible because the recorded neurons (which act as a reservoir) already implement a sufficient, nonlinear transformation of the input. More complicated methods (e.g. support vector machines) do not show a relevant performance increase. We can parsimoniously assume that evolution would settle for simpler solutions (i.e. linear readouts) if they suffice—unless unknown selective pressures existed.

The term mixed selectivity is used in [126,127] to underscore that neurons from the reservoir in these experiments do not respond to simple (i.e. somehow linear) features of the stimulus. Measures akin to the r^S and r^G introduced above are computed in [126,127] for a set of neural recordings while monkeys perform a series of tasks. It is shown how the classification accuracy grows with the dimensionality of the space (which corresponds to larger r^S) into which the neural recordings project the input stimuli. The underlying reason is, again, that this larger r^S allows more different binary classifiers to be allocated among the data.

The baseline story is that biological neural systems in the neocortex (and potentially other parts of the brain) exhibit all the ingredients needed to implement RC. Most importantly, a lot of meaningful information can be retrieved from real neural activity using simple linear readouts. From an evolutionary point of view, it would feel suboptimal to perform further complicated operations (especially provided that they do not improve performance [125,126]).

(b). The body as a reservoir

In [64], Hauser et al. implemented a reservoir using two-dimensional networks of springs (figure 3c). Inputs are provided as horizontal forces that displace some springs from their resting states. Such perturbations propagate through the network similarly to activity in other reservoirs. Simple linear readouts can be trained to pick up both vertical and horizontal elongations (these would constitute the internal state $\overline{x}(t)$ of the system), and thus perform all kind of computations upon the input signals. It seems trivial that such a reservoir will work as long as the springs present a variety of elastic constants (hence providing the richness of dynamics that RC demands). But a more important conceptual point is made in [64]: the possibility that bodies can function as reservoirs, with springs modelling muscle fibres and other sources of mechanical tension.

A more explicit implementation is explored in [65,128], where the muscles of an octopus arm are simulated and used as a reservoir. Torques at the base of the arm serve as inputs. These forces propagate along the arm, perturbing modules of coupled springs. A simple linear classifier reads the elongation of the various springs. The readouts are trained using standard RC methods until they reproduce a desired function of the output. Alternatively, readout activity is fed back to the arm and trained so that it displays a target motion. Octopuses have a central brain with approximately 50 million neurons versus a distributed nervous system with approximately 300 million neurons [65]. The computational power of nerve cells along the arms is beyond doubt [129,130]. But this approach is telling us something much more important: a lot of the nonlinear calculations needed to process and control an arm’s motion could be provided for free by spurious mechanical forces picked up by simple linear classifiers.

The nonlinearities and unpredictable behaviour of soft tissue could have been a nuisance in robotics. They could have been perceived as untameable systems, very costly to simulate, that a central controller would need to oversee in real time. But a recent trend termed morphological computation [131–133] exploits these nonlinearities, self-organization and in general the ability that soft tissues and compliant elements have shown to carry out complex computations. This framework includes simple behaviours such as passive walkers [134,135], materials optimized to provide sensory feedback [136] or collective self-organization of smaller robots [137]. The approach of the body as a reservoir (demonstrated by the networks of springs just described [64,65,128,138–140] or by tensegrity structures that can crawl controlled by RC-based feedback [141,142]) offers a principled way to develop a sound theory of morphological computation [64,133,138].

Other bodily elements besides physical tensions can work as a reservoir. Recently, Gabalda-Sagarra et al. [143] have shown how the GRNs in a range of cells (from bacteria to humans) present a structure quite suited for RC. Empirically inspired GRNs are simulated and used as reservoirs to solve benchmark problems as well as known optimal topologies. They also show how an evolutionary process could successfully train output readouts stacked on top of those GRNs. These examples with physical bodies and gene cross-regulation highlight a potential abundance of repertoires in nature.

They also highlight the importance of embodied computation [132,144]—the fact that living systems develop their behaviour within a physical reality whose elements (including bodies) can participate in the needed calculations, become passive processors, expand an agent’s memory, etc. In robotics, this opens up huge possibilities [145,146]—e.g. to outsource much of the virtual operations needed to simulate robot bodies. From an evolutionary perspective, the powerful and affordable computations that RC offers through compliant bodies raise a series of questions. For example, with animal motor control in mind: since RC is a valid approach to the problem and it seems to provide so much computational power for free, why is it not more broadly used? Why would, instead, a centralized model and simulation of our body (such as the one harboured by the sensory-motor areas of the cortex) become so prominent instead? What were the evolutionary forces shaping this process, which somehow displaced computation from its embodiment to favour a more virtual approach? It is still possible that, unknown to us, RC actually takes place with the body (or parts of the body) as a reservoir—after all, the paradigm has only been introduced recently (and already; see [133,147,148] for examples falling close enough). However, the most salient features of advanced motor control (e.g. sensory-motor cortices, the central pattern generator that regulates gait, some peripheral circuits implementing reflexes) do not resemble RC much. So the above questions can still teach us something about how RC endures different selective pressures. A possibility that we explore further in the next section is that RC could be an unstable evolutionary solution.

4. Discussion: evolutionary paths to reservoir computing

RC is a very cheap and versatile paradigm. By exploiting a reservoir capable of extracting spatio-temporal, nonlinear features from arbitrary input signals, simple linear classifiers suffice to solve a large collection of tasks including classification, motor control, time-series forecasting, etc. [42–53]. This approach simplifies astonishingly the problem of training RNNs, a job plagued with hard numerical and analytic difficulties [39,40]. Furthermore, as we have seen, reservoir-like systems abound in nature: from nonlinearities in liquids and GRNs [69,143], through mechanoelastic forces in muscles [64,65,128], to the electric dynamics across neural networks [34,41,63], a plethora of systems can be exploited as reservoirs. Reading off relevant, highly nonlinear information from an environment becomes as simple as plugging linear perceptrons into such structures. Adopting the RC viewpoint, it appears that nature presents a trove of meaningful information ready to be exploited and coopted by Darwinian evolution or engineers so that more complex shapes can be built and ever-more intricate computations can be solved.

When looking at RC from an evolutionary perspective these advantages pose a series of questions. Where and how is RC actually employed? Why is this paradigm not as prominent as its power and simplicity would suggest? In biology, why is RC not exploited more often by living organisms (or is it?); in engineering, why is RC only so recently making a show? This section is a speculative exercise around these points. We will suggest a series of factors that, we think, are indispensable for RC to emerge and, more importantly, to persist over evolutionary time. Based on these factors, we propose a key hypothesis: while RC shall emerge easily and reservoirs abound around us, these are not evolutionarily stable designs as systems specialize or scale up. If reservoirs evolve such that signals need to travel longer distances (e.g. over bigger bodies), integrate information from senses with wildly varying time scales, or carry out very specific functions (such that the generalizing properties of the reservoir are not needed anymore), then the original RC paradigm might be abandoned in favour of better options. Then, fine-tuned, dedicated circuits might evolve from the raw material that reservoirs offer. A main goal of this speculative section is to provide testable hypotheses that can be tackled computationally through simulations, thus suggesting open research questions at the interface between computation and evolution.

First of all, we should not dismiss the possibility that RC has been overlooked around us—it might actually be a frequent computing paradigm in living systems. It has only recently been introduced, which suggests that it is not as salient or intuitive as other computing approaches. There was a lot of mutual inspiration between biology and computer science as perceptrons [149], attractor networks [150] or self-organized maps [151] were introduced. Prominent systems in our brain clearly seem to use these and other known paradigms [19,21,32,152–154]. We expect that RC is used as well. We have reviewed some evidence suggesting that it is exploited by several neural circuits [109,117–121,124–127], or by body parts using the morphological computation approach [147,148]. All this evidence, while enticing, is far from, e.g. the strikingly appealing similarity between the structure of the visual cortices and modern, deep convolutional neural networks for computer vision [20,30,32] (figure 1e). Altogether, it seems fair to say that RC in biology is either scarce or elusive, even if we have only recently begun looking at biological systems through this optic.

The two main advantages brought about by RC are: (i) very cheap learning and (ii) a startling capability for parallel processing. Its main drawback compared to other paradigms is the amount of extra activity needed to capture incidental input features that might never be actually used. We can view these aspects of RC as evolutionary pressures defining the axes of a morphospace. Morphospaces are an insightful picture that has been used to relate instances of natural [155–158] and synthetic [159–161] complex systems to each other guided by metrics (sometimes rigorous, other times qualitative) that emerge from mathematical models or empirical data. Here we lean towards the qualitative side, but it should also be possible to quantitatively locate RC and other computational paradigms in the morphospace that follows. That would allow us to compare these different paradigms, or different circuit topologies within each paradigm, against each other under evolutionary pressures.

A first axis is straightforwardly the dynamical cost (C, figure 4a) of the reservoir, since RC demands so much more activity than it eventually uses. This would prevent RC at large organism scales with costly metabolism, but still allows myriad smaller physical systems (such as muscles or tiny bodies) to behave as free-floating reservoirs ready to be exapted.

Figure 4.

Evolutionary paths to and from RC. (a) We propose a morphospace to locate RC-based circuits (this could be extended to other computing paradigms). The axes are determined by the circuit cost (C) and two aspects of an underlying fitness landscape that drives circuit selection: its ruggedness (R) and the average lifetime (τ) that computational tasks contribute to fitness. RC should ensue when both C and τ are low and R is high (green volume). Low τ and high R demand, respectively, cheap learning and multitasking. Possible evolutionary paths of non-RC circuits into the RC area (and vice versa) are depicted. (b) (Red curve in panel a.) Readout units implement simple linear classifiers (indicated by the grey rectangle). Among the many computations implemented by this reservoir, the red units are solely responsible for one specific, valuable computation that turns out to reliably contribute a lot of fitness. The other readouts and their tasks (as well as some components of the reservoir) gradually become spurious and evolutionary dynamics get rid of them. Eliminating some nodes can prompt a reorganization of the original units implementing the valuable computation (solid red units). Such a circuit migrates through the morphospace from the RC region into a position with little ruggedness and stable peaks of the landscape. The cost, too, should have been decreased. (c) (Also red curve in panel a, traversed in the opposite direction.) A simple circuit gradually becomes more complex. New tasks are implemented as components are appended. If true RC is reached, feedback between the outputs (marked with red links) should be lost and readout units should implement linear classifiers (again, grey rectangle). At some point, this should be noted as a certain symmetry breaking between reservoir and readout units. This might leave an empirical trace in biological systems. (d) (Blue trajectories in panel a.) A genetic mutation produces two copies of a very specialized circuit. One of the copies keeps implementing its very specific task (blue ball in the upper left corner of the morphospace, panel a). This represents a lasting, singled-out peak in fitness landscape and the implementation is costly because mistakes would cause a fatal failure. The circuit copy is released from the cost of failure (jump represented by the dashed blue arrow in panel a). It can then wander the morphospace freely (different blue trajectories), probably becoming cheaper as some further functionality is lost. Its very rich dynamics make it a perfect candidate to be hijacked by simple, linear readouts that start using the circuit as a reservoir. (Online version in colour.)

This prompts the next question: given these freely available reservoirs (specifically supported by the spring and GRN examples), when will they be exploited? To answer this, let us focus on the two main advantages of RC mentioned above, starting with the cheap learning. Let us also assume that there exists an underlying fitness landscape that tells us whether a feature (e.g. solving a specific computation) contributes to the success of a living organism. We conceive learning as a process much faster than evolutionary dynamics. Looking at the fitness landscape, we would expect features that offer fitness over long evolutionary periods to be hard-wired, not learned. We would not need a reservoir to capture these, but rather a robust, efficient and dedicated structure fixed by evolution. Features that are rather learned, on the other hand, offer fitness on a time-scale briefer than a lifetime. We are talking about short-lived peaks of the landscape, so voluble or unpredictable that it becomes preferable to keep a learning engine rather than hardwiring a fixed design. Some notion of an average lifetime (τ, figure 4a), during which a feature contributes to fitness, defines a second axis of our morphospace.

Similarly, to exploit the parallel processing abilities of RC, the underlying fitness landscape should be peaked with multiple optima that represent different useful computations. Thus ruggedness (R, figure 4a) defines the last axis of our morphospace. If one or a few tasks contribute much more fitness than others, parts of the reservoir dedicated to them would be reinforced over evolutionary time and unimportant components would fade away, eventually thinning down the reservoir and dismissing the less fit computations (figure 4b). Over very short time-scales, the ruggedness and peak lifetime axes shall become indistinguishable: a quickly shifting single peak resembles a rugged landscape when looked at from afar.

These wildly speculative hypotheses suggest a research program bridging evolution and computation. We can craft artificial fitness landscapes with computing tasks at their peaks. We could then mutate and select reservoirs with costs associated to their dynamics, and with a reward collected as they solve tasks around the shifting landscape. We expect RC to fade away if some of the conditions are removed, for example, if a reservoir grows in physical size so that communicating dynamical states over long distances becomes metabolically costly. Might this have happened in motor control for larger bodies? How would the interplay between the advantages be afforded by RC and a growing morphology? What would be the evolutionary fate of the components of a reservoir? Under what conditions do reservoirs retain their full original architecture with redundant dynamics? When do they thin down to a subset of dedicated parts that capture specific signals and ignore others (figure 4b)? Could some conditions prompt the development of more complex reservoirs? Can we observe a reservoir complexity ratchet—a threshold beyond which RC becomes evolutionarily robust? These are all issues that can be tackled through simulations in a systematic and easy manner. Also, these questions extend beyond RC. We used its properties as guidelines to design morphospace axes that, we think, could optimally distinguish this computing strategy from others. But both the lifetime and ruggedness of tasks in our landscape are independent of the strategy used to tackle each problem. A dynamical cost can also be calculated in other computing devices. Hence, we expect that the morphospace will be useful in locating RC among a larger family of RNN and other paradigms. The eventual picture could contain hybrids as well, thus potentially revealing a continuum of computing options.

Back to RC: up to this point we have assumed that a fully-fledged reservoir exists that, depending on external constraints, might shift to simpler computing paradigms and lose part of its structure (figure 4b). This will be relevant for the kind of reservoirs provided for free by nature, as discussed above. But there are alternative questions at the other end of the spectrum: what would be plausible evolutionary paths for non-RC circuits to progress towards RC? If cortical microcolumns actually implement RC in our brains, we could then wonder how they got there by building upon non-RC elements. Some non-exhaustive possibilities include:

• —Gradually, over evolutionary time, a small, specialized circuit acquires more tasks at the same time that it becomes more complex—e.g. by incorporating more parts and a more convoluted topology (figure 4c). For this to result in RC, partial computations needed for the acquired tasks must be distributed over several circuit components in a way that makes them difficult to disentangle. Otherwise, the system would more likely evolve into smaller, separate and task-specific structures. Importantly, at some point, a kind of symmetry-breaking between the reservoir and the readouts should take place. All the costly, nonlinear calculations should be relegated to the parts that will constitute the reservoir. Readouts or circuit effectors, on the other hand, can become simpler—with linear classifiers eventually sufficing to do the job. Given the way that learning works in RC, which only affects the readouts, this symmetry breaking between parts should also be reflected by a specialization of feedbacks controlling synaptic plasticity—a point that we will discuss again below.
• —An already complex, yet specialized circuit is freed of its main task, thus becoming raw material that can be coopted to solve other computations (figure 4d). This evolutionary path to RC could be explored, e.g. if a complex circuit gets duplicated and one of the copies keeps implementing the original task—so that outlandish explorations are not penalized. This is a mechanism exploited elsewhere in biology, e.g. by duplicated genes with one of the copies exploring the phenotypic neighbourhood of an existing peak in fitness landscape. (Indeed, we could look at such pools of duplicated genes as a reservoir of sorts; and we could wonder whether the structure, variability and frequency of such gene reservoirs could help us quantify aspects of our RC morphospace.) The similarities between central pattern generators (CPGs) in the brain stem and columns in the neocortex have been noted at several levels, including dynamical, histological, biomolecular, pathological and structural [162]. A key difference is the versatility and plasticity of microcolumns when compared with the stereotypical behaviour of CPGs (a versatility that would be very costly for CPGs, since they would fail to implement their main task). RC is explicitly suggested in [162] as a paradigm to frame the differences between CPGs and cortical columns. The hypothesis that microcolumns followed this path to RC from a shared evolutionary origin with CPGs becomes tempting.

These paths to RC, again, work as evolutionary-computational hypotheses that can be easily tested through simulations. It is perhaps also possible to derive some mean-field models of average circuit structure and their computational power and address some limit cases analytically. These are all engaging topics for the future, but a look at the foundation of our speculations already offers hints to answer some of the questions above—notably, why is RC not a more prominent paradigm?

While circuits and complex systems with the ability to work as reservoirs abound in nature, situations that sustain evolutionary pressures with a rugged, shifting landscape (demanding multitasking and adaptability within times much shorter than the lifetime of the reservoir) might not be as common. One possibility is that a peak of the fitness landscape becomes more prominent for an evolving species—e.g. by a process of niche construction. Then, a single task among the several solved by a reservoir can provide enough fitness. Redundant tasks and components can be consequently lost (figure 4b). As suggested in the previous section, against the convenience of bodies as a reservoir, we propose that this might be a reason why nerves at large evolved towards a more sequential and archetypal wiring. As bodies grew bigger, peaks in the motor-control landscape might have become more prominent (e.g. just a few coarse-grained commands seem enough for CPGs to coordinate gait behaviour at large [163,164]). Even though bodies as reservoirs still serve the information needed to solve motor control, the archetypal circuitry is more stable in the long run. The need to integrate visual cues (which can hardly be incorporated into a mechanoelastic reservoir) and long-term movement planning hint at yet other evolutionary pressures against the RC solution. This suggests that smaller or more primitive organisms, if any, shall exploit RC more clearly. Just as RC appears to be a relevant tool to develop a principled theory of morphological computation [64,133,138], it also seems a great addition to think about liquid and solid brains along the lines of this issue, especially as embodied and non-standard computations with small living organisms are explored [165].

Cortical microcolumns, on the other hand, are likely to operate based on RC or to incorporate most RC principles. These are the circuits in charge of the more abstract and complex tasks, such as language or conscious processing. These tasks appear indeed shifting in nature, and often present a wide variety of solutions (e.g. as indicated by the different syntax and grammars that implement language equally well). These features loosely correspond to rugged landscapes whose peaks either shift in time or cannot be anticipated from the long evolutionary perspective—thus fitting nicely in our speculation. Efforts to clarify whether RC is actually exploited in the neocortex and other neural circuits are underway, as reviewed earlier [109,117–121,124–127]. This work focuses mostly on the richness of the dynamics and on the ability of simple linear classifiers to pick up nonlinear input features based on recorded neural activity alone.

We would like to add an alternative empirical approach: as mentioned above, RC implies that training focuses on the linear readouts. This must be reflected at several levels. For instance, the target behaviour must be made available to the readouts (e.g. to implement back propagation or Hebbian learning), but is not needed elsewhere. On the other hand, we have explored a series of computation-theoretical features that reservoirs should preferably display. These include a tendency to criticality and a simultaneous maximization of the separability and generalization properties (as measured by r^S and r^G). All these pose diverging evolutionary targets for reservoir and readout plasticity. This should result in differences in the mechanisms guiding neural wiring—perhaps even at the molecular level. Trying to spot such differences empirically should be within the reach of current technology.

We close with a reflection to link this paper with the general theme of this issue. In §1, we anticipated that most authors would be exploring liquid or solid brains as referring to the thermodynamic state of the physical substrate in which computation happens—i.e. whether computing units are motile (ants, T-cells, etc.) or fixed (neurons, semiconductors, etc.). Instead, this paper rather tackled aspects of the software. RC largely resembles a liquid in the signals involved (sometimes literally so [69]), specially through the abundance of spurious dynamics elicited. Evolutionary costs could limit these generous dynamics so that RC is lost (e.g. figure 4b). This could somehow crystallize the available liquid signals to a handful of stereotypical patterns. This could, in turn, either lower the demands on the hardware (as it requires less active dynamics) and otherwise free resources that could be invested, e.g. in hardware motility. We expect non-trivial interplays between afforded liquidity at the software and hardware levels. It then becomes relevant to determine whether and how the evolutionary pressures discussed above can constrain the liquid or solid nature of brain substrates, not only of its signal repertoire.

Data accessibility

This article has no additional data.

Competing interests

We declare no competing interests.

Funding

This work has been supported by the Botín Foundation, by Banco Santander through its Santander Universities Global Division, a MINECO FIS2015-67616 fellowship, and the Secretaria d’Universitats i Recerca del Departament d’Economia i Coneixement de la Generalitat de Catalunya.