Computational Principles of Supervised Learning in the Cerebellum

Abstract

Supervised learning plays a key role in the operation of many biological and artificial neural networks. Analysis of the computations underlying supervised learning is facilitated by the relatively simple and uniform architecture of the cerebellum, a brain area that supports numerous motor, sensory, and cognitive functions. We highlight recent discoveries indicating that the cerebellum implements supervised learning using the following organizational principles: (a) extensive preprocessing of input representations (i.e., feature engineering), (b) massively recurrent circuit architecture, (c) linear input-output computations, (d) sophisticated instructive signals that can be regulated and are predictive, (e) adaptive mechanisms of plasticity with multiple timescales, and (f) task-specific hardware specializations. The principles emerging from studies of the cerebellum have striking parallels with those in other brain areas and in artificial neural networks, as well as some notable differences, which can inform future research on supervised learning and inspire next-generation machine-based algorithms.

1. INTRODUCTION

Many of the computations that machines, animals, and humans must perform to effectively interact with the world can be implemented through supervised learning. The central feature of supervised learning is that feedback about a system’s performance is used to adjust internal parameters and thereby improve future performance. This is done through an iterative process whereby the system’s response to a given input is evaluated against a desired outcome, and deviations from the desired outcome (i.e., errors) are then used to adjust adaptive elements within the system. Engineers have used supervised learning to train adaptive filters for applications such as system identification and noise cancellation (Jenkins et al. 1996); mathematicians and computer scientists have demonstrated its power for solving a variety of classification and regression problems, from image and speech recognition to forecasting stock prices or energy consumption (Alpaydin 2014, Hagan et al. 2014). Supervised learning is also thought to play a key role in the development and maintenance of many brain functions (Doya 2000, Knudsen 1994), although the organizational principles governing the underlying neural computations have been elusive (Marblestone et al. 2016). In this review, we summarize recent discoveries about how supervised learning is implemented in the cerebellum, a neural system that is tightly interconnected with multiple brain areas and plays a key role in the adaptive control of motor, sensory, and cognitive functions (Sokolov et al. 2017).

The essential architecture for supervised learning comprises three core elements (Figure 1) in both artificial and biological neural networks: (a) The first element is an adaptive processor. To learn how to generate an appropriate response from each of its inputs, the network that performs the input-output transformation must have internal parameters that can be adaptively tuned. This tuning is achieved by adjusting connection weights in artificial neural networks (Alpaydin 2014, Hagan et al. 2014, Jenkins et al. 1996) or by adjusting the strength of synapses or other properties of neurons in the brain (i.e., neural plasticity) (Titley et al. 2017). (b) The second element is input preprocessing. Before inputs are fed into the adaptive processor of a supervised learning system, it is often necessary to transform the raw input. In artificial neural networks, the process of finding a suitable representation of the input data is called feature engineering and is a critical step that often determines whether the algorithm will succeed or fail (Bengio et al. 2013, LeCun et al. 2015). (c) Instructive signals compose the third element. The main distinction between supervised and unsupervised learning is that the supervised network receives feedback about its performance and uses these instructive signals to adjust the internal parameters of the adaptive processor. The instructive signals are computed by comparing the response of the network with the desired response; in other words, they signal error.

Figure 1.

Core architecture of supervised learning networks. The essential computational frameworks for implementing supervised learning are shown for (a) an artificial neural network and (b) the cerebellar network. A preprocessing stage (cyan) transforms the input signals to create representations that are a suitable substrate for supervised learning. Those representations are then sent to an adaptive processor (green) that generates responses. Errors are computed by comparing the actual response of the network with the desired response (orange) and are used as instructive signals to adjust the internal parameters (e.g., synaptic weights) of the adaptive processor until it learns to generate the desired response.

It has long been recognized that the cerebellum possesses these three core elements of supervised learning systems (Albus 1971, Marr 1969) (Figure 1). Input signals arrive in the cerebellum via mossy fibers and are preprocessed by local circuits in the granule cell layer of the cerebellar cortex. These signals are then sent to Purkinje cells, which are the sole output of the cerebellar cortex. The Purkinje cells are a key part of the adaptive processor that generates responses and distributes them, via the cerebellar nuclei, to numerous brain areas, including sensory, decision-making, and motor centers (Sokolov et al. 2017). The inferior olive nuclei in the brainstem compute instructive signals and convey them to the cerebellum via the climbing fibers. Importantly, Purkinje cells form closed loops with neurons in the cerebellar nuclei and the specific set of cells in the inferior olive that generate a particular instructive signal (Apps & Hawkes 2009, Chaumont et al. 2013) (Figure 1b).

In the following sections, we review recent research that has uncovered key elaborations of the basic functional architecture for supervised learning in the cerebellum. The general principles emerging from this work can inform studies of supervised learning in other areas of the brain and inspire next-generation machine-based algorithms.

2. PREPROCESSING OF INPUTS

The input layer of the cerebellum performs feature engineering by converting the mossy fiber signals it receives into granule cell representations that are suitable for supervised learning. More than half of the neurons in the mammalian brain are cerebellar granule cells; thus, the processing of mossy fiber inputs by the granule cells represents a dramatic expansion of coding space, which could support feature engineering functions such as pattern separation and the generation of temporal basis sets.

2.1. Pattern Separation

Early theories suggested that the highly divergent architecture of the granule cell layer is ideally suited to perform pattern separation (Albus 1971, Marr 1969): Similar input representations, encoded by overlapping groups of mossy fibers, would be transformed into linearly separable representations by activation of sparse and nonoverlapping groups of granule cells. Support for this idea comes from the demonstration that different mossy fiber streams converge onto individual granule cells (Chabrol et al. 2015, Huang et al. 2013, Ishikawa et al. 2015) (multimodal granule cell in Figure 2; see also Sawtell 2010), a process that would favor pattern separation through combinatorial sampling of the mossy fiber signals (Albus 1971, Marr 1969). However, in some parts of the cerebellar cortex, granule cells are driven by mossy fibers carrying similar information (Bengtsson & Jorntell 2009) (unimodal granule cell in Figure 2). Also, representations may not be as sparse as originally envisioned, because widespread activation of many granule cells can occur during natural behaviors and in response to sensory stimulation (Giovannucci et al. 2017, Jorntell & Ekerot 2006, Knogler et al. 2017, Ozden et al. 2012, van Beugen et al. 2013, Wagner et al. 2017). Lower levels of sparseness could improve the capacity of the network to generalize what it learns (Spanne & Jorntell 2015) without compromising pattern separation; the number of mossy fiber inputs per granule cell is low, which should support decorrelation of spatially overlapping mossy fiber patterns and the generation of high-dimensional representations in granule cells (Billings et al. 2014, Cayco-Gajic et al. 2017, Litwin-Kumar et al. 2017).

Figure 2.

Circuit diagram illustrating how supervised learning is implemented in the cerebellar network. The numbers 2–7 correspond to the computational principles described in Sections 2–7 of the main text and indicate the location(s) where each principle is implemented in the cerebellar network. Feedforward paths (black lines) and feedback paths (red lines) are indicated. Each mossy fiber pattern represents an information stream of a specific modality. An example of a multimodal granule cell (filled with four patterns) and a unimodal granule cell (filled with a one-dot pattern) are shown. Abbreviations: CN, cerebellar nucleus; GrC, granule cell; GoC, Golgi cell; IO, inferior olive; MLI, molecular layer interneuron; PkC, Purkinje cell.

2.2. Dynamic Representations

Learning what response to make to a given input is useful but often insufficient. In many cases, learning when to make the correct response is just as important, because the desired response (Figure 1) often varies as a function of time. Artificial neural networks often solve this problem with a preprocessing step that transforms time-invariant inputs into dynamic representations that enable the adaptive system to approximate the desired response over its entire time course (Jenkins et al. 1996). Artificial network models based on the architecture of the cerebellar granule cell layer can generate temporal basis sets (i.e., time-varying patterns of granule cell activations in response to temporally impoverished mossy fiber inputs) (for reviews, see D’Angelo & De Zeeuw 2009, Medina & Mauk 2000). Such temporal basis sets in the granule cells of cerebellum-like circuits of electric fish have been described (Kennedy et al. 2014); however, technical challenges have impeded empirical studies of temporal patterning in the mammalian cerebellum. Moreover, recent work suggests that molecular mechanisms intrinsic to the Purkinje cells may learn to time a response (Johansson et al. 2014, 2015). Thus, the extent to which the dynamics of the granule cell layer contribute to the cerebellum’s ability to support precise learned timing remains unresolved.

2.3. Plasticity of Representations

Artificial deep networks revolutionized image recognition and related classification problems by allowing the network to learn the representations of inputs suitable for supervised learning, thereby automating the feature engineering function and making it highly adaptive (Bengio et al. 2006, Coates et al. 2011). Likewise, synaptic plasticity in the granule cell layer of the cerebellar cortex (D’Angelo & De Zeeuw 2009, Sgritta et al. 2017) could enable it to implement adaptive feature learning during preprocessing of mossy fiber inputs.

3. MASSIVELY RECURRENT ARCHITECTURE

Although the cerebellar circuit was historically characterized as a traditional, primarily feedforward perceptron network (Albus 1971, Marr 1969) (Figure 1b), recent work has uncovered ubiquitous, recurrent connections (Figure 2). In machine learning, recurrent neural networks, which incorporate feedback paths between the different layers of the network, have been particularly effective for tasks that require processing of sequences, such as natural language processing and time-series prediction (Medsker & Jain 1999). The highly recurrent architecture in the cerebellum may support similar functions, including language processing and the coordination of sequential movements (Hikosaka et al. 1999, Leggio & Molinari 2015, Penhune & Steele 2012, Sokolov et al. 2017).

3.1. Local Feedback Within the Cerebellar Cortex

It has long been recognized that negative feedback loops between excitatory granule cells and inhibitory Golgi cells (GoCs) could play a role in pattern separation and the generation of dynamic representations in the granule cell layer (Albus 1971, Billings et al. 2014, D’Angelo & De Zeeuw 2009, Litwin-Kumar et al. 2017, Marr 1969, Medina & Mauk 2000) (see Section 2). Moreover, feedback is now known to influence computation at multiple additional levels of the local circuit in the cerebellar cortex (Figure 2). First, there are reciprocal connections between groups of excitatory unipolar brush cells (UBCs) (Dino et al. 2000), between groups of inhibitory GoCs (Hull & Regehr 2012), and between groups of inhibitory molecular layer interneurons (MLIs) (Rieubland et al. 2014). This type of recurrent architecture can act as a temporal integrator to prolong the response of the system to transient inputs and build a short-term memory (Maex & Gutkin 2017, van Dorp & De Zeeuw 2015). Second, a positive feedback loop between the axons of granule cells and MLIs (Astorga et al. 2015) forms a recurrent local circuit that is predicted to sharpen the spatial contrast of the input signals delivered to Purkinje cells. Third, Purkinje cells are reciprocally connected to each other in the adult brain (Witter et al. 2016) and also send inhibitory feedback to granule cells (Guo et al. 2016) and MLIs (Witter et al. 2016).

3.2. Long-Range Feedback from Cerebellar Nuclei to the Cerebellar Cortex

We have known for over 40 years that the final output of the cerebellum is sent from the cerebellar nuclei back to the cerebellar cortex (for a review, see Houck & Person 2014), but only recently have some of the details critical for understanding the function of this nucleocortical feedback connection come to light. In many (but not all) regions of the cerebellum (Trott et al. 1998a,b), nucleocortical feedback is part of a closed loop among granule cells (input layer), Purkinje cells (sole output of the cerebellar cortex), and neurons in the cerebellar nuclei (output to other brain areas). This recurrent path can be configured as either a positive or a negative feedback loop, with excitatory neurons in the cerebellar nuclei targeting both excitatory and inhibitory cells in the granule cell layer (Houck & Person 2015) and inhibitory neurons in the cerebellar nuclei targeting inhibitory cells selectively (Ankri et al. 2015). These nucleocortical feedback loops could play several computational roles in supervised learning by (a) enabling the cerebellum to work as a dynamical system to learn sequences of responses (Brandi et al. 2013) and to perform state estimation and time-series prediction using a forward model configuration (Miall et al. 1993, Wolpert et al. 1998), (b) stabilizing learning even in cases where only indirect information about the correct response is available for computing the error signal (Porrill & Dean 2007, Porrill et al. 2004), (c) providing a mechanism for regulating input signals and amplifying or dampening the gain of the system based on the current response (Gao et al. 2016, Lisberger & Sejnowski 1992), and (d) implementing an architecture for reservoir computing capable of generating dynamically rich input signals in the granule cell layer of the cerebellar cortex (Rossert et al. 2015).

4. LINEAR COMPUTING

The goal of supervised learning is to find a mapping function: a transformation of the input signals that generates the desired correct response (Figure 1). Although some machine learning applications have accomplished this with networks of linear units (i.e., units that produce an output that is a weighted sum of their inputs) (Hagan et al. 2014, Haykin 2013), units with nonlinear activation functions are more frequently used. In highly nonlinear networks, it is often extremely difficult to find any obvious relationships between the output of the individual units and any of the features of the input stream or the final response of the system. In contrast, the firing rate of many cerebellar neurons is a linear function of task-related parameters such as the direction of a moving stimulus or the kinematic properties of an action being performed (for reviews, see Ebner et al. 2011, Medina 2011). This linear coding of task-related parameters has been found at all levels of the cerebellar circuit, including mossy fibers (Laurens et al. 2013, Lisberger & Fuchs 1978), granule cells (Arenz et al. 2009, Chadderton et al. 2014, Chen et al. 2017, Powell et al. 2015), MLIs (Chen et al. 2017, Gaffield & Christie 2017, Jelitai et al. 2016, ten Brinke et al. 2015), Purkinje cells (Chen et al. 2016; Dugue et al. 2017; Herzfeld et al. 2015; Hong et al. 2016; Medina 2011; Medina & Lisberger 2007, 2009; Shidara et al. 1993; Sun et al. 2017), and the cerebellar nuclei (Ebner et al. 2011, Heiney et al. 2014, Kleine et al. 2003, ten Brinke et al. 2017). Recent studies have begun to shed light on the mechanisms underlying these linear transformations in the cerebellum (Park et al. 2012; Person & Raman 2011; Steuber & Jaeger 2013; Turecek et al. 2016, 2017; Walter & Khodakhah 2006, 2009). Although cerebellar neurons and synapses have nonlinear intrinsic properties (Hoebeek et al. 2010, Ishikawa et al. 2015, Jahnsen 1986, Loewenstein et al. 2005, Person & Raman 2011), these properties are disengaged or compensated in ways that keep the cerebellar network operating in a linear regime (Alvina et al. 2008, Rossert et al. 2014, Schonewille et al. 2006, Turecek et al. 2017, van Vreeswijk & Sompolinsky 1996). This linear processing would not preclude supervised learning of nonlinear mapping functions by the network if the granule cell layer provided an appropriate basis set (see Section 2.2), because it is possible to approximate any nonlinear function with a linear combination of basis functions (Dean & Porrill 2011, Pouget & Snyder 2000, Spanne & Jorntell 2013).

5. SMART INSTRUCTIVE SIGNALS

Errors play a critical role during supervised learning; they are the instructive signals that guide appropriate adjustments in the adaptive processor (Figure 1). In many machine learning applications, instructive signals are provided that precisely specify the desired network output for a set of training inputs (e.g., “Image 1 is a Welsh terrier, not an Airedale terrier.”). In contrast, the error signals available to guide supervised learning in the cerebellum and other biological networks are often more indirect and delayed (e.g., visual feedback about where the tennis ball landed rather than a specification of what the commands to each muscle should have been). Solutions to this distal teacher problem (Jordan & Rumelhart 1992,Miall et al. 1993) have been reviewed elsewhere (Ito 2013, Porrill et al. 2013). In this section, we focus on recent experimental work showing that instructive signals in the cerebellum are much richer than previously thought.

5.1. Modulation of Instructive Signals

Neurons of the inferior olive, which send their climbing fiber axons to Purkinje cells (Figure 2), are a key source of instructive signals for supervised learning in the cerebellum (De Zeeuw et al. 1998, Ito 2013, Simpson et al. 1996). Although early experiments suggested that climbing fibers signal errors by generating an all-or-none complex spike response in Purkinje cells (Eccles et al. 1966), subsequent work has revealed that the potency of the climbing fiber signal can be exquisitely regulated by several factors (Najafi & Medina 2013). These factors include (a) intrinsic network mechanisms that can influence the state of Purkinje cells (Callaway et al. 1995, Wang et al. 2000) and of the neurons in the inferior olive (Bazzigaluppi et al. 2012, De Gruijl et al. 2012, Lefler et al. 2014, Mathy et al. 2009, Tokuda et al. 2013); (b) signals from neuromodulatory centers in the brain (Carey & Regehr 2009); and (c) environmental parameters related to the current behavioral state (Apps 2000, Lawrenson et al. 2016), the training context (Kimpo et al. 2014), the salience of error-related cues (Najafi et al. 2014a,b), or the prior history of errors (Maruta et al. 2007, Yang & Lisberger 2017). Because climbing fibers trigger the induction of plasticity at multiple sites in the cerebellar cortex (for a review, see Gao et al. 2012), the modulation of their efficacy has important functional implications for determining how much the cerebellum learns from a given error signal (Carey & Regehr 2009; Mathy et al. 2009; Yang & Lisberger 2014a, 2017). Thus, mechanisms exist for regulating the potency of instructive signals in the cerebellum, and these mechanisms could be used to adjust the rate of learning on the basis of factors such as the relevance of the error and the degree of certainty that the current response of the network causally contributed to the error (Wei & Kording 2009) (for more on rate of learning, including its adaptive modulation, see Section 6).

5.2. Temporal-Difference Instructive Signals

In applications where the desired response varies as a function of time, a time-varying error signal can be computed by comparing the response of the network to the desired response at each time point. Neurons in the inferior olive are well positioned to implement this continuous error computation by adding excitatory inputs related to the desired response and inhibitory inputs related to the actual response generated by the cerebellar nuclei at each point in time (De Zeeuw et al. 1998, Ito 2013, Simpson et al. 1996) (Figure 1b). However, recent work indicates that the way the inferior olive computes dynamic error signals may be more consistent with the method of temporal differences (Ohmae & Medina 2015), whereby an error signal is generated whenever there is a difference between the response of the network at the current time and the prediction of the desired response of the network in the next time step (Sutton 1988). Neural signals resembling temporal-difference errors have been found in other parts of the brain known to play a critical role during reinforcement learning tasks, including the dopaminergic neurons of the midbrain (O’Doherty et al. 2003, Schultz 1998, Seymour et al. 2004), and are also used in machine learning applications to improve performance (Schultz et al. 1997).

5.3. Purkinje Cells as a Source of Instructive Signals

Climbing fibers are not the only neural instructive signal available in the cerebellum to guide learning; the output of Purkinje cells can also carry information about the learning that is required (Miles & Lisberger 1981, Popa et al. 2016, Raymond & Lisberger 1998). Cerebellar learning performance is better predicted by the instructive signals carried by both the climbing fibers and the Purkinje cell output during training than by the climbing fibers alone, and learning can occur under training conditions where the climbing fibers do not carry instructive signals (Ke et al. 2009, Raymond & Lisberger 1998). Moreover, direct stimulation of either the climbing fibers or the Purkinje cells can be used to induce learning in the absence of the normal sensory feedback about errors (Lee et al. 2015, Nguyen-Vu et al. 2013). Purkinje cells fire spontaneously at high rates and can thus carry information and influence the responses of the network by decreasing or increasing their firing. Pauses in Purkinje cell firing, by disinhibiting their targets in the cerebellar nuclei (Heiney et al. 2014), may be particularly effective at driving learning mechanisms at those sites (Lee et al. 2015, McElvain et al. 2010, Medina & Mauk 1999, Person & Raman 2010, Zheng & Raman 2010). The use of multiple neural instructive signals may enhance the dynamic range of the network by allowing learning under a broader range of conditions. The relative contribution of climbing fiber-triggered versus Purkinje cell-triggered plasticity to cerebellar learning may vary with the training conditions and over time (as discussed further in Section 6).

6. MULTIPLE TIMESCALES OF PLASTICITY

In machine learning, the choice of the learning rate for adjusting network parameters is critical—if changes are too big, the network may oscillate or otherwise fail to converge, but if they are too small, learning will take too long. A solution is to adjust the learning rate during training, increasing the rate to make bigger changes when the network is far from a solution or when the statistics of the inputs change and decreasing the learning rate under more stable conditions (Schaul et al. 2012). In the cerebellum, one mechanism for adjusting the learning rate is to modulate the potency of the instructive signals carried by climbing fibers (as described in Section 5.1). A second mechanism, observed in the cerebellum and in other brain regions, is to implement learning with distinct forms of neural plasticity that operate over different timescales (Dudai et al. 2015, Kukushkin & Carew 2017, Squire et al. 2015). This may allow the brain to both adapt and forget quickly if errors are transitory (e.g., in response to muscle fatigue) and to retain what it has learned if errors persist for long periods (e.g., if an increase in body weight requires long-lasting changes in the motor commands) (Kording et al. 2007).

6.1. Seconds: One-Shot, Volatile Cerebellar Plasticity

Cerebellum-dependent learning is incremental, requiring hundreds or thousands of training trials to approach asymptote. However, the error signals carried by the climbing fibers not only trigger small changes that accumulate across many trials but also have a more powerful yet transient influence that supports rapid adaptation to recent experience. Following a single climbing fiber response to an error on one trial, there is an approximately 5% change in the output of the Purkinje cell targets of this climbing fiber and in the behavioral response on the next trial (Medina & Lisberger 2008). These changes are roughly 100-fold larger than expected from the cumulative changes that accrue across multiple trials, but they decay quickly, in approximately 6 s (Khilkevich et al. 2016, Kimpo et al. 2014, Yang & Lisberger 2014b). One candidate synaptic mechanism underlying these rapid, transient changes in the neural and behavioral responses is short-term associative plasticity of the granule cell-Purkinje cell synapses (Brenowitz & Regehr 2005, Suvrathan et al. 2016).

6.2. Minutes-Hours: Incremental Changes in Purkinje Cell Output

Over minutes to hours of training with repeated trials, there is a learned change in the response of the Purkinje cells (for a review, see Medina 2011). Early models attributed this learning process to climbing fiber-triggered, long-term plasticity of the granule cell-Purkinje cell synapses (Ito 2000), which is known to last for hours once it has been induced. However, the extent to which this mechanism, rather than other plasticity mechanisms, contributes to the changes in Purkinje cell output is still unresolved after many decades (Mauk et al. 1998, Schonewille et al. 2011, Yamaguchi et al. 2016), most likely because the recruitment of plasticity at the granule cell-Purkinje cell synapses is highly sensitive to the parameters of training (Boyden et al. 2006, Nguyen-Vu et al. 2013). Notably, different Purkinje cells exhibit a correlate of learning at different times within a training session (Yang & Lisberger 2014b), with some individual cells exhibiting changes only transiently for a few dozen trials and then apparently handing off the memory trace to other Purkinje cells (Li et al. 2011). Thus, there appear to be multiple learning time constants and mechanisms of plasticity within the minutes-hours range.

6.3. Hours-Days: Systems Consolidation

The cerebellar cortex is essential for generating the learned response in the first several hours after training, but it is not essential 24 h later (Kassardjian et al. 2005, Shutoh et al. 2006). A structural correlate of the transient dependence on the cerebellar cortex is a decrease in α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptors at the granule cell-Purkinje cell synapses 1 h after training, which recovers by 24 h after training (Aziz et al. 2014, Wang et al. 2014). Thus, the memory trace seems to be transferred from the cerebellar cortex to a different site, where more durable mechanisms of plasticity support long-term retention of the learned response; the prime candidate for this site is the immediately downstream cerebellar nuclei, where molecular, electrophysiological, and structural correlates of learning have been described (Boele et al. 2013, Gomi et al. 1999, Kleim et al. 2002, Medina et al. 2000b, ten Brinke et al. 2017). This systems consolidation of cerebellar learning depends on neural activity in the cerebellar cortex during the initial hours after training (Cooke et al. 2004) and is reminiscent of findings from other brain areas, such as the hippocampus or the amygdala, where memory traces are gradually transferred to different sites in the brain for long-term storage (Dudai et al. 2015, Medina et al. 2002, Squire et al. 2015). Notably, although the role of the cerebellar cortex is reduced over time, it is not completely eliminated, as persistent changes in synapse number a day or more after training have been reported (Aziz et al. 2014, Black et al. 1990), and certain aspects of learning, such as the learned timing of a response and, possibly, the pattern of generalization of the learning, retain a long-term dependence on the cerebellar cortex (Boyden et al. 2004, Medina et al. 2000a).

6.4. Interaction Between Different Timescales of Plasticity

Slower, long-term plasticity can be observed in the absence of faster, transient plasticity in the cerebellum (Kimpo et al. 2014, Yang & Lisberger 2014b), suggesting that the mechanisms underlying these two types of plasticity may be at least partially independent of each other, as has been described for other neural circuits (Sutton & Carew 2000). However, there is also evidence that, during cerebellar learning tasks, short-term changes can facilitate the induction of long-term changes (Joiner & Smith 2008, Yang & Lisberger 2010). One candidate link between short-term and long-term plasticity is Purkinje cell activity, which serves the dual functions of (a) adaptively adjusting the response of the system by serving as a conduit for the expression of rapid, shorter-term plasticity in the cerebellar cortex while (b) providing instructive signals for inducing slower, more durable changes that integrate and store the learned response over longer timescales in sites downstream of the cerebellar cortex (Lee et al. 2015, Medina et al. 2000b, Miles & Lisberger 1981, Nguyen-Vu et al. 2013, Popa et al. 2016, Raymond & Lisberger 1998). Finally, the interaction between different timescales is bidirectional—not only does rapid, short-term plasticity influence the induction of long-term plasticity, but the longer-term statistics of experience can also scale up or down the amount of rapid learning that takes place during cerebellar tasks (Herzfeld et al. 2014, Yang & Lisberger 2017), similar to strategies used in machine learning (Schaul et al. 2012).

7. HARDWARE SPECIALIZATION

A striking feature of the cerebellum is its highly regular circuit architecture, which is often described as crystalline because it comprises a series of modules of similar anatomical structure (Figure 2). These modules form closed loops among groups of parasagittally aligned Purkinje cells, neurons in the cerebellar nuclei, and the inferior olive and are repeated throughout the cerebellum. The uniform circuit structure suggests that the different cerebellar modules perform the same, canonical computation on the inputs they receive from different sensory, motor, and cognitive brain areas. However, this interpretation has been called into question by the observation of molecular, physiological, and even anatomical variations across cerebellar modules (Cerminara et al. 2015, Chan-Palay 1977, Urbano et al. 2006).

More than 50 genes are differentially expressed in the parasagittal bands of the cerebellar cortex, giving it a striped appearance in sections stained for any of their markers (Cerminara et al. 2015, Hawkes 2014). The cerebellar Purkinje cells, for example, are a well-defined type of neuron on the basis of their distinct morphology and their role as the sole output neurons of the cerebellar cortex; however, on the basis of differences in gene expression profiling, they could be divided into multiple different subgroups (Cerminara et al. 2015). Heterogeneity is not restricted to the Purkinje cells; there are additional differences in gene expression, packing density, and even the size of the cell bodies of the granule cells and interneurons in different cerebellar modules (Cerminara et al. 2015). There are also physiological differences, which correlate with the molecular variations across modules. For example, Purkinje cells expressing the molecule zebrin II have a lower basal firing rate (Xiao et al. 2014; Zhou et al. 2014, 2015) and lower propensity for associative plasticity of their granule cell inputs, as well as a larger response to their climbing fiber input (Tang et al. 2017), than Purkinje cells that are zebrin II negative (Hawkes 2014, Wadiche & Jahr 2005). This heterogeneity is not simply biological noise but rather systematic variation that is highly reproducible across individuals and that obeys the anatomical boundaries between different cerebellar modules (Cerminara et al. 2015, Tsutsumi et al. 2015, Valera et al. 2016).

Recent findings suggest that the variations between cerebellar modules may simply reflect a fine-tuning of the supervised learning computation rather than fundamentally different computational principles. For example, different regions of the cerebellar cortex have different timing requirements for associative synaptic plasticity at the granule cell-Purkinje cell synapses (Suvrathan et al. 2016). This heterogeneity allows the cerebellum to solve the same temporal credit assignment problem (Minsky 1963, Sutton 1984) across modules that receive instructive signals with different feedback delays: Plasticity is induced selectively in those synapses that were active when an error was generated rather than when the feedback about that error reached the cerebellum. Similarly, cerebellar modules engaged during oculomotor learning have anatomical specializations that could support a functional requirement of those behavioral tasks, namely the generation of long time constants for temporal integration. This could be supported by two characteristic features of oculomotor-related modules: (a) a higher density of UBCs that generate sustained responses to input (Dino et al. 2000, Mugnaini & Floris 1994, Rossi et al. 1995) and (b) a higher density of recurrent loops via feedback projections from Purkinje cells to granule cells (Guo et al. 2016). Other specializations across cerebellar modules might compensate for different mean firing rates in the mossy fiber inputs to keep the input layer of the cerebellar cortex in an optimal dynamic range for coding (Cayco-Gajic et al. 2017, Litwin-Kumar et al. 2017) or to prevent saturation of plasticity (Nguyen-Vu et al. 2017).

Thus, variations on the crystalline circuit architecture of the cerebellum may reflect, not fundamentally different computations, but specializations that allow different cerebellar modules to implement the same supervised learning computation on inputs with different statistics and in a way that meets numerous different task requirements. The different cerebellar modules support many functions, including fine motor control and navigation (Lefort et al. 2015), learned fear and other emotions (Perciavalle et al. 2013, Strata 2015), reward (Mittleman et al. 2008), and cognitive processing (Hoche et al. 2016, Koziol et al. 2014, Sokolov et al. 2017). The cerebellum contributes to these different functions through reciprocal connections with most areas of the cerebral cortex as well as subcortical structures (for a review, see Bostan et al. 2013). In the context of these broader networks, the cerebellum may act as a universal neural chip for implementing supervised learning computations, serving different purposes depending on the specific inputs it receives and the way that it is embedded within the larger networks of the brain (Figure 3).

Figure 3.

Network configurations for supervised learning applications. Depending on the way the adaptive processor is embedded in the broader network, supervised learning can be used for (a) system identification, where the adaptive processor learns to generate a response that mimics that of an unknown system; (b) inverse system identification, where the adaptive processor learns to generate a response that mimics the original input signal before it was transformed or corrupted by an unknown system; (c) noise cancellation, where the adaptive processor learns to generate a response that is a clean version of a signal that was embedded in unwanted noise (in this application, the error signal converges to the signal itself, rather than converging to zero); and (d) prediction, where the adaptive processor takes past values of the input signal to learn to generate a response that predicts what the future values of the input signal will be. These different network configurations have been leveraged extensively in engineering applications. The cerebellum could be embedded in larger brain networks in similar configurations to help support a variety of sensory, motor, and cognitive functions, including supervised learning of inverse models for generating suitable motor commands, as well as supervised learning of forward models for predicting the sensory consequences of our actions and for cancelling out neural noise generated by self-motion.

8. CONCLUSIONS AND OPEN QUESTIONS

A clear picture is beginning to emerge of the fundamental principles that allow the cerebellum to implement a general algorithm for supervised learning. (a) Extensive preprocessing of signals at the input layer is used to perform pattern separation and, possibly, to generate time-varying representations. (b) The architecture of the network is highly recurrent. (c) Computations are performed with neurons that have linear input-output functions. (d) Multiple neural elements provide error signals; these error signals encode predictive information and their potency can be modulated to control the rate of learning. (e) Adaptive changes occur at several nodes of the network during learning and differ with regard to both their rate of induction and their persistence. (f) Although the architecture of the cerebellum is generally uniform, there is also task-related specialization of certain properties of the cerebellar hardware.

The principles of supervised learning discovered in the cerebellum have many parallels with those in other biological and artificial neural networks, as well as some key differences. An important principle shared with many other vertebrate and invertebrate biological neural networks is that learning is not a unitary process—multiple instructive signals guide the induction of plasticity at distributed sites in the network, and there are different changes over different timescales. This flexibility could allow learning to be highly specific yet have a broad dynamic range. Massive recurrence is also a nearly ubiquitous feature of neural networks and is now known to be more prominent in the cerebellum than previously recognized. Meanwhile, molecular and functional heterogeneity among neurons of a given type, which have long been recognized in the cerebellum, have also been discovered in other regions of the brain (Cembrowski et al. 2016, Giocomo & Hasselmo 2008, Soltesz 2005). A key difference between the cerebellum and other brain areas is the extraordinary amount of neural hardware devoted to input preprocessing in the cerebellum, which is roughly equal to the number of neurons in the rest of the brain combined. Yet the computational functions that have been attributed to the cerebellar preprocessing stage are similar to those that have been described for other brain areas—decorrelation, pattern separation, and the generation of temporal basis sets (Eichenbaum 2014, Finnerty et al. 2015, Pitkow & Meister 2012, Yassa & Stark 2011). Additional work is needed to identify the computational principles that arise uniquely from the specific architecture of the cerebellum versus those shared with other neural circuits. The relatively simple and uniform circuit architecture of the cerebellum, combined with powerful new tools for visualizing and manipulating neural activity, makes the cerebellum an ideal system for studying general computational principles in the brain and discovering new ways to apply those principles to improve the performance of machine learning algorithms.

There are striking parallels between supervised learning in the cerebellum and the strategies used for supervised learning in artificial neural networks, including the regulation of the learning rate by modulating the potency of the instructive signals that induce changes in the network. One of the most notable differences is that machine learning has not widely employed passive decay of learned changes in the network, whereas this is prominent in the cerebellum and other vertebrate and invertebrate biological networks (Kukushkin & Carew 2017). Future work will determine how much this passive decay is a limitation of the biology (i.e., a bug) or a feature that allows the network to effectively adapt to a dynamic world. Finally, findings from artificial neural networks provide insights about potential functions of supervised learning in the cerebellum (Figure 3). A definitive answer to the question of what the cerebellum learns will need to take into account the possibility that different cerebellar regions may be implementing the same universal supervised learning algorithm but be specialized for different functions depending on the system configuration, the connections to other parts of the brain, and the kind of information that is carried by the input and output signals.