An adaptive filter model of cerebellar zone C3 as a basis for safe limb control?

Abstract

The review asks how the adaptive filter model of the cerebellum might be relevant to experimental work on zone C3, one of the most extensively studied regions of cerebellar cortex. As far as features of the cerebellar microcircuit are concerned, the model appears to fit very well with electrophysiological discoveries concerning the importance of molecular layer interneurons and their plasticity, the significance of long-term potentiation and the striking number of silent parallel fibre synapses. Regarding external connectivity and functionality, a key feature of the adaptive filter model is its use of the decorrelation algorithm, which renders it uniquely suited to problems of sensory noise cancellation. However, this capacity can be extended to the avoidance of sensory interference, by appropriate movements of, for example, the eyes in the vestibulo-ocular reflex. Avoidance becomes particularly important when painful signals are involved, and as the climbing fibre input to zone C3 is extremely responsive to nociceptive stimuli, it is proposed that one function of this zone is the avoidance of pain by, for example, adjusting movements of the body to avoid self-harm. This hypothesis appears consistent with evidence from humans and animals concerning the role of the intermediate cerebellum in classically conditioned withdrawal reflexes, but further experiments focusing on conditioned avoidance are required to test the hypothesis more stringently. The proposed architecture may also be useful for automatic self-adjusting damage avoidance in robots, an important consideration for next generation ‘soft’ robots designed to interact with people.

Sean Anderson received the MEng degree in Control Systems Engineering from the Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield, UK, in 2001, and the PhD degree from the Department of Chemical and Process Engineering, University of Sheffield, in 2005. After working as Research Associate in the Centre for Signal Processing in Neuroimaging and Systems Neuroscience at the Department of Psychology, University of Sheffield from 2005 to 2010 he moved to the Department of Automatic Control and Systems Engineering where he is currently a lecturer with interests in adaptive and optimal biological control, and bioinspired robotics. Paul Dean received the MA degree in physiology with psychology from the University of Cambridge, Cambridge, UK, and the DPhil degree from the University of Oxford, Oxford, UK. He is currently an Emeritus Professor with the Department of Psychology and a Member of the Centre for Signal Processing in Neuroimaging and Systems Neuroscience, University of Sheffield, Sheffield, UK. His research interests include producing computational models of neural systems that are based on both biological data and developments in control engineering, signal processing and robotics, which serve as a vehicle for two-way communication between biological and physical sciences, allowing roboticists to use new discoveries in biology and biologists to interpret their findings in light of current developments in signal processing. Henrik Jörntell received his PhD degree in Neurophysiology from Lund University. He is currently employed as a researcher at Lund University where he heads that lab ‘Neural Basis for Sensorimotor Control’ at the Medical Faculty. His interests is the neurophysiological analysis of neuronal microcircuits involved in movement control, spanning cerebellar, spinal, brainstem and neocortical circuitry as well as models of how these structures interact during movement performance. John Porrill received the BA degree in Mathematics and the PhD degree from the University of Cambridge, Cambridge, UK, where he worked with J. Stewart on topics in classical general relativity. He is currently a Professor in Psychology and a member of the Centre for Signal Processing in Neuroimaging and Systems Neuroscience in the Department of Psychology, University of Sheffield, Sheffield, UK. His research centres around the computational modelling of the neural processes controlling sensory and motor systems, the role of the cerebellum in their adaptive calibration, and the application of these biological principles to the control of biomimetic robot devices.

The Trolleholm meeting held in May 2011 celebrated the work of Carl-Fredrik Ekerot, one of whose major contributions has been a series of discoveries relating to the C3 zone of the cerebellar cortex. This zone receives climbing fibre input from a specific region of the inferior olive (rostral part of dorsal accessory olive), and projects to a specific part of the deep cerebellar nuclei (anterior interpositus nucleus). Ekerot and colleagues have focused on the forelimb region of the C3 zone, situated in lobules IV and V of the anterior lobe (Fig. 1), whose climbing fibre input, among other modalities, carries information about the location of nociceptive stimuli applied to the ipsilateral forelimb (e.g. Ekerot et al. 1991b). The C3 zone sends its output, via the anterior interpositus nucleus, primarily to the M1 motor cortex and the magnocellular part of the red nucleus (Gibson et al. 1987; Ekerot et al. 1995; Jörntell & Ekerot, 1999)

Figure 1. Cerebellar zone C3.

Regions of the inferior olive send their climbing fibres to sagittal strips of cerebellar cortex, termed zones. Parts of six such zones are shown in this anterior/dorsal view of cat cerebellum. Zone C3 receives climbing fibres specifically from the rostral part of the dorsal accessory olive, and like other zones projects to a specific target in the cerebellar nuclei (in the case of C3, the anterior interpositus nucleus). A sagittal zone is thus defined by its olivary inputs and corticonuclear outputs, and by the type of afferent system that activates the climbing fibres (Oscarsson, 1980; Voogd & Bigare, 1980; see also Apps & Garwicz, 2005). A zone can in turn be divided into constituent microzones, each a narrower sagittal strip of cortex, which can span several contiguous folia of the cerebellar cortex, and that receives climbing fibre afferents corresponding to very specific peripheral inputs. There is in addition a coarse somatotopic map running somewhat across the sagittal zonation, so that the control of different parts of the body, and different parts of the neocortex, are localized to different lobules (see Ito, 1984). Lobules III–V (anterior lobe) and VI (posterior lobe) are shown in the figure. The electrophysiological recordings to be discussed were from lobules IV and V of zone C3, which are concerned with control of the forelimbs.

In a series of studies, the Ekerot group have recorded from mossy fibres, climbing fibres, granule cells, molecular layer interneurons (MLIs) and Purkinje cells in this area, and have investigated how pairing mossy and climbing fibre inputs can alter responses of MLIs and Purkinje cells. This work is important to modellers for two reasons. First, because the organization of the basic cortical microcircuit appears very similar throughout the cerebellum, new information concerning the behaviour of the microcircuit in zone C3 is relevant to models that attempt to specify the basic computational algorithm carried out by this microcircuit. Secondly, each individual region of the cerebellar cortex has unique connections, indicating a unique functional role. Models of the basic algorithm used in conjunction with knowledge of that connectivity can therefore be helpful in suggesting hypotheses about the functions of individual areas, in this case the C3 zone in lobules IV and V.

These two issues are reviewed in order below. The review focuses on the adaptive filter model of the cerebellar microcircuit, originally introduced by Fujita (1982) as a development of the original ideas of Marr (1969) and Albus (1971). Variants of the adaptive filter are widely used in models of cerebellar motor control (references in Dean et al. 2010), and one reason for their popularity is that they deal conveniently with the time-varying signals often used for sensory processing and motor control. In contrast, Marr's original model seems only to have been implemented once in detail (Tyrrell & Willshaw, 1992), and even then not in the context of motor control. And although Albus's cerebellar model articulated or arithmetic controller is widely used, its treatment of the cerebellar microcircuit as a look-up table that stores the desired output to a given set of inputs lend itself much more readily to pattern classification and feature detection than to the dynamic processing of time-varying sensorimotor signals (Porrill et al. 2013). Moreover, in addition to its convenience, the adaptive filter combines a plausible explanation of microcircuit features with demonstrated computational power (see below). This combination of qualities makes the adaptive filter a plausible candidate for the basic cerebellar algorithm.

Microcircuit features of C3 zone

The broad resemblance between the structure of the adaptive filter and that of the cerebellar cortical microcircuit is shown in Fig. 2. In the adaptive filter (Fig. 2A); filter inputs are split into a large number of component signals, which are then individually weighted and recombined to produce the filter output. The weights are adjusted by a teaching signal. This structure looks like a simplified version of the cerebellar cortical microcircuit (Fig. 2B), where mossy fibres carry the filter inputs, parallel fibres the component signals, and Purkinje cells produce the filter output. The teaching signal is provided by the climbing fibre input to Purkinje cells. The similarities shown in Fig. 2 are thought significant in part because they address two strikingly unusual features of cerebellar cortex. One is the enormous proliferation of granule cells, which are estimated to constitute ∼80% of all neurons in the human brain (Herculano-Houzel, 2009). The adaptive filter model needs such numbers to provide a set of (possibly non-linear) component signals that is large enough to allow synthesis of all desired output signals. The second feature is the unusual behaviour of climbing fibres. These fire on average at ∼1 Hz, apparently too low a frequency to make a significant contribution to Purkinje cell output (simple spike average ∼40 Hz). However, a single climbing fibre action potential produces a large widespread calcium transient throughout the Purkinje cell dendritic tree (e.g. Ohtsuki et al. 2009) in a manner thought to be related to plasticity at the estimated 150,000 parallel fibre synapses on the tree. This combination of properties is exactly that required by an adaptive filter teaching signal, which must alter all the weights appropriately without affecting the filter output.

Figure 2.

A, adaptive filter. A commonly used adaptive filter architecture is the analysis–synthesis filter. A bank of fixed filters G_i‘analyses’ the input signal x(t) into component signals p_i(t) = G_i[x(t)]. This figure shows three leaky-integrator filters with different time constants. The component signals are recombined (‘synthesized’) to form the output signal y(t), with the amount of a given component in the output controlled by an adjustable weight w_i so that y(t) = Σw_ip_i(t). Weights are adjusted automatically by the learning rule, δw_i=–β〈ep_i〉, where δw_i is the change in weight, e(t) is a teaching signal carrying information about errors in filter output, p_i is the input signal to the weight w_i and 〈ep_i〉 denotes the covariance of e with p_i. This is called the covariance learning rule (Sejnowski, 1977), or the least mean squares rule, because when e is the error in the output y(t), it can be shown to minimize the mean square performance error, or the decorrelation learning rule because learning stops when errors e are uncorrelated with all filter inputs p_i. Although the filter output is linear in the weights w_i, by including appropriate non-linear component filters G_i[x(t)] in the analysis layer, it can be used to model non-linear filters. Adapted from fig. 1A of Dean & Porrill (2010). B, simplified diagram of a cerebellar cortical microcircuit. The input signals to the cerebellum are carried by mossy fibres, which synapse on granule cells (GCs). Granule cells axons bifurcate and form parallel fibres (PFs), which extend over the surface of the cerebellum in PF beams, synapsing extensively on Purkinje cells (PCs), which are the output cells of the cerebellum, and causing them to produce simple spikes. In addition to its many PF inputs a PC takes input from a single climbing fibre (CF), which winds around the cells’ dendrites and produces complex spikes on a one-to-one basis. Note that this simplified figure omits many details of the microcircuit, such as the ascending GC axon inputs, inhibitory projections from GCs to PCs via stellate and basket cells, and the recurrent connection of GCs via Golgi cells. The Marr–Albus interpretation of the microcircuit maps on to the adaptive filter architecture shown in Fig. 1. Processing of the mossy fibre input signal in the GC layer is interpreted as analysis by a bank of filters, to produce component signals carried on the PFs. Combination of these PF inputs weighted by the efficacies of the PF/PC synapses to produce PC output is interpreted as the synthesis stage. The CF input is interpreted as a teaching signal, which adjusts synaptic weights according to a spike-timing-dependent plasticity rule in which weights are decreased (long-term depression) when the PF and CF input to a synapse are positively correlated and increase (long-term potentiation) when they are negatively correlated; this learning rule is equivalent to the covariance learning rule described in Fig. 1. Adapted from fig. 1A of Porrill et al. (2004).

However, the discoveries of the Lund group in relation to cerebellar zone C3 require the simple diagram of Fig. 2B to be expanded and refined. A critical question for the adaptive filter model therefore is whether it can accommodate these new microcircuit findings. Because we have discussed this issue in detail previously (Dean et al. 2010), here we mention briefly three particularly important new findings.

First, electrophysiological recordings from individual Purkinje cells in the forelimb C3 zone indicate that for each cell complex spike firing is exquisitely sensitive to painful stimulation of a restricted area of the forelimb skin (Ekerot et al. 1991b; Ekerot & Jörntell, 2001) (Fig. 3A, top row). These complex spike cutaneous receptive fields can be compared with the receptive fields for simple spikes (Ekerot & Jörntell, 2001) (Fig. 3A, bottom row), which are mapped using light touch (there was no simple spike response to painful skin stimulation). It can be seen that the excitatory receptive fields for simple spikes never overlapped or resembled the complex spike receptive field, although inhibitory fields when present typically did overlap with the complex spike field. This organization was taken to suggest ‘a novel learning mechanism, in which a small subset of the parallel fibre synapses on the Purkinje cell is selected to be potentiated’ (Ekerot & Jörntell, 2001, p. 1309). This idea was tested by measuring the effects on simple spike fields of parallel fibre stimulation unaccompanied by climbing fibre spikes (Jörntell & Ekerot, 2002). Dramatic (>20-fold) increases in receptive fields were observed (Fig. 3B), which could be reversed by subsequent parallel fibre stimulation that was paired with complex spikes (not shown). These findings point to a bidirectional form of plasticity where a given parallel fibre input causes synaptic depression or potentiation depending on the presence or absence of a climbing fibre input (Jörntell & Hansel, 2006; Jörntell & Ekerot, 2011), a form that appears to correspond very well to the covariance learning rule used by the adaptive filter (Fig. 2).

Figure 3.

A, receptive fields of Purkinje cells in forelimb C3 zone. Sample receptive fields of Purkinje cells along a beam of PFs in left cerebellar anterior lobe, lobule V. Receptive fields of complex spikes (dark blue) and simple spikes (yellow-to-red gradient) are shown for a sequence of Purkinje cells recorded from medial to lateral. Based on Ekerot & Jörntell (2001, 2003). B, induction of increased simple spike receptive fields in Purkinje cells. Changes in a simple spike receptive field induced by the PF burst protocol (Jörntell & Ekerot, 2002) in which PF burst stimulation is delivered unaccompanied by climbing fibre discharge. Inputs from different skin regions were quantified in peristimulus histograms (see Jörntell & Ekerot, 2011). Excitatory receptive fields on the left (ipsilateral) forelimb are indicated by a yellow-to-red colour gradient, the redder shades indicating stronger excitatory input. Note the particularly early appearance of a strong excitatory input from the skin area corresponding to the climbing fibre receptive field, a consistent finding, which may reflect the inputs from the ascending granule cell axons (Jörntell & Ekerot, 2002) (see also circuitry diagram of Fig. 7). The time course indicated in hours and minutes is relative to the end of the PF burst protocol. The complex spike receptive field is indicated on the left. c, contralateral; i, ipsilateral; FL, forelimb; HL, hindlimb; rf incl., receptive field included.

Second, the simple adaptive filter model shown in Fig. 2 has weights that can take either positive or negative values, but actual synapses are constrained to be either excitatory or inhibitory. A solution to this problem would be to include a parallel pathway from parallel fibres to Purkinje cells via inhibitory interneurons [some consequences of such a pathway are discussed in Porrill & Dean (2008)]. Synaptic weights between parallel fibres and MLIs (stellate and basket cells) would then behave as if they were negative weights between parallel fibres and Purkinje cells. It therefore follows that these weights too must show plasticity (but with the opposite sign) in accordance with the covariance rule. Recordings from the C3 forelimb zone (Jörntell & Ekerot, 2002, 2011; Ekerot & Jörntell, 2003) suggest that the pathway from granule cells to Purkinje cells via the inhibitory MLIs (stellate and basket cells) has exactly the properties predicted by the adaptive filter model (Jörntell et al. 2010). A number of other recent studies of MLIs can be found in the literature (e.g. Dizon & Khodakhah, 2011; Abrahamsson et al. 2012; Gao et al. 2012; Mathews et al. 2012; Park et al. 2012; Consalez & Hawkes, 2013).

Third, a central feature of the adaptive filter algorithm is the analysis of input signals into different components (Fig. 2), which in the cerebellum has been assumed to take place in the granular layer (Fujita, 1982). Overcoming the technical difficulties of recording from granule cells in vivo (Chadderton et al. 2004), the Lund group found that in the C3 forelimb zone the firing patterns of granule cells do not support the idea that the granule layer performs complex signal decomposition (Jörntell & Ekerot, 2006; Ekerot & Jörntell, 2008; Bengtsson & Jörntell, 2009), but suggest rather a simple noise filtering of submodality specific mossy fibre input (Dean et al. 2010). These findings raise important questions about the role of the granular layer in the adaptive filter model, and suggest further experiments exploring, in particular, the behaviour of granular layer Golgi cells, and the possibility that there are differences in granular layer processing between different cerebellar zones (Arenz et al. 2009; D’Angelo & De Zeeuw, 2009; Dean & Porrill, 2010). A number of reviews have addressed the issue of granular layer processing (D’Angelo & De Zeeuw, 2009; D’Angelo et al. 2009, 2011; Yamazaki & Tanaka, 2009; Galliano et al. 2010; Carey, 2011; D’Angelo, 2011; Gao et al. 2012; Spanne & Jörntell, 2013). Of particular relevance to the adaptive filter model are the conclusions that the ‘structure of the granular layer network and its mossy fibre inputs is well suited for spreading diverse sets of information’ (Gao et al. 2012, p. 625), and that ‘[p]lasticity in the granular layer may increase the diversity of coding, whereas plasticity in the Purkinje cell network may facilitate selection of the appropriate coding’ (Gao et al. 2012, p. 631). A computational method for creating diversity in the context of zone C3 based on the information provided by the mossy fibre systems rather than plasticity in the granule layer is described by Spanne and Jörntell (2013).

The above suggests that although the adaptive filter model accounts well for certain features of the cerebellar microcircuit, there may be others, discovered by research groups both in Lund and elsewhere, that have yet to be fully evaluated. We have argued elsewhere that one way of looking at this evaluation is to regard it as a means of testing the adaptive-filter hypothesis (Dean & Porrill, 2011). Here we ask the question: if the adaptive filter hypothesis were correct, what could it tell us about functions of the C3 zone?

Functions of C3 zone

The second desirable property of the adaptive filter model is its computational power and flexibility. The covariance learning rule is equivalent to the least mean square rule in artificial systems (Widrow & Stearns, 1985), and can be shown to be optimal in the sense of minimizing the mean square difference between desired and actual output. Its effect for appropriately connected filters is to decorrelate all the component signals from the teaching signal. Because of this optimality the adaptive filter can be wired to perform a very wide series of signal processing and control tasks (e.g. Widrow & Stearns, 1985). Such power and flexibility is relevant to a key feature of cerebellar function, that a uniform microcircuit can be used for many different purposes. It has long been recognized that the cerebellar microcircuit appears broadly similar throughout the entire cortex, whereas each individual microzone of cerebellar cortex has a unique pattern of connections with external neural structures (Ito, 1970), and has given rise to the ‘chip’ metaphor of cerebellar organization (Ito, 1997).

This metaphor is illustrated in Fig. 4, which shows in schematic form a functional subregion of cerebellar cortex represented by an identical internal structure and idiosyncratic inputs and outputs. The question addressed here is how the computational capacities of the adaptive filter model could be harnessed to offer insight into aspects of C3 zone function.

Figure 4. The cerebellar ‘chip’ metaphor.

The arrangement of neurons and their connections within the cerebellar cortex is similar over the entire cortex, whereas individual regions of the cerebellar cortex have unique patterns of connections with external neural structures (Herrick, 1924). This combination has suggested what has been termed the ‘chip’ metaphor of cerebellar organization (Ito, 1997), shown here in simplified form. The climbing fibre and output connections are unique to each microzone, whereas some of a microzone's mossy fibre inputs may be shared with other microzones. The climbing fibre teaching signal specifies the learning goals of a particular chip, and historically this connectivity has been central to defining individual microzones. The Purkinje cell output is connected to a region in the deep cerebellar (or vestibular) nuclei, which contributes to achieving this goal. The mossy fibre input connections convey a wide array of possibly relevant sensory and motor signals, from which those signals actually relevant to the task will be chosen by the learning procedure. Adapted from fig. 2 of Porrill et al. (2013)

Sensory noise cancellation

We suggest that a possible link between the adaptive filter model and C3 function is the notion of motor noise cancellation, as explained in the next two sections. First, we consider the problem of adaptive sensory noise cancellation.

Adaptive filters have been applied to the generic problem of adaptive noise cancellation using a circuit of the form shown in Fig. 5A (Widrow & Stearns, 1985). The problem to be solved concerns a signal of interest s(t) that is contaminated by noise n(t), as, for example, in the case of an electrophysiological recording distorted by the subject's heartbeat (Widrow et al. 1975). Some information about the noise source is available, for example a recording of the heartbeat, termed reference noise r(t). The reference noise is used as the input to an adaptive filter, whose function is to provide an estimate of the noise that is added to the signal. It can be seen that if n(t) and r(t) were identical, i.e. the noise added to the signal were identical to the reference noise picked up by the heartbeat recording- the task would be very straightforward, because the output of the filter would be identical to its input. However, the properties of r(t) have been changed, in unknown ways, before it is added to the signal of interest, a process represented in Fig. 5A by the box labelled ‘noise channel’. The noise channel is regarded as a fixed filter with unknown properties (e.g. high pass), so the task for the adaptive filter is to learn to mimic those properties.

Figure 5. Architecture for adaptive noise cancellation.

A, adaptive noise cancelation architecture. The problem addressed in noise cancelation is to suppress noise n(t) that additively corrupts a signal of interest s(t). A known reference noise r(t), which is correlated but not identical with the unknown disturbance noise, is used as input to an adaptive filter (here labelled ‘Forward Model’ to denote its function when learning is complete). The predicted noise signal produced by the forward model is used to cancel the noise from the observed signal, resulting in a prediction of the signal of interest, this signal also acts as teaching signal. When learning is successful, the filter also transforms its input r(t) into n(t) exactly as the noise channel does. Thus, when the filter's output (, ‘predicted noise’) is subtracted from the noisy audio signal (x(t) =s(t)+n(t)), the result (=s(t) +n(t) −, noise cancelled output) corresponds exactly to the original signal s(t). B, reafferent signal cancelation. A special case of noise cancellation is when the noise derives from the animal's (or robot's) own movement (reafferent signal’) and interferes with the detection and analysis of external (‘exafferent’) signals. The adaptive noise cancelation architecture in (A) can be used directly to overcome the reafference problem by substituting motor commands for reference noise. Hence, the organism or robot can learn to predict the sensory consequences of its own movements and cancel these reafferent components from observed sensory signals. A and B were adapted from fig. 5 of Porrill et al. (2013). C, photograph of the mobile whiskered robot exploring the environment using its bilateral array of active artificial whiskers. Movement of these whiskers (‘whisking’) produces reafferent whisker signals that interfere with the detection of external signals, as shown in B. From fig. 2 of Anderson et al. (2012). D, version of the forward-model architecture (B) found in some electric fish (Bell et al. 2008). Principal cells in ‘cerebellar-like’ structures receive an efference copy of relevant motor commands delivered to their apical dendrites via parallel fibres. The basal dendrites receive input from the periphery, i.e. sensory afferents that carry, e.g. electroreceptive information contaminated by reafferent signals. The input pathway via the parallel fibres/apical dendrites appears to perform the function of a forward model, and the model's predictions are subtracted from the contaminated sensory signals that arrive via the basal dendrites. The output of the structure is the prediction of the exafferent signal. The principal cell thus embodies the complete noise cancellation scheme, i.e. adaptive filter plus comparator. Associative learning is driven by correlation between the principal-cell output and the parallel fibre inputs. An important difference between this architecture and the one shown in B is that the output of the principal cell (predicted exafferent signal) can serve as the required teaching signal. In contrast, the output from the cerebellum is the forward model prediction of the noise, which is not suitable as a teaching signal. An additional structure is therefore required to act as comparator to predict the exafferent signal, and an additional pathway is required to feed the error signal back to the Purkinje cell to drive associative learning, i.e. the climbing fibre.

When learning is successful, the filter also transforms its input r(t) into n(t) exactly as the noise channel does. Thus, when the filter's output is subtracted from the noisy audio signal, the result corresponds exactly to the original signal s(t) so that perfect noise cancellation has been achieved. Nevertheless, how can the adaptive filter learn to mimic the noise channel? The decorrelation learning rule typically used in adaptive filters is in fact beautifully suited to precisely this task. The output of the noise canceller is used as the teaching signal. If it contains any contamination by noise, then there will be a residual correlation between system output and components of the filter input r(t). The learning rule is designed to remove any such correlation, so that when learning is complete, there will no longer be any contamination of filter output by noise (provided the appropriate component filters are available).

From a biological perspective, the crucial feature of the circuit in Fig. 5A is that it can be transferred directly to the problem of predicting the sensory effects of movement in biological systems, as shown in Fig. 5B. The observed signal now comes from a biological sensor, combining information about the external world (‘exafferent signal’) and interference produced by the animal's own movement (‘reafferent signal’). Information about these movements is provided by their motor commands. This ‘efference copy’ information is in effect reference information r(t) about the noise source, and so can be used as input to an adaptive filter (in the cerebellum) that learns to mimic the transformation of motor commands into sensory signals. The cerebellum in effect learns a ‘forward model’ of the box labelled ‘motor plant’ in Fig. 6B, which refers to the mechanical properties of the muscles and associated tissue whose combination can be represented by an appropriate fixed filter, similar to the noise channel in Fig. 5A. Once learning has been achieved, the adaptive filter mimics this fixed filter, and its output is an explicit prediction of the effects of the animal's own movements on the sensory signal provided by the whiskers. This prediction is then subtracted from the raw sensory input to provide an estimate of the sensory signal generated by stimuli in the outside world. Again it is this signal estimate that is used as a teaching signal; thus removing any component of the signal that has a component correlated with (and so is predictable by) the organism's movement alone.

Figure 6.

A, vestibulo-ocular reflex (VOR). An illustration of how walking produces up-and-down movements of the head. Unless the eyes move to counteract these movements (VOR), the retinal image will move and visual information will be lost. The VOR can thus be regarded as an example of noise cancellation, where the noise is produced by the organism's own movement. B, architecture for learning accurate tracking movements. The architecture shown in Fig. 5B removes the effects of self-movement from an internal estimate of a sensory input. An alternative strategy, adopted in the VOR, is to counteract the self-movement (in this case the head) by moving the sensor itself (in this case the retina). This strategy requires three major modifications to the architecture shown in Fig. 5B so that can be used to learn the required accurate tracking eye movements. (1) The output of the cancellation module is now a motor command to the oculomotor plant rather than an estimate of the self-induced slip, when learning is complete retinal slip will be decorrelated from these motor command, i.e. it will have no self-generated component. (2) As the cancellation of head velocity by eye velocity is now physical rather than internal, the signal required as the teaching signal is no longer an internal signal but is the measured retinal slip itself. This sensory signal must be made available on the climbing fibre. (3) A copy of the motor command signals must be made available to the adaptive filter. In addition, as the direct path via the brainstem contributes to the VOR, the cerebellum only learns a ‘partial’ forward model. Diagram adapted from fig. 3B of Dean & Porrill (2010). C, Robot VOR Robot ‘head’ for which camera image was stabilized using the architecture shown in B (Lenz et al. 2008, 2009).

The relevance of the circuit shown in Fig. 5B to the cerebellum is supported by three lines of argument. First, the viability of the circuit has been examined in a robot designed to mimic the whisking behaviour of rats (Fig. 5C). In this robot, the signal of interest comes from sensors that monitor the movements of the (artificial) whiskers. This sensory signal is contaminated by ‘self-noise’, generated by the robot's own movements of the whiskers. Information about these whisking movements is provided by the commands sent to the motors that move the whiskers. The adaptive filter uses this information to learn a forward model of the robot whisker plant, which substantial improves detection of whisker deflections produced by objects in the outside world (Anderson et al. 2010). Secondly, adaptive noise cancellation has been established as a function of precerebellar structures (Fig. 5D). Thirdly, learning to predict the sensory effects of movement is widely thought to be a central cerebellar function (Wolpert et al. 1998). A homely example concerns the difficulty of tickling oneself: the argument is that this difficulty is caused by a prediction of the sensory effects of one's own movement that is used to diminish the actual sensory effects. Imaging and other evidence strongly point to the involvement of the cerebellum in this process (Blakemore et al. 1999). Much more generally, the predicted effects of movement have widespread uses in motor control (Wolpert et al. 1998), so that the cerebellum's capacity to learn a model of plant dynamics (the forward model) is of great potential importance (Porrill et al. 2013).

‘Motor’ noise cancellation

Although these arguments provide general support for a role for the cerebellum in sensory noise cancellation, there are as yet few detailed proposals for underlying neural circuits that could implement specific applications of the scheme (Anderson et al. 2012). Historically attention has focused on a closely related scheme, in which sensory interference is prevented by appropriate movement. An example is the vestibulo-ocular reflex (VOR), which helps to stabilize the retinal image when the head moves (Fig. 6A). Movements of the head also displace the eyes, which results in movement of the whole image across the retina. In sensory noise cancellation such retinal slip would be treated as a sensory consequence of head movement that can be predicted using an architecture similar to that shown in Fig. 5B. Here head motion is regarded as a source of noise in the retinal slip signal, which otherwise would indicate actual movement of the organism's surround (in practice such movement is rare, so the uncorrupted retinal slip signal is typically zero). To remove self-generated retinal slip a reference signal related to head movement would be sent to the adaptive filter. (In the case of the VOR, this signal is provided by the vestibular system rather than an efference copy of the head movement command.) The adaptive filter can then learn to predict the retinal slip produced by head movement, and subtract it from the actual retinal slip signal.

Unfortunately, this direct application of the sensory noise cancellation architecture of Fig. 5B to head movement interference has a serious disadvantage. Even if the filter successfully produced an uncontaminated retinal slip signal, this would not remove the effect of image movement on visual processing. Image movement degrades the visual signal, for example by losing information about fine detail that can never be restored. Retinal slip therefore needs to be prevented, by moving the eyes to counteract the effects of the head movement. This is indeed the function of the VOR, which ensures that noise cancellation is carried out in the external world, rather than within the signal processing system itself. This new ‘motor’ cancellation strategy requires three important changes to the architecture shown in Fig. 5B, as illustrated in Fig. 6B.

First, the output of the filter is no longer an estimate of the retinal slip caused by head movement, but is instead a contribution to the motor command sent to the eye muscles (the remainder of the command is provided by the brainstem, which also receives vestibular information).

Secondly, because the output of the active noise canceller (dotted box in Fig. 6B) is no longer an estimate of the uncorrupted retinal slip signal, it cannot be used as a teaching signal. For example, if the VOR becomes perfectly calibrated, retinal slip will always be zero, yet eye movement command cannot be zero when the head is moving. A new error signal is required that is directly related to retinal slip.

The third change to the architecture of Fig. 5B is needed to cope with the mechanical effects of the oculomotor plant, which is in the pathway between system output and actual movement of the eyes. The oculomotor plant consists of the eye muscles together with tissues in the orbit, and its mechanical properties can be represented by a fixed filter (see above). This filter is a potential source of noise in the signal of interest (retinal slip) because if its properties change, for example should the eye muscles weaken, the system's previously appropriate motor commands now produce eye movements too small to counteract movements of the head. Movement-related retinal slip will therefore reappear.

A possible solution to this problem is to send an efference copy of the eye movement commands to the adaptive filter, allowing it to learn to decorrelate the commands from retinal slip. The basis of the learning is the same as for sensory noise cancellation: if there is a correlation between the motor command and retinal slip then the motor commands are incorrect. Removing this correlation ensures that any residual slip was not caused by inaccurate eye movement commands. In this fashion, the cerebellum can learn to produce accurate eye movements (Dean et al. 2002). This architecture has been successfully applied to gaze stabilization in a robot head (Lenz et al. 2009) (Fig. 6C), and appears broadly consistent with connections of the flocculus, the relevant region of the cerebellum (Porrill et al. 2013).

Motor noise cancellation, avoidance and zone C3

The above indicates how the cerebellum could play a role in motor noise cancellation whenever the sensory ‘noise’ is better avoided than cancelled. This is clearly the case when the sensory noise in question signals pain (and potential tissue damage). Thus, if an organism's movement causes pain, the motor noise cancellation architecture could be used to modify the movement so that pain is avoided. Given that the climbing fibre input to zone C3 responds to noxious stimulation of the skin, and potentially also to group III and group IV muscle afferents (Jörntell et al. 1996) signalling muscle overload (Rybicki et al. 1985; Rotto & Kaufman, 1988), one possible function of the zone would be to use inputs related to movement to avoid painful stimulation that those movements would otherwise produce. This would provide an automatic adaptive mechanism for preventing the organism's own movements from causing damage.

Organisation of complex spike receptive fields

The examples illustrated in Fig. 3A show that the complex spike receptive fields of Purkinje cells in the forelimb C3 area for painful skin stimulation have rather distinctive spatial organizations. In fact, about 30 distinct subclasses of spatial organization have been observed, suggesting that ‘the organization of each receptive field (and thus subclass) is related to a specific motor function. Since the proximal borders of the receptive fields are located close to joints their activity may be related to movements caused by contraction in single muscles or small groups of muscles acting on the same proximal joint’ (Ekerot et al. 1991b, p. 270).

The authors propose a hypothesis that ‘the forelimb area of the C3 zone is composed of microzones, each of which receives information via climbing fibres from spinal multimodal pathway reflex arcs acting on a single muscle or group of synergistic muscles’ (p. 272). The cutaneous information provided to the climbing fibres is mediated via primary sensory afferents directly or indirectly synapsing on spinal neurons that provide the axons of the postsynaptic dorsal column pathway, which in turn synapse in the cuneate nucleus before the information is forwarded to the inferior olive (Ekerot et al. 1991a). The postsynaptic dorsal column pathway thus mediates cutaneous nociceptive information to cerebellar climbing fibres in the cat. Hence, the cutaneous information to the climbing fibres could be regarded as a tap-off of the sensory input to these reflex arcs, which means that implicit in the sensory coordinate of the receptive field being activated lies information about the motor coordinate, i.e. which single muscle or synergy is presently being activated (Garwicz et al. 2002).

It is important to point out that even though the spinal motor circuitry is described here in terms of reflex arcs and withdrawal reflexes, they constitute parts of a general circuitry on which all motor commands, generated by motor systems such as the corticospinal tract, also act (Jankowska, 1992; Kitazawa et al. 1993; Raphael et al. 2010). The term ‘reflex’ is hence in this context a misnomer, since the system is clearly involved in the control of voluntary movements (Van Kan et al. 1993a,b; Horn et al. 2010), but it has served as a useful descriptive frame of reference against earlier work carried out on spinal cord circuitry organization.

Proposed circuitry for avoidance of painful stimuli

Figure 7 represents a summary of some of the main features of circuit organization in the C3 zone of the normal, adult cat. It focuses on one of the 30–40 modules of which this system is believed to be composed (Apps & Garwicz, 2005) and thus represents the control structure for only one of the controlled movement directions. This would be the structure of a well-adapted circuitry that would be compatible with the function of pain avoidance. The function of pain avoidance could then be achieved as follows: zone C3 uses relevant information to predict when painful tactile stimulation will occur, and learns to use this predictive information to alter ongoing movements to avoid the pain. Thus, ‘interneurons may have the role of learning to predict conditions that would lead to climbing fibre activation’ and prevent them from occurring (Jörntell & Ekerot, 2003, p. 9630). This could be achieved during reaching movements, when the output of the module is such ‘…that the efferent action of a module is to withdraw the [climbing fibre] receptive field from an external stimulus’ (Ekerot et al. 1995, p. 365).

Figure 7. Organisation of the C3 zone circuitry in relation to its external corrections.

The principles of input connectivity are shown for PCs of two different microzones. The climbing fibre input to the PCs of a single microzone is driven from one specific skin receptive field (shown in blue), whereas the active parallel fibre input, presumably reflecting the parallel fibre synapses active on these PCs, is driven from a completely different skin receptive field (shown in yellow and red). The movement controlled by a single microzone (via the DCN cells, movement indicated by large empty arrows) is of a synergistic type, covering multiple joints, which would have the effect of withdrawing the climbing fibre receptive field from an external stimulus touching that field during reaching. The active parallel fibre input to the PC has, in the case illustrated, ‘an opposite location’ with respect to the movement controlled. The inhibitory interneurons of the molecular layer, in this figure represented by a stellate cell, are driven by parallel fibre input activated from skin receptive fields that overlap that of the climbing fibre receptive field within the same microzone. Mossy fibres and granule cells tend to form clusters where they are activated by the same input. This input has a high degree of similarity to the input to the climbing fibres in the overlying microzone. The Golgi cell receives excitation from the same inputs as the local granule cells, which it inhibits. The inputs to the interneurons and Golgi cells are similar to the receptive fields of the local granule cells, so these cells may be a target of active synapses made by ascending granule cell axons. In contrast, the active parallel fibre input to the PC comes primarily from granule cells underlying other microzones located further away. The ‘interrupt symbols’ for connections with the limb indicate: (1) that afferent input travels to the CNS via the same sensory primary afferents but the information takes different synaptic pathways before it reaches the cerebellum via the mossy fibres and climbing fibres; (2) the efferent information from the DCN cells does not target alpha-motoneurons directly but is mediated via motor systems for limb control. DCN, deep cerebellar nucleus; PC, Purkinje cell. Based on Ekerot et al. (1991b, 1995); Ekerot & Jörntell (2001, 2003); Jörntell & Ekerot (2002, 2003, 2006).

In this example, the predictive information would concern the animal's own movements and would require the availability of a pure sensory mossy fibre signal activated during the movement (Edin & Johansson, 1995; Edin, 2001), which is available from the cuneocerebellar mossy fibre system (Garwicz et al. 1998), and an efference copy signal, which is available, for example, through the lateral reticular nucleus (Clendenin et al. 1974) to which the motor cortex project directly (Wiesendanger & Wiesendanger, 1987). For the function of pain avoidance, the system could then use any signal, whether motor or sensory or a combination thereof, to predict states that would normally lead to pain. One potential predictor of pain to a region of skin may be previous touch to that region. If so, receptive fields of inhibitory interneurons would be expected to overlap with climbing fibre receptive fields (inhibiting Purkinje cells facilitates the withdrawal reflex) (e.g. Ekerot & Jörntell, 2003). Indeed, in the normal circuitry, this is precisely the location of the receptive field of the inhibitory interneuron of the module, as well as the location of the inhibitory receptive fields of the Purkinje cell (Ekerot & Jörntell, 2001, 2003; Jörntell & Ekerot, 2002, 2003).

The circuitry illustrated in Fig. 7 suggests that there might be a version of motor noise cancellation architecture (Fig. 6B) that could learn to predict and avoid painful and possibly damaging stimuli to the limb when these result from caused by unsuitable movements. A candidate version is shown in Fig. 8. Depending on how the body and external world are configured, certain motor commands will cause the limb to collide either with another part of the body, or with some object in the external world. This collision is assumed to produce a signal from the area of the limb that makes the contact adequate to produce a withdrawal reflex of the kind discussed above (which may not require actual activation of nociceptors as tactile input from the corresponding skin area also excites the spinal reflex arcs Schouenborg et al. 1995). It is also used as a teaching signal for the adaptive filter. If information is available that predicts the collision, the filter can use the decorrelation learning rule to activate the reflex before collision, and thus avoid the pain. Possible sources of information include efference copy of the command, proprioceptive inputs describing body configuration, and touch signals that can either indicate that the limb is moving in a certain direction or non-painful contact that could potentially lead to nociceptive activation.

Figure 8. Possible adaptive filter architecture for pain avoidance in co-ordinated limb control.

For a given configuration of the body and limbs, particular movement commands will result in a collision between the limb and a part of the body, producing a localized pain signal in the skin of the limb. Before the adaptive filter learns, this painful stimulus triggers a withdrawal reflex organized in part by circuitry within the spinal cord. However, if the adaptive filter receives information that predicts the collision, then that same withdrawal circuitry can be used to avoid it using the decorrelation rule as in noise cancellation. Possible sources of such information include an efference copy of the command and proprioceptive information related to body configuration. Touch signals can indicate limb movement on its own, or non-painful contact between the limb and part of the body. It should be noted that the climbing fibres to zone C3 are also driven by tactile skin receptors, with receptive fields that overlap those of the nociceptors. Thus even if the powerful tactile stimulus generated by a bump into another part of the body was insufficient to trigger the nociceptors of the skin, it would still be enough to drive the climbing fibre. Such a tactile input is also a drive for the withdrawal reflex arc (though less potent than a nociceptive input Schouenborg et al. 1995).

The architecture shown in Fig. 8 is consistent with the connectivity of zone C3 as outlined above. (i) The teaching signal input to the adaptive filter corresponds to the climbing fibre input to zone C3 in terms of signalling localized pain. (ii) It is a tap off of the sensory input to withdrawal reflex arcs. (iii) The main inputs to the adaptive filter correspond to the mossy fibre inputs to zone C3 observed experimentally. (iv) The filter output makes use of existing withdrawal reflex circuitry.

Features of proposed circuit

As mentioned above, even though the brainstem and spinal motor circuitry is described here in terms of reflex arcs and withdrawal reflexes, they constitute parts of a general circuitry on which all motor commands, generated by motor systems such as the corticospinal tract, also act (Jankowska, 1992; Kitazawa et al. 1993; Raphael et al. 2010). This issue is relevant to the control scheme proposed by Herreros and Verschure (2013), a scheme related to the one discussed here that focuses on how reactive control (corresponding to the withdrawal reflex mode) can be blended with adaptive control (corresponding to predictive avoidance). It is also relevant to the feedback-error-learning architecture in which control is transferred adaptively from a conventional feedback controller to a feedforward controller (Kawato, 1990; Gomi & Kawato, 1993).

A second feature relates to the overlap of climbing fibre and mossy fibre inputs to some regions of cerebellar cortex (e.g. Pijpers et al. 2006). We have previously suggested that such overlap might be related to the requirements of an internal model for novelty detection (Anderson et al. 2012), as past inputs to a particular region of the body may be particularly useful for predicting future inputs to that region. If a similar consideration applies to the prediction of localized pain signals, then granule cells in a given microzone will receive mossy fibre inputs with receptive fields similar to the climbing fibre receptive fields for that microzone. The proposed involvement of zone C3 in predictive withdrawal indicates that such inputs need to inhibit the Purkinje cells of the microzone, thus releasing the anterior interpositus nucleus to facilitate the reflex predictively. This inhibition has to be exercised by MLIs, so the prediction is that the receptive fields for MLIs will resemble those of the climbing fibres for that microzone. This resemblance has been observed experimentally (Ekerot & Jörntell, 2001, 2003; Jörntell & Ekerot, 2002, 2003).

Relation to previous work

Our hypothesis is that one of the functions of zone C3 in limb control is to prevent limb movements that would cause damage, either through collision with other parts of the body, or because of muscle overload. This safety function is obviously important for survival, and is likely to be relevant for certain kinds of soft robot (Trivedi et al. 2008) as well as for biological organisms, yet to the best of our knowledge has not been discussed extensively in the literature.

It is, however, related to the problem of collision avoidance, where the locomoting animal or robot learns to alter its trajectory to prevent damaging collisions, and here a recent study has demonstrated in simulation the utility of a cerebellum-based controller (Herreros & Verschure, 2013). In both cases the cerebellum is used not simply for prediction (e.g. Miall & Wolpert, 1996; Wolpert et al. 1998; Blakemore et al. 1999), but is placed in an architecture that allows predicted sensory events to be avoided. In the ‘motor noise cancellation’ architecture discussed here, the internal model (incorporating an adaptive filter) is used specifically to alter ongoing movement commands to make them safe, a purpose that as far as we aware has not hitherto been suggested.

This point is also relevant to previous experimental evidence concerning zone C3 and pain avoidance. As discussed above, a number of studies have pointed to a strong connection between zone C3 and limb withdrawal reflexes, and it has been suggested that an ability to predict when such reflexes are activated could be used for pain avoidance. The proposal here focuses on a specific and previously unaddressed aspect of this general idea, namely that contact with one's own limbs can in principle be a powerful source of withdrawal reflexes.

Finally, a number of studies of classically conditioned limb withdrawal reflexes in animals and humans have suggested involvement of the intermediate cerebellum, which includes zone C3 (Timmann et al. 1996, 2000; Kolb et al. 1997; Bracha et al. 1999; Apps & Lee, 2002). Moreover, classical eyeblink conditioning also involves zone C3, although the critical region is in lobule VI located posteriorly to the regions considered here in lobules IV and V (e.g. Mostofi et al. 2010). It has been suggested that the role of the cerebellum in eyeblink can be explained at a general level by the ideas of Marr and Albus (e.g. Yeo & Hesslow, 1998), and more specifically by an adaptive filter type model (Medina et al. 2000). In addition, possible links between the role of the cerebellum in eyeblink conditioning and adaptation of the VOR have been discussed (e.g. Raymond et al. 1996; De Zeeuw & Yeo, 2005). However, the relation between classical conditioning and pain avoidance is not straightforward (see below), and in any event the idea that a likely (the most likely?) source of damage to the cornea is one's own movements (e.g. when grooming or scratching the face) does not seem to have been a major influence on these discussions.

New work

As mentioned above, previous studies have implicated zone C3 in classically conditioned limb withdrawal reflexes. However, although it has been claimed that ‘the adaptive acquisition of anticipatory defensive responses is commonly studied using a classical conditioning paradigm’ (Bracha et al. 1999, p. 77), this paradigm is in fact defined by its ensuring there is no link between the conditioned response and the effectiveness of the unconditioned stimulus. Eyeblinks for example are classically conditioned when the unconditioned stimulus is an electric shock to the skin round the eye. In contrast, if the unconditioned stimulus is an airpuff, and the eyelids are allowed to move, learned eyelid closure now does protect the cornea, and the paradigm is now longer classical but instrumental – i.e. avoidance – conditioning. It is currently unclear whether classically conditioned and avoidant eyeblink responses differ significantly (references in Lepora et al. 2010). It may therefore be prudent to study C3 function specifically with conditioned avoidance tasks, preferably where the stimulus to be avoided is generated by the organism's own movement. Moreover, techniques capable of inactivating individual zones (e.g. Pijpers et al. 2008) could be helpful in relating any avoidance defects found more precisely to zone C3.

Finally, the present hypothesis may be of relevance to robotics. There is increasing interest in using biomimetic robots made from ‘soft’ materials (Trivedi et al. 2008), not least because they are likely to be safer and more trustworthy in domestic contexts. However, such robots will be much prone to damage, highlighting the importance of anticipatory defensive responses. Attempting to implement the scheme outlined in Fig. 8 in an actual robot will illuminate the real world problems associated with self-protection. For example, we have only considered the adaptive-filter model in the context of single joint movements, and an interesting aim for future work would be to extend this model to the kind of multijoint movements associated with zone C3 (Van Kan et al. 1993b). As this step would entail a number of complications such as non-linear intersegment dependencies of the limb, implementation in a physical robot would be especially helpful.

Glossary

MLI

molecular layer interneuron

VOR

vestibulo-ocular reflex

Additional information

Competing interests

None.

Funding

This paper describes research supported by the UK Engineering and Physical Sciences Research Council, under the Novel Computation Initiative (GR/T10602/01), and the European Union (BIOTACT, ICT-215910; REALNET, 270434 FP7). H.J. was supported by grants from NINDS/NIH (R01 NS040863), The Hand Embodied (THE) (an Integrated Project funded by the EU under FP7, project no. 248587) and the Swedish Research Council (VR Medicine).