State estimation in the cerebellum

Abstract

An exciting hypothesis about the cerebellum is that its role is one of state estimation - a process that combines efferent copies of motor commands with afferent sensory signals to produce a representation of the current status of the peripheral motor system. Sensory inputs alone cannot provide a perfect state signal because of inevitable delays in their afferent pathways. We have recently reported the effects of transcranial magnetic stimulation (TMS) over the ipsilateral cerebellum as healthy subjects made rapid reaching movements towards visually defined targets (Miall et al PLoS Biology, 5: 2733-2744, 2007). Errors in the initial direction and in the final finger position of this reach-to-target movement were consistent with the reaching movements being planned and initiated from an estimated hand position that was about 138 ms out of date. This interval is consistent with estimates of the delays in sensory motor pathways that would inform the CNS of the peripheral status. We now report new data using the same paradigm, testing the effects of varying the TMS stimulus train from 1, 2 or 3 pulses. We show that the errors in movement are relatively insensitive to the TMS pulse-train duration. The estimated time interval by which the hand position is mislocalised varied by only 12 ms as the TMS train duration increased by 100ms. Thus this interval is likely to reflect physiological processes within the cerebellum, rather than the TMS-stimulus duration. This new evidence supports our earlier claim that the cerebellum is responsible for predictevely updating a central state estimate over an interval of about 120-140 ms. Dysfunction of the cerebellum, whether through disease or experimental procedures, leads to motor errors consistent with a loss of knowledge of the true state of the motor system.

Introduction

Recent work in computational or theoretical motor control has been based around the concepts of internal models [1], optimal control [2] and probability [3]. In essence this broad literature seeks to understand how the central nervous system controls a highly complex non-linear and often unstable biomechanical system comprising the joints and muscles of the body. It must use optimal – or at least sufficient – strategies in the face of uncertainty about the exact properties of the system, due to noise and delay in sensory systems, in central neural pathways and in motor execution. And it is assumed that the CNS maintains information in the form of internal models – neural representations of the properties of the system it is controlling. The cerebellum is thought to be important for learning and storage of these internals models [4–6], and much experimental research has addressed this hypothesis. A particularly powerful idea is that the cerebellum forms a “forward model” – an internal model that generates a prediction of the expected outcome of motor commands [7]. This predictive signal could be used to plan actions, to help control on-going movements, to coordinate two or more simultaneous actions, and to update a central representation of the current state of the body. It is this last hypothetical role that we address in this paper.

The CNS needs an internal “state estimate” because it does not know the current state of the peripheral motor system, because of unavoidable conduction delays in efferent and afferent pathways. Hence sensory afferents are out of date by the time they reach the CNS, while any recent motor commands may not have yet affected the musculature [8]. Sensory-motor delays vary across modality, but may be in the order of 100-300 ms [9–12]. Thus the brain generates an estimate of the true state of the peripheral motor system, integrating incoming sensory information with predictions of the consequences of outgoing motor commands [5;13–15]. Hence state estimation is closely coupled with forward modelling [7]. The cerebellum receives ascending proprioceptive inputs and efferent copies of descending motor commands, and it outputs to cortical and brain stem motor nuclei [4]. It also has the necessary adaptive mechanisms to support this hypothesised role, as forward model predictions must be refined and maintained by experience-based motor learning [4;7]. So to test the role of the cerebellum in these processes, we have recorded the response properties of Purkinje cells in the lateral cerebellum of monkeys performing a visually guided tracking task, and have shown that they encode the direction of movement of a cursor on the screen even when the monkeys’ hand moves in the opposite direction [16]. Thus they appear to code the visual consequences of the actions, as predicted by the forward model hypothesis. Others have reported similar results suggesting the lateral cerebellum encodes the future kinematics of arm movements, and not dynamics [17;18]. The responses of the human cerebellum measured by fMRI also confirm its forward model role in coordinating actions, apparently using predictive information from one effector to help control the motion of another [19;20]. Of course, the cerebellum is interconnected with other brain systems, and the forward model and state estimation processes may be shared with cortical structures including the superior parietal cortex [11;21–23] or elsewhere, depending on task [24].

In a recent paper [25] we reported the effects of brief disruption of the cerebellum during a reach-to-target action. We used transcranial magnetic stimulation (TMS) over the ipsilateral cerebellum during subject’s reaction time to respond to an auditory cue and to point, without vision, to a previously observed target location. We predicted that TMS-induced disruption of the cerebellum would lead to inaccuracies in movement because the state estimation signals would be lost. A rapid reaching action made without accurate knowledge of the current state of the hand (its position and velocity) would overshoot [4], analogous to the hypermetria of cerebellar patients [26]. Moreover, if the reaching action started while the hand was already in motion, the initial direction of the movement towards the target would be inaccurate. For example, during a rightwards movement, plans to move forward to the target should reflect the continuously updated estimate of the hand’s current position. Without an accurate the state estimate, the reach-to-target would be misdirected, reflecting the mislocated starting position of the hand. Both these predictions were borne out – there were small but significant increases in terminal errors when 100 ms of TMS was applied over the cerebellum but not elsewhere, and cerebellar TMS induced movement direction errors when the reaching movement started during a rightwards or leftwards action, but not if the hand was initially stationary [25]. This last result reflects the expected insensitivity of movement direction if the starting posture is known and fixed, as the current state estimate is then no longer time dependent.

However, this work could not resolve an unknown relationship between the fixed duration of the TMS stimulation that we used and our estimate of the functional effect of about 138 ms. It was not clear if the 138 ms merely reflected the TMS duration (100 ms) plus an additional neuronal recovery time (38 ms).

In the present work, we directly test this relationship by repeating the original experiment, but varying the TMS stimulus train from 1 to 3 pulses at 50 ms intervals. In other words, we varied stimulus duration from 0, 50 to 100 ms and tested its effects on terminal errors, on directional errors and on the duration of the functional effect. We hypothesize that the state-estimation interval is unrelated to the TMS duration, and instead reflects the interval over which the cerebellum predictively estimates the current state of the motor apparatus. Our null hypothesis is that the interval is directly related to the duration of the TMS-induced disruption of the cerebellum.

Methods

Thirteen subjects viewed a virtual image of a static target in 3D space ahead of them, and started each trial by lifting their right index finger from a start key and moving slowly and steadily towards their right (Figure 1A). LCD goggles blocked the view of their hand and of the target as soon at the start key was released. An auditory go cue, 500-1500 ms after trial onset, instructed them to make a rapid upward and leftward pointing movement to the virtual target. We refer to this task as “dynamic”. Their index finger had typically moved laterally 10-40 cm from its original position when the go cue was delivered. On a random 50% of the 90 trials in each block, TMS was delivered within their reaction time after the auditory go-cue, as a 1,2 or 3 pulse train of stimuli separated by 50 ms. Vision was allowed after the reach-to-target motion was complete, avoiding any slow drifting of accuracy across trials. In separate blocks, subjects also performed a “static” task. Here, the start button was moved laterally by 20 cm and the subject was instructed to lift the index finger off the button, but to hold stationary until the go cue; they then made a reach to the target from this static, constant, position. For other details of the task, refer to our previous paper [25]. Participants gave informal consent and the experiments was approved by local ethics committee.

Figure 1.

A: The state-estimation task. Subjects made an initial right-ward movement for a randon period until interrupted by an auditory go-cue. They then reached to the virtual target position. TMS (1, 2 or 3-pulses) was delivered in the reaction time of the reach-to-target action. B: Group mean trajectories (n=10 subjects) for TMS trials and non-TMS trials.

Results

Screening

We predict effects on azimuth angles in the dynamic task [25] and so we first screened participants for an effect of TMS on this measure, with a very liberal inclusion criterion of p<0.5. We rejected data from 3 participants who showed no differences between TMS and non-TMS trials, probably because of inter-subject differences in stimulation threshold or in coil-to-cerebellar distance (the coil was positioned relative to the inion). The remaining 10 data sets were then analysed in detail. Group mean trajectories for TMS and non-TMS trials in the dynamic task are shown in Figure 1B. We concentrate in these results only on the dynamic task results. There were no significant differences in terminal errors between the static and dynamic task (ANOVA, F(1,6)=0.35, NS) or for elevation angles (F(1,6)=0.24, NS); nor were the condition by TMS interactions significant. Expected differences in azimuth angle between the two conditions [25] are described below.

Terminal Error

The increase in end point error for TMS trials (Figure 2A) confirmed the original results, with a significant increase in error in all 3 TMS conditions compared to non-TMS trials (t(9)>6.3, p<0.0001, 1-sample t-tests). There was no evidence of an increase in error with the increasing number of TMS pulses, and although the errors were somewhat smaller than those observed with the original, large sample, a comparison between the new and old 3-pulse conditions was not statistically significant (t(40)=1.6, p=0.117).

Figure 2.

A: TMS-induced increase in terminal errors. Each bar is the mean of the within-subject difference in end-error over TMS vs non-TMS trials (+ 1 SEM). 1, 2 or 3 pulse TMS was applied over the cerebellum during rightward movements (Dynamic, n=10) or when stationary (Static, n=10). B: TMS-induced change in azimuth angles and in elevation angles (C). Equivalent data from our original report [25] which also used 3-pulse TMS is included in each panel for comparison (n=32).

Directional error

Azimuth angle

For the dynamic task, measurement of the difference in average direction between the TMS and non-TMS trials showed a significant TMS-induced change in azimuth angles for all 3 TMS conditions (Figure 2B; t(9)>4.8, p<0.001), and again no significant difference between the new and old 3-pulse conditions (t(40)=0.78, p=0.437). As expected, there were highly significant differences between the dynamic and static conditions (F(1,6)=20, p=0.004), as azimuth angles were not significantly different to zero in the static condition, and did not change significantly with increasing TMS number. The interaction between task condition and TMS number was not significant (F(2,12)=2.76, p=0.10).

Elevation angle

There were also significant changes in elevation angles (Figure 2C; t(9)>2.7, p<0.02), and no difference between the new and old groups (t(40)=0.82, p=0.417). There was a trend towards an increase in the angular deviation as the number of TMS pulses increased, although this was not statistically significant in azimuth (F(2,18)=0.398, p=0.68). For elevation, this was significant (F(2,18)=4.39, p=0.028), with a linear increase in elevation error with increasing TMS pulses (F(1,9)=13.1, p=0.006).

State estimation interval

This is determined from the angular deviation in azimuth (Figure 2B). The mean deviation was about 4 degrees clockwise, consistent with a leftwards mis-localization of the hand position (i.e. backwards in time). We can therefore backtrack along the average movement trajectory to find the position at which the start-to-target angle is shifted by the same ~4 degree amount, and use the velocity of the hand to convert this distance to a time interval. These results shown in Figure 3, again in comparison with the previous data [25]. These intervals are smaller than previously seen, although the difference is not statistically significant (t(40)=0.79, p=0.435). The trend for the intervals to increase with increasing number of TMS pulses was not significant (F(2,18)=0.2, p=0.82),

Figure 3.

Estimated state estimation intervals. The TMS-induced changes in azimuth angles (Figure 2B) were converted to a time interval by comparison with the initial velocity of the hand. There are no significant differences between these estimates, although a linear trend is evident. The original estimate of 138 ms is shown for comparison [25]. Black horizontal bars indicate the TMS stimulus duration (0, 50 or 100 ms).

Discussion

These results confirm that TMS over the lateral cerebellum during dynamic actions resulted in a directional deviation of the reaching movement, and in increased positional error. These results confirm previous findings that the effects depend on the task, and TMS in the static condition does not generate significant errors in azimuth. Moreover, they strongly argue that the magnitude of the behavioural effect observed is not simply a reflection of the duration of the TMS train, as there was only a limited sensitivity to 1, 2 or 3 TMS pulses. Of particular importance is the finding that the estimated time by which the hand is mislocalised (Figure 3) varied only 12 ms as the TMS stimulus train increased by 100 ms.

It is not certain what duration of effect a single TMS pulse would have on processing within the cerebellar system. Judging by its influence on primary motor cortex, the effect could be brief: there is a narrow window of 5-10 ms within which motor cortical excitability is reduced after each cerebellar shock [27;28]. However, this indirect measure of cerebello-thalamic-cortical modulation does not indicate how long the cerebellar cortex itself takes to recover, and it is likely to be much longer. Judging by the effects of TMS over visual cortex on image detection [29], or over motor areas on reaction times [30], neocortical recovery takes 30-60 ms after a supra-threshold pulse. Adding ~30 ms to the 1, 2 or 3-pulse trains, we might expect our TMS protocol to disrupt the cerebellar hemisphere for about 30, 80 and 130 ms respectively. The latter is close to the observed interval (Figure 3). But it is also close to the delay of sensory afferents informing the CNS about the previous hand positions [9;10;15]. And since the differences between 1, 2 and 3 pulses were not significant (except in elevation angle) we suggest that the original 138 ms state estimation interval, based on a large sample size, is instead a good measure of the time over which the cerebellum anticipates the state of the ongoing hand movements.

There was however a trend for an increased effect with longer TMS-trains, and the elevation angles were significantly related to the number of TMS pulses. This may indicate temporal summation of the effect, such that the 3-pulse stimulus is more effective in disrupting the cerebellum than a single pulse [31]. Given that even 3-pulse TMS does not completely block processing in the relevant arm areas of the anterior and/or posterior cerebellum, our estimate of ~120-140 ms may be a slight underestimate, but is not likely to be badly wrong.

Another issue is the localization of these TMS effects in the cerebellum. Prior research using TMS to condition the excitability of motor cortex [27;28] is consistent with the brief excitation of the Purkinje cells, leading to cerebellar inhibition. It is also clear that the target site affects the hand area of motor cortex, consistent with activation of lateral cerebellar cortex, probably in lobules V and VI. It is not clear how widespread within neighbouring areas of the cerebellum the effect is.

It also remains for us to test whether the effects we report depend only on the cerebellum, or whether the state estimation process is shared with other areas [32], and whether the cerebellar state estimates are used to update other cortical regions. These will be the aims of further experiments.

Suggested Reviewers.

Response to Reviewers: We thank the reviewer for helpful comments.

>1) Adding to Figure 1 a diagram of the movement without and >with TMS and the error measures would be helpful.

We have added an additional figure (1) to make this clear.

>2) The Static condition is never defined in the paper. >This could be done with one sentence in the Methods.

We have defined this condition in the Methods (yellow highlight sections)

>3) The statistics reported in the Results section do not >clearly state whether they are referring to the Dynamic >condition (presumably this is what is reported). The >statistics for both Dynamic and Static conditions should be >reported. What is not discussed is the obvious decrease in >the azimuth error in the static condition with increasing >number of pulses and whether this is significant.

We have added the required azimuth statistics for the static condition - in fact the trend the reviewer noticed is not statistically reliable. We have now included statistics testing that the static and dynamic conditions do not significantly differ for terminal error or elevation angle.

>4) Finally, adding a few sentences on the question of the >localization of the TMS effect to the cerebellum is >desirable. Undoubtedly, this is a critical issue for a >reader.

Yes, a good point - we have added to the discussion to mention this issue.