Multiple timescales in the adaptation of the rotational VOR

Abstract

Goal-directed movements, such as pointing and saccades, have been shown to share similar neural architectures, in spite of the different neuromuscular systems producing them. Such structure involve an inverse model of the actuator being controlled, which produces the commands innervating the muscles, and a forward model of the actuator, which predicts the sensory consequences of such commands and allows online movement corrections. Recent studies have shown that goal-directed movements also share similar motor-learning and motor-memory mechanisms, which are based on multiple timescales. The hypothesis that also the rotational vestibulo-ocular reflex (rVOR) may be based on a similar architecture has been presented recently. We hypothesize that multiple timescales are the brain's solution to the plasticity-stability dilemma, allowing adaptation to temporary and sudden changes while keeping stable motor-control abilities. If that were the case, then we would also expect the adaptation of reflex movements to follow the same principles. Thus we studied rVOR gain adaptation in eight healthy human subjects using a custom paradigm aimed at investigating the existence of spontaneous recovery, which we considered as the hallmark of multiple timescales in motor learning. Our experimental results show that spontaneous recovery occurred in six of eight subjects. Thus we developed a mathematical model of rVOR adaptation based on two hidden-states processes, which adapts the cerebellar-forward model of the ocular motor plant, and show that it accurately simulates our experimental data on rVOR gain adaptation, whereas a single timescale learning process fails to do so.

research on how the brain produces precise and rapid goal-directed movements, in spite of the variability of both internal and external factors, has made great progress over the past 20 yr. How do we learn to control reaching movements e.g., while turning or under the water? How does the brain cope with muscle fatigue and aging? The neural mechanisms producing movements are constantly monitored in terms of their performance; motor learning is the process that tunes their synaptic weights to keep actions quick and accurate. A key concept in the current understanding of motor-control principles is that of an internal model, i.e., a neural mechanism that can mimic the input-output characteristics of an actuator, or its inverse (Kawato 1999). Indeed, several studies have shown that goal-directed movements, such as pointing and reaching movements or saccades, share a similar control structure based on the involvement of at least two internal models: an inverse internal model of the actuator being controlled (Shadmehr and Mussa-Ivaldi 1994; Shidara et al. 1993), which produces the appropriate commands once the motor plan has been chosen, and a forward internal model, predicting the sensory consequences of such commands before the availability of sensory feedback and providing the basis for state estimation (Jordan and Rumelhart 1992; Wolpert and Miall 1996). Such forward models were hypothesized to lie in the cerebellar cortex (Miall et al. 1993) and to be fed by an efference copy of the motor command, i.e., the output of the inverse model, to predict the sensory consequences of a movement before sensory feedback is available and thus allow its online correction in case of a discrepancy with the desired movement. These theories, although mainly developed in studies of upper-limb movements in time-varying force fields, found neurophysiological grounds only when considered in the domain of eye movements (Green and Angelaki 2010; Lisberger 2009).

Indeed, although the term “internal model” had not yet been introduced, studies on the vestibulo-ocular reflex (VOR) frequency response led Robinson (1981) and his colleague Skavenski (1973) to the hypothesis that the sum of weighted direct and integrator pathways acts as an inverse model of the eye-plant dynamics and is used to transform the desired action into the corresponding motor command. Recent neurophysiological studies have shown that burst-tonic/tonic neurons in the nucleus prepositus hypoglossi (commonly referred to as PH-BT cells) encode and relay an efference copy of the motor command, i.e., the output of the inverse model (Green et al. 2007), thus confirming the early hypotheses. Furthermore, the existence of a forward model in a cerebellar-feedback loop was suggested by Robinson and colleagues (Zee et al. 1976) in the context of the saccadic mechanism: the ability, found in two patients with spinocerebellar degeneration showing slow saccades, to reverse these saccades in midflight led to hypothesizing the existence of a local-feedback loop in the cerebellum (Zee et al. 1976), later perfected by Jürgens and colleagues (1981) with the concept of a resettable integrator. This integrator is fed with an efference copy of the command to the ocular motor plant and produces an internal estimate of eye displacement, allowing saccades to stop precisely on target, despite that they are usually over by the time visual feedback is available [see Ramat et al. (2007) for a review]. Several recent studies have explicitly hypothesized the existence of a forward model of the ocular motor plant in the cerebellar flocculus (Glasauer 2003; Green and Angelaki 2010; Green et al. 2007; Lisberger 2009), emphasizing that the organization of the vestibulo-ocular system may be similar to that proposed for the control of goal-directed movements.

More recently, several studies on motor adaptation in response to external perturbations, such as force fields affecting arm movements or target shifts in the context of saccadic eye movements, have shown that motor memory of goal-directed movements evolves through at least two processes with different timescales, establishing a continuum between short-term, i.e., in the order of hours or less (Ethier et al. 2008; Kojima et al. 2004; Smith et al. 2006), and long-term, i.e., in the order of days (Kording et al. 2007; Robinson et al. 2006), adaptation. The model proposed by Smith and colleagues (2006) is based on two hidden states: a fast one that learns quickly from motor error but has poor retention and a slow one that learns slowly but has stronger retention. Such an approach can account for different adaptation phenomena, such as saving, i.e., a faster relearning of the same task after the first adaptation, and spontaneous recovery, i.e., the re-emergence of a previously learned, adapted state after its extinction (Robbins 1990). Motor learning probably includes several timescales, with time constants ranging from minutes to days and longer (Wong and Shelhamer 2011a), which explains the way the brain learns new behaviors and calibrates reflexive movements. Multiple timescales thus appear to be a constant characteristic of adaptive processes involving voluntary movements. Our study thus aimed at investigating whether the VOR shares such learning principles with goal-directed movements to understand whether the hypothesis that the rotational VOR (rVOR) shares the organization of volitional movements (Green and Angelaki 2010) also extends to sharing similar adaptation processes.

Despite several studies that account for adaptation in saccadic and in reaching movements with multiple timescales of memory, little is known about these processes in the adaptation of reflex movements in humans, such as the rVOR. The existence of multiple plasticity mechanisms in the adaptation of the horizontal rVOR was shown in mice by testing the interaction (Boyden and Raymond 2003) and generalization (Kimpo et al. 2005) of gain-increase and gain-decrease adaptation.

The VOR is a phylogenetically old reflex that uses head velocity information transduced by the semicircular canals to drive the movement of the eyes to maintain a stable gaze and hence, clear vision during head movements. Its neural circuitry comprises a classic, three-neuron arc with vestibular afferents projecting to the vestibular nuclei, which in turn, projects to the abducens nuclei innervating the lateral recti, and a side circuit involving the cerebellum, receiving input from mossy fibers and exerting an inhibitory action on floccular target neurons in the vestibular nuclei, which in turn, projects to ocular motor neurons (Lisberger et al. 1994) through Purkinje cells. The VOR is a very short latency (7–10 ms) reflex, a characteristic that makes it the best tool available to the brain to respond to head-movement perturbations. Such timing, nonetheless, is too brief to use visual feedback to correct for errors in performance, at least during the first 100 ms after the beginning of the head movement (Carl and Gellman 1987; Lisberger and Westbrook 1985; Waespe and Henn 1987). Thus the VOR operates as an open-loop control system, at least during such initial part of the response, so that adaptive processes are required to keep the VOR calibrated consistently.

In humans, a number of rVOR adaptation studies [see review by Schubert and Zee (2010)] have shown the ability to recalibrate the reflex using vision, i.e., by coupling head motion with target motion to generate a velocity error signal (retinal slip). Retinal slip is a strong adaptive stimulus, demonstrated, for instance, by the pioneering studies on long-term adaptation by Gauthier and Robinson (1975), who used magnifying lenses, and those by Gonshor and Jones (1976), who used inverting prisms. However, other error signals, even discrete ones, such as position errors (Eggers et al. 2003; Scherer and Schubert 2010), can also be used to induce adaptation of the rVOR.

Traditionally, rVOR gain-adaptation studies have been performed during passive, low-frequency, sinusoidal head rotations. On the other hand, recent human studies have explored gain adaptation of the rVOR during both self-generated and manually delivered, passive, head-only impulsive rotations (head impulses or thrusts)—a more physiological stimulus than single-frequency sinusoids. These studies have shown that the rVOR can be adapted efficiently using a retinal velocity error (retinal slip) signal with a gradually increasing adaptive demand (Migliaccio and Schubert 2013; Schubert et al. 2008). Similarly, Zhou and colleagues (2002, 2003) showed the efficacy of brief, passive head translations using either retinal slip or discrete position errors toward the adaptation of translational VOR in monkeys.

Being easily adaptable, the VOR has been long considered as a model system for motor adaptation, and the first hypotheses on the involvement of cerebellar control in the long-term adaptation of the rVOR date back to the early 1970s [see Ito (2002)]. The cerebellar flocculus is necessary for learning in gain adaptation of the rVOR (Ito et al. 1974; Robinson 1976). and recent evidence in cats and monkeys has shown that early (short-term) and late (long-term) motor memories are stored at different loci: the floccular complex and the brain stem nuclei, respectively (Anzai et al. 2010; Kassardjian et al. 2005). A second site of plasticity in the brain stem, besides the cerebellar cortex, also allows high-frequency, continuous learning in the presence of visual-feedback delay (Porrill and Dean 2007).

The main purpose of this study was to understand whether multiple timescales also exist in human rVOR adaptation and thereby, provide further evidence to support the hypothesis that reflex movements are not only controlled by the brain using the same approach as with goal-directed movements but also that their learning mechanism is similar. Our hypothesis is that the multiple timescales of memory, observed in the adaptation of goal-directed movements, represent the brain's solution to the plasticity-stability dilemma (Abraham and Robins 2005), i.e., the need for a tradeoff faced by all learning systems that require both stability to retain information and plasticity for new learning within the network. This hypothesis would likely imply that the brain also uses the same approach for other motor-learning needs, such as the tuning of reflex movements. Therefore, we chose to study whether short-term, gain-down rVOR adaptation, followed by a shorter, gain-up de-adaptation, would give rise to spontaneous recovery when movements are performed without visual feedback. Thus we used catch (CH) trials, in which the visual stimulus is turned off as soon as the movement starts to observe whether rVOR gain reverted to a previously learned, low-gain state. We then modeled motor memory in the VOR as a two hidden-states model, learning the parameters of the cerebellar-forward model of the ocular motor plant, and studied whether it could explain our experimental data, also compared with a single hidden-state model.

MATERIALS AND METHODS

Subjects and Experimental Paradigm

Subjects.

We recorded eight healthy human subjects (seven men and one woman; mean age 37; SD 9.8) at the Department of Neurology, University Hospital Zurich (Zurich, Switzerland). Participation in this study was voluntary, and all subjects signed a written consent form before taking part in the study. The experimental protocol was approved by the local Ethics Committee.

Vestibular stimulation.

Rotational stimuli were delivered in complete darkness by a three-axis rotational stimulator, driven by three servo-controlled, motorized axes (Acutronic, Bubikon, Switzerland), controlled with Acutrol software and hardware (Acutrol, Huntsville, AL), and interfaced with LabVIEW software (National Instruments, Austin, TX). Subjects were seated comfortably in a chair so that the center of the head was positioned at the center of rotations to obtain purely rotational stimuli. Individually adjusted masks, made of a thermoplastic material, were molded to the contour of the head after warming, with openings in the mask made for the eyes, video-oculography system, and mouth. The mask was attached to the back of the chair and restricted head movements very effectively without causing discomfort. Rotational stimuli consisted of 120 ms constant acceleration at 460°/s², immediately followed by a symmetric deceleration interval, resulting in a peak head velocity of ∼100°/s and an overall rotation of ∼20° (Fig. 1D). Subjects were instructed to hold their gaze on a projected laser dot or during CH, on the location at which they remembered the laser dot. Each stimulus lasted 5.42 s: 3 s of rest [comprised of 1 s in darkness; a 1.5-s flash of a lateral, white light-emitting diode (LED) lamp, aimed at the eyes of the subject to limit dark adaptation; and 0.5 s with the laser on]; 0.42 s of chair rotation; and 2 s to reposition the chair and bring it back to the start position.

Fig. 1.

A: schematic representation of the visual stimulus during head rotation. In the space-fixed (SF) trial stimulus, the dot remains fixed in space for the entire rotation duration; in the catch (CH) trial stimulus, the target turns off when the chair starts rotating; and in the adaptation-stimulus (AS) trials, the dot moves during the rotation with a percentage of the chair velocity, depending on the planned adaptation requirement. B: experimental paradigm: 20 SF and 20 CH stimuli to assess basal condition; 150 AS trials of gradual gain-down adaptation; 50 AS trials of constant gain-down adaptation; 70 AS trials of gradual gain-up adaptation; 80 CH trials. C: block diagram of the rotational vestibulo-ocular reflex (rVOR) model and its motor-learning mechanism (gray lines). Head velocity estimated by the vestibular nuclei (VN) is conveyed to motor neurons, directly and indirectly, via a neural integrator, forming an inverse model of the ocular motor plant; an efference copy of the innervation sent to the ocular motor nuclei is conveyed from the output of the inverse model to the forward model in the floccular-feedback loop to estimate the resulting eye velocity. The latter is then compared with the desired eye velocity to compute the error and refine the motor command. The movement of the target causes retinal slip, and such error is assigned, in part, to an inaccurate eye movement (a motor error) and, in part, to world disturbances (an externally caused error) by the Credit Assignment block. Only the part credited to a motor error leads to the recalibration of the forward model of the eye plant. D: head velocity (vel; black solid line) and target velocity (gray dotted line) during chair rotation.

Laser projection.

We used a red laser and real-time, two-dimensional, mirror-deflection system for displaying a 2-cm diameter visual target on a curved isovergence projection screen at 140 cm in front of the subject. Laser position was controlled using the online horizontal position of the chair.

We considered three types of trials: 1) space-fixed (SF) trials aimed at assessing rVOR in normal visuo-vestibular interaction conditions, 2) CH for assessing rVOR gain in the absence of visual feedback, and 3) adaptation-stimulus (AS) trials for inducing gain adaptation. During these different types of trials, the visual stimulus behaved as described below.

Space fixed.

The laser dot turned on 0.5 s before the beginning of chair rotation in the straight-ahead position and remained fixed in space for the entire rotation duration, asking for an ideal rVOR gain of ∼1 (Fig. 1A).

Catch trial.

The laser dot turned on 0.5 s before chair rotation in the straight-ahead position, turned off when the chair started rotating, and stayed off until the following trial. Thus this condition provided no error feedback, with the following visual target being displayed only after chair repositioning (Fig. 1A).

Adaptation stimulus.

The laser dot turned on 0.5 s before chair rotation in the straight-ahead position and moved during the rotation with a percentage of the chair velocity, depending on the planned adaptation requirement (Fig. 1D). If the target moved in the direction of head rotation, then its fixation requires rVOR gain reduction, whereas a movement in the opposite direction asks for rVOR gain enhancement (Fig. 1A). In this study, we induced short-term adaptation of the rVOR using a gradually increasing target movement and hence, an rVOR gain-adaptation request (Schubert et al. 2008). Such an incremental adaptation approach was chosen both to reduce the chance of the subjects becoming aware of the target-movement manipulation and to adopt a paradigm that would more likely induce adaptation of the reflex (Kagerer 1997; Schubert et al. 2008). Thus during such adaptation trials, the visual stimulus behavior, i.e., the coefficient determining the laser movement as a fraction of the head movement, was changed gradually and progressively to induce an increasing retinal slip either for gain reduction or for gain increase.

Experimental paradigm.

The overall experimental paradigm (shown in Fig. 1B) was then organized as follows: rVOR gain was first assessed with a block of 20 SF trials (i.e., rVOR gain request of one), followed by a block of 20 CH trials. These were followed by one block of 150 AS trials, incrementally asking for a decrease in rVOR gain from 1.0 to 0.6; i.e., the target movement was a fraction of the chair movement that increased by ∼0.0027 (0.27%) on each trial and a last block of 50 trials constantly asking for a gain of 0.6 (40% reduction; Fig. 2A). Subjects then underwent a block of 70 AS trials of reversed, i.e., gain-up, incremental adaptation, asking for an increase in rVOR from 1.0 to 1.2; i.e., the target moved opposite of the chair, increasing its velocity by ∼0.0028 (0.28%) of chair velocity at each trial. A final block of 80 CH trials was then supplied to evaluate whether spontaneous recovery of motor memory occurred.

Fig. 2.

A: adaptation ratio (AR) for all subjects (gray points); black solid line represents the mean AR, averaged over all subjects at each trial; dashed line represents adaptation request. The labels indicate the groups of movements used in statistical analysis and in E: (1) 1st 10 SF; (2) last 10 AS gain down; (3) 1st 10 AS gain up; (4) last 10 AS gain up; (5) from 41 to 50 CH. B: AR for 1 representative subject (S3). pert, perturbation. C and D: comparison of CH trial recorded during the main experiment and in the control experiment. C: mean data over all subjects in the main and the control (ctrl) experiments and linear regression of the 1st 60 CH. D: each data point represents the mean, together with the SD of the 6 values obtained by averaging 10 consecutive movements for each subject. The 1st 2 values refer to SF trials; the last 8 to CH trials. One-way ANOVA does not reveal any significant difference for CH control (gray line). CH after adaptation (post; black line) are significantly lower than SF, and post hoc analysis reveals a statistically significant difference between the 1st bin of CH (group 3) and the last 4 bins (P < 0.001 for bins 7–9; P < 0.05 for bin 10). E: AR for the movements labeled in A. Bar plots represent the mean and SD of AR for each subject in each condition.

Control paradigm.

After at least 1 mo, six of the original eight subjects were recorded again in a protocol consisting in 20 SF trials and 80 CH trials, without previous adaptation stimuli. Such paradigm was aimed at verifying whether the block of CH, per se, induced any rVOR gain reduction.

Data Acquisition and Analysis

Data acquisition.

Horizontal and vertical right eye position was measured using the EyeSeeCam infrared video system (sampling rate, 220 Hz; EyeSeeTec GmbH, Fürstenfeldbruck, Germany). Head angular velocity was measured using the EyeSeeCam three-axis inertial measurement unit sensor. Target position and chair position signals were recorded at 220 Hz with 16 bits resolution through an NI USB-6211 data acquisition system (National Instruments). All data were acquired and synchronized by the EyeSeeCam system software and saved in Matlab format (MathWorks, Natick, MA) for offline analysis. A 15-point grid (from −20° to +20° with a 10° step on the horizontal axis; from −10° to +10° with a 10° step on the vertical axis), shown on the isovergence projection screen, was used to calibrate eye position. To limit dark adaptation and avoid the risk of losing the tracking of the eye due to excessive pupil dilation, the recorded eye was briefly (1.5 s) illuminated using a white light from a battery of LEDs, lying outside of the subject's field of view, at the beginning of each trial. Such technique was efficient in limiting the size of the pupil during the head movement and in allowing its proper tracking.

Data analysis.

Data were analyzed offline using Matlab (MathWorks) custom-developed software (Colagiorgio et al. 2013). Raw eye position was calibrated using a second-order polynomial fit and filtered using a second-order Butterworth low-pass filter (30 Hz cutoff frequency). Eye velocity data were calculated using the Savitzky-Golay four-point derivative filter (Ramat et al. 1999). Onset of each head movement was found using a fitting curve composed of a constant value and a quadratic function, and all trials were aligned based on such instant in time (Ramat and Zee 2003).

We have chosen to analyze the gain of rVOR responses, ∼100 ms from head onset, thus measuring the open-loop response of the rVOR before the availability of visual feedback. Any trials in which the subject broke fixation or made a saccade within the first 120 ms from the onset of the head movement were manually rejected.

The effect of motor learning was evaluated by computing the adjustment of eye velocity in terms of adaptive change [adaptation ratio (AR)] with respect to the basal SF trials. AR of the n^th trial was thus computed as follows

$$AR\left(n\right)=\frac{VM\left(n\right)-V{M}_p}{V{M}_p}$$ (1)

where VM is the mean eye velocity computed between 80 and 120 ms (Carl and Gellman 1987; Lisberger and Westbrook 1985; Waespe and Henn 1987) from head-movement onset, and VM_p is the mean VM assessed in the initial 20 SF trials.

Statistical analysis.

Student's t-test was used to assess the difference between the initial 10 SF and the last 10 movements of AS to test adaptation, the difference between the initial 10 SF and the first 10 movements of reversal AS to test retention, and the difference between the first 10 reversal AS and the last 10 movements of reversal AS to test de-adaptation (all described in Fig. 2A).

To assess spontaneous recovery, we analyzed the trend of AR data relative to the initial 20 SF and the final set of 80 CH trials of the main and the control experiments (Fig. 2, C and D). Data were normalized with respect to the initial 20 SF trials, and then, for each subject, we binned consecutive movements in groups of 10, thus obtaining 10 average values (first two for SF trials, last eight for CH trials; Fig. 2D). The bins were then compared with a repeated-measures ANOVA, followed by a multiple comparison correction using Tukey's least-significant difference procedure. In analyzing the main experiment data, we therefore obtained eight mean values for each bin, corresponding to the eight participants. The comparison of the main and control experiments considered six values for each bin, corresponding to the six subjects who performed both experiments.

A further analysis assessed the gain trend within the first 40 CH performed at the end of adaptation and in the control experiment by testing the correlation of AR against the trial number (Fig. 2C). The significance of such correlation is reported as an indicator of its reliability. We chose to compare the trend in the first 40 trials of the control and the adaptation experiments, since the model predicts no gain decay for the control experiment and a decay of the free response of the fast state on our mean data in ∼35 trials [5 × fast time constant (τ_f); see results] for the adaptation experiment.

For the same reason, to evaluate spontaneous recovery within each subject (Student's t-test), the adaptation level was assessed between trials 41 and 50 of the CH paradigm and compared with the last 10 movements of reversal AS (both in Fig. 2A).

Mathematical modeling.

Semicircular canal afferents encode head velocity signals representing the vestibular contribution to the desired compensatory eye-movement response to head rotation; velocity signals, produced by the vestibular nuclei responsible for multisensory integration [see Cullen (2012) for a review] and velocity storage (Raphan et al. 1979; Robinson 1977), are conveyed to motoneurons both directly and indirectly via a neural integrator. The parallel pathway model (Robinson 1981; Skavenski and Robinson 1973) represents an implementation of an inverse dynamic model of the eye-plant dynamics. A forward model of the same motor dynamics, possibly lying in the cerebellar flocculus, could be used to compute the predicted sensory consequences of the motor command. The estimated eye velocity may be used to compute the error between predicted and desired action and such error exploited in a feedback loop to refine the motor command (Dean et al. 2010; Ghasia et al. 2008; Glasauer 2003; Green and Angelaki 2010; Green et al. 2007; Porrill et al. 2013).

The schematic in Fig. 1C shows the proposed rVOR model, based on that proposed by Glasauer (2003) and combining it with the proposed motor-learning mechanism. We modeled the oculomotor plant (the relationship between eye velocity Ė and motoneuron rate M) with a second-order linear system with time constants of 224 and 13 ms (Keller 1973)

$$\frac{\dot{E}\left(S\right)}{M\left(S\right)}=\frac{s}{\left(1+0.224s\right)\left(1+0.013s\right)};$$ (2)

We converted the transfer function to its state-space form

$$\dot{x}\left(t\right)=Ax\left(t\right)+Bu\left(t\right);$$ (3)

$$y=Cx\left(t\right).$$ (4)

$$A=\left[\begin{array}{cc}-81.72& -344.82\\ 1& 0\end{array}\right];\hspace{0.17em}B=\left[\begin{array}{c}1\\ 0\end{array}\right];\hspace{0.17em}C=\left[\begin{array}{cc}344.82& 0\end{array}\right];\hspace{0.17em}D=0.$$ (5)

The plant is thus a second-order system so that matrices A, B, and C are 2 × 2, 2 × 1, and 1 × 2, respectively.

To obtain an inverse model of the simplified, first-order eye plant, which produces the motor command u(t), the gain of the parallel pathway was set to 0.224 s, equal to the dominant eye-plant time constant and thus effectively canceling its pole in the overall transfer function of the final ocular motor pathway (Skavenski and Robinson 1973).

The cerebellar-forward model reproduces the dynamics of the ocular motor plant and thus predicts the state of the eye, i.e., its velocity, based on the previous state and the efference copy of the motor command u(t)

$$\widehat{x}\left(t+1\right)=\widehat{A}\widehat{x}\left(t\right)+\widehat{B}u\left(t\right).$$ (6)

$$\widehat{y}\left(t\right)=\widehat{C}\widehat{x}\left(t\right).$$ (7)

where Â, B̂, and Ĉ are estimates of matrices A, B, and C in Eqs. 3 and 4. The cerebellar estimate of eye velocity is compared with the desired eye velocity carried by the mossy fibers to estimate an error in the ongoing movement, which is used in a feedback loop to correct the motor command.

We then transformed all model transfer functions (forward and inverse model and plant) to discrete time using a time step of 4.5 ms, corresponding to the sampling time of the data acquired through the EyeSeeCam system.

Sensory feedback is computed at the retina in the form of retinal slip (err): the velocity with which the image of the visual target moves on the retina, quickly deteriorating vision, i.e., err = T_v − H_v − E_v + b, where H_v, E_v, and T_v are the mean over the interval between 80 and 120 ms from chair movement onset of head velocity, eye velocity, and target velocity, respectively, and b represents a bias, that is, the portion of head velocity, measured over the same interval, that is not compensated by the VOR during the initial SF trials (Aw et al. 1996). Retinal slip is a strong, adaptive stimulus, which induces the brain to modify its motor response to cancel it, and has been used frequently for inducing rVOR adaptation (Gauthier and Robinson 1975; Gonshor and Jones 1976). Recent studies (Chen-Harris et al. 2008; Kluzik et al. 2008; Wei and Körding 2009) suggest that depending on the size of the error information, the nervous system assigns the potential cause of an observed error, in part, to a motor error, i.e., an inaccurate eye movement, and, in part, to external disturbances; i.e., some change occurred in the external world. This process tends to assign small errors as mainly due to internal causes, i.e., motor error, and as the size of the error increases, an increasing portion of it is assigned to external causes. Thus the brain estimates the relevance of the error in its motor performance and adapts strongly only to errors due to motor causes (e_m). We modeled this credit-assignment (Fig. 1C) problem with a normalized probability of relevance varying with the size of error: the value is 1; i.e., the error is assigned entirely to motor causes for 0°/s of retinal slip disturbance, and it decreases with increasing error size. Given a sensory error err, the estimate of the corresponding motor error was thus given by

$$e_m\left(n\right)=e^{-\frac{{\mathit{e}\mathit{r}\mathit{r}\mathit{\left(}\mathit{n}\mathit{\right)}}^2}{2{\sigma }^2}}$$ (8)

where σ is a parameter to be estimated based on experimental data, and n indicates the n^th trial. When a motor error occurs, the brain estimates that the forward model is not accurately reproducing the dynamics of the eye plant. Therefore, during adaptation, the motor-learning process responds to motor error by updating the estimate B̂ modulating the effect of the efference copy of the motor command u(t) in the forward model, following the rule, B̂(n + 1) = B̂(n) + δ(n + 1), where δ is the learning parameter computed by motor learning. The latter was modeled as a two hidden-states linear system (Smith et al. 2006), with one state representing a fast process that adapts quickly but has poor retention and one representing a slow process that learns slowly but has better retention. Each state was modeled with a linear differential equation with a learning term and a forgetting term. The update equations were therefore the following

$$\delta \left(n\right)={\delta }_f\left(n\right)+{\delta }_s\left(n\right);$$ (9)

$${\delta }_f\left(n+1\right)=a_f\ast {\delta }_f\left(n\right)+b_f\ast e_m\left(n\right);$$ (10)

$${\delta }_s\left(n+1\right)=a_s\ast {\delta }_s\left(n\right)+b_s\ast e_m\left(n\right).$$ (11)

The learning rates for the fast and slow states (δ_f and δ_s, respectively) are 1 > b_f > b_s > 0, and their forgetting rates are 1 > a_s > a_f > 0.

We then used the model to study the predicted changes in eye velocity caused by the adaptation of the forward model parameters following an approach similar to that used in the context of saccadic adaptation (Chen-Harris et al. 2008; Ethier et al. 2008). We considered that at the beginning of the experiments, the system is well calibrated, and  = A, B̂ = B, and Ĉ = C; i.e., the forward model is a perfect replica of the plant. The learning and forgetting parameters a_s, a_f, b_s, and b_f, as well as the σ parameter, which determined the coefficient of the credit-assignment problem, were then estimated by minimizing the root mean square error between experimental and simulated VM data using a nonlinear least-squares solver (Matlab).

The goodness of fit of the model simulations was evaluated using the R² coefficient of determination. Based on the R² statistics, we compared these results with those obtained by following the same fitting procedures and credit-assignment function yet using a single hidden-state model for determining the update of B̂ in the cerebellar-forward model of the plant. To assess whether the expected increase in goodness of fit related to the use of more model parameters reflected overfitting or a more appropriate model for explaining the data, we compared the two models using the Bayesian information criterion (BIC) statistics (Schwarz 1978), which considering that our models were fitted using least squares, can be computed as (Burnham and Anderson 2002)

$$BIC=n\cdot \ln \left({\sigma }_e^{\hat{2}}\right)+k\cdot \ln \left(n\right);$$ (12)

where n is the number of data points (350 in our case), k is the number of parameters in the model, i.e., three for the single hidden-state model (a and b in the state-space equation and σ in the credit-assignment function) and five for the two hidden-states model (a_s, a_f, b_s, b_f, and σ), and ${\sigma }_e^{\hat{2}}$ is the estimate of the variance of the error, computed as the sum of squared errors (residuals) normalized by the number of samples. With the use of such criterion, which penalizes the increase in the number of variables more than the Akaike information criterion does, models with lower BIC are preferable for explaining the observed data.

RESULTS

Time Course of Adaptation

We induced rVOR adaptation during passive, impulsive, whole-body rotations (at constant acceleration) in eight healthy human subjects, using an incremental velocity error stimulus. As described, we used five sets of stimuli in our experiment: SF trials, assessing the baseline rVOR gain in natural conditions, followed by CH to assess gain in the absence of visual feedback. A set of four blocks of AS trials, inducing gain-down rVOR adaptation, was followed by a set of reversal AS trials aimed at rVOR gain-up adaptation. In the fifth set of trials, we assessed the state of the rVOR gain without introducing visual endpoint error feedback using CH trials. The effect and the course of motor learning were evaluated by computing the change in VM between 80 and 120 ms from head-movement onset with respect to the baseline response assessed in the SF trials (see materials and methods).

Figure 2A shows the evolution during the entire experiment of the mean AR (Eq. 1), computed over all subjects on a trial-by-trial basis, whereas Fig. 2B shows the AR evolution for a representative subject (S3). Figure 2E shows, for each tested subject, the mean and SD of AR, as assessed during the various phases of the experiment, shown in Fig. 2A. Statistical analysis on adaptation data was thus performed on each subject based on the Student's t-test, comparing experiment phases using the same data shown in Fig. 2E.

Statistical analysis provided the following results: 1) all subjects had a consistent AR reduction (mean −28.1%, SD 1%, for a final request of −40%) at the end of the gain-down adaptation (P < 0.001) AS trials [difference between (1) and (2) in Fig. 2. A and E], showing the efficacy of the designed gain-down adaptation paradigm; 2) to test adaptation retention, we assessed the level of adaptation at the beginning (first 10 trials) of the reversal gain-up adaptation block, during which the target movement went from 0 to 2.7% of chair movement and was thus almost fixed in space. AR resulted significantly smaller compared with initial SF trials (mean −11%, SD 2%, P < 0.001) in all subjects [difference between (1) and (3) in Fig. 2, A and E]; 3) reversed, gain-up adaptation was successful in inducing an increase of rVOR gain, as six subjects showed a significant increase in AR in the last 10 trials of reversal adaptation with respect to the beginning 10 trials (mean +7%, SD 3%, P < 0.05), whereas S4 and S7 showed no statistically significant AR changes [difference between (3) and (4) in Fig. 2, A and E].

Spontaneous Recovery

The final set of 80 CH trials was then analyzed to investigate the evolution of eye velocity in the absence of visual feedback and assess whether spontaneous recovery toward the previously learned, low-gain state occurred in our subjects.

We performed an analysis on individual subjects that underwent the main experiment (eight subjects), comparing AR between phases (4) and (5) in Fig. 2, A and E: six subjects showed a significant AR reduction (P < 0.05), consistent with the occurrence of spontaneous recovery, whereas S4 and S6 showed no statistically significant changes.

We then compared the trend of AR in the main and the control experiments (for the six subjects that performed the control experiment) using repeated-measures ANOVA (over each of the 10 bins in Fig. 2D): 1) in the main experiment (Fig. 2D), the test rejected the null hypothesis that all bins were equal (P < 0.001, both considering the eight subjects that performed the main experiment and limiting the subjects to the six who performed the control); multiple comparison post hoc test showed that AR values in the first bin of CH (bin 3 in Fig. 2D) were significantly lower than those during SF (bins 1 and 2 in Fig. 2D; P < 0.001). Furthermore, AR values in the first bin of CH (bin 3) were significantly higher than during the last four bins (bins 7–10 in Fig. 2D; P < 0.05); 2) in the control experiment, (Fig. 2D), the test did not reject the null hypothesis that all bins were equal (P = 0.9).

A further test showed that the correlation of AR with trial number in the first 40 CH (averaged over all subjects) was not statistically significant for the control experiment (P = 0.11), whereas a highly significant, negative correlation was found for the CH recorded after adaptation (P < <0.001; Fig. 2C).

Both tests argue that in the main experiment, CH trials showed a decreasing trend toward the low-gain state, acquired during the first gain-down adaptation (i.e., arguing for spontaneous recovery), whereas in the control experiment, CH trials did not show any trend.

Multiple States of Motor-Memory and Forward-Model Adaptation

We tried to explain the changes in eye velocity during rVOR adaptation in terms of recalibration of the cerebellar-forward model of the eye plant, which by predicting the sensory consequences of the movement along the floccular-feedback loop (Fig. 1C) allows online movement correction. The motor-memory process, controlling the adaptation of the estimated eye-plant dynamics in the forward model, was presented as a linear state-space model (Eqs. 9–11), supported by two hidden states (Chen-Harris et al. 2008; Ethier et al. 2008; Smith et al. 2006). The states adapt as a function of their value in the previous trial (coefficient a_s, a_f) and of the motor error observed on each trial (coefficient b_s, b_f).

Retinal slip (target velocity-gaze velocity) was assigned to two different sources, based on its size: motor error or external causes, such as target displacement. Only the part assigned to a motor error leads to a recalibration of the rVOR gain. The credit assignment was ruled by Gaussian functions, such as that shown in Fig. 3G, where the results of fitting Eq. 8 to the overall mean data (Eq. 8: σ = 6.9) and to individual subjects are presented. In our experiment, the error is induced gradually: at the start of training, when the error is relatively small, the model attributes the whole error to motor causes, and as training proceeds, and the error becomes larger, the model progressively assigns part of the error to external causes, e.g., a target displacement (Fig. 3B).

Fig. 3.

Model predictions. A: mean eye velocity between 80 and 120 ms from head-movement onset (VM) for the simulated movement (black solid line) and recorded data averaged on all subjects (gray data points). Goodness of fit: R² = 0.83. B: error magnitude (retinal slip velocity: difference between the required and the recorded eye velocity, err) and motor error (black line, e_m). C and D: model fitting on 2 representative subjects. E: final CH trials (gray dash-dotted line), model prediction (solid black line), and linear fit of the 1st 40 trials (dashed black line). F: adaptation is the result of the recalibration of the forward model of the eye plant by changing the value of the coefficient B̂ (black dash-dotted line) that increases or decreases the effect of the motor command. The update of the parameter was modeled with 1 fast process (δ_f, black dashed line) that adapts quickly but has poor retention and 1 slow process (δ_s, gray solid line) that learns slowly but has better retention. The sum of the 2 states (δ) is the black solid line. G: credit assignment function for individual subjects (gray lines) and for mean data simulations (black line). Smaller errors are more likely to be assigned to motor error, i.e., function values close to 1, due to a noncalibrated-forward model of the eye plant. H: Single hidden-state model fit on average data.

At the beginning of adaptation, the fast state quickly responds to error (Eqs. 10 and 11: b_f = 0.0085; b_s = 0.0009; sensitivity to error b_f/b_s∼ 10), but at the end of the gain reduction-adaptation trials, most of the motor adaptation is due to the slow state (δ_s; Fig. 3F). In the reverse adaptation period, the errors reverse sign and are suddenly large, so that in spite of credit assignment, the large motor error produces a rapid change in the fast state (δ_f; Fig. 3F) and a slower change in the slow state. The reverse adaptation block is followed by a set of CH, during which the fast state rapidly returns to zero, whereas the slow state declines more slowly. In this condition, the system is autonomous; i.e., it is not driven by an input e_m(n), and changes in the memory states are only due to the forgetting terms (Eqs. 10 and 11: a_s = 0.999; a_f = 0.85). The different time constants governing the retention of the slow (τ_s = 1,000 trials; i.e., ∼83 min) and fast (τ_f ∼ 7 trials; i.e., ∼35 s) states thus result in the model prediction of spontaneous recovery of the previously learned behavior, as illustrated in Fig. 3, A and E. The eye velocity trajectories simulated by the model in the different stages of the experiment are also reported in Fig. 4, A–D, showing good agreement with the recorded data. Note that model simulations of both the mean data and that of individual subjects clearly predict spontaneous recovery, correctly following the trend of the experimental data, in spite of the fact that the eye velocity predicted at the end of reversed adaptation is still lower than that at the initial, nonadapted state.

Fig. 4.

A–D: black lines are model-simulated eye velocities, and gray lines recorded movements of 1 representative subject (S3). Dashed lines represent the rVOR outside of the open loop response. A: initial CH and 1 simulation. B: recorded movements during gain-down adaptation and simulations at the beginning and at the end of adaptation. Only after at least 200 ms, at or after peak head velocity, covert saccades are generated to regain fixation. C: recorded movements during gain-up adaptation and simulations at the beginning and at the end of adaptation. D: final CH and 1 simulation. E–H: detail of the data recorded during the 1st 200 ms, corresponding to A–D, respectively.

As a quantitative test for the existence of multiple timescales in rVOR adaptation, we set out to verify how the alternative hypothesis of a single timescale process governing rVOR gain adaptation would explain the experimental data. R² goodness-of-fit results for each subject and for the mean over all subjects are compared for the two models in Table 1. Values for the two hidden-states models are significantly higher than those for the one-state model (P = 0.01). The single-state model simulation of the mean data is presented in Fig. 3H, showing how the single-state model fails to predict the fast increase of gain following the beginning of reversed adaptation, as well as spontaneous recovery, as the value of the single state can only continue to decrease toward the nonadapted state.

Table 1.

Comparison of R² statistics and BIC values between predictions of the single hidden-state (1 HS) and the 2 HS models on individual subjects and on mean data

Subject	R2 2 HS	R2 1 HS	BIC 2 HS	BIC 1 HS
S1	0.65	0.31	895	646
S2	0.67	0.28	849	561
S3	0.74	0.61	741	603
S4	0.39	0.34	939	923
S5	0.49	0.26	1105	971
S6	0.71	0.61	926	821
S7	0.46	0.17	910	761
S8	0.55	0.13	1024	745
Mean	0.83	0.55	686	125

BIC, Bayesian information criterion.

The BIC computed on the mean data simulation was 686 for the one hidden-state and 125 for the two hidden-states model.

DISCUSSION

To deepen our understanding of the analogies between the control of goal-directed movements and the control of reflex movements, such as the rVOR, we investigated the time course of short-term motor learning in the rVOR and developed a mathematical model to account for our experimental results. As an experimental proof of the existence of multiple timescales in the short-term motor learning of the rVOR, we specifically sought to investigate whether it would show the phenomenon of spontaneous recovery (Ethier et al. 2008; Kojima et al. 2004; Myers and Davis 2002). Thus we developed a custom paradigm (Fig. 1A), exposing the rVOR to an initial assessment of gain, followed by an adaptive paradigm calling for gain reduction, a shorter period of reversed, gain-increase adaptation, and a final set of CH, in which the subjects had no feedback on their rVOR performance, as the target disappeared at the beginning of the head movement.

Short-Term Incremental rVOR Adaptation Using Passive Head (and Body) Impulses and Spontaneous Recovery

Our experimental results show that the incremental adaptation paradigm was successful in inducing a statistically significant reduction of the gain of the rVOR in all tested subjects, obtaining a mean decrease in gain with respect to the initial baseline of 28% for a request of 40% (difference shown in Fig. 2A). The amount of adaptation assessed as the average over the first 10 movements at the beginning of the de-adaptation (Fig. 2A) trials was significantly lower than that achieved at the end of the gain-down adaptation session (Fig. 2A). Yet, as can be appreciated both on the mean and on individual subjects' data (Fig. 3, A, C, and D), such a difference is mainly due to the rapid de-adaptation that occurs during the 10 repetitions considered for the assessment. In other words, when faced with the large errors induced by reversed adaptation, the fast state quickly responds to the changed requirements, and the resulting rVOR gain value quickly increases during the initial 10 responses considered for computing the mean. The 70 repetitions of the reversal adaptation trials were also successful in significantly increasing rVOR gain in six of the eight tested subjects, with S4 and S7 showing no significant change between the beginning and the end of the reversed adaptation paradigm (differences in Fig. 2A). The average AR achieved at the end of reversal adaptation was still lower than the baseline by 7% (Fig. 2A).

AR values showed a significant reduction of gain during the final set of CH, arguing for the phenomenon of spontaneous recovery toward the previously learned, lower gain state. Both individual subject and averaged data showed a slowly increasing, yet not statistically significant, gain in the last 20 trials of the CH paradigm (Fig. 2A), which may reflect the free response (i.e., not driven by an input) of the motor-learning process once the fast state has returned to zero, and only the slow state still shows residual adaptation. Only the first 40 trials, i.e., approximately six fast states' time constants, were then considered to assess the evolution of adaptation during CH and the maximum level of spontaneous recovery (data reported in Fig. 2E).

Comparison of the AR behavior in such a final set of CH with that recorded during the control experiment consisting of the same number of CH, yet not preceded by adaptation, showed that subjects presented no significant trend in gain during the control experiment, whereas a highly significant decrease in gain was shown during the main experiment.

A Common Architecture for Motor Control

Previous studies have demonstrated that the brain may deal with the tuning and continuous recalibration of the rVOR through a control circuitry that is similar to that used for goal-directed movements: a forward model of the actuator in a cerebellar-feedback loop is fed with an efference copy of the motor command and provides the brain with a prediction of the consequences of movements before sensory feedback is available (Wolpert and Miall 1996). The error being made during the online movement can then be monitored by comparing the sensory-feedback signal with its prediction and may be used for online correction. Motor commands, on the other hand, are produced by an inverse model of the same actuator, thereby canceling its dynamics and issuing a command that will cause a movement that faithfully reproduces the desired behavior (Shadmehr and Mussa-Ivaldi 1994). Neural correlates for such a control structure were found for the control of eye movements and the control of the rVOR, in particular: the existence of an inverse model was first hypothesized by Robinson [in Skavenski and Robinson (1973)] and then confirmed recently by Green and colleagues (Green et al. 2007), which found that the “burst-tonic” and “tonic” cells in the prepositus hypoglossi and medial vestibular nuclei (PH-BT cells) encode the output of the inverse model, i.e., an efference copy of the motor command. The existence of a cerebellar-forward model for eye movements was also hypothesized by Robinson and colleagues [in Zee et al. (1976)] and later, by several other studies [see Green and Angelaki (2010) for a review], and the finding that PH-BT neurons project to the flocculus, thereby providing the input to the forward model, together with the similarity between the activity of eye-head neurons receiving projections from the cerebellar flocculus and eye velocity (Ghasia et al. 2008), provides support for such a hypothesis (Green and Angelaki 2010).

Mathematical Model of Motor-Learning Dynamics

Thus we developed a mathematical model of the rVOR circuitry, implementing the described control structure and including such inverse and forward models based on the model proposed by Glasauer (2003). We provided the model with a motor-learning mechanism based on credit assignment (Wei and Körding 2009) and a two hidden-states process (Smith et al. 2006) acting on the parameters of the cerebellar-forward model. The latter is the motor-learning process, comprising a fast state that quickly learns from motor error but has poor retention and a slow state that learns more slowly but has greater retention. The mechanism behind adaptation of the rVOR response through adaptive changes of the cerebellar-forward model is described. During gain-down adaptation, the nervous system faces a consistent retinal slip, as the VOR produces eye velocities that are faster than required for target stabilization (Fig. 4B; corrective saccades in opposite direction than rVOR). With our incremental adaptation paradigm, the error is initially small and is thus entirely assigned to motor error. The brain then starts to believe that the motor command sent to the motor plant has a stronger effect on the dynamics of the eye than its forward model predicts; i.e., the eyes are faster than predicted. The motor-learning process (Eqs. 4–7) thus increases the effect of the motor command u(t) in Eq. 3 by increasing the value of B̂, which causes the forward model to predict a faster eye movement for a given command. A higher predicted eye velocity, in turn, reduces or changes signs to the feedback error, which diminishes the velocity command to the eyes. During gain-up adaptation, the situation is reversed, and the eye movements are consistently less than compensatory (Fig. 4C; corrective saccades in the same direction as the rVOR); thus the contribution of u(t) in the predicted eye velocity is decreased, and the resulting command is increased.

Error Signal for rVOR Motor Learning

One issue that is still debated with respect to motor learning in the saccadic system is that of the error signal used by the brain in the motor-learning mechanism. Saccades are goal-directed eye movements, for which a well-defined error value may be computed for each movement performed. Based on the motor-control model in Fig. 1C, several error signals may be available for learning in the VOR: for instance, 1) a sensory error computed at the retina based on visual feedback and corresponding to the difference between real movement and intended movement, i.e., the estimate of retinal slip used in this work; 2) a predictive error between the desired and the predicted movement outcome; and 3) a predictive error between the predicted and the actual movement outcome, i.e., the predicted and the actual retinal slip, with the predicted outcome signals based on the output of the cerebellar-forward model [e.g., Wong and Shelhamer (2011b, 2012)]. The error between the desired and the predicted movement outcome (number 2 in the previous list) is used for online movement correction along the feedback loop; i.e., it is the mechanism of action of the forward model to change the amplitude of the ongoing responses but does not take into account visual information and cannot, therefore, drive motor learning. The prediction error (number 3 in the previous list) can indeed be used for learning, yet specific experimental paradigms are needed to clarify which is the error signal used by the brain in driving adaptation. Further error signals could be computed by considering an estimate of target movement (Chen-Harris et al. 2008) or by considering errors based on gaze position.

Regardless of which signal is being considered though, saccade error naturally corresponds to a scalar value for each movement performed. This is not the case with the rVOR, whose goal is that of maintaining the image of the external world, stable on the retina throughout the entire duration of the head rotation, so that a time-based vector of instantaneous errors may be computed in terms of real or predicted retinal slip, i.e., a velocity-based error signal, for each trial. Thus we hypothesized that the brain might use a scalar error value computed for each trial based on an average performance over time; i.e., we computed the error in a single trial as the average of the corresponding time-based signal over the same 80- to 120-ms interval from head-movement onset that we considered for the analysis of experimental data. Such a choice evidently represents an assumption for which we have no direct proof; i.e., we do not know how the brain evaluates the performance of the rVOR. Nonetheless, the consideration of a time interval within the open-loop portion of the ocular motor response may be ecological in the sense that it would provide the brain with useful information related to the performance of the sole rVOR. This would be advantageous with respect to a measure including the performance of the visually mediated movements, such as pursuit or the optokinetic reflex, since it is the performance of the rVOR that is critical in the open loop in response to high-frequency perturbations, which benefits more from a correctly calibrated pair of forward and inverse internal models. On the other hand, the hypothesis of a discrete error signal, driving VOR adaptation on a trial-by-trial basis, may be challenged by the traditional experimental approach to VOR adaptation based on continuous sinusoidal oscillations (Baloh et al. 1982). In other words, adaptation to continuous sinusoidal stimuli may raise the question of which discrete error signal could be used by the brain to drive adaptation in a condition in which the concept of trial is missing. We hypothesize that even in such a setting, the brain may still use a discrete approach to learning, which could be based on a discrete error signal that may either be computed by sampling the performance of the reflex, e.g., retinal slip, or by exploiting the error driving the corrective saccades that are typically produced during VOR gain training. The finding that a discrete position error at the end of transient head rotations is effective in driving adaptation (Eggers et al. 2003; Scherer and Schubert 2010) and that a corrective saccade is important in such a learning mechanism (Eggers et al. 2003) may support such a hypothesis. In both scenarios, the sampling of retinal slip or the triggering of a saccade would represent the discretizing event, producing a discrete error quantifying the reflex performance and causing the evolution of one step of the learning mechanism.

Credit Assignment

Credit assignment is a mechanism that given the information on an error, estimates its relevance to motor adaptation by assigning it to internal or external causes based on the size of the error and considering that only the error attributed to internal causes, i.e., the motor error, should be taken into account for motor learning (Chen-Harris et al. 2008; Ethier et al. 2008; Wei and Körding 2009). On the other hand, the role of such a credit-assignment mechanism may be subtle and may need to be investigated using specific experimental paradigms. Smith and colleagues (2006), for instance, did not use a credit-assignment block in modeling the adaptation of reaching to a suddenly introduced, velocity-dependent force field. Yet, the studies by Kluzik and colleagues (2008) showed that in such a learning task, the modality in which the error is introduced, i.e., gradual vs. abrupt changes in the dynamics of the manipulandum, influences the generalization properties of the learned behavior. In fact, abrupt changes of the force field induce a form of learning that transfers more to the other arm than learning caused by gradual changes, hinting that in the first condition, when errors are larger, the brain is more likely to consider the change as due to external causes, i.e., the behavior of the manipulandum, which therefore needs to be taken into account also when reaching with the other arm. Gradually introduced perturbations, instead, may be considered as due to a change of the arm's dynamics and therefore induce a motor learning that is limited to the arm performing the experiment (Kluzik et al. 2008; Malfait and Ostry 2004). Following the principle that when facing small errors, the brain is more likely to assign them to internal causes, we fitted the variance of a zero-mean Gaussian curve to our data, considering retinal slip as global error and the probability of it being of motor origin as the portion of such error attributed to motor causes and thus driving adaptation. The SD of the Gaussian distribution found for credit assignment was 7.1°/s when simulating the mean data; the mean value over the eight subjects was 6.6 ± 1.4°/s. Further research specifically tailored to the understanding of credit assignment is needed to explore such a phenomenon in detail and evaluate its potential implications, for instance, with vestibular rehabilitation or training paradigms. The difference in adaptive behavior between abrupt and gradual perturbations might be helpful for investigating the existence of a credit-assignment mechanism, as well as approaches, such as the extinction paradigm, or for studying the transfer of adaptation to other contexts. Also investigating a subject's perception of the error as a function of its amplitude might provide additional insights.

Motor Learning in the rVOR

As a theoretical argument on the structure of the motor-learning process, we compared the ability to fit the experimental data, i.e., the AR evolution curve, between a single hidden-state and a two hidden-states model. We thus fitted each model to the experimental data and assessed the goodness of fit of each simulation using the R² coefficient of determination statistics. All fits were improved using the two hidden-states model (R² ranging from 0.32 to 0.68; mean 0.57), with respect to the single hidden-state one (R² ranging from −0.13 to 0.57; mean 0.25). The BIC provided further support in favor of a two hidden-states memory process, showing that the increase in goodness of fit for the second-order model was indeed an indicator of a more appropriate model for the experimental data and not a consequence of overfitting. Longer experiments and more experimental paradigms are needed to answer the question of how many timescales are used by the nervous system. However, we did investigate whether a third timescale would improve the fit of our data but found that it was not the case.

We have thus shown that the rVOR may quickly adapt its gain in response to consistent visual errors in the form of retinal slip and that such adaptation may, in fact, result from changes in the forward model of the ocular motor plant, possibly residing in the cerebellar flocculus. Here, the main adaptation paradigm consisted of 200 trials, corresponding to ∼18 min, and the entire experiment of 390, lasting 35 min altogether. Such short-term adaptation may reflect, for instance, the general mechanism by which the brain maintains an appropriate reflex performance when facing muscle fatigue or adapts to the needs for a new sensorimotor transformation, such as that required by wearing glasses. On the other hand, it would seem odd that if the need for an altered response becomes permanent, such as with aging, then the brain achieves such adaptation by maintaining a wrong forward model of the plant in its cerebellar synapses. Indeed, studies on cats (Kassardjian et al. 2005), mice (Shutoh et al. 2006), and monkeys (Anzai et al. 2010) have shown that motor memories for adaptation of the horizontal rVOR are stored at different sites, depending on their consolidation. Motor memories acquired with short-term adaptive training are stored immediately in the cerebellar flocculus so that its pharmacological inactivation disrupts the recently learned changes, whereas the changes learned during the previous day's training are preserved and are therefore stored elsewhere, possibly in the brain stem (Anzai et al. 2010). These and other studies (Joiner and Smith 2008; Tanaka et al. 2012) have shown that when a memory-learning process is explained by a two hidden-states model, it is the slow state that may form a long-term memory being consolidated, whereas the fast state only reflects short-term learning. Indeed, recent work by Criscimagna-Hemminger and colleagues (2010) has provided new evidence that the fast-state process, allowing learning from large errors, resides in the cerebellum and that it is anatomically distinct from that responsible for learning from small errors. Therefore, one may question whether the incremental paradigm used in our work stimulated only the mechanism responding to small errors. On the other hand, the reversal of adaptation considered in our experiments did present the subjects with sudden large errors, to which they responded by a rapid gain adaptation, whereas the final set of CH, revealing spontaneous recovery, clearly produces adaptive changes that can only be explained by the involvement of two learning processes having distinct time constants. The issues related to consolidation of motor memory and to the role played by the slow-state process in such consolidation though go beyond the scope of our study, as we have not investigated retention of the learned behaviors to longer time intervals. Yet, it would appear reasonable that adaptive changes deserving to be consolidated become incorporated in the brain stem inverse model of the ocular motor plant, while always leaving to the floccular-forward model the task of learning the changes needed to respond to short-term adaptation requests.

Conclusions

In conclusion, both our experimental data and our modeling results strongly support the hypotheses that short-term adaptation of the rVOR is mediated by the adaptation of a forward model of the ocular motor plant residing in the cerebellar flocculus and that such adaptation is controlled by a multiple hidden-states process that modifies the parameters of the forward model based on motor error. For movements to be quick and accurate, the commands controlling them need to be adjusted continuously to reflect the state of our body, such as the level of fatigue, and of our interaction with the environment. The brain then faces the classical plasticity-stability dilemma, i.e., the need to find a compromise allowing sufficient plasticity to adapt to changes, while being stable enough to benefit from learning and behave consistently. Our results showing that the brain faces adaptation, even of a reflex movement, using the same approach used with goal-directed movements, lead to the hypothesis that multiple timescales in motor learning may be considered as a general principle, a general-purpose solution of the central nervous system to the cited dilemma, allowing the system to adapt quickly to errors, while having a relatively stable, slowly evolving reference behavior.

GRANTS

Support for this work was partially funded by Fondazione Cariplo Project No. 2009-2936, “Internationalization of the ICT and Bioengineering PhD Schools,” and by Italian Ministry PRIN Grant 2010R277FT_004 (to S. Ramat). Financial support was also provided from the Betty and David Koetser Foundation for Brain Research (Zurich, Switzerland).

DISCLOSURES

The authors declare no competing financial interests.

AUTHOR CONTRIBUTIONS

Author contributions: P.C., G.B., and C.J.B. performed experiments; P.C. and S.R. analyzed data; P.C. and S.R. interpreted results of experiments; P.C. prepared figures; P.C. and S.R. drafted manuscript; G.B., C.J.B., D.S., and S.R. edited and revised manuscript; D.S. and S.R. approved final version of manuscript; S.R. conception and design of research.