Learning on Multiple Timescales in Smooth Pursuit Eye Movements

Abstract

We commonly think of motor learning as a gradual process that makes small, adaptive steps in a consistent direction. We now report evidence that learning in pursuit eye movements could start with large, transient short-term alterations that stoke a more gradual long-term process. Monkeys tracked a target that started moving horizontally or vertically. After 250 ms of motion had produced a preinstruction eye velocity close to target velocity, an orthogonal component of target motion created an instructive change in target direction that was randomly in one of the two directions along the orthogonal axis. The preinstruction eye velocity in each trial expressed single-trial learning as a bias toward the direction of the instruction in the prior trial. The single-trial learning was forgotten within 4 to 10 s. Two observations implied that single-trial learning was not simply cognitive anticipation. First, the magnitude of the trial-over-trial change in eye velocity depended on the ongoing eye velocity at the time of the instruction in the prior trial. Single-trial learning was negligible if the prior trial had provided a well-timed cue without evoking any preinstruction eye velocity. Second, regular alternation of the direction of the instructive target motion caused reactive rather than anticipatory trial-over-trial changes in eye velocity. Humans showed very different responses that appeared to be based on cognitive anticipation rather than learning. We suggest that single-trial learning results from a low-level learning mechanism and may be a necessary prerequisite for longer-term modifications that are more permanent.

INTRODUCTION

Motor learning is a dynamic process that improves movement accuracy on the basis of real-time reports of prior inaccuracies. Behavior appears to change gradually and the first theories about motor learning assumed that the gradual behavioral learning would be mediated by a similar, gradual cellular plasticity at a single locus in the neural circuit for the behavior (Ito 1982). However, current thinking about the mechanisms of learning has moved toward explanations in terms of many interacting sites, with cellular mechanisms of learning distributed through the neural circuit for a given behavior (e.g., Ke et al. 2009).

The cerebellum is a central structure for motor learning and cellular analyses now have revealed plasticity on timescales that are short (Brenowitz and Regehr 2005; Brown et al. 2003) or long (Ito 2001; Lev Ram et al. 1995; Linden et al. 1991), and at over half the possible cerebellar sites (e.g., Hansel et al. 2001). It follows that cerebellum-dependent motor learning may comprise many separate components mediated at multiple neural sites by plasticity that operates over timescales ranging from single movements to days. Indeed, studies in behaving animals have provided evidence that behavioral learning has multiple facets and time courses in the vestibuloocular reflex (Boyden et al. 2004). In arm movements (Smith et al. 2006) and saccadic eye movements (Ethier et al. 2008), analysis of the progression of learning, forgetting, and relearning has implied the existence of rapid learning that is quickly forgotten and runs in parallel (Lee and Schweighofer 2009) with a slower and more memorable long-term component.

Smooth pursuit eye movements provide an excellent system to understand the neural mechanisms that work together to mediate motor learning. Prior reports have studied mostly the steady-state learned behavior present after repeated instructive target motions. For example, repeated changes in the speed (Kahlon and Lisberger 1996) or direction (Medina et al. 2005) of target motion causes acquisition of a “learned” response that peaks around the time of the instructive stimulus and that is present even on probe trials where the instructive stimulus is withheld. However, unexpectedly large and rapid changes may be occurring during the learning process. For example, Medina and Lisberger (2008) showed a large single-trial effect in which the occurrence of a complex spike in cerebellar Purkinje cells on one trial was linked to a large and properly timed reduction in simple-spike firing on the next trial.

The finding of large single-trial changes in the responses of cerebellar Purkinje cells led us to study single-trial learning using an approach developed by Cheng and Sabes (2006, 2007). We provide a direct demonstration of a component of learning that 1) occurs on the time course of single trials, 2) requires the conjunction of ongoing movement and an instructive error signal, and 3) is forgotten within 4–10 s. We also show important roles in the overall learning process for other, more conventional learning mechanisms that operate on longer timescales.

METHODS

Three male rhesus monkeys (Macaca mulatta, 8–11 kg) served as subjects in most of the experiments. Their eye movements were recorded with the magnetic search coil method. Each monkey was well trained on pursuit tasks but was naive to all the direction-learning paradigms. We prepared the monkeys for the study with surgical and training procedures that we have described previously (Medina et al. 2005). All experimental procedures had been approved in advance by the Institutional Animal Care and Use Committee of the University of California, San Francisco (UCSF) and were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals. In addition, a limited set of experiments were repeated on two human subjects, one naïve to the purpose of the experiments and one an author. In human subjects, eye movements were measured using a dual Purkinje image tracker. Details are available in Garbutt and Lisberger (2006). Procedures had been approved in advance by the Institutional Review Board at UCSF and informed consent was obtained.

Experimental design

Fixation and pursuit targets comprised bright 0.3 or 0.5° spots, respectively, on a dark background. They were presented on a 20-in. CRT monitor with a refresh rate of 85 Hz and a spatial resolution of 1,280 × 1,024 pixels. The screen was positioned 30 cm from the monkey and subtended 59° horizontal × 47° vertical. All experiments were performed in a dimly lit room.

Stimuli were presented as a sequence of behavioral trials. Each trial started with a random-duration fixation period of 600 to 1,000 ms followed by a sequence of target motions; each trial was terminated by another fixation period of duration 400 ms. At the start of motion, the target underwent a 3° eccentric step and a ramp back toward the original fixation position at 20°/s. The direction of the initial target motion was called the “pursuit direction.” Many experiments began with a prelearning block of 100 trials in which the initial ramp target motion continued uninterrupted for 1,050 s. Thereafter, all trials provided an instructive component of motion that started 250 ms after the onset of target motion in the pursuit direction and superimposed motion at 30°/s for 400 ms in an orthogonal direction that we will call the “learning” direction. Then, the target continued to move in the pursuit direction for an additional 400 ms before stopping. In some experiments, the pursuit direction was along the horizontal axis and the learning directions were up and/or down. In others, the pursuit direction was along the vertical axis and the learning directions were right and/or left. Except as noted in results, the data were nearly identical for the two axes. Given the evidence we provide that 70–90% of single-trial learning is lost within one trial or 10 s, it did not make sense to include postlearning blocks of unidirectional target motion without instructive stimuli, although we did on some experiments.

We used three different daily experimental designs that differed in the sequence of instructive trials. Most of our data come from the “random” paradigm, in which the learning directions varied randomly from trial to trial between the two directions along the learning axis. In most random paradigm experiments, we allowed the natural pace of the experiment to determine the interval between an instruction on one trial and the target motion that probed its effects on the next trial, creating “instruction–test intervals” that ranged from 2,050 to 2,450 ms. In a few experiments, we used much longer trials that contained two step-ramp target motions so that we could control the instruction–test interval exactly. The “repeated” paradigm contained hundreds of trials, with the same learning direction presented repeatedly in 100-trial blocks before the opposite learning direction was tested similarly. The “alternating” paradigm provided a regular sequence of trials with learning directions that alternated between the two choices along the learning axis.

Monkeys were required to hold their gaze within a 2–3° window centered on the moving target. However, to avoid punishing the monkey for his inescapable response latencies, fixation requirements were suspended for the 250 ms between the onset of the step-ramp target motion and delivery of the instructive change in target motion. If the monkey failed the fixation requirements, then the trial was aborted. Successful execution on a trial was rewarded with a drop of juice or water. In each daily experiment, monkeys attempted >2,000 trials and their success rates were >90% (monkey M, 95%; J, 92%; and P, 90%).

Data analysis

Eye position and velocity signals were sampled at 1 kHz on each channel and stored for off-line analysis. Horizontal and vertical eye velocity data were provided by an analog circuit that differentiated signals at frequencies <25 Hz and filtered out signals at higher frequencies. The traces from each trial were inspected visually and trials were discarded if saccades occurred in the analysis interval, which started 200 ms before the onset of target motion and ended 450 ms later at the time of the instructive, orthogonal target motion. To assess the size of learning on each trial, we computed the mean difference between eye velocity on the nth and n − 1st trials across the last 150 ms before the end of the analysis window. For the random and alternating paradigms, we excluded data from the first 100 learning trials to ensure that we were measuring the trial-over-trial changes in eye velocity in a stationary state. For the repeated paradigm, we excluded data from the rapid acquisition of learning in the first 10 trials, again to measure performance in a stationary state.

A model for single-trial learning

We used a simple linear dynamic system to infer the contribution of learning to the trial-over-trial change in eye velocity and to model the dynamics of motor learning in the random paradigm (Cheng and Sabes 2006, 2007). We described the eye velocity on any given trial as

$${\dot{E}}_i=\left(1-{\omega }_{\text{R}}\right){\dot{E}}_{i-1}+{\omega }_{\text{L}}{\dot{I}}_{i-1}+\eta$$ (1)

where Ė_i and Ė_i−1 are the eye velocities in trial i and i − 1 of a consecutive pair and İ_i−1 is the instructive signal in the first of the pair of trials. The three components of the model describe: 1) the decay of eye velocity back toward zero learned response between two trials, according to the weight ω_R; 2) the learning that occurred between two trials, driven by the instruction in the prior trial according to the weight ω_L; and 3) the noise unaccounted for by the events in the prior trial, defined as distribution η.

We rewrote Eq. 1 to predict the quantity we measured, i.e., the trial-over-trial change in eye velocity, as

$$\Delta {\dot{E}}_{i,i-1}=-{\omega }_{\text{R}}{\dot{E}}_{i-1}+{\omega }_{\text{L}}{\dot{I}}_{i-1}+\eta$$ (2)

We then optimized the values of the three weights for the data from each daily experiment. Optimization was performed separately for each direction of instructive target motion and each millisecond in the data. The weights were chosen by exhaustive search for the values of ω_R and ω_L that minimized the root-mean-square (rms) difference accumulated across all trials between the predictions of the model and eye velocity. The values were similar across the 150-ms analysis interval, failing to show any interesting dynamics, and we report only those for the time that provided the peak size of learning.

RESULTS

Directional learning from the prior trial

Eye velocity on any given trial depended in a subtle but consistent way on the instructive change in target direction in the prior trial. Figure 1 shows the experimental design we used to study trial-over-trial effects. Each trial consisted of target motion that started in one direction and added “instructive” motion in an orthogonal direction 250 ms later. In the left column, for example, motion on the n − 1st trial (leftmost, black icon) was initially rightward, added an instructive upward component, and then returned to purely rightward. In the subsequent, nth trial, target motion could be identical to the n − 1st (Fig. 1A, top blue icon) or could impose a downward rather than upward instructive target motion (Fig. 1C, bottom blue icon). In Fig. 1, in both A and C, the vertical eye velocity on the nth trial (blue traces) had an upward deflection relative to the n − 1st trial (arrowheads). The deflection started before the instructive change in target direction and reached a peak at about the time of the instruction. In these trials, we refer to the axis of the original pursuit as the “pursuit” direction and the axis of the orthogonal, instructive motion as the “learning” direction.

Fig. 1.

Example trials showing the experimental design used to study single-trial learning in one trial based on the instructive target motion in the prior trial. A–D: comparisons of the eye and target movements in 2 successive trials, dividing according to the direction of the instructive target motion in each panel, illustrated by the icons to the left of each panel showing the 2-dimensional trajectories of target position. The sequences of instructive target motions were: A, up-up; B, down-down; C, up-down; D, down-up. In each panel the 2 sets of traces show horizontal and vertical motions, with dashed vs. solid traces indicating target and eye velocity and black vs. colored traces indicating the prior and current trials. Diagonal arrowheads point out the brief deflections associated with single-trial learning before the onset of the instructive target motion. E and F: millisecond-by-millisecond changes in eye velocity between 2 successive trials, for upward (E) and downward (F) instructive target motions. Gray traces superimpose many responses for a single experiment, colored traces show the data from A–D, and bold black traces show the averages across trials. Time zero indicates the onset of the instructive target motion.

The small upward response to the upward instruction on the n − 1st trial appeared in the nth trial, whether the nth trial presented an upward (Fig. 1A) or downward (Fig. 1C) instructive change in target direction. The converse situation occurred when the instructive change in target direction in the n − 1st trial was downward (Fig. 1, B and D, black traces). In the nth trial (red traces), there was a small downward deflection of vertical eye velocity (arrowheads) that again preceded the time of the instructive change in target direction and was present in trials that presented either an upward (Fig. 1D) or downward (Fig. 1B) instructive signal.

Many responses from a day's experiment are superimposed in the graphs at the bottom of Fig. 1 to illustrate the statistical effect of the instructive stimulus in the prior trial on the eye velocity in any given trial. As shown by the bold, black traces in Fig. 1, E and F, the mean trial-over-trial eye difference in velocity was in the direction of the instruction on the prior trial for both upward and downward instructions. Although there was considerable trial-by-trial variation of the difference in eye velocity in the last 150 ms before the onset of the instructive signal, the individual responses were in directions biased toward the instruction in the prior trial: 64% upward versus 36% downward in Fig. 1E, and 63% downward versus 37% upward in Fig. 1F. Indeed, the four examples from Fig. 1, A–D (colored traces in Fig. 1, E and F) were chosen to illustrate the phenomenon and represent some of the largest examples of single-trial learning in these experiments.

Fig. 3.

Duration of the memory for single-trial learning. Schematic diagram at the top of the figure shows the temporal sequence of target velocities in a single long trial, defining the “instruction–test interval” as the time from the onset of an instructive change in target direction to the subsequent step-ramp target motion used to probe memory. The solid and dashed traces in the instruction target velocity for the test target motion indicate that it was randomized to be in either the same or the opposite direction from the first instructive target motion. A–D: the mean trial-over-trial change in eye velocity in the 150 ms before the instructive target motion is plotted as a function of the instruction–test interval for vertical (A, C) and horizontal (B, D) instructive target motions and for monkey P (A, B) and J (C, D). Different symbols show different directions of instructive target motion along each axis. Error bars are 1SD over the data from 4 experiment days.

Figure 1 indicates that an instruction on one trial biases eye velocity on the subsequent trial, without having a strong enough effect to mitigate tracking errors on the subsequent trial. Indeed, we will show later that single-trial learning accounts for only a fraction of the trial-by-trial variance in eye velocity. We will thus argue that single-trial learning is a short-term effect that itself does not correct errors very effectively, but may be a mechanistic prerequisite for more permanent, larger, and more adaptive long-term learning.

Time course of learning and forgetting

Instructive signals were provided on all trials in random order, so that eye velocity in the learning direction on any given trial could be affected by instruction in any prior trial. To assess the effect of the entire prior history, we divided the full sample of trials into different sets according to the direction of the instruction on the prior nth trial, where n was 1, 2, 3, 4, or 5. When trials were divided according to the n − 2nd instruction, for example, either direction of instruction could have occurred on the n − 1st, n − 3rd, n − 4th, or n − 5th trial. As illustrated in Fig. 2, the instruction in the n − 1st trial had the greatest effect on eye velocity in the subsequent trial and the effect declined quickly as the instruction moved backward in history.

Fig. 2.

Changes in eye velocity caused by instructive target motion in each of the prior 5 trials. The 3 columns show data for 3 monkeys. A–C: average traces showing the time course of the difference between eye velocity and the nth and n − xth trial, divided according to whether the instructive target motion in the n − xth trial was upward or downward. The value of “X” was 1, 2, 3, 4, or 5. The error band on each trace indicates 1SD of the averages across daily experiments. D–F: the change in eye velocity across the last 150 ms before the instructive target motion is plotted as a function of how far the instructive target motion was in the past. G–I: the variance of the change in eye velocity is plotted as a function of how far the instructive target motion was in the past. Data in the graphs were averaged across 4 daily experiments and triangles and circles indicate data for horizontal and vertical directions of instructive target motion. Error bars indicate 1SD of averages across daily experiments. In D–I, data are plotted separately for experiments with horizontal and vertical learning directions, but are combined across the 2 directions along each axis, where there were no significant differences in these measures from the data.

The larger effect of more recent instructions appears both in time averages of the trial-over-trial difference in the eye velocity along the learning direction (Fig. 2, A–C) and in graphs that show the mean across 4 experimental days (Fig. 2, D–F) in all three monkeys for both horizontal (triangles) and vertical (circles) learning directions. The variance of the trial-over-trial difference in eye velocity (Fig. 2, G–I) decreased somewhat for instructive trials closer to the trial in which the effect of the instruction was assessed. The decrease in variance indicates that dividing the trials into two sets according to the instruction in the most recent prior trial assigns some of the variance in the trial-over-trial changes in eye velocity to the recent history of target motion. In addition, there is considerable variance that must be attributed to other sources.

In most of our experiments, the interval between the start of two trials varied randomly within the interval from 2,050 to 2,450 ms. To assess the duration of memory in the absence of further instructions, we contrived longer trials that contained a delay we could control between two target motions. As illustrated in the schematic diagram at the top of Fig. 3, the first half of each target motion contained an instructive change in direction, and the second half provided a new target motion at a controlled time to test memory as a function of the interval between the instruction and the test target motion. The test target motion included an instruction that was randomized to be in either direction (continuous vs. dashed traces), to ensure that the monkeys were expecting a change in target direction and did not suppress learned responses because they would never be needed. The “instruction–test interval” between the two instructive target motions was randomized across trials to be 1.5, 2, 4, or 10 s. Experiments with horizontal or vertical learning directions were conducted on separate days.

The trial-over-trial difference in eye velocity declined as a function of increases in the instruction–test interval (Fig. 3). For both horizontal and vertical directions, the single-trial learning has been forgotten when the interval between the instruction and the probe is 10 s. For horizontal learning directions, the single-trial effect failed to endure for 4 s in either monkey (Fig. 3, B and D), whereas for vertical learning directions (Fig. 3, A and C) it appeared to endure longer for downward instructions (open symbols) but not for upward instructions (filled symbols). We cannot explain the absence of trial-over-trial changes in eye velocity with an instruction–test interval of 1.5 s for rightward trials in monkey J (Fig. 3D, filled symbols), except to say that it was a consistent finding across days.

Components of trial-over-trial differences in eye velocity

To understand the factors leading to the sequence of eye velocity responses to the randomly ordered sequence of instructive stimuli, we fitted the trial-over-trial sequence of responses with a linear dynamic system model (Cheng and Sabes 2007). The model (Eq. 2) assumed that there were three contributions to the trial-over-trial change in eye velocity: 1) relaxation (or forgetting) from the prior response back toward a baseline that was assumed to be zero; 2) learning driven by the prior instructive signal provided by the orthogonal direction of target motion; and 3) noise. By separating the sequence of trial-over-trial changes in eye velocity into these three components, we can estimate the contribution of learning to the trial-over-trial change in eye velocity.

Rather than attempting to compute the optimal values of ω_R and ω_L, we simply computed the predicted eye velocity in each trial based on the eye velocity and instruction in the prior trial for all reasonable combinations of the weights. We then determined the pair of weights that minimized the rms difference between the predicted and actual sequence of eye velocities from color maps of the weights like those in Fig. 4A. In each monkey, there is a region of minimum prediction error (dark blue) that is centered at about 0.8 on the relaxation weight axis and at positive or negative learning weights on for upward (top row of graphs) versus downward (second row) instructions. The relaxation weights were similar for all monkeys and both directions of vertical instructive target motion (Fig. 4, B and C) and 70–90% of vertical eye velocity in the n − 1st trial was lost by the nth trial. We found an up-down asymmetry in learning weights (Fig. 4, D and E), with rather low values for downward instructive target motions. We obtained a similar picture for horizontal instructions presented during vertical tracking in the two monkeys we tested (Fig. 5), but with symmetrical relaxation and learning weights for rightward versus leftward instructions.

Fig. 4.

Quantitative analysis of single-trial learning for vertical instructive target motions in terms of a linear dynamic system model. In A, the color maps show prediction error as a function of the weights of relaxation and learning, ω_R and ω_L. Different color maps indicate data for the 3 monkeys (columns) and upward vs. downward instructive target motions (rows). B–D: bar graphs showing the mean and SD of the weights across 3 experimental days in each of 3 monkeys. Different graphs show the relaxation weights for upward (B) and downward (C) instructive target motions and the learning weights for upward (D) and downward (E) instructions.

Fig. 5.

Quantitative analysis of single-trial learning for horizontal instructive target motions in terms of a linear dynamic system model. In A, the color maps show prediction error as a function of the weights of relaxation and learning, ω_R and ω_L. Different color maps indicate data for the 2 monkeys (columns) and leftward vs. rightward instructive target motions (rows). B–D: bar graphs showing the mean and SD of the weights across 3 experimental days in each of 2 monkeys. Different graphs show the relaxation weights for rightward (B) and leftward (C) instructive target motions and the learning weights for rightward (D) and leftward (E) instructions.

How can the linear dynamic system model report a large up-down asymmetry (Fig. 4, D and E) when the actual trial-over-trial changes in eye velocity are quite symmetric (Fig. 6, A and B)? The answer lies in the absolute eye velocity after upward and downward instructive target motions, which also had a large up-down asymmetry (Fig. 6, C and D). The very small learning weight for downward instructive target motions is related to the upward mean absolute eye velocity. The trial-over-trial change is negative after downward instructions because the relaxation of eye velocity toward zero and the weak learning are working together, but are not strong enough to drive eye velocity to be negative on average. For horizontal instructive target motions, both the trial-over-trial difference in eye velocity (Fig. 6, E and F) and the absolute eye velocity (Fig. 6, G and H) are symmetric after rightward versus leftward instructions. As a result, both the relaxation and learning weights in the linear dynamic model for horizontal instructions also are symmetric (Fig. 5, B–E). We note that this interpretation of our data is model dependent, albeit using a model that provides a good account of the features of single-trial learning.

Fig. 6.

Explanation of how learning can be asymmetric for up vs. down instructive target motions when the trial-over-trial changes in eye velocity are symmetric. A, B, E, and F: time course of trial-over-trial changes in eye velocity after instructive target motions that were upward (A), downward (B), rightward (E), or leftward (F). C, D, G, and H: time course of actual eye velocity after instructive target motions that were upward (C), downward (D), rightward (G), or leftward (G). In each panel, the gray traces indicate single-trial responses and the bold black traces plot averages across trials.

Relationship between learning in single trials and longer timescales

To test how single-trial effects interacted with mechanisms that operated over longer timescales, including strategies based on knowing (or not knowing) the sequence of trials, we used three different experimental designs. In the “random” paradigm, the direction of the instructive target motion was randomly in one of the two directions along the learning axis, as described herein so far: the monkey could not correctly anticipate the direction of the instructive target motion. In the “alternating” paradigm, the direction of the instructive signal was perfectly predictable: it was in opposite directions along the learning axis in successive trials. In the “repeated” paradigm, the instructive target motion was in the same direction on all trials.

When the sequence of instruction directions followed the repeated paradigm, all three monkeys showed a rapid acquisition of learning in the first few trials, followed by a steady increase or decrease in the learned eye velocity over the subsequent 100 trials (Fig. 7, X symbols and small circles). Comparison of the time course of learning in the repeated paradigm with the predictions from the linear dynamic system model obtained from the random paradigm revealed some aspects of the data that could, and others that could not, be accounted for in terms of single-trial learning. The parameters from the model of single-trial learning reproduced well the fast, early component of learning in each 100 trial block for the repeated paradigm. However, the model for single-trial learning did not predict the continued steady increase in the size of learning, but instead predicted an asymptote (solid and dashed curves) that was achieved quickly and retained throughout the 100-trial learning block. We conclude that single-trial learning describes a short-term component of pursuit learning, but that other, longer-term components are recruited when the same instructive target motion occurs repeatedly.

Fig. 7.

Comparison of the time course of pursuit learning when the same instructive target motion is repeated many times, with the predictions based on the linear dynamic systems model for single-trial learning. The 3 graphs show data for 3 monkeys. The X symbols and dots show the eye velocity averaged across 150 ms before upward or downward instructive target motions as a function of trial number in the repeated paradigm. The solid and dashed curves show the predictions calculated using best-fit learning and relaxation weights derived from the random paradigm. The starting point (trial 1) for the predictions was taken as the response on the first trial after a switch of the direction of the instructive target motion in the repeated paradigm.

Further analysis revealed that single-trial learning could be modulated by cognitive knowledge of the pattern of instructive target motions, but did not show the features we would have expected if single-trial learning were actually cognitive anticipation. In the alternating paradigm, the predictable reversal of the direction of the instructive target motion on each trial should favor anticipatory trial-over-trial changes in the direction opposite to the instructive target motion in the prior trial. This is what Wells and Barnes (1998) reported for their human subjects and is what we found when human subjects performed our version of the alternating paradigm: the trial-over-trial change in eye velocity was in the direction of the instructive target motion on the present trial. In our data on monkeys, in contrast, the trial-over-trial change in eye velocity in the alternating paradigm is reactive: it is in the direction of the prior instructive target motion, as in the random paradigm (Fig. 8). We did find that the trial-over-trial effects were weaker in the alternating versus the random paradigm in monkeys (Fig. 8, long dashed vs. solid traces).

Fig. 8.

Time course of the mean trial-over-trial change in eye velocity for different sequences of instructive target motions. Solid, long dashed, and short dashed curves show results for the random, alternating, and repeated paradigms. The 3 graphs show data for 3 monkeys, averaged across 3 daily experiments.

The trial-over-trial change in eye velocity is quite small in the steady-state phase of the repeated paradigm (Fig. 8, short dashed traces), as one might expect given the gradual acquisition of the long-term learned behavior. Thus the regularity and repeatability of the sequence of instructive target motions did affect the size of the trial-over-trial difference in eye velocity, but not the direction as predicted if single-trial learning was due to cognitive anticipation. Careful selection of segments from the random paradigm with alternating or repeated directions of instructive target motion demonstrated that the modulatory effects in the alternating paradigm were due to knowledge of the pattern of instructive target motions during a longer session rather than a simple effect of recent history. The modulatory effects are multiplicative and thus are not predicted by the linear model of Eq. 2. Finally, we did not find any drift in the size of the trial-over-trial changes in eye velocity across the duration of experiments with >2,000 trials. Trial-over-trial changes were the same when comparing results computed separately for the first, second, or third 1/3 of the data.

Necessary conditions for single-trial learning

We conducted one additional test of whether single-trial pursuit learning is a genuine adaptive change that is driven by providing an instruction during ongoing pursuit or whether it behaves as might be expected for cognitive anticipation that is time-locked to the onset of target motion or any other properly timed signal. To discriminate these possibilities, we tested single-trial learning with the target motions diagrammed in the schematic at the top of Fig. 9. Our goal was to create a series of trials where the target motions differed but the cues should be equally strong for timing the onset of instructive target motion.

Fig. 9.

Effect of eye velocity at the time of the instructive target motion on the size of single-trial learning. The schematic at the top left indicates the sequence of target motions and defines the “cue duration” as the duration of target motion in the learning direction during the 250 ms before the instructive change in target direction. The schematic at the top right indicates the temporal sequence of target velocities for a cue duration of 50 ms. A and B: time course of the average size of learning, with different colors showing the data for different cue durations. Dashed vs. solid traces show the learning for downward vs. upward instructive target motions. Time zero indicates the onset of the instructive target motion. The 2 graphs show data for 2 monkeys. C: the mean trial-over-trial change in vertical eye velocity is plotted as a function of the mean eye speed in the horizontal direction just before the instructive target motion. Filled vs. open symbols show data for upward vs. downward instructive target motions. Black symbols show data for experiments where the cue duration was varied and the target speed was 20°/s. Red symbols show data for experiments where the cue duration was 250 ms and the target speed was 5, 10, or 20°/s. Error bars indicate 1SD.

Each trial began with a cue that occurred 250 ms before the onset of the instructive vertical target motion to provide a time reference for future anticipation or learning. In most cases, the cue was a step-ramp of target motion along the horizontal axis. At one extreme, which we will call a cue duration of 250 ms, the target motion continued through the 250-ms interval, exactly as in our prior experiments. At the other extreme, which we will call a cue duration of 0 ms, the target changed size only from 0.3 to 0.5° but did not move. In other experiments, the target moved for cue durations of 50, 100, 150, and 200 ms and then stopped for 250 ms minus the cue duration. The target started to move horizontally again at the same time the vertical instruction was delivered, 250 ms after the initiation of the step-ramp of target motion. Different cue durations were tested on different experimental days so that the monkey would have the opportunity to become quite familiar with the time contingencies and use the timing information in the cue if he was able to do so. If single-trial learning were actually cognitive anticipation, based on the time difference between the cue onset and the time of the instructive change in target direction, then we would expect the trial-over-trial change in eye velocity to be the same for all cue durations as long as the cue allowed accurate timing of the change in target direction.

The trial-over-trial change in eye velocity change depended strongly on cue duration. In both monkeys tested (Fig. 9, A and B), learning was best when the cue duration was 250 ms (blue traces, meaning no pause in horizontal target motion). The change in eye velocity declined as cue duration decreased (meaning less motion and longer pauses) until there was essentially no trial-over-trial change when the cue duration was 0 ms (black traces). The effect of cue duration can be understood as a systematic effect of horizontal eye velocity at the time of the instruction on the trial-over-trial change in vertical eye velocity. Plotting the trial-over-trial eye change in velocity as a function of the horizontal eye speed at the time of the instructive target motion yielded a linear relationship (Fig. 9C).

To provide a proper control for the relationship between the trial-over-trial change in eye velocity and ongoing pursuit eye velocity, we presented instructive vertical target motions in randomized directions during tracking at 5, 10, or 20°/s, with uninterrupted horizontal target motion before delivery of the instructive vertical target motion (cue duration 250 ms). Different target speeds were used on different experimental days. The trial-over-trial change in vertical eye velocity increased as a function of horizontal eye velocity at the time of the instruction (Fig. 9C, red symbols) and followed the same linear relationship that described the data for target motion at 20°/s with different cue durations. We conclude that an instructive signal for a change in pursuit direction is effective in proportion to the magnitude of the ongoing pursuit eye velocity at the time of the instruction. If eye velocity is zero, then nothing is learned, even if a reliable cue occurred 250 ms before the instructive vertical target motion.

Single-trial learning and anticipatory smooth eye movements in humans

As a test of whether “single-trial learning” in monkeys is actually cognitive anticipation, we have conducted a subset of our experiments on two subjects from the species that anticipates best: humans. Our human subjects were not told the sequence of target movements in advance and they reported after each experiment that they were able to predict the direction of the next change in target direction in the repeated and alternating paradigms, but not in the random paradigm. Figure 10, A and B shows that two human subjects showed reasonable directional learning in pursuit when the same instructive stimulus was repeated on every trial. The time course of learning was generally shorter in humans than that in monkeys and the magnitude of learning was generally a bit smaller than that in monkeys.

Fig. 10.

Cognitive anticipation vs. single-trial learning in humans. A and B: the 2 graphs show data for 2 humans. The X symbols and dots show the eye velocity averaged across 150 ms before upward or downward instructive target motions as a function of trial number in the repeated paradigm. C, D; A–C: average traces showing the time course of the difference between eye velocity and the nth and n − 1st trial, divided according to whether the instructive target motion in the n − 1st trial was upward or downward. Red and black traces show responses when the instruction in the prior trial was upward or downward. Continuous and dashed traces show average eye velocity when the direction of the instruction change in target motion was randomized vs. alternated. The error band on each trace indicates 1SD of the averages across daily experiments.

When the direction of the instructive change in target motion occurred in random order from trial to trial, humans showed essentially no single-trial learning (Fig. 10, C and D, continuous traces). When the direction of the instructive change in target motion alternated from trial to trial, humans anticipated the direction of the motion in the current trial. Thus the smooth eye movements were more upward on the trial after one that presented a downward instructive target motion and vice versa. The anticipatory eye movements in humans are opposite in direction to the responses of monkeys under the same stimulus conditions, but are as large as the manifestations of single-trial learning recorded in monkeys. We conclude that the single-trial changes in smooth eye movement in monkeys are reactive to the prior instructive stimulus and are not anticipatory, at least not in the same sense that anticipation has been used in studies of smooth pursuit eye movements in humans.

DISCUSSION

It has been common to think of motor learning as the result of many successive small changes in one or several mechanisms of cellular plasticity, gradually altering a movement in a way the reduces movement errors. In smooth pursuit eye movements, prior work had shown that 50–100 repetitions of a change in the direction or speed of target motion causes learning of a smooth eye movement that is well timed to, and starts just before, the time of an instructive stimulus (Kahlon and Lisberger 1996; Medina et al. 2005). Now we have provided direct measurements of a process of single-trial learning where the eye movement on one trial is altered by the instructive stimulus presented in the prior trial. The single-trial effects in our data are quite different from those reported some time ago by Kowler et al. (1984), in that our effects are much larger (0.5–1°/s vs. 2 min arc/s) and that our effects are produced by moving targets, whereas theirs appeared in relation to saccades produced by step changes in target position.

Our study shows the existence of single-trial learning directly and thus substantiates prior conclusions based on fitting models to behavioral learning curves. For both reaching (Smith et al. 2006) and saccadic eye movements (Ethier et al. 2008), two components of learning with different time courses are obligated by data from a clever sequence of learning in one direction, relearning in the opposite direction, and finally probes that do not cause learning. Even though single-trial learning in pursuit eye movements has a much shorter time course of acquisition and forgetting than that predicted by the models for saccades and reaching, it may be related to their faster component of learning.

Properties of single-trial learning

We suggest that single-trial learning works as a stimulus–response machine on the basis of the observation that trial-over-trial changes in eye velocity occur in proportion to the ongoing eye velocity at the time of the instruction in the prior trial. We suggest that learning occurs in terms of conventional plasticity mechanisms, such as at a site in the brain where signals related to ongoing eye velocity and visual errors converge. In this scenario, the strength of the learning would be determined by the size of the eye velocity signal at the time of arrival of the visual error signal. Three aspects of our data imply that single-trial learning in the pursuit of monkeys is different from the cognitive anticipation demonstrated by Wells and Barnes (1998). First, single-trial learning is nonexistent in stimulus conditions that provide a reliable cue for timing the instructive change in the direction of target motion without causing an eye movement to drive the stimulus–response machine. In the cognitive anticipation studied by Wells and Barnes (1998), anticipation does not require the presence of ongoing eye velocity at the time of the change in target motion that the system learns to anticipate. Second, in conditions where the direction of target motion alternated predictably, Wells and Barnes (1998) found, and our experiments on humans substantiated, that eye movements were truly anticipatory in the sense that they were in the direction of the current instructive target motion. In monkeys, in contrast, single-trial learning in the alternating paradigm was reactive in the sense that the trial-over-trial changes in eye velocity were related to the instruction on the prior trial rather than to anticipation for the current trial. Third, in the random stimulus condition that caused the largest single-trial learning in monkeys, humans showed essentially zero learning, indicating that their well-developed anticipatory mechanisms could not account for the results obtained in monkeys.

Prior work has suggested that pursuit eye movements result from convergence of signals from two separate processes. One process is visual motion processing, which estimates target direction and speed from the responses of neurons in the extrastriate visual motion pathway (Newsome et al. 1985). The other process uses signals that emanate from the smooth eye movement portion of the frontal eye fields to regulate the strength, or “gain,” of visual motor transmission for pursuit (Schwartz and Lisberger 1994; Tanaka and Lisberger 2001). Tabata et al. (2008) demonstrated that the gain of visuomotor transmission is modulated by the presence versus the absence of target motion on the prior trial and that the modulation is expressed in both directions along the axis of target motion. In contrast, the learning demonstrated herein is directional (and axial) and behaves as if the pursuit system has acquired a new response to a given target motion rather than as modulation of an existing response. Although single-trial learning appears to be the result of a low-level nuts-and-bolts learning mechanism, the modulation of the size of learning in the random versus alternating paradigms could be an effect of gain control or of other cognitive processes such as attention.

Neural site of single-trial learning

The cerebellar cortex is one putative site of motor learning. According to the cerebellar learning hypothesis, action potentials in a climbing fiber input signal movement errors and cause long-term depression of the strength of synapses on Purkinje cells from parallel fibers that are active at the time of the climbing fiber response (Gilbert and Thach 1977; Ito 1972). A previous study from our laboratory has demonstrated single-trial depression of the simple spike firing of floccular Purkinje cells if, and only if, a properly timed complex spike occurred during the prior movement (Medina and Lisberger 2008). Thus single-trial plasticity in the cerebellar cortex can occur very quickly, on the same timescale as that of the behavioral effects documented here. Further, Brown et al. (2003) and Brenowitz and Regehr (2005) previously demonstrated cannabinoid-mediated short-term plasticity in the cerebellar cortex that is induced by complex spikes and, like single-trial behavioral learning, endures for only a few seconds. We conclude that cerebellar plasticity could be a mechanism of single-trial learning in pursuit. In saccades, Criscimagma-Hemminger et al. (2010) have come to the parallel conclusion that the rapid component of learning depends on an intact cerebellum.

A similarity in the kinematics of Purkinje cell responses during pursuit and single-trial learning is compatible with a locus of single-trial learning in the floccular cortex. The floccular complex is tightly linked to the oculomotor output because of the disynaptic pathway from its Purkinje cells to extraocular motoneurons (Highstein 1973; Lisberger 1994). Many floccular Purkinje cells show strong modulation of both simple and complex spike responses in relation to pursuit eye movements (Lisberger and Fuchs 1978; Miles and Fuller 1976; Stone and Lisberger 1990a), with most of the cells showing increased simple spike firing for eye movements either toward the side of the recording or downward (Krauzlis and Lisberger 1996). The complex spikes of the same Purkinje cells respond to image motion in the opposite direction, either away from the side of the recording or upward (Stone and Lisberger 1990b). Thus taking into account the two sides of the cerebellum, floccular complex spikes have the potential to instruct upward, rightward, and leftward learning. However, the paucity of Purkinje cells with complex spikes that increase for downward visual image motion means that there is an up-down asymmetry in the climbing fiber inputs to the floccular complex. This asymmetry in the climbing fiber input to the floccular complex parallels our finding of an up-down asymmetry in the size of single-trial learning and suggests a floccular locus of single-trial learning in pursuit.

Learning on multiple timescales

Historically, motor learning has been synonymous with cerebellar learning and cerebellar learning has been viewed mainly in terms of climbing fiber mediated long-term depression of synapses from parallel fibers onto Purkinje cells. Analysis of the neural basis for learning in several behavioral systems, however, has suggested that there are at least two sites of learning: one in the cerebellar cortex that could be related to long-term depression mediated by climbing fibers and one in the deep cerebellar nuclei (Hirata and Highstein 2001; Lisberger 1994; Medina and Mauk 2000). Further, there seem to be multiple sites and mechanisms of motor learning in the vestibuloocular reflex (Boyden et al. 2004; Ke et al. 2009) and pursuit eye movements (Medina and Lisberger 2008), as well as multiple components in saccades and arm movements (Ethier et al. 2008; Smith et al. 2006).

At the same time, exploration of cellular mechanisms of plasticity within the cerebellum has identified a wide range of potentiation and depression mechanisms that operate over a range of timescales and at diverse sites in the cerebellar microcircuit (Aizenman and Linden 2000; Hansel and Linden 2000; Jorntell and Ekerot 2003; Pugh and Raman 2008; Telgkamp and Raman 2002; Zhang and Linden 2006). Finally, single-trial depression occurs during learning in the simple spike responses of floccular Purkinje cells (Medina and Lisberger 2008) and, on the basis of its rapid induction and short duration, seems to be related to but different from cerebellar long-term depression (Lev Ram et al. 1995; Linden et al. 1991). One appealing concept is that short-term learning in the cerebellar cortex is a step toward induction of long-term learning in the deep cerebellar nuclei (Hirata and Highstein 2001; Medina and Mauk 2000; Miles and Lisberger 1981), although single-trial learning at the parallel fiber to Purkinje cell synapse could also lead to more permanent effects at the same cellular locus.

We suggest that single-trial plasticity and learning, which occur quickly but are forgotten within 10 s, may be a feature of all motor learning systems. On the basis of the properties of the single-trial learning, we suggest that motor learning results from many processes that work together over multiple timescales and that single-trial changes are the result of one of those processes. Single-trial learning would operate at the start of motor learning and could be a companion or a necessary prerequisite for gradual long-term changes that are remembered for longer times. The similar conclusions of fast and slow components of learning across many different movement systems link nicely to the knowledge of potential sites and mechanisms of learning in pursuit eye movements, suggesting that similar neural mechanisms might operate across movement systems.

GRANTS

This research was supported by the Howard Hughes Medical Institute and National Institute of Mental Health Grant MH-077970.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).