Saccade Adaptation as a Model of Flexible and General Motor Learning

Abstract

The rapid point-to-point movements of the eyes called saccades are the most commonly made movement by humans, yet differ from nearly every other type of motor output in that they are completed too quickly to be adjusted during their execution by visual feedback. Saccadic accuracy remains quite high over a lifetime despite inevitable changes to the physical structures controlling the eyes, indicating that the oculomotor system actively monitors and adjusts motor commands to achieve consistent behavioural production. Indeed, it seems that beyond the ability to compensate for slow, age-related bodily changes, saccades can be modified following traumatic injury or pathology that affects their production, or in response to more short-term systematic alterations to post-saccadic visual feedback in a laboratory setting. These forms of plasticity rely on the visual detection of accuracy errors by a unified set of mechanisms that support the process known as saccade adaptation. Saccade adaptation has been mostly studied as a phenomenon in its own right, outside of motor learning in general. Here, we highlight the commonalities between eye and arm movement adaptation by reviewing the literature across these fields wherever there are compelling overlapping theories or data. Recent exciting findings are challenging previous interpretations of the underlying mechanism of saccade adaptation with the incorporation of concepts including prediction, reinforcement and contextual learning. We review the emerging ideas and evidence with particular emphasis on the important contributions made by Josh Wallman in this sphere over the past 15 years.

As humans rely heavily on vision, it will come as no surprise that amongst the knowledge recorded on papyrus in ancient Egypt was documentation of various disorders of vision and accompanying treatments (Bryan, 1930). Ancient anatomists such as Galen were also aware of the extraocular muscles and their actions (May, 1968). Fascinatingly, however, the rapid discontinuous movements of the eyes, known as saccades, were not recognized until the pioneering work of Porterfield (1737), who documented fast and slow phases of nystagmus through the use of afterimages. It is now recognized that saccades are one of the most common movements in the human repertoire of behavioral output. They are used to very rapidly (up to 800°/s) orient the relatively small area (1°) of our best vision (the maximally receptor-dense region of the retina called the fovea) to objects of interest in our environment.

1. The origins of saccade adaptation

Research regarding saccades and eye movements in the late 1960s and the 1970s established the foundation of our knowledge on saccade adaptation. With the technological advances and the development of better devices to measure eye movements, it was possible to observe that there was variation in saccade gain from individual to individual (gain: the ratio of saccade amplitude to target eccentricity or step-size), and it was soon realized that both pathological (Kommerell et al., 1976; Abel et al., 1978), and normal saccade gain (McLaughlin, 1967), could be modified by experience. Kommerell, Abel and their collaborators (Kommerell et al., 1976; Abel et al., 1978) documented patients who made low-gain saccades with one eye, and relatively normal movements with the other. They both found that forcing the patient to view the world through only the weakened eye, over several days, had the effect of restoring relatively normal saccadic gain. Meanwhile, McLaughlin (1967) found that, by arranging saccades to trigger consistent 1° intr a-saccadic target steps (ISSs) backward relative to the direction of the movement, saccadic gain could be noticeably decreased in just a handful of trials. These were the beginning of the study of what is now known generally as saccade adaptation. An example of a typical McLaughlin-style, ISS adaptation session is presented in Figure 1. At the time, because of the extremely rapid adjustment that McLaughlin demonstrated (on the order of minutes) and the relatively slow adjustment observed by Kommerell and Abel (on the order of days), it was thought that these two cases of gain adjustment were supported by distinct mechanisms. However, it was later shown by Scudder and colleagues (1998) that the two forms of adaptation shared the same time course, when compared under similar conditions.

Figure 1.

Illustration of gain change during saccade adaptation. A typical intrasaccadic step (ISS)-adaptation session. Saccade gain is slightly bellow 1 in the baseline-phase, where there are no ISSs. In the adapt-phase, gain decreases progressively to an asymptote. Note that adaptation is incomplete. In the post-phase, where there are again no ISSs, gain progressively increases toward the baseline level.

Another important result came from Optican and Robinson (1980) who weakened extraocular muscles in monkeys, producing similar saccade gain asymmetries to the patients of Kommerell and colleagues: low saccade gain in one eye, and normal saccade gain in the other. Motivated in part by the knowledge that humans with certain forms of cerebellar degeneration show lasting saccade dysmetria (Zee et al., 1976), Optican and Robinson were able to show, via ablation, that the ability to adapt saccade gain required midline cerebellar structures (oculomotor vermis and fastigial nuclei). A host of subsequent studies have confirmed the importance of the cerebellum in saccade adaptation (for a review, see Robinson, Fuchs, and Noto, 2002).

Following the discovery of saccade adaptation and the establishment of the main brain structures responsible, came the question of the underlying mechanism. Saccade adaptation occurs in response to the detection of an error or a discrepancy between the planned movement and its outcome. But one may wonder 1) is this error a simple visual error?, 2) how are errors detected? and 3) how are errors used? For example, saccade adaptation has long been thought of as a very simple error feedback (“servo mechanism”) that operates to reduce error over time, but is this error a simple visual error, or does it incorporate higher order information or predictions? How are error signals detected and selected in a cluttered visual environment? How are the spatiotemporal statistics of the history of errors taken into account, and to what extent can different contexts influence the adaptive use of errors? The rest of this review will focus on the evidence emerging to answer these three conceptual questions.

2. What is the error signal?

2.1 From retinal error to prediction error

In the absence of pathology, or an experimental perturbation, saccade accuracy stays roughly constant over a lifetime (Warabi et al., 1984; Munoz et al., 1998) despite changes in the oculomotor plant during ageing, such as degradation of the extraocular muscles (Miller, 1975; Muller-Hocker et al., 1992; McKelvie et al., 1999). Ergo, saccade gain is actively maintained by the brain. Further, Miller and collaborators (1981) showed how specific adaptation could be to a movement direction and amplitude, suggesting the existence of some “error signal” which is instructive as to the direction and magnitude of ongoing saccade dysmetria. The distance between target and gaze at primary saccade completion or retinal error (Optican and Robinson, 1980) arose naturally as a candidate for the error signal. Another possibility was that the requirement to produce corrective saccade(s) to acquire the target was the actual error signal (see, for example, Albano and King, 1989). In other words, one question is whether the adaptation mechanism cares about reducing the visual or motor consequences of inaccuracy?

By showing that adaptation can occur in the near absence of corrective saccades, two studies demonstrated that post-saccadic visual information must be the driving force behind saccade adaptation (Wallman and Fuchs, 1998; Noto and Robinson, 2001). However, the conclusion that the error signal must be visual in nature does not amount to a conclusion that retinal error is the error signal determining saccade adaptation. Bahcall and Kowler (2000) found that when they instructed subjects to saccade part-way to a target that then underwent a backwards ISS, gain-decrease adaptation occurred, despite the conclusion of each primary saccade in a positive (onwards) retinal error. They explained these results in terms of the ISS introducing a prediction error between the expected and actual visual error after the saccade. Given the evidence for active internal monitoring during saccades of its trajectory (for a review, see Sommer & Wurtz, 2008), the idea that an internal prediction of error might govern adaptive changes to saccade gain after the saccade is highly plausible (Chen-Harris et al., 2008).

2.2 General concept of sensory prediction errors (SPE)

Indeed, in the study of motor learning more broadly, there is now wide agreement that adaptive modifications of behavioral output are driven by sensory prediction errors (SPEs): the difference between the predicted and actual sensory consequences of a movement (Shadmehr et al., 2010). This central concept in modern motor learning theory is illustrated in Figure 2, which demonstrates how the SPE caused by an ISS would be identical regardless of whether a conventional saccade adaptation paradigm was used or the “part-way” paradigm of Bahcall and Kowler (2000) in which subjects deliberately aim short of the target, and can therefore explain the similar adaptation in both cases despite the opposite retinal errors.

Figure 2.

Schematic of the Sensory Prediction Error (SPE, magenta) concept. Whether considering the “normal” saccades (green) used in many studies of saccade adaptation induced by intrasaccadic target steps (ISSs, dark red), or the “part-way” saccades (blue) used by Bahcall & Kowler (2000), the resulting SPE is unchanged, despite a large difference in the retinal error (orange), thanks to offsetting by the sensory prediction (purple). Note that the SPE and the ISS are shown as identical in this example because: (1) the illustrated sensory prediction does not account for noise in the pre-saccade estimate of target eccentricity, and (2) the internal estimate of saccade amplitude (on which the sensory prediction must also rely) is illustrated as accounting for the saccadic execution noise, which it cannot. In actuality, the SPE would vary in magnitude relative to the ISS as a result of both noise sources.

A particularly striking example of the SPE demonstrated that its action could override a conscious strategy in a pointing task (Mazzoni and Krakauer, 2006). In this study, subjects performed rapid hand movements and visual feedback of these were rotated by 45 degrees anticlockwise during the movements. One group of subjects was unaware of the rotation and adapted their hand trajectories progressively clockwise to counter the imposed rotation. A second group was informed of the rotation and told to aim 45 degrees clockwise of the initial target location. This group thus landed on target at the start of the session thanks to this conscious strategy, but amazingly adapted in the same way as the first group, leading to increasingly inaccurate movements (gradually adapting to nearly the same extent as the unaware group). The commonality between the aware and unaware group was that an SPE was present in both, convincingly demonstrating the explanatory power of this idea. In the same way that the instruction to saccade “part-way” in the Bahcall and Kowler (2000) study caused different retinal errors but resulted in the same SPE, here the instructions caused different angular errors, but resulted in the same SPE.

The SPE concept can also explain how an undershoot bias persists in saccade aiming. In non-adaptation experiments in which subjects simply saccade to targets (no ISS), subjects typically undershoot the target by about 10%, frequently followed by short latency onward corrective saccades. It has been reported that arranging for the target to land on the fovea at the end of the saccade leads to a progressive decrease in saccade size (Henson, 1978; Robinson et al., 2003), as if to deliberately re-establish the undershoot (although Robinson et al. proposed their data were due to an artefact of their monkeys making thousands of saccades in the dark causing gain to drift down). The reason for this undershoot is still uncertain, but may reduce the overall time spent in flight acquiring the target, and thus the “costs” of poor vision during movement (Harris, 1995). Regardless of why we undershoot, the explanation of why the undershoot is not eradicated by a saccade adaptation mechanism can be ascribed to the SPE: although there is a retinal error at the end of the saccade there is likely no error in the sensory prediction (SPE =0). Clearly, this argument relies on accurate predictions of saccade landing positions, and there is good evidence that these are available (Collins et al., 2009; Collins, 2010); most likely arising from efference copies of the motor command rather than proprioception (Lewis et al., 2001).

2.3 Evidence in saccade adaptation in favor of SPE

The strongest evidence to date for SPE in saccade adaptation comes from three recent studies. Wong and Shelhamer (2011a) took advantage of the undershoot bias to put retinal error and prediction error into direct competition. Because subjects undershoot by nearly 2 degrees to a 20 degree eccentric target, they could backstep the target by ~1 degree during the saccade (negative ISS) while leaving a positive retinal error (which they kept constant by varying the ISS trial-by-trial). They found subjects adapted backwards in accordance with SPE rather than forwards as retinal error would predict.

In their recent study, Collins and Wallman (2012) directly tested the relative effectiveness of retinal error and prediction error in eliciting adaptation by comparing the amount of saccade gain change occurring during two different sessions. In the first session, the target was turned off if the saccade was shorter than the running median amplitude (or in a different set of experiments, was longer than the median) i.e. remained on for large saccades (or in a different set of experiments, for small saccades). In this session subjects therefore experienced mostly negative retinal errors which led to a decrease in saccade amplitude. Note that there were no target displacements, therefore, when the target was not extinguished, it was at the expected position. In the second session, targets were displaced (during saccade execution) to a position that reproduced the trial-by-trial retinal error experienced by the subject in the first session. Despite the retinal error of the first and second sessions being identical, subjects adapted much more during the second session (Figure 3). The key difference was that, in the second session, the target did not appear at the predicted location. This result supports the hypothesis that a prediction error signal is mostly responsible for saccade adaptation.

Figure 3.

The effect of prediction errors on saccade adaptation. Targets were always extinguished upon saccades made to them. In two experiments, during Session 1, targets were either re-illuminated in their original position if the saccade was larger than the running median saccade amplitude (as illustrated by the light versus dark shading of the Large-On distribution, inset), or re-illuminated if the saccade was smaller than the median saccade amplitude (Small-On, inset). Hence, theoretically, there was no prediction error in Session 1 as the target reappeared in its original position. In Session 2 of each experiment, the target was re-illuminated on the same trials as in Session 1 (irrespective of landing position) and at a displaced position to recreate the exact pattern of retinal errors experienced in Session 1, but now with prediction errors. Percent adaptation was much greater in Session 2 across the seven subjects (circles), as shown by the group averages (dark gray lines) and the 95% confidence intervals (lighter gray boxes). Data replotted from Collins and Wallman 2012 with permission.

The most recent study to support SPE in saccade adaptation does so by showing that it is the ISS, not the retinal error, that drives adaptation (Herman et al., 2013). Because of the close connection between ISS and SPE (Figure 2), this evidence for ISS is strongly consistent with the SPE hypothesis. Herman and colleagues studied the effects of ISS smaller than the endpoint variability of saccades. In separate sessions, they explored five ISS values from 0.1 deg to 1 deg (to initial target jumps of 10 deg). Similar to Wong and Shelhamer (2011a), they found that despite frequently undershooting the final target location subjects adapted downwards and that retinal error was a bad predictor of adaptation gain changes in regression analyses. Importantly, they showed that saccade adaptation is proportional to the ISS, with gain decreasing by an approximately constant 70% of the ISS. This implies that the well established fact that gain changes asymptote at less than complete adaptation (Straube et al., 1997; Robinson et al., 2003) is neither the consequence of saturating motor learning, nor minimizing retinal error to within a given tolerance level: as ISS size increases, a motor limit would result in gain changes decreasing as a proportion of ISS, whereas a retinal error limit would imply an increasing level of adaptation when expressed as a proportion of ISS. While supporting the SPE hypothesis, this new study highlights a critical and basic outstanding issue: what is the error sensitivity of saccade adaptation and its relation to ISS?

Remarkably, little is known about the error sensitivity of human saccade adaptation to different ISS. The only other human data were reported from two subjects and showed similar effects to the Herman study, adapting by a constant 60% of their two much larger ISSs (25% and 50%; Miller et al., 1981). The lone monkey report used a different design: fixed within-session retinal errors, such that ISSs increased continually as adaptation progressed in order to maintain a fixed post-saccadic retinal error (Robinson et al., 2003). Hence, it is hard to directly compare the monkey and human data, but the general patterns appear different in that the monkey data showed a rapidly increasing adaptive sensitivity as errors increased to 25% of the initial step size, before decreasing for larger errors. The issue of error sensitivity of adaptation is not only a fundamental empirical question, but has many potential theoretical implications beyond the asymptote example above. If human saccade adaptation data is found to have a nonlinear sensitivity to error, does this reflect larger errors being credited to external sources of error rather than internal sources and thereby inducing weaker adaptive change (Ethier et al., 2009, Wei and Kording, 2009)? Many models make an assumption that error sensitivity (“cost functions”) increase quadratically with error, but is this primarily a mathematical convenience (Todorov, 2005) or a biological reality?

2.4 Reinforcement learning versus error feedback

There is much evidence in monkeys that reinforcement schedules act on saccade and can increase saccadic speeds, reduce reaction times, and straighten trajectories (Lauwereyns et al., 2002; Takikawa et al., 2002; Watanabe et al., 2003). Similarly, it has been shown in humans that reward may alter saccade speeds (Xu-Wilson et al., 2009), and saccade latency distributions (Madelain et al., 2007) and variability in saccade gain (Paeye and Madelain, 2011). Regarding saccade accuracy, one very biological ‘cost function’ is how visibility changes with eccentricity. Perhaps clear vision of the target, by allowing to accurately perform visually guided tasks, is rewarding and it is the increased information input close to the fovea that drives adaptation rather than a minimization of retinal error per se. Because visibility and error are usually correlated, dissociating between access to visual information and error-minimization would be difficult in many circumstances. One possibility to disentangle visibility from reinforcement is to reward specific saccadic gain in the absence of visual error.

A recent study by Madelain and colleagues (2011) found that it was possible to use amplitude-dependent reinforcement to drive changes in saccade gain while abolishing the post-saccadic position error. For example, in one paradigm the target was always absent at saccade completion and only a tone reinforcer was given to the subject when their saccade amplitude met the criteria. The authors observed a progressive change in saccade amplitude, although slower than conventional adaptation, showing that post-saccadic visual feedback about the target location is not required for saccade adaptation. Importantly, conventional and reinforced adaptations were similar in terms of magnitude of gain change, degree of transfer to new spatial locations and new amplitudes, as well as speed and amount of recovery. They proposed that conventional adaptation may rely on reinforcement learning in which clear vision of the target constitutes the reinforcer. It is somewhat surprising that this work is the only one to have explored the possible role of motivation or task demands in saccade adaptation, or more general forms of associative learning such as operant or classical conditioning. This is probably due to the fact that the dominant presumed function of saccade adaptation has always been a simple motor recalibration mechanism. The work of Madelain and colleagues (2011), and the work of Herman and colleagues (2009) outlined in section 4, points towards greater adaptive flexibility and the possible use of saccade adaptation as a model for general learning. The notion that reinforcement may also have a role in arm movement learning is now emerging (Dam et al., 2013). This new area of study opens many new questions regarding the interactions of reinforcement and prediction errors in driving adaptation. More generally, is some sort of internal reinforcement signal an implicit component of all error-based adaptation?

2.5 Attention as an error signal

Finally, another putative description of saccadic error (which is not incompatible with SPE) is the comparison between gaze location after the saccade and the locus of attention, rather than the target location. Attention is intimately linked to saccade targeting since it is required for object selection, and is thought to move to the target location before every saccade (Castet et al. 2006; Deubel and Schneider 1996; Hoffman and Subramaniam 1995; Kowler et al. 1995). The interplay of attention and saccades might also play a role in maintaining the accuracy of the latter. After a saccade the oculomotor system would need only to know where the eyes were with respect to the locus of attention, rather than needing to keep track of all the other potential targets. Attention could be a placeholder that allows the oculomotor system to automatically keep track of the target across the saccade (Cavanagh et al., 2010). It has been previously shown that only the attended visual information after a saccade is utilized in saccade adaptation (Ditterich et al. 2000). If the locus of attention coincides with that of the fovea just after a saccade, the saccade could be regarded as accurate, even if not on the visual target per-se; if the fovea is consistently on the far side of the locus of attention, the saccade could be regarded as too big, and this error signal could induce downward adjustment of subsequent saccade amplitudes. Interestingly, when the presaccadic shift of attention has itself been adapted, by measuring the time course of shifting of attention and shifting cues at this time, long before the saccade, it led to significant decreases in saccade amplitudes (McFadden et al., 2002). Hence, this provides evidence suggesting that adapting attention can adapt saccade size.

The main purpose of invoking a role for attention in the error signal for saccade adaptation is in cluttered visual environments where the selective filtering power of attention can be useful in keeping track of relevant error signals. This is the topic we explore next.

3. How does the brain detect errors?

In addressing the question of what constitutes the error signal, it was assumed that localizing the target after the saccade (for the purposes of calculating retinal error, for instance) is trivial. While it is true that in the natural environment, we do sometimes make saccades to lone, easily discriminated stimuli, it is also common to direct our gaze to one of many similar objects, or to explore a single object, or in fixating portions of less structured visual stimuli such as textures or patterns. In the laboratory, however, saccade adaptation has traditionally been studied using small discrete targets – deemphasizing the potential challenges faced by the oculomotor system in extracting an error signal from the information-dense image it is confronted with upon saccade completion. That is, in most studies of saccade adaptation, it is safe to assume that bottom-up vision constitutes the “actual sensory information” for SPE, what happens when more is required?

3.1 Isolated target-objects

As a first step to richer target environments, Madelain and collaborators (2010) confronted subjects with two targets after a saccade (the original, and a similar distractor) and found that adaptation was selective for the error associated with the original target. If the target was not displaced during the saccade, the adaptation mechanism ignored the position error associated with the distractor appearance; conversely, if the target was displaced, with distractor appearing in its place, adaptation followed the displacement of the target (Figure 4). Importantly, this target-specific adaptation was indistinguishable from adaptation with a single target, despite the presumed additional visual processing that would be required to calculate an error specific to the target, or ignore the distractor. This also underlines the critical role of object identity in saccade adaptation, which is surprising from the conventional context of saccade adaptation as a simple servo-mechanism, but which is compatible with the idea that seeing the post-saccadic target reinforces gain changes (Madelain et al., 2011).

Figure 4.

Saccade adaptation is selective for target identity. A- Illustration of the temporal sequence of the protocols used in Madelain et al. 2010. After a primary 10deg step, on saccade detection the target (a green circle in this example) could 1) step-back by 2deg and be replaced by a distractor (left panel) or 2) remain in place, a distractor appearing 2deg on its left (right panel). B- Individual saccade gains and lowess smoother for a single subject in 3 conditions. Red: Step-back condition, with the appearance of a distractor at the initial target location. Blue: Conventional step-back condition, without the appearance of a distractor. Green: No step-back of the target, but distractor appearing on the left of the target. The gray zone indicates the adaptation phase. Figure reproduced from Madelain et al. 2010 with permission (copyright: Association for Research in Vision and Ophthalmology, ARVO ©).

If active maintenance of saccadic accuracy functions successfully in the natural environment – as appears possible (Kommerell et al., 1976; Abel et al., 1978) – saccades to objects may represent a uniquely good opportunity for detecting errors in the oculomotor system. Such movements have a well-defined goal, rendering the process of determining saccadic dysmetria relatively straightforward compared to making saccades to unstructured stimuli or within a larger object. In accordance with this, Collins and collaborators found that ISSs only cause adaptation when saccades are made between objects rather than within an object (Collins et al., 2007). Their finding is supported by work on two-saccade sequences, which demonstrates that secondary saccades only compensate for ISSs during primary saccades when these saccades are made between, not within, objects (Vergilino-Perez et al., 2006).

3.2 Targets on backgrounds

Those studies that have explored adaptation with both a target and a background present have shown that objects do play a significant role in saccade adaptation. In their study of the interactions between target and background during saccade adaptation, Ditterich and colleagues (1999) found that the size of a superimposed target determined the extent to which intra-saccadic background shifts contributed to adaptation. When the target was small, shifts of the background had no effect on adaptation, but when the target was a large circle, shifting the background along with the target caused greater adaptation than shifting it in opposition to the target. Thus, the saccadic error resulting from intrasaccadic displacement is not automatically computed using all available information. This result instead suggests that target-specific visual information is actively selected to estimate error. In keeping with these findings, Robinson and colleagues (2000) found that adaptation with a point-like target in monkeys was reduced when a background was present compared to the case of a target alone, regardless of whether it moved with the target or remained stationary. This latter finding suggests that the addition of the background may increase the difficulty of localizing the target post-saccadically, or calculating an error associated with it, as delaying the presentation of error is known to reduce its efficacy (Shafer et al., 2000).

More recently, Madelain and collaborators (2013) investigated the effects of the background shift on adaptation but, for the first time, using natural image scenes. Moreover, during the saccade execution, the target and background were masked to exclude motion signals. In agreement with previous studies, it was only displacements of the target that determined gain changes; displacements of the background neither induced adaptation in the presence of a static target, nor added to the gain changes when the target was also displaced. However, if the target was turned off at the start of the saccade and remained off, the displacement of the background did induce some adaptation (Figure 5).

Figure 5.

Effects of natural scene backgrounds and/or targets displaced during saccades. Mean and standard deviation of the percent change of gain for each subject in the 4 experiments of the study from Madelain and collaborators (2013): Background displaced - Target static (BdTs), Background static - Target displaced (BsTd), Background displaced - Target displaced (BdTd) and Background displaced - Target extinguished (BdTe). Figure reproduced from Madelain et al. 2013 with permission (copyright: Association for Research in Vision and Ophthalmology, ARVO ©).

Altogether, results exploring the impact of a background-image on saccade adaptation show that target selection is very influential. But what happens when there is no clearly defined target? The results from the Background displaced - Target extinguished condition in Madelain and collaborators (2013) study indicate that, in the absence of a target, information from the background is used. Two previous studies also found that intra-saccadic shifts of an unstructured background alone can cause adaptation (Deubel, 1991; Ditterich et al., 1999). This again suggests that the impotence of background shifts (in contributing to adaptation) in the presence of a target is the result of an active selection of target information (or suppression of non-target information). A notable difference between these studies, however, is that Madelain and colleagues’ masking during the saccade implies that their finding relies exclusively on trans-saccadic comparison of background features, i.e. comparing the spatial properties of the pre- and post-saccadic visual images. The earlier studies could have been driven by intra-saccadic motion signals; i.e. by detecting target or background shifts during the saccade. Although intra-saccadic displacement detection is severely impaired (compared to such detection during fixation) due to saccadic suppression of vision (for example, Bridgeman et al., 1975), it has been shown that low frequency motion signals can be detected during saccades (Castet and Masson, 2000). A recent study even suggested that intra-saccadic vision may contribute to saccade adaptation (Panouilleres et al 2013).

These results about the effect of a background on saccade adaptation raise a second issue of the visual mechanism of how errors are detected more broadly. Experiments exploring the use of visual landmarks in pre- and post-saccadic localizations of the target have suggested that they can influence the perceptual detection of intrasaccadic target displacement. Deubel (2004) found that intrasaccadically shifting nearby objects along with a target modulated the perceived direction of the target’s displacement (at least when the target is extinguished during and briefly after a saccade). However, it is not clear if this type of perceptual intra-saccadic detection is related to that relied upon by the oculomotor system for adaptation. When such targets are not extinguished during and after the saccade, perceptual reports regarding intrasaccadic displacement are much poorer (Deubel et al., 1996). Indeed, it seems that subjects are more apt to report a continuously visible stimulus as intrasaccadically stable regardless of whether it moved or not (Deubel et al., 2010). Also, though extinguishing these targets during and after a saccade facilitates detection of their displacement (Deubel et al., 1996), it also significantly decreases the impact of that displacement on adaptation (Shafer et al., 2000).

In summary, it seems that in the presence of a spatially well-defined target, only the visual information arising from the target (or its immediate proximity) are used in the detection of error i.e. the background has little influence. Meanwhile, if such a target is absent, the oculomotor system does not limit the scope of the visual information used in detecting errors. Furthermore, there is a broad gap in our understanding of the relationship between the perceptual detection of error, and that at the service of saccade adaptation. On the one hand, subjects are essentially unable to detect intrasaccadic image displacement (Deubel et al., 1996); on the other, such displacement can cause adaptation (Deubel et al., 1991; Ditterich et al., 1999; Madelain et al., 2013), even when the image is of a point-like target (Panouilleres et al., 2013). What is the difference between perceptual error detection and such detection at the service of adaptation? More work is required to clarify this important distinction.

Given the context of this review, we end this section by noting that the impressive selectivity of the foveal target over its background shown in Figure 5 can be cast as conceptual evidence in favour of a reinforcement function, rather than a motor recalibration function, of saccade adaptation. If the sole function of saccade adaptation were to establish during development and then maintain accurate calibration throughout life, why would the brain ignore error signals from >99% of the visual field? Muscle weakness or fatigue would lead to the whole visual field being inaccurate by the same amount. In other visual error feedback systems such as emmetropization, the peripheral signals play an important role (for review see Smith, 2013, in this issue). Clearly, foveal vision has much more value for task-oriented behaviour, be this purely visual information accrual or in guiding arm movements and the like. Due to motor variability, some saccades bring the fovea close to the target and some further away. From the point of view of a conditioning system, the presence of the target near the fovea after the saccade could be seen as a reward and the corresponding motor plan would be reinforced. However, reinforcement and calibration functions are not mutually exclusive propositions; a point that we return to below.

4. How does the brain use errors?

Though framing saccade adaptation with the question-headings chosen here attempts to address different aspects of the adaptation process independently, it is likely that processes such as the calculation or detection of error and its utilization in driving adaptation are intimately intertwined. For instance, using a point-like target, Havermann and Lappe (2010) found that increasing noise in retinal error by systematically varying it from trial to trial attenuated the induced adaptation (compared to the case with no such additional variation), though the average retinal error was identical between compared conditions. In the following, while we attempt to address only the way that errors are used in driving adaptation, we recognize that it may be impossible to do so without considering the nature of the error signal or its detection.

4.1 Vector and context specificity

Many experiments have demonstrated the ways in which error can be used to adapt gain in a saccade-specific way. Several different types of specificity have been demonstrated: leftward saccades may be adapted independently from rightward saccades (Semmlow et al., 1989), it is possible to adapt different saccade amplitudes in the same direction (Miller et al., 1981; Frens and Van Opstal, 1997), and several studies have emphasized some degree of independence in adapting different saccade types (for example reactive versus voluntary saccades; Erkelens and Hulleman, 1993; Deubel, 1995a, 1995b; Fujita et al., 2002; Gaveau et al., 2005; but see Fuchs et al., 1996; Deubel, 1999; Hopp and Fuchs, 2002; for reviews see Hopp and Fuchs, 2004; Pélisson et al., 2010).

It is possible that adaptation specificity (or lack thereof) may fall under the more general aegis of context; perhaps differential adaptation of distinct saccade types is the manifestation of those types of saccadic performance constituting distinct contexts for action. In the motor learning literature writ large, it has been suggested that the cerebellum may support the formation, utilization, and adjustment of distinct sensory predictions through distinct, paired “forward” and “inverse” models (Wolpert et al., 1998). Forward models are used to predict the sensory and motor consequences of movement (position, velocity, et cetera) using the outgoing motor command and current sensory data; inverse models are used to determine the appropriate motor command for a given desired motor consequence. Note that though this formalism has more traditionally been used to represent ideas about feedback corrections that occur during a movement, conceptually it applies equally well to the offline error feedback correction needed to support saccade adaptation (Chen-Harris et al., 2008). The adjustment of the forward model could then be made based on the difference between the forward model prediction of the sensory consequences of a movement and the actual outcome, which is essentially SPE, discussed above. Such paired forward and inverse models could support contextual motor learning if each context has independent pairs of models. Work from several labs has demonstrated that saccade adaptation can be sensitive to a variety of contextual cues. These include target eccentricity and depth (Chaturvedi and van Gisbergen, 1997), horizontal and vertical orbital eye position (Shelhamer and Clendaniel, 2002a; Alahyane and Pélisson, 2004; Zimmermann and Lappe, 2011) or head orientation (Shelhamer and Clendaniel, 2002b). Though Deubel (1995a) found that a target’s visual appearance was not an effective contextual cue, this result was challenged by Herman and collaborators (2009).

Herman and collaborators (2009) showed that selective adaptation based on the target visual appearance was possible. They used two different targets, a flickering or non-flickering dot, one of which would back-step during the primary saccade and not the other one. They showed a difference in the gain between the movements toward the two types of targets, with a large decrease of saccade amplitude for the target which was making the back-steps (Figure 6; on average, subjects decreased by 15% from their baseline gains to the step-back target). The effect of context was not absolute with some transfer to the target that did not step back (decreasing by 6% on average from the baseline gain), but the contextual difference was highly significant. This type of transfer is typical in contextual saccade adaptation due to the dominance of gain-decreasing over gain-increasing (or in this case gain-stable conditions), and the contextuality is usually only assessed by average gain performances. A more stringent test of contextuality is to look at individual transitions between contexts: does the context, which is defined before the movement, have an immediate effect on the first saccade after context change, or are several movement outcomes (e.g. several trials of the not stepping back context) necessary to effect change? Analyzing just the average behavior cannot detect this stronger contextuality. Herman and colleagues found that early in the adaptation switching between short runs of each context did not give rise to this immediate transitional adaptation, but that this strong contextuality emerged gradually over several hundred trials, and persisted very significantly in a random switching phase. Interestingly, in a second experiment, they demonstrated that this “differential feedback” (switching between short blocks of each context) was critical to establishing the contextual adaptation. If one target was presented on ~90% of trials, always stepping backwards on saccade, with the other target not stepping back on the other 10% of trials (randomly distributed across the adaptation session), no contextual adaptation developed: the more infrequent context had identical gains to the frequently presented target type.

Figure 6.

Contextual saccade adaptation driven by purely visual properties of the target. The individual saccade gain for each trial (dots) in one experiment in a representative subject are plotted with their moving averages (+/− 95% confidence intervals, shaded curves) for each context. On each trial, the target was either a flickering spot (luminance switching between two values at 5 Hz), or a non-flickering spot target. In the Blocked phase, 3–10 consecutive trials were of one target type, before switching to the other target context; in the Random phase, target contexts switched randomly trial-to-trial. After the Baseline phase, one target type always stepped backwards during saccades (ISS), and the other type did not (No ISS). The gradually separating curves show the build-up of contextual adaptation to the target types: subjects learned to associate the visual context into their movement gain. Data replotted from Herman et al. 2009 with permission.

These data are greatly at odds with the traditional view that saccade adaptation is essentially for purely motor recalibration purposes. On the one hand, one can easily imagine that contexts such as orbital position or head tilts could be useful for a motor recalibration mechanism since different orbital positions or head tilts require different sets of motor commands. On the other hand, it is much harder to imagine how forming different motor outputs based on visual feature contexts would be useful to a motor recalibration mechanism. Hence, the surprisingly strong visual contextuality found by Herman and colleagues implies that saccade adaptation, even if its raison d’être is to service motor calibration, can be used for other purposes. Perhaps other processes in the brain, such as those involved in associative learning, can augment or piggy-back on top of the existing saccade adaptation machinery. Evidence supporting augmentation of existing motor contexts can be seen by reports that visual appearance (as well as an auditory and a more ambient context) can enhance the effects of a head-orientation-defined context (Beaton et al., 2010). The Herman finding that the pattern of differential feedback was crucial to establishing the visual contextuality is also more consistent with more implicit pattern recognition types of learning than would be expected of simple recalibration.

This question of augmenting adaptation or being a part of “true” saccade adaptation is reminiscent of the earlier discussion raised by the new evidence for operant conditioning in saccade adaptation (Madelain et al., 2011). Our intuition is that the visual contextual adaptation would fall more under an augmentation of adaptation, whereas operant conditioning with its close connection between retinal error and visual reward would be less of an augmentation process and more integral to “true” adaptation. However, ongoing experiments may yet reverse that viewpoint, or point to both phenomena being augmentative to a simpler error feedback calibration process. Although those are crucial issues, at a pragmatic level, if the saccade system can change its gain parameters in these more colourful ways, and we can measure them, our view is that that is its own reward. It would enable saccade adaptation to be a much broader model for learning and a more useful tool for studying behavioural neuroscience. The many details of the functional connectivity that have been mapped out in the saccade system might inform learning models, and vice versa.

4.2 Spatiotemporal properties of error

What of the use of error within a context, however? Putting aside for a moment the idea that error is used in a specific way, we must also consider how error might be used to adjust saccadic gain in one set of circumstances. Specifically within the field of saccade adaptation, several studies have shown that gain-increase adaptation is slower and less substantial than gain decrease, suggesting perhaps that onwards errors are used differently from backwards errors (Miller et al., 1981; Deubel et al., 1986; Fitzgibbon et al., 1986; Deubel, 1987; Straube et al., 1997; Scudder et al., 1998; Kroller et al., 1999). Recent work suggests that gain-decrease adaptation may act by adjusting the online control of saccades, while gain-increase adaptation more closely resembles a remapping of target position (Ethier et al., 2008, but see Straube and Deubel, 1995; Alahyane and Pélisson, 2005). An alternative explanation came from Harris (1995) who showed that a model that monitors total-flight-time over the course of many saccades, reproduces the gain-increase gain-decrease rate asymmetry.

The way that errors vary over time may also influence their impact on adaptation. As mentioned above, Havermann and Lappe (2010) found that increasing spatial noise in retinal error (in a fixed-error, variable-ISS paradigm) decreased induced adaptation, despite mean error remaining constant. However Srimal and colleagues (2008), findings in modeling adaptation with constant or random ISSs suggested that more spatially variable errors are used no differently than those with less variability. Meanwhile, it appears that gradually introduced errors are more effective in driving changes than abruptly introduced errors, at least for gain-increase adaptation (Wong and Shelhamer, 2011b). Indeed, several motor learning studies have suggested differences in the impact of gradually versus abruptly introduced errors (Kagerer et al., 1997; Klassen et al., 2005; Hatada et al., 2006; Michel et al., 2007; Huang and Shadmehr, 2009). This may be because smaller errors are treated differently from larger errors (Wei and Körding, 2009; Criscimagna-Hemminger et al., 2010; Robinson et al., 2003). It has also been suggested that the use of error is dependent on whether the error is ascribed to an internal or external cause (Berniker and Kording, 2008).

Most generally, it has been suggested that errors are recorded by memory systems with multiple different timescales of “remembering” and “forgetting,” and that error is stored (remembered) and retained (not forgotten) at rates which correspond to the timescale of the error-source (Kording et al., 2007; Joiner and Smith, 2008; Huang and Shadmehr, 2009). That is, memories are stored more slowly and retained for a longer period if the error is persistent, but stored and forgotten rapidly if the error is more short-term. However, there are differing interpretations of the method by which the timescale of the error leads to changes in memory rates. One suggestion is that this process is implicit: as the error persists, memory stored at a slower rate has more of an opportunity to build and be retained (Joiner and Smith, 2008). The other is that the nature or statistics of the learning-event leads to an active choice of memory rates which are appropriate to the event; if the brain deems a memory worth remembering, it is retained for a longer period (Huang and Shadmehr, 2009). It is not clear whether these views are, in fact, incompatible, or simply two different ways of interpreting the same phenomenon.

5. Conclusion

In conclusion, we have displayed evidence that the saccade adaptation system is more flexible, and both more specific and general than previously accounted for. Flexibility is displayed in the sources of information it makes use of. Not only is retinal input critical, so too must be extra-retinal input for the formation of predictions. Meanwhile, if a well-defined target is unavailable post-saccadically, whatever visual information remains may be incorporated in error estimation. This is a crucial point that requires further investigation as it will provide a clearer definition of the likely limited relationship between ISS adaptation in the laboratory and adaptive maintenance of saccade gain in the natural environment. Adaptation is also specific in being highly selective of target-identity in its calculation of post-saccadic error when faced with competing alternatives to the targets from distractors or background displacements. A different, and perhaps more surprising, selectivity is seen in its ability to use target features to implement different movement plans in different contexts, which may hint towards saccade adaptation being susceptible to a more associative form of learning than would be expected for a simple motor recalibration mechanism. The generality of saccade adaptation may be seen in the many properties it shares with other forms of motor learning; its being subject to the influence of reinforcement (an exciting new direction which begs several questions), and as with nearly every area of brain research its ability to form predictions. We believe that efforts to explore saccade adaptation and saccadic behavior under generally more demanding conditions has built on earlier work, demonstrated the power of this approach, and exposed several intriguing new avenues for continued research.

• We review recent saccade adaptation data, highlighting emerging conceptual advances.

• The adaptation is driven by sensory prediction error and influenced by reinforcement.

• Active target selection is evident in adaptation in more information-rich scenes.

• The adaptation is contextual and has spatiotemporal features of other motor learning.

• Its flexibility and commonality make saccade adaptation a model learning system.