Brain circuits for the internal monitoring of movements

Abstract

Each of our movements causes spurious sensations that must be discriminated from the inputs that really matter – those representing our environment. Here we consider circuits for dealing with this problem in the primate brain. Such circuits convey a copy of each motor command, known as a corollary discharge (CD), to brain regions that use sensory input. In the visual system, CD signals have been hypothesized to produce a stable visual percept from the jumpy images resulting from our rapid eye movements. A candidate pathway for providing CD for vision ascends from the superior colliculus to the frontal cortex in the primate brain. This circuit conveys warning signals about impending eye movements that are used for planning subsequent movements and analyzing the visual world. Identifying this circuit has provided a model for studying CD in other primate sensory systems and may lead to a better understanding of motor and mental disorders.

Introduction

A major function of nervous systems is to process sensory input in order to detect changes in the environment. Yet animals have evolved not only to analyze but also to act, and actions have a multifold impact on sensory processing. A cat may notice a movement in a field and pounce toward a mouse. The cat's locomotion causes its legs to brush against grass, evoking touch sensations that must be ignored to avoid withdrawal reflexes. At the same time, the cat's head and eye movements disrupt its percept of auditory and visual space. It seems rather remarkable that a mouse is ever caught. Even at rest, if a cat perks up its ears to hear something better, the action momentarily disrupts the very sense it is meant to help. Nervous systems require some way of dealing with these myriad, movement-related disruptions. A major mechanism is to provide warning signals to sensory systems from within the movement generating systems themselves.

The importance of this motor-to-sensory feedback may be emphasized by considering what happens if there is no such link. Imagine you are a security guard watching images of a room from a video camera. If a suspicious person enters, you can quickly re-center the camera by moving a joystick. This mobility comes at the price of momentary disorientation, however, because the entire image leaps across your monitor. Our eyes are our personal security cameras, and the brain faces the same problems as the guard. About twice per second we move our eyes to inspect a new part of the visual scene, but we have no overt awareness that our visual input is constantly lurching about. We perceive the world as stable. The fundamental difference between the security camera and the eye is internal feedback of movement information. The visual system is not surprised by eye movements because it receives a warning signal about each one. This warning signal is corollary discharge (CD).

The hypothesis that CD exists in the service of vision is a venerable one (Grüsser 1995) and was discussed most influentially by von Helmholtz in the 19^th century (1925; translation by Southall), who argued for the importance of an “effort of will.” The first strong experimental evidence for the hypothesis was presented in the 20^th century from two groups working independently. Sperry (1950) coined the term corollary discharge, and his contemporaries von Holst and Mittelstaedt (1950) referred to essentially the same principle as efference copy. We will use the term CD to refer to the general concept of neuronal signals that provide internal information about movements.

In this review we concentrate on the demonstration of CD in the primate brain and its contribution to perception and motor planning. Our discussion focuses on the visual and oculomotor system because that is where the physiological basis of CD is best understood. We omit some phenomena that likely depend on CD because little data on them exist at a circuit level, and we do not cover equally important aspects of CD that have been revealed in studies of invertebrates and simpler vertebrate systems. We briefly consider the relevance of CD to other systems in the brain and to human disease.

Identifying CD in a Complex Brain

A model system for studying CD: saccades and vision

In our example of the security camera, we emphasized the disruption of vision as the camera shifted rapidly from one point of the scene to another. For the eyes such rapid movements are called saccades. Saccades play the essential role in vision of moving the fine grain analyzer of the retina, the fovea, onto salient objects in the visual scene. The generation of saccades is among the more thoroughly understood motor processes of the primate brain. The saccadic network therefore provides an attractive model system for investigating the sources of CD and the pathways that distribute CD to sensory areas.

The visuosaccadic network extends from retinal registration of visual input to muscle contraction for movement output (Fig. 1A). Visual input arrives at area V1 of visual cortex (far right) through a retinal-thalamic pathway (not shown). Movement output descends from the superior colliculus (SC), through the midbrain and pontine reticular formation, to the oculomotor nuclei that innervate each of the six eye muscles. In between these primary input and output pathways lie cortical processing networks that range from V1 through subsequent visual processing areas (Fig. 1A, yellow arrows), and which include the Lateral Intraparietal (LIP) area of parietal cortex and the Frontal Eye Field (FEF) of frontal cortex. These parietal and frontal areas contribute to the generation of voluntary saccades (see reviews by Colby & Goldberg 1999, Andersen & Buneo 2002, Schall 2002). LIP, FEF, and associated cortical areas in turn project to the intermediate layers of the SC to influence lower motor pathways (Fig. 1A, red arrows). Ancillary circuits (including one from FEF through the basal ganglia to SC) and a second visual pathway (from SC superficial layers through pulvinar to visual cortex) are not shown.

Figure 1.

Visual and saccadic circuits in the monkey brain. A. Outline of the major components of the circuit for generating visually guided saccades. Visual signals (yellow) travel from area V1 (far right) through dorsal and ventral processing streams. A major intermediary region is LIP. Visual information from both streams converges in the FEF. Signals are sent downstream (red) to influence the SC and deeper motor areas that trigger a saccade. B. Candidate pathways for conveying saccade-related CD to cerebral cortex. The SC-MD-FEF pathway has been studied in detail. Other routes may exist as well (grey arrows). C. The CD signal in MD thalamic relay neurons that project to FEF. The MD neurons have a tonic increase in firing rate terminated by a burst of activity just before saccade initiation (top). Spatially, the neurons fire only for specific ranges of contralateral saccades (bottom).

A CD path to frontal cortex

Given this background about the saccadic system, we can now ask what a saccadic CD circuit should look like. In principle, a CD pathway should run counter to the normal downward flow of sensorimotor processing. One such pathway (Fig. 1B, red arrows) ascends from the intermediate layers of the SC, to the Medial Dorsal nucleus (MD) of the thalamus and then to FEF in frontal cortex. Other pathways may convey CD as well (grey arrows) but they have not yet been studied physiologically.

The pathway in Figure 1B was elucidated anatomically (as discussed below) but nothing was known about it functionally. The key question was, does it carry CD? Pathways for CD have been elucidated at the single neuron level in simpler nervous systems (Poulet & Hedwig 2007). In the highly complex primate brain, it would seem to be even more of a challenge to establish that a neuron carries CD as opposed to a movement command. By definition both the command and the corollary should look the same. We therefore developed a set of criteria (Wurtz & Sommer 2004) that, if met, would satisfy us that we were studying a CD pathway.

Four criteria for identifying CD

The first criterion was that the CD must originate from a brain region known to be involved in movement generation, but it must travel away from the muscles. The SC-MD-FEF pathway met this criterion. The pathway originates in the intermediate layers of the SC, an area clearly involved in saccade generation. Neurons in the area discharge before saccades, electrical stimulation in the area produces saccades, and inactivation in the area disrupts saccades (Sparks & Hartwich-Young 1989). Anatomical evidence for the pathway came from retrograde labeling and anterograde degeneration studies (Benevento & Fallon 1975, Goldman-Rakic & Porrino 1985) as well as trans-synaptic retrograde labeling using herpes virus (Lynch et al 1994). By combining the classic physiological techniques of antidromic and orthodromic stimulation, individual MD relay neurons were shown to be activated by SC input and to project to the FEF (Sommer & Wurtz 1998, Sommer & Wurtz 2004a). Taken together, these results confirm that the pathway originates in a strongly saccade-related area but travels away from the eye muscles to innervate the frontal cortex.

The second criterion was that the CD signal should occur just prior to the movement and represent its spatial and temporal parameters. For the MD neurons identified as relay neurons, about three quarters increased their activity just before the saccade (Fig. 1C, top) (Sommer & Wurtz 2004a). Most MD neurons had peak activity for a limited range of saccadic amplitudes and directions, and the best direction was into the contralateral visual field (Fig. 1C, bottom). Thus the second criterion, a functional one, was met; neurons in this pathway are active before saccades and convey information about the parameters of the upcoming saccade.

The third criterion was that the neurons should not contribute to producing the movement. Silencing the pathway should not affect movements in simple tasks that do not require a corollary discharge. In accord with this criterion, inactivation of MD relay neurons did not alter saccadic accuracy and speed (Sommer & Wurtz 2002, Sommer & Wurtz 2004b).

The fourth criterion was the most critical: it required that perturbing the putative CD pathway does disrupt performance on a task that depends on a CD. Perturbing the CD pathway did disrupt performance in such a task. This evidence was so central to illuminating the nature of the CD that we devote the next section to these experiments.

CD and Sequential Saccades

Influence of CD on behavior: double step saccades

Any experiment for evaluating the influence of CD on behavior must reduce other factors that could mask the influence of CD. First and foremost, the behavior under study must prohibit visual feedback about performance, because primates make use of visual information whenever it is available. Elimination of visual feedback is achieved in the double step task (Hallett & Lightstone 1976) which has become standard in both the laboratory and the clinic. In this task, two targets are flashed sequentially. The subject's task is to make a saccade to the first target and then to the second. The first target may be flashed to the right, for example, and then the second target straight upward from there (Fig. 2A, yellow dots). In correct performance, saccades would follow the targets as depicted by the solid arrows. No visual feedback is available during the task because the saccades are performed in total darkness; the fixation point and the two flashed targets are gone before the saccades begin. If the second saccade were made only on the basis of remembering visual information, it would go up and 45 degrees to the right (Fig. 2A, dashed arrow) because that was the location of the target when the eye was originally fixating. To make the correct second saccade, this memory of the target location must be adjusted to take into account where the eye lands at the end of the first saccade. Internal information about the execution of the first saccade is needed. Because such information is not provided by visual feedback and is unlikely to come from proprioception (see Sidebar), CD must be the source.

Figure 2.

Double step task used to evaluate the influence of CD. A. The pattern of double step saccades (arrows) that would be expected with intact CD vs. total loss of CD. B. Inactivation of the putative CD pathway: the GABA agonist muscimol was injected at the site of MD relay neurons. C. Average accuracy deficit, measured as the horizontal shift in second saccade endpoints. There was a clear deficit for trials with contralateral first saccades (right), but no deficit following ipsilateral first saccades (left). D. Precision deficit in compensating for trial-by-trial variability in first saccades. Compensation is illustrated for second saccade vectors in an example session. The vectors are spread out in the vertical dimension for clarity, but their relative horizontal positions are maintained (as indicated by axis at bottom). Deficit across all sessions (n=22) is summarized with the bar graph (inset). Bars show means and SEs; on average there was a 58% deficit ([97%-39%]/97%).

Proprioception.

When visual feedback is absent or unreliable, extraretinal signals are required to monitor eye position. CD is one such signal, and proprioception of eye muscles is another (Wang et al 2007). While CD is of central origin and reaches cerebral cortex before a saccade, proprioception arises peripherally and reaches the cortex after a saccade. A series of experiments has shown that proprioception provides negligible information for everyday saccadic behavior. The common approach was to use microstimulation to displace a monkey's eye position while it prepared a voluntary saccade to a visual target. The question was, could a monkey internally monitor the displacement and compensate for it? The extent of compensation indicated the availability of extraretinal information. Stimulating the muscle-innervating motor neurons caused an eye displacement but no compensation, even though proprioception was available (Schiller & Sandell 1983, Sparks & Mays 1983, Mays et al 1987). But stimulating higher in the circuitry, e.g. in the SC, caused a saccade that was immediately followed by full compensation, presumably because a concomitant CD signal was evoked. Compensation following SC stimulation persists even after the proprioceptive nerves are cut (Guthrie et al 1983). Proprioception therefore seems insufficient as an extraretinal signal for making successive saccades; instead, it may contribute to longer term adjustments (Steinbach 1987, Lewis et al 2001).

Quantifying the influence of CD on behavior requires an observable measure of that influence. CD itself is an internal signal and hidden from direct psychophysical assessment. In the double step task, the observable event is the second saccade, and the parameter of interest is its vector. A second saccade vector that lands at the second target location, fully compensating for the first saccade, implies perfect CD. Inaccuracies of the second saccade reveal inaccuracies of CD.

Inactivation of the CD pathway

The hypothesis that the pathway to frontal cortex provides CD information about the first saccade was tested by reversibly inactivating the MD relay node (Fig. 2B) in the pathway. This left both the SC and FEF untouched while severing the link from SC to FEF (Sommer & Wurtz 2002, Sommer & Wurtz 2004b). If inactivation totally eliminates CD, the monkey should make the first saccade correctly but should not have internal information that it did so. In the absence of information about this first saccade, the second saccade should be inaccurate; it should be directed toward the initial, retinotopic location of the second visual target. The average endpoints of second saccades were analyzed to see how they changed during inactivation.

Inactivation of MD did cause second saccade endpoints to shift as if CD of first saccades was impaired, as long as those first saccades were directed into the contralateral hemifield (Fig. 2C). The impairment was significant, but smaller than expected from total CD elimination. This partial deficit suggests that we did not silence the path completely, or that other CD pathways exist. Both hypotheses are reasonable but have yet to be tested.

What makes the results more compelling is that the deficit was limited to trials in which the first saccades were made into the contralateral visual field (Fig. 2C, right). This direction was the same preferred direction of the MD relay neurons. Double step configurations in which first saccades went into the ipsilateral visual field were randomly interleaved, and these trials were unaffected (Fig. 2C, left). In sum, there was a modest deficit in the average accuracy of second saccades during inactivation but a deficit that matches the direction of saccades represented in the affected MD nucleus.

Similar double-step deficits have been found in patients subsequent to thalamic injury (Gaymard et al 1994, Bellebaum et al 2005), parietal injury (Duhamel et al 1992b, Heide et al 1995), and parietal transcranial magnetic stimulation (Morris et al 2007). Such deficits have not been reported for human FEF lesions, but a major confound is that FEF damage causes impairment to saccade generation that complicates analysis of CD deficits (discussed in Sommer & Wurtz 2004b).

Successive saccades depend on CD

A finer-grained analysis of the data, focusing on the precision of individual saccades, revealed a more striking deficit (Sommer & Wurtz 2004b). During normal behavior, monkeys made first saccades that landed at slightly different places from trial to trial, sometimes short of where the first flashed target had been, sometimes beyond it. It therefore seemed likely that if there is a CD with each saccade as predicted from the neuronal activity found in MD, there should also be a CD-driven compensation unique to each trial. As predicted, the individual second saccade vectors (Fig. 2D, blue lines) rotated forward or backward depending on where they started (i.e. where the first saccades landed). Hence in its normal state, the brain precisely monitors even small fluctuations in saccade amplitude and adjusts subsequent saccades accordingly.

Now if the CD were altered by inactivation this compensation should be altered. It was – the second saccades during inactivation (Fig. 2D, orange lines) failed to rotate as a function of where the first saccades ended. Compensation was reduced by more than half (Fig. 2D, inset). Monkeys could still compensate slightly after MD inactivation, so the deficit was partial as it was for the average accuracy deficit. And once again, the effect occurred only for saccades into the contralateral visual field.

These inactivation experiments suggest that activity in the SC-MD-FEF pathway not only correlates with what is expected of a CD signal, but actually functions as a CD signal. With this demonstration, the signals in the pathway from SC to FEF via MD satisfy the four criteria we regard as essential for identifying CD. The signals originate from a known saccade-related region, they encode the timing and spatial parameters of upcoming saccades, their removal does not affect the generation of saccades where CD is not required, and their removal does disrupt saccades where CD is required.

CD and Visual Perception

Influence of CD on vision: shifting receptive fields

After gaining evidence that the information conveyed upward through the SC-MD-FEF pathway is a CD signal, we now return to the fundamental hypothesis: that the world appears stable across eye movements because CD informs the visual system about each imminent saccade. Whether cortical visual neurons truly use a CD signal was unknown until landmark experiments on parietal cortical area LIP by Duhamel, Colby, and Goldberg (1992a) that revealed changes in neuronal activity that must result from CD input.

Neurons in the visual system have a restricted “window on the world” known as a receptive field (RF). The neurons respond only if light falls on a restricted zone of the retina – the RF – and conventional thought is that the location of this zone changes little, if at all, when the eyes move. Duhamel et al. (1992a) found, however, that some neurons in LIP behave differently. A new zone of retinal sensitivity emerges at a location dependent on the saccade that is about to occur. Such a “shifting RF” is only possible if the neuron receives information about the imminent saccade.

The fundamental nature of a shifting RF is depicted in Figure 3A. Suppose that during fixation (left panel), a neuron has an RF (blue ellipse) aligned directly below the fovea (orange dot). The neuron would respond to an apple in its RF but not to a distant pepper. If the monkey plans to make a saccade to the point just above the pepper, however (middle panel), a neuron with a shifting RF shows a crucial change. It suddenly becomes sensitive to the pepper (magenta ellipse), because the pepper lies at the location where the RF will reside after the saccade – referred to as the Future Field (FF). Then the eyes move (right panel), and the presaccadic FF becomes the RF again. Such shifting RFs (also called remapped RFs) have been studied in many regions including LIP (Duhamel et al 1992a, Colby et al 1996, Batista et al 1999, Kusunoki & Goldberg 2003, Heiser & Colby 2006), FEF (Umeno & Goldberg 1997, Umeno & Goldberg 2001, Sommer & Wurtz 2006), extrastriate visual areas (Tolias et al 2001, Nakamura & Colby 2002), and the SC (Walker et al 1994). They are likely to be present in the human brain as indicated by fMRI studies (Medendorp et al 2003, Merriam et al 2003, Merriam et al 2007), and their relation to corpus callosum function has been examined (Berman et al 2005, Heiser et al 2005, Berman et al 2007).

Figure 3.

Shifting visual receptive fields. A. Concept shown schematically. B. Example of an FEF neuron that shows the shift. A visual probe was flashed in the RF in some trials (blue) or in the FF in other trials (magenta). No probe appeared in control trials (black). Curves show mean and SE of average neuronal activity aligned to probe onset (left and center panels) or to saccade initiation (right panel).

Shifting RFs are not studied in the laboratory with apples and peppers, of course, but with well-controlled flashes of light (“probes”). The activity of a neuron with a shifting RF demonstrates its presaccadic change in visual sensitivity across space (Fig. 3B, an FEF neuron). During initial fixation the neuron responded only to probes flashed at the RF (left) and just before the saccade only to probes flashed at the FF (middle). Because the visual activity associated with the FF stimulation appeared around the time of the saccade (right), a fair question is whether the activity was related to making the saccade. This possible artifact was tested with interleaved trials in which no probe appeared, but the same saccade was made (black traces). Such control trials elicited no activity. The activity required a visual stimulus in the FF.

Our hypothesis was that, in FEF neurons, shifting RFs are caused by CD input from the MD-mediated pathway. This led to three predictions. First, the spatial characteristics of the shifting RF should reflect those of the CD. Second, the temporal dynamics of the shift should match those of the CD. If both of these first two predictions were confirmed, the CD would be established as appropriate for driving the shift. The third prediction was that the shift should be reduced if the CD were interrupted. Such a result would show that the CD is necessary for the shift.

Spatial match between CD and shifting RFs

The first prediction about shifting RFs was that they should have spatial properties similar to the CD itself. The CD in the pathway to FEF originates in the SC (Fig. 4A), a structure with exquisite topographic organization of saccade vectors. If SC activity provides the CD that causes the FEF shift, the shift should move as if prompted by a vector input. It should jump to its new location (Fig. 4B) rather than spreading continuously from RF to FF. The prediction was testable by flashing probes not only at the RF and FF, but also at the midpoint between them. If the RFs jump, the midpoint would exhibit little or no activity; if they spread, the midpoint would become highly active.

Figure 4.

Shifting RFs: spatial. A. Neurons with shifting RFs were recorded in the FEF. B. The prediction was that shifting RFs would jump, as expected from a vector CD input. There should be a significant change in activity (asterisk) at the FF but no significant difference (n.s.d.) at the midpoint. C. (Top) Example FEF neuron shows a jump, not a spread of activity. (Bottom) The same result in the average of the FEF population (n=13 neurons). D. Experiment to see if shifting RFs travel toward the saccadic endpoint (“?” trajectory). The data support, instead, a trajectory parallel to the saccade (average data from n=12 neurons). See text for details.

Data from an example neuron (Fig. 4C, top) revealed that while activity increased dramatically at the FF just before a saccade (right), no change occurred at the midpoint (middle). The result from this example neuron was reproduced in the overall population (Fig. 4C, bottom). There was a small average decrease in activity at the RF just before a saccade (left), no change at the midpoint (middle), and a large increase at the FF (right). The data supported a jump, not a spread, of activity, compatible with a vector CD input. The small amount of activity at the midpoint, which did not change significantly before the saccade, was caused by the probe sometimes falling on the edges of the RF and FF (Sommer & Wurtz 2006).

In this analysis the implicit assumption has been that the shifting RFs move parallel to the saccade as implied by the horizontal shift with a horizontal saccade in Fig. 3A. Shifting RFs in visual area V4, however, were reported to move toward the saccadic endpoint (Tolias et al 2001). The direction of the shift is crucial because the FF location provides clues about the perceptual function of the shift, as will be discussed below. To examine the issue in FEF neurons, probes were introduced near the saccadic endpoint (Fig. 4D, top). As the green activity trace shows (Fig. 4D, bottom), the shift at that location was minimal. It was no different from activity at the edge of the FF as defined by a parallel shift (purple) and much less than activity the center of the FF so defined (black trace). The data supported our hypothesis that shifting RFs in the FEF move parallel to the saccade.

Temporal match between CD and shifting RFs

The next prediction was that the timing of shifting RFs should match the timing of the CD signal in the SC-MD-FEF pathway. The recording study (Sommer & Wurtz 2004a) had demonstrated that the pathway conveys a gradual increase in activity and then a burst of activity aligned with saccade onset (recall Fig. 1C). If CD from the pathway causes the shift, the shift should be time-locked with the saccade.

We analyzed whether shifting RF activity was better related to stimulus onset or to saccade initiation. Shifting RF activity requires visual stimulation in the FF, and Figure 5A shows for one neuron the temporal profile of visual probe presentation in the FF and the neuronal activity that it evoked. The probe appeared at time 0 and disappeared after 50 ms. There was no activity at the neuron's normal visual latency of 80 ms (purple arrow; bold orange trace is average firing rate). Rather, activity appeared much later, and with appreciable variability. In individual trials (rows), the start of the activity seemed closely linked to the start of each saccade (green dots). The correlation between activity onset and saccade onset was highly significant for this neuron and for the population (Sommer & Wurtz 2006). Alignment of the average visual activity improved when the same data were plotted relative to saccade initiation (Fig. 5B, higher peak and narrower SEs). The take-home message was that a shifting RF is visual activity synchronized to a motor act.

Figure 5.

Shifting RFs: timing. A. Shifting RF activity aligned to probe onset for an example neuron. Start of the shift is much later than the neuron's normal visual latency. Also shown are times of saccade onset in each trial (dots) corresponding to the eye position data (traces at bottom). B. Shifting RF activity aligned to saccade initiation. For this neuron, shift onset time and saccadic onset time were correlated with Pearson R = 0.97; for the population, R = 0.50 (both p < 0.01).

Although saccade-gated visual activity may seem bizarre, it makes sense; if the shifting RF has anything to do with maintaining perceptual visual stability, it would be important for it to occur only if the generation of the saccade were inevitable. Saccades can be cancelled up to 100 ms before their initiation (Hanes & Schall 1995), so only by synchronizing the shifting RF activity to begin after 100 ms prior to saccade initiation can it be guaranteed that the eyes are beyond the “point of no return” for moving.

Shifts were time-locked to saccades, but their exact onset times varied from neuron to neuron as indicated by four examples (Fig. 6A). In some neurons (orange and blue traces) the shift started before the saccade began (arrows indicate shift onset times). In others, the shift started after the saccade began (green and red traces). It should be noted that for all of the neurons, probes were flashed at identical times (yellow range to left) and always appeared and disappeared in the FF of the neuron. Even though some shifts began after saccade initiation, they were caused by a visual event happening before saccade initiation and outside of the classical RF.

Figure 6.

Shifting RFs: timing of CD influence. A. Shift onset times for four example FEF neurons. The orange trace is from the neuron in Fig. 5A, and the red trace is from the neuron in Figure 3B. The blue and green traces show data from two other neurons. Shift magnitudes are normalized to each other to facilitate comparison of timing. B. Distribution of shift onset times for the sample of FEF neurons (n=26). C. Distribution of CD onset times (the start of the saccade-related bursts) in the population of MD relay neurons that project to FEF.

Overall, the shift onset times for the population of neurons were distributed approximately normally, over a range of 100 ms before to 200 ms after saccade initiation (Fig. 6B). The average shift onset time was 24 ms after the saccade started, which is in mid-flight of a typical saccade. How does this distribution of shift onset times compare with the CD signal that we posit is causing the shift? Figure 6C shows the range of onset times for the CD signal that is provided to the FEF by MD relay neurons. Just as shift onset times in the FEF were measured by a sudden increase in visual activity of shifting neurons, CD onset times were measured by a sudden increase in saccade-related activity of MD relay neurons. The average CD onset time was 72 ms before the saccade, demonstrating that the CD signal is feasible as a causal trigger for the shift. It appears to take about 100 ms (72 + 24 = 96 ms) for CD arriving at the FEF to trigger a shifting RF. This delay implies that shifts are not constructed by single, MD-recipient FEF neurons, but rather develop through a multisynaptic network.

Necessity of CD signal for Shifting RFs

The final prediction was that interrupting the CD signal should reduce the shift. Up to this point the comparison of CD to shift was one of spatial and temporal correlation; perturbing the CD allows us to move from correlation to causation. The CD signal was inactivated at the level of MD relay neurons while an FEF neuron with a shifting RF was studied (Fig. 7A) (Sommer & Wurtz 2006). If the prediction were correct, the FEF neuron would retain its classic visual response (in the RF) but lose its ability to shift the response (into the FF).

Figure 7.

Shifting RFs: necessity of CD pathway. A. Inactivation of the CD pathway while an FEF neuron with a shifting RF is recorded. B. Example of a shifting RF that was impaired by MD inactivation. C. (Left) The deficit in the population of neurons. Reductions in activity were seen only in the FF, not the RF, and only for contralateral, not ipsilateral, saccades. (Right) Task configurations for testing shifting RFs that accompanied contralateral (contra) saccades vs. ipsilateral (ipsi) saccades. **, significant changes at p < 0.0001 level.

Data from an example FEF neuron are shown in Figure 7B. Activity before inactivation is depicted with black traces. The neuron had a strong visual response in the RF during fixation (left panels), an intense shift to the FF just before the saccade (lower middle panel), and tight alignment of this shift with saccadic onset (right panel). The neuron maintained its activity in the RF even as it gained sensitivity at the FF, as many neurons did (upper middle panel). During inactivation (orange traces), the FF response was drastically reduced (bottom row). The classic RF response, however, was unchanged (top row). In sum, the neuron retained its visual sensitivity, and the monkey retained its ability to make a saccade, when MD was inactivated. But the communication between saccade generation and visual processing was gone; the neuron seemed to lose advance information about the saccade (i.e. CD), severely impairing its ability to shift visual sensitivity to the FF location.

The deficits found in the example neuron were replicated individually in most neurons tested (7 out of 8) and were highly significant in the pooled data (Fig. 7C, left). Overall, the shift was reduced by an average of about 50% when it accompanied contralateral saccades, those represented by the SC-MD-FEF pathway. Shifts that accompanied ipsilateral saccades were unimpaired.

In summary, our predictions were supported: shifting RFs had spatial and temporal dynamics consistent with the CD signal, and the shifts depended on the CD signal. This provided systematic evidence for a functional link between the CD arising from the midbrain and visual processing in the FEF. This result has an intriguing further implication when considering the position of the FEF in the broader network of cortical visual areas. The FEF may play a role in relaying CD information, or shifting RFs themselves, to other cortical areas such as LIP where shifting RFs are found. Functional, reciprocal connections are known to link FEF and parietal cortex (Chafee & Goldman-Rakic 2000).

Shifting RFs and visual stability

The previous section and Figures 3-7 have described shifting RFs. Because of their unique characteristics, shifting RFs have been thought to play a role in the perception of visual stability across saccades (Duhamel et al 1992a). To see why this hypothesis is plausible, it is necessary to discuss in more detail how CD signals influence shifting RFs, and how shifting RFs, in turn, could influence visual analysis.

The CD signals produce a spatial effect in the retinotopic coordinate system used for visual processing. This effect is a new receptivity to visual stimulation at the FF. The location of the FF is determined by the vector of the impending saccade conveyed by the CD. How the motor-to-sensory transformation between the saccadic system and the visual map could be made has been modeled by Quaia, Optican, and Goldberg (1998). The salient point is that for a neuron with a shifting RF, the CD signal enables visually responsiveness at a specific location on the visual map, and the information conveyed by the neuron is about visual events in visual coordinates.

In temporal terms, saccadic CD has a gating effect on visual activity. A normal visual response, to a stimulus in the RF, has a brisk onset and fixed latency relative to stimulus onset. But activity evoked by a visual stimulus in the FF has no such brisk onset; as was shown in Figure 5, it is tied not to stimulus onset but to saccade onset. The visual activity may be held in check for as long as 1000 ms until the saccade begins (Umeno & Goldberg 2001). Only around the time of the saccade is the visual activity released in a surge of action potentials. As noted above, saccadic gating of the FF visual activity is a strange but seemingly critical feature; it guarantees that a shifting RF occurs only if a saccade occurs.

The above description of shifting RFs makes it clear that there are two phases to the phenomenon: an early phase of visual sensitivity at the FF that can begin long before the saccade, and a later phase of unleashed visual activity that begins with the saccade. These two phases fit well with the nature of the CD signal (recall Fig. 1C, top), which has a low-frequency, prelude phase that could induce visual sensitivity at the FF and a high-frequency, burst phase that could trigger the saccade-aligned visual activity.

With these properties of shifting RFs in mind, the hypothesis that they contribute to the perception of visual stability can be evaluated (Colby & Goldberg 1999, Sommer & Wurtz 2006). We think the hypothesis is compelling because of a specific, key feature of shifting RFs: the presaccadic FF location and the postsaccadic RF location sample the same absolute location in visual space. For an illustration of this overlap, see Figure 3A and note that the FF location (middle box) is identical to the postsaccadic RF location (right box). Information is available, therefore, about a single region of the visual world before and after the saccade. If the presaccadic image equals the postsaccadic image, objects must have remained stable across the saccade. If there is a presaccadic-postsaccadic difference, objects must have moved while the eyes moved. In this manner shifting RFs could allow single neurons to assess the stability of the world across saccades.

Confirmation or refutation of this hypothesis calls for two lines of research. One would address in a conceptual manner how trans-saccadic information provided by shifting RFs could lead to a percept of visual stability. What are the exact computational principles involved? Does an overt representation of stable space have to emerge from the comparison of presaccadic FF with postsaccadic RF, or can it remain implicit at a network level? Answering these questions will almost certainly depend on developing more sophisticated models of how neuronal activity leads to perceptual experiences.

The other line of research would empirically test the link between shifting RFs and perceptual stability. On this issue, the experiments on the SC-MD-FEF pathway provide a new opportunity. Because we have established that at the neuronal level we can disrupt the shifting RFs in FEF by interrupting the ascending pathway, we can now test whether interrupting this pathway also disrupts visual stability when saccades occur. The approach would be to train a monkey to report when an object moves in front of it as opposed to when an object seems to move as a result of its own saccades. Our prediction is that during inactivation of the CD pathway, the monkey will erroneously report that an object moves during a saccade when it is actually still. Such a demonstration would provide the final link in the mechanistic chain from identified CD signal, to the shifting RF phenomenon, to the percept of visual stability across saccades.

CD Beyond Vision

The nature of the CD signal

Until recent years, the mechanistic nature of CD in the primate has largely been a matter of speculation. Now, with insight into one example of the neuronal basis of CD, we can compare speculation with observation. Three salient concepts about how CD exerts its influences have been suppression, cancellation, and forward models.

The first CD concept is also the simplest: the CD simply suppresses visual information. Evidence for such suppression with saccades has been found in single neurons of the superficial visual layers in SC (Robinson & Wurtz 1976). The suppression resulted from a CD rather than from proprioceptive feedback because preventing the proprioception did not eliminate the suppression (Richmond & Wurtz 1980). This was the first demonstration of the effect of CD in the primate brain. The recent demonstration of an inhibitory input from the saccade-related intermediate layers of the SC to the visual superficial layers (Lee et al 2007) suggests a mechanism for the effect. Clear suppression that may also result from CD has been demonstrated in the primate pulvinar (Robinson & Petersen 1985) and extrastriate area MT (Thiele et al 2002); the lateral geniculate nucleus shows only modest suppression (Ramcharan et al 2001, Reppas et al 2002, Royal et al 2006).

The second concept, cancellation, was put forward most forcefully by von Holst and Mittelstaedt (1950): “…the efference leaves an ‘image’ of itself somewhere in the CNS to which the reafference (i.e. sensory signs of the resulting movement) compares as the negative of a photograph compares to its print” (von Holtz quoted in MacKay 1972). Cancellation mechanisms have been found in some CD circuits, for example in electric fish (Bell 1982), but have seemed less likely in the primate visual system (Bridgeman 2007). In our experiments, the activity representing the CD in MD (a vector encoding movement) and that representing visual activity in FEF (a response field on a retinotopic map) are strikingly different. Simple cancellation makes little sense, so we find no support for this interpretation of CD in the primate for the visual-saccadic system.

The third and most recently promoted concept, the forward model, has been particularly useful in the analysis of skeletal movements (Wolpert & Miall 1996) and has been invoked occasionally for the oculomotor system (Robinson et al 1986, Vaziri et al 2006, Murthy et al 2007). Figure 8 illustrates the concept. A copy of the motor command, or CD in our terminology, is routed into a forward dynamic model. This feedback about a new movement is used to create a feedforward model of the predicted sensory consequences of the movement, the forward output model. The prediction of the sensory input is compared with the actual, reafferent sensory input to determine the extent of any sensory discrepancy – similar to the cancellation interpretation of CD. The key advance of forward models over previous cancellation hypotheses, however, is the explicit recognition that the copy of the motor command, in motor coordinates, must be transformed into sensory coordinates so that it can be compared directly with the sensory input (Wolpert & Miall 1996, Webb 2004). As we have noted above, such a motor to sensory transformation is exactly what we find in the FEF, which uses CD in saccadic motor coordinates to generate shifting receptive fields in visual coordinates. Hence the FEF may contain elements of a forward model (Crapse & Sommer 2008).

Figure 8.

The forward model concept. CD (left) is combined with the current state of the system to generate a prediction of the sensory input (upper right). This prediction is compared with the actual, reafferent sensory input (lower right). Resultant discrepancies inform the brain about external influences and miscalibrations. Modified from Wolpert & Miall (1996) with permission from Elsevier.

Results from a variety of studies in the primate visual-saccadic system therefore support two major concepts of CD: its role in suppression mechanisms and forward models. Little evidence supports a cancellation mechanism for saccades.

CD and human disease

For as long as CD has been recognized, there has been speculation about the relation between CD and human disease. von Helmholtz (1925) described how eye muscle paralysis causes a mismatch between impaired saccades and intact “effort of will”, as he called CD. He reported that the mismatch induces disturbing visual percepts, and subsequent work has refined his initial observation (Kornmueller 1931, Stevens et al 1976). The reverse condition, in which muscles are intact but CD is lost, results from central nervous system lesions. Thalamic and parietal lesions cause CD deficits in double step tasks, as discussed above, and parietal damage has been reported to impair the awareness of movement intentions (Sirigu et al 2004). Frontal damage may cause CD-related deficits in perception (Teuber 1964), and visual cortex lesions can cause an illusion that the visual world is moving when the eyes move across a static background scene (Haarmeier et al 1997).

Broader damage to CD networks, reaching beyond focal lesions, may have more devastating and long-term consequences. Schizophrenia has been hypothesized to involve a disruption to CD that could impair the self-monitoring of thoughts or actions so that they are misattributed to external sources (Feinberg 1978, Feinberg & Guazzelli 1999). CD deficits in schizophrenia could arise from the disrupted subcortical-thalamic-cortical circuits that are known to characterize the disease (Andreasen et al 1999). Schizophrenics are impaired in tasks requiring CD (Malenka et al 1986), and their auditory systems show particularly strong evidence for loss of CD-related modulation (Ford et al 2001, Ford et al 2007). Neurophysiological assessment of CD circuits (e.g. the SC-MD-FEF pathway) could be performed in monkey models of schizophrenia as induced with dopaminergic or glutamatergic drugs (Condy et al 2005, Stone et al 2007) or fetal irradiation (Selemon et al 2005). While a contribution of impaired CD to schizophrenic positive symptoms, e.g. hallucinations, is the most obvious tie-in of the function to the disease, it has been argued that impaired CD could contribute to negative and cognitive symptoms as well (Frith 1987).

Deficits in CD have been implicated, too, in disorders having a significant motor component such as Huntington's disease (Smith et al 2000) and developmental coordination disorder (Katschmarsky et al 2001). We expect that further insights into the possible role of CD impairments in disease will continue to develop as circuits for CD in the normal brain become better understood.

Conclusion

We have described the specific issue of using CD from the saccadic system to influence visual processing, and have indicated more generally how CD might be involved in other brain movement systems and in their pathology. Our focus was on a CD pathway that emanates from the midbrain and influences the cerebral cortex.

Our final point is that although we have concentrated on a CD arising from one level of a sensorimotor pathway in the primate brain, CD could arise at any of multiple levels along such a pathway (Fig. 9). At each level, the characteristics of the descending sensorimotor signals change, and because the CD is a copy of the motor signal, the characteristics of the ascending CD would change as well. The particular visual-saccadic CD that we considered branches off at a point where saccadic vectors are represented on a retinotopic map (the SC), and this spatial CD is well-suited for visuomotor functions such as shifting RFs that require input related to that map. A CD that branches from lower in the descending pathway, in the pontine circuitry, should convey information much more tightly related to saccadic timing parameters such as velocity. CD copies of such signals play an important role in low-level saccadic control circuits (Robinson 1975) and may feed back to the SC (Soetedjo et al 2002). CD signals that branch from the lowest levels would provide information about muscle innervation. At the upper extreme, CD signals may emerge from areas of cerebral cortex such the FEF; they may convey more tentative plans to move rather than distinct retinotopic vectors of the impending movement. Finally, the information at each level in the chain may diverge to influence multiple different targets. What CD goes where would depend on the requirement of the target sensory area.

Figure 9.

Hypothesized chain of saccadic CD in the primate brain. Descending visuomotor levels (right) extend from the cortical areas that detect a visual stimulus (top) to the brainstem motor neurons that contract the muscles (bottom). Ascending CD signals may branch from any of these levels (left). The pathway that branches from SC up to the FEF conveys a CD signal rich with spatial vector information, because that is the nature of the signal found in the SC and sent downstream from it. Other CD signals, branching from above and below the SC level, would reflect other aspects of movement. It may be more accurate to consider CD as a set of signals representing the state of each saccade-related motor level, in contrast to a unitary CD signal representing the saccade itself.

Our overall view is that for any movement there may be a set of CD signals, with each CD conveying the state of the specific sensorimotor level from which it emerges. While we may still speak of CD in loose terms as representing the state of a movement, it is more precise – and more beneficial to experimenters – to realize that in the primate brain, a CD signal represents the state of a movement structure such as the SC. The information content of the CD signal is limited by the signals available to the structure, and the potential targets of the CD signal are constrained by the anatomical reach of the structure's efferent pathways. With these principles in mind it will be important to identify further CD pathways in the primate brain.

Glossary

Afference

Sensory inflow evoked by changes in the environment

Corollary Discharge

A copy of a motor command sent to brain areas for interpreting sensory inflow (also called efference copy)

Efference

Motor outflow that causes movements

Reafference

Sensory inflow evoked by one's own movements

Saccade

Eye movement that rapidly redirects the fovea

Abbreviations

CD

Corollary Discharge

FEF

Frontal Eye Field (of frontal cortex)

LIP

Lateral Intraparietal area (of parietal cortex)

MD

Medial Dorsal nucleus (of thalamus)

RF

Receptive Field

SC

Superior Colliculus (of midbrain)