How the brain makes the world appear stable

Abstract

Space constancy, the appearance of a stable visual world despite shifts of all visual input with each eye movement, has been explained historically with a compensatory signal (efference copy or corollary discharge) that subtracts the eye movement signal from the retinal image shift accompanying each eye movement. Quantitative measures have shown the signal to be too small and too slow to mediate space constancy unaided. Newer theories discard the compensation idea, instead calibrating vision to each saccadic target.

All visual information arrives in the brain through the retinal receptors that are in nearly continuous motion as the eyes flit from one fixation to another. Yet the world appears perfectly stable. A stable visual world establishes the platform on which all other visual functions rest, allowing judgments about positions and motions of the self and other objects. How is the stabilization accomplished? This is the problem of space constancy.

1. Efference copy

Historically the problem of visual stability has been simplified to address only a fixed visual world with a fixed head. Until a few decades ago the dominant theory evoked a signal emanating from motor areas of the brain to inform the visual system about eye movements. At the time of an eye movement this signal would be subtracted from the resulting retinal image shift, achieving space constancy. Because the idea requires that visual centers receive a copy of the neural efference to the eye muscles, von Holst and Mittelstaedt (1950) named it efference copy. Formal descriptions originated with Bell (1823) and Purkinje (1825), who described it independently. Helmholtz (1866 [1962]) expanded the empirical base for efference copy with observations of neurological patients who had eye muscle damage. He called the intention to change gaze position a ‘Willensanstrengung’, an effort of will. Another extraretinal signal, proprioception from the eye muscles, was later proposed as a source of information that could contribute to space constancy (Sherrington 1918). The analysis below shows that it makes only a minor contribution.

Not all theorists have accepted this position; J J Gibson asserted that motion of the entire visual field would always result in a perception of self-motion. Asking whether the world should appear to move when the visual field sweeps by, he answered, “Why should it move? The retina is proprioceptive” (Gibson 1966 p 256). Simply pressing on one eye while closing the other disproves this assertion, for then the whole world appears to move.

Von Holst and Mittelstaedt (1950) inverted the head of the blowfly Eristalis, holding it with a piece of wax. The fly would circle either clockwise or counterclockwise at random. In darkness, though, the fly's locomotion seemed normal. The conclusion was that the fly compared the output of its locomotor system with the retinal flow field (since the fly's eye is fixed to its head, and in the experiment is also fixed to the body, the locomotor system is also the oculomotor system). The ‘Efferenzkopie’ would subtract from the retinal signal to stabilize locomotion (figure 1). Inverting the head converted the negative feedback to positive feedback—a random nudge in one direction would feed back a signal to ‘correct’ in the same direction, resulting in a further deviation in the same direction. Continuous circling would result. Independently, Sperry (1950) made similar observations in a fish with a surgically inverted eye, naming the signal corollary discharge. These papers formalized the compensation idea that had dominated for more than a century. But there had always been problems.

Figure 1.

Use of efference copy to stabilize a perceived visual world, redrawn to American engineering conventions. Information flows into the system from the left and out to the right. Efference copy is subtracted (−) from the visual input, afference (+), at a comparator to present a stable representation to the brain.

2. Limitations of efference copy

First, efference copy informs the brain of where the eyes should be rather than where they are. As such it cannot be exact—it should drift with time, and inaccuracies cannot be corrected. Yet perceptual space constancy is perfect—the world does not appear to jump in the slightest when the eyes move. Something must be supplementing the efference copy. Further, quantitative measures of saccadic suppression of displacement showed that, at the optimal timing of image displacement and saccade, the perceptual threshold was nearly 1/3 as large as the saccade (Bridgeman et al 1975). Any orientation mechanism with an error this large had no idea where the world was, and could support neither space constancy nor visual-motor calibration. Efference copy could not be the only mechanism mediating space constancy.

There were other problems as well. Experiments exploiting alternating eye movements, either from rapidly repeated eye presses (Ilg et al 1989) or from rapidly alternating saccadic eye movements (Nagle et al 1980; Grüsser et al 1987), demonstrated that both motor and sensory compensation were effective at low temporal frequencies but failed at higher frequencies. Thus any efference-copy-based system fails at rates well within the temporal range of normal perception.

Space constancy was further explored by pressing with a finger on the eyelid. The eyepress has been misunderstood since Purkinje's 1825 assumption that it elicits a passive eye movement; the real situation is exactly the reverse. The reader can easily verify this—pick a fixation target, then close one eye while slowly pressing on the other through the lid. You will be able to hold your gaze on your fixation target while the visual world, target and all, appears to move. Thus the retina is not moving with respect to the visual world. The motion must come from involuntary mechanisms that hold the eye on target. Rather than revealing the effect of passive eye movement, the eyepress shows the effect of active compensation for oculomotor disturbance. Efference copy alone can drive motion perception (Bridgeman and Stark 1991). Pressing on the viewing eye changes efference without changing gaze position, but pressing on the covered eye should change only proprioception. The covered eye when pressed should rotate under the eyelid, because the press does not result in any corrective signal from visual error feedback. The situation in pressing the viewing eye is now more complicated, because efference copy and proprioception work in opposite directions. Proprioception will come from the deviated, covered eye, and altered efference copy will be driven by the active compensation for the press of the viewing eye (suggested by Wenshun Li of Columbia University). Thus we could quantify gains of both efference copy and proprioception. Presses on the viewing or the covered eye could be used to recover the internal proprioception and efference copy signals by algebraic reconstruction of the measured signals (Bridgeman and Stark 1991). The resulting gains were 0.61 for efference copy and 0.26 for proprioception; even with perfect summation of the gains at 0.87, the brain would underestimate how far the eye really moved.

The missing gain should have perceptual consequences. In fact it accounts for previously unexplained data on the perceived deviation of an eccentric target from the midline; targets are perceived as more eccentric if their position is judged in peripheral vision, with the eyes straight ahead, than if they are fixated (Morgan 1978). The gain from Morgan—s data is 0.13, precisely the ‘missing’ gain from the eyepress experiments. Efference copy, proprioception, and illusion gains sum to 1.00, closing the circle on the signals used to register gaze position and consequent perceptions. Two centuries of work had led to the conclusion that efference copy dominates; we can now see that the reason for the domination is that its gain is about 2.4 times greater than the proprioceptive gain. Thus efference copy explains a wide range of results and clinical observations better than proprioception.

3. Modern alternatives

Efference copy clearly cannot support space constancy, but for years no alternative was available. It was finally discarded as a mechanism for space constancy with a new theory. O'Regan (1992) asserted that there need be no memory for the content of previous fixations, because the information remains in the world and can be reacquired whenever an observer wants it. The brain codes only the presently available retinal information. This idea turned out to be too radical, but not by much. Two years later another reanalysis appeared (Bridgeman et al 1994). We knew by then that no brain area contained a panoramic, high-acuity representation that corresponds to our spatially stable perception. The alternative was a ‘calibration’ solution: spatiotopic positions are calculated anew from proprioception, efference copy, and retinal sources for each fixation. Previous fixation positions are not taken into account; the world appears to be in the same place because nothing tells the brain that it is not. Extraretinal signals, then, do not compensate retinal position but destroy it. Perception begins anew with each fixation.

An elaboration of this idea was motivated by data showing that a saccadic target object that is present continuously appears to be stabilized, while objects that are interrupted near the time of a saccade appear to jump, even if it is the continuous target that really jumps (Deubel et al 2004). To explain this, Deubel et al conclude that, before a saccade is executed, attention shifts to a reference object whose location and attributes are stored in a transsaccadic memory. After the saccade, the visual system searches for the reference object. The world is perceived as stable if it is found. Information from the previous fixation is discarded or ignored, and localization proceeds using currently available information. Other objects are then localized relative to the reference object. Only if the object is not found are efference copy and other information sources applied. Extraretinal signals are used in static conditions, though, especially to control motor behavior.

Evidence for this position comes from a number of sources, including change blindness, the difficulty of identifying changes in naturalistic scenes if they are masked by a brief blank (Simons 1996; Rensink et al 1997). The importance of change blindness is that inattention to previous images prevents their interfering with present perception. Little is carried over from one fixation to the next; we do not build a visual world by pasting together samples calibrated with efference copy, but simply perceive what is currently available, plus a gist and a few previously attended objects. The stable, rich visual world of our perception is more promise than physiological reality. A second conclusion is that many mechanisms and strategies combine to achieve visual stabilization. All of this happens automatically, effortlessly, and normally with astonishing effectiveness.