Summary
The human motor system is remarkably proficient in the online control of visually guided movements, adjusting to changes in the visual scene within 100 ms [1–3]. This is achieved through a set of highly automatic processes [4] translating visual information into representations suitable for motor control [5, 6]. For this to be accomplished, visual information pertaining to target and hand need to be identified and linked to the appropriate internal representations during the movement. Meanwhile, other visual information must be filtered out, which is especially demanding in visually cluttered natural environments. If selection of relevant sensory information for online control was achieved by visual attention, its limited capacity [7] would substantially constrain the efficiency of visuomotor feedback control. Here we demonstrate that both exogenously and endogenously cued attention facilitate the processing of visual target information [8], but not of visual hand information. Moreover, distracting visual information is more efficiently filtered out during the extraction of hand compared to target information. Our results therefore suggest the existence of a dedicated visuomotor binding mechanism that links the hand representation in visual and motor systems.
Graphical Abstract
Highlights
-
•
Reafferent visual hand information is processed independently from attention
-
•
In contrast, online processing of target information is facilitated by attention
-
•
Visual hand information is filtered more efficiently than visual target information
-
•
A dedicated visuomotor binding mechanism links visual and motor systems
The brain utilizes visual information for online motor control with astonishing efficiency. Reichenbach et al. provide evidence for a specialized visuomotor mechanism that links visual hand information to corresponding motor control centers and functions independently of visual attention, which in turn promotes processing of target information.
Results and Discussion
Skilled motor control demands the simultaneous processing of different sources of visual information. For example, when several basketball players jump for the ball at the same time, a player must track visual information pertaining to the target (exafference; the basketball) and the controlled limb (reafference; one’s own hand) while ignoring distracting visual information (e.g., other players’ hands or the background). How does the visuomotor system accomplish this task efficiently? In perceptual tasks, selection of relevant sensory information is considered to be a function of attention [7]. Likewise, one might suggest that visual attention also facilitates processing of all relevant visual information during the control of movement. Directing visual attention to the target of a reach accelerates the initiation of the movement, and overt attention (gaze direction) is therefore typically focused on this target [9, 10]. Consequently, covert visual attention would have to be allocated to the moving limbs to achieve accurate feedback control. When one reaches with both hands to two separate targets simultaneously [11], for instance, this division of resources would place high demands on a limited-capacity visual attention system [7].
To study the role of attention in visual feedback control, we challenged the visuomotor system with a bimanual reaching task. Participants held a robotic manipulandum with each hand and moved these simultaneously to two targets presented in the left and right visual fields while keeping their eyes fixated at a central location (enforced via eye tracking). During the task, participants’ hands were occluded by a horizontally mounted monitor, which displayed the cursors and targets (Figure 1A). The task therefore required participants to simultaneously process visual information from two targets and two cursors. To test whether spatial visual attention influences the processing of visual target and hand information during online control, we manipulated the locus of attention via exogenous cuing. Immediately after the onset of the movement, covert attention was attracted by briefly increasing the luminance [13] of a target or cursor (“flashes”; see Figure 1A). We assessed the influence of attention by displacing the position of one of the targets or cursors perpendicular to the reaching direction 100 ms after the flash [14] (Figure 1A). The side of the displacement was independent of the side of the preceding attention manipulation, but it occurred always on the same object type (target or cursor) as the attentional cue. The displacement evoked an automatic feedback response with the corresponding hand: rightward for target displacements to the right and leftward for cursor displacements to the right [1, 4, 15]. The forces with which participants pushed into “force channels” [16] during interspersed probe trials provided a sensitive assay of the early corrective motor response [1].
We found fast (onsets around 165 ms) and consistent responses to both target and cursor displacements. These indicate that the sensorimotor system, rather than relying exclusively on proprioceptive or efference copy information, is exquisitely sensitive to visual feedback from the hands [17, 18], even when it tracks both hands simultaneously. The force response to target displacements (Figures 1B and 1D) was modulated by the locus of attention. Displacements preceded by the exogenous cue elicited significantly stronger initial responses (for statistical details, see Figure 1D) and produced earlier onsets of the correction (Figure 1E) than uncued displacements. The size of the attentional modulation on feedback responses was of similar magnitude as for simple reaction time tasks [19]. In contrast, exogenous cuing did not modulate the responses to cursor displacements (Figures 1C–1E). The interaction for displacement type (target or cursor) × attention was significant for both response strength (F1,13 = 7.129, p = 0.019) and onset (F1,13 = 10.005, p = 0.008).
To corroborate these findings, we conducted a second experiment in which we manipulated covert attention endogenously using a secondary perceptual task. At the start of a trial, we presented an arrow near fixation that pointed to the left or right (cue validity for the perceptual task: 83%). During the reaching movement, the luminance of one of the targets (or, in separate blocks, the luminance of one of the cursors) was subtly changed for 350 ms. After completing the movement, participants reported whether the luminance had increased or decreased. During the movement, we probed feedback control by displacing either the left or right target (or, in separate blocks, the cursor) 100 ms before the luminance change.
Accuracy on the brightness discrimination task was better on the cued side (F1,18 = 18.449, p < 0.001; Figure 2A), demonstrating successful attention manipulation both for the cursor and target conditions. Corrective motor responses to target displacements were also significantly greater for the attended side (Figure 2B includes statistical details). In contrast, the response to cursor displacements was not modulated by the cue. As in the exogenous case, the displacement type (target or cursor) × attention interaction was significant (F1,18 = 5.030, p = 0.038). The time of response onsets revealed a similar pattern, although here the interaction did not reach significance (Figure 2C).
These results indicate that both exogenous and endogenous visual attention modulate the processing of target information [9]. In contrast, the processing of visual information about the moving limb, although fast and efficient, seemed to be independent of either attentional manipulation. Our failure to detect an effect of the attentional modulation on responses to cursor perturbations is unlikely to be caused by a simple lack of statistical power. The power for detecting an effect of the size of the attentional modulation on responses to target displacements was >80% with our design (82.1% and 98.7% for experiments 1 and 2, respectively). Furthermore, the significant interaction shows that an attentional modulation effect was clearly larger for target displacements than for cursor displacement—if the latter was present at all. Finally, the absence of attentional modulation on cursor displacement responses is unlikely to be a result of a ceiling effect limiting the size of the maximal response to cursor displacements. In an additional control experiment (Supplemental Information available online), we introduced visual distractors alongside targets and cursors to reduce the size of the feedback response. Even though the demand on visual processing increased, additional allocation of visual attention facilitated only target, but not cursor, processing, ruling out a ceiling effect. We therefore suggest that the binding of reafferent visual information about the movement to the corresponding motor command is achieved without (or at least compared to target processing with much less) aid of visual attention. Therefore, we propose the existence of a separate visuomotor binding mechanism that confers a privileged status on visual information representing the kinematics of a moving limb.
If different mechanisms underlie attentional and visuomotor binding, they may also differ in their ability to filter out and ignore distracting objects. Due to the limited capacity of visual attention [7], filtering constitutes a problem in complex natural scenes. When reaching in such a cluttered environment, the visuomotor system faces the same challenge. To test for differences between attentional and visuomotor binding, we conducted a unimanual reaching experiment requiring participants to filter out numerous distracting objects. In order to match perceptual difficulty for the filtering of target and hand information, we presented a target that moved toward the starting position (Figure 3A; Supplemental Experimental Procedures), and participants were instructed to intercept the target with a movement of their right hand while maintaining central fixation (Figure 3A). Between zero and four distractors moved alongside the target and the cursor (the number of distractors matched for the target and the cursor). The target, target distractors, and cursor distractors followed a minimum jerk trajectory, with the onset and speed drawn randomly from the distribution of participants’ own reaching movements (Figure 3B). Given that fixation was tightly controlled, the visual information pertaining to target and hand were very closely matched.
With no distractors, responses to cursor displacements were 32% smaller than responses to target displacements (0.57 N versus 0.39 N; Figure 3E), most likely because hand estimates integrate both visual information (indicating a displacement) and proprioceptive information (indicating no displacement) [20]. Although responses to both target (Figure 3C) and cursor (Figure 3D) displacements decreased with increasing number of distractors, the response to target displacements declined much more rapidly (interaction: F3,27 = 8.571, p < 0.001; Figure 3E). Participants also showed small erroneous responses to distractor displacements. These, however, were not significantly different between perturbation conditions (main effect perturbation [target or cursor]: F1,9 = 0.041, p = 0.844; interaction: F2,18 = 0.079, p = 0.924; Figure 3E, light-blue and orange lines).
The differential performance in filtering was especially apparent after we accounted baseline responsiveness differences by normalizing the data in the distractor conditions to the corresponding data without distractors (Figure 3F). When distractors were introduced, we observed that the normalized response strength for cursor displacements remained significantly higher than did responses to target displacements (F1,9 = 19.968, p = 0.002; Figure 3F, bold lines). Interestingly, the difference in the filtering properties for target and cursor information was especially pronounced in the very early phase of the response (F1,9 = 25.843, p < 0.001; Figure 3F, thin lines). When averaged over the first 140–170 ms, the response to a cursor perturbation with up to two distractors did not decline from the level of response without distractors (t9 < 0.44, p > 0.2).
These results demonstrate that responses to cursor displacements are more robust against the presence of distracting objects than responses to target displacements. This dissociation further supports the existence of an attention-independent visuomotor binding mechanism that extracts visual information about the body. The difference in filtering efficiency can only be attributed to different processing mechanisms because we carefully matched low-level visual characteristics such as motion energy, visual hemifield, and visual eccentricity. Our result with four distractors also shows that even the filtering through visuomotor binding is resource limited—but taken together, the results indicate that these resource limitations are different from those imposed by attentional processes.
A dedicated visuomotor binding mechanism during voluntary movements would explain the efficiency and speed with which humans can execute multiple goal-directed movements at the same time [11]. This is not because visuomotor binding provides faster responses than other processes, but because its independence of visual attention frees those limited resources for allocating them to the current target during reaching [9] and to potential alternative targets or interfering objects [8]. Why the brain has developed a specialized mechanism for processing reafferent visual information, instead of relying on the general-purpose mechanisms of attention, remains to be answered by future research. We can only speculate that the necessity to react very rapidly to divergent visual information about both the target and the hand has been evolutionary important enough to justify the development of a specialized mechanism. While attention is already involved in the detection and selection of targets before movement onset [8, 9], it seems sensible that it should retain this function during online control, supporting the flexibility of the visuomotor system to adjust movement goals midreach. In contrast, the effector is rarely changed within a movement, and visuomotor binding heavily depends on efference copy and proprioceptive information. These special characteristics may have further promoted the emergence of a dedicated process.
Note that we are not claiming that processing of the target during the online control of movement is conscious or voluntary. On the contrary, there is substantial evidence that online corrections to target displacements can bypass voluntary control [4, 21] and even occur without conscious awareness [15]. The same has been shown for processing of visual hand information [1, 20]. What we show here is that this automatic target processing can be facilitated by the allocation of visual attention—whereas the processing of hand information cannot. Thus, our results demonstrate that even though processing of visual target and hand information share some features regarding their automatic and involuntary nature, the processing of visual hand information appears to occur through a dedicated channel that is uninfluenced by the allocation of visual attention.
We suggest that the visuomotor binding mechanism detects spatiotemporal correlation between objects in the visual scene and internal state estimates of moving limbs. This internal estimate is informed by proprioceptive information, and the predictions arising from a forward model through an efference copy [22] of the executed motor commands [23–25]. Visuomotor binding is complicated by the fact that the spatial relationship between movement and visual consequences is often highly task dependent. However, the ease with which we handle tools [26] or remotely controlled objects such as computer cursors [27] suggests that it is a highly flexible process that can learn new mappings between motor commands and visual outcomes. The factors influencing these adaptive processes and their relationship to the adaptation of dynamic forward models [28], however, have yet to be elucidated.
Assignment of visual input to one’s own action has been debated extensively as “intention or action attribution” or “agency” in the context of conscious perception [29, 30]. These studies show that sensory events judged to be consequences of one’s own actions are perceived differently from sensory events judged to arise from external causes [31–33]. Our results emphasize that the detection of visual stimuli pertaining to our own movements is a fundamental process for the online control of reaching movements. The link between actions and visual consequences allows the motor system to respond rapidly to visual feedback signaling reach errors, even in the face of potentially distracting visual information as present in naturalistic visual scenes. Visuomotor binding is most likely a phylogenetically old mechanism common to all species that rely on vision for movement guidance. It must therefore constitute a central concept in theories of visuomotor feedback control. Indeed, it is possible that this process forms the basis upon which the percept of a “sense of agency” is founded.
The proposal of a dedicated mechanism raises the question of how visuomotor binding is implemented in the nervous system. Visual attention arises through top-down influences of parietal and prefrontal areas onto sensory regions, increasing signal-to-noise in these regions through an increase in sensitivity and contrast gain of relevant neurons [34–37]. Visuomotor binding could act through a conceptually similar yet independently implemented mechanism. For example, one may hypothesize that the link is established by neural synchronization between sensory areas and corresponding motor cortical regions [38] or upstream premotor and parietal regions coding visual space near the hand [39]. Sudden sensory changes would then be directly transmitted to the respective motor circuits for rapid feedback control.
While more work remains to be done to further characterize the proposed mechanisms behaviorally and shed light on its neural implementation, our current results provide the first strong evidence for the existence of a visuomotor binding mechanism that is dissociable from general visual attention. This specialized mechanism constitutes the connecting link between sensory and motor systems by providing a privileged channel for reafferent visual information.
Experimental Procedures
Further details about the experimental procedures and the control experiment are described in the Supplemental Information.
Acknowledgments
We thank M. Husain, D. Wolpert, P. Haggard, and E. Heerey for helpful comments on earlier versions of the manuscript. This research was funded by the Biotechnology and Biological Science Research Council (BBSRC; BB/J009458/1), a postdoctoral fellowship of the Deutsche Forschungsgemeinschaft (DFG; RE 3265/1-1) to A.R., and a Wellcome Trust RCD Fellowship to D.W.F. (WT091547MA).
Footnotes
This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/3.0/).
Supplemental Information
References
- 1.Franklin D.W., Wolpert D.M. Specificity of reflex adaptation for task-relevant variability. J. Neurosci. 2008;28:14165–14175. doi: 10.1523/JNEUROSCI.4406-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Reichenbach A., Thielscher A., Peer A., Bülthoff H.H., Bresciani J.P. Seeing the hand while reaching speeds up on-line responses to a sudden change in target position. J. Physiol. 2009;587:4605–4616. doi: 10.1113/jphysiol.2009.176362. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Soechting J.F., Lacquaniti F. Modification of trajectory of a pointing movement in response to a change in target location. J. Neurophysiol. 1983;49:548–564. doi: 10.1152/jn.1983.49.2.548. [DOI] [PubMed] [Google Scholar]
- 4.Pisella L., Gréa H., Tilikete C., Vighetto A., Desmurget M., Rode G., Boisson D., Rossetti Y. An ‘automatic pilot’ for the hand in human posterior parietal cortex: toward reinterpreting optic ataxia. Nat. Neurosci. 2000;3:729–736. doi: 10.1038/76694. [DOI] [PubMed] [Google Scholar]
- 5.Andersen R.A., Snyder L.H., Bradley D.C., Xing J. Multimodal representation of space in the posterior parietal cortex and its use in planning movements. Annu. Rev. Neurosci. 1997;20:303–330. doi: 10.1146/annurev.neuro.20.1.303. [DOI] [PubMed] [Google Scholar]
- 6.McGuire L.M., Sabes P.N. Sensory transformations and the use of multiple reference frames for reach planning. Nat. Neurosci. 2009;12:1056–1061. doi: 10.1038/nn.2357. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Pashler H.E. MIT Press; Cambridge: 1999. The Psychology of Attention. [Google Scholar]
- 8.Tipper S.P., Lortie C., Baylis G.C. Selective reaching: evidence for action-centered attention. J. Exp. Psychol. Hum. Percept. Perform. 1992;18:891–905. doi: 10.1037//0096-1523.18.4.891. [DOI] [PubMed] [Google Scholar]
- 9.Neggers S.F., Bekkering H. Ocular gaze is anchored to the target of an ongoing pointing movement. J. Neurophysiol. 2000;83:639–651. doi: 10.1152/jn.2000.83.2.639. [DOI] [PubMed] [Google Scholar]
- 10.Rushworth M.F., Ellison A., Walsh V. Complementary localization and lateralization of orienting and motor attention. Nat. Neurosci. 2001;4:656–661. doi: 10.1038/88492. [DOI] [PubMed] [Google Scholar]
- 11.Diedrichsen J., Nambisan R., Kennerley S.W., Ivry R.B. Independent on-line control of the two hands during bimanual reaching. Eur. J. Neurosci. 2004;19:1643–1652. doi: 10.1111/j.1460-9568.2004.03242.x. [DOI] [PubMed] [Google Scholar]
- 12.Cohen J. Second Edition. Lawrence Erlbaum Associates; Hillsdale: 1988. Statistical Power Analysis for the Behavioral Sciences. [Google Scholar]
- 13.Posner M.I., Cohen Y. Components of visual orienting. In: Bouma H., Bouwhuis D., editors. Attention and Performance X. Lawrence Erlbaum Associates; London: 1984. pp. 531–555. [Google Scholar]
- 14.Klein R.M. Inhibition of return. Trends Cogn. Sci. 2000;4:138–147. doi: 10.1016/s1364-6613(00)01452-2. [DOI] [PubMed] [Google Scholar]
- 15.Prablanc C., Martin O. Automatic control during hand reaching at undetected two-dimensional target displacements. J. Neurophysiol. 1992;67:455–469. doi: 10.1152/jn.1992.67.2.455. [DOI] [PubMed] [Google Scholar]
- 16.Scheidt R.A., Reinkensmeyer D.J., Conditt M.A., Rymer W.Z., Mussa-Ivaldi F.A. Persistence of motor adaptation during constrained, multi-joint, arm movements. J. Neurophysiol. 2000;84:853–862. doi: 10.1152/jn.2000.84.2.853. [DOI] [PubMed] [Google Scholar]
- 17.Saunders J.A., Knill D.C. Humans use continuous visual feedback from the hand to control fast reaching movements. Exp. Brain Res. 2003;152:341–352. doi: 10.1007/s00221-003-1525-2. [DOI] [PubMed] [Google Scholar]
- 18.Saunders J.A., Knill D.C. Visual feedback control of hand movements. J. Neurosci. 2004;24:3223–3234. doi: 10.1523/JNEUROSCI.4319-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Posner M.I., Snyder C.R., Davidson B.J. Attention and the detection of signals. J. Exp. Psychol. 1980;109:160–174. [PubMed] [Google Scholar]
- 20.Sarlegna F., Blouin J., Bresciani J.P., Bourdin C., Vercher J.L., Gauthier G.M. Target and hand position information in the online control of goal-directed arm movements. Exp. Brain Res. 2003;151:524–535. doi: 10.1007/s00221-003-1504-7. [DOI] [PubMed] [Google Scholar]
- 21.Day B.L., Lyon I.N. Voluntary modification of automatic arm movements evoked by motion of a visual target. Exp. Brain Res. 2000;130:159–168. doi: 10.1007/s002219900218. [DOI] [PubMed] [Google Scholar]
- 22.von Holst E., Mittelstaedt H. Das Reafferenzprinzip. Naturwissenschaften. 1950;37:464–476. [Google Scholar]
- 23.Desmurget M., Grafton S. Forward modeling allows feedback control for fast reaching movements. Trends Cogn. Sci. 2000;4:423–431. doi: 10.1016/s1364-6613(00)01537-0. [DOI] [PubMed] [Google Scholar]
- 24.Wolpert D.M., Ghahramani Z. Computational principles of movement neuroscience. Nat. Neurosci. 2000;3(Suppl):1212–1217. doi: 10.1038/81497. [DOI] [PubMed] [Google Scholar]
- 25.Wolpert D.M., Ghahramani Z., Jordan M.I. An internal model for sensorimotor integration. Science. 1995;269:1880–1882. doi: 10.1126/science.7569931. [DOI] [PubMed] [Google Scholar]
- 26.Johnson-Frey S.H. The neural bases of complex tool use in humans. Trends Cogn. Sci. 2004;8:71–78. doi: 10.1016/j.tics.2003.12.002. [DOI] [PubMed] [Google Scholar]
- 27.Brenner E., Smeets J.B. Fast corrections of movements with a computer mouse. Spat. Vis. 2003;16:365–376. doi: 10.1163/156856803322467581. [DOI] [PubMed] [Google Scholar]
- 28.Kawato M., Furukawa K., Suzuki R. A hierarchical neural-network model for control and learning of voluntary movement. Biol. Cybern. 1987;57:169–185. doi: 10.1007/BF00364149. [DOI] [PubMed] [Google Scholar]
- 29.de Vignemont F., Fourneret P. The sense of agency: a philosophical and empirical review of the “Who” system. Conscious. Cogn. 2004;13:1–19. doi: 10.1016/S1053-8100(03)00022-9. [DOI] [PubMed] [Google Scholar]
- 30.Haggard P. Conscious intention and motor cognition. Trends Cogn. Sci. 2005;9:290–295. doi: 10.1016/j.tics.2005.04.012. [DOI] [PubMed] [Google Scholar]
- 31.Chaminade T., Decety J. Leader or follower? Involvement of the inferior parietal lobule in agency. Neuroreport. 2002;13:1975–1978. doi: 10.1097/00001756-200210280-00029. [DOI] [PubMed] [Google Scholar]
- 32.Farrer C., Frith C.D. Experiencing oneself vs another person as being the cause of an action: the neural correlates of the experience of agency. Neuroimage. 2002;15:596–603. doi: 10.1006/nimg.2001.1009. [DOI] [PubMed] [Google Scholar]
- 33.Moore J.W., Ruge D., Wenke D., Rothwell J., Haggard P. Disrupting the experience of control in the human brain: pre-supplementary motor area contributes to the sense of agency. Proc. Biol. Sci. 2010;277:2503–2509. doi: 10.1098/rspb.2010.0404. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Moran J., Desimone R. Selective attention gates visual processing in the extrastriate cortex. Science. 1985;229:782–784. doi: 10.1126/science.4023713. [DOI] [PubMed] [Google Scholar]
- 35.Reynolds J.H., Pasternak T., Desimone R. Attention increases sensitivity of V4 neurons. Neuron. 2000;26:703–714. doi: 10.1016/s0896-6273(00)81206-4. [DOI] [PubMed] [Google Scholar]
- 36.Fries P., Reynolds J.H., Rorie A.E., Desimone R. Modulation of oscillatory neuronal synchronization by selective visual attention. Science. 2001;291:1560–1563. doi: 10.1126/science.1055465. [DOI] [PubMed] [Google Scholar]
- 37.Niebur E., Hsiao S.S., Johnson K.O. Synchrony: a neuronal mechanism for attentional selection? Curr. Opin. Neurobiol. 2002;12:190–194. doi: 10.1016/s0959-4388(02)00310-0. [DOI] [PubMed] [Google Scholar]
- 38.Fries P. Neuronal gamma-band synchronization as a fundamental process in cortical computation. Annu. Rev. Neurosci. 2009;32:209–224. doi: 10.1146/annurev.neuro.051508.135603. [DOI] [PubMed] [Google Scholar]
- 39.Brozzoli C., Gentile G., Petkova V.I., Ehrsson H.H. FMRI adaptation reveals a cortical mechanism for the coding of space near the hand. J. Neurosci. 2011;31:9023–9031. doi: 10.1523/JNEUROSCI.1172-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.