Skip to main content
PMC home page
Proceedings of the National Academy of Sciences of the United States of America logoLink to Proceedings of the National Academy of Sciences of the United States of America
. 1999 Aug 31;96(18):10418–10421. doi: 10.1073/pnas.96.18.10418

Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position

Michael S A Graziano 1,*
PMCID: PMC17903  PMID: 10468623

Abstract

A central problem in motor control, in the representation of space, and in the perception of body schema is how the brain encodes the relative positions of body parts. According to psychophysical studies, this sense of limb position depends heavily on vision. However, almost nothing is currently known about how the brain uses vision to determine or represent the location of the arm or any other body part. The present experiment shows that the position of the arm is represented in the premotor cortex of the monkey (Macaca fascicularis) brain by means of a convergence of visual cues and proprioceptive cues onto the same neurons. These neurons respond to the felt position of the arm when the arm is covered from view. They also respond in a similar fashion to the seen position of a false arm.

The sense of limb position is necessary for the control of movement. For example, in order to move your hand accurately toward a glass of wine, you must process the location of both the glass and the hand. These two pieces of information can be combined to determine the location of the target relative to the hand, a crucial piece of information for guiding the movement (1–4). We use two types of cues to determine the position of the hand: the felt position (proprioception) and the seen position (5–8). However, almost nothing is known about how these signals are combined in the brain. Although dozens of neurophysiological studies, especially in monkeys, have examined the role of vision in the control of movement (9–12), these studies concentrate on vision of a target to which the monkey is reaching. Vision of the hand is an equally important, parallel, but neglected part of the visuomotor pathway.

The premotor cortex of monkeys is a site of convergence of visual, tactile, and proprioceptive information; it is also involved in the control of movement of the mouth, head, and arms (12–15). In the arm portion of the somatotopic map in premotor cortex, many neurons have a tactile response on the hand or arm and also a visual response to stimuli placed near the tactile receptive field (RF), within about reaching distance (13–18). When the eyes move, these visual RFs do not move with the eyes. That is, they are not in retinocentric coordinates. When the arm is moved, the visual RFs move through space in rough correspondence with the arm (17, 18). Although the amount of movement of the visual RF, and the closeness of match to the arm, varies across neurons, an ensemble of these neurons could encode the locations of nearby stimuli relative to the arm, in arm-centered coordinates. This is precisely the information necessary to guide reaching. Indeed, premotor cortex projects to M1 (primary motor cortex) and also directly to the spinal cord (19–23); many of its neurons are active during reaching movements (24); and electrical stimulation of this part of the brain elicits arm movements (24). Premotor cortex, therefore, may be a primary site where the sense of limb position contributes to the control of movement. The current study asked whether premotor neurons use visual, proprioceptive, or both types of information to assess the position of the arm.

MATERIALS AND METHODS

Responses of single neurons in premotor cortex were studied in four tame male Macaca fascicularis (4–5 kg). For details of the experimental procedures, see ref. 17. Many of the neurons were tested in other paradigms for other experiments (16–18). Here we report on the 36 neurons that were tested in the current paradigm. Daily recording sessions were conducted on each monkey while the animal was seated in a primate chair with its head fixed and the arm contralateral to the recording chamber placed in an adjustable holder. A hydraulic microdrive was used to lower a tungsten microelectrode into the portion of premotor cortex just ventral to the spur of the arcuate sulcus, where a high proportion of neurons have tactile RFs on the arm and visual RFs near the arm. Once a neuron was isolated, it was tested for somatosensory and visual responsiveness. Somatosensory RFs were plotted by manipulating the joints and stroking the skin, and visual RFs were plotted with objects presented on a wand.

Neurons that showed a tactile response on the arm or hand and a visual response to objects near the arm were tested further with a 5-cm-diameter ball mounted on the end of a computer-controlled robot. The ball was moved through the space near the arm along the trajectories shown in Fig. 1A. For each stimulus presentation, the stimulus was first moved to the starting position of the trajectory; then it paused for a 10-sec intertrial interval; then it moved toward the monkey for 1 sec at 10 cm per sec; then it paused for 2 sec before moving to the start of the next trajectory. Neurons were tested under four different arm configurations: the arm was held on the contralateral side; the arm was held on the ipsilateral side; the arm was contralateral and covered from the monkey’s view with an opaque plastic barrier; the arm was ipsilateral and covered. The configuration of the arm was changed every four trials during an extended intertrial interval of about 1 min. The four stimulus trajectories and four arm configurations yielded 16 different stimulus conditions, which were presented in an interleaved fashion, usually 10 trials per condition.

Figure 1.

Figure 1

Open in a new tab

Visual responses of a typical premotor neuron with a tactile RF (hatched) on the forearm and hand, and a visual RF within 10 cm of the tactile RF. (A) On each trial, the arm contralateral to the neuron was fixed in one of two positions and the visual stimulus was advanced along one of four trajectories (1–4). For this neuron, the two arm positions were chosen to align the visual RF near the hand and forearm with trajectories 2 and 3. For other neurons, the arm was moved to different extents depending on the location of the visual RF, to better capture the movement of the visual RF with the arm. (B) Responses of the neuron to the four stimulus trajectories when the arm was visible to the monkey. When the arm was fixed on the right, the response was maximum at position 3. When the arm was fixed on the left, the maximum response moved to the left, to position 2. (C) Responses of the neuron when the arm was covered. The movement of the visual RF with the arm was reduced but not eliminated, indicating that the neuron combined both proprioceptive and visual information about the position of the arm. Each point is a mean of 10 trials. Error bars are standard error.

Of the 36 neurons studied in the manner described above, 17 were also tested in a separate block of trials as follows. The monkey was seated with its contralateral arm fixed in the arm holder on the contralateral side. An opaque plastic barrier was placed over the arm. On top of the barrier was placed an arm that had been stuffed and prepared by a taxidermist. The arm was from a monkey of the same species. Thus, the monkey’s real arm was blocked from view while the stuffed arm was visible. The proximal end of the stuffed arm was covered from view by a portion of the monkey’s chair, and the arm extended from the approximate region of the monkey’s shoulder to produce a life-like effect to the eye of the experimenter. We tested each neuron under two arm configurations: when the stuffed arm was on the contralateral side, directly over the real arm; and when the stuffed arm was angled toward the ipsilateral side. The four stimulus trajectories and two arm configurations yielded 8 different stimulus conditions, which were presented in an interleaved fashion, usually 10 trials per condition.

In a previous experiment (16) we showed that the visual RFs in premotor cortex remain anchored to the arm, at the same location, whether or not the monkey performs a fixation task. However, the magnitude of the visual response is reduced for some neurons during the performance of the fixation task. Five of the 36 neurons in this experiment were tested both while the monkey was fixating on a small light placed 23 cm away and while the monkey was not performing any fixation task. The pattern of results was the same in both these conditions for all five neurons, confirming that fixation control was not necessary for this experiment. The remaining 31 neurons were tested without fixation control. Below, I report on the data collected without fixation control.

RESULTS AND DISCUSSION

Fig. 1 shows the results for an example neuron with a tactile RF on the forearm and hand and an adjacent visual RF. The neuron responded best to visual stimuli that moved downward in the region of the visual RF. It also responded somewhat to other directions of motion of a visual stimulus, including radially away from the monkey, toward the monkey, rightward and leftward, and also to stationary stimuli in the space near the tactile RF. For the tests shown in Fig. 1, we used a stimulus that moved along a straight trajectory toward the monkey. When the arm was held on the right and uncovered, visible to the monkey (Fig. 1B, open squares), the neuron responded best to a visual stimulus moving along trajectory 3, directly over the forearm. When the arm was bent toward the left (Fig. 1B, filled diamonds), the visual RF moved toward the left, and the neuron responded best to trajectory 2, again directly over the forearm. That is, the visual RF was anchored to the tactile RF on the forearm and moved in spatial register with that part of the arm. Fig. 1C shows the result for the same neuron when the arm was covered, thereby removing visual information about the position of the arm. Again, when the arm was on the right, the neuron responded best to trajectory 3. When the arm was on the left, the visual RF shifted slightly toward the left, and the neuron responded equally well to position 2 and 3. Because the response depended on arm position, even though the arm was not visible, this neuron must have received proprioceptive input about the position of the arm. However, covering the arm reduced the amount that the visual RF moved with the arm, indicating that the sight of the arm also contributed to the neuron’s information about arm position.

To quantify the amount of shift of the visual response, a shift index (SI) was calculated. First the weighted spatial average, or center of mass, of the visual response was calculated by using the formula CM = (R1 + 2R2 + 3R3 + 4R4)/(R1 + R2 + R3 + R4), where R1 through R4 = the mean neuronal response in spikes/sec during stimulus presentation along trajectories 1 through 4. For the data in Fig. 1B, the center of mass when the arm was on the right was 2.78 and the center of mass when the arm was on the left was 2.30. The SI was then calculated by subtracting one center of mass from the other. For this example neuron, when the arm was uncovered and visible to the monkey (Fig. 1B) SI = 0.48; and when the arm was covered (Fig. 1C), SI = 0.29. Thus, removing the sight of the arm caused a reduction in the effect of arm position on the response of the neuron.

Fig. 2 shows the mean result for all 36 neurons. The mean SI when the arm was “uncovered,” that is, when the monkey had both visual and proprioceptive cues about arm position, was significantly greater than zero, indicating that on average the visual RFs moved when the arm moved (see legend of Fig. 2 for details of statistics). This result is expected, and is a replication of our previous study (17). The SI varied from neuron to neuron, ranging from −0.09 to 0.78. Thus, not all neurons had a visual RF that moved as the arm moved, also in agreement with our previous results. For different neurons, the tactile RF was located on the upper arm, the forearm, and hand, or encompassed the whole arm. Since these parts of the arm moved to different extents, the associated visual RF also moved to different extents in different neurons. This effect also contributed to the variability in the SI.

Figure 2.

Figure 2

Open in a new tab

Mean SI (amount that the visual RF shifted when the arm was moved) under three conditions: first, when the arm was uncovered and the monkey had both visual and proprioceptive information about arm position; second, when the arm was covered and the monkey had only proprioceptive information about arm position; and third, when the monkey’s arm was stationary and blocked from view, and a detached, stuffed monkey arm was placed in view and moved to two different positions. N = number of neurons per mean. Error bars are standard error of the mean. The SI for Arm Uncovered was significantly greater than zero: mean = 0.18, SEM = 0.04, t = 4.97, P < 0.0001. The SI for Arm Covered was significantly greater than zero: mean = 0.08, SEM = 0.03, t = 2.75, P < 0.01. The SI for Visual + Proprioceptive was significantly greater than the SI for Proprioceptive: t test for correlated data, t = 2.77, P < 0.05, adjusted with the Bonferroni correction for multiple tests. The SI for Fake Arm was significantly greater than zero: mean = 0.12, SEM = 0.06, t = 2.20, P < 0.05.

The mean SI when the arm was “covered,” that is, when the monkey had only proprioceptive information about arm position, was also significantly greater than zero (see legend to Fig. 2), indicating that proprioceptive information alone was able to influence the responses of the neurons. Finally, the mean SI was significantly less when the arm was covered than when it was in view, indicating that the sight of the arm also contributed to the neuronal representation of arm position. These results show that both visual and proprioceptive cues about arm position converge on the same neurons.

To explore the effect of the sight of the arm on premotor neurons, we used a monkey arm that had been stuffed by a taxidermist. Of the 36 neurons described above, 17 were further tested in a separate block of trials, during which the monkey’s real arm was covered from view with the opaque barrier and the stuffed arm was placed on top of the barrier. We asked whether the visual RFs of any of the neurons would move as the stuffed, or “visual” arm was moved. Note that in this experiment, the position of the stuffed arm can be in conflict with the felt position of the real arm. Will this conflict nullify the effect of the stuffed arm? Psychophysical evidence from humans shows that when the sight of the arm is dissociated from the real position of the arm by using prisms (6) or a rubber arm (8), the subjects’ sense of arm position is indeed influenced by the visual information. The sensed position lies between the actual position and the seen position. Thus we hypothesized that the sight of the stuffed arm would influence the behavior of the neurons in premotor cortex.

For each neuron, we calculated an SI for the fake arm; that is, the amount that the visual RF moved when the fake arm moved. As shown in Fig. 2, the mean SI for the fake arm was significantly greater than zero, indicating that these neurons were influenced by the sight of the fake arm. When the fake arm moved, the visual RFs on average also moved in the same direction, even though the monkey’s real arm was stationary.

The results were analyzed further as follows. For each of the 17 neurons, three SIs were obtained. SIvisual+proprioceptive indicates the amount that the visual RF shifted with the monkey’s own arm, when it was uncovered and the monkey had both visual and proprioceptive cues about the change in arm position; SIproprioceptive indicates the amount that the visual RF shifted with the monkey’s arm when it was covered from view and the monkey had only proprioceptive cues about the change in arm position; and SIvisual indicates the amount that the visual RF shifted with the stuffed arm, when the monkey was provided with a purely visual signal about a change in arm position. It was hypothesized that these three quantities should be systematically related. Specifically, the difference between SIvisual+proprioceptive and SIproprioceptive should indicate the amount that the neuron is influenced by the sight of the monkey’s own arm, while SIvisual should indicate the amount that the neuron is influenced by the sight of the fake arm. If the neurons are indeed influenced by visual information about arm position, then these two experimentally independent measures should be correlated. As shown in Fig. 3, across the sample of neurons the two measures were highly correlated (r = 0.84, P < 0.0001). There was a range of neurons: some were relatively more influenced by visual information about the monkey’s own arm, and thus were influenced by the sight of the stuffed arm; whereas others were relatively more influenced by proprioceptive information about the monkey’s arm position, and thus were not affected by the sight of the stuffed arm.

Figure 3.

Figure 3

Open in a new tab

Effect of the sight of the monkey’s own arm and of a detached, stuffed monkey arm on responses of premotor neurons. For each neuron, three SIs were calculated. SIvisual+proprioceptive indicates the amount that the visual RF moved with the monkey’s arm, when the arm was uncovered and within view. SIproprioceptive indicates the amount that the visual RF moved with the arm, when the arm was covered from view. SIvisual indicates the amount that the visual RF moved when the monkey’s arm was blocked from view and a stuffed arm was placed in view and moved to different positions. SIvisual+proprioceptive − SIproprioceptive provides an estimate of the amount that the sight of the arm influences the neuron. SIvisual provides an independent measure of the same property. These two measures were highly correlated. The line at slope = 1 is also plotted.

Two neurons showed an especially intriguing property. The visual RFs for these neurons moved with the arm; however, the RFs moved most when the monkey could not see his arm. When he could see his arm, the movement of the visual RF with the arm was reduced. Thus, the quantity SIvisual+proprioceptive − SIproprioceptive was negative. This result suggests that for these two neurons, the visual input was connected up paradoxically, in opposition to the proprioceptive input. The purpose of such crossed signals is not clear. We would expect, however, that when tested with a fake, “visual” arm, the visual RFs of these two neurons would actually move in the opposite direction as the fake arm, moving to the left when the fake arm moved to the right, and vice versa. That is, the SIvisual should also be negative. Such a counterintuitive result would be strong confirmation that, indeed, the fake arm was mimicking the visual effect of the real arm on these neurons. This is precisely the result obtained (left-most two data points in Fig. 3).

We do not know whether or how a monkey is aware of its arm position, or whether the fake arm successfully “fooled” the animal. Instead, these data show that neurons in premotor cortex, part of the neuronal machinery that guides movement, are influenced by the position of the arm; that this influence is both visual and proprioceptive; and that the two influences converge on single neurons. Because premotor cortex lies along the complex neuronal route from visual input to motor output, and because many of the bimodal, visual-tactile neurons in premotor cortex are known to be active during voluntary movement, we suggest that these neurons use their arm position information to guide reaching.

Where does proprioception and vision of the limb become synthesized into a coherent arm-position signal? Do separate visual and proprioceptive inputs converge on premotor neurons, or does the convergence occur at an earlier stage in the processing, such as in the parietal lobe? One group has suggested that vision of the limb and proprioception are already combined in the medial parietal lobe (25), which sends projections to premotor cortex (26). Some of the neurons in this parietal region respond to changes in arm position during a reaching task. Because these neurons behave differently in the light than in the dark, the authors suggested that the sight of the arm influences the behavior of the neurons. However, turning off the lights changes more than the monkey’s vision of his arm, and therefore it is premature to conclude that the sight of the arm affected the neurons. Further study, perhaps with a fake, “visual” arm, will be needed to determine if the sight of the arm and the feel of the arm join to form a coherent arm-position signal in the parietal lobe.

Acknowledgments

I thank C. G. Gross, S. E. Alisharan, and V. Gomez for their help. This work was supported by National Institutes of Health Grant EY11347 and McDonnell Pew Grant 90-16.

ABBREVIATIONS

RF

receptive field

SI

shift index

References

  • 1.Gordon J, Ghilardi M F, Ghez C. Exp Brain Res. 1994;99:97–111. doi: 10.1007/BF00241415. [DOI] [PubMed] [Google Scholar]
  • 2.McIntyre J, Stratta F, Lacquaniti F. J Neurosci. 1998;18:8423–8435. doi: 10.1523/JNEUROSCI.18-20-08423.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Soechting J F, Flanders M. J Neurophysiol. 1989;62:582–594. doi: 10.1152/jn.1989.62.2.582. [DOI] [PubMed] [Google Scholar]
  • 4.Tipper S P, Lortie C, Baylis G C. J Exp Psychol Human Percep Perf. 1992;18:891–905. doi: 10.1037//0096-1523.18.4.891. [DOI] [PubMed] [Google Scholar]
  • 5.Berkinblit M B, Fookson O I, Smetanin B, Adamovich S V, Poizner H. Exp Brain Res. 1995;107:326–330. doi: 10.1007/BF00230053. [DOI] [PubMed] [Google Scholar]
  • 6.Rossetti, Y., Desmurget, M. & Preblanc, C. (1995) 74, 457–463. [DOI] [PubMed]
  • 7.van Beers R J, Sittig A C, Denier van der Gon J J. Exp Brain Res. 1996;111:253–261. doi: 10.1007/BF00227302. [DOI] [PubMed] [Google Scholar]
  • 8.Botvinick M, Cohen J. Nature (London) 1998;391:756. doi: 10.1038/35784. [DOI] [PubMed] [Google Scholar]
  • 9.Andersen R A, Snyder L H, Bradley D C, Xing J. Annu Rev Neurosci. 1997;20:303–330. doi: 10.1146/annurev.neuro.20.1.303. [DOI] [PubMed] [Google Scholar]
  • 10.Rizzolatti G, Luppino G, Matelli M. Electroencephalogr Clin Neurophysiol. 1998;106:283–296. doi: 10.1016/s0013-4694(98)00022-4. [DOI] [PubMed] [Google Scholar]
  • 11.Wise S P, Boussaoud D, Johnson P B, Caminiti R. Annu Rev Neurosci. 1997;20:25–42. doi: 10.1146/annurev.neuro.20.1.25. [DOI] [PubMed] [Google Scholar]
  • 12.Graziano M S A, Gross C G. Curr Opin Neurobiol. 1998;8:195–201. doi: 10.1016/s0959-4388(98)80140-2. [DOI] [PubMed] [Google Scholar]
  • 13.Fogassi L, Gallese V, Fadiga L, Luppino G, Matelli M, Rizzolatti G. J Neurophysiol. 1996;76:141–157. doi: 10.1152/jn.1996.76.1.141. [DOI] [PubMed] [Google Scholar]
  • 14.Gentilucci M, Scandolara C, Pigarev I N, Rizzolatti G. Exp Brain Res. 1983;50:464–468. doi: 10.1007/BF00239214. [DOI] [PubMed] [Google Scholar]
  • 15.Rizzolatti G, Scandolara C, Matelli M, Gentilucci M. Behav Brain Res. 1981;2:147–163. doi: 10.1016/0166-4328(81)90053-x. [DOI] [PubMed] [Google Scholar]
  • 16.Graziano M S A, Gross C G. Exp Brain Res. 1998;118:373–380. doi: 10.1007/s002210050291. [DOI] [PubMed] [Google Scholar]
  • 17.Graziano M S A, Hu X, Gross C G. J Neurophysiol. 1997;77:2268–2292. doi: 10.1152/jn.1997.77.5.2268. [DOI] [PubMed] [Google Scholar]
  • 18.Graziano M S A, Yap G S, Gross C G. Science. 1994;266:1054–1057. doi: 10.1126/science.7973661. [DOI] [PubMed] [Google Scholar]
  • 19.Barbas H, Pandya D N. J Comp Neurol. 1987;256:211–228. doi: 10.1002/cne.902560203. [DOI] [PubMed] [Google Scholar]
  • 20.Dum R P, Strick P L. J Neurosci. 1991;11:667–689. doi: 10.1523/JNEUROSCI.11-03-00667.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Godschalk M, Lemon R N, Kuypers H G J M, Ronday H K. Exp Brain Res. 1984;56:410–424. doi: 10.1007/BF00237982. [DOI] [PubMed] [Google Scholar]
  • 22.Leichnetz G R. J Comp Neurol. 1986;254:460–492. doi: 10.1002/cne.902540403. [DOI] [PubMed] [Google Scholar]
  • 23.Muakkassa K F, Strick P L. Brain Res. 1979;177:176–182. doi: 10.1016/0006-8993(79)90928-4. [DOI] [PubMed] [Google Scholar]
  • 24.Gentilucci M, Fogassi L, Luppino G, Matelli M, Camarda R, Rizzolatti G. Exp Brain Res. 1988;71:475–490. doi: 10.1007/BF00248741. [DOI] [PubMed] [Google Scholar]
  • 25.Ferraina S, Garasto M R, Battaglia-Mayer A, Ferraresi P, Johnson P B, Lacqaniti F, Caminiti R. Eur J Neurosci. 1997;9:1090–1095. doi: 10.1111/j.1460-9568.1997.tb01460.x. [DOI] [PubMed] [Google Scholar]
  • 26.Johnson P B, Ferraina S, Caminiti R. Exp Brain Res. 1993;97:361–365. doi: 10.1007/BF00228707. [DOI] [PubMed] [Google Scholar]

Articles from Proceedings of the National Academy of Sciences of the United States of America are provided here courtesy of National Academy of Sciences

RESOURCES