Abstract
Goodale et al. (1991) reported a striking dissociation between vision for perception and action. They examined DF, a human patient who had damage to her ventral visual stream and suffered from visual form agnosia. She was unable to perceive an object's size but could match the opening of her hand to the object's size during grasping. It was concluded that grasping relied on a separate representation of visual size in the dorsal stream and required no visual input from the ventral stream. This observation inspired the influential perception–action model, which claimed separate visual streams for perception and action. However, in grasping (but not in corresponding perceptual tasks), participants receive haptic feedback after each trial. Using this feedback, DF might compensate for her impaired size-vision. I reexamined DF's grasping behavior using a mirror apparatus to dissociate the image of an object from its physical presence. DF's grasping was only normal when she received haptic feedback. Thus, in grasping, DF can rely on haptic feedback to compensate for her deficit in size-perception. This can explain why her grasping is significantly better than her perceptual performance. The findings emphasize the extent of early interstream interactions and highlight the multimodal nature of sensory processing in the dorsal stream.
Introduction
Goodale and Milner (1992) proposed that vision for perception and action is processed in separate pathways. They adopted the distinction between a ventral system, passing information from V1 to areas in the temporal lobe, and the dorsal system, a visual stream terminating in the parietal lobe (Ungerleider and Mishkin, 1982). In this model, the ventral system serves object and scene recognition and the dorsal system uses vision to guide actions (Milner and Goodale, 2006). But in recent years, many of the model's claims were challenged (Schenk and McIntosh, 2010; Schenk et al., 2011), most notably, its claim that action is immune to perceptual illusions (Smeets and Brenner, 2006; Franz and Gegenfurtner, 2008), its claim that the dorsal stream does not provide observer-invariant visual information (Konen and Kastner, 2008) and does not make a contribution to perception (Schenk, 2006), the assertion that visual form agnosia and optic ataxia constitute a proper double-dissociation (Pisella et al., 2006), and the assumption that memory-based action is not processed by the dorsal stream (Himmelbach and Karnath, 2005; Himmelbach et al., 2009). In response to this critique, Goodale and colleagues noticed that critics fail to take the evidence from their patient, DF, into account and argued that one cannot dispute the two-visual stream hypothesis unless an alternative account for DF's surprisingly good visuomotor behavior is provided (Milner and Goodale, 2008, Goodale and Milner, 2010, Westwood and Goodale, 2011).
Recently, I described an alternative account for DF's visuomotor robustness (Schenk, 2010). I argued that DF's preserved ability for visually guided action might reflect the fact that many visuomotor tasks can be solved on the basis of multiple sensory cues. Some of those cues are not even visual and therefore will not be affected by DF's visual disorder. Grasping provides an example. Our ability to adjust the opening of our hand to the object's size is informed by visual and haptic information about the object's size (Bingham et al., 2007). Haptic feedback is received at the end of a grasp. It allows the subject to adjust their initial visual estimate of the object size and to generate more accurate grip-size adjustments in future grasping movements. The presence of haptic feedback during grasping could thus explain one of the most widely cited studies in support of the perception–action model. In 1991, Goodale and colleagues reported that DF successfully grasps objects whose size she cannot perceive. If the above account is correct, I would expect that DF fails to show normal grasping behavior when haptic feedback is withdrawn.
Materials and Methods
Participants
DF (aged 53 years at the time of testing) and two samples of 10 age-matched healthy controls took part in the study. The first sample of controls (10 females; mean age, 55 years; SD, 5.6 years) took part in tasks 1 and 2; the second sample (9 females; mean age, 50 years; SD, 14.0 years) took part in tasks 3–5. All subjects were right-handed, had normal or corrected-to-normal vision, and volunteered for participation after being informed about the purpose and conduct of the experiment in agreement with the Declaration of Helsinki. The study was performed at the University of Durham, UK and approved by the local ethics committee.
DF suffered a carbon monoxide intoxication in 1988 that led to extensive bilateral brain damage mainly affecting the lateral aspects of the occipital gyri extending into the parasagittal occipitoparietal region (for more details, see James et al., 2003). DF suffers from profound visual form agnosia. A detailed description of her perceptual deficits can be found in Milner et al. (1991).
Testing procedure and apparatus
Except for tasks 1 and 2, the setup (mirror apparatus) presented in Figure 1, a and b, was used. Figure 1c depicts the three different grasping objects. The grasping movements were recorded with a CMS70-motion registration system (spatial resolution, 0.1 mm; sampling frequency, 50 Hz; Zebris). Participants used their right hand. Three markers were placed on the hand, one each at the index finger, thumb, and wrist. At the start of each grasp, participants placed their hand on a start button with their index finger and thumb in contact. The grasping hand was placed behind the mirror. Participants could, therefore, not see their hand during the experiment. LCD goggles (PLATO; Translucent), worn by all participants, turned translucent at the start of the trial and became opaque after trial completion to ensure that participants did not see the experimenter's manipulations between trials. Participants were instructed to grasp the object and to respond as fast and accurately as possible. For each grasping condition, participants were given 20 practice trials.
Tasks
Size-discrimination task (task 1).
In each trial, two objects of different diameters were presented. Three levels of difficulty were used: easy (small vs big objects), medium (medium vs big objects), and difficult (small vs medium objects). Participants were asked to point to the bigger object; their responses were recorded by the experimenter. Each combination was presented 20 times, thus leading to a total of 60 trials.
Manual estimation task (task 2).
Participants were asked to indicate with the index finger and thumb of their right hands the diameter of the presented object. Distance between those fingers, measured with the Zebris System, provided the value for the grip aperture, which was used in the computation of grip-size-scaling slopes (see Data analysis, below). I used the same objects as in task 1 (each object presented 15 times, 45 trials in total, presented in a randomized sequence).
Standard grasping (task 3).
In this condition, participants saw the object at the same position where the objects was felt and picked up. I used the middle position (Fig. 1b). Each of the three different objects was presented 15 times, resulting in a total of 45 trials (Fig. 2a).
Grasping without haptic feedback (task 4).
In this condition, there was no object behind the mirror. In all other respects, the task was identical to task 3 (Fig. 2b).
Grasping with intermittent haptic feedback (task 5).
An object was present behind the mirror in half of all trials. Trials with and without physical object were randomly interleaved. The presence of a physical object behind the mirror was signaled by a red LED (Fig. 2c,d). This meant that participants knew beforehand whether they would come into contact with a physical object or not. This condition was introduced to examine whether knowledge about the presence or absence of a physical object behind the mirror affected grasping performance. In healthy subjects, there is evidence that such intermittent haptic feedback will lead to grasping performance that is almost indistinguishable from grasping with full haptic feedback (Bingham et al., 2007). It is worth noting that the influence of haptic feedback on grasping comes from preceding trials. Since the frequency of haptic feedback in preceding trials will have been the same for the two types of trials, performance in trials with and without object will have benefited from haptic feedback to the same extent. There were six different trials (with vs without physical object and three different objects); each type of trial was repeated 10 times, producing a total of 60 trials presented in a randomized order.
Grasping with dissociated positions (task 6).
There are two relevant differences between manual estimation and standard grasping. One is the availability of haptic feedback in grasping but not manual estimation. The second is the fact that in manual estimation, participants direct their pretend-grasp to a position that differs from the position of the target object. In task 4, I examined the role of haptic feedback; in task 6, I examined the role of dissociating the position of the target object from the grasping position. To dissociate target and grasping position, an object was presented in the middle position but asked participants to pick up a corresponding but invisible object in the far position. This position was indicated by a red LED (Fig. 2e). Importantly, a physical object was present at the far position (behind the mirror), meaning that participants received veridical haptic feedback. Each of the three objects was presented 15 times in a randomized sequence, resulting in a total of 45 trials.
Data analysis
Maximum grip aperture (i.e., maximum distance between index and thumb) was calculated for all grasping tasks. To obtain a single measure for the subjects' ability to match their hand-opening to the size of the object, the slope of the grip-size function was computed using the following formula: [(GMm − GMs)/(Wm − Ws) + (GMb − GMm)/(Wb − Wm)]/2, where GMb, GMm, and GMs are the maximum grip aperture and Wb, Wm, and Ws and the width for big, medium, and small objects, respectively. To test for the presence of classical dissociations, I used Crawford et al.'s criteria and tests (Crawford and Howell, 1998; Crawford and Garthwaite, 2002; Crawford et al., 2003). For task, 1 a binomial test was used.
Results
Size discrimination (task 1)
All healthy observers achieved error-free performance. DF erred in 35% of all trials. Her performance was only above chance in the easy condition (easy: 85% correct, p < 0.003; medium: 65%, p < 0.263; difficult: 45%, p < 0.824). Accordingly, her performance differed significantly from that of healthy observers (t(9) = −333,712, p < 0.00001).
Manual estimation (task 2)
Healthy observers matched the size of the objects well, producing a mean slope of 0.98 (SD, 0.25). In comparison, DF's slope was 0.05. The modified t test confirmed the abnormality of DF's performance (t(9) = −3.585, p < 0.0029).
Standard grasping (task 3)
The results are illustrated in Figure 3. This figure presents the maximal grip aperture of one healthy observer (Fig. 3, left) and DF (Fig. 3, right) as a function of object size. The grip-size slope values for all grasping tasks are presented in Figure 4. DF's grip-size slope was in the normal range (t(9) = −0.445, p < 0.33; Fig. 3). The differences between the size discrimination and grasping and between manual estimation and grasping were more pronounced for DF than for healthy subjects (size vs grasping: t(9) = 63.72, p < 0.000001; manual estimation vs grasping: t(9) = 2.55, p < 0.0307). Therefore, both comparisons (standard grasping vs size discrimination and standard grasping vs manual estimation) fulfilled the criteria of a classical dissociation.
Role of haptic information (task 4)
Without haptic feedback, DF's performance was abnormal (t(9) = −3.708, p < 0.00243; slope measures for healthy participants ranged between 0.76 and 1.56; DF's grip-size slope was 0.07; Fig. 4). The difference between task 3 (standard grasping) and task 4 (no haptic feedback) was significantly bigger for DF than for healthy controls (t(9) = 3524, p < 0.00648).
Role of knowledge (task 5)
DF's grip-size scaling in this task with intermittent haptic feedback was in the normal range. This was true both for trials without object (t(9) = −1.695, p < 0.0621) and trials with object (t(9) = −1307, p < 0.118). Admittedly, her performance was in the lower part of the normal range, but there were also healthy participants with comparably small scaling factors. I conducted a revised standardized difference test (Crawford et al., 2003) to examine whether the performance difference between the condition with and without object was more pronounced for DF than for healthy controls. No significant difference was found (t(9) = 0.435, p < 0.673), indicating that DF's grip-size adjustment was not significantly more affected by the knowledge about the presence or absence of a physical object than that of healthy controls. This is consistent with the assumption that haptic feedback on the previous trial, which is comparable for trials with and without object (see Materials and Methods, above), but not knowledge, which is different for trials with and without object, determines DF's performance. The revised standardized difference test was also applied to examine whether the performance difference between the condition with partial haptic feedback (task 5) and the condition without haptic feedback (task 3) was more pronounced for DF than for healthy controls. This difference was significant (t(9) = 2.348, p < 0.0434), indicating that the (partial) introduction of haptic feedback benefited DF significantly more than healthy controls.
Grasping with dissociated positions (task 6)
DF's grip-size adjustment (i.e., the slope of the grip-size function) was clearly abnormal in this task (t(9) = −3.39, p < 0.004). Moreover, the difference in DF's grip-size adjustment for standard grasping and grasping with dissociated positions was significantly greater than the difference found for healthy subjects (t(9) = 2339, p < 0.044). Thus, there was a classical dissociation between standard grasping and grasping with dissociated positions.
Discussion
Our findings suggest that the critical difference between manual estimation, size discrimination, and grasping is that one task, namely grasping, provides access to sensory information that is unavailable in the other tasks. One such source of information is haptic feedback. Without haptic feedback, DF's grasping performance was not better than her performance in the manual estimation task. When intermittent haptic feedback was provided, her performance improved, but more importantly, DF's performance in this task suggests that it is really haptic feedback and not the expectation of encountering a physical object that is important. This is confirmed by the fact that her grasping performance is the same for trials where she expected an object and for those where no object was expected.
But haptic feedback is not everything. When haptic feedback was provided but target and grasp positions were dissociated, DF failed to produce normal grip-size scaling. It is not entirely clear why DF's performance was so poor in this condition. One possibility is that the dissociation of target and grasp position deprived DF of egocentric cues. In the standard condition (i.e., target and grasp position are identical), index finger and thumb can be directed to the visible edges of the object (Smeets and Brenner, 1999). In this case, it is not necessary to compute the object width, it is sufficient to code the position of the object's grip surfaces relative to the position of the grasping fingers. When target and grasp position are dissociated, the relative distance between the position of fingers and the grip surfaces of the object no longer correspond to the amplitude and direction of the required finger movements and thus a different strategy and different visual cues are required. The fact that DF was successful in the standard but not in the dissociated condition suggests that she was not relying on the one visual cue that was available in both conditions, namely the object's width. It seems that DF relies on a combination of different sensory information, such as haptic feedback, egocentric cues, and probably also visual feedback. While in our study, participants could not see their hand and therefore did not receive visual feedback, visual feedback was available in a study by Westwood and colleagues (2002). This might explain why in their study, DF could adjust her grip-size to 2D objects. Such 2D objects do not provide veridical haptic feedback. DF's success in grasping them might therefore suggest that she does not have to rely on haptic feedback but can also use online visual feedback to guide her fingers to their final grasping positions.
In conclusion, it appears that DF's superior performance in grasping reflects the abundance of sensory information in this condition. Our findings, therefore, challenge the conventional interpretation of one of the most widely cited studies in support of the perception–action model (Goodale et al., 1991). Providing an alternative account for one piece of evidence does not challenge the perception–action model in its entirety, but it suggests that the task that the model's proponents (for example, see Westwood and Goodale, 2011) have assigned to their critics, namely finding alternative accounts for DF's visuomotor behavior, can be solved. These findings also suggest a greater emphasis on interstream interactions and stress the important role of multimodal integration for the successful guidance of actions.
The fact that the range of conditions under which DF can perform normal grip-size scaling is significantly narrower than that of healthy people suggests that ventral stream areas make a critical contribution to the ability of healthy subjects to perform successful grasping movements under a wide range of sensory conditions. In the past, Milner and Goodale (2008) argued that the contribution of the ventral stream to the control of action is restricted to memory-guided action and aspects of motor planning. In contrast, motor programming was seen as the exclusive domain of the dorsal stream (Goodale and Milner, 2010). While this distinction between planning and programming is not without its problems (Schenk, 2010), it is quite uncontroversial that grip-size scaling should be seen as an example of motor programming (Dijkerman et al., 2009). The demonstration of deficits in grip-size scaling after extensive (but not exclusive) ventral stream damage therefore implies that ventral stream areas also contribute to aspects of motor programming. This conclusion is well in line with a recent fMRI study. Verhagen and colleagues (2008) examined the brain activity associated with grasping objects presented at different slants and under different viewing conditions (i.e., monocular vs binocular viewing). An object's orientation in depth can either be decoded on the basis of binocular cues or on the basis of pictorial cues. It is assumed that such pictorial cues are processed in ventral stream areas. Verhagen and colleagues (2008) found that in the absence of binocular cues, activity of ventral stream area LO (lateral occipital area) and its functional connectivity to areas in the dorsal stream significantly increased with increasing object slant. The authors interpreted their findings as evidence for an early interaction of ventral and dorsal stream areas in the sensory control of movements.
The critical role of haptic feedback for DF's success in grasping also highlights the multimodal nature of the sensory control of grasping. It seems that the visual information in the dorsal stream about the target object is not enough to generate an accurate grasping movement. Either haptic information, visual information from the ventral stream, or online visual feedback is also required to produce accurate grasping. This reminds us that the posterior parietal cortex as part of the dorsal stream is not just a visuomotor system but a multimodal integration area (Jackson, 2010).
Footnotes
I thank D.F. and the healthy participants for their participation.
The author declares no financial conflict of interest.
References
- Bingham G, Coats R, Mon-Williams M. Natural prehension in trials without haptic feedback but only when calibration is allowed. Neuropsychologia. 2007;45:288–294. doi: 10.1016/j.neuropsychologia.2006.07.011. [DOI] [PubMed] [Google Scholar]
- Crawford JR, Garthwaite PH. Investigation of the single case in neuropsychology: confidence limits on the abnormality of test scores and test score differences. Neuropsychologia. 2002;40:1196–1208. doi: 10.1016/s0028-3932(01)00224-x. [DOI] [PubMed] [Google Scholar]
- Crawford JR, Howell DC. Comparing an individual's test score against norms derived from small samples. Clin Neuropsychol. 1998;12:482–486. [Google Scholar]
- Crawford JR, Garthwaite PH, Gray CD. Wanted: fully operational definitions of dissociations in single-case studies. Cortex. 2003;39:357–370. doi: 10.1016/s0010-9452(08)70117-5. [DOI] [PubMed] [Google Scholar]
- Dijkerman HC, McIntosh RD, Schindler I, Nijboer TC, Milner AD. Choosing between alternative wrist postures: action planning needs perception. Neuropsychologia. 2009;47:1476–1482. doi: 10.1016/j.neuropsychologia.2008.12.002. [DOI] [PubMed] [Google Scholar]
- Franz VH, Gegenfurtner KR. Grasping visual illusions: consistent data and no dissociation. Cogn Neuropsychol. 2008;25:920–950. doi: 10.1080/02643290701862449. [DOI] [PubMed] [Google Scholar]
- Goodale MA, Milner AD. Separate visual pathways for perception and action. Trends Neurosci. 1992;15:20–25. doi: 10.1016/0166-2236(92)90344-8. [DOI] [PubMed] [Google Scholar]
- Goodale MA, Milner AD. Two visual streams: interconnections do not imply duplication of function. Cogn Neurosci. 2010;1:65–68. doi: 10.1080/17588920903511635. [DOI] [PubMed] [Google Scholar]
- Goodale MA, Milner AD, Jakobson LS, Carey DP. A neurological dissociation between perceiving objects and grasping them. Nature. 1991;349:154–156. doi: 10.1038/349154a0. [DOI] [PubMed] [Google Scholar]
- Himmelbach M, Karnath HO. Dorsal and ventral stream interaction: contributions from optic ataxia. J Cogn Neurosci. 2005;17:632–640. doi: 10.1162/0898929053467514. [DOI] [PubMed] [Google Scholar]
- Himmelbach M, Nau M, Zündorf I, Erb M, Perenin MT, Karnath HO. Brain activation during immediate and delayed reaching in optic ataxia. Neuropsychologia. 2009;47:1508–1517. doi: 10.1016/j.neuropsychologia.2009.01.033. [DOI] [PubMed] [Google Scholar]
- Jackson SR. Is the visual dorsal stream really very visual after all? Cogn Neurosci. 2010;1:68–69. doi: 10.1080/17588920903513177. [DOI] [PubMed] [Google Scholar]
- James TW, Culham J, Humphrey GK, Milner AD, Goodale MA. Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. Brain. 2003;126:2463–2475. doi: 10.1093/brain/awg248. [DOI] [PubMed] [Google Scholar]
- Konen CS, Kastner S. Two hierarchically organized neural systems for object information in human visual cortex. Nat Neurosci. 2008;11:224–231. doi: 10.1038/nn2036. [DOI] [PubMed] [Google Scholar]
- Milner AD, Goodale MA. The visual brain in action. 2 edition. Oxford: Oxford UP; 2006. [Google Scholar]
- Milner AD, Goodale MA. Two visual systems reviewed. Neuropsychologia. 2008;46:774–785. doi: 10.1016/j.neuropsychologia.2007.10.005. [DOI] [PubMed] [Google Scholar]
- Milner AD, Perrett DI, Johnston RS, Benson PJ, Jordan TR, Heeley DW, Betucci D, Mortara F, Mutani R, Terazze E, Davidson DL. Perception and action in ‘visual form agnosia.’. Brain. 1991;114:405–428. doi: 10.1093/brain/114.1.405. [DOI] [PubMed] [Google Scholar]
- Pisella L, Binkofski F, Lasek K, Toni I, Rossetti Y. No double-dissociation between optic ataxia and visual agnosia: multiple sub-streams for multiple visuo-manual integrations. Neuropsychologia. 2006;44:2734–2748. doi: 10.1016/j.neuropsychologia.2006.03.027. [DOI] [PubMed] [Google Scholar]
- Schenk T. An allocentric rather than perceptual deficit in patient D.F. Nat Neurosci. 2006;9:1369–1370. doi: 10.1038/nn1784. [DOI] [PubMed] [Google Scholar]
- Schenk T. Visuomotor robustness is based on integration not segregation. Vis Res. 2010;50:2627–2632. doi: 10.1016/j.visres.2010.08.013. [DOI] [PubMed] [Google Scholar]
- Schenk T, McIntosh RD. Do we have independent visual streams for perception and action? Cogn Neurosci. 2010;1:52–63. doi: 10.1080/17588920903388950. [DOI] [PubMed] [Google Scholar]
- Schenk T, Franz V, Bruno N. Vision-for-perception and vision-for-action: which model is compatible with the available psychophysical and neuropsychological data? Vis Res. 2011;51:812–818. doi: 10.1016/j.visres.2011.02.003. [DOI] [PubMed] [Google Scholar]
- Smeets JB, Brenner E. A new view on grasping. Motor Control. 1999;3:237–271. doi: 10.1123/mcj.3.3.237. [DOI] [PubMed] [Google Scholar]
- Smeets JB, Brenner E. 10 years of illusions. J Exp Psychol Hum Percept Perform. 2006;32:1501–1504. doi: 10.1037/0096-1523.32.6.1501. [DOI] [PubMed] [Google Scholar]
- Ungerleider LG, Mishkin M. Two cortical visual systems. In: Ingle DJ, Goodale MA, Mansfield RJ, editors. Analysis of visual behavior. Cambridge, MA: MIT; 1982. [Google Scholar]
- Verhagen L, Dijkerman HC, Grol MJ, Toni I. Perceptuo-motor interactions during prehension movements. J Neurosci. 2008;28:4726–4735. doi: 10.1523/JNEUROSCI.0057-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Westwood DA, Goodale MA. Converging evidence for diverging pathways: neuropsychology and psychophysics tell the same story. Vis Res. 2011;51:804–811. doi: 10.1016/j.visres.2010.10.014. [DOI] [PubMed] [Google Scholar]
- Westwood DA, Danckert J, Servos P, Goodale MA. Grasping two-dimensional images and three-dimensional objects in visual-form agnosia. Exp Brain Res. 2002;144:262–267. doi: 10.1007/s00221-002-1068-y. [DOI] [PubMed] [Google Scholar]