On the matching of seen and felt shape by newly sighted subjects

Abstract

How do we recognize identities between seen shapes and felt ones? Is this due to associative learning, or intrinsic connections these sensory modalities? We can address this question by testing the capacities of newly sighted subjects to match seen and felt shapes, but only if the subjects can see the objects well enough to form adequate visual representations of their shapes. In light of this, a recent study by R. Held and colleagues fails to demonstrate that their newly sighted subjects' inability to match seen and felt shape was due to a lack of intermodal connections rather than a purely visual deficit, as the subjects may not have been able visually to represent 3D shape in the perspective-invariant manner required for intermodal matching. However, the study could be modified in any of several ways to help avoid this problem.

1. A famous question

In 1688, the Irish politician William Molyneux wrote to John Locke, asking whether a man born blind who could recognize shapes by touch would, upon having his sight restored and the same shapes placed before his eyes, be able to identify seen shapes with felt ones (Locke 1979). To see what is at stake here, consider the way we recognize people by their voices. Upon hearing a familiar voice it will often be possible to identify by sight the person whose voice it is, but only because of associative connections that have been established through the experience of seeing people while hearing them speak. But is this true in general of the relationships between perceptual representations in different sensory modalities? In particular, think of the way that shape and other spatial properties are perceived in sight and touch, of how we can tell right away whether a seen shape is the same as some felt one. Molyneux's question asks: can we do this only because of learned associations built up in the course of past experience, or are the representations of these properties related somehow intrinsically?

Though Molyneux's question assumes that a philosophical dispute can be resolved by scientific experiment, in practice this is no simple matter (Degenaar 1996). In particular, addressing Molyneux's question experimentally requires at least two things. First and most obviously, the subjects who are tested must not have been able at any point to associate sight and touch through perceptual learning. Second, these subjects must be able to see well enough to represent visually the shapes of the objects they are presented with: otherwise failure to match seen shapes with felt ones would not prove any lack of an intrinsic connection between the representations of vision and touch, as opposed to a purely visual deficit. Several philosophers have suggested that demands like these cannot be met, and so Molyneux's question cannot be answered experimentally at all (Evans 1985; Jacomuzzi et al 2003; Noë 2004; Gallagher 2005). However, recent experimental work suggests that those who are given sight late in life may be able to see immediately, challenging the orthodox view of an early ‘critical period’ necessary for visual development (Fine et al 2003; Maurer et al 2005; Ostrovsky et al 2006; Mandavilli 2006), and giving hope for an empirical resolution to this longstanding philosophical debate.

2. The question answered?

In this light, a recent study by R. Held and colleagues purports to resolve Molyneux's question definitively (Held et al 2011; and cf Held 2009). As Held and colleagues note, many past studies on intermodal matching in the newly sighted were carried out ‘without consideration of the visual discriminability’ of the stimuli, and so it could not be assured that failure in intermodal matching was not due simply to the subjects' limited visual abilities. Held and colleagues therefore tested their subjects' visual acuity, as well as their ability to match a visual stimulus object with a target object presented to sight (the ‘vision-to-vision’ task). This is sufficient, they suppose, to ensure that the subjects' visual abilities met the requisite standard.

Strikingly, despite their high rates of success in the two intramodal tasks, all four subjects were near chance in the intermodal touch-to-vision task, where they identified which of two seen stimuli matched a felt one. On this basis, the authors conclude that ‘the answer to Molyneux's question is likely negative’, as ‘the newly sighted subjects did not exhibit an immediate transfer of their tactile shape knowledge to the visual domain’. Within a few days, however, performance in this crucial task had improved, and thus the authors claim that the ability to match seen shape with felt ‘can apparently be acquired after short real-world experiences’.

Given that the subjects tested in this study had no capacity for visual form perception prior to their surgery and were tested as soon as they were allowed to see, it does seem that they cannot have formed learned associations between visual and tactile percepts before they were tested. We should be careful, however, in assuming that the subjects' success in the vision-to-vision task guarantees that they had an appropriate capacity for visual form perception at the time of the experiment. For in principle it is possible that the subjects succeeded in this intramodal task based on a more simple visual representation (eg of approximate overall appearance, object outlines, or identifications of distinct subsections of the object) than the relatively viewpoint-invariant shape representation required for intermodal matching. Moreover, though Held et al permitted eye and head movements during presentation of the visual stimulus objects, these were not recorded, and so there is no way to know if subjects incorporated the richer visual information provided by a range of viewing angles. Thus it is possible that the subjects failed the vision-to-touch task not because of a lack of intrinsic intermodal connections, but because their visual representations of 3D object shape were insufficient to be compared with those of touch. In this case, the experiment would leave Molyneux's question unresolved.

This skeptical assessment is supported by other several other recent studies on the visual capacities of newly sighted subjects. For example, Fine et al (2003) describe a patient who several months after surgery to restore vision in his right eye could identify simple forms but failed to recognize 3D images such as Necker cubes. Similarly, Ostrovsky et al (2009) carried out a series of experiments with three congenitally blind subjects, between 2 weeks and 3 months after surgical restoration of their vision. When presented with simple 3D stimuli (such as a shaded image of a cube or pyramid), these subjects ‘were unable to integrate the facets [of the shape] into the percept of a single three-dimensional object’, perceiving each image as containing several distinct objects instead. The same was true when these subjects were shown photographic images of familiar objects: their visual systems ‘greatly oversegmented the images and partitioned them into meaningless regions, which would be unstable across different views and uninformative regarding object identity’ (emphasis added). Taken together, this is evidence that the surgical restoration of vision provides only a limited capacity for 3D form perception in the period immediately after the surgery, and thus it seems reasonable to infer that Held et al's subjects failed to match seen shape with felt, not because the percepts of touch and vision are intrinsically unrelated, but because they could not form the requisite visual representations of their shapes in the first place. The question of the relationship between the visual and tactile representations of shape remains wide open.

3. Fixing things

Is there a way to modify the Held et al experiment to address this difficulty? A first strategy, which is necessary in any case, would be to test the newly sighted subjects' abilities to identify visual stimulus objects across a range of viewing angles, to ensure that they were forming relatively viewpoint-invariant visual representations of them. Second, it is worth noting that Fine et al (2003) found that their subject could identify 3D shapes with perfect accuracy when the stimuli included visual cues simulating object motion; similarly, Ostrovsky et al (2009) found that their subjects were better at identifying photographic images of objects that are frequently seen in motion in everyday life. These findings suggest that information from object motion is especially useful in visual form perception for newly sighted subjects, and in light of this the Held et al study could be re-run with the stimulus objects presented on a rotating platform, making it easier for subjects to form the robust visual shape representations that intermodal matching demands.

Finally, if these modifications are insufficient then it seems that the stimuli themselves must be simplified: perhaps it would be possible to use geometrically simpler shapes, such as the cube and sphere in Molyneux's original proposal, or planar figures instead of three-dimensional solids, as there is some evidence that newly sighted subjects may be better at identifying 2D shapes than 3D ones (Gregory and Wallace 1963: 17; Sacks 1995: 115, 126; Fine et al 2003; Mandavilli 2006; Ostrovsky et al 2009). While the significance of such a study still might not be entirely unequivocal if subjects fail to match touch and vision (as the question could remain whether it was due to their limited visual capacities), some such simplification may be necessary in order to test for the existence of connections between vision and touch prior to associative learning, as opposed to the ability of newly sighted subjects simply to see the shapes of things at all.

Biography

John Schwenkler received his Ph.D. in 2009 from the University of California, Berkeley, where he studied with Alva Noë and John Campbell. He now teaches in the Department of Philosophy at Mount St. Mary's University in Emmitsburg, Maryland, and has published papers in journals such as Mind & Language, Philosophical Studies, Philosophical Explorations, and Philosophy Compass. His research spans the areas of epistemology and the philosophy of mind and action, especially concerning issues of human knowledge, agency, and the philosophy and science of perception. For more information, visit sites.google.com/site/jlschwenkler