Summary
Humans, many animals, and certain robotic hands have deformable fingertip pads [1, 2]. Deformable pads have the advantage of conforming to the objects that are being touched, ensuring a stable grasp for a large range of forces and shapes. Pad deformations change with finger displacements during touch. Pushing a finger against an external surface typically provokes an increase of the gross contact area [3], potentially providing a relative motion cue, a situation comparable to looming in vision [4]. The rate of increase of the area of contact also depends on the compliance of the object [5]. Because objects normally do not suddenly change compliance, participants may interpret an artificially induced variation in compliance, which coincides with a change in the gross contact area, as a change in finger displacement, and consequently they may misestimate their finger’s position relative to the touched object. To test this, we asked participants to compare the perceived displacements of their finger while contacting an object varying pseudo-randomly in compliance from trial to trial. Results indicate a bias in the perception of finger displacement induced by the change in compliance, hence in contact area, indicating that participants interpreted the altered cutaneous input as a cue to proprioception. This situation highlights the capacity of the brain to take advantage of knowledge of the mechanical properties of the body and of the external environment.
Highlights
-
•
Pushing a finger against a soft surface provokes an increase of the contact area
-
•
This increase in contact area potentially provides a cue to finger displacement
-
•
We ran psychophysical experiments to test this hypothesis
-
•
Results revealed a novel proprioceptive cue, i.e., the change in contact area
Pushing a finger against an external surface provokes an increase of the contact area. Moscatelli, Bianchi, et al. show with psychophysical experiments that this increase in contact area provides a cue to finger displacement, similarly to looming in vision. Their results show that the change in contact area provides a novel proprioceptive cue.
Results
The size of the image that an object projects on the retina, referred to as the retinal size of the object, changes with the distance of the object to the observer. In order to use the retinal size as a depth cue, observers assume heuristically that two otherwise identical objects in the visual scene, such as two identical columns in a colonnade, have the same physical size [4]. A violation of this assumption produces a misestimate in the perceived depth. For instance, the baroque architect Borromini scaled the physical size of the columns in the colonnade of Galleria Spada to produce an illusory sensation of depth [6]. Similarly, a progressive increase in the retinal size of an object produces the sensation of the object approaching the observer, a phenomenon known as visual looming [7, 8, 9]. In this study, we investigated whether an analog of looming exists in the sense of touch and whether the tactile system can be deceived, similarly to vision, by altering unbeknownst to the observer some of the object properties that are usually assumed to remain stable.
The formation of contact between the skin of the fingertip and an external object produces a progressive recruitment of strained tissue that evokes characteristic neural responses [10, 11, 12]. Thus, in principle, an observer could infer the relative displacement of the finger from this change in the area of contact, because a larger area corresponds to a greater finger displacement from initial contact (Figure 1). The rate of change of the area of contact also depends on the compliance of the object that the observer can estimate by combining multisensory cues and prior knowledge [5, 13]. Compliance is usually a stable property of each specific object. If the compliance of a given object would suddenly change between two sequential tactual interactions, an observer might attribute the resulting change in the contact area to a difference in the finger indentation. This occurrence should result in a misestimate of the relative finger displacement. Therefore, by modifying the compliance of an object in a controlled manner, we can quantitatively assess the contribution of the change in gross contact area as a cue to relative finger displacement.
Figure 1.
The Change in Contact Area
(A) The area of contact between the skin and the silicon increases as the finger moves toward the bottom edge of the object.
(B) The increase in the area recorded from the camera of the softness display for a surface compliance comparable with the silicon. The expanding contact area can be seen as successive frames of a movie of a looming stimulus.
(C and D) A more-compliant object is associated with a higher rate of change of the area of contact.
Each trial consisted of a reference and a comparison stimulus randomly presented in two sequential time intervals. In each interval, the elastic surface of the apparatus (Figure 2) was lifted to come in contact with the participant’s index finger and to passively move it up and down. Participants reported in which of the two intervals the extent of the angular displacement of the finger (i.e., the angle in Figure 2B) was greater. In each of the two intervals and before the elastic surface contacted the finger, we modified its compliance by means of a computer-controlled device that adjusted the stretch of the surface [14, 15]. The experimental apparatus also ensured that the actual finger displacements and interaction forces were uncorrelated with the surface compliance (Supplemental Experimental Procedures). The participants were not informed that the compliance could change between the two intervals. In two different blocks, the surface was either “softer” or “stiffer” in the comparison than in the reference, thus leading, for equal finger displacements, to a “larger” or “smaller” contact area in the comparison, respectively (Figure 3A). The order of the blocks (subsequently named “large versus small” condition) was counterbalanced across participants. The testing procedures were approved by the Ethical Committee of the University of Pisa, in accordance with the guidelines of the Declaration of Helsinki for research involving human subjects. Informed written consent was obtained from all participants involved in the study.
Figure 2.
Experimental Setup and Protocol
(A) The setup included the lift, the softness display FYD-2, and the angle encoder. The FYD-2 modified the compliance of an elastic contact surface by adjusting its stretching state. A rotational spring produced a linear increase of the force exerted by the finger on the surface through the lifting phase of the stimulus. (Adapted with permission from Moscatelli, Bianchi, et al., Eurohaptics Conference 2014, Paris.)
(B) In each interval, the elastic surface (in red in the figure) was lifted to contact the participant’s index finger and to passively move it up and down. The angle of the metacarpo-phalangeal joint, γ, was measured from the angle encoder and used to control the movement of the lift.
Figure 3.
Results
(A) The area of contact between the fingertip and the surface changes with the displacement of the finger and the compliance of the surface (results from a representative participant). The tonality of gray of each curve, from black to light gray, stands for the compliance of the surface. In (A)–(C), the dashed line indicates the extent of the finger displacement in the reference stimulus, γ0 =12°.
(B) The psychometric functions and the responses of a representative participant in the two experimental conditions. The horizontal bars indicate the PSE ± the SE (PSElarge = 9.7° ± 0.5°; PSEsmall = 12.8° ± 0.6°).
(C) The PSE ± 95% confidence interval in the two experimental conditions (n = 11).
See also Figures S1–S3.
We considered three possible hypotheses: (1) participants were not sensitive to the change in the area of contact; (2) participants were sensitive to the change in the area of contact and attributed it to the change in compliance of the object; or (3) participants were sensitive to the change in the area of contact and attribute it, mostly or entirely, to the indentation of the finger, hence to its relative displacement. If hypotheses 1 or 2 were true, the change in the area of contact should not have affected the perceived finger displacement. In contrast, hypothesis 3 would predict a perceptual bias. To test these hypotheses, we fit the binary responses of each participant with two psychometric functions, one for the large and one for the small condition (Figure 3B). In order to quantify perceptual biases, we computed the point of subjective equality (PSE) from each psychometric function corresponding to the stimulus value yielding a response probability of 0.5. If participants used the change of the contact area as a cue for the finger displacement, as in hypothesis 3, the PSEs would be significantly different between the two experimental conditions, with . We analyzed the data of all participants (n = 11) using a paired t test and confirmed the result with a generalized linear mixed model (GLMM) [16].
Overall, the value of the was significantly lower than the (paired t test; = 4.47; p = 0.001), in accordance with hypothesis 3 (Figure 3C). The mean of the difference was 2.2°, corresponding to 18% of the reference stimulus (12°). The GLMM confirmed the result of the t test. Accordingly, the 95% confidence interval (CI) of the difference in PSEs did not include zero (95% CI of the difference ranging from 1.5° to 2.7°). The PSE was significantly lower than the reference stimulus in the large condition () and significantly higher in the small condition (). The GLMM fit is illustrated in Figure S1 and shows that the effect sign was consistent across participants. In summary, a more-compliant surface produced a greater expansion of the area of contact, which was in turn associated with a greater perceived angular displacement of the finger.
In experiment 1, the difference in compliance between the two conditions generated small differences in finger kinematics. The initial movement of the lift (Figure 2) produced an increase in the contact area, i.e., an indentation of the finger in the compliant surface, without any finger or joint movement. Because of the difference in compliance, the duration of this indentation phase, hence the finger motion onset, differed between the two experimental conditions. We performed a second experiment (experiment 2a) to ascertain that this difference had a negligible impact on the perceptual bias of finger displacement. Participants wore a rigid thimble covering the pad of their index finger that contacted the device. Apart from the use of the thimble, the apparatus and the task were the same as in experiment 1. As in the first experiment, the more- and the less-compliant conditions (subsequently termed “soft versus stiff” condition) differed in terms of finger kinematics, duration of the indentation phase, the temporal evolution of the load on the finger pad (the stiff condition was associated with a steeper increase of the load), and possibly other cues such as duration of uncontrolled vibrations in the apparatus. The soft and the stiff conditions, however, were identical in terms of change in gross contact area. An absence of perceptual bias, estimated from the difference in PSE, would imply that the effect in the first experiment was primarily due to a change in contact area and not due to any of the other differences between the two conditions. In experiment 2b with covered fingertip, the PSEs were not significantly different between the soft and the stiff conditions (mean of the difference = 0.03°; = 0.05; p = 0.96). Likewise, the 95% CI of the difference in PSEs included zero (95% CI ranging from −0.8° to 0.3°), indicating the importance of the contact area spread as a cue to proprioception. The GLMM fit is illustrated in Figure S2. We further analyzed the PSEs of experiments 1 and 2a together using a 2 × 2 nested ANOVA with factors experiment (1 versus 2a) and stimulus type (more versus less compliant). The interaction between the two factors was statistically significant (), confirming the different effect of stimulus type in the two experiments. In an additional control experiment (experiment 2b), we reproduced the delays of the motion onset observed in experiment 1 by controlling the movement of a rigid platform lifting the participants’ bare fingertip. Experiment 2b confirmed that the delay of the motion onset did not produce any significant perceptual bias (Supplemental Experimental Procedures).
Our results imply that observers were sensitive to a change in the area of contact. Hence, if we informed the participants that the compliance could change across trials, they should have been able to discriminate the difference in compliance based on the change in the area of contact. To verify this hypothesis, we conducted a third experiment (Supplemental Experimental Procedures) where the finger was immobilized and where participants were instructed to discriminate the differences in compliance of the surface based on the differences in the area of contact. In accordance with previous studies [5, 17, 18, 19], participants were able to discriminate the stimuli. That is, the slope parameter of the GLMM, which reflected discriminability for compliance, was significantly different from zero (slope = 1.36; p < 0.001). The just noticeable difference (JND) was equal to 0.49 mm N−1 (95% CI ranging from 0.33 to 0.96 mm N−1).
In summary, we provided evidence that observers were sensitive to the change of the area of contact with an external surface and used it as a cue to the relative motion of the finger, i.e., as a cue for proprioception. The change in contact area is qualitatively different from other displacement cues because it provides information that is expressible in units of surface and not in units of length. This observation raises the question about the integration of this newly described cue with other proprioceptive cues. Landy et al. proposed a model for the integration of qualitatively different cues in vision, known as “modified weak fusion” [20]. We applied a similar model to our results (Supplemental Experimental Procedures and Figure S3). The model was based on a two-step algorithm. The contact area cue was first calibrated (or promoted) using auxiliary information (i.e., with auxiliary proprioceptive cues conveying information expressed in units of length). In the second step, the calibrated cue was combined with the other cues to provide the fused estimate. The model predicted that the perceptual bias was proportional to the difference in contact area (computed at the same angular displacement of 12°) between the reference and the comparison stimulus. Hence, the model predicted a negative bias for the large condition and positive bias for the small condition and a larger absolute value of the bias in case of the large condition. The two predictions were consistent with the results of the experiment 1. The weight of the area-based cue in the fused estimate changed between the two experimental conditions, in accordance with a robust estimation hypothesis [20].
Visual and tactile looming are both ambiguous cues. In vision, the retinal size of an object depends multiplicatively on its actual size and on its proximity to the viewer. In touch, the tactile size of a deformable object depends multiplicatively on its compliance and on finger displacement. In touch, as in vision, the observer can resolve these ambiguities by assuming object constancy, i.e., by assuming that objects have a constant size [4], or a constant compliance. A Bayesian model assuming an observer’s prior belief that the compliance of a given object does not change fits our behavioral data with the consequence that a violation of the constancy assumption generates a perceptual illusion. To use Palmer’s words [4], “constancy and illusion are therefore opposite sides of the same perceptual coin.”
Discussion
Many mammals, birds, and other species have soft pads on the volar side of their extremities. The soft pads rapidly conform to external surfaces ensuring secure grips and stable interactions with objects [1]. Soft fingers have been also used in robotic hands to increase grasp stability [2, 21]. Besides having advantages for grasping, the patterns of pad deformation (such as the change in contact area and the deformation due to slip and roll motion) also provide rich information to the tactile system [12]. The evolution of skin strain patterns during tactile slip provides relative motion information similar to optic flow in vision [22]. Observers can reproduce displacement paths by integrating tactile slip motion over time [23] and experience the shape of curved objects from a rolling interaction of finger pads with an object [24]. The evolution of the gross contact area provides information about the softness of touched objects [5, 15, 17, 18, 19], a finding which is confirmed by our third experiment.
Although the role of pad deformation in cutaneous touch is well established, its contribution to proprioception is less clear. In the present study, we demonstrated that a change in the contact area provides a cue to finger displacement relative to an object. According to classical studies in physiology, muscle spindles, Golgi tendon organs, and receptors in the joints provide crucial information on the static position and movement of our limbs [25]. Information from cutaneous mechanoreceptors also contributes to our sense of position [26, 27, 28]. Stretching the skin around the proximal interphalangeal joint, i.e., around the second knuckle, induced a vivid sensation of movement in anesthetized fingers [29]. Furthermore, during the movement of the elbow joint, skin stretch in a direction in line with muscle stretch applied simultaneously with external vibrations increased the perceived movement sensation [30]. The literature has largely overlooked the role of finger pad deformation due to object interaction as a cue to relative motion. In the current study, changes in the gross contact area produced during the indentation of an elastic surface induced a sensation of relative finger motion. Recently, it was found that, when pushing with a finger against a stiff, stationary object, microscopic fluctuations in the counter-surface could elicit a sensation of finger displacement [27]. These results provide converging evidence that an important source of proprioceptive information comes from skin deformation during interaction with external objects.
In the experiments presented here, the change in gross contact area provided a motion cue that could be compared to looming in vision. This effect, termed “tactile looming,” supports the hypothesis that similar motion detection processes are implemented in vision and touch [31]. In the two sensory systems, a 2D sensory sheet (i.e., the retina or the skin) provides important information about the relative motion of our own body with respect to external objects. Moreover, previous studies showed an analog of visual looming in audition [32, 33], which might suggest a canonical computation of looming stimuli across different senses.
Author Contributions
A.M., M.B., A.S., V.H., M.O.E., and A.B. conceived and designed the experiments. M.B., A.M., and A.S. performed the experiments. A.M. and M.B. analyzed the data. A.T. developed the explanatory model. All authors interpreted results of experiments. A.M., M.B., and A.S. prepared figures. A.M. and M.B. drafted the manuscript. All authors edited, revised, and approved the final version of the manuscript.
Acknowledgments
This work was partially supported by the European Commission funded projects WEARHAP (project 601165), SOMA (project 645599), and SOFTPRO (project 688857) and by ERC Advanced Grants 291166 SoftHands and 247300 Patch. Preliminary results from experiment 1 (sample size = six participants) were presented at the Eurohaptics Conference 2014 (Paris) and were awarded the Best Paper Award (poster presentation). We thank Simone Fani, Omar Al Atassi, Simone Ciotti, and Mattia Poggiani for helping with the apparatus and Irene Senna, Cesare V. Parise, Jessica Hartcher-O’Brien, and Francesco Lacquaniti for useful comments and suggestions.
Published: April 7, 2016
Footnotes
Supplemental Information includes three figures and Supplemental Experimental Procedures and can be found with this article online at http://dx.doi.org/10.1016/j.cub.2016.02.052.
Contributor Information
Alessandro Moscatelli, Email: alessandro.moscatelli@uni-bielefeld.de.
Matteo Bianchi, Email: matteo.bianchi@centropiaggio.unipi.it.
Supplemental Information
References
- 1.Cartmill M. The volar skin of primates: its frictional characteristics and their functional significance. Am. J. Phys. Anthropol. 1979;50:497–509. doi: 10.1002/ajpa.1330500402. [DOI] [PubMed] [Google Scholar]
- 2.Arimoto S., Nguyen P.T.A., Han H.-Y., Doulgeri Z. Dynamics and control of a set of dual fingers with soft tips. Robotica. 2000;18:71–80. [Google Scholar]
- 3.Serina E.R., Mockensturm E., Mote C.D., Jr., Rempel D. A structural model of the forced compression of the fingertip pulp. J. Biomech. 1998;31:639–646. doi: 10.1016/s0021-9290(98)00067-0. [DOI] [PubMed] [Google Scholar]
- 4.Palmer S.E. MIT Press; 1999. Vision Science: Photons to Phenomenology. [Google Scholar]
- 5.Bicchi A., Scilingo E.P., De Rossi D. Haptic discrimination of softness in teleoperation: the role of the contact area spread rate. IEEE Trans. Robot. Autom. 2000;16:496–504. [Google Scholar]
- 6.Martinez-Conde S., Macknik S.L. The neuroscience of illusion. Sci. Am. Mind. 2013;22:6–9. [Google Scholar]
- 7.Regan D., Beverley K.I. Looming detectors in the human visual pathway. Vision Res. 1978;18:415–421. doi: 10.1016/0042-6989(78)90051-2. [DOI] [PubMed] [Google Scholar]
- 8.Regan D., Vincent A. Visual processing of looming and time to contact throughout the visual field. Vision Res. 1995;35:1845–1857. doi: 10.1016/0042-6989(94)00274-p. [DOI] [PubMed] [Google Scholar]
- 9.Brenner E., Van Den Berg A.V., Van Damme W.J. Perceived motion in depth. Vision Res. 1996;36:699–706. doi: 10.1016/0042-6989(95)00146-8. [DOI] [PubMed] [Google Scholar]
- 10.Johansson R.S., Birznieks I. First spikes in ensembles of human tactile afferents code complex spatial fingertip events. Nat. Neurosci. 2004;7:170–177. doi: 10.1038/nn1177. [DOI] [PubMed] [Google Scholar]
- 11.Adams M.J., Johnson S.A., Lefèvre P., Lévesque V., Hayward V., André T., Thonnard J.L. Finger pad friction and its role in grip and touch. J. R. Soc. Interface. 2012;10:20120467. doi: 10.1098/rsif.2012.0467. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Jörntell H., Bengtsson F., Geborek P., Spanne A., Terekhov A.V., Hayward V. Segregation of tactile input features in neurons of the cuneate nucleus. Neuron. 2014;83:1444–1452. doi: 10.1016/j.neuron.2014.07.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Di Luca M., editor. Multisensory Softness. Springer London; 2014. [Google Scholar]
- 14.Serio, A., Bianchi, M., and Bicchi, A. (2013). A device for mimicking the contact force/contact area relationship of different materials with applications to softness rendering. IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4484–4490.
- 15.Bianchi M., Serio A. Design and characterization of a fabric-based softness display. IEEE Trans. Haptics. 2015;8:152–163. doi: 10.1109/TOH.2015.2404353. [DOI] [PubMed] [Google Scholar]
- 16.Moscatelli A., Mezzetti M., Lacquaniti F. Modeling psychophysical data at the population-level: the generalized linear mixed model. J. Vis. 2012;12:1–17. doi: 10.1167/12.11.26. [DOI] [PubMed] [Google Scholar]
- 17.Fujita, K., and Ohmori, H. (2001). A new softness display interface by dynamic fingertip contact area control. Proceedings of the 5th World Multiconference on Systemics, Cybernetics and Informatics, pp. 78–82.
- 18.Yokota, H., Yamamoto, A., Yamamoto, H., and Higuchi, T. (2007). Producing softness sensation on an electrostatic texture display for rendering diverse tactile feelings. Proceedings of the 2nd Joint IEEE EuroHaptics Conference, 2007 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 584–585.
- 19.Kimura, F., Yamamoto, A., and Higuchi, T. (2009). Development of a contact width sensor for tactile tele-presentation of softness. Proceedings of the 18th IEEE International Symposium on Robot Human Interactive Communication, pp. 34–39.
- 20.Landy M.S., Maloney L.T., Johnston E.B., Young M. Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Res. 1995;35:389–412. doi: 10.1016/0042-6989(94)00176-m. [DOI] [PubMed] [Google Scholar]
- 21.Catalano M.G., Grioli G., Farnioli E., Serio A., Piazza C., Bicchi A. Adaptive synergies for the design and control of the Pisa/IIT SoftHand. Int. J. Robot. Res. 2014;33:768–782. [Google Scholar]
- 22.Bicchi A., Scilingo E.P., Ricciardi E., Pietrini P. Tactile flow explains haptic counterparts of common visual illusions. Brain Res. Bull. 2008;75:737–741. doi: 10.1016/j.brainresbull.2008.01.011. [DOI] [PubMed] [Google Scholar]
- 23.Moscatelli A., Naceri A., Ernst M.O. Path integration in tactile perception of shapes. Behav. Brain Res. 2014;274:355–364. doi: 10.1016/j.bbr.2014.08.025. [DOI] [PubMed] [Google Scholar]
- 24.Dostmohamed H., Hayward V. Trajectory of contact region on the fingerpad gives the illusion of haptic shape. Exp. Brain Res. 2005;164:387–394. doi: 10.1007/s00221-005-2262-5. [DOI] [PubMed] [Google Scholar]
- 25.Proske U., Gandevia S.C. The proprioceptive senses: their roles in signaling body shape, body position and movement, and muscle force. Physiol. Rev. 2012;92:1651–1697. doi: 10.1152/physrev.00048.2011. [DOI] [PubMed] [Google Scholar]
- 26.Edin B.B., Abbs J.H. Finger movement responses of cutaneous mechanoreceptors in the dorsal skin of the human hand. J. Neurophysiol. 1991;65:657–670. doi: 10.1152/jn.1991.65.3.657. [DOI] [PubMed] [Google Scholar]
- 27.Terekhov A.V., Hayward V. The brain uses extrasomatic information to estimate limb displacement. Proc. R. Soc. B Biol. Sci. 2015;282:20151661. doi: 10.1098/rspb.2015.1661. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Collins D.F., Refshauge K.M., Gandevia S.C. Sensory integration in the perception of movements at the human metacarpophalangeal joint. J. Physiol. 2000;529:505–515. doi: 10.1111/j.1469-7793.2000.00505.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Edin B.B., Johansson N. Skin strain patterns provide kinaesthetic information to the human central nervous system. J. Physiol. 1995;487:243–251. doi: 10.1113/jphysiol.1995.sp020875. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Collins D.F., Refshauge K.M., Todd G., Gandevia S.C. Cutaneous receptors contribute to kinesthesia at the index finger, elbow, and knee. J. Neurophysiol. 2005;94:1699–1706. doi: 10.1152/jn.00191.2005. [DOI] [PubMed] [Google Scholar]
- 31.Pack C.C., Bensmaia S.J. Seeing and feeling motion: canonical computations in vision and touch. PLoS Biol. 2015;13:e1002271. doi: 10.1371/journal.pbio.1002271. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Ghazanfar A.A., Neuhoff J.G., Logothetis N.K. Auditory looming perception in rhesus monkeys. Proc. Natl. Acad. Sci. USA. 2002;99:15755–15757. doi: 10.1073/pnas.242469699. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Maier J.X., Ghazanfar A.A. Looming biases in monkey auditory cortex. J. Neurosci. 2007;27:4093–4100. doi: 10.1523/JNEUROSCI.0330-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.