Sensory information from a slipping object elicits a rapid and automatic shoulder response

Abstract

Humans have the remarkable ability to hold, grasp, and manipulate objects. Previous work has reported rapid and coordinated reactions in hand and shoulder muscles in response to external perturbations to the arm during object manipulation; however, little is known about how somatosensory feedback of an object slipping in the hand influences responses of the arm. We built a handheld device to stimulate the sensation of slipping at all five fingertips. The device was integrated into an exoskeleton robot that supported it against gravity. The setup allowed us to decouple somatosensory stimulation in the fingers from forces applied to the arm, two variables that are highly interdependent in real-world scenarios. Fourteen participants performed three experiments in which we measured their arm feedback responses during slip stimulation. Slip stimulations were applied horizontally in one of two directions, and participants were instructed to either follow the slip direction or move the arm in the opposite direction. Participants showed shoulder muscle responses within ∼67 ms of slip onset when following the direction of slip but significantly slower responses when instructed to move in the opposite direction. Shoulder responses were modulated by the speed but not the distance of the slip. Finally, when slip stimulation was combined with mechanical perturbations to the arm, we found that sensory information from the fingertips significantly modulated the shoulder feedback responses. Overall, the results demonstrate the existence of a rapid feedback system that stabilizes handheld objects.

NEW & NOTEWORTHY We tested whether the sensation of an object slipping from the fingers modulates shoulder feedback responses. We found rapid shoulder feedback responses when participants were instructed to follow the slip direction with the arm. Shoulder responses following mechanical joint perturbations were also potentiated when combined with slipping. These results demonstrate the existence of fast and automatic feedback responses in the arm in reaction to sensory input to the fingertips that maintain grip on handheld objects.

INTRODUCTION

Imagine that you are looking at your smartphone while your partner is asking you a question. After you fail to respond to the question, your partner decides to get your attention by pulling your phone out of your hand. In this situation, your partner’s action would initiate a combined response of your upper limb and hand to stabilize your grasp and secure the device. How the nervous system rapidly uses haptic and proprioceptive feedback to appropriately respond in such complex real-world scenarios is an important question in sensorimotor neuroscience (Mathis et al. 2019; Mazurek et al. 2018).

Previous reports have shown evidence that the nervous system automatically increases grip force to prevent an object from falling when slip is detected (Cole and Abbs 1988; Jones and Hunter 1992; Johansson 1996; Johansson and Westling 1984). In the case of self-initiated movements, these grip-force modulations are highly predictive (Danion and Sarlegna 2007; Diamond et al. 2015; Flanagan and Wing 1997; Hadjiosif and Smith 2015; Wolpert and Flanagan 2001). Within the arm, humans generate rapid and flexible motor responses in response to mechanical perturbations that compensate for the coupling between joints (for review, see Pruszynski and Scott 2012) and are modulated by task goals (Pruszynski et al. 2008; Pruszynski et al. 2016; Weiler et al. 2019).

Previous work has mainly characterized grip and upper limb responses independently; it is clear, however, that hand and arm responses need to be tightly coordinated for successful object manipulation (Smeets et al. 2019). To explore this coordination, Crevecoeur et al. (2016) applied loads to the arm joint while participants held an object in precision grip. Their results showed that hand muscles rapidly accounted for the perturbation direction in a goal-dependent manner. Thus, perturbation in the upper limb modulates grip force. Here, we ask the complementary (yet independent) question of whether a distal perturbation to the fingers (slip) would result in a change in the rapid proximal response in the upper arm. In other words, it is unknown whether there is a fast an automatic coupling between sensory information from the fingers (e.g., slipping object) and arm feedback responses.

To study how somatosensory information at the finger tips modulates arm responses, we designed a new device to emulate the sensation of an object slipping during grasping. Importantly, the object slip could be manipulated independently from any loads applied to the shoulder or elbow joints. In real life, when somebody pulls an object you are holding, part of the force will be transmitted to your arm and sensed via the muscle spindles, resulting in a direct compensatory response of the arm muscles (Dimitriou 2014). Hence, any arm response in this scenario could be the result of proprioceptive information from the arm rather than from somatosensory information from the finger tips. To be able to disentangle these two sources of information, we mounted the device on a robotic exoskeleton, such that the forces inducing the slip sensation at the fingertips could be uncoupled from the forces applied to the arm. This allowed us to investigate the effect of the somatosensory information from the fingers without the confounding influence of proprioceptive information at the arm.

We hypothesized that the sensation of an object slipping may trigger a rapid shoulder muscle response to compensate for the slipping direction. A priori, it was not clear whether such an automatic response would involve the arm following the direction of slip or opposing the direction of slip. Therefore, in experiment 1, we compared responses under a “follow” or “against” instruction and found a much more rapid response when participants followed the direction of slip. In experiment 2, we tested how the speed and distance of the slip would influence the rapid shoulder muscle response. Finally, experiment 3 investigates how this mechanism interacts with mechanical perturbations applied to the shoulder joint, as occurs in real-world scenarios. This design allowed us to study somatosensory and proprioceptive perturbations in the hand and shoulder independently as well as the interaction between them when these perturbations are combined.

MATERIALS AND METHODS

Participants

Fourteen human participants (aged 22.7 ± 3.7; 6 males, 8 females) with no known musculoskeletal or neurological diseases were invited to perform three experiments described below. Participants reported to be right-hand dominant and had normal or corrected-to-normal vision. The Office of Research Ethics at Western University approved all experimental procedures according to the Declaration of Helsinki, and all participants signed a consent form before participating in an experiment.

Apparatus

Participants performed the experiments using a robotic exoskeleton (KINARM, Kingston, ON, Canada) that permits flexion and extension movements of the shoulder and elbow joints in the horizontal plane intersecting the shoulder joint (Scott 1999). The KINARM robot can independently apply mechanical loads to the shoulder and/or elbow and record kinematic variables of these joints. Mechanical stimuli were delivered to the fingertips using a custom-built, computer-controlled stimulator box designed to produce a slipping sensation at each of the five fingers (Fig. 1A). The stimulator box was mounted to the KINARM (fixed in the hand plate), and participants grasped it during the task. The stimulator allowed position and speed control of the contact surfaces in one dimension for all fingers. The surface that contacted the fingertip was flat and had fine sandpaper (grit 800) as a surface finish. This stimulus surface was chosen to obtain sufficiently high friction between the contact surface and the skin without restraining the slider movement. The contact surface for each finger was 18 mm in the vertical plane and 40 mm in the horizontal plane. During the tactile stimulation (slip), the fingers usually had some contact with surrounding housing. Depending on the finger size, there is a range of ∼1 mm of finger movements before they were stopped by the housing. Given that in all of our experiments the sliders move a minimum of 8 mm on each trial, we know that a slipping does occur regardless of whether there might or might not be concomitantly finger movements. The full range of movement of the sliders was 18 mm driven by high-speed digital servos (Power HD 3688HB; operation speed 0.07 s/60°; stall torque 2.8 kg·cm). To measure the grip force of each individual finger, two load sensors (Honeywell FSG020WNPB) per finger were placed behind the sliders. Because the hand, the arm, and the case of the finger-stimulation box were all fixed to the KINARM exoskeleton, slip stimuli delivered to the fingers did not induce any torque in the elbow or shoulder joints. The setup included an overhead screen and semitransparent mirror to show visual information. Each segment length of the robot was adjusted to fit the participant’s arm. Arm supports were selected according to the arm size, and foam padding was used to reduce any undesirable arm movement. Throughout the experiment, direct vision of the entire arm and hand was occluded so that responses were guided only by somatosensory information.

Fig. 1.

Stimulator device and experimental setup. A: left, right, and holding view of the stimulator box. Blue arrows indicate the movement of the sliders. B: in all experiments, participants held the stimulator box that could trigger a slipping sensation at the fingertips. Visual feedback of the device position (white circle) and the target position (red circle) were displayed in the same plane of motion. C: before each trial, participants were instructed to align the device visual feedback with the target feedback while accomplishing the baseline state conditions of position, grip force, and muscle pre-activation (see Experimental Paradigm). All visual feedback was then removed for the start of a trial (i.e., before the delivery of a mechanical slip, mechanical joint perturbation, or both).

Experimental Paradigm

Experiment 1: rapid feedback responses.

We hypothesized that the sensation of the object slipping in the finger tips would cause a rapid response in the arm. A priori we did not know whether this response would cause the arm to follow the object slip (to stabilize the object) or whether it would move the arm in the opposite direction (to resist the perturbation). Therefore, we designed a postural task in which the participants held the stimulator box while they felt the slip in one of two directions either inward or outward with respect to the hand. In separate blocks, participants were instructed to either “follow the slip” or “move against the slip.” If there exists a rapid and automatic coupling between slip sensation and arm response, the reaction in the “natural” direction should be substantially faster. The procedure began with the participant grasping the stimulator while seated in the exoskeleton. During all trials, the direct visual feedback of the hand and arm was occluded; however, during the initial part of the experiment, a visual cursor (white circle: 1 cm in diameter) indicating the position of hand was projected onto the mirror (Fig. 1B). To start a trial, the participant had to fulfill three conditions: 1) Using visual feedback, participants had to align their hand (white cue) with the home target (red circle: 2 cm in diameter), whose position corresponded to a shoulder angle of 45° and an elbow angle of 90° (Fig. 1C); 2) after entering the home target, the exoskeleton was gradually applied a background torque of 2 Nm to either the flexor or extensor muscles of the shoulder (arm pre-activation), and participants were instructed to keep their hand at the home target while grasping the stimulator; and 3) participants had to apply a grip force of 0.5 N ± 0.1 N between the thumb and the rest of the fingers. Once participants achieved these three conditions, all visual feedback was removed. Then, if participants maintained this baseline state for a random period between 250 and 500 ms (uniform distribution), the trial started. If participants failed to achieve/maintain this baseline state for 1 s, the trial restarted from the beginning. For experiment 1, participants were instructed to move their arm as fast as they could in either the same (to follow) or the opposite (go against) direction of the slip. To avoid any constraints on the movement, participants did not receive any instructions pertaining to the distance they should move. The slider displacement was 16 mm with a speed of 20 mm/s in either the inward or outward directions. Participants completed 240 trials in two blocks. Half of the participants received the instruction of “follow the slip” first, and the other half received the instruction of “move against the slip” first. The order of background load direction was balanced and randomized across trials. This ensured that slipping direction was not predicted from the background load direction. The order of slipping direction was randomized, and participants completed 120 trials in each block of a total of 30 trials per condition (2 slip directions and 2 background force directions). About 20 min was required to complete experiment 1.

Experiment 2: speed and distance of the slip.

To test whether speed and distance of the slip could modulate the arm response, participants performed an accuracy task. We asked participants to precisely compensate for the slip of the sliders with an arm movement. Thus, if the participant felt that the sliders moved 1 cm in the forward direction within the device, the hand was required to also move 1 cm in the forward direction. We asked participants to move without delay from the slip onset. As in experiment 1, a trial in experiment 2 started when participants accomplished and maintained the baseline state. Mechanical slip occurred at one of two different distances and two speeds. Participants completed a total of 96 trials in this experiment. The instruction was to follow the direction of the slip as accurately as possible. The order of slipping distance (8/16 mm), velocity (10/20 mm/s), was randomized (24 trials per condition). About 20 min was required to complete experiment 2.

Experiment 3: combined slip and arm perturbations.

In experiment 3, we studied the interaction between simultaneous perturbations to the arm and slip stimulation at the fingertips. In this experiment, participants performed a postural task that required holding and keeping the stimulator box centered at a target. A mechanical load was applied at the shoulder joint (extension) either alone or in combination with a slip stimulation to the fingers. Based on previous results, we used 16 mm of slipping distance and a velocity of 20 mm/s in the “out” direction. The instructions to accomplish the baseline state were the same as in experiments 1 and 2. At the moment of perturbation, the stimulator moved the sliders and/or the KINARM robot applied a mechanical load at two different strengths (1 Nm or 2 Nm) at the shoulder joint. Participants were asked to move the hand back to the original position (without visual feedback) as quickly as possible after perturbation onset. Participants completed a total of 96 trials in this experiment. The order of slip stimulation (present/absent) and strength of joint perturbation (1 Nm/2 Nm) was randomized (24 trials/condition). About 20 min were required to complete experiment 3.

All participants underwent all three experiments. They performed these experiments in order (experiments 1, 2 and 3 of the paper) and in one testing session. The conditions within each experiment were counterbalanced. Rest breaks were given throughout when requested.

Muscle Activity

Electromyographic signals (EMG) were obtained from four upper-limb muscles involved in flexion or extension movements at the elbow and/or shoulder joints [pectoralis major clavicular head (PEC), shoulder flexor; posterior deltoid (PD), shoulder extensor; biceps brachii long head (BI), shoulder and elbow flexor and wrist supinator; triceps brachii lateral head (TRI), elbow extensor]. Prior to electrode placement, the skin was cleaned and abraded with rubbing alcohol, and the electrode contacts were covered with conductive gel. Electrodes (DE-2.1; Delsys, Boston, MA) were placed on the belly of the muscle and oriented along the muscle fiber, and the reference electrode (Dermatrode; American Imex, Irvine, CA) was attached to the clavicle. To assess the quality of each EMG signal, we performed a set of maneuvers known to elicit high levels of activation for each muscle in the horizontal plane. EMG signals were amplified (gain = 103) and band-pass filtered (20–450 Hz) by a commercially available system (Bagnoli; Delsys) and then digitally sampled at 1,000 Hz.

Data Analysis

Data processing and statistical analyses were performed using Matlab (The Mathworks, Natick, MA). All joint kinematics (i.e., hand position and joint angles) were sampled at 1,000 Hz and then low-pass filtered (12 Hz, 2-pass, 4th-order Butterworth). EMG data were band-pass filtered (20–500 Hz, 2-pass, 2nd-order Butterworth) and full-wave rectified. EMG data were normalized to their own mean activity over the 200-ms period before slip perturbation onset when either shoulder flexor or extensor muscles were loaded by the exoskeleton (i.e., shoulder flexion or extension torque preload, 2 Nm). All data were aligned on perturbation onset that could be either a mechanical slipping, a mechanical joint perturbation, or both at the same time.

To estimate the temporal onset of task-related EMG activity for each participant, we used each participant’s EMG activity from two conditions to generate a time series receiver operator characteristic (ROC) from 0 ms to 200 ms relative to perturbation onset. Briefly, ROC curves quantify the probability that an ideal observer could discriminate between two stimuli conditions; a value of 0.5 represents chance-level discrimination, whereas a value of 0 or 1 represents perfect discrimination (Green and Swets 1966). ROC curves were generated from the pectoral or deltoid muscle EMG activity, depending on the condition. We then fit the time-series ROC curves with a linear regression technique, which estimates the temporal onset of task-related EMG activity by determining when the time series ROC curve diverges from chance-level discrimination (i.e., ∼0.5; see Weiler et al. 2015). We will refer to this time point as the divergence onset time. We used the EMG activity to perform additional statistical test sampling the EMG time series. Statistical tests were performed using the average of 10 time points centered in the divergence time calculated by the ROC.

In-plane hand position and tangential velocity were used to determine the end of the hand trajectories. The arm responses for the participants were not goal-directed reaching movements. Thus, over many trials, participants did not completely stop the arm from moving. Rather, they smoothly returned to the home target. Thus, we needed a robust criterion that would allow us to determine the length of the movement. Empirically, 30% of the maximum velocity was a good approximation that could be applied systematically to all the subjects in this task. We performed different statistical tests such as paired t-test and ANOVA when appropriate for each of the three experiments. Details of these procedures are provided below in results. Experimental results were considered statistically significant if the P value was <0.05.

RESULTS

Experiment 1: Automatic Arm Response in the Direction of Slip

In experiment 1, participants were instructed to move the hand position via the shoulder joint as fast as possible in either the same (to follow) or the opposite (go against) direction of the slip. If there exists a rapid and automatic coupling between slip sensation and arm response, the reaction in the “natural” direction should be substantially faster.

Figure 2A shows the task design, in which participants performed backward or forward movements for the two slip directions (2 × 2 design; Fig. 2E). The mean kinematics of the shoulder joint are shown in Fig. 2, B and F, for forward and backward arm movement, respectively. For both arm movements, we found that following the slip (red traces) resulted in faster responses compared with moving against the slip (blue traces). The EMG data also revealed a faster ramping activity of agonist muscle when the participants followed the slip (Fig. 2, C and G). To quantify the difference in timing, first we estimated the onset of divergence from baseline activity for the two conditions (follow and against) in each participant. Indeed, for the forward arm movement (Fig. 2C), participants performed faster responses when they moved in the same direction of the slip (mean onset time = 60.0 ms; SE = 0.2) compared with when they moved in the opposite direction (mean onset time = 148.1 ms; SE = 0.5). Then, we calculated the divergence time between the two conditions for each arm movement. In both cases, the divergence between in and out conditions was close to 67 ms (forward 67.1 ms SE 0.1 and backward 67.1 ms SE 0.2). This behavior was similar for the backward arm movement (Fig. 2G), showing a faster arm response when participants moved in the same direction of the slip (mean onset 78 ms) compared with when they moved to the opposite direction (mean onset time = 153 ms). A 2 × 2 ANOVA showed a significant main effect of the direction of the slip [F_{(1, 13)} = 6.45, P = 0.02]. No significant effect of the arm direction [F_{(1, 13)} = 2.08, P = 0.17] or interaction between the arm direction and the slip direction [F_{(1, 13)} = 2.52, P = 0.13] was found. Figure 2, D and H, shows time series ROC curves from an exemplar participant fit with the linear regression technique that indicates the divergence onset time (green line) between follow and oppose movements in panels Fig. 2, C and D, respectively.

Fig. 2.

Shoulder responses related to slipping direction. A and E: during experiment 1, participants received slip stimulation in 2 directions (A), and they were instructed to move the arm either in the same (follow) or the opposite (against) direction of the slip (E). B and F: average kinematics of the shoulder joint. C and G: normalized muscle activity. D and H: receiver operator characteristic (ROC) curve of the divergence between follow and against conditions for representative subjects. B–D: results for a forward arm movement. F–H: results of backward arm movement. Shaded areas represent the standard error of the mean. ROC panels indicate in gray the ROC curve and in black the best-fitted line. Green line indicates the timing of a significant difference of the muscle response for both conditions (red and blue). All muscle activity traces correspond to the agonist shoulder muscle for each arm movement. AU, arbitrary units; deg, degrees; EMG, electromyographic signals; PD, posterior deltoid; PEC, pectoralis major clavicular head. All data are aligned on slipping onset; n = 14.

These results show that the arm feedback response is faster when the arm movement is in the same direction of the slip as compared with when the participant moves in the opposite direction. If there is an automatic response to follow the direction of a perceived slip, we would expect that some of the feedback responses under the “move against” instruction are produced in the wrong direction (i.e., in the direction of the slip). To test for this possibility, we carefully analyzed the paths of the hand during the trials. Figure 3A shows the average displacement trace of the hand position for each participant, showing that participants generally followed the instruction. However, on individual trials, participants made a number of errors. We defined an error as individual trials when the participant moved more than 1 mm away from the home position (either in the x- or y-axis) in a direction different from the correct quadrant (i.e., 2nd quadrant for the forward movement, 4th quadrant for the backward movement; Fig. 3B). Participants showed only a small number of errors when the arm movement followed the slip (3.1% of total trials; mean = 0.96, SD = 1.43) compared when the slip was opposite to the arm movement (26.9% of total trials; mean = 8.07, SD = 7.22). This difference was significant for both forward [t(13) = 3.59, P = 0.001] and backward movements [t(13) = 3.21, P = 0.002] during the first block in which there was no adaptation to the instruction (follow or opposite). These results suggest that the response to follow a slipping object with the arm is not only fast but also automatic; that is, it can intrude on a voluntary response and induce errors (Haith and Krakauer 2018).

Fig. 3.

Hand paths for experiment 1. A: each trace indicates the average path of each participant (n = 14) for both instructions, follow (red) and against (blue), and both directions of arm movement (forward and backward). Paths start on the trial onset (at home position 0,0) and finish after 600 ms. B: zoomed view of the home position in the forward movement for a representative subject. Thicker lines indicate the average trace of all the individual trials (thin lines) for each instruction.

Experiment 2: Fast Feedback Responses Vary With Speed But Not With the Distance of Slip

In experiment 1, we showed an automatic response of the arm that follows the slip sensation on the fingers. It has been shown that rapid responses can be modulated in a task-dependent manner to maintain limb stability (Shemmell et al. 2010). Therefore, we tested whether the characteristics of the slipping stimulus modulate the arm response or whether the arm responds equally to any slip sensation. We used two speeds and two distances for the slip stimuli (Fig. 4A). To limit the overall number of conditions, we chose to study only forward arm movement with slipping in the direction out of the hand. Overall, we found that the ROC divergence time for faster slips (orange color in Fig. 4C) elicited earlier (mean onset time = 67.0 ms, SE 0.6) muscle activity compared with slower slips [green color in Fig. 4C; mean onset time = 114.3 ms, SE 0.7; t(13) = 3.99, P = 7.6e-4]. However, the muscle activities resulting from the two slip distances using the same slip speed (solid vs. dashed lines of the same tone) were not significantly different for either slow slip [t(13) = 0.89, P = 0.194] or fast slip [t(13) = 1.36, P = 0.097]. These results suggest that the speed of the slipping has a stronger effect on the early arm response as compared with slip distance (Fig. 4D).

Fig. 4.

Shoulder responses according to different slip characteristics. A: during experiment 2, participants received slip stimulation in the “out” direction using 2 speeds and 2 distances, and they were instructed to move the arm following the slip. B: average kinematics of the shoulder joint. C: normalized muscle activity. D: gray line shows the difference between fast long and fast short (distance), whereas black line shows the difference between fast long and slow long (speed). All muscle activity traces correspond to the agonist shoulder muscle for each arm movement (n = 14). AU, arbitrary units; Deg, degrees; EMG, electromyographic signals; PEC, pectoralis major clavicular head.

The explicit task goal in experiment 2 was to move the hand the same distance as the sensed slip (i.e., the displacement of the device sliders). Although participants’ movements did not exactly match the distance (8 or 16 mm), the average displacement showed a clear effect of the slip characteristics on the final position of the participant’s hand (Fig. 5A). A 2 × 2 ANOVA showed a significant main effect of the distance [F_{(1, 13)} = 24.4, P = 0.3e-3] and speed [F_{(1, 13)} = 6.092, P = 0.028] of the slip. However, the interaction between slip speed and distance was not significant [F_{(1, 13)} = 1.8, P = 0.2]. Although the instructions emphasized an accurate compensation for the slip distance, the speed of slip also had a significant influence on hand displacement for both the short [t(13) = 1.83, P = 0.044] and long slips [t(13) = 2.19, P = 0.023].

Fig. 5.

Hand paths for experiment 2. A: each color line indicates the average path of each participant (n = 14) for each condition for the forward arm movement. Gray line indicates the mean path of the group. Paths start on the trial onset (at home position 0,0) and finish when the participant stops movement (tangential velocity <30% of the maximum velocity of each trial). B: average hand displacement from the home target to the end of the movement for each condition. *Statistical signifiance.

Overall, these results show that the initial arm response is mostly dictated by the speed of the slip. In contrast, the overall response of the arm took into account the displacement of the slip to achieve the behavioral goal but still was slightly biased by the initial speed.

Experiment 3: Slip Modulates Response to Arm Perturbation

In real-world scenarios, our nervous system needs to integrate information from the finger tips with information from the arm to optimally resist perturbations delivered to a handheld object. Our setup uncoupled these sources of information between the hand and the arm, allowing us to observe the effect of slip stimulation in isolation. But how do feedback from the hand and arm interact when perturbations occur simultaneously with slip stimulation? It is possible that the local arm feedback loop completely overwrites any modulation from the sensation from the fingertips. Alternatively, the two sources of information may be combined in the final response. In experiment 3, we investigated whether the slip sensation at the fingers modulates the arm’s response to a slipping object during an external arm perturbation (either 1 Nm or 2 Nm). We asked participants to bring the object back to the home position as fast as they could after the perturbation. Figure 6A shows the task setup and Fig. 6B the response of the arm to an external mechanical shoulder extension perturbation alone (dashed lines) and to an external perturbation plus slipping in the opposite direction (i.e., out of the hand; solid lines).

Fig. 6.

Arm responses related to combined torque and slip. During experiment 3, participants received either an extension torque (1 Nm or 2 Nm) or an extension torque plus slip stimulation (out direction) A and D: participants were instructed to move the stimulator cursor back to the original position (without visual feedback). B: average kinematics of the shoulder joint. E and F: normalized muscle activity (n = 14). C: receiver operator characteristic (ROC) curve of the 2 conditions (torque and torque plus slip) using 2 Nm of a representative subject. ROC panel indicates in gray the ROC curve and in black the best-fitted line of a representative subject. Green line indicates the timing of a significant difference of the muscle response for both conditions. All muscle activity traces correspond to the agonist shoulder muscle for each arm movement.

As expected, the 2-Nm torque produced larger arm displacements than the 1-Nm perturbation (Fig. 6B). For both perturbation levels, however, the position of the arm moved back to the original position faster when the slip was included in the perturbation as compared with when it was absent (torque alone). Although the initial ramping of the EMG activity did not change significantly, the EMG signal showed a significantly higher activity when the slipping stimulation was present (Fig. 6, E and F). To determine the onset of this modulation, we computed the area under the ROC curve for each time point and determined the divergence between trials with and without slip present using linear regression (see materials and methods). The mean onset time for 1 Nm was 98.2 ms, SE 0.9, whereas for 2 Nm we found a mean onset time of 71.3 ms, SE 0.9 (Fig. 6C). A 2 × 2 ANOVA using the EMG signal showed a significant main effect for the presence of the slip [F_{(1, 13)} = 16.15, P = 1.5e-3] and for the torque strength [F_{(1, 13)} = 16.7, P = 1.3e-3]. However, the interaction between slip and torque was not significant [F_{(1, 13)} = 0.74, P = 0.4]. This result suggests that the direct perturbation in the arm does not override the slip sensation from the fingertips but that both are integrated to produce a combined feedback response.

Participants were relatively accurate in returning to the home target when they received a mechanical torque in the arm. Figure 7A shows the average hand path of each participant for each condition. As expected, the stronger perturbation (2 Nm) resulted in higher variability in the end position of the hand, but overall, participants stopped close to the home position. When the slip was present, however, participants tended to overshoot, ending the movement farther away from the home position compared with the respective control (torque alone). A 2 × 2 ANOVA comparing the individual end positions showed a significant main effect of the slip [F_{(1, 13)} = 24.62, P = 0.3e-3]. We also found a significant interaction between torque and slip [F_{(1, 13)} = 16.92, P = 1.2e-3]. The difference between the control and combined condition (torque plus slip) was higher for the 1-Nm perturbation [t(13) = 5.38, P = 1.2e-4] than for the 2-Nm perturbation [t(13) = 2.73, P = 0.017] (Fig. 7B). Overall, slip information biased participants to respond more strongly to the perturbation, ultimately leading to a less accurate performance.

Fig. 7.

Hand paths for experiment 3. A: each color line indicates the average path of each participant (n = 14) for each condition. Gray line indicates the mean path of the group. Paths start on the trial onset (at home position 0,0) and finish when the participant stops the movement (tangential velocity <30% of the maximum velocity of each trial). B: average hand displacement from the home target to the end of the movement for each condition. *Statistical significance.

DISCUSSION

Taken together, our results establish the existence of a fast and automatic arm response that follows the direction of an object slipping from the hand. We were able to reveal this response by artificially uncoupling the slip sensation on the fingertips from the forces acting on the shoulder joint, two variables that are often coupled in real-world situations. In our experiment, the stimulator device was fixed to the robot structure, and the hand and arm of the participant were secured with foam padding to prevent any undesired movement within the device. Thus, the slip stimulation did not produce a torque to the arm, and the torque applied to the arm did not cause slip of the device, allowing us to assess the arm responses associated with the slipping sensation alone. We report three principal findings. First, we found a fast and automatic feedback response in shoulder muscles when following the direction of a slip stimulus at the fingertip with an onset latency of ∼67 ms. Second, this rapid feedback response of the shoulder muscles was modulated by the speed but not by the distance of the slip. Third, responses to mechanical perturbations applied to the upper limb were potentiated when combined with object slip in the direction opposite to the perturbation.

Automatic Response Following a Slipping Object

Previous work has long demonstrated that the sensation of slip at the fingertips can trigger very rapid increases in grip force (Cole and Abbs 1988; Crevecoeur et al. 2017; Delhaye et al. 2014; Häger-Ross et al. 1996; Häger-Ross and Johansson 1996; Johansson and Westling 1984; Jones and Hunter 1992). Here, we found that slip at the fingers also induces a rapid and automatic shoulder muscle response that moves the arm in the direction of the slip. This automatic response was revealed by instructing participants to either follow the slipping direction or move against it, a paradigm similar to anti-saccade or anti-reach approach (Day and Lyon 2000; Gail and Andersen 2006; Munoz and Everling 2004). Specifically, we found substantially faster responses when the participants were instructed to move their arms in the same direction of the slip as compared with when instructed to move in the opposite direction. If the responses had been arbitrary and fully deliberate, both instructions should have led to the same latency.

A related observation comes from a bimanual haptic tracking task (Rosenbaum et al. 2006). In this study, participants were instructed to follow a moving object using the tactile information from the fingertip that made contact with the object. The results show that participants could follow two independent spatial trajectories with their two hands without interference, something that is very hard to achieve during voluntary movements (Kennerley et al. 2002). The lack of interference clearly argues for the existence of an automatic response that guides the arm in the direction of a perceived slip.

What is the functional relevance of this automatic response? It is most likely that it serves to facilitate stability of a handheld object. When an object slips from our grasp, it is essential to follow the movement of the object with the arm to prevent the object from completely slipping from our grasp. Even smaller movements of the object within the grasp should be prevented, as the finger grasp positions are chosen to balance the object in the hand to avoid object rotation (MacKenzie and Iberall 1994).

Consistent with a functional role in object stabilization, we showed in experiment 2 that the arm responses scale with the initial speed of the slip. We believe that such difference in the responses is related to the intensity of the stimuli (e.g., frequency of the mechanoreceptor stimulation). For grip force increases, such modulation has been well demonstrated (Cole and Abbs 1988; Crevecoeur et al. 2017; Häger-Ross and Johansson 1996). In contrast, we found no modulation in the initial shoulder muscle responses when the grasped object slipped at two distinct distances. This was expected, as at the onset of slipping in either condition (short or long distance), the same somatosensory information was transmitted to the nervous system. Therefore, the differences between the two distances would become available only when the short distance perturbation was completed. Indeed, the later responses and hand distance traces were clearly influenced by the length of the slip. These results provide evidence that the automatic response takes into account afferent feedback from the digits in an adaptive, time-sensitive, and appropriate manner, but the contribution of tactile and/or muscle afferent feedback remains to be elucidated.

The muscle activity latency of the following response of the arm (∼67 ms) indicates that the response can be produced faster than normal voluntary responses, which usually have a time scale of 100–150 ms. Similar latencies have been reported in previous work for other automatic responses, including the increase in grip force following a load perturbation in the fingertip (Cole and Abbs 1988; Crevecoeur et al. 2017) or a perturbation to the upper limb (Crevecoeur et al. 2016; Pruszynski et al. 2008). The short latency indicates that these responses are not generated by the normal polysynaptic cortical circuits that underlie voluntary and potentially arbitrary responses. The ∼67-ms response also suggests that these automatic responses are not generated exclusively at the level of the spinal cord, as known spinal reflexes (i.e., to muscle stretch) occur within ∼20–50 ms (Pierrot-Deseilligny and Burke 2012; Weiler et al. 2019). Feedback responses following mechanical perturbations that arise >50 ms can potentially engage spinal, subcortical, and cortical areas (Cheney and Fetz 1984; Evarts and Tanji 1976; Herter et al. 2009; Omrani et al. 2016; Pruszynski et al. 2011; Pruszynski et al. 2014; for review see, Scott 2016). Although the neuroanatomical substrate that underlies these automatic responses remains to be determined, our study predicts that, somewhere in the nervous system, neurons that project to shoulder muscles must receive relatively direct sensory input from tactile sensors in the hand. The response we describe here is similar to the nociceptive withdrawal reflex, where cutaneous inputs drive muscle responses to move the body away from a potentially dangerous stimulus (Sherrington 1910). Indeed, careful mapping of the withdrawal reflex has revealed an intricate relationship between the location of the nociceptive stimulus and which muscles are recruited to best move the limb away from the stimulus (Levinsson et al. 1999; Schouenborg and Kalliomäki 1990). A similar mapping and neural substrate could potentially underlie the responses observed here. It should be noted, however, that the direction of function of the following response is substantially different from the withdrawal reflex and thus may require different descending modulation and/or directly engage brainstem and cortical circuits also known to receive rapid somatosensory inputs (Scott 2016).

Combination of Slip Information with Local Muscle Stretch

In our experimental setup, we artificially dissociated the slip information and the torques acting on the arm. In real-world scenarios, however, a perturbation to a handheld object will induce both slip of the object in the hand and a torque at the shoulder joint. In other possible scenario, the salience of the torque in the shoulder joint (proximal proprioceptive) will be higher in comparison with the stimulation on the fingertips (distal somatosensory), resulting in a preponderant response to the local perturbation in the joint. If the automatic response revealed in the first two experiments indeed functions to stabilize the handheld object, it must also be functional in combination with stretch to the shoulder joint itself. The results from experiment 3 clearly show that the automatic response to a slip is not overridden by the presence of a perturbation to the shoulder but rather combines with this locally generated response.

The experimental situation corresponds to the natural scenario in which a perturbation to the arm causes a sudden acceleration of the limb. The inertia of the object then induces a slip of the object in the opposite direction. If such slip is detected, the resistive reaction of the arm is amplified, stabilizing the grasp on the object. Although not reported here, pilot experiments also indicated that this amplification was not observed when the object slip was in the same direction of the arm perturbation. This arises from forces that are applied directly to the object, in which case the arm should be more compliant to maintain a stable object grasp.

Processing of sensory information from the hand and the upper limb has been largely studied in isolation (Delhaye et al. 2018; Scott 2016); however, the integration of these two sources of information for limb control suggest a confluence of these sensory sources on motor structures. For example, spinal, subcortical (i.e., thalamus), and cortical (i.e., somatosensory cortex) structures are known to receive information from both tactile sensors and muscle spindles (Delhaye et al. 2018; Kim et al. 2015; Picard and Smith 1992; Scott 2016). Despite the fact that our experiment did not provide data to test a specific way of integration, one possibility is that the observed combination might take place in regions that receive both types of information. Alternatively, it remains possible that the signals are processed separately, and the combination arises during convergence onto spinal motor neurons.

One limitation of our experiments is that we could only study a limited set of slip directions in the horizontal plane. However, if the function of this automatic response is to stabilize handheld objects, the arm’s response to slip should adapt flexibly to the configuration of the arm in space and to the configuration of the object in the hand. This would imply that slip at the fingertips can also modulate automatic responses around the elbow joint. Such flexibility remains to be experimentally shown. Another limitation of our setup is that regardless of us trying our best to constrain the arm and hand movement in the exoskeleton, it is impossible to completely suppress any small change in finger configuration, and as a consequence, afferent feedback from the finger muscles was also likely contributing to some extent.

In summary, our paper demonstrates that somatosensory information at the hand elicits rapid motor corrections in the shoulder that are suitable to stabilize handheld objects, are sensitive to the slipping direction and speed, and are integrated with local reflex responses at the shoulder.

GRANTS

This work was supported by a Scholar award from the James S. McDonnell foundation and a discovery grant from the National Science and Engineering Research Council of Canada (RGPIN-2016-04890, both to J.D.). Additional support came from the Canada First Research Excellence Fund (BrainsCAN). C.R.H.-C. received a postdoctoral fellowship from the Brain and Mind Institute. R.S.M. received a salary award from CNPq/Brazil. J.A.P. received a salary award from the Canada Research Chairs program.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

C.R.H.-C., R.S.M., J.A.P., and J.D. conceived and designed research; C.R.H.-C. and R.S.M. performed experiments; C.R.H.-C. and R.S.M. analyzed data; C.R.H.-C., R.S.M., J.A.P., and J.D. interpreted results of experiments; C.R.H.-C. and R.S.M. prepared figures; C.R.H.-C. and R.S.M. drafted manuscript; C.R.H.-C., R.S.M., J.A.P., and J.D. edited and revised manuscript; C.R.H.-C., R.S.M., J.A.P., and J.D. approved final version of manuscript.