Fast grip force adaptation to friction relies on localized fingerpad strains

Abstract

During object manipulation, humans adjust the grip force to friction, such that slippery objects are squeezed more firmly than sticky ones. This essential mechanism to keep a stable grasp relies on feedback from tactile afferents innervating the fingertips, that are sensitive to local skin strains. To test if this feedback originates from the skin-object interface, we asked participants to perform a grip-lift task with an instrumented object able to monitor skin strains at the contact through transparent plates of different frictions. We observed that, following an unbeknown change in plate across trials, participants adapted their grip force to friction. After switching from high to low friction, we found a significant increase in strain inside the contact arising ~100 ms before the modulation of grip force, suggesting that differences in strain patterns before lift-off are used by the nervous system to quickly adjust the force to the frictional properties of manipulated objects.

Fast grip force adaptation to friction relies on localized fingerpad strains.

INTRODUCTION

In a seminal paper in the 80s, Johansson and Westling described how efficiently human participants handle objects of different textures and frictions (1). They observed that when lifting objects, participants scaled the grip force (GF; the force applied normally to the surface) to the frictional properties of the surface, such that an object with a slippery surface (low friction) was gripped more firmly than one with a sticky surface (high friction). Moreover, it was found that a change in the frictional properties of the object from one trial to the next elicited a GF adjustment that was observable only 100 ms after contact with the surface. This adaptation was canceled under local anesthesia, underlining the essential role of afferent feedback (2–5). A rapid feedback loop is thus able to take into account tactile information about the surface efficiently (6, 7), but the mechanism by which friction is signaled remains unknown.

The surfaces used in (1) had very different textures (i.e., sandpaper, suede, and silk), it is therefore unclear whether the feedback provided by the tactile afferents was related to the topography of the material, thereby quickly eliciting the recall of a motor memory related to the surface, or if it was directly related to friction, such that the motor system could scale the GF accordingly (8). Notably, it was later demonstrated that humans can adapt to changes in friction (9), even those that are not directly associated with a change in texture (10). In this later study, using coatings to alter friction, participants adjusted the level of GF to the coefficient of friction but not the texture when lifting objects. There must therefore exist a sensory signal related to friction that elicits this adaptation.

Here, we hypothesize that the sensory signal eliciting adjustments to friction is not friction per se, but a warning signal about an impending loss of grip, in the form of surface skin strain resulting from partial, but not complete, slip. During tangential loading of the fingerpad, the object-finger interface is subjected to partial slips, i.e., localized loss of grip between the skin and the surface (11–14). These partial slips also occur during active manipulation of objects (15, 16). Partial slips are associated with substantial skin strains (up to 30%) in the contact area (17) that trigger strong afferent responses (18). Reducing friction accelerates the progress of partial slips and leads to an earlier discharge of the tactile afferents, which can potentially inform the central nervous system about an impending slip (18, 19). Furthermore, generating artificial skin strains at the contact interface with the object during lifting also leads to an increase in GF (20).

To test this hypothesis, we measured where and when skin strains associated with partial slip take place following an unexpected change in friction and if those allow participants to adapt their GF accordingly. To this end, we used a custom-made manipulandum able to record interaction force and skin deformation at the contact area between the object and the fingertips (Fig. 1A) (15, 16). We asked participants to repeatedly grip and lift a manipulandum, while the friction was changed between trials unbeknownst to them (Fig. 1, B to D). We found that participants adjusted their GF to a change in friction, on average, around 100 ms after liftoff of the object, suggesting that most of the sensory information about the friction change was available before the liftoff. To further understand the mechanisms underlying such adjustments, we simultaneously recorded images of the contact area between the index finger and the object to track the skin strains resulting from partial slip (Fig. 1, E to G). These image recordings before the grip modulation revealed a localized strain contrast after changes in friction early in the trial, i.e., before liftoff. Our findings thus support the hypothesis that humans make use of localized strain patterns to adjust the GF to unexpected changes in friction.

Fig. 1. Experimental setup, typical trial, and experimental procedures.

(A) Participants lifted a manipulandum in a precision grip with both fingers in contact with transparent glass plates. The device includes force sensors on both fingers and an imaging system allowing imaging the object-index contact [see (E)]. (B) Evolution of the vertical position of the manipulandum, forces, contact area, and average surface shear during a typical trial. (C) Participants performed 10 blocks of five trials. Transparent plates with high (blue) and low (red) friction were interchanged between each block. Half of the participants started with high friction. The first trial of each block is called a “post-change” trial, as opposed to a “normal” trial. (D) Participants were split into two groups, starting with either the low or the high friction plate. Group means (n = 9 and n = 9) of the GF at the end of the movement [static phase (SP)]. The shaded areas show the SD. (E) Typical image, with contact area, depicted in green. Only the index finger was monitored. (F) Throughout the paper, the object-index contact is shown as if viewed through the finger, with the distal part on the right. (G) Heatmaps of the rates of area change (e_a) and max shear (e_s) rates obtained from a pair of consecutive frames, as described in (17). Strains are observed at the periphery of the contact area. The central stuck zone remains undeformed. The total strain is shown on the right. (H) Coefficient of friction of the index finger as a function of GF for a typical participant obtained using the method described in (22).

RESULTS

Participants performed a series of grip and lift trials using a custom-made manipulandum held in a precision grip (Fig. 1A). Following an auditory cue, they were instructed to grip and lift the object vertically to reach a visual target (Fig. 1B; transport phase) and then hold it stationary (static phase). This lifting movement was followed by three up-and-down movements that are not shown and analyzed in this study [see (16)].

The manipulandum was equipped with force sensors that allowed us to monitor the GF and the load force (LF; acting tangentially to the surface and due to the object’s weight and inertia). A typical movement was accompanied by one LF peak and one LF depression, related to the acceleration and deceleration phases of the movement (Fig. 1B). Note that the LF peak was paired with a GF peak. We synchronized all trials at the instant of liftoff, defined as the point where LF exceeds the weight of the object. During the static phase, LF remained fairly constant (= the object weight: 2.22 ± 0.07 N, mean ± SD across participants) as did GF.

After each experimental block consisting of five trials, the participants were asked to sit on a chair with their backs turned away from the experimental setup, and the experimenter quickly interchanged the surfaces without the participants noticing (Fig. 1C). Two sets of glass plates having different levels of friction were used (see Methods). We defined the first trial following a surface change as a “post-change trial,” since it included an unexpected change in friction, and the other four trials were called “normal trials.” The first two blocks were considered to be “training blocks” as GF decreased significantly during those for all participants and were thus excluded from the data analysis (Fig. 1, C and D). All trials were successful, in that the participants always managed to keep the device stably in their hands, and no object drop was observed.

The index fingertip contact with the object surface was monitored through the glass plates using a high-speed, high-resolution camera (Fig. 1, E and F). Image processing techniques allowed us to track fingerprint movements and evaluate surface skin strains during the lift movement [Fig. 1G, (12, 17), and see Methods]. The rate of change of skin strains resulting from LF increase during the lift of the manipulandum was observed at the periphery of the contact area (Fig. 1, B and G). In Fig. 1G, the strain tensor was decomposed into a component related to a net change in the area (e_a) and a component of maximum shear (e_s) (see Methods) (21). During the lifting movement, contraction was observed in the lower part of the contact area (shown in red in Fig. 1G), and expansion was observed in the upper part (in blue). Shear was observed over the whole periphery of contact. Given that it was the largest component, it is the variable that we chose to analyze in detail. In most trials, the center of the fingertip remained stuck and nondeformed. The contact area was an approximately ellipsoidal shape and increased together with the GF, consistent with previous observations (Fig. 1, B and G) (11). The strain rate peaked around the time of liftoff (Fig. 1, B and G), which slightly preceded the time when the smallest stuck area was reached (peak cross-correlation of −0.75 ± 0.2 with a lag of 40 ms, mean ± SD across participants). The total strain created by one lifting movement was of the order of a few percent and had an annular shape (Fig. 1G, right).

The friction between the fingertips, index and thumb, and the two sets of plates was measured at the end of the experiment. It was done at the end of the session to reduce potential cues about the different materials that could have been used during the manipulation task. Note that all participants, except three, reported that they did not notice that different materials were used before the friction test. Following (22) (see Methods), we characterized the coefficient of friction over a range of GF relevant to our experiment (see Fig. 1H). The data were obtained for both fingers of all participants and fit with a negative power law. No difference in friction was observed between the thumb and the finger [t test: t₃₅ = −0.21, P = 0.834, high and low friction aggregated]; therefore, the average value was considered. We observed that the coefficient of friction of the low-friction glass remained lower than the one of the high-friction glass across all levels of normal force tested, as shown in Fig. 1H for a typical participant. From the fits obtained for each participant, we summarized the coefficient of friction by a single value, being the average coefficient of friction over the range of 1 to 8 N of normal force (which approximately corresponds to the 5 to 95 percentile range of GF used in this study), and across fingers.

GF adapts to changes in friction

Before assessing whether the plate changes had any impact on the participant’s behavior, it is necessary to verify that the coefficient of friction between the plates and the fingers was actually different. We found that, indeed, friction was always higher in the case of the high-friction material, and the average relative difference was larger than 20% (Fig. 2, A and C, Wilcoxon signed-rank test: n = 18, Z = −3.72, P = 0.000, median (Mdn): −21.5%). Given that we aimed to observe behavioral adaption to changes in friction, we required a sufficient difference between materials and set a lower bound to a relative difference of 10%. Accordingly, one participant was removed from all subsequent analyses (Fig. 3 onward) because the relative friction difference was too low (only 4%; see near line dot in Fig. 2A and lower right dot in Fig. 2C). In summary, then, the two flat and transparent materials used in this study showed a consistent difference in friction.

Fig. 2. Adaptation to friction during normal trials.

(A) Coefficient of friction for high-friction versus low-friction materials averaged over the observed manipulation range (1 to 8 N) for each participant (n = 18). (B) GF for high vs low friction averaged during the static phase of normal trials for all participants (n = 18). (C) Relative GF change from high to low friction versus friction change. The change was computed according to (low-high)/(low/2 + high/2) and is expressed in percent. (D) GF versus slip force for each participant and each friction condition. Significant t tests are shown with asterisks (**P < 0.01 and ***P < 0.001).

Fig. 3. GF adaptation to friction during post-change trials.

(A) GF change during the static phases for post-change trials under the low-friction condition versus normal trials under the high-friction condition (post-change low) (n = 18). (B) Same as (A), but for post-change trials under the high-friction condition versus normal trials under the low-friction condition (post-change high). (C) Relative change of GF during the static phases for the three kinds of changes. Computed as (after-before)/(after/2 + before/2) (D) Same as (C), but at liftoff. Significant one-sample t tests are shown with asterisks (*P < 0.05, **P < 0.01, and ***P < 0.001). L, low; H, high.

Next, we sought to assess whether this difference in friction elicited different gripping behavior. First, we tested if participants could adapt to the difference in friction by using a consistently higher GF for the lower friction during the normal trials, i.e., those not following an unexpected change in friction. We found that, indeed, all except two participants used a higher level of GF for the lower friction as averaged during the static phases (Fig. 2B), although the level of GF varied widely across participants. Overall, the relative difference was statistically significant (Fig. 2C, one sample t test: t₁₇ = 3.36, P = 0.004, mean: 13.7%). Thus, participants spontaneously adjusted the GF level to the friction condition despite the absence of a difference in texture. Moreover, the relative GF change was correlated with the friction change. We observed that the participant that lacked difference in friction showed a slight decrease of GF for the low-friction material (Fig. 2C, lower right dot).

Note that most participants used a relatively high safety margin for the high friction condition, such that they did not actually require any GF adjustment to maintain a stable contact for the low-friction condition (Fig. 2D). However, this adjustment was nevertheless observed consistently.

GF adjustment takes place during the first movement of post-change trials

Having observed that GF was indeed adjusted to friction during normal trials, we then tested if a change in friction elicited a quick adjustment of GF, as observed after the lifting movement, during the static phase of the post-change trials. To that end, we compared the static GF (Fig. 1B) of the post-change trials to the static GF developed during the normal trials with the other plate. The post-change trials could be of two types: (i) low-friction post-change trials, following an adapted exposure to high friction, were referred to as “post-change low,”, and (ii) high-friction post-change trials, following an adapted exposure to low friction, were referred to as “post-change high.” The difference or change between two conditions (post change minus normal) really emphasizes the effect of a change in friction.

For the post-change low trials, which required an urgent increase in GF because the decrease in friction increased the risk of slip of the object, we found that GF was already higher during the static phase (Fig. 3A). The adaptation was close to 20%, thus already of the same magnitude as the adaptation learned throughout many trials (Fig. 3C, one-sample t test: t₁₇ = 4.22, P < 0.001, mean: 19.4%). For the “post-change high” trials, for which the urgency was lower because the risk of slip decreased compared to preceding trials, we also found that the GF decreased after only one movement (Fig. 3B), although the relative difference of GF was about −9% on average (Fig. 3C, one-sample t test: t₁₇ = −2.31, P = 0.033, mean: −8.8%), thus not yet completely adapted.

Briefly, participants adapted GF to the friction condition, and this adaptation was already present after the first movement following a post-change trial and for both directions in friction change (Fig. 3C). However, while normal trials showed GF profiles that already diverged between conditions at the liftoff (Fig. 3D, one-sample t test: t₁₇ = 2.48, P = 0.024, mean: 6.4%), this divergence was not yet significant at that moment for post-change trials (Fig. 3D, post-change low Wilcoxon signed-rank test: n = 18, Z = 1.72, P = 0.085 and post-change high one-sample t test: t₁₇ = 1.43, P = 0.17).

GF adjustments start just after the liftoff of post-change trials

Because GF was already adjusted to the friction level after the first movement following a post-change trial, but not yet at the time of lift-off, we investigated the temporal evolution of GF during these post-change trials to determine when the changes in friction evoked the change in GF (Fig. 4). The participants produced movements with very similar kinematics across conditions, as shown by the object height and the LF curves, which could not be distinguished across conditions (Fig. 4, A to C, top two rows). We observed that GF curves of post-change trials progressively diverged from those of normal trials (Fig. 4, A to C, third row).

Fig. 4. GF adjustments to friction during the post-change trials.

(A) Average evolution of object height, LF, and GF as a function of time for post-change low trials. Trials were synchronized on the liftoff. (B) Same as (A) for the post-change-high trials. (C) Averaged evolution of the paired difference between post-change and normal trials. Data were aggregated in 50-ms bins and systemically tested for significance. Only significant results are shown with asterisks (*P < 0.05, **P < 0.01, and ***P < 0.001). Lines are averages, and shaded areas or error bars are SEM across participants (n = 18). Error bars on contact time and static phase show SD across participants.

For the post-change low trials (Fig. 4, A and C), we found that the GF difference reached statistical significance very early (paired t test; see Fig. 4 caption), i.e., just after liftoff (the 100 to 150 ms bin after liftoff was the first significant). This timing corresponded to approximately 300 ms after initial contact (contact time was 212 ± 7 ms before liftoff, mean ± SD across participants). This difference was substantial, as it peaked, on average, at around 0.8 N during the movement. For the post-change high trials (Fig. 4, B and C), the difference in GF never reached statistical significance during the movement. Given that the post-change high did not elicit fast adjustments of GF but rather a slow and progressive release, which might underlie a different mechanism, we did not analyze those trials further.

Note that participants tended to apply similar levels of GF (i.e., not statistically different) at the very beginning of the contact (i.e., before liftoff) of post-change trials no matter the sign of the friction change (i.e., post-change low or post-change high). This suggests that participants did not anticipate the friction for the post-change trials and that the adjustment was purely made based on the tactile feedback. In summary, we showed that GF reaches a level that is significantly different from the level of the normal trials during the first lift following a change in friction, approximately 100 ms after liftoff for the post-change low and much later on, during the static phase, for the post-change high trials.

Surface skin strain at the object-finger interface can provide feedback about contact stability

Next, we sought to characterize the skin strain taking place at the fingertip object interface during the early phase of the lift and also to anticipate what change in strain a post-change trial should elicit. To that end, we decided to focus on the maximal shear strain rate, e_s (Fig. 1G, lower row), which was the most salient component and produced clear annular strain patterns (Fig. 5A).

Fig. 5. Skin strains during the lifting movement.

(A) Evolution of the maximum shear strain as a function of time for a variety of example trials randomly taken from the dataset. Trials were synchronized on the liftoff. Participant number and condition (normal high, normal low, post-change high, and post-change low) are shown on the right. (B) Subset of the examples shown above at the instant of liftoff. Four ellipses are overlaid, showing the band of interest, the warning zone, between the inner ellipse and the third ellipse (in black). (C) e_s rate in the warning zone as a function of GF at the instant of liftoff for each participant. (D) Stick radius as a function of GF at the instant of liftoff for each participant. (E) Relative change of e_s rate in the warning zone from high to low friction at liftoff for the three kinds of changes. Computed as (low-high)/(low/2 + high/2). Only significant results are shown with asterisks (**P < 0.01 and ***P < 0.001). Dots are averages, and error bars show SD across trials.

Because our setup uses a lightning trick to obtain a high contrast between elements in contact and elements not in contact (see Methods), the initiation of contact leads to some artifact in our tracking algorithm, visible at the very beginning of the contact (first couple of frames), during the steep growth of the contact area (Fig. 1G, first frame, and Fig. 5A). It is therefore not clear if a stereotypical pattern develops at that moment and if it is influenced by friction (23). Nevertheless, just after this period develops a very clear annular pattern, which is not divergent as would be expected by the initial, purely normal contact (23) but dominated by shear and asymmetrical with stretch on the top and contraction on the bottom (Fig. 1G) as would be expected by tangential loading (15, 17). Although the timing of the evolution of the contact, the orientation of the contact contour, and the location of the stuck nondeformed area varied widely across participants and trials, the main annular pattern is very well preserved (Fig. 5A). The largest strain rates were systematically observed at the instant of liftoff.

We used Hertz contact theory to model the surface skin strain for different levels of stick radii, representing the proportion of the contact area in a stable, nonslipping state (see Methods), similar to those observed in the data (fig. S1A). We found that our simulations predicted that most of the difference in strain rate across conditions would take place in an elliptic band that does comprise neither the center of contact, which remains undeformed if no slip occurs, nor the external parts of the contact, which are always deformed. This middle band, the warning zone, arbitrarily set for a radius between ¼ and ¾ of the external contact radius following our simulations results, is extremely sensitive to changes in GF or friction (fig. S1B). To summarize e_s into one scalar value that possibly signals impending slip, we therefore averaged the e_s values over the middle band (Fig. 5B). We found that this averaged e_s rate in the warning zone at the liftoff scaled nicely with GF (Fig. 5C) as did the stick radius (Fig. 5D).

Looking at the relative change of e_s between conditions (Fig. 5E), we found a moderate change across normal trials, that is, e_s was slightly larger for low-friction trials (t test: t₁₄ = 3.22, P = 0.006, mean: 15.7%). Critically, e_s was much larger for post-change low trials when compared to preceding normal high trials (t test: t₁₄ = 4.51, P < 0.001, mean: 34.8%). This trend was not observed on post-change high trials (t test: t₁₄ = −1.55, P = 0.144, mean: −9.6%), probably because the expected change is lower (fig. S1). This strongly suggests that e_s can be used as a warning signal to adjust the GF.

Contrasts in skin strain rate before liftoff are cues for GF adjustments

To test if e_s can be used as a warning signal to adjust the GF, we looked at the time evolution of both GF and strain rate. We restricted our analyses to the post-change low trials, for which we observed a significant change in GF with respect to normal high trials (paired test, see Fig. 6 caption) around 100 ms after liftoff (Fig. 6, A and B, first row).

Fig. 6. Differences in skin strains preceding the force adaptation.

(A) Average evolution of GF, GF rate (dGF/dt), average maximum shear strain in the middle annulus of the contact ellipse, and stick radius as a function of time for post-change low trials. Trials were synchronized on the liftoff. (B) Averaged evolution of the paired difference between post-change low and normal high trials. Data were aggregated in 50-ms bins and systemically tested for significance. Only significant results are shown with stars (*P < 0.05, **P < 0.01, and ***P < 0.001). Lines are averages, and shaded areas or error bars show SEM across participants (n = 15). Error bars on contact time and static phase show SD across participants. (C) Excess GF rate (between 50 and 200 ms after liftoff) as a function of excess maximum shear strain in the middle annulus (between −100 and 100 ms after liftoff) for post-change low trials with respect to normal trials (n = 48). w.r.t., with respect to.

To probe the exact timing of the reactive behavior in response to a particular sensory event, we looked at the rate of change in GF (Fig. 6, A and B, second row). Note that in Fig. 6, in contrast to Fig. 4, we excluded the data from two participants having consistently poor image quality (therefore, n = 15) and we also only included the trials in which the images reached high-quality standards (84% of trials among the remaining participants, see Methods for more details). As observed in Fig. 6B, the ∆GF rate trace diverged just after liftoff, becoming significant in the 50- to 100-ms bin after liftoff, suggesting that online corrections of GF in response to changes in friction arise, on average, around this moment.

Those GF responses to post-change low trials suggest that a sensory signal informative about the frictional properties of the material was triggered before this time. Taking into account the conduction delays, which for tactile-motor responses are close to 90 ms on average (24), the sensory signal should start to emerge 50 ms before liftoff. Accordingly, we observed that the maximum shear strain was higher in post-change low trials compared to normal trials around the time of liftoff (Fig. 6A, third row), and the stick radius was at the same time lower (Fig. 6A, last row). Looking at the difference between post-change low and normal, we observed a significantly higher strain rate in the bin starting 50 m before liftoff (Fig. 6B, third row). Likewise, the stick radius was consistently smaller for post-change low trials around the time of liftoff (Fig. 6B, last row). This observation suggests that this sensory signal was available for the central nervous system to react by an appropriate adjustment of the GF.

To rule out the possibility that the effects observed are due to the learning of a particular sequence of blocks, we analyzed separately the first transition from high to low friction (see Fig. 1C). That is, we used transition from block 2 to 3 for subjects #10 to #18 and from block 3 to 4 for subjects #1 to #9 (transitions from block 1 to 2 for subject #1 to #9 were excluded because of the excessive GF applied, see Fig. 1D). For those transitions, the learning of the sequence was impossible because the subjects were not exposed to the sequence earlier. We found very similar trends: the ∆e_s rate was significantly different from 0 in the bins 0 to 100 ms before liftoff (paired t test, P < 0.01 and P < 0.05), and the ∆GF rate was significantly different from 0 in the bins 0 to 100 ms after liftoff (paired t test, P < 0.05). So, the effect was already observed in the first or second transition from high to low friction. Furthermore, to confirm that this GF increase was not systematic in post-change trial, we also separately analyzed the first transition from low to high friction. We found that, indeed, neither ∆e_s rate nor ∆GF rate reached significance at any time during those trials, suggesting that the GF rate increase observed during the first post-change to low is not solely a result of the block transition.

To confirm the causal link between strain rate in the warning band and GF rate increase, we averaged, for each post-change low trial, the excess of e_s rate around the time of liftoff (±100 ms) with respect to normal trials and the excess of GF rate on the same time window length but 150 ms later (Fig. 6C). While the trend observed in Fig. 6B is reproduced, that is, both variables are mostly positive, no correlation was found. The absence of such a correlation is discussed below.

The data related to post-change to high trials did not reveal any significant trend. The slow release of the GF might be explained by a slightly lower level of deformation than in normal trials at the liftoff (that did not reach significance in our analyses) or could be an automatic progressive release in the absence of a “warning signal.”

In summary, we observed the onset of a motor response resulting from the friction change approximately 50 ms after liftoff, and this online GF correction was consistent with a sensory signal resulting from an increase in the strain rates closer to the central parts for the contact area happening approximately 150 ms before the motor response. This subtle but essential sensory signal, therefore, is the information that was used in the brain to adjust GF control.

DISCUSSION

We quantified how fast humans could adjust their GF to a change in friction specifically (not texture) with flat transparent surfaces swapped across blocks of trials. We suggest that this adjustment was triggered by local strain patterns that took place in the contact area with the manipulated object and could signal an unstable grip. Specifically, we show that when confronted with lower friction than expected, slip and surface skin strains progress more centrally in the contact area. The differences in skin strains with respect to a normal trial were already significant very early after the initial increase of the LF and more than 100 ms before the GF response, which is a reasonable delay to explain it (24, 25). These differences could thus constitute a warning signal that allows the central nervous system to adjust GF to the friction condition. A characteristic of our results was the asymmetry of the participants’ behavior between post-change high and post-change low trials (Fig. 4). It is worth noting that a short reaction time in the post-change low condition is critical: It is urgent to increase GF because not correcting it could lead to a marked slip and drop of the object. In contrast, the excessive GF in the post-change-high trials only results in a temporary slight excess of energy expenditure, which does not require reacting quickly.

Large variability

We observed that the levels of skin strains varied greatly from participant to participant (Fig. 5, C and D), an effect mainly driven by the variable level of GF. We have shown similar levels of variation of skin strains in a previous study where participants had to perform oscillations in a precision grip (15). We also observed in that study that a larger amount of skin strains is linked to a lower stick radius and that the stick ratio also varied greatly between participants. The stick radius was also measured in a study where participants had to perform a grip-lifting task (26). The authors hypothesized that humans control the level of GF to maintain a constant amount of partial slip (~40% of the contact area) when lifting objects of known friction and weight, but they mentioned in their paper that the validity of this hypothesis has to be verified, as only three participants were tested. It is worth noting that humans perceive slippage at very different levels of partial slip in a passive setting (27). The variability in the levels of strains and stick ratio during manipulation and the variability in the partial slip between participants seem to point toward strategies of manipulation that vary from person to person. This requires further inquiring by performing experiments with tasks of different natures.

Weakness or lack of correlation on trial-by-trial basis

It can be surprising that we did not find a strong correlation between the sensory signal and the motor response on a trial-by-trial basis. However, several factors can explain this finding. First, our manipulandum can only observe the contact with the index finger. Our analyses, therefore, exclude all potential sensory information coming from the thumb. Given that the forces are similar on both fingers (because the object was moved vertically and kept upright), and given that the measured frictions were similar across fingers, we could expect similar strain patterns. Given the variability of the observed strain across trials, it is likely that the contribution of the thumb is as important as the contribution of the index finger. Second, most participants manipulated the object with a high level of safety margin; therefore, the presence of a higher strain might not always trigger a motor response, given that the grip is stable. Given the variability of the shape of the contact, most probably also of the pressure distribution, of the level of deformation, this variable might not ideally capture the best sensory signal, which can probably not be reduced to a single value. More controlled experiments might be more suited to finding the best candidate. Last, large variability between perception and skin deformation has already been observed before (23, 27).

Friction

Although we characterized the friction between the fingers and the manipulated object by a constant scalar value per participant-condition pair (Fig. 2, B and C), this is a gross approximation of a very complex phenomenon. The tribology of the skin is complex and varies significantly at different temporal and spatial scales (28–30). One aspect of this complexity is related to the complex geometry and mechanics of the finger: The fingertips are composed of several layers, from the bone in the interior of the fingertip to the epidermis in the exterior. The epidermis of the glabrous skin is characterized by the presence of ridges and furrows that form the fingerprints and present a complex topography (31). This complex geometry and mechanics are likely to impact the friction on a trial-to-trial basis, depending on how the finger contacts the object. Another source of complexity is the fact that the contact interface evolves over time: Sweat pores are present at the surface of the skin and produce humidity, which varies over the course of several seconds and heavily affects the level of friction when gripping a material (32, 33). The occlusion phenomenon defines humidity evolution and plasticization of the skin when in close contact with the material for a few seconds (30). The level of friction thus varies during a single trial as moisture varies due to occlusion. In our experiment, the moisture level is however probably partially maintained when trials are close enough to one another. Thus, in the context of this study, the differences in friction measured between materials, which could vary widely between participants, are to be considered approximations of their actual values, which can vary during manipulation. Last, while the surface used are flat and without texture, most encountered materials have a much higher degree of roughness. The sensory signal arising from partial slip in the case of natural surface might be amplified in a more ecological context by two different phenomena: first, by the more sudden slip or release of portions of the contact, instead of an almost continuous progression in the case of glass, and by the generation of vibrations due to the interaction between the rough elements and the fingerprint ridges in the slipping part. Some vibrations were measured by a previous study during partial slip (1).

Initial contact

A recent study suggested that the information necessary to judge the friction of a material is provided under the initial normal contact (23). A more slippery contact lets the skin expand more than a sticky contact when pushing normally to the surface, leading to measurable differences in skin deformation. The study found that, indeed, a relatively large change in friction (50%) could be perceived by only pressing the finger on the surface. The study reported radial strains of around 3%, and strain rates peaking at 5 to 10%/s. Our setup did not enable us to properly quantify the strain at the initial contact, which might well be very informative about friction. While we cannot reject that this phenomenon occurred here, we suggest that the strains related to the tangential loading could be much more informative about contact stability. First, it enabled to adjust GF to much smaller changes in friction, around 20% in this study. Second, the observed deformations are larger, with strain rates peaking at around 20%/s. Finally, they are present also in all subsequent movements and can provide continuous monitoring of contact stability.

Multiple senses are involved when performing fine object manipulation, of which our sense of touch has particular importance. It sends critical information to the central nervous system to adjust GFs to various parameters of the manipulation such as friction. We have shown that when gripping and lifting small objects, the skin strains depended on the level of friction at the interface of the contact. The localized differences in skin strains between conditions during the loading phase were consistent with the timing of the first signs of adaptation of the GF and were necessary to explain them.

METHODS

Participants

Eighteen volunteers (five females; ages 20 to 65) participated in the experiment. All of them provided written informed consent to the procedures and the study was approved by the ethics committee at the host institution (UCLouvain, Brussels, Belgium).

Apparatus

At rest, the device was standing on a table with a hole to allow the passage of the cables coming from the bottom of the device (Fig. 1A). Its weight (540 g) was partially compensated by a counterweight (320 g) attached to a system of pulleys. The device is described at length in a recent publication (15). Succinctly, forces were measured under each fingertip using two six-axis force and torque sensors (ATI Mini27 Ti, ATI-IA, Apex, NC, USA). From those measurements, the GF and LF were computed, as described in the “Data analysis” section. The position was measured using an optical distance sensor (DT20-P224B, SICK Sensor Intelligence). The position and forces were sampled at 200 Hertz with a NI-DAQ card (NI6225, National Instruments).

A custom optical system allowed to image the index fingerprints in contact with the glass (Fig. 1E). Because of constraints in the design of the manipulandum, it was only possible to monitor one finger, as the light is emitted by an array from the side where the thumb is, blocking its observation. This system is based on the principle of frustrated total internal reflection and enables a high contrast between the points in contact with the glass and those that are not. Images are recorded at 50 frames/s with a camera (GO-5000 M-PMCL, JAI, monochrome, 2560 × 2048 full pixel resolution). The image size is 1696 × 1248 pixels with a resolution of 4096 pixels/mm², which corresponds to an area of 26.5 mm × 19.5 mm.

Two kinds of glass plates were used to generate different levels of friction. The first set of plates is simple transparent optically flat plates of glass. They are referred to as “high friction.” A process called “glass frosting” was used to alter friction in the second set of plates. In brief, a chemical process was used to imprint a nanoscale pattern on the surface of the glass. With the right set of parameters (height and roughness), this decreased the real area of contact between the finger and the plate and thus the coefficient of friction (30, 34–38). This nanostructured glass was referred to as the “low-friction” surface. The transparent plates are indistinguishable to the naked eye.

Experimental procedures

Participants stood in front of a table on which the device was positioned. After an auditory cue, they were instructed to grip and lift the device to a height of about 20 cm within 0.8 s and then hold it still for 1.5 s (Fig. 1B). They then performed three fast point-to-point movements (0.8 s) with pauses (1.5 s) in-between. Auditory cues were used to pace each movement. We observed that participants’ movements were slightly slower than what was instructed, resulting in slightly longer movements and shorter static phases. The participants were requested (and often reminded during the experiment) to use a minimal amount of GF. We observed in a preliminary study that participants tended to use an excessive amount of GF naturally, probably because this device composed of a camera and sensors seems fragile and looks heavy. The glass plates were cleaned with alcohol between each trial. This served the purpose of getting images as clean as possible. Also, this procedure removed sweat that could alter the topography of the glass plates at a microscopic level and thus the level of friction. After each block of five trials, participants were instructed to take a break and turn away from the setup, such that they could not see the experimenter manipulating the device. During that break, the experimenter interchanged the plates such that the friction was changed from high to low or from low to high (for both the index finger and the thumb). This procedure was quick and took a maximum of 2 min. Half of the participants started with the high-friction condition and the other half with the low friction (Fig. 1, C and D). This caused no difference in their adaptation to friction, as measured by the difference in GF between conditions during the static phase of normal trials. The coefficient of friction was measured for each material at the end of the experiment (Fig. 1H and see the “Friction measurement” section). In total, the experiment lasted between one and a half and 2 hours for each participant.

Friction measurements

We measured the coefficient of friction between the participants’ fingers and both materials at the end of the experiment using the method described by (22). Briefly, participants were instructed to rub their index finger and thumb on the glass plates for three periods of 15 s at different levels of normal force. The approximate range of normal force for each period was 0 to 2.5 N for the first, 2.5 to 6 N for the second, and 6 to 10 N for the third. The moment of slippage was detected by finding the maximum ratio of tangential force over normal force at the start of each rubbing motion. This ratio was measured and was our estimation for the static coefficient of friction corresponding to the normal force applied at that moment. The data were obtained for both fingers of all participants (see Fig. 1H) and fit with a negative power law [μ = k(NF)^(n–1), where μ is the coefficient of friction] (39). From the fits, we computed a single coefficient of friction value for each participant and each material for the 1 to 8 N range that corresponded to the approximate range that the participants used for manipulation (close to the 5 to 95 percentiles). The friction was averaged across both fingers. The first measurement of the friction was always performed with the same material as the one used in the last block of trials.

Data analysis

Forces and position

Data were filtered with a fourth-order low-pass Butterworth filter with a cutoff frequency of 40 Hz. The GF is defined as the mean of the norm of the forces normal to the surface of the object exerted by each finger. The LF is defined as the norm of the sum of the forces applied tangentially to both surfaces by each finger. The slip force (SF) is the minimum normal (grip) force needed to avoid slip. It was obtained from the power function fit for each subject as a function of the LF [LF/2 = μSF = k(SF)ⁿ]. The instant of liftoff was defined as the first sample after LF exceeded the weight of the object (220 g). The initial contact was obtained once GF exceeded 5 SD of the background noise. The static phase was defined on the basis of thresholds for maximal value on the velocity (<3 cm/s) and LF (2.2 ± 0.15 N). Those values were arbitrarily defined by visual inspection of the trials.

Image processing

A previously described image processing pipeline was used to evaluate the skin strains from the raw images [see (12, 17)]. This pipeline is only summarized here. First, a custom-made machine-learning algorithm was used to detect the area of contact between the finger and the glass plate for each image. This algorithm was trained for each participant separately with manually detected areas of contact for randomly selected images. Then, feature points were selected automatically from several frames of the sequences of images (at the beginning, during, and at the end of each movement). Their position was tracked forward and backward in time from frame to frame using an algorithm of optical flow (40, 41). Delaunay triangulation was then computed, and the evolution of the shape of the triangles allowed measuring the local strain rate along three dimensions (ε_xx horizontal, ε_yy vertical, and ε_xy shear strain). The strains were assigned to the center of each triangle. From those three components, we obtained for each triangle and each pair of frame the principal strains (e₁ and e₂) by eigenvalue decomposition (17) and, lastly, the area change [e_a = (e₁ + e₂)/2] and the maximum shear strain [e_s = (e₁ − e₂)/2] (42). In this study, we were mostly interested in comparing the amount of strain rate according to the condition of friction and the adaptation of GF rather than the specific description of these strains (15).

The first images of the contact were difficult to interpret. The fingertip skin can be rough and stiff on a small scale, depending on the moisture content of the individual’s skin. When it enters into contact with a stiff surface such as glass, the initial real area of contact is low. However, during the first tens of milliseconds of the contact, moisture secreted by the sweat pores hydrates the skin, rendering it softer and elasticizing it. The skin then enters in closer contact with the surface and the real area of contact increases (43–45). As a rapidly changing real contact area not associated with skin strains was problematic for the interpretation of the results of our image-processing pipeline, we decided to discard the images directly following the time of contact between the skin and the surface from our analysis. We used the first image of the loading phase, defined as the moment when a participant starts applying tangential force to lift the object after the preloading phase (46) as the first image in our image processing pipeline. This guaranteed that the apparition of moisture would only play a negligible role in our measurement of strains and that the strains caused by the vertical lifting of the object would be included in our analysis.

Stick radius

The stick radius was obtained as the square root of the instantaneous ratio of the stuck area to the total contact area. Each triangle was defined as stuck or slipping for each pair of frames by setting a threshold of 0.25 pixels of movement of its center between frames, a value chosen to be well above noise level but still very sensitive. While we report the stick radius in some analyses, we prefer to focus on the skin strain rates, which are the determinants of the afferent responses (18).

Inspection and sorting of trials

As mentioned in Results, the first two blocks were considered to be “training blocks” as the participant GF decreased significantly during those for all participants and were thus excluded from the data analysis (Fig. 1, C and D). The third block was the first included in our analyses. For some trials, participants placed their index finger outside of the field of the camera or displaced it outside of the field during the trial due to slipping or rolling. Some trials were therefore not included in the analysis. Only the parts of the trials where the finger got out of the field of the camera were removed. After a close inspection of each trial, 99 of 720 were at least partly removed because the images were unusable during some part of the trial. Those trials were still used for the kinematics and dynamics analysis of Figs. 2 to 4. Also, three participants were removed for the image and forces analyses of Figs. 5 and 6 because they had very dry skin, and the image quality was insufficient to obtain reliable strain data.

Full slip trials

To get a different look at our data, we counted the proportion of trials for which a full slip occurred during the first movement, which yielded the following results: A full slip occurred in 11.33% of the high-friction normal trials, 12.75% of the low-friction normal trials, 11.77% of high-friction post-change trials, and 31.03% of low-friction post-change trials. A full slip is said to occur when all the feature points whose positions are tracked from frame to frame are measured as moving with respect to the glass. That is, in some cases, the strain wave reached the central point of the contact. Note that even if full slip occurred for some trials, the extent of slippage was small and was quickly stopped by a corrective GF [slipping distance of the central part of the contact area of 0.19 mm (± 0.25 mm, mean ± SD), for trials in which the full slip was reached]. The full slip trials were included in the strain analysis similar to all other trials because none of them resulted in object drop. We did not observe any particular behavior specifically related to full slip trials.

Statistical analyses

All statistical analyses were performed with MATLAB using the functions corr (for Pearson correlation), and ttest (for one-sample or paired t tests). Data normality was tested on with the function adtest (Anderson-Darling test), and the t test was replaced with a Wilcoxon signed-rank test (function signrank) if the null hypothesis was rejected. The test performed, the number of degrees of freedom, and the t statistics are always mentioned with the P value.

Supplementary Materials

This PDF file includes:

Fig. S1