Representing multiple object weights: competing priors and sensorimotor memories

The present study is the first to examine the specificity of learning processes underlying adaption of long-term priors while also examining the ability to form sensorimotor memories for multiple sets of objects simultaneously. In a follow-up study, we directly test the hypothesis that sensorimotor memory involves explicit working memory resources.

Abstract

When lifting an object, individuals scale lifting forces based on long-term priors relating external object properties (such as material and size) to object weight. When experiencing objects that are poorly predicted by priors, people rapidly form and update sensorimotor memories that can be used to predict an object's atypical size-weight relation in support of predictively scaling lift forces. With extensive experience in lifting such objects, long-term priors, assessed with weight judgments, are gradually updated. The aim of the present study was to understand the formation and updating of these memory processes. Participants lifted, over multiple days, a set of black cubes with a normal size-weight mapping and green cubes with an inverse size-weight mapping. Sensorimotor memory was assessed with lifting forces, and priors associated with the black and green cubes were assessed with the size-weight illusion (SWI). Interference was observed in terms of adaptation of the SWI, indicating that priors were not independently adjusted. Half of the participants rapidly learned to scale lift forces appropriately, whereas reduced learning was observed in the others, suggesting that individual differences may be affecting sensorimotor memory abilities. A follow-up experiment showed that lifting forces are not accurately scaled to objects when concurrently performing a visuomotor association task, suggesting that sensorimotor memory formation involves cognitive resources to instantiate the mapping between object identity and weight, potentially explaining the results of experiment 1. These results provide novel insight into the formation and updating of sensorimotor memories and provide support for the independent adjustment of sensorimotor memory and priors.

NEW & NOTEWORTHY

The present study is the first to examine the specificity of learning processes underlying adaption of long-term priors while also examining the ability to form sensorimotor memories for multiple sets of objects simultaneously. In a follow-up study, we directly test the hypothesis that sensorimotor memory involves explicit working memory resources.

on a daily basis, we are required to lift a multitude of objects that can differ in size and material. To efficiently lift an object, the ability to accurately predict the weight of the to-be-lifted object is essential (Flanagan et al. 2006; Wolpert and Flanagan 2001). For example, when lifting an object slightly above a lifting surface, people typically smoothly increase vertical load force to a level that is slightly higher than the weight of the object such that the object can be accelerated upward during the lift. When this prediction is well matched to the object weight, the object is efficiently lifted up.

When lifting an object for the first time, people make use of visual and haptic information related to object size (Flanagan and Beltzner 2000; Gordon et al. 1991a, 1991b) and material (Buckingham et al. 2009; Gordon et al. 1993) to predict its weight. Such predictions are thought to be based on well-learned size-weight and material-density priors (Cole 2008; Flanagan et al. 2008). Not only are these priors used in lifting new objects, they also underlie weight illusions. An example of these priors can be seen in the size-weight illusion (SWI) (Brayanov and Smith 2010; Buckingham and Goodale 2013; Charpentier 1891; Flanagan et al. 2008; Flanagan and Beltzner 2000; Grandy and Westwood 2006; Mon-Williams and Murray 2000; Ross 1969), in which people judge the smaller of two equally weighted objects to be heavier because it is heavier than expected. The SWI is resistant to change, even with extensive experience with illusion-eliciting stimuli, and persists when subjects have been told that the objects are equal in weight (Flanagan and Beltzner 2000; Flournoy 1894). Thus short-term interactions with objects that violate the normal mapping between size and weight have little effect on the SWI, and the same holds for the material-weight illusion (Buckingham et al. 2011), in which people judge equally weighted items that look like they are made from lighter materials as heavier than items made from heavier materials (Harshfield and DeHardt 1970).

Flanagan et al. (2008) provided evidence that distinct sources of information are utilized when lifting objects and when making perceptual judgments about their weight. This study demonstrated that with extensive experience with a set of objects whose size and weight are inversely related (e.g., the smallest objects were the heaviest) the SWI becomes reversed, so that the larger of two equally weighted cubes feels heavier. This study also demonstrated that lifters are able to correct lift forces at a much more rapid rate than the effects of experience have on the perceptual SWI, suggesting that the sensorimotor memory used in lifting behavior is distinct from the long-term priors utilized during perceptual weight predictions. One plausible reason for these differing timescales is the amount of previous experience each is based on. Priors used when judging weight may change slowly because they are based on a lifetime of well-established correlations between size and weight for large sets of related objects. In contrast, sensorimotor memory supporting lifting behavior adapts at a faster rate, as it is tuned based on much smaller sets of specific objects to be manipulated. In combination, this allows priors to change when there is ample evidence present in the external world but to resist temporary perturbations in our environment.

A number of questions naturally follow from these results. First, there has yet to be an examination of the specificity of learning processes underlying the adaptation of long-term priors; for example, it is not known how adaptation of priors for one set of unusually weighted objects influences priors for normal objects lifted at the same time. Next, previous studies (including our own) have only examined the formation of sensorimotor memories for a single set of objects. In our day-to-day lives we encounter multiple sets of objects concurrently, and therefore an examination of how sensorimotor memories are formed in this scenario is prudent. Finally, little is known about the cognitive resources involved in the encoding and recall of sensorimotor memories. Previous research has demonstrated that the ability to use arbitrary visual cues for predictive control of fingertip forces during object lifting is affected by natural aging, likely because of failures in associative learning (Cole and Rotella 2002). However, theories of practice-based automaticity suggest that well-learned skill execution requires few cognitive resources (Anderson 1982; Langer and Imber 1979). As the act of lifting an object based on sensorimotor memory is one that we presumably have years of experience with, it is plausible that the use and updating of sensorimotor memories are somewhat impervious to the effects of cognitive load. In two experiments, we examined each of these questions by training participants to lift two families of objects that had opposite size-weight maps. In experiment 1, participants lifted inverted size-weight green objects, normal size-weight black objects, or both. Load force recordings taken during the object lifts were used to examine predictions of weight during object manipulation. Also, at the beginning of the experiment, and then on each subsequent day of testing, the SWI was tested to examine weight perception. We predicted that participants would experience interference between the green and black cubes when judging weight. This prediction is based on the idea that size-weight priors, used when judging weight, would be expected to be largely independent of color and that adaptation of such priors might also be expected to generalize across color. In contrast, we predicted that participants would show limited or no interference when lifting the trained objects, as reflected in the load forces applied during lifts of the green and black cubes. This latter prediction stems from the hypothesis that the sensorimotor system can quite quickly learn the weights of specific objects when these weights are not well predicted by priors. In a follow-up study, we examined whether cognitive resources are required for the learning and implementation of the arbitrary associations between object size, color, and weight to which participants were exposed. To test the role of cognitive resources during arbitrary associations between object size and weight, we examined participants' lift forces in a dual-task situation in which they had to perform an additional associative task involving an arbitrary visuomotor mapping. On the basis of the results of experiment 1, we predicted that when lifting objects in a dual-task paradigm lifting performance would be negatively affected by the presence of the arbitrary visuomotor association (AVA) task.

MATERIALS AND METHODS

Participants

The Queen's University General Research Ethics Board approved all experimental procedures. Twenty-seven undergraduates (17 men, 10 women; mean age = 20 yr) participated in experiment 1 and performed either in the experimental condition GB6 (3 green objects and 3 black objects, n = 9) or in one of two control conditions, GC3 (3 green objects, n = 10) and BC3 (3 black objects, n = 8). Seventeen undergraduate students (7 men, 10 women; mean age 21 yr) participated in experiment 2 and performed in one of two conditions (interference, n = 8; no interference, n = 9). All participants received compensation at the rate of $10/h for their participation or credit toward an introductory psychology course and provided written informed consent.

Apparatus

Participants sat at a table and lifted objects either off or onto a tabletop platform that was instrumented with four force/torque sensors (Nano 17 F/T sensors, ATI Industrial Automation, Garner, NC) (see Fig. 1A). Each of the force sensors was capped with flat circular disks. These force sensors allowed for the precise measurement of the vertical load force applied to each object during lifting, up until the point at which the object was lifted off the supporting disk. The upper surface of the platform was located 0.5 cm below the sensor surfaces, with the sensor surfaces protruding through four circular holes (diameter of 3.4 cm) in the platform. An LCD data projector was used to provide participants with instructions during the experiment, which projected its image down onto the tabletop via a 45° mirror.

Fig. 1.

Experimental apparatus and stimuli. A: while seated, participants lifted and replaced 1 of 2 objects located on force sensors embedded in a platform. A data projector, located above the participant, was used to indicate which object to lift on a given trial. B: relationship between volume and mass for the 3 size-weight inverted green cubes, the 3 normally weighted black cubes, and the small and large equally weighted green and black size-weight illusion stimuli. C: 2 examples of the arbitrary visuomotor association (AVA) task instructions used in experiment 2. The instructions specify which direction the participant should translate each object in the horizontal plane after lifting it vertically. For example, in the first set of instructions, the large black cube (BL) should be moved away from the participant and to the left. In contrast, the medium-sized green cube (GM) should be moved toward the participant and to the right. D: load force functions from 2 lifts of a 7-N object. In one lift (gray curves) the initial increase in load force undershot object weight, and in the other lift (black curves) the initial increase in load force accurately reached the weight.

Materials

Different groups of participants lifted three inverted size-weight green cubes, three normal size-weight black cubes, or all six objects (see Fig. 1B). The sizes of the small, midsized, and large black and green cubes were 70, 275, and 457 cm³, respectively. The small, midsized, and large black cubes were all constructed from the plastic Delrin (1.356 g/cm³) and weighed 95, 429, and 714 g, respectively. The small, midsized, and large green cubes were constructed from lead (10.263 g/cm³), Delrin, and balsa wood (0.118 g/cm³) and weighed 714, 429, and 54 g, respectively. All three green cubes were covered with a thin sheet of balsa wood and thus had the same outer appearance, while the black cubes' outer appearance was consistent with Delrin, resulting in stimuli being distinct in both color and apparent material. To test the SWI, we used equally weighted (390 g) large and small black cubes and equally weighted small and large green cubes. All four cubes weighed 390 g, and the small and large cubes in both pairs were equal in size to the small and large cubes used for repeated lifting.

In experiment 2, participants were required to perform an out-and-back movement, in the horizontal plane, after lifting each object and before placing it back down. The direction of this secondary movement depended on the color and size of the object being lifted, and the mapping between the movement direction and the color and size of the object was specified via instructions displayed on the tabletop (see Fig. 1C). The aim was to determine whether this AVA task interfered with the scaling of lift forces for the black and green cubes, which may also involve remembering and implementing an AVA.

Procedure

Experiment 1.

Participants completed 10 lifting sessions on separate days. In most cases, successive sessions were run on successive days. However, there could be a gap of 1 or 2 days between sessions (e.g., over a weekend). Participants in the experimental (green-black) condition lifted the three green cubes and the three black cubes, with each of the objects being lifted 40 times per day. Participants in the green control condition lifted only the three green cubes, and participants in the black control condition lifted only the black cubes. In the control conditions, each block was lifted 80 times per day. Thus all three groups of participants performed 240 lifts a day—40 lifts of six blocks in the experimental condition or 80 lifts of three blocks in the control conditions.

Participants were required to lift each object ∼2.5 cm off the lifting surface, hold it stationary for ∼1 s, and then set it down in the location specified. In a given lift, an object could be moved either from the tabletop to a sensor or from a sensor to the tabletop. To instruct participants to place a particular object on a particular sensor, an image was projected onto the location where the object was to be set down. The size of the presented image corresponded to the size of the target object, and the image was either filled or an outline of the object to indicate whether the green or black variant was to be lifted, respectively. To instruct participants to lift a particular object from a sensor, a small circle was projected onto the center of the object. Forces from the force sensors were used to determine when each instructed task was completed. The object to be lifted in any given trial was randomly selected, with the constraints that 1) the same object could not be placed on a sensor and then immediately lifted off the sensor during the following trial and 2) all objects were lifted an equal number of times within a lifting session. In trials in which an object was placed on a sensor, the sensor was randomly selected from among the unoccupied sensors. On average, two objects were placed on the sensors at any time.

In the green-black condition, the SWI was tested, for both the black and green cubes, on the first day before any lifting had occurred (which we refer to as day 0) and on days 1–10 after all lifting had occurred. In the green and black control conditions, the SWI was also tested for both the black and green cubes before lifting on the first day and after lifting on day 10. However, on days 1–10, participants in the green and black conditions were only tested on the SWI with the green and black cubes, respectively. To test the SWI, the small and large equally weighted cubes (green or black) were placed on the two sensors closest to the participant while the participant closed his/her eyes. To measure the strength and direction of the SWI, an absolute-magnitude-estimation procedure (Flanagan and Beltzner 2000; Zwislocki and Goodman 1980) was used, a method that results in data with ratio properties that are similar to standard forms of technical measurement (Meilgaard et al. 2006). Participants lifted each cube once. After each lift, they were asked to assign a numerical value representing the weight of the object. The procedure was explained to participants ahead of time, and participants were told they could use any numbers they wished. No range was provided. The order in which the large and small cubes were lifted was randomized across participants and days. In addition, when the SWI was tested for both the green and black cubes, the order was randomized. No instructions regarding object color and weight were provided to the participants.

Experiment 2.

The general procedure in experiment 2 was the same as the experimental condition (GB6) of experiment 1. Two groups of participants were tested. Both groups completed a single lifting session in which they lifted the three green and three black objects 40 times each for a total of 240 lifts. Participants in the interference condition were required to perform the AVA task (described above) after each lift. A new AVA instruction image was presented every 10 trials. These instructions were displayed for as long as was necessary for the participant to memorize the new relationship between movement and block features (typically ∼30 s). The instruction image was not available to the participant while the objects were being lifted; therefore, participants were required to keep these arbitrary mappings in working memory. Participants in the no-interference condition simply lifted the objects (as in experiment 1).

Data Analysis

Vertical forces from each of the force sensors were sampled at 500 Hz. The raw force signals were low-pass filtered with a fourth-order, zero-phase lag Butterworth filter with a cutoff frequency of 14 Hz. A signal representing the vertical force applied to the object by the hand (i.e., the vertical lifting or load force) was obtained by subtracting the vertical force corresponding to the weight of the object when fully supported by the force sensor from the recorded signal. This processed signal was then differentiated with respect to time with a first-order central difference equation to obtain the rate of change in load force, or load force rate.

When lifting an object, the weight of which is well predicted, just off a surface (as in the present study), people generate a bell-shaped load force rate profile such that the load force rate is small when the object lifts off. Because people tend to lift objects of varying weight in roughly the same amount of time, the initial peak rate of load force rate can provide an index of predicted object weight (Flanagan and Beltzner 2000; Johansson and Westling 1988). The load force at the time of the initial peak load force rate also provides an index of expected weight, with the value being approximately half the weight of the object (Flanagan et al. 2008). When the object is heavier than expected, the initial peak load force rate (associated with the initial increase in load force) occurs prior to liftoff. However, when the object is lighter than expected, liftoff may occur before the peak load force rate. Because in the present study we only measured load force prior to liftoff, we cannot use the measures in cases when the object may be lighter than expected. For this reason, we focused our analysis on the heavier objects (i.e., the small green and large black cubes), which were either equal to or heavier than participants' weight expectations. In previous work, we have shown that when simultaneously lifting unusually light and unusually heavy objects, the rate of lift force adaptation follows a similar time course for all objects (Flanagan and Beltzner 2000). Thus, by examining load forces applied to the small green and large black cubes, we could assess sensorimotor adaptation in general. For each lift, we first determined the onset of the load phase, during which load force is increased prior to liftoff, as the time when load force first exceeded 0.5 N. We then determined the load force at the time of the initial peak in the load force rate (see Fig. 1D). To assess the direction and strength of the SWI, a signed percent change score was used. Specifically, we determined the percent increase from the smallest to the largest numerical values provided by participants for the equally weighted large and small cubes and assigned a positive sign to this number if the small object was perceived as heavier (the regular SWI) and a negative sign to this number if the larger object was perceived as heavier (an inverse SWI).

To compare lifting force adaptation both across and within groups, ANOVAs were utilized. Because of the variability observed within the GB6 condition, additional comparisons were made across cubes (on days 5 and 7) to ensure sampling throughout all 10 days of lifting without having to test on each day to avoid unnecessary statistical comparisons.

RESULTS

Experiment 1

Changes in weight perception across days.

Figure 2 shows the strength and direction of the SWI as a function of days of lifting for the green and black control conditions (GC3 and BC3) (Fig. 2A) and the green-black experimental condition (GB6) (Fig. 2B). To test for the presence of a SWI, one-sample t-tests (comparing the magnitude of the illusion to 0 on each day) were used, with a Holm-Bonferroni correction to control the family-wise error rate. Each control condition was tested separately (GC3 and BC3). To directly compare the magnitude of the SWI for both the green and black cubes, Holm-Bonferroni-corrected paired-samples t-tests were used to directly compare the strength of the illusion on day 0 with that observed on day 10. For the experimental condition (GB6), a similar method statistical approach was taken. First, the presence of an illusion was tested with Holm-Bonferroni one-sample t-tests on each day for the two sets of blocks. Next, paired-samples, Holm-Bonferroni-corrected t-tests compared the signed percent change scores of the green and black blocks on each day. Finally, a comparison of the magnitude of the illusion on day 0 and day 10 was made by using paired-samples t-tests for both the black and green cubes. First consider the green control condition. The solid gray line in Fig. 2A represents the mean signed percent change score for the equally weighted green cubes—tested at the start of day 1 (day 0) and after lifting on all 10 days—and the filled black circles represent the mean scores for the equally weighted black cubes—tested on days 0 and 10. The gray circles along the bottom of the figure indicate whether the illusion was significantly different from zero (filled circles) or not (open circles) based on t-tests (P < 0.05; n = 10). Participants began the experiment with a positive (i.e., normal) SWI illusion for both the green cubes and the black cubes, in both cases judging the smaller cube to be ∼175% heavier than the large cube [t(9) = 6.82 P < 0.001; t(9) = 7.85, P < 0.001, respectively]. After lifting on day 1, the SWI for the green cubes remained (i.e., the mean percent change score was slightly but significantly greater than zero) [t(9) = 4.79, P < 0.001]; however, no SWI was observed on days 2 and 3. Beginning on day 4, and lasting for the remainder of the experiment, an inverted SWI was observed (i.e., the mean signed percent change score was significantly less than 0%) such that participants, on average, judged the larger green cube to be heavier than the smaller green cube, an inverse SWI [t(9) = −3.96, −3.18, −9.09, −3.65, −4.94, −4.66, −4.43, all P < 0.05, for days 4–10, respectively]. This inversion of the SWI following extensive lifting is consistent with our previous work (Flanagan et al. 2008). After lifting on day 10, the mean signed percent change score for the black cubes was also significantly less than 0, and paired-samples t-tests revealed that the signed percent change score was significantly lower on day 10 compared with day 0 for both the green cubes and the black cubes [t(9) = −6.79, P < 0.001 and t(9) = −1.633, P = 0.027, respectively]. Thus changes in the perceived weights of the green cubes, associated with repeatedly lifting the green size-weight inverted cubes, generalized to the black cubes.

Fig. 2.

Perceptual results—experiment 1. A: signed percent change scores as a function of day for the black and green control cubes tested on days 0–10. The additional testing points within the control conditions are a result of testing the illusion with the alternate colored cubes. B: signed percent change scores as a function of day for the black and green experiment cubes. The height of each vertical bar and the shaded regions represent ±1 SE. Scores were calculated by determining the percent increase from the smallest to the largest numerical values and assigning a positive sign to this number if the small object was perceived as heavier and a negative sign if the larger object was perceived as heavier. The gray and black circles along the bottom indicate whether the green and black illusions were significantly different from 0 (filled circle), based on t-tests. A P value of 0.05 was considered statistically significant for all tests.

Consider next the black control condition shown in Fig. 2A. The solid black line represents the mean signed percent change score for the equally weighted black cubes—tested at the start of day 1 (day 0) and after lifting on all 10 days—and the filled gray circles represent the mean scores for the equally weighted green cubes—tested on days 0 and 10. The black circles along the bottom of the figure indicate whether the illusion was significantly different from zero (filled circles) or not (open circles) based on t-tests (P < 0.05; n = 8). Participants began the experiment with normal SWIs for both the black and green cubes, although the initial strength of the illusion was visibly weaker than that observed for participants in the green control condition [t(7) = 2.74, P = 0.029; t(7) = 4.904, P = 0.002]. A normal SWI with the black cubes remained throughout all days of lifting. However, the strength of the illusion tended to increase across days. Indeed, a paired-samples t-test revealed that the signed percent change scores were significantly higher on day 10 compared with day 0 for the black cubes [t(7) = −3.62, P = 0.008]. On day 10 after lifting signed percent change scores were significantly different from zero for the green cubes [t(7) = 5.68, P < 0.001; t(7) = 6.16, P < 0.001], and a paired-samples t-test revealed that the signed percent change score for the green cubes was significantly higher on day 10 compared with day 0 [t(7) = 2.85, P = 0.02]. Thus changes in the perceived weights of the black cubes, associated with repeatedly lifting the black normally weighted cubes, generalized to the green cubes.

Finally, consider the green-black experimental condition shown in Fig. 2B. The gray and black lines show the mean signed percent change scores for the green and black cubes, respectively, obtained on days 0–10. The gray and black circles along the bottom of the figure indicate whether the green and black illusions, respectively, were significantly different from zero (filled circles) or not (open circles) based on t-tests (all P < 0.05; n = 9). Direct comparisons were made between the signed percent change scores for the green and black condition at each day of lifting with paired-samples t-tests, and no significant differences were observed (all P > 0.05). Before any lifting (day 0), the normal SWI was observed for both the green and black cubes [t(8) = 6.13, P < 0.001; t(8) = 3.58, P = 0.008]. Thus the smaller green and black cubes were judged to be heavier than the larger but equally weighted green and black cubes at the onset of the experiment. The SWI for the black cubes remained on each day of lifting, and there was no significant difference in the strength of the illusion between day 0 and day 10 [t(8) = 1.74, P = 0.119]. For the green cubes, the signed percent change scores were not significantly higher than zero, once lifting had begun. There was a significant decrease in the signed percent change score of the green cubes when day 0 was compared to day 10 [t(8) = 5.53, P < 0.001]. For both the green and black cubes, the mean signed percent change score never dropped below 0% throughout the duration of the experiment, indicating that the normal SWI did not reverse with extensive exposure to the inverse size-weighted green cubes. To compare the effects of lifting the green and black cubes either together or in isolation, we performed independent-samples t-tests comparing the illusions on day 10 and day 0 in the green-black condition and the control conditions. On day 0, there were no significant differences between control and experimental groups (all P > 0.05). On day 10, for the green cubes the signed percent change score was significantly lower in the green control condition compared with the green-black condition [t(17) = −4.24, P < 0.001]. Similarly, for the black cubes the signed percent change score was significantly higher [t(15) = 4.95, P < 0.001] for the black control condition compared with the GB6 condition. Thus clear interference between the two sets of cubes was observed in terms of changes in the SWI.

Sensorimotor learning during lifting of inverted size-weight green cubes and normal black cubes.

To examine sensorimotor learning, we focused only on those lifts off the force sensor onto the tabletop of the heaviest objects, i.e., the large black cube and the small green cube. These objects were selected as they allow for the most accurate measurement of the initial peak in load force rate. When lifting the large, light cube, participants typically overestimated the weight, causing object liftoff to occur as load force was still increasing. In light of previous results (Flanagan et al. 2008), we assume that adaptation to the lighter-weight objects follows a time course similar to adaptation to the heavy objects. For each object and day, lifts off the sensor were binned into four successive blocks of 10 lifts for the control conditions and 5 lifts for the GB6 condition.

Figure 3A shows the load force and load force rate records for the first 10 lifts of the small heavy green cubes on days 1, 2, 3, and 10 of the green control condition (GC3), whereas Fig. 3B shows the first 10 lifts of the large heavy black cube on the same days of the black control condition (BC3), from typically performing subjects. In the green control condition, all but one participant were able to lift the small heavy cube in one smooth increase in load force, resulting in a single well-defined peak in load force rate. Similarly, in the black control condition, all but one participant were able to lift the large heavy cube with a single peak in load force rate. Figure 3C shows the load force rate records from the first five trials lifting the large black cube and the small green cube on days 1, 2, 3, and 10 of the green and black six-object condition (GB6). Qualitatively, three patterns of results emerged when looking at the load force traces, exemplified by the presented load force and load force rate records. Some participants were able to scale their lift forces accurately to the weights of both objects after a single day of lifting (n = 4), demonstrable in the single peak in load force rate (e.g., 1st row, Fig. 3C). Furthermore, these participants were able to maintain this performance throughout the duration of the experiment. In comparison, other participants showed poor scaling of their lift forces to the weights of both the green and black objects on day 1 but improved over the duration of the experiment, resulting in accurately scaled lift forces for both sets of blocks on day 10 (e.g., 3rd row, Fig. 3C) (n = 2). Finally, a subset of participants demonstrated a probing strategy of lift forces on day 1 and showed little improvement throughout the duration of the experiment (e.g., 4th row, Fig. 3C) (n = 3).

Fig. 3.

Load force rate during lifting—experiment 1. A and B: first 10 lifts of the heavy small green cubes of the green control condition (A) and the black control condition (B) on days 1, 2, 3, and 10. C: load force rate records from the first 5 trials lifting the large black cube (black traces) and the small green cubes (gray traces) on days 1, 2, 3, and 10. Each row represents data from an individual exemplar subject, identified by subject code, across days 1, 2, 3, and 10.

To demonstrate this variability in lifting performance, the load forces at the initial peak in load force rate for all subjects' lifts of the large black cube and the small green cube can be seen in Fig. 4, A–D. All participants in the BC3 and GC3 conditions (Fig. 4, A and C) were able to accurately scale load forces to the block weight at the end of day 10. The majority of participants in the GC3 condition showed load force rates indicative of improvement over the duration of the experiment, indicated by accurate scaling of lift forces to both the green and black cubes on day 10. However, not all participants were able to learn the appropriate lifting forces (Fig. 4, B and D).

Fig. 4.

Initial peak in load force rate—experiment 1. All lifts of the large black cube and the small green cube are plotted separately for each participant in the black (A) and green (C) control conditions. Lifts of the large black cube (B) and the small green cube (D) of the GB6 condition are also plotted. The mean lifting forces for the BC3 and GC3 conditions were significantly different (E); however, the GC3 condition (F) demonstrated increased variability and a nonsignificant difference between lifts of the large black cube and the small green cube. Shaded areas represent ±1 SE.

We next examined whether adaptation occurred for the green and black cubes in both control and experimental conditions. To accomplish this, two ANOVAs were conducted on the load forces at peak load force rate (LF@PeakLFR). First, a 2 (day 1 vs. day 10) × 2 (GC3 vs. BC3) between-subjects ANOVA was performed. No significant results were observed, suggesting that adaptation in both conditions likely occurred on the first day of lifting. To examine adaptation within the GB6 condition, a 2 (day 1 vs. day 10) × 2 (green cube vs. black cube) repeated-measures ANOVA was performed. Main effects of both day [F(1,8) = 10.38, P = 0.012] and block [F(1,8) = 11.01, P = 0.011] were observed. Specifically, participants lifted the blocks with greater force on day 10 compared with day 1 (2.90 N vs. 2.28 N) and had a higher lifting force when lifting the green cubes than when lifting the black cubes (2.87 N vs. 2.38 N). To confirm that adaptation within the GC3 and BC3 conditions occurred within the first day of lifting (as suggested by the ANOVA), we examined the median value of the LF@PeakLFR for each successive block of five lifts of each object for the GC3 and BC3 conditions. This resulted in eight blocks of trials for each of the GC3 and BC3 conditions. A trial block (time block 1 vs. time block 8) × experimental condition (GC3 vs. BC3) between-subjects ANOVA found a significant time × condition interaction [F(1,16) = 9.21, P = 0.008]. Decomposing the interaction found a significant increase in lifting forces within the GC3 condition over time (mean difference = 1.18, P = 0.002) but no difference in lifting forces within the BC3 condition as a function of time (P > 0.10) (Fig. 5). Finally, we examined whether the mean lifting forces used to lift the large black and small green cubes are influenced by combining the two separate families of objects used in each of the control conditions (GC3 and BC3) into the green and black experimental condition. To accomplish this, we compared the lifting forces used to lift the small green cube on day 10 of the GC3 condition (gray trace, Fig. 4E) with the lifting forces used to lift the small green cube on day 10 of the GB6 condition (gray trace, Fig. 4F), using Holm-Bonferroni-corrected independent-samples t-tests. No significant difference was found between the means [t(17) = 1.045, P = 0.311]. When comparing the lifting forces used to lift the large black cube on day 10 of the BC3 condition (black trace, Fig. 4E) to the GB6 condition (black trace, Fig. 4F), Levene's test for equality of variances found a significant difference between the variances of the BC3 and GB6 conditions, as would be expected from the individual subject plots of Fig. 4 [F(1,15) = 7.982, P = 0.013]. An independent-samples t-test with equal variances not assumed found no difference between means [t(15) = 0.658, P = 0.52].

Fig. 5.

Initial peak in load force rate: experiment 1, day 1. First peak in load force rate for the large black and small green cubes. The mean peak in load force rate of all subjects is shown for the first 5 and last 5 trials of each block. A significant increase in lifting force was observed for the small green cubes, indicating sensorimotor adaptation over the first day of lifting. Error bars represent ±1 SE. *P value of <0.05, corrected, considered a statistically significant difference between means.

Arbitrary Visuomotor Association Task Impairs Lift Force Scaling (Experiment 2)

Lifts off the sensor and onto the tabletop were divided into four blocks of five lifts per object (20 lifts per object). As each object could be lifted off a sensor or the tabletop, the number of scored lifts is equal to half the total number of lifts participants engaged in (40 lifts per object). The analysis once again focused on lifting performance during trials in which the large black cube or the small green cube was being lifted. Lifting performance was examined at two different time points—before any lifting was completed and at the end of the experiment after all 240 lifts were complete. There were no individual differences in the rate at which participants learned the visuomotor association task or in the accuracy of completing the task.

To determine the effects of the interfering visuomotor association task on lifting performance, a repeated-measures ANOVA on the load force at initial peak load force rate was performed with object (green vs. black) and trial block (1 vs. 4) as within-subjects factors and AVA task (present vs. absent) as a between-subjects factor (see Fig. 6). A significant main effect of trial block [F(1,15) = 6.930, P = 0.019] was found, revealing that participants lifted the black (Fig. 6A) and green (Fig. 6B) cubes with greater initial peak in load force rate at the end of the experiment compared with the beginning of the experiment. Importantly, a significant effect of the between-subjects manipulation of the AVA task was observed. Participants who had to concurrently perform both the object lifting and the AVA task were substantially impaired in their lifting behavior [F(1,15) = 8.548, P = 0.010]. This suggests that the AVA task interfered with sensorimotor learning and/or utilization of different families of object weight, whereas participants who did not perform the AVA task showed significantly more efficient lifting behavior. Finally, there was a significant trial block × AVA task interaction, indicating that participants who completed lifting without the AVA task demonstrated a greater increase in lifting forces across time blocks than those participants who lifted while performing an AVA task [F(1,15) = 13.049, P = 0.003].

Fig. 6.

Initial peak in load force rate: experiment 2. First peak in load force rate for the large black (A, black) and small green (B, light gray) cubes. The mean peak in load force rate of all subjects is shown for the first and last blocks for the no interference (filled) and interference (open) conditions. Error bars represent ±1 SE.

DISCUSSION

The present study examined the roles of both long-term priors and sensorimotor memory when participants repeatedly lifted, over multiple days, a set of objects that included black cubes with a normal size-weight mapping and green cubes with an inverted size-weight mapping. Experiment 1 demonstrated that when lifting two competing size-weight maps there was considerable interference at the perceptual level. Specifically, there was little evidence of an inversion of the SWI for the green cubes in the GB6 condition, in stark contrast to what was observed in the GC3 condition. This result suggests that at the perceptual level priors for the green and black cubes were not adjusted independently. When examining sensorimotor memory, approximately half of participants in the GB6 condition rapidly learned to scale lift forces appropriately whereas the remaining half of participants demonstrated poor scaling of lifting forces throughout the duration of the experiment, suggesting that individual differences in available cognitive resources may be involved. A follow-up study directly assessed the role of cognitive resources in the formation and utilization of sensorimotor memory during an object lifting task and found that the use of sensorimotor memory was significantly impaired during a dual-task condition.

Recent studies suggest that weight prediction can be based on short-term information obtained from previous lifting experience, known as sensorimotor memory, to adapt lifting forces (Cole 2008; Flanagan et al. 2008). For instance, Cole (2008) asked participants to lift a large brown bottle 20 times. After a brief delay, participants lifted a slightly smaller bottle of similar visual appearance. Even though participants were explicitly unaware of the slight reduction in bottle size, they were able to scale their lifting forces for the smaller bottle. This result suggests that people combine sensorimotor memory of object density with an analysis of object size to make weight predictions. Flanagan et al. (2008) found further support that sensorimotor memory is engaged when lifting objects, while also providing evidence that distinct sources of information are engaged when lifting objects vs. when judging their weights. On the basis of these results, we predicted that interference between two opposing size-weight priors could occur when perceptually judging object weight, a prediction that was verified in the results of experiment 1. We also hypothesized that sensorimotor memory must be fine-tuned to specific sets of objects and, as such, participants should experience only marginal interference in lifting forces when lifting objects belonging to two opposing size-weight families. The presented lift forces of experiment 1 partially support this hypothesis. Many of the nine participants in the GB6 condition were able to consistently lift in a smooth, efficient manner resulting in a single peak in load force rate during lifting on the final day of the experiment. However, the increased variability observed in both the individual participant data (Fig. 4, B and D) and the overall participant means (Fig. 4F) provides evidence that not all participants are able to learn these two sets of blocks simultaneously. Another interesting, and perhaps counterintuitive, result when participants were lifting the inversely weighted green cubes and normally mapped black cubes is that lifting performance of both the green and black cubes was similar within participants. A valid hypothesis would have been that participants would have been very good at predicting the weights of the normal size-weight black cubes and would only show difficulty with the novel inversely weighted green cubes. This finding suggests that participants were attempting to learn the green and black objects as an entire set and attempting to learn the mapping of all objects at once, using the conjunction of color and size to predict weight.

The perceptual results obtained with the SWI provide evidence that the priors formed throughout the multiple days of lifting in the present experiment are represented at a very broad level. We believe this result reflects the relative unimportance of object color in weight perception, and therefore when lifting objects with varying size-weight relationships primarily indicated by object color there is considerable interference between competing size-weight mappings. This hypothesis is supported by previous research showing that, although there are color-weight illusions (with darker colors being judged as heavier than lighter colors) (De Camp 1917; Walker et al. 2010; Warden and Flynn 1926), this illusion is much smaller than those based on size (Charpentier 1891) or apparent materials (Buckingham et al. 2009; Seashore 1899). Further support comes from studies demonstrating that although it is possible to learn arbitrary links between object color and object weight these links are not formed as readily as those incorporating features such as size (Li et al. 2009). Future studies could examine whether such broad generalizations are formed when size-weight relationships are linked to object properties more naturally predictive of object mass.

Data collected from experiment 1 indicated that not all participants learned to scale lift forces accurately when lifting multiple objects belonging to families with different size-weight maps. This result suggested that the sensorimotor system is capable of learning two opposing size-weight maps without interference, but not in all participants. One plausible explanation for these individual differences in lifting performance is that some participants had difficulty learning and/or using the arbitrary mapping between size and color and weight. Previous research has demonstrated that the ability to use arbitrary visual cues for predictive control of fingertip forces during object lifting is affected by natural aging, likely as a result of failures in associative learning (Cole and Rotella 2002). To test the role of associative learning directly, we examined lift forces in a dual-task situation in which participants had to respond to an arbitrary visuomotor mapping while lifting the green and black cubes (experiment 2). We found that participants in the AVA condition scaled their lifting forces less accurately during the dual-task condition compared with participants who did not have to perform this additional visuomotor association task. This suggests that the AVA task interferes with learning and/or utilizing competing size-weight families of objects. In other words, the generation of correct lifting forces, at least during conditions of conflicting size-weight maps, is a cognitive process that requires attentional resources. This result fits nicely with recent research suggesting that increased cognitive load during lifting objects with arbitrary associations between object weight and object color has a detrimental effect on anticipatory grip force scaling (Li et al. 2009). Interestingly, Li et al. (2009) found that even though trial-to-trial performance metrics were negatively impacted when participants had to perform a simultaneous memory task, associative memory was still formed (although impaired) and participants could utilize this latent learning in future interactions with the arbitrary color-weight stimuli.

On the basis of the present results, it is unclear whether force adaptation when lifting two series of blocks with unique size-weight relationships is a result of learning two size-weight mappings that are then utilized to predict object weight or if participants learn to associate a particular force to each separate object. We believe the latter case more plausible. In particular, with the present stimuli, one would have to learn a “negative” density with the inversely weighted blocks in order to make accurate weight predictions. This seems like an unlikely scenario, and therefore we believe the learning of particular forces associated with each block to be the more parsimonious explanation. A third possibility is that participants remember specific densities associated with the cubes, with one density learned for the black normally weighted blocks and then applied to each of the three sizes and three unique densities for the inversely weighted green cubes. As the present study was not designed to dissociate these alternatives, future work is required to determine which explanation is correct.

One difference between the GB6 and the GC3 and BC3 conditions was in the number of times each object was lifted. Specifically, in the control conditions each object was lifted 80 times and in the experimental GB6 condition each object was lifted 40 times. It is conceivable that the extra exposure participants received to each of the individual blocks was partially responsible for the pattern of results that we observed. Although we are not able to rule out this potential alternative interpretation, when examining the rate at which both the perceptual and lifting forces adapted in the control conditions the level of exposure within the experimental condition should have been sufficient to foster changes in these measures. Furthermore, previous studies examining the role of lifting experience on size-weight priors (Flanagan et al. 2008) have shown effects with as few as 20 lifts of each object, leading us to believe that the level of exposure to the blocks is not playing a significant role.

A surprising finding within the perceptual data was the difference in the SWI between the green and black control conditions. As both groups of participants likely had similar experiences before the experiment began, a difference in the magnitude of the SWI between the green and black cubes at the onset of the experiment was unexpected. Although previous research has shown that object color can have an impact on the perception of weight, with darker colors being perceived as heavier than lighter colors (De Camp 1917; Walker et al. 2010; Warden and Flynn 1926), this research does not provide an adequate explanation as to why the magnitude of the illusion would be larger in one condition than another. Rather, the SWI is highly variable, and the apparent difference between groups at the onset is the result of this variability. Specifically, as both groups should have begun the experiment with equivalent illusion magnitudes, and the BC3 group reaches the magnitude of the GC3 group at training onset, we believe that the BC3 group reported an abnormally low perceptual illusion on day 0. This is further supported by the fact that the stimuli in the BC3 condition have a normal density, and therefore lifting experience should not alter the magnitude of the illusion, with changes over time representing a regression to the mean. Importantly, despite this variability, the magnitude of the illusion decreased, to the point of reversal, in the GC3 condition, a pattern that was not replicated when the green cubes and black cubes were lifted in the GB6 condition. This latter result suggests that there is interference between priors, used in judging weight, developed for the green and black cubes. However, we cannot be sure whether participants learn two priors for the green cubes and the black cubes, respectively, or whether they learn priors for each object.

In summary, the present study demonstrates significant individual differences in the formation of sensorimotor memories and demonstrates that priors related to the objects being lifted are not independently adjusted as a result of lifting experience. Instead, the SWI data provide evidence that the priors formed throughout the experiment are represented at a very broad level (experiment 1). In a follow-up experiment, we examined the detrimental effects of completing a secondary task on learning the appropriate lifting forces when lifting novel objects. This impairment suggests that sensorimotor memory requires substantial cognitive resources. When taken together, these results provide evidence that working memory and/or attentional resources may be required to successfully retrieve and utilize size-weight maps and that such size-weight information is broadly categorized within memory used for predicting object weight. Future research, therefore, would be well served by examining the precise nature of the cognitive resources required for efficient lifting behavior, as well as the timing in relation to the lifting task.

GRANTS

This work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC), the Canadian Institutes of Health Research, the Swedish Research Council (Project 08667), and the Strategic Research Program in Neuroscience at the Karolinska Institute, Stockholm, Sweden. L. A. Baugh was supported by an NSERC Collaborative Research and Training Experience (CREATE) Training Grant.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

L.A.B., A.Y., R.S.J., and J.R.F. conception and design of research; L.A.B. and A.Y. performed experiments; L.A.B., A.Y., and J.R.F. analyzed data; L.A.B., A.Y., R.S.J., and J.R.F. interpreted results of experiments; L.A.B., A.Y., and J.R.F. prepared figures; L.A.B., A.Y., and J.R.F. drafted manuscript; L.A.B., A.Y., R.S.J., and J.R.F. edited and revised manuscript; L.A.B., A.Y., R.S.J., and J.R.F. approved final version of manuscript.