Energy margins in dynamic object manipulation

Abstract

Many tasks require humans to manipulate dynamically complex objects and maintain appropriate safety margins, such as placing a cup of coffee on a coaster without spilling. This study examined how humans learn such safety margins and how they are shaped by task constraints and changing variability with improved skill. Eighteen subjects used a manipulandum to transport a shallow virtual cup containing a ball to a target without losing the ball. Half were to complete the cup transit in a comfortable target time of 2 s (a redundant task with infinitely many equivalent solutions), and the other half in minimum time (a nonredundant task with one explicit cost to optimize). The safety margin was defined as the ball energy relative to escape, i.e., as an energy margin. The first hypothesis, that subjects converge to a single strategy in the minimum-time task but choose different strategies in the less constrained target-time task, was not supported. Both groups developed individualized strategies with practice. The second hypothesis, that subjects decrease safety margins in the minimum-time task but increase them in the target-time task, was supported. The third hypothesis, that in both tasks subjects modulate energy margins according to their execution variability, was partially supported. In the target-time group, changes in energy margins correlated positively with changes in execution variability; in the minimum-time group, such a relation was observed only at the end of practice, not across practice. These results show that when learning a redundant object manipulation task, most subjects increase their safety margins and shape their movement strategies in accordance with their changing variability.

most experimental paradigms in computational neuroscience have used highly constrained tasks such as rapidly reaching to a point target (Bhushan and Shadmehr 1999; Meyer et al. 1988; Schmidt et al. 1979). The underlying rationale is that insight can be gained into how the brain controls movement by pushing the sensorimotor control system to its limit in terms of spatial and/or temporal accuracy. Although this is a useful approach, tasks performed in everyday life rarely match this scenario, as is evident when placing a cup of coffee onto a table. This action is not performed rapidly, nor is the accuracy of cup placement critical. What matters is that the cup reaches the table without coffee being spilled. This task is redundant, with an infinite number of movement solutions. This redundancy raises many questions: Are all movement strategies or “solutions” equally valid? If not, what preferences do humans display, or what “cost functions” do they develop with practice? How consistently do individuals choose one or a subset of solutions? This study addresses these questions by introducing a new experimental task that emulates transporting a cup of coffee.

When redundancy is discussed, it is usually with respect to the overdetermined nature of the human musculoskeletal system (Bernstein 1967), e.g., most joints are crossed by more muscles than needed to actuate the joint's degrees of freedom (however, see Loeb 2000). Similarly, a task can be redundant if it affords multiple ways to be accomplished. If the goal is to transport an object in an unspecified movement time or to an approximate location, an infinite number of equally valid solutions are available. In contrast, when transporting an object as fast as possible, redundancy is eliminated due to the optimization of one explicit performance criterion, assuming fixed constraints on actuation. Here we refer to optimality at the task level, i.e., as a “goal post” that humans strive to obtain. Although studies have investigated redundant tasks, such as pointing to a line (Berret et al. 2011; Diedrichsen et al. 2010), a plane (Schlerf and Ivry 2011), or throwing a ball at a target (Cohen and Sternad 2009; Sternad et al. 2011), they have focused on performance in terms of the end-state goal with a defined end-state error. In tasks such as transporting a cup of coffee, performance is constrained along the entire movement trajectory, because a safety margin must be maintained to prevent spilling coffee.

The modulation of continuous safety margins has been studied in relation to picking up, transporting, or oscillating rigid objects, where humans have to maintain a grip force exceeding the slipping threshold to prevent dropping the object (Danion and Sarlegna 2007; Flanagan and Lolley 2001; Forssberg et al. 1991; Johansson and Cole 1994; Sarlegna et al. 2010). For rigid objects there is a straightforward relation between object acceleration and the load force that must be overcome to avoid slipping. Consequently, humans quickly adapt their grip force in response to a novel load within a single movement (Johansson and Westling 1988). However, an object like a cup filled with coffee has more complex dynamics, and the risk of spilling requires a different and less intuitive safety margin due to the moving liquid. In addition, the control of the safety margin is coupled to the object motion in a nonlinear fashion, i.e., the safety margin cannot be controlled independently from object kinematics. It is safe to assume that this more complicated safety margin will require more than one movement to learn. How humans establish safety margins while at the same time trying to satisfy end-state goals in a redundant task is an open question.

A key factor that may modulate how individuals discover and use safety margins in a redundant task is motor variability, which arises in part due to noise in the neuromuscular system (Faisal et al. 2008). Previous studies have shown that humans are sensitive to the effects of variability and seek solutions that are tolerant to error and noise (Chu WT, Sternad D, Sanger TD, unpublished observations; Harris and Wolpert 1998; Sternad et al. 2011). This predicts that individuals with greater variability will utilize larger safety margins, since task failure may result if movement errors or unforeseen perturbations occur. One way to test this prediction is to compare individuals' choices of safety margin with their variability: those with greater variability should choose larger safety margins, i.e., there should be a correlation. An even stronger test is to examine changes across practice within an individual and assess how safety margins are modulated with practice. If humans are sensitive to their variability, then decreases in trial-to-trial variability should be accompanied by smaller increases in safety margins, and vice versa.

The present study developed an experimental paradigm for a complex task that presents redundancy and has continuous safety margin constraints, similar to the real-world task of transporting a cup of coffee. These features were reproduced by having participants transport a virtual ball in a cup to a spatial target without letting the ball escape from the cup. To investigate the effects of task redundancy on performance, two different task conditions were compared. In a redundant version of the task, subjects were asked to transport the ball and cup in a comfortable time of 2 s without letting the ball escape. On the basis of these instructions, there is redundancy, because subjects could use a variety of movement strategies to place the cup onto the target location. In addition, the constraint of not losing the ball is not a discrete criterion, but rather an inequality. To create a nonredundant task for comparison, a second group of subjects was instructed to complete the cup transit in minimum time, a task goal that has a single theoretical optimum given physiological constraints on action.

The first aim was to investigate how task redundancy affects performance variability. We expected that, as is typical in motor learning, subjects in both groups would improve end-state performance while also decreasing trial-to-trial variability. However, we anticipated that subjects in the target-time group would choose different spatiotemporal solutions due to task redundancy, whereas those in the minimum-time group would converge to the same strategy (hypothesis 1).

The second aim was to investigate how manipulation of task redundancy affects safety margins. As detailed below, the safety margin was quantified in terms of ball energy relative to escape, i.e., as an energy margin. We expected that, with practice, subjects performing the nonredundant minimum-time task would shift from a conservative strategy to one that was more risky. Specifically, safety margins should decrease as subjects become familiar with the task dynamics and make faster movements (hypothesis 2a). In contrast, in the target-time task, subjects should take advantage of redundancy and seek strategies with greater safety margins (hypothesis 2b).

The third aim was to investigate the relation between variability and safety margins. If subjects are sensitive to their own variability and seek error-tolerant performance, then safety margins should change in concert with changes in variability. Subjects who exhibit large decreases in variability should be under less pressure to seek safe movement strategies compared with those exhibiting only marginal decreases in variability. If this is a general tendency, then there should be a positive correlation between safety margins and variability at the end of practice in both redundant and nonredundant tasks (hypothesis 3a). A more individualized expectation is that within each individual, we should see that safety margins are adapted to their changing variability (hypothesis 3b).

METHODS

Model of the Task

Task dynamics.

The model task emulated the everyday activity of transporting a cup of coffee, simplified to transporting an arc (the cup) containing a ball (Fig. 1). The motion of the ball within the cup was dictated by pendular dynamics. The cup was constrained to move along one horizontal dimension and had a semicircular shape, which was equivalent to the ball's pendular path. This system is underactuated, because control inputs can only be applied to the cup and not to the ball. The equations of motion were the same as for a cart (cup) and pendulum (ball) system (Hinrichsen and Pritchard 2005) and were derived using Euler-Lagrangian mechanics (for more detail, see appendix). The first equation of motion is

$$\left(m+M\right)\ddot{x}=F_{\text{A}}+F_{\text{B}},$$ (1)

and the second is

$$\ddot{\theta }=\frac{\ddot{x}}{\ell }\text{cos}\theta -\frac{g}{\ell }\text{sin}\theta ,$$ (2)

where M is the cup mass, m is the ball mass, θ, θ̇, and θ̈ are the ball angle, angular velocity, and angular acceleration, respectively, ẍ is the cup's horizontal acceleration, ℓ is the pendulum length, g is gravitational acceleration, F_A is an external horizontal force applied to the cup, and F_B is the horizontal reaction force of the ball on the cup, given by

$$F_{\text{B}}=m\ell \ddot{\theta }\cos \theta -m\ell {\dot{\theta }}^2\sin \theta .$$ (3)

A positive cup displacement is defined when the cup moves toward the right, and a positive increase in the pendulum angle is defined when the pendulum swings in a clockwise direction. Although a variety of models could have been used (e.g., a ball that slides or rolls inside the cup surface), the cart-and-pendulum model was chosen because it is a well-studied model task in physics and engineering. Also, although a veridical model of the full fluid dynamics of a cup of coffee would be interesting, it was not necessary for the purposes of this study.

Fig. 1.

Progression from actual task (A) through the conceptual model (B) to the mathematical model (C) and its implementation in a virtual environment (D). In the mathematical model, x is the cup position, M is the cup mass, m is the ball mass, θ is the ball angle, θ_ESC is the escape angle, and ℓ is the pendulum length.

Energetics.

The ball-and-cup system can be characterized in the inertial reference frame by computing the net kinetic energy, KE, of the ball and cup as

$$KE=\frac{1}{2}M{\dot{x}}^2+\frac{1}{2}m\left({\dot{x}}^2-2\ell \dot{x}\dot{\theta }\cos \theta +{\ell }^2{\dot{\theta }}^2\right),$$ (4)

where ẋ is the cup velocity, and the potential energy, PE, as

$$PE=mg\ell \left(1-\cos \theta \right),$$ (5)

which together give the total ball and cup system energy, TE,

$$TE=PE+KE.$$ (6)

The movement of the ball within the cup can be characterized in terms of the kinetic energy of the ball, KE_BALL, computed in the local reference frame of the cup as

$$K{E}_{\text{BALL}}=\frac{1}{2}m{\left(\ell \dot{\theta }\right)}^2.$$ (7)

The gravitational potential energy of the ball, PE_BALL, is the same as in the inertial reference frame, i.e., PE_BALL = PE.

Safety margin.

The cup imposes an angular constraint on the ball. The ball angle θ should not exceed the maximum angle subtended by the cup, termed the escape angle θ_ESC, because this is equivalent to losing the ball. This safety constraint can be quantified as the height of the ball relative to the cup rim, i.e., PE_BALL relative to the maximum allowable potential energy PE_MAX, which occurs when the ball is at the cup rim (Fig. 2A), i.e., when θ = θ_ESC. This maximum potential energy PE_MAX is given by

$$P{E}_{\text{MAX}}=mg\ell \left(1-\cos {\theta }_{\text{ESC}}\right),$$ (8)

and a safety margin can be defined in terms of a potential energy margin EM_PE, given by

$$E{M}_{\text{PE}}=\left(P{E}_{\text{MAX}}-P{E}_{\text{BALL}}\right)/P{E}_{\text{MAX}}.$$ (9)

If EM_PE is positive, then the ball is in the cup; if it is negative, then the ball has escaped. A smaller EM_PE value corresponds to a smaller safety margin.

Fig. 2.

Illustration of different safety margin calculations. The safety margin is quantified in terms of a ball energy margin EM relative to ball escape. Three variants are shown: a potential energy margin EM_PE that only depends on ball height relative to the rim height (A), EM_PE+KE, which includes both potential (PE) and kinetic energy (KE) of the ball (B), and EM_(PE+KE+PSE), which accounts for ball potential and kinetic energy and subject inputs via cup acceleration (C). PSE, pseudo energy.

A significant limitation is that EM_PE ignores whether and how fast the ball is moving within the cup. Even if the ball is at the bottom of the cup (maximum EM_PE), it may have a large velocity and might escape in the immediate future. To account for this, the energy margin calculation needs to be appended by adding the ball kinetic energy KE_BALL (Fig. 2B) to define the energy margin EM_PE+KE as

$$E{M}_{\text{PE}+\text{KE}}=\left[P{E}_{\text{MAX}}-\left(P{E}_{\text{BALL}}+K{E}_{\text{BALL}}\right)\right]/P{E}_{\text{MAX}}.$$ (10)

This formulation uses the same energy limit PE_MAX, because if the ball is right at the cup rim (i.e., PE_BALL = PE_MAX), then the ball must have zero velocity within the cup (KE_BALL = 0); otherwise, the ball will escape at a future time.

A limitation of this calculation is that it assumes that the cup is not accelerating. In other words, if there is a positive safety margin, EM_PE+KE > 0, it is assumed that the ball will oscillate within the cup indefinitely and never escape. This is not true if the cup is accelerating, i.e., ẍ ≠ 0 (Fig. 2C). In such an accelerating, non-inertial reference frame, there is a pseudo force that will change the ball's energy, which must be accounted for. The pseudo ball energy, PSE_BALL, is given by

$$PS{E}_{\text{BALL}}=\{\begin{array}{cc}\ddot{x}\ge 0& -m\ddot{x}\ell \sin \theta +m\ddot{x}\ell \\ \ddot{x}<0& -m\ddot{x}\ell \sin \theta -m\ddot{x}\ell .\end{array}$$ (11)

The conditional nature of PSE_BALL results from the use of a zero-energy reference (mẍℓ) that avoids negative energies. Specifically, when ẍ is positive, the zero-energy reference is when the ball or pendulum is aligned with the left horizontal (θ = 90°); when ẍ is negative, the zero reference is the right horizontal (θ = −90°). Instead of using this case distinction, we can equivalently take the absolute value of the acceleration term, giving a single equation

$$PS{E}_{\text{BALL}}=-m\ddot{x}\ell \sin \theta +m\left|\ddot{x}\right|\ell .$$ (12)

Details of the PSE_BALL derivation can be found in the appendix. Hence, the total energy of the ball TE_BALL in the local cup reference frame is given by

$$T{E}_{\text{BALL}}=K{E}_{\text{BALL}}+P{E}_{\text{BALL}}+PS{E}_{\text{BALL}}.$$ (13)

A new energy limit is defined, called the escape energy, E_ESC, which modifies PE_MAX by adding terms to account for PSE_BALL such that

$$E_{\text{ESC}}=mg\ell \left(1-\cos {\theta }_{\text{ESC}}\right)-m\left|\ddot{x}\right|\ell \sin {\theta }_{\text{ESC}}+m\left|\ddot{x}\right|\ell .$$ (14)

In contrast to Eq. 12, the absolute value of ẍ is taken in the first PSE_BALL term as well as the reference offset term. This “corrects” for the fact that there are actually two escape angles, θ_ESC and −θ_ESC, and the pertinent one depends on the sign of ẍ (see appendix). As long as TE_BALL < E_ESC, the ball will stay below the escape angle, assuming ẍ remains constant. This new EM is given by

$$EM=\left(E_{\text{ESC}}-T{E}_{\text{BALL}}\right)/E_{\text{ESC}}.$$ (15)

The EM subscript PE+KE+PSE is not included for simplicity. The difference E_ESC − TE_BALL represents how close the current ball energy is to exceeding the escape energy. This quantity is normalized to E_ESC to ease interpretation: if EM is between 0 and 1, the ball will not escape, but if EM is negative, it will escape, assuming ẍ is not changed (alternative normalizations could be used; see appendix). Note that E_ESC depends on ẍ and therefore changes during cup transportation. It should be emphasized that EM extrapolates, i.e., it takes the instantaneous energy of the ball and predicts whether the ball will escape in the future with constant ẍ. Accordingly, E_ESC is not a “hard” constraint and can be exceeded for brief periods, provided an appropriate and timely correction is made before the ball reaches the cup rim. If E_ESC is exceeded, the minimum time available to make a correction is the sensorimotor delay time. If ball energy is below E_ESC, then the time to escape is infinite and the ball will oscillate within the cup until ẍ is changed.

Experimental Design and Procedures

Participants.

A total of 18 young (21–35 yr) healthy male and female adults participated in the study. Subjects were randomly assigned to either a target-time group or a minimum-time group, with nine subjects in each group. One subject participated in both groups. No exclusions were made on the basis of sex, race, or ethnicity. Subjects were not told about the specific purpose of the study. Before participating, subjects were informed of all experimental procedures and read and signed an informed consent document approved by the Institutional Review Board at Northeastern University.

Instrumentation.

Subjects were seated 2.4 m from a rear-projection screen on which they saw a virtual cup displayed as a simple arc with a ball (Fig. 1). They could manipulate the cup by applying forces to a custom-made handle attached to a robotic manipulandum (HapticMaster, Moog, The Netherlands; Van der Linde and Lammertse 2003). Although the robot had three translational degrees of freedom, it was constrained to medial-lateral motion in the horizontal plane. The robot used admittance control with dedicated haptic and graphic servers operating at 2,500 and 120 Hz, respectively. All graphic programming and computations related to the ball were performed on the graphic server via a custom C++ program. At each program iteration, the graphic server queried the haptic server for the current robot arm kinematics (x, ẋ, ẍ) and computed the acceleration of the ball θ̈ using Eq. 2. To increase the task challenge, the ball was made more responsive to movements of the cup by multiplying the cup acceleration ẍ by a gain z and then solving for θ̈ in Eq. 2, where z = 5. With this modified θ̈, the ball θ and θ̇ values were computed using a fourth-order Runge-Kutta integrator, and the force of the ball on the cup F_B was computed by using the modified ball kinematics and solving Eq. 3. This ball force was sent to the haptic server, which computed the resulting medial-lateral acceleration of a virtual mass (m + M) according to

$$\ddot{x}=\frac{F_{\text{B}}+F_{\text{A}}}{m+M}.$$ (16)

The visual scene was then updated, and the robot motors moved the manipulandum according to ẍ. The parameters were as follows: θ_ESC = 35°, ℓ = 0.35 m, M = 3.5 kg, m = 0.3 kg, and g = 9.81 m/s².

Visual feedback.

Subjects saw neither the pendulum nor the pivot point; they only saw the ball at the end of the pendulum (Fig. 1B). The cup was drawn as an arc that subtended an angle of 2θ_ESC, which moved horizontally with the pivot point but was drawn below the pivot point so that the cup appeared to hold the ball. Two filled green rectangles were displayed, one serving as the start box and one as the goal box (Fig. 3). The on-screen distance between the centers of the boxes was 1.0 m; however, the actual physical distance D that subjects needed to traverse with the manipulandum was 0.4 m (the projected feedback was magnified by a factor of 2.5). The on-screen cup width w_CUP was 5.75 cm, and the width of the start and goal boxes w_GOAL was 9.20 cm (1.6·w_CUP).

Fig. 3.

Virtual task and visual display. Subjects were instructed to transport the cup from the left green start box to the right green target box in exactly 2 s without letting the ball escape from the cup. After the cup left the start box, a white box descended with constant velocity and overlaid the target box after 2 s, visually signaling the target time to subjects. At the target time, the cup had to be stopped inside the target box. A second group of subjects performed the task in minimum time without the white timing box. Note that the distance between the start and end boxes has been reduced in this depiction; the actual distance (0.40 m) appeared longer. T, time.

Protocol.

Data were collected while participants practiced the transportation task in 5 blocks of 60 trials (300 total), with brief rest breaks between blocks. A numeric timer display started as soon as the right edge of the cup passed the rightmost edge of the start box and terminated when the cup was stopped (ẋ < 0.02 m/s) inside the boundaries of the goal box, after which the robot arm locked and the trial ended. To prevent participants from spending a disproportional amount of time trying to keep the cup still in the goal region, the goal box was made “sticky” by applying a damping force F_D = −26ẋ to the cup when both cup edges of the cup were inside the goal, i.e., when

$$D-0.5\left(w_{\text{GOAL}}-w_{\text{CUP}}\right)<x<D+0.5\left(w_{\text{GOAL}}-w_{\text{CUP}}\right).$$ (17)

No damping was applied to the ball. The ball was not required to be still upon reaching the spatiotemporal goal; it only needed to stay inside the cup. If the ball escaped from the cup at any time, including after the cup was stopped in the goal, the robot arm locked and the ball was shown rolling out of the cup and falling toward the floor (accompanied by a “failure” sound), ending the trial.

Condition 1: target-time task.

In this condition the goal was a spatiotemporal end state, i.e., move the cup to a spatial goal in a specific time. In pilot work subjects were asked to transport the ball and cup to the spatial goal “at a comfortable speed” without letting the ball escape. No feedback was provided regarding the movement time, and no explicit rewards were provided. After initial transients (related to learning object dynamics), subjects converged to a movement time close to 2.0 s. Therefore, this time was chosen for the target-time task, in which subjects were instructed to move the cup and stop in the goal box in a goal time of T_GOAL = 2.0 s without letting the ball escape. This could be achieved with an infinite number of strategies because the 2.0-s target time was well within the subject's capabilities.

This movement time was visually signaled, so subjects did not need to learn an abstract temporal interval. Once the cup left the starting region, a white rectangular timing box descended from above the goal box with constant velocity (Fig. 3), overlaying the goal when 2.0 s had passed. The cup was to be brought to a stop at this moment, i.e., the arrival of the timing box and the cup should be simultaneous. After the end of the trial, subjects were shown their temporal error ΔT = T_GOAL − T. If ΔT < 50 ms, a sound signaled successful performance; if ΔT < 10 ms, a different sound signaled excellent performance. Points were awarded, which increased exponentially to 100 (maximum) as ΔT decreased toward zero (an exponential increased subjects' motivation). No points were awarded if ΔT exceeded 1.0 s. If the ball escaped, 100 points were subtracted from the running score.

Condition 2: minimum-time task.

Subjects were instructed to transport the ball and cup to the goal box in minimum time, for which there was a single optimal solution that depended on task constraints. If there were no constraints, then the movement time could be driven toward zero by applying forces of infinite magnitude to give infinite cup accelerations. However, because humans cannot apply infinite forces, there is a single time-optimal solution for a given set of physiological constraints. In contrast to the redundant target-time task, the white timing box was not displayed, subjects were shown their movement time (instead of a temporal error) after each trial, and points increased as times grew shorter (100 points if T < 1.5 s, 0 points if T > 2 s). All other aspects of the task were the same, including the requirement for keeping the ball in the cup and the penalty for ball escape.

Data Analysis

Data analysis was performed with MATLAB. The raw data included time histories for x, ẋ, ẍ, θ, θ̇, θ̈, and F_A, which were filtered with a dual-pass fourth-order low-pass Butterworth digital filter. Only trials in which the target location was reached and the ball was not dropped were analyzed. To facilitate averaging movement patterns across subjects for graphical presentation, time histories were normalized to a unitary movement time (0–100%) using linear interpolation.

The safety margin was assessed using the energy margin EM derived above. Variability in subjects' movement patterns across trials was quantified by the intertrial standard deviation of the average total energy, TE_STD. The total energy TE is a global measure of task performance, representing the combined effects of subject's control inputs and movement of the ball within the cup. Note that TE is the total energy of the ball and cup in the global reference frame (Eq. 6), which is different from the total energy of the ball TE_BALL in the local cup reference frame (Eq. 15). Movement time alone was not a reliable indicator of the intertrial variation in subjects' movement patterns, because large changes in the movement time could result from relatively minor adjustments of cup position near the goal box. This could occur when the cup was stopped just outside the goal box and a small adjustment was needed to bring the cup fully inside the target region.

To assess whether subjects in the target-time group used a wider range of strategies than those in the minimum-time group, subjects' F_A profiles were compared within the two groups. For this comparison the last 10 trials were interpolated (0–100% of movement time), averaged, and scaled to the peak positive force. Each subject's representative F_A profile was compared with other subjects' profiles in a pairwise manner using dynamic time warping (Berndt and Clifford 1994). This method takes two time-dependent sequences, warps or shifts them in time to find an optimal match, and provides a metric representing the distance between the two warped sequences. This analysis focuses on differences in the shapes of the force profiles, rather than only time-related stretching and/or compression from varying velocities. Each subject's average force profile was compared with all others in the group, and an average distance cost was computed for each subject.

Statistical Analysis of Experimental Data

To assess changes in end-state performance, we used paired t-tests to compare the mean absolute temporal error (target-time group) and movement times (minimum-time group) early and late in practice (first vs. last 30 trials). A similar procedure was used to assess changes in trial-to-trial variability of temporal error/movement time and total ball and cup energy profiles TE_STD with practice. A two-sample t-test was used to test whether the average distance costs for the target-time group F_A profiles were different from those of the minimum-time group, i.e., to test whether one group had a greater diversity of movement strategies than the other (hypothesis 1). For both groups, changes in the average EM with practice were assessed using paired t-tests of the first vs. the last 30 trials (hypotheses 2a and 2b). Pearson product-moment correlations were used to determine if any relation existed between the late-practice EM and trial-to-trial variability TE_STD (last 30 trials; between-subjects comparison; hypothesis 3a). Correlations were also performed to assess the relationship between changes in the energy margin, ΔEM, and changes in trial-to-trial variability, ΔTE_STD (first 30 vs. last 30 trials; within-subjects comparison; hypothesis 3b). For all statistical tests, two subjects in the target-time group were identified as outliers and excluded. In contrast to the other seven subjects in the group, these subjects increased the variability of their movement patterns and used a different high-acceleration movement strategy (see results). Significance was set at P < 0.05 for all tests.

RESULTS

Changes in Task Performance and Variability with Practice

Figure 4A shows how subjects in each group improved their end-state performance. The target-time group decreased their absolute temporal error from an initial 213 ± 129 ms to 57 ± 12 ms late in practice (P = 0.016). The minimum-time group started with an average movement time near 2 s (2.01 ± 0.161 s) and decreased this time by the end of practice (1.54 ± 0.064 s; P < 0.001). At the end of practice, the minimum-time group was still improving by a small amount (Fig. 4A); however, this was not problematic because the goal was to elicit a significant improvement in end-state performance, rather than examine asymptotic performance. The fastest time achieved by any subject in the minimum-time group without letting the ball escape was 1.33 s. Both groups decreased their trial-to-trial variability in their movement timing (Fig. 4B) with practice, as expected (target-time group: P = 0.007; minimum-time group: P = 0.002). Given that timing variability was partly influenced by homing-in adjustments, we also evaluated trial-to-trial variability in subjects' energy profiles, quantified by the standard deviation of the average total ball and cup mechanical energy, TE_STD (Fig. 4C). The target-time group showed a small but significant decrease in TE_STD with practice (P = 0.040), whereas the minimum-time group increased TE_STD (P < 0.001). This shows that although the minimum-time group's movement timing became more consistent with practice, their movement profiles became increasingly variable across trials. Also shown for the target-time group is the average for the two outlier subjects (dashed line), who increased TE_STD over practice instead of showing a decrease; however, the magnitude and variability of the timing error for these two subjects (Fig. 4, A and B, respectively) were mostly within the range of the other subjects' (±1 SD) and are therefore not shown separately. The time constants of the individually fit exponentials for subjects were not different between the groups (movement timing, P = 0.272; movement timing variability, P = 0.512; total energy variability, P = 0.098).

Fig. 4.

Movement time, variability in movement timing, and variability in energy profiles in the target-time and minimum-time groups. A: subject movement time relative to the task goal. The target-time group (left) had to achieve a target movement time of 2 s, and therefore the absolute temporal error from this target time reflects performance. The minimum-time group (right) had to move in minimum time, and therefore movement time reflects performance. Data were averaged in 10 consecutive trial bins, and each bin was averaged across subjects (filled circles). Vertical dashed lines represent brief rest breaks. Error bars indicate ±1 between-subjects standard deviation. Solid black curves represent exponentials fit to the average group data (target-time R² = 0.89, time constant = 35 trials; minimum-time R² = 0.89, time constant = 46 trials). B: changes in trial-to-trial variability of subjects' absolute temporal error (left) and movement time (right) from trial to trial (target-time R² = 0.85, time constant = 29 trials; minimum-time R² = 0.67, time constant = 40 trials). C: changes in trial-to-trial variability of subjects' energy profiles, quantified by standard deviations of the average total ball and cup mechanical energy, TE_STD (target-time R² = 0.12, time constant = 15 trials; minimum-time R² = 0.82, time constant = 64 trials). Dashed line represents the average for 2 outlier subjects in the target-time group.

Exemplary Movement Profiles: Kinematics and Energetics

Figure 5 shows the time course of kinematics and energy profiles for the first (dashed lines) and last (solid lines) 10 trials of 2 representative subjects. For the target-time task, cup velocity ẋ increased with displacement x, which caused the ball to move backwards relative to the cup, indicated by positive θ, followed by about two back-and-forth oscillations of the ball as the cup was transported to the goal. The ball escape angle (±35°) is indicated by the black “ball escape” boundary. During cup transit, the total energy of the ball and cup TE increased and then decreased. It remained nonzero at the end of cup movement, since the ball was permitted to oscillate as long as it did not go past the cup rim. For the minimum-time task, the magnitude of ẋ increased substantially with practice and the ball came much closer to the escape angle; correspondingly, TE was considerably higher.

Fig. 5.

Ball and cup kinematics and energetics for 2 representative subjects from each condition. Cup position x, cup velocity ẋ, ball angle θ, and total mechanical energy of the ball and cup system TE for the first (dashed lines) and last (solid lines) 10 trials are shown in a subject from the target-time group (A) and in a subject from the minimum-time group (B). The angle θ at which the ball would escape was ±35° and is indicated by the black “ball escape” boundaries.

Average Force Profiles

Figure 6 shows the average time-normalized F_A profile of all subjects in the two groups at the end of practice. Each line represents one subject's average profile; the shaded band denotes ±1 SD. Figure 6 highlights differences between subjects and groups. Note that the dashed lines represent the two outlier subjects in the target-time group. These subjects showed a rapid initial rise in their applied force, causing a large initial cup acceleration. To address hypothesis 1, the target-time group's average F_A profiles were compared with the minimum-time group's profiles using dynamic time warping, which provided a distance metric representing the similarity of the profiles within each group. The distances were not different between the groups (average normalized cumulative distance between subjects: target-time group, 2.2 ± 1.5; minimum-time group, 3.3 ± 1.3; P = 0.139). This result does not support hypothesis 1, which specified that the target-time group would display more varied movement strategies (greater distances) than the minimum-time group.

Fig. 6.

Average applied force F_A profiles for the last 10 successful trials for all subjects. A: target-time group. Dashed lines show 2 outlier subjects who had a much larger rate of initial force development. B: minimum-time group. The mean (solid line) and SD (shaded band) are shown for each subject. Time was normalized to the total movement time.

Energy Margin

It is instructive to visualize the energy margin EM and ball escape energy E_ESC in execution space (Fig. 7A). This space is spanned by three variables, θ, θ̇, and ẍ, that together define EM. Two different perspectives of one successful (Fig. 7A) and one failed trial (Fig. 7B) are shown for one subject in the target-time group. The two-dimensional surface (blue mesh) denotes the E_ESC manifold, where EM = 0. As long as the trajectory remains inside this manifold (where EM is between 0 and 1), the ball will never escape (under the assumption of constant ẍ). If outside the manifold where EM is negative, the ball energy exceeds E_ESC, the ball will escape unless ẍ is changed by making a “correction.” In the successful trial, the trajectory stayed largely within the E_ESC manifold, but in the failed trial it moved well outside the manifold and the ball escaped. Note that EM does not consider time directly; it might not be possible to initiate a correction before the ball escapes. For reference, the manifold delimiting a 120-ms (an approximate sensorimotor delay; Jeannerod 1988) time-horizon for ball escape is shown in pink in Fig. 7, A and B. If the trajectory is outside this outer manifold, recovery is no longer possible because subjects would need to initiate a correction in less than 120 ms. Note that this does not account for the fact the subjects could use “preplanned” corrections, which circumvent this 120-ms boundary.

Fig. 7.

Safety margin defined as an energy margin relative to ball escape. A: an exemplar successful trajectory (trial) in 3-dimensional execution space, spanned by ball angle θ, ball angular velocity θ̇, and cup horizontal acceleration ẍ, which together specify the ball energy in the cup reference frame. The escape energy, E_ESC, describes a 2-dimensional manifold (meshed blue surface). Assuming ẍ is unchanged, when the trajectory is inside the E_ESC manifold, there is a positive safety margin and the ball will remain in the cup (green trajectory); if the trajectory is outside (black trajectory), there is a negative safety margin and the ball will escape unless a correction is made (i.e., a change in ẍ). The pink outer boundary represents a time to ball escape of 120 ms, an approximate human sensorimotor delay. If the time to escape falls below this time, it is impossible to make a reactive correction to prevent ball escape. B: in the failed trial the trajectory moved outside the outer boundary (red trajectory) and escaped a short time later (magenta square). C: exemplary time series of θ̇ and ẍ for the target-time task (left) and corresponding safety margin measures (right). Three variants are shown: 1) the energy margin EM (solid green/black line, PE+KE+PSE), which accounts for the ball energy and the subject inputs through ẍ, 2) EM_PE+KE, which only considers ball potential and kinetic energy (purple dashed line), and 3) EM_PE, which is a function of only the ball potential energy (blue dashed line).

Figure 7C shows the ball and cup kinematics and the corresponding EM for a different successful trial (target-time task). For comparison, Fig. 7C also shows the two simpler EM measures, which ignore cup acceleration ẍ (EM_PE and EM_PE+KE) and ball velocity (EM_PE). EM_PE and EM_PE+KE differ most when the ball swings through the bottom of the cup, whereas EM is most different at the beginning and end of the movement, where the magnitude of ẍ is the greatest. As the cup accelerates and the ball swings toward the cup rim, EM decreases sharply, varies as the movement progresses, and reaches a minimum near the end. Because EM was positive at the end of the movement, the ball stayed in the cup (once the cup is brought to a stop in the goal, the manipulandum locks and the net ball energy does not change). Although the two other EM measures, EM_PE and EM_PE+KE, appear less complex and potentially more accessible to subjects, the full EM measure is needed to characterize the safety margin because both ball position and velocity define the state of the ball, and ẍ reflects the immediate control actions applied to the cup, which change the ball energy.

Ball Escapes

Figure 8 depicts the number of failed trials as a function of location (A) and movement time (B), i.e., when and where the ball was lost during the trial. As expected, the number of ball escapes was much larger in the minimum-time group compared with the target-time group (∼19 vs. ∼2% of all trials). The histograms in Fig. 8 illustrate that in both tasks most of the ball escapes occurred near the end of movement and, to a smaller degree, soon after the start. Rarely did subjects fail near the middle. The large EM dip at the end is consistent with the large negative acceleration when the cup is brought to a stop (see Fig. 7C and Fig. 9, A and B). Based on this EM dip and the high percentage of failures near the end of the cup transit (in both distance and time; Fig. 8, A and B), this can be viewed as the most “dangerous” portion of the task. While the maximum EM dip occurred before the end of movement, ball escapes generally occurred immediately at movement termination or after a small delay. This is because EM is an extrapolative measure that predicts future escape, so there will be a delay until the ball actually escapes. Also shown in Fig. 8C is the number of failed trials as a function of practice. As expected, the number of failures decreased with practice in the target-time task. Conversely, the failure rate steadily increased in the minimum-time group, except for the last block, which showed a decrease. For clarity, the target-time outliers are not shown; however, their failure rates were similar to those of the other subjects in the group.

Fig. 8.

Failures (ball escapes). Histograms show the total number of failures for all subjects aggregated across the practice sessions for the target-time and minimum-time groups. The same failure data are presented as a function of the cup position at failure (A), movement time at failure (B), and stage of practice (C).

Fig. 9.

Changes in energy margin with practice. A: continuous energy margin EM over the time of a trial (normalized to 100%) for 4 exemplary subjects from the target-time group. Dark/light shading indicates a decrease/increase in EM (between the average for first vs. last 30 trials). B: data for 4 exemplary subjects from the minimum-time group. C: summary changes in EM with practice. Each data point represents the average for the first (early practice) or last (late practice) 30 trials. Subjects performed either the target-time (triangles) or minimum-time experiments (circles). Open triangles denote the 2 outlier subjects; arrows identify the 1 subject who completed both experiments.

Energy Margin Changes With Practice

To address hypothesis 2, the energy margin EM was first examined over the course of a trial. Figure 9 shows EM for four exemplary subjects from the target-time group (A) and the minimum-time group (B). The averages for early and late practice trials (first/last 30 trials) are shown. The change with practice is highlighted by gray (increased EM) and black shading (decreased EM). On average, the target-time group increased EM, whereas the minimum-time group decreased EM, as expected from hypothesis 2.

A summary view of EM changes was obtained by averaging EM across each trial and submitting these averaged data to statistical tests. Figure 9C displays the changes in EM for the target-time group and the minimum-time group from early to late practice. Each data point represents the first or last 30 trials averaged for 1 subject. The minimum-time group decreased EM with practice (P = 0.025), whereas the target-time group increased EM (P = 0.006). One subject who participated in both groups is identified with small black arrows. Despite exposure to both conditions, this subject increased/decreased EM in the same way as the other subjects in each group. These results are consistent with the hypothesis that the minimum-time group would decrease their safety margins (hypothesis 2a) and the target-time group would increase theirs (hypothesis 2b).

Energy Margin and Trial-to-Trial Variability

Given these supportive results, this issue is taken one step further in hypothesis 3: Do individuals adapt their energy margin in accordance with their change in variability? To this end, the trial-to-trial variability of subjects' movement strategies was quantified by the standard deviation of average total ball and cup mechanical energy across trials, TE_STD, and was compared with the average EM. Figure 10 shows correlations between these quantities for both subject groups. At the end of practice (Fig. 10A), there was a significant positive correlation between EM and TE_STD for the minimum-time group, i.e., subjects with higher variability had larger safety margins. However, there was no correlation for subjects in the target-time group. Different results were found when analyzing these relations within each subject by correlating the change in the EM (ΔEM) with the change in TE_STD (ΔTE_STD) from early to late practice (Fig. 10B). In this case, the target-time group exhibited a significant positive correlation. Subjects with large decreases in variability had small increases in their safety margin; those with small decreases in variability had larger safety margin increases. There was no correlation for the minimum-time group.

Fig. 10.

Changes in energy margins as a function of trial-to-trial variability and task redundancy. A: correlations between energy margin EM and trial-to-trial variability TE_STD during late practice over the last 30 trials for the target-time group (triangles) and the minimum-time group (circles). B: correlations between the change in EM (ΔEM) and change in TE_STD (ΔTE_STD) from early (first 30 trials) to late practice (last 30 trials) within each subject. Open triangles denote the 2 outlier subjects; arrows identify the 1 subject who completed both experiments.

DISCUSSION

Task and Quantification of Safety Margin

Compared with the multitude of data from studies examining reaching and/or manipulation of rigid objects, little is known about how humans control objects with underactuated dynamics or internal degrees of freedom. To date, only a few studies have investigated such “flexible” objects (Dingwell et al. 2002, 2004; Nagengast et al. 2009; Svinin et al. 2006). Importantly, in these studies no constraints were imposed on the motion of the object, except at the beginning and end of movement. Inspired by the real-life task of carrying a cup of coffee, where the coffee cup must be manipulated in a way that prevents spillage at all times, a new model task was created that required subjects to transport a ball in a shallow cup without letting the ball escape from the cup. In this task large and/or rapid fluctuations of the ball (the “coffee”) are associated with high-energy states, and an energy threshold determines the risk of ball escape (“spilling coffee”). As in many daily life tasks, these energetic constraints must be satisfied in conjunction with end-state goals, i.e., a safety margin must be maintained and a target must be reached. It is important to understand how both of these constraints impact subjects' control strategies.

A first challenge was to define a measure that quantified the continuous energetic constraint. To this end, an energy margin EM was defined, which extrapolates the state of the ball, assuming constant cup acceleration, and provides a measure of the total ball energy relative to the escape energy. The EM was calculated at each instant in time and was continuously updated to reflect the changing cup control by the subject. As shown in Fig. 7, the “escape” condition can be visualized as a two-dimensional manifold in the three-dimensional execution space. Exceeding the escape energy is equivalent to breaching this manifold. Because the EM measure is an extrapolation based on constant cup acceleration, it is possible that a large and sudden change in acceleration could cause this boundary to be crossed without the ball escaping from the cup. However, this is unlikely because EM was continuously updated (at 120 Hz), so sudden acceleration changes are quickly reflected in the EM (in <10 ms).

The EM is conceptually similar to other calculations with a “predictive horizon,” which have been used to understand catching and driving (Lee 1976; Lee et al. 1983) and in the assessment of postural control. For the latter, the instantaneous position, velocity, and acceleration of body center of mass (Hasson et al. 2008, 2009) or center of pressure (Slobounov et al. 1997) relative to the base of support boundary is extrapolated, assuming constant acceleration, to predict when the boundary will be crossed (a short time is less “safe”). One important difference is that in postural control there is always a finite time to contact, because upright stance is an unstable configuration. In contrast, in the ball and cup task, this time can be infinite if the ball is below the escape energy (EM > 0): for constant acceleration, the ball will keep oscillating within the cup and never escape.

It is important to note that the EM measure does not account for physiological constraints on action, such as the maximum force that subjects apply to the cup or the maximum rate of force development that subjects could achieve. Including these constraints would add considerable complexity. For example, defining an appropriate maximal applied force is not trivial, because the maximum force that a muscle can produce depends on its length and velocity (Gordon et al. 1966; Hill 1938) and on other time-dependent properties, such as force enhancement (Edman et al. 1978) and depression (Edman et al. 1993). Although simpler EM measures based on ball position and/or velocity appear feasible, these are not adequate for two reasons. 1) Cup acceleration due to an external force changes the ball energy and reflects the immediate control actions applied to the cup. It is therefore necessary to include. 2) Other research suggests that the nervous system is likely to use acceleration information (in addition to position and velocity) in its modulation of motor commands (Lockhart and Ting 2007).

Hypothesis 1: Redundancy and Changes in End-State Performance and Variability with Practice

The ball and cup transportation task was used to investigate how task redundancy influenced subjects' movement strategies. Redundancy was “created” by giving only sparse instructions: subjects who performed a target-time task were asked to complete the cup transit in a comfortable time of 2 s. Although subjects in this task used different movement strategies, at the end of practice most applied force gradually over the first half of movement but had a more variable second half. This observation is consistent with a two-component model consisting of a preplanned or ballistic movement followed by online feedback regulation, i.e., a “homing-in” phase (Woodworth 1899). Two subjects within the target-time group had a much greater initial rate of force development. A large initial force rapidly accelerates the cup and covers distance quickly, allowing a slower and hence more accurate target approach. However, the trade-off is that a large acceleration is more risky and may lead to a more variable and less predictable cup trajectory, which is corroborated by the observed decrease in the safety margin and increase in the trial-to-trial variability for these subjects compared with others.

In a contrasting task condition, redundancy was “removed” by explicitly asking subjects to move the cup as fast as possible (minimum-time group). In this case, there is a single explicit time-optimal solution, given limits on the rate and amplitude of muscular force production. This optimal control problem has been studied extensively in engineering for a model system, a gantry crane that transports a load (Karihaloo and Parbery 1982). Because there are limits on the torque capacity of the motor and on the amplitude of the load's oscillations, there is a unique time-optimal solution for moving the load with the crane (Golafshani and Aplevich 1995; Manson 1982; Van de Ven 1983). The idealized dynamics of the crane and load are almost identical to those of the ball and cup task. The latter is based on a model of a cart that translates with a pendular load and a human acting as the “motor” that pushes/pulls the cart.

It was therefore expected that with practice subjects would converge to the same strategy in the minimum-time task, which presents an optimal solution given physiological constraints (as in the crane transportation problem). However, over the 300 practice trials, subjects continued to use a range of different strategies to achieve the task goal. Although this does not support the hypothesis, it does agree with results of Dingwell et al. (2002), who showed that subjects adopted different movement strategies when transporting a virtual mass-spring object to a spatial target. However, there were several key differences with respect to the present study. The task used by Dingwell and colleagues required the mass-spring to be still at the goal location, the object dynamics were linear, and there were no constraints during the movement (i.e., the mass could not “escape”). In the present study, the ball could oscillate at the goal as long as it remained in the cup, the ball and cup dynamics were nonlinear, and the ball could escape. The goal was also made “sticky,” which reduced the emphasis on actively damping oscillations at the goal. At the end of practice, subjects had a nonzero applied force at movement termination; this suggests that subjects incorporated the goal damping into their movement strategies.

One reason why the minimum-time group did not converge to the same strategy may be because subjects had different weightings for costs such as control effort (Guigon et al. 2007; Todorov and Jordan 2002), movement smoothness (Ben-Itzhak and Karniel 2008; Flash and Hogan 1985), or risk sensitivity (Nagengast et al. 2010). Differences between subjects could also arise from varying physiological properties. This is supported by the relatively large differences in trial-to-trial variability between subjects, which suggests varying amounts of intrinsic neuromotor noise and/or uncertainty (Faisal et al. 2008). Also, subjects' varying strategies could have arisen from the stochastic selection from several “good enough” (Raphael et al. 2010) movement plans, as shown by Kodl et al. (2011).

Hypothesis 2: Safety Margins and Task Redundancy

We aimed to determine how task redundancy affected safety margins. In the cup transportation task, the safety margin was defined in terms of energy, i.e., as an energy margin EM, which was a nonlinear function of the ball position and velocity and cup acceleration. Subjects were not informed of the nature of the safety margin; therefore, they had to discover which movements were safe and which were more risky. This is more challenging compared with previous studies in which the safety margin was explicitly defined as the static position of an object in relation to the arm, as when avoiding an obstacle (Abend et al. 1982; Dean and Brüwer 1994; Hamilton and Wolpert 2002; Sabes and Jordan 1997).

It was expected that with practice subjects in the minimum-time task would shift from a conservative strategy to one that was more risky (decreased energy margin), because increased cup accelerations are needed to reduce movement time. Although intuitive, this is not a foregone conclusion, because large accelerations are not by default “dangerous”; the effect is dependent on the state of the ball. For instance, if the ball is pushed toward the left side of the cup (positive angle), an acceleration to the right (positive) is safer than an acceleration to the left (negative). This is illustrated by the asymmetrical nature of the escape energy E_ESC manifold in the execution space (Fig. 7, A and B). A decreased safety margin following practice is consistent with the findings of previous research on grip force adaptation (Crevecoeur et al. 2009; Gordon et al. 1993). These studies concluded that as subjects became familiar with the dynamics of the gripped object with repeated lifts, they reduced their grip force and decreased their safety margins, presumably to minimize energy expenditure.

In contrast to the minimum-time task, which encouraged subjects to push the limits of their capabilities, subjects in the target-time task had a “choice” in developing their movement strategies. It was hypothesized that in this task subjects would take advantage of redundancy and find strategies with greater safety margins. This is because large safety margins confer an advantage, as they reduce the risk of ball escape, allowing subjects to focus less on the ball and more on accurate achievement of the end-state goal. Consequently, the results showed that subjects increased their safety margin with practice, supporting the hypothesis. This agrees with Nagengast et al. (2010), who used a risk-sensitive optimal feedback control model of reaching to show that most subjects are not risk neutral but are risk adverse. It is further consistent with the findings of Sternad and colleagues (Müller and Sternad 2004, 2009; Sternad et al. 2011), who have shown that in a redundant throwing task, subjects develop strategies that are tolerant to performance errors due to execution variability.

Error tolerance quantifies the sensitivity of a given movement strategy to noise in repeated task executions (Cohen and Sternad 2009; Müller and Sternad 2004, 2009; Sternad and Abe 2010; Sternad et al. 2011). If tolerance is high, then variability in execution will have relatively little effect on the task outcome or result. This previous work on error tolerance has used a discrete throwing task where the result variable, the scalar task error, is completely described by the execution variables of the arm (angle and velocity) at ball release. This mapping from two variables to one result creates redundancy. However, application of the tolerance analysis introduced by Müller and Sternad (2004) to the continuous cup transportation task is not straightforward: in the ball-and-cup task, the result variable is the energy margin EM, which is defined at every moment along the cup trajectory and is a priori unknown to subjects. The execution variables include the ball states, angle and velocity, and the cup acceleration, which together define EM. Alternatively, the execution and result variables could be defined with reference to the end state goal, instead of the ball-in-cup constraint. Here, a redundant set of cup and ball trajectories (the execution variables) can be defined from which subjects will chose a subset to achieve the result variable, i.e., the target time. Current research is developing a method to analyze tolerance with respect to the end-state goal in continuous tasks.

Hypothesis 3: Variability and Safety Margins are Jointly Modulated With Practice

The final purpose of this study was to examine how safety margins are modulated with respect to motor variability and how such a relation might depend on task redundancy. The results showed that the way in which subjects' safety margins changed with practice in a redundant task was explained in large part by how their trial-to-trial variability changed. Specifically, analysis within individuals revealed that smaller decreases in variability were associated with larger increases in safety margins, and vice versa. Those subjects who converged toward a consistent movement pattern may have been more confident in their ability due to a relatively small amount of execution noise and therefore did not need a large safety margin. On the other hand, subjects with greater trial-to-trial variability may have chosen a larger safety margin to accommodate the greater uncertainty associated with more variable movements. This agrees with previous work suggesting that variability plays a central role in movement control such that the motor system optimizes movements to minimize the effects of variability on task goals (Chu WT, Sternad D, Sanger TD, unpublished observations; Cohen and Sternad 2009; Gepshtein et al. 2007; Harris and Wolpert 1998; Hudson et al. 2010; Sternad et al. 2011; Trommershäuser et al. 2005). The novelty of the present findings is that they span a period in which significant learning has occurred, and they support the hypothesis that individuals shape their control strategies in accordance with their variability.

Whereas in the minimum-time task, changes across practice were not observed, a clear positive correlation between safety margins and variability was observed at the end of practice after subjects converged to their preferred strategies. In this condition, subjects were instructed to push the limits of their capability by moving as fast as they could. This drive toward limit performance is supported by the fact that the trial-to-trial variation in subjects' energy profiles, captured by the standard deviation of the average total ball and cup energy between trials, increased with practice. High-velocity movements tend to increase variability, because fast movements require greater muscle activation, and hence greater signal-dependent noise (Harris and Wolpert 1998). However, even under these more variable conditions, the relation between variability and safety margins was clearly evident at the end of practice. In addition to signal- or velocity-dependent noise, the nonlinear dynamics of the ball-and-cup system are likely to contribute to trial-to-trial variability. A forced-pendulum can exhibit a rich array of behaviors due to nonlinearities and sensitivity to initial conditions (Van Dooren 1996). Compared with the target-time task, in the minimum-time task the applied forces were larger, and the ball reached larger angles with higher energy states, which may have caused kinematic profiles to diverge more rapidly from a planned trajectory.

Although at the task level, the target-time task has an infinite number of solutions, in principle, any redundant task can be made nonredundant by introducing a cost function. For example, the task of pointing toward a line is redundant, because subjects could point to any location on the line and fulfill the task goal. However, if we assume that a subject's behavior is guided by a criterion cost that minimizes one or a combination of task-relevant variables, then a single optimal movement trajectory can always be found. Indeed, Berret et al. (2011) recently showed that subjects' trial-to-trial behavior in a redundant line-pointing task could be explained by stochastic optimal feedback control (Todorov and Jordan 2002) if it is assumed that subjects minimize mechanical energy expenditure and movement smoothness. In the present study, if subjects' behavior was indeed guided by a cost function, it may likely include terms related to the safety margin in combination with typical costs such as control effort (Guigon et al. 2007; Todorov and Jordan 2002) or movement smoothness (Ben-Itzhak and Karniel 2008; Flash and Hogan 1985). It is likely that the points awarded to subjects were also factored into the overall cost. The influence of points or monetary rewards on movement control has been studied by Trommershäuser et al. (2005) using relatively simple pointing movements, where the explicit reward cost may have been the chief consideration for subjects. However, the task used in the present experiment is likely to have multiple weighted costs (effort vs. smoothness vs. reward); separating out the weights of such costs is not trivial. Although this can be done, each added cost introduces free parameters that would inevitably increase the fit to experimental data. Instead of specifying a cost function, we used a data-driven approach to examine the trade-off between variability and safety margins.

Conclusion

To conclude, this study has shown that when given the option, most humans prefer movement strategies with increased safety margins and shape their movements in accordance with their changing variability. More broadly, the results suggest that highly constrained tasks such as the minimum-time task may be most informative when the research question is focused on understanding optimized movement strategies formed after a period of practice. Less-constrained tasks such as the target-time task may be useful when the aim is to understand the principles that guide the formation of movement strategies with motor learning.

GRANTS

This research was supported by National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS) Ruth L. Kirschstein National Research Service Award for Individual Postdoctoral Fellows 1F32 AR061238 (to C. J. Hasson) and National Institute of Child Health and Human Development (NICHD) R01 HD045639, National Science Foundation NSF-DMS0928587, and American Heart Association AHA-11SDG7270001 Grants (to D. Sternad).

DISCLAIMER

The contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIAMS, NICHD, NSF, or AHA.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

Author contributions: D.S. conception and design of research; C.J.H., D.S., and T.S. conception and development of model and theoretical approach; D.S. and C.J.H. designed experiment; C.J.H. performed experiments; C.J.H. and T.S. analyzed data; C.J.H. and D.S. interpreted results of experiments; C.J.H. prepared figures; C.J.H. drafted manuscript; C.J.H. and D.S. edited and revised manuscript; C.J.H., T.S., and D.S. approved final version of manuscript.

APPENDIX: DETAILED MATHEMATICAL DESCRIPTION OF THE TASK

Derivation of Equations of Motion

The ball-and cup-system is analogous to the standard model of a pendulum hanging from a cart (Fig. 1C). This system has two degrees of freedom. The kinetic energy of the ball, KE_BALL, in the inertial reference frame is

$$\begin{array}{c}K{E}_{\text{BALL}}=\frac{1}{2}m\left[{\left(\dot{x}-\ell \dot{\theta }\cos \theta \right)}^2+{\left(\ell \dot{\theta }\sin \theta \right)}^2\right]\\ =\frac{1}{2}m\left({\dot{x}}^2-2\ell \dot{\theta }\cos \theta +{\ell }^2{\dot{\theta }}^2\right),\end{array}$$ (A1)

where m is the ball mass, ℓ is the pendulum length, θ is the ball (pendulum) angle, and θ̈ is the ball angular velocity. The potential energy of the ball, PE_BALL, depends only on the ball height,

$$P{E}_{\text{BALL}}=mg\ell \left(1-\cos \theta \right).$$ (A2)

where g is gravitational acceleration. For simplicity, the cart will henceforth be referred to as the “cup.” The kinetic energy of the cup, KE_CUP, is

$$K{E}_{\text{CUP}}=\frac{1}{2}M{\dot{x}}^2,$$ (A3)

where M is the cup mass and ẋ is the cup horizontal velocity. The Lagrangian L is given by

$$L=\left[K{E}_{\text{CUP}}+K{E}_{\text{BALL}}\right]-P{E}_{\text{BALL}}=\left[\frac{1}{2}M{\dot{x}}^2+\frac{1}{2}m\left({\dot{x}}^2-2\ell \dot{x}\dot{\theta }\cos \theta +{\ell }^2{\dot{\theta }}^2\right)\right]-mg\ell \left(1-\cos \theta \right).$$ (A4)

Note that the gravitational potential energy of the cup is constant because the cup's vertical position does not change; therefore, this energy is irrelevant. The Euler-Lagrange equation,

$$\frac{\partial L}{\partial q}=\frac{d}{dt}\frac{\partial L}{\partial \dot{q}},$$ (A5)

is used to find the equations of motion. Using the generalized coordinates x and θ, we have

$$\frac{d}{dt}\frac{\partial L}{\partial \dot{x}}-\frac{\partial L}{\partial x}=F_{\text{A}}$$ (A6)

and

$$\frac{d}{dt}\frac{\partial L}{\partial \dot{\theta }}-\frac{\partial L}{\partial x}=0,$$ (A7)

where F_A is an external force applied to the cup.

First equation of motion.

The first equation of motion for the cup and ball is derived by computing the partial derivative of the Lagrangian with respect to ẋ, giving

$$\frac{\partial L}{\partial \dot{x}}=\left(m+M\right)\dot{x}-m\ell \dot{\theta }\cos \theta .$$ (A8)

Its time derivative is

$$\frac{d}{dt}\frac{\partial L}{\partial \dot{x}}=\left(m+M\right)\ddot{x}-m\ell \ddot{\theta }\cos \theta +m\ell {\dot{\theta }}^2\sin \theta .$$ (A9)

The partial derivative of the Lagrangian with respect to x is zero,

$$\frac{\partial L}{\partial x}=0.$$ (A10)

Plugging Eq. 9 into the Euler-Lagrange equation gives the first equation of motion for the generalized coordinate x,

$$\frac{d}{dt}\frac{\partial L}{\partial \dot{\theta }}-\frac{\partial L}{\partial \theta }=\left(m+M\right)\ddot{x}-m\ell \ddot{\theta }\cos \theta +m\ell {\dot{\theta }}^2\sin \theta =F_{\text{A}}.$$ (A11)

Rearranging gives Eq. 1:

$$\left(m+M\right)\ddot{x}=\left(m\ell \ddot{\theta }\cos \theta +m\ell {\dot{\theta }}^2\sin \theta \right)+F_{\text{A}}.$$ (A12)

Second equation of motion.

For the second equation of motion for cup and ball, the partial derivative of the Lagrangian with respect to θ̇ is

$$\frac{\partial L}{\partial \dot{\theta }}=-m\ell \dot{x}\cos \theta +m{\ell }^2\dot{\theta },$$ (A13)

and its time derivative is

$$\frac{d}{dt}\frac{\partial L}{\partial \dot{\theta }}=-m\ell \ddot{x}\cos \theta +m\ell \dot{x}\dot{\theta }\sin \theta +m{\ell }^2\ddot{\theta }.$$ (A14)

The partial derivative of the Lagrangian with respect to θ is

$$\frac{\partial L}{\partial \theta }=m\ell \dot{x}\dot{\theta }\sin \theta +mg\ell \sin \theta .$$ (A15)

Plugging into the Euler-Lagrange equation gives

$$\frac{d}{dt}\frac{\partial L}{\partial \dot{\theta }}-\frac{\partial L}{\partial \theta }=\left(-m\ell \ddot{x}\cos \theta +m\ell \dot{x}\dot{\theta }\sin \theta +m{\ell }^2\ddot{\theta }\right)-\left(m\ell \dot{x}\dot{\theta }\sin \theta -mg\ell \sin \theta \right)=0.$$ (A16)

After simplification,

$$m\ell \left(\ell \ddot{\theta }-\ddot{x}\cos \theta +g\sin \theta \right)=0,$$ (A17)

which can be rearranged to give the second equation of motion for the generalized coordinate θ. The mass disappears if we assume that m and ℓ cannot be zero, giving Eq. 2:

$$\ddot{\theta }=\frac{\ddot{x}}{\ell }\cos \theta -\frac{g}{\ell }\sin \theta .$$ (A18)

Derivation of Pseudo Energy

In the local reference frame of the cup, it is assumed that the total energy is conserved. This allows the explicit characterization of the energy contributions due to the cup acceleration ẍ. In an accelerating reference frame, there is a pseudo energy, PSE_BALL, that must be derived. Assuming that cup acceleration is constant, we begin with zero rate of change in TE_BALL,

$$\frac{dT{E}_{\text{BALL}}}{dt}=0.$$ (A19)

We then take the derivative as

$$\frac{d\left(K{E}_{\text{BALL}}+P{E}_{\text{BALL}\left(\text{Net}\right)}\right)}{dt}=\frac{dK{E}_{\text{BALL}}}{dt}+\frac{dP{E}_{\text{BALL}\left(\text{Net}\right)}}{dt}=0.$$ (A20)

Here, TE_BALL is the sum of the ball's kinetic energy, KE_BALL, and the net potential energy, PE_BALL(Net):

$$\frac{d\left(\frac{\text{1}}{\text{2}}m{l}^2{\dot{\theta }}^2\right)}{dt}=\frac{dP{E}_{\text{BALL}\left(\text{Net}\right)}}{dt}=0$$ (A21)

$$m{\ell }^2\ddot{\theta }\ddot{\theta }+\frac{dP{E}_{\text{BALL}\left(Net\right)}}{d\theta }\dot{\theta }=0.$$ (A22)

Combining terms gives us

$$\frac{dT{E}_{\text{BALL}}}{dt}=\dot{\theta }\left(m{\ell }^2\ddot{\theta }+\frac{dP{E}_{\text{BALL}\left(\text{Net}\right)}}{d\theta }\right)=0.$$ (A23)

If we assume θ̇ ≠ 0, then we have

$$\frac{\frac{dT{E}_{\text{BALL}}}{dt}}{\dot{\theta }}=m{\ell }^2\ddot{\theta }+\frac{dP{E}_{\text{BALL}\left(\text{Net}\right)}}{d\theta }=0.$$ (A24)

Just taking the two right-hand terms gives

$$m{\ell }^2\ddot{\theta }+\frac{dP{E}_{\text{BALL}\left(\text{Net}\right)}}{d\theta }=0.$$ (A25)

After rearranging, we arrive at

$$\frac{dP{E}_{\text{BALL}\left(\text{Net}\right)}}{d\theta }=-m{\ell }^2\ddot{\theta }.$$ (A26)

Inserting the second equation of motion, θ̈ = ẍ/ℓ cos θ − g/ℓ sin θ, into the above equation, gives

$$\frac{dP{E}_{\text{BALL}\left(\text{Net}\right)}}{d\theta }=-m{\ell }^2\left(\frac{\ddot{x}}{\ell }\cos \theta -\frac{g}{\ell }\sin \theta \right).$$ (A27)

Combining terms, we arrive at

$$\frac{dP{E}_{\text{BALL}\left(\text{Net}\right)}}{d\theta }=-m\ddot{x}\ell \cos \theta +mg\ell \sin \theta .$$ (A28)

and integrate with respect to θ for PE_BALL(Net)

$$dP{E}_{\text{BALL}\left(\text{Net}\right)}(\theta )=-m\ddot{x}\ell \sin \theta +mg\ell \cos \theta .$$ (A29)

Note that we assume ẍ is constant. PE_BALL(Net) is the sum of the gravitational potential energy, PE_G, and a pseudo energy PSE_BALL due to cup acceleration, i.e.,

$$P{E}_{\text{BALL}\left(\text{Net}\right)}=PS{E}_{\text{BALL}}+P{E}_G.$$ (A30)

Substituting Eq. 29 for PE_BALL(Net) gives

$$-m\ddot{x}\ell \sin \theta -mg\ell \cos \theta =PS{E}_{\text{BALL}}+P{E}_{\text{G}}.$$ (A31)

Now we solve for PSE_BALL,

$$PS{E}_{\text{BALL}}=-P{E}_{\text{G}}-m\ddot{x}\ell \sin \theta -mg\ell \cos \theta .$$ (A32)

The potential energy of the ball due to gravity alone (using a zero-energy reference of θ_REF = 90° or θ_REF = −90°) is given by

$$P{E}_{\text{G}}=-mg\ell \cos \theta .$$ (A33)

We substitute PE_G into Eq. 32, giving the pseudo energy PSE_BALL,

$$PS{E}_{\text{BALL}}=mg\ell \cos \theta -m\ddot{x}\ell \sin \theta -mg\ell \cos \theta .$$ (A34)

Simplifying gives

$$PS{E}_{\text{BALL}}=-m\ddot{x}\ell \sin \theta .$$ (A35)

Redefinition of Energy References

The gravitational potential energy of the ball, PE_G, is dependent on its height, which is given by Eq. A33. To prevent negative energy values, we redefine the reference so there is zero gravitational potential energy when the pendulum is at the bottom of its arc (θ = 0°). To do this, we add a constant mgℓ to Eq. A33, giving

$$P{E}_{\text{G}}=-mg\ell -mg\ell \cos \theta ,$$ (A36)

or we can write this as Eq. 5:

$$P{E}_{\text{G}}=P{E}_{\text{BALL}}=mg\ell \left(1-\cos \theta \right).$$ (A37)

We also define the reference for PSE_BALL to avoid negative energies. Similar to the way that PE_G was re-referenced by adding an offset, an offset is added here, but g is replaced with ẍ. Hence, the offset is mẍℓ and Eq. A35 becomes

$$PS{E}_{\text{BALL}}=-m\ddot{x}\ell \sin \theta +m\ddot{x}\ell .$$ (A38)

Here, when there is a positive cup acceleration ẍ, the zero-energy reference is the left horizontal, i.e., θ = 90. However, if the cup acceleration is negative, the zero-energy reference changes to be the right horizontal (i.e., θ = −90). This is achieved by adding a negative sign to the offset: −mẍℓ. Based on the conditional nature of the offset, this can be written as

$$PS{E}_{\text{BALL}}=\{\begin{array}{cc}\ddot{x}\ge 0& -m\ddot{x}\ell \sin \theta +m\ddot{x}\ell \\ \ddot{x}<0& -m\ddot{x}\ell \sin \theta +m\ddot{x}\ell .\end{array}$$ (A39)

Instead of using this case distinction, we can write one equation by taking the absolute value sign of the second acceleration term, i.e., Eq. 12:

$$PS{E}_{\text{BALL}}=-m\ddot{x}\ell \sin \theta +m\left|\ddot{x}\right|\ell .$$ (A40)

Definition of Escape Energy

We define the escape energy E_ESC as the amount of total ball energy that, if exceeded, will result in the ball escaping from the cup sometime in the future under the assumption of constant cup acceleration ẍ. To find the limit E_ESC, we know that if the pendulum angle is equal to the escape angle (θ = θ_ESC), then the ball kinetic energy must be zero. Therefore, we only need consider the net potential energy as defined earlier,

$$PS{E}_{\text{BALL}\left(\text{Net}\right)}=P{E}_{\text{G}}+PS{E}_{\text{BALL}}=mg\ell \left(1-\cos \theta \right)-m\ddot{x}\ell \sin \theta +m\left|\ddot{x}\right|\ell ,$$ (A41)

and insert the θ_ESC for the angle to give E_ESC,

$$E_{\text{ESC}}=mg\ell \left(1-\cos {\theta }_{\text{ESC}}\right)-m\ddot{x}\ell \sin {\theta }_{\text{ESC}}+m\left|\ddot{x}\right|\ell .$$ (A42)

Note that there is an absolute value sign on the last term, which ensures a correct reference for the pseudo energy as explained in the previous paragraph. There is also the possibility that the cup (cart) may be accelerating in the negative direction. In this case the relevant escape angle is negative (−θ_ESC), and a conditional statement is needed to account for the changing escape angle sign, for instance,

$$E_{ESC}=\{\begin{array}{cc}\ddot{x}\ge 0& E_{ESC}=mg\ell \left(1-\cos {\theta }_{ESC}\right)-m\ddot{x}\ell \sin \left({\theta }_{ESC}\right)+m\left|\ddot{x}\right|\ell \\ \ddot{x}<0& E_{ESC}=mg\ell \left(1-\cos {\theta }_{ESC}\right)-m\ddot{x}\ell \sin \left(-{\theta }_{ESC}\right)+m\left|\ddot{x}\right|\ell .\end{array}$$ (A43)

This conditional set of equations can be simplified by taking the absolute value sign of the first acceleration term (and using the positive escape angle, θ_ESC) to arrive at a more compact form, giving Eq. 14:

$$E_{\text{ESC}}=mg\ell \left(1-\cos {\theta }_{\text{ESC}}\right)-m\left|\ddot{x}\right|\ell \sin {\theta }_{\text{ESC}}+m\left|\ddot{x}\right|\ell .$$ (A44)

If TE_BALL > E_ESC, then the ball will escape; otherwise, the ball will remain in the cup (assuming constant cup acceleration ẍ).

Definition of Escape Envelope

All possible combinations of ball states (θ, θ̇) and constant cup accelerations ẍ that result in the ball staying within the cup for infinite time form a region in the task space that is bounded by an “escape energy manifold” (Fig. 7). Outside the manifold, the ball will escape after some time. For a given constant ẍ, the equilibrium angle of the pendulum θ_EQ will shift to a new angle, defined as

$${\theta }_{EQ}={\tan }^{-1}\left(\ddot{x}/g\right).$$ (A45)

If the pendulum state is θ = θ_EQ, θ̇ = 0, and ẍ = g tan θ_EQ, then the pendulum will not oscillate but will remain exactly at the equilibrium angle. In all other cases, the ball will oscillate. Whether or not the ball stays in the cup is dependent on the size of the oscillations relative to the size of the cup. For a given constant ẍ, the minimum and maximum ball angles θ_MIN and θ_MAX (which define the size of the oscillation) that allow the ball to remain in the cup are

$$\left[{\theta }_{\text{MIN}},{\theta }_{\text{MAX}}\right]={\theta }_{\text{EQ}}+\left({\theta }_{\text{ESC}}-{\theta }_{\text{EQ}}\right),$$ (A46)

and the minimum/maximum ball angular velocity θ̇_MIN and θ̇_MAX is

$$\left[{\dot{\theta }}_{\text{MIN}},{\dot{\theta }}_{\text{MAX}}\right]=\pm \sqrt{\frac{2}{\ell }\left[g\left(\cos {\theta }_{\text{ESC}}-\cos {\theta }_{\text{EQ}}\right)+\ddot{x}\left(\sin {\theta }_{\text{ESC}}-\sin {\theta }_{\text{EQ}}\right)\right]}.$$ (A47)

Energy Margin

As long as the ball energy stays below the escape energy (TE_BALL < E_ESC), the ball will stay below the escape angle θ_ESC, assuming ẍ remains constant. An energy margin can be defined as

$$EM=\left(E_{\text{ESC}}-T{E}_{\text{BALL}}\right)/E_{\text{ESC}}.$$ (A48)

The difference E_ESC − TE_BALL represents how close the current ball energy is to exceeding the escape energy. This quantity is normalized to E_ESC to ease interpretation: if EM is between 0 and 1, the ball will not escape, but it will if EM is negative, assuming ẍ is not changed. Note that E_ESC depends on ẍ and therefore changes during cup transportation. For reference, alternative normalizations can be used. For example, a different energy margin EM′ could be defined as

$$EM\prime =\left(E_{\text{ESC}}-T{E}_{\text{BALL}}\right)/\left(E_{\text{ESC}}-E_{\text{EQ}}\right).$$ (A49)

where

$$E_{EQ}=mg\ell \left(1-\cos \theta {\prime }_{\text{EQ}}\right)-m\left|\ddot{x}\right|\ell \sin \theta {\prime }_{\text{EQ}}+m\left|\ddot{x}\right|\ell$$ (A50)

and

$$\theta {\prime }_{\text{EQ}}+{\tan }^{-1}\left(\left|\ddot{x}\right|/g\right).$$ (A51)

The equation for E_EQ is the same as that for E_ESC with θ_ESC replaced by θ′_EQ. Whereas EM normalizes the energy difference E_ESC − TE_BALL to the critical escape energy E_ESC, EM′ normalizes E_ESC − TE_BALL to the difference between E_ESC and the “least critical energy state,” E_EQ, which is when the ball is at the equilibrium angle θ′_EQ, assuming constant ẍ.