Implicit Guidance to Stable Performance in a Rhythmic Perceptual-Motor Skill

Abstract

Feedback information about error or reward is regarded essential to aid learners to acquire a perceptual-motor skill. Yet, simple error feedback does not suffice in guiding the learner towards the optimal solutions, when tasks have redundancy where the mapping between execution and performance outcome is unknown. The present study developed and tested a new means of implicitly guiding learners to acquire a perceptual-motor skill, rhythmically bouncing a ball on a racket. Due to its rhythmic nature, this task affords dynamically stable solutions that are resistant to small errors and noise, a strategy that is independent from simply reducing error. Based on the task model implemented in a virtual environment, a state-dependent manipulation was designed that shifted the range of ball-racket contacts that achieved to dynamically stable solutions. In two experiments, subjects practiced with this manipulation that guided them to impact the ball with more negative racket accelerations, the indicator for the strategy with dynamic stability. Subjects who practiced under normal conditions took longer time to acquire this skill, although error measures were identical between the control and experimental groups. Unlike in many other haptic guidance or adaptation studies, the experimental groups not only learned but also maintained the stable solution after the manipulation was removed. These results are a first demonstration that more subtle ways to guide the learner to better performance are needed to assist performance improvements, especially in tasks with redundancy, where error feedback may not be sufficient.

Introduction

From learning to walk or playing a drum set, to relearning to write after a stroke, humans acquire a vast collection of motor skills over their lifetime. Many strive to improve advanced motor skills, while others have to relearn basic skills, such as coordinating a knife and fork to eat. Consequently, much research has been dedicated to understand and facilitate motor learning and recovery. Following decades of research on operant conditioning or reinforcement learning in the 1940-60s, a host of subsequent studies adopted a more cognitive approach to enhance skill learning, many under the umbrella of schema theory or generalized motor programs (Bilodeau 1966; Bilodeau 1969; Schmidt 1975; Newell 1976; Adams 1987; Magill and Anderson 2010; Schmidt and Lee 2011). Despite many unresolved issues, one consensus from these studies was that presenting augmented (as opposed to only intrinsic) information about the outcome following the action, or knowledge of results, enhances performance and learning (Salmoni et al. 1984). Numerous experiments aimed to identify its optimal frequency, temporal delay, and precision of delivery that may lead to optimal performance, although few generalizable results arose. This quantitative terminal feedback was sometimes separated from knowledge of performance, defined as augmented kinematic and kinesthetic information about the movement itself (Newell and Walter 1981). However, only relatively few studies were conducted to further detail its potential benefit, probably due to the lack of virtual technology.

More recently, error-based learning and feedback information have been studied in a discrete reaching paradigm requiring adaptation to external force fields and visuomotor rotations. Using virtual environments, several studies validated that presenting continuous and terminal error information aids in learning the new visual-motor remapping or the compensation of external forces (e.g., Hinder et al. 2008; Shabbott and Sainburg 2010). Besides visual information about the endpoint effector, additional proprioceptive information was examined to support and accelerate adaptation (Scheidt et al. 2005; Scheidt et al. 2010). More recently, the role of terminal reinforcement or reward scores and credit assignment has been highlighted (Abe et al. 2011; Wolpert et al. 2011; Galea et al. 2015). Characteristic to these studies is that adaptations to new conditions faded relatively quickly, when the perturbation was removed. Further, regardless of the error information, subjects adapted to new rotations or force fields in as few as 10 trials, and return to initial performance (after-effects) has been even shorter (Kitago et al. 2013). These fast performance changes highlight that reaching and its transformations are easy to learn and present relatively little challenge. The correct solution is uniquely defined by the target location and the desired straight-line trajectory, which makes error information very effective (Scheidt et al. 2005; Scheidt et al. 2010).

In contrast to this adaptation of reaching movements, acquiring a novel skill takes considerably longer, from weeks to years, and presenting knowledge of results is not as straightforward and effective (Crossman 1959; Cesqui et al. 2012; Park et al. 2013; Park and Sternad 2015). Any athlete or coach knows that a single-numbered score, as for example given by judges, is not sufficient information to improve performance in a complex motor skill. After all, finding ways that shape behavior towards a desired goal is what makes or breaks a good coach. By definition, in a novel skill the relationship between motor execution and task outcome is unknown and the mapping is typically very complex. Not only does the redundancy in the neuro-mechanical system afford infinitely many ways to produce the desired movement, the task itself can also be achieved in infinitely many ways and performance improvement may proceed in a non-monotonic fashion. The manifold of solutions in the result space is typically nonlinear, with some solutions more favorable than others (Sternad et al. 2011; Berret et al. 2011; Campolo et al. 2013; Ganesh and Burdet 2013; Sternad et al. 2014). Specifically, some solutions tend to be more tolerant to perturbations and noise.

Previous research by Sternad and colleagues highlighted this redundancy in a discrete throwing task, where the set of zero-error solutions described a nonlinear manifold (Müller and Sternad 2004; Sternad et al. 2010). When practice was examined up to 16 days, invariably noise-tolerant solutions were found first (Cohen and Sternad 2009; Cohen and Sternad 2012; Abe and Sternad 2013). In a similar vein, work on a rhythmic ball bouncing task highlighted that zero-error performance can be achieved in a multitude of ways (Schaal et al. 1996; Sternad et al. 2000b; Dijkstra et al. 2004; Wei et al. 2007; Wei et al. 2008). Due to its rhythmic nature, there exists a subset of solutions that are dynamically stable, i.e. where small errors or noise are automatically corrected without requiring explicit corrections. These “smart” solutions attenuate the propagation of error and are therefore computationally less demanding. Several studies showed that such dynamically stable solutions require practice and are the hallmark of expert performance (Wei et al. 2008). Error scores alone do not reflect this advantageous strategy, and hence, cannot be the only feedback driving performance improvement (Huber et al. 2015). Therefore, other implicit information or guidance is needed to “coach” the subject. How can subjects be guided to such dynamically stable solutions?

To date, only relatively few studies in motor neuroscience have used the virtual environment to design interventions that shape behavior in implicit ways, guiding performance to achieve lasting changes in skill. One way to enhance awareness of the desired motion is amplifying the error in the visual feedback. Patton and colleagues have demonstrated the benefit of this method for healthy subjects and also stroke patients, although their task consisted of straight-line reaches, and after-effects remained a critical issue when using the adaptation paradigm (Hasson et al.; Wei et al. 2005; Patton et al. 2006; Reinkensmeyer and Patton 2009; Milot et al. 2010; Sharp et al. 2011). Using a shuffleboard task, Chu et al. (2013) manipulated the variability of the puck releases and showed that behavior can be shaped: decreasing variability by filtering over past trials improved performance, even in severely handicapped children, while increasing variability induced healthy children to perform with more risk awareness. Note that the shuffleboard task was constrained to have no redundancy. Using a redundant line reaching task, (Manley et al. 2014) attempted to guide subjects to reach in directions for which they receive a monetary reward by adding noise to undesired reaching directions. However, the added noise did not help subjects find the most robust solutions as initially expected. Interestingly, not even a gradient in the noise amplitude guided subjects towards the desired direction, except when subjects were made explicitly aware that noise was added.

It is noteworthy that all of these prior studies on learning and adaptation were focused on discrete movements and none have looked at rhythmic movements. Rhythmic movements are ubiquitous, ranging from locomotion to many forms of tool use. Behavioral, modeling, and neuroimaging results have shown that rhythmic movements follow different principles and may constitute a different “primitive” (Sternad et al. 2000a; Schaal et al. 2004; Ronsse et al. 2009; Ikegami et al. 2010; Howard et al. 2011; Sternad et al. 2013) Hence, it is likely that also the underlying mechanisms of learning are different. For example, explicit quantitative feedback was not needed to acquire and retain a bimanual rhythmic skill (Park et al. 2013; Park and Sternad 2015).

This research pursues to develop interventions that accelerate novel skill learning of a rhythmic task in a virtual environment. Going beyond simple reward signals, the study aims to identify elements of performance in a redundant skill to guide learners towards desired dynamically stable and noise-tolerant solutions. The experiment capitalizes on previous research on the task of rhythmically bouncing a ball with a racket that has shown to have redundancy with a subset of solutions displaying dynamic stability (Schaal et al. 1996; Sternad et al. 2000b; Dijkstra et al. 2004; Ronsse and Sternad 2010). In this task, the subject is instructed to rhythmically bounce a virtual ball to a target line using a real racket. Mathematical analysis of a model system indicated dynamically stable solutions, when the racket hit the ball in the decelerating upward swing, assuming sinusoidal motion. This strategy is advantageous because the performer need not adapt his/her racket movements to every small deviation of the ball to maintain successful performance.

Prior experiments by Sternad and colleagues have shown that while novices initially hit the ball with positive racket acceleration, they learn to exploit dynamic stability, as indicated by a shift to negative racket acceleration at impact after approximately 30 minutes of practice (Sternad et al. 2001; Ehrlenspiel et al. 2010; Huber et al. 2015). Interestingly, performers were not aware of their control of contact, nor of their change in strategy. Dynamic stability has been shown in other rhythmic tasks, the most notable of which is locomotion. This finding is robust across different types of gait, as many animals, including guinea fowl and cockroaches, also exploit dynamic stability in locomotion (Ting et al. 1994; Daley et al. 2006). In fact, people who are at increased risk of falling, such as amputees or patients with sensory neuropathy, walk slower to improve dynamic stability of the upper body during level walking, even at the cost of increased variability (Dingwell et al. 2000; Dingwell and Marin 2006; Beurskens et al. 2014).

The purpose of this study was to design a manipulation in the ball bouncing task that guides subjects to these dynamically stable solutions earlier in practice. How can subjects be made aware of these attractor solutions? One approach is to physically shape the subject’s movement during the task using robotic assistance, driving them to exploit the dynamically stable solutions. However, Marchal-Crespo et al. (2014) showed that such haptic guidance actually hampers learning in a similar rhythmic ball bouncing task. Another approach is to manipulate the task itself to guide behavior, without creating obvious dynamic perturbations such as force fields that subjects need to match exactly to be successful. As the ball bouncing task is performed in a virtual environment, the physical laws that generate the ball movements can be modified to manipulate the attractors in the task. Previously, Morice et al. (2007) showed that shifting the position and velocity of the virtual racket shifted the attractor solutions, but this manipulation created a new perceptual-motor mapping that needed to be matched. Hence, like in visuo-motor adaptation studies, the learned behavior disappeared immediately after removal of the manipulation.

The goal of this research is to guide subjects to better solutions in the present task. The important evaluation of success is that the learned behavior should persist after terminating the intervention. Before the intervention is detailed and the specific hypotheses formulated, the task, the model, and the proposed intervention will be laid out.

Experimental Methods and Design

The Task and the Model

In the experiment, the subject is instructed to rhythmically bounce a virtual ball to a target line using a real racket. This deceptively simple task requires considerable perceptually-guided coordination to intercept the ball at the right moment and with the right racket velocity to impart the necessary energy to the ball to hit the target height. The core challenge is that control of the ball is confined to the extremely short moments of ball-racket collisions. Further, because impacts occur in a repeating fashion, errors in one impact also determine the next contact: a higher ball amplitude leads to a higher ball velocity at the next contact, that will require a smaller racket velocity at the next contact to compensate. This model task exemplifies rhythmic interaction with an object as is pervasive in tool use, such as hammering, sawing, sweeping, and typing on keyboard.

The model for this task is a well-studied nonlinear dynamical system, originally developed for a particle bouncing on a vibrating surface and then used for a series of human studies (Guckenheimer and Holmes 1983; Tufillaro et al. 1992; Schaal et al. 1996; Sternad et al. 2001). This simple model consists of a planar surface moving sinusoidally in the vertical direction to repeatedly impact a ball (Figure 1). The vertical position of the virtual ball x_b between the kth and the k+1th racket-ball impact follows ballistic flight:

$$x_b\left(t\right)=x_b\left(t_k\right)+{\nu }_b^{+}(t-t_k)-g∕2{(t-t_k)}^2$$

where t_k is the time of the kth ball-racket impact, ${\nu }_b^{+}$ is the velocity of the ball just after impact, and g is the acceleration due to gravity (9.81 m/s²). To determine the ball velocity just after impact ${\nu }_b^{+}$, an instantaneous impact is assumed that has energy loss at the collision quantified by a coefficient of restitution α:

$$\alpha ({\nu }_b^{-}\left(t_k\right)-{\nu }_r^{-}\left(t_k\right))=-({\nu }_b^{+}\left(t_k\right)-{\nu }_r^{+}\left(t_k\right))$$

where ν_b and ν_r are the racket and ball velocities just before (−) and after (+) impact. Further, the mass of the racket is assumed to be much larger than the mass of the ball, such that the racket velocity does not change during impact:

$${\nu }_r^{-}\left(t_k\right)={\nu }_r^{+}\left(t_k\right)={\nu }_r\left(t_k\right).$$

Fig. 1.

Model of the racket-ball system. The vertical ball position between each instantaneous impact follows ballistic flight, which depends on three execution variables: ball ${\nu }_b^{-}$ and ${\nu }_r^{-}$ racket velocities just before impact and racket position x_r at impact.

Thus, the ball velocity just after impact is determined by:

$${\nu }_b^{+}\left(t_k\right)=(1+\alpha ){\nu }_r\left(t_k\right)-\alpha {\nu }_b^{-}\left(t_k\right).$$

By assuming sinusoidal racket motion, the racket and ball system is a continuous dynamical system. A Poincare section at the moment of the collision rendered a discrete map with two state variables, ball velocity just after impact ${\nu }_b^{+}$ and racket phase at impact θ_k.

Local linear stability analysis of this discrete map identified a fixed-point attractor, when racket acceleration at impact a^r satisfied the inequality (Schaal et al. 1996; Dijkstra et al. 2004):

$$-2g\frac{(1+{\alpha }^2)}{{(1+\alpha )}^2}<a_r<0$$

For α = 0.6 and g = 9.81 m/s², as in the experiment, the range with dynamic stability was between 0 and −10 m/s². Simulations of the ball bouncing map illustrate that when the impact occurs during negative racket acceleration of the upward racket swing (Figure 2A), the ball exhibits stable period-1 behavior. The map possesses other attractors besides the period-1 attractor, including “sticking” behavior, where the ball sticks to the racket and follows the racket trajectory. This “sticking” behavior results when the ball-racket impact occurs during positive racket acceleration (Figure 2A). Further, non-local Lyapunov stability analyses narrowed the range of acceleration values to −2m/s² to −5m/s² for the given parameters α and g (Schaal et al. 1996), although these values are not hard boundaries.

Fig. 2.

(A) Simulation of the ball-racket system. Assuming sinusoidal racket movement, the racket trajectory has a segment with positive acceleration followed by negative acceleration before its peak position during the upward swing. Ball trajectories with initial contact at different phases of the upward swing (from 1.51π rads to 2.49π rads at an interval of .3 rads) were simulated. When the racket impacted the ball during the decelerating portion of the racket’s upward motion, the ball-racket system was dynamically stable. All simulations with initial contact during negative racket accelerations led to the same stable ball amplitude without requiring any changes in the racket trajectory. If the ball impacted the racket during the accelerating portion of the racket’s upward motion, the system was unstable. All simulations with initial contacts during positive racket acceleration led to unstable behavior, where the ball finally stuck to the racket. The only way to achieve and maintain a stable pattern was to correct for errors in the ball amplitude by a change in the racket trajectory. (B) Simulation of the ball-racket system with time-dependent perturbation. If the ball impacted the racket during the accelerating portion of the racket’s upward motion, the system was unstable. When the racket impacted the ball during the decelerating portion of the racket’s upward motion, the system was unstable if the racket phase was less than .05ω; otherwise, the perturbed ball-racket system was dynamically stable.

Unlike the ball bouncing map that only describes feedforward dynamics, novice participants who hit with positive racket acceleration are able to compensate for the errors arising from such unstable performance. Based on visual information about the error, they can actively correct for errors by adjusting their racket trajectory to propel the ball either higher or lower than the previous bounce (de Rugy et al. 2003; Wei et al. 2007; Siegler et al. 2010). However, with practice, participants learn to hit the ball with negative acceleration. By exploiting this efficient solution, small errors need not be corrected, reducing the necessity for active correction (Wei et al. 2008).

In the virtual environment, the model was rendered exactly and there were no uncontrolled aspects or simplifying assumptions, such as drag or spin, as would occur in a real experiment.

Design of the Intervention

Given this known dynamics, the question is how the system can be tweaked to guide subjects to more stable behavior? The two complementary principles are penalizing undesirable solutions or enhancing desirable solutions. As Figure 2A showed, contact points that ensure stability were on the upper segment of the upward sinusoidal trajectory. To be exact, the period-1 attractor arose from hitting at a segment with racket acceleration between −0 and −10m/s² (Figure 3A).

Fig. 3.

Dynamically stable solutions (gray) depicted in on the assumed sinusoidal racket trajectory under the (A) control condition and (B) time-shifted racket velocity condition.

To encourage novices to hit with more negative racket acceleration, the period-1 attractor was “shifted” to a segment of the racket trajectory with more negative acceleration (Figure 3B). This manipulation of the dynamically stable solutions was achieved by using the racket velocity 50ms prior impact, as opposed to the racket velocity at impact, to determine the release velocity of the ball. The temporal shift not only made the negative racket acceleration regions more dynamically stable, but also made the positive racket accelerations more unstable, leading to sticking solutions. For the implementation, the racket velocity at impact ν_r was set equal to the racket velocity 50ms before the time of impact t_r. Hence, the ball velocity just after impact was determined by:

$$v_b^{+}=(1+\alpha ){\nu }_r(t_k-.05)-\alpha {\nu }_b^{-}\left(t_k\right).$$

As in the original map, the ball exhibited “sticking” behavior, if the impact occurred during the positive racket accelerations (Figure 2B). However, there was an additional segment of 50ms that produced sticking solutions, where subjects were “penalized”. Only the more negative racket accelerations continued to produce stable period-1 behavior. Given prior findings that humans seek dynamically stable solutions, we expected that subjects performing the task with this manipulation would hit with more negative racket acceleration compared to subjects performing the task under the normal condition.

To test that this state-determined intervention did not only add noise to the ball release, at least in the perception of the subject, but did contain the intended directional information, an additional control condition was included. In this condition, random noise was added to the racket velocity before calculating the ball trajectory. At each bounce, a value was drawn from a Gaussian distribution with mean zero. For better comparison with the time-shifted condition, the standard deviations of the Gaussian were matched with the ones obtained from the velocity shifts in the experimental condition: σ=0.4 m/s.

Based on these manipulations, the present study tested the following three hypotheses: (1) Subjects who practice with the time-shifted racket velocity learn to hit with negative racket acceleration earlier in practice, compared to those who practiced with no manipulation. (2) After removing the manipulation, they have a higher degree of dynamic stability than those who practice under normal or noise conditions. (3) The attained dynamic stability persists throughout extended practice. Two experiments were conducted to test these hypotheses. In the first experiment, the experimental group practiced for 6 blocks of 4 trials with the manipulation followed by one test block without the manipulation. The purpose of this experiment was to assess the effect of the time-shifted racket velocity on learning and if performance changed upon the removal of the time-shift manipulation. In the second experiment, the experimental group practiced for 4 blocks followed by 3 test blocks without the manipulation. This experiment further assessed how long this enhanced performance persisted after removing the manipulation.

Participants

A total of 29 students (15 males and 14 females, mean age 20.45 ±3.31 years) from Northeastern University participated in the experiment after signing the consent form approved by the Institutional Review Board of Northeastern University. They received partial fulfillment of a course requirement in exchange for their participation. None had any prior experience with the virtual ball bouncing task. They all self-reported to be right-hand dominant and performed the task with their dominant hand.

Experimental Task and Apparatus

The participant manipulated a real table tennis racket to rhythmically bounce a virtual ball to a target height in a 2D virtual environment (Figure 4A). The participant stood 2m in front of a rear projection screen holding the racket in his or her dominant hand. A light rigid rod with two hinge joints was attached to the racket surface, which could translate in the vertical direction while free to tilt around all three axes. The latter was included to minimize friction. However, only the vertical component of the racket displacement moved the virtual racket. As the virtual ball movement was confined to the vertical dimension, the racket movements did not deviate very much in other directions. To measure vertical racket displacement, the rigid rod moved a wheel, whose rotations were registered by an optical encoder at a sampling rate of ~500Hz with a spatial resolution of 0.27mm (Bourns Inc, Riverside, CA). In addition, a wireless accelerometer attached to the center of the racket surface measured the racket accelerations directly at a sampling rate of ~250Hz (Myon 320, Schwarzenberg, Switzerland). A PC (2.4-GHz Pentium CPU, Windows XP) controlled the experiment and generated the visual stimuli with a graphics card (Radeon 9700, AMD, Sunnyvale, CA). The same PC also acquired the data using a 16-bit A/D card (NI-USB6229BNC, National Instruments, Austin, TX). The delay between real and virtual racket movement was measured in a separate experiment and was on average 22 ± 0.5ms. The images were displayed by a rear-projector (DepthQ-WXGA, Lightspeed Design, Bellevue, WA) consisting of 1024 × 768 pixels with a 60Hz refresh rate.

Fig. 4.

(A) Side and front view of the virtual experimental setup for ball bouncing. Participants were positioned in front of a screen and manipulated a real table tennis racket to rhythmically bounce a virtual ball to a target height in a 2D virtual environment. (B) Schematic of Experiment 1 design. Each group performed of 6 practice blocks with respective manipulations to racket velocity and one test block with no manipulation. Each block consisted of 4 trials, and each trial lasted 40 seconds long. (C) Schematic of Experiment 2 design. The time-shifted group performed of 4 practice blocks and 3 test blocks with no manipulation. The control group was the same from Experiment 1.

The vertical ball position, and consequently the maximum ball height of each bounce, was fully determined by the ball velocity, racket velocity, and racket position at impact. In the task model, the coefficient of restitution was set to 0.6. Just before the virtual ball hit the virtual racket, a trigger signal was sent out to a mechanical brake that acted on the rod. The brake was controlled by a solenoid and applied a brief braking force pulse to the rod to create the feeling of a real ball hitting the racket (Magnet-Schultz type R 16 ×16 DC pull, subtype S-07447). The trigger signal was sent 15ms before the ball–racket contact to overcome the mechanical and electronic delay of the solenoid and brake. The duration of the force pulse (30ms) was consistent with the average impact duration observed in a real ball–racket experiment (Katsumata et al. 2003).

Figure 4 shows the virtual environment displayed on the projection screen, which consisted of a virtual racket (horizontal red line, 0.2m × 0.02m), target line (horizontal yellow line, 1.0m × 0.02m), and ball (white circle, 0.02m radius). The vertical position of the virtual racket was determined by the measured position of the real racket; the target line was positioned 1.0m above the minimum racket position. At the start of each trial, the ball was positioned at the left edge of the screen atop the target line and proceeded to roll along the target line towards the center of the screen. Once it reached the edge of the target line, the ball dropped towards the racket. The participant was instructed to bounce the ball for the duration of the 40s trial such that the maximum ball height of each bounce was coincident with target line. The trajectory of the virtual ball after the ball-racket impact was determined using the ballistic flight and instantaneous impact equations above.

Design and Procedure

In both experiments, all subjects performed a total of 7 blocks, with 4 trials per block, leading to a total of 28 trials. As the duration of each trial was 40s, with a brief break between blocks, the entire experiment lasted approximately 35 minutes.

In Experiment 1, 9 subjects practiced the task with the manipulated racket velocity (time-shifted group), and 9 subjects practiced with no manipulation (control group) (Figure 4B). An additional group of 3 subjects practiced with a random noise term added to the racket velocity at impact (noise group). The noise group served as supplementary control to determine that it was the time-dependent nature of the manipulation that presented implicit guidance to dynamic stability, not noise alone. Only 3 subjects were collected as it became immediately evident that they did not understand the result of their actions, and hence, did not change performance. As this random condition was extremely frustrating to subjects, we stopped after collecting 3 subjects. After the 6 practice blocks, all subjects performed one test block of 4 trials with no manipulation to the racket velocity. Subjects were not informed that the conditions changed between the practice blocks and test block.

In order to further assess how much practice with the manipulation was needed to establish dynamic stability and how long this persisted, a second experiment was conducted. In Experiment 2, 9 subjects practiced with manipulated racket velocity (time-shifted group) for 4 practice blocks, followed by 3 test blocks (Figure 4C). The same control group from the first experiment was used for comparison.

Dependent Measures

Error was defined as the signed distance between the maximum ball amplitude and the target line. Absolute error was defined as the absolute value of error. The median of absolute error and the interquartile range of the signed error over all bounces in each 40s trial served as performance measures. Shapiro-Wilk tests revealed that on average, the distribution of error and absolute error in each block deviated from normal distribution in approximately 14.5 and 26 out of 28 blocks, respectively, for each subject (Shapiro and Wilk 1965). Thus, the median and interquartile ranges of these measures were used.

As introduced above, racket acceleration at impact served as the measure of dynamic stability. The racket acceleration signal was measured by an accelerometer on top of the real racket. The signal was then resampled at a fixed frequency of 500 Hz and filtered by a fourth-order Savitzky-Golay filter with a window size of 10ms on both sides (Savitzky and Golay 1964). Racket acceleration at impact was defined as the racket acceleration in the y-direction 6ms prior to ball-racket impact to avoid capturing any artifacts due to the activation of the mechanical brake. Given that impacts occurred in the upward movement, this temporal interval was conservative, as it biased the estimated value towards positive values. Again, median racket acceleration at impact over all bounces in each 40 s trial was used as Shapiro-Wilk tests revealed that the distribution of racket accelerations in each block was not normal in approximately 9.7 out of 12 blocks for each subject.

The dependent measures of each trial were averaged across trials and blocks for each experimental group.

Statistical Analyses

Experiment 1

To assess the effect of the time-shifted racket velocity on learning a 2 (group: control vs. time-shifted) × 6 (block: practice blocks 1 through 6) ANOVA was conducted on each dependent measure (Hypothesis 1). To assess performance changes from the last practice block to the test block, a two-way 2 (group: control vs. time-shifted) × 2 (block: practice block 6 vs. test block) ANOVA was conducted on each dependent measure (Hypothesis 2). Only the time-shifted and control groups were considered in the statistical analyses. The noise group was omitted, as it had only a very limited number of participants.

Experiment 2

As in Experiment 1, a 2 (group: control vs. time-shifted) × 4 (block: practice blocks 1 through 4) repeated-measures ANOVA was conducted on each dependent measure (Hypothesis 1). A 2 (group: control vs. time-shifted) × 2 (block: practice block 4 vs. test block 1) ANOVA was conducted on each dependent measure to assess performance changes upon removing the time-shift manipulation (Hypothesis 2). An additional 2 (group: control vs. time-shifted) × 3 (block: test blocks 1 through 3) ANOVA was conducted on each dependent measure to further assess the persistence of performance after removing the time-shift manipulation (Hypothesis 3).

For all ANOVAs, group was a between-subjects factor and block was a within-subjects factor, and the Greenhouse-Geisser correction factor was applied to the within-subject effects (Greenhouse and Geisser 1959). The significance level was set to α = 0.05. A test of simple effects was calculated a significant interaction was present. In Experiment 2, one participant in the time-shifted group had an average median racket acceleration at impact across trials that was more than 4 standard deviations above the mean of the time-shifted group. This participant was excluded from the statistical analyses.

Results

Experiment 1

Exemplary time series

Figure 6 shows exemplary trials from one subject in each experimental group. Each trial lasted 40s and typically contained between 40-50 bounces. The trials shown in Figure 6 are the first trial in practice block 1, the last trial in practice block 6, and the first trial in the test block immediately after the manipulations to racket velocity was removed. The data illustrate that despite the initially perturbed performance in the time-shifted group, subjects achieved consistent performance by block 6. This performance persisted without visible change in the test block. The trials of the noise group illustrate how the added noise could lead to sticking solutions throughout practice. Figure 7A-C shows the group means of the dependent variables racket acceleration, absolute error and variability of error, plotted across the 6 practice blocks and the 1 test block for the 3 experimental groups.

Fig. 6.

Exemplary trials of three subjects (one per experimental group) in practice blocks 1 and 6 and the test block. The first 25s of the 40s time series of racket (gray) and ball (black) position are depicted. The dashed line indicates the target height.

Fig. 7.

Dependent measures of Experiment 1 over blocks. The measure of each trial were averaged across blocks and finally averaged over the subjects in each experimental group. Each point represents the group average per block, and the error bar represents the standard error across subjects in each group. (A) Median racket acceleration. The dynamically stable solutions between −2m/s² and −5m/s² (shaded region below the dashed line) are desired. (B) Median absolute error. (C) Interquartile range of signed error.

Learning with time-shift manipulation

Consistent with prior results, subjects of all 3 groups initially hit with positive racket acceleration in block 1 (time-shifted: M = 2.66m/s², SD = 4.87m/s²; control: M = 2.40m/s², SD = 2.06m/s²). In subsequent blocks, all subjects decreased their racket acceleration, albeit at different rates and to different degrees (Figure 7A). Hypothesis 1 stated that subjects who practice with the time-shifted racket velocity learn to hit with negative racket acceleration earlier in practice, compared to those who practiced with no manipulation. The ANOVA on median racket acceleration at impact revealed a significant main effect of block, F(1.65, 26.42) = 28.69, p < .001, as well as group, F(1, 16) = 8.03, p = .012. These main effects were qualified by a significant interaction between group and block, F(1.65, 26.42) = 3.65, p = 0.047. A test of simple effects revealed that the time-shifted group had significantly lower racket accelerations at impact compared to the control group in blocks 3 through 6 (ps < .015). These results indicate that the time-shifted manipulation could guide subjects to hit with negative racket acceleration earlier in practice (Hypothesis 1). While the noise group was not subjected to statistical testing, the median racket acceleration at impact remained positive over practice, indicating that the time-dependent nature of the manipulation was necessary for the desired change in behavior (Figure 7A).

To determine if practicing with the time-shifted racket velocities interfered with task performance, ANOVAs on error measures were conducted. The ANOVA on median absolute error revealed a main effect of block, F(1.68, 26.88) = 16.2, p < 0.001. As expected, median absolute error decreased with practice for both groups (Figure 7B). However, neither the main effect of group, F(1, 16) = 1.58, p = .23, nor the Group × Block interaction, F(1.68, 26.88) = 1.45, p = .25, were significant. The statistical results of the mean absolute error were mirrored in the analysis of the interquartile range of error, as seen in Figure 7C. The ANOVA on interquartile range of error revealed a significant main effect of block, F(3.92, 52.61) = 11.11, p < 0.001, but not for group, F(1, 16) = 1.35, p = .26. The Group × Block interaction was also non-significant, F(3.92, 52.61) = .67, p = .59. In short, while both groups improved overall task performance over time, there were no significant differences between the time-shifted and control groups in these error measures. Visual inspection of Figures 7B-C shows that the noise group has performed visibly worse over the course of practice compared to the control and time-shifted groups.

Removing time-shift manipulation

To determine whether performance changed upon removal of the time-shifted manipulation, the dependent measures in practice block 6 and the test block were compared (Figure 7A-C). The ANOVA on median racket acceleration at impact revealed a significant main effect of group, F(1, 16) = 18.44, p = .001, but not for block, F(1, 16) = 1.17, p = .30. The analysis also revealed a significant interaction between group and block, F(1, 16) = 7.91, p = 0.013. A test of simple effects revealed that the time-shifted group significantly increased racket acceleration at impact from practice block 6 (M = –4.61m/s², SD = 1.1m/s²) to the test block (M = –3.73m/s², SD = 1.13m/s²), p = .014, whereas the control group did not, p = .24. While the significant increase in racket acceleration in the timed-shifted group was inconsistent with Hypothesis 2, it should be kept in mind that, while statistically different, the acceleration values did not differ much in their physical meaning. As stated above, a wide range of negative values ensures dynamic stability and an “optimal region” as determined by Lyapunov stability analysis spans –2 to –5m/s² (Schaal et al. 1996). While the time-shifted group hit with racket acceleration values in this optimal range during the test block (M = –3.73m/s², SD = 1.13m/s²), the control group did not (M = –1.20m/s², SD = 2.06m/s²).

The ANOVA on median absolute error did not reveal any significant main effects (block: F(1,16) = 1.39, p = .26; group: F(1,16) = .013, p = .91), nor a significant interaction, F(1,16) = 2.09, p = .17. Similarly, the ANOVA on the interquartile range of error did not reveal any significant main effects (block: F(1,16) = .53, p = .48; group: F(1, 16) = .007, p = .94), nor a significant interaction, F(1, 16) = 1.52 , p = .24. Thus, removing the manipulation did not interfere with task performance.

Experiment 2

Figure 8A-C shows the group means of dependent measures plotted across the 4 practice blocks and the 3 test blocks. The control group was the same group from Experiment 1.

Fig. 8.

Dependent measures of Experiment 2 over blocks. The measure of each trial were averaged across blocks and finally averaged over the subjects in each experimental group. Each point represents the group average per block, and the error bar represents the standard error across subjects in each group. (A) Median racket acceleration. The dynamically stable solutions between −2m/s² and −5m/s² (shaded region below the dashed line) are desired. (B) Median absolute error. (C) Interquartile range of signed error.

Learning with time-shift manipulation

Just as in Experiment 1, the time-shifted group initially hit with positive racket acceleration in block 1 (M = 2.20m/s², SD = 3.81m/s²). The ANOVA on median racket acceleration at impact in the 4 practice blocks revealed a significant main effect of block, F(1.92, 28.76) = 16.07, p < .001 (Figure 8A). The main effect of group did not reach significance, F(1, 15) = 3.63, p = .076. These effects were qualified by a significant interaction between Group and Block, F(1.92, 28.76) = 3.44, p = .047. A test of simple effects revealed that the time-shifted group had significantly lower racket acceleration at impact compared to the control group in blocks 2 and 4 (ps < .015). This result indicates that even after just 4 practice blocks, the time-shifted manipulation guided subjects to hit with negative racket acceleration earlier in practice (Hypothesis 1).

The ANOVA on median absolute error revealed a main effect of block, F(1.91, 28.62) = 18.05, p < .001 (Figure 8B). However, neither the main effect of group, F(1, 15) = 2.54, p = .13, nor the Group × Block interaction, F(1.91, 28.62) = 1.63, p = .21, were significant. The ANOVA on interquartile range of error similarly revealed a significant main effect of block, F(2.46, 36.91) = 3.04, p = .050, but not for group, F(1, 15) = .32, p = .58 (Figure 8C). The Group × Block interaction also was not significant, F(2.46, 36.91) = 1.17, p = .33. As in Experiment 1, there were no significant differences between the time-shifted and control groups in these error measures over the practice blocks.

Removing time-shift manipulation

To determine whether performance changed upon removal of the time-shifted manipulation, the dependent measures in practice block 4 and the test block 1 were compared (Figure 8A-C). The ANOVA on median racket acceleration at impact revealed a significant main effect of group, F(1, 15) = 7.94, p = .013. Pairwise comparisons revealed that in the time-shifted group hit with significantly lower racket acceleration than the control group in both practice block 4 and test block 1 (ps < .04) (Hypothesis 2). The main effect of block was marginally significant, F(1, 15) = 4.47, p = .052, and the interaction between group and block was not significant, F(1, 15) = .001, p = .98.

The ANOVA on median absolute error revealed a significant main effect for block F(1, 15) = 4.60, p = .049, but not for group, F(1, 15) = 1.43, p = .25. The interaction was also not significant F(1, 15) = .33, p = .57. Similarly, the ANOVA on interquartile range of error revealed a significant main effect for block, F(1, 15) = 5.93 p = .028, but not for group, F(1, 15) = .36, p = .56. The interaction was also not significant, F(1, 15) = .64, p = .44. Consistent with Experiment 1, these results indicate that removing the manipulation did not interfere with task performance.

Persistence

In Experiment 2, the time-shifted group performed 3 test blocks under normal task conditions after removing the perturbation. The ANOVA on median racket acceleration at impact in the 3 test blocks revealed a significant main effect of group, F(1, 15) = 7.71, p < .014, as seen in Figure 8A (Hypothesis 3). Pairwise comparisons revealed that the time-shifted group hit with significantly lower racket acceleration than the control group in test blocks 1 and 2 (ps < .035). Neither the main effect for block, F(1.29, 19.39) = 1.16, p = .31, nor the interaction between group and block were significant, F(1.29, 19.39) = 3.68, p = .061. As previously reported, by the end of the experiment, the control group hit with racket acceleration values that were still outside the optimal region of dynamic stability. Like in Experiment 1 and consistent with Hypothesis 3, the time-shifted group in Experiment 2 learned and maintained hitting with racket acceleration at impact in this region until the end of the experiment (M = −2.61m/s², SD = 1.55m/s² in test block 3).

The ANOVAs on median absolute error and interquartile range of error did not reveal significant main effects for block and group, nor interactions in the test blocks, Fs < 1, ps > .50.

Discussion

This study examined how the manipulations of a task in a virtual environment can implicitly guide subjects towards a desired behavior that also persisted after removal of the guidance. Using the rhythmic perceptual-motor skill of bouncing a ball, this experiment introduced a task-based intervention to subtly shape performance. Our approach capitalized on virtual environments, which has become a prominent tool in motor neuroscience and rehabilitation to deliver performance feedback that can go beyond delivering quantitative information after completion of performance (Holden and Todorov 2002; Huber et al. 2010; Lange et al. 2012). The results supported our three hypotheses: Subjects could be guided to learn the dynamically stable solution faster than controls (Hypothesis 1). Immediately after the manipulation was removed, they maintained their performance under normal conditions, without any undesired after-effects (Hypothesis 2). This enhanced performance persisted over extended practice (Hypothesis 3).

Unlike most previous studies on learning and adaptation, we chose a rhythmic task, complementing the numerous learning and adaptation studies on discrete reaching movements. Based on previous theorizing and experimental support, it may be expected that learning rhythmic movements has different characteristics and may obey different principles (Hogan and Sternad 2007; Hogan and Sternad 2013). This study focused on dynamic stability, a characteristic inherent to rhythmic movements, ranging from ball bouncing to locomotion. Previous theoretical and experimental research on the ball bouncing task showed that dynamic stability afforded a solution that obviated error correction and was less sensitive to noise. Skilled experts robustly converged to these solutions, characterized by negative racket accelerations, as shown in several different experimental set-ups and instructional conditions (Schaal et al. 1996; Katsumata et al. 2003; Wei et al. 2007; Wei et al. 2008; Huber et al. 2015). Importantly, subjects are not aware of how their strategy changes with practice. Hence, explicit information about the desired strategy or the central variable is probably less effective or even unnecessary, although this remains to be tested.

This study developed a subtle intervention that could steer subjects to the dynamically stable behavior. Unlike in many motor adaptation studies using force fields or visuo-motor rotations, where the after-effects are very short-lived when the manipulation was removed, the learned behavior in this study persisted. In Experiment 1, the test block showed that racket accelerations stayed negative, although there was a significant, yet small, increase from −5.32m/s² to −4.40m/s². However, this increase should not be over interpreted, as both values remained within the range of stability between −2 and −5m/s² (Schaal et al. 1996). In fact, prior studies showed that experienced subjects converged to hitting with a racket acceleration value of approximately −3m/s² (Wei et al. 2008). Hence, this increase may be interpreted in line with this previous observation. Experiment 2 further tested the persistence by including three test blocks that showed again that the impacts maintained the signature of dynamic stability. It should be pointed out that 3 blocks with 12 trials of approximately 60 bounces each amounted to ~720 single bounces and presents a relatively long test phase. By the end of the experiment, the two time-shifted groups learned to use solutions with local linear stability, whereas the control group only learned solutions outside of this desired range. While the results of these two experiments demonstrate that this efficient behavior persisted after removing the manipulation, further investigation is needed to determine if it is retained in the longer term, and when there are breaks between the practice and the test blocks.

Going Beyond Error to Measure Task Performance

It should be highlighted that over the course of practice, there was no significant difference in the error measures between the control group and the time-shifted group. Hence, analysis of error alone would not have differentiated between the two groups, as these performance changes differed from the paradigmatic error-based learning (Diedrichsen et al. 2010). As introduced above, bouncing a ball rhythmically has redundancy and low error can be achieved with different strategies. While the dynamically stable strategy is advantageous as it is more noise-tolerant, it is not necessary for task achievement. Previous work highlighted how novices and experts employ different degrees of error correction versus exploitation on these passive error compensation mechanisms (Wei et al. 2007; Wei et al. 2008). It should also be pointed out that hitting with more negative racket acceleration was not an immediate response to the manipulation. Rather, it took several blocks of practice until the time-shifted group had reached significantly lower negative racket accelerations at impact.

The observation that the errors did not differ between the experimental and the control group raised the question: what signaled and guided subjects to the dynamically stable behavior? We previously argued that the stable solutions are computationally less demanding because, in principle, they do not require active monitoring of the ball trajectory to correct errors. In contrast, under unstable conditions, continuous corrections of perceived errors in the ball amplitude of the previous bounce are required. Such continuous monitoring requires perceptual and control processes, which subjects might sense as computational effort and seek to gradually minimize. Such arguments are consistent with studies that have shown support for effort minimization (e.g. Emken et al. 2007).

Another type of effort that may play a role in optimizing performance is mechanical energy. Considering one bounce in isolation, the maximum amplitude of the ball is determined by racket velocity at contact, together with the pre-impact ball velocity and the absolute height of the impact. Hence, it may be argued that subjects should hit the ball at peak velocity of the racket trajectory to impart maximum energy to the ball for a given racket amplitude. Subjects may then decrease the racket amplitude, while still impacting the ball at sufficiently high peak velocities. This was actually proposed in previous work on juggling robots, but has never been observed in our human experiments (Bühler et al. 1990; Bühler et al. 1994). Nevertheless, to examine whether racket amplitude decreased with practice, facilitated by the time-shift condition, racket amplitudes were analyzed. Consistent with prior findings, the racket amplitudes neither increased nor decreased throughout practice. This permitted the conclusion that mechanical efficiency did not play a role.

Time-shifted Manipulation is Different Than Adding Random Noise

The random noise condition was added to test whether it was indeed the subtle velocity manipulation that had the desired effect on performance, and it was not only perceived as noise. As all measures show, performance in the noise condition was significantly the worst. In fact, subjects in this group very quickly showed frustration, because none of their attempts resulted in the desired performance. Hence, only three subjects were collected as the prolonged task performance with random outcome was thoroughly annoying to subjects. One other rationale behind adding extrinsic noise was that such amplification of intrinsic neuromotor noise may aid in the exploration of the execution space to discover the more advantageous solutions. However, this rationale proved not effective in the current experiment. One reason may have been that the added noise was relatively large. After the experiment, subjects in the noise-group reported a sense of helplessness, as they felt that they had no control over their own performance. Manley et al. (2014) similarly reported in a redundant line-reaching task that neither adding nor amplifying noise led subjects to discover the optimal solution, unless they were explicitly aware of the task-relevant variable.

Designing Interventions for Persistent Behavior

This persistence occurred because subjects most likely did not perceive any change in the task or environment when the manipulation was removed. One reason why subjects did not notice the removal of the manipulation is that the manipulation did not cause a change in the visuo-motor mapping. A previous experiment by Morice et al. (2007) on the same rhythmic task did not achieve persistent behavior after removal of a manipulation that altered the visuo-motor mapping. With the goal to examine how subjects learned new attractor solutions, they temporally shifted the racket position and velocity, which altered the visual display of the racket. This created a mismatch between proprioceptive and visual feedback and a new perceptual-motor mapping had to be learned. Hence, similar to visuo-motor adaptation studies, this acquired behavior disappeared very fast after the display returned to normal conditions. To avoid such fast return to baseline behavior, the present study only manipulated the racket velocity, which did not alter the visual display of the racket and introduce new visuo-motor mappings.

A second reason why the removal of the perturbation did not affect behavior is that the changed behavior also resulted in successful performance of the task under normal conditions. Even though subjects experienced initial difficulties when they performed with their typical novice strategy, mostly sticking solutions (Figure 5), they quickly learned to hit the ball later in the decelerating racket trajectory, which led to the desired decrease in the error and its variability. By the time the manipulation was removed, subjects had found solutions that led to dynamic stability both in the manipulated and unmanipulated versions task. Hence, removing the manipulation did not increase error and variability, and subjects did not perceive a reason to alter their behavior.

Fig. 5.

Exemplary time series of racket (black) and ball (gray) to illustrate dependent measures. Error was defined as the unsigned difference between the target height and the maximum ball amplitude at each bounce. Racket acceleration at impact was defined as the racket acceleration 6ms before the ball-racket impact of each bounce.

Implications for Motor Rehabilitation

While rhythmic ball bouncing is an experimental toy task, it has many similarities to walking. Most centrally, the contact fully determines the flight phase. Dynamic stability has also been a central research topic in the area of passive dynamic walking (McGeer 1990; Garcia et al. 2007). A recent series of studies by Reisman and colleagues on walking in post-stroke patients has demonstrated the utility for such implicit interventions on gait asymmetries (Reisman et al. 2007; Reisman et al. 2009; Reisman et al. 2013). Using a split-belt treadmill, patients walked on two parallel belts and their asymmetries in step length during locomotion were exaggerated by moving the belt for the paretic leg at a slower pace than the non-paretic leg. Increasing the asymmetries drove the nervous system to make the necessary corrections in step length and patients achieved better symmetry between the two step lengths. Furthermore, this improvement transferred to overground walking, which Reisman et al. (2009) attributed to the fact that the improved symmetry in step length may have been beneficial in terms of energy cost, balance, or efficiency. In contrast, when control subjects completed the same split-belt training, the training induced asymmetries that did not transfer to overground walking. Consistent with the findings of the current study, these results suggest that interventions for motor rehabilitation and motor learning must guide subjects towards the correct behavior in unaugmented task conditions.

Robot-assisted therapy is another example of an intervention that guides patients toward the correct behavior. Unlike robot-assisted therapy for the upper limbs, the robotic interventions for improving locomotion have fallen short so far. The most prominent robotic intervention for locomotion is to passively move the limbs through a predefined kinematic pattern. However recent studies have shown that this type of training does not improve gait (Hornby et al. 2008; Hidler et al. 2009). Marchal-Crespo et al. (2014) similarly showed that in the rhythmic ball bouncing, such robotic guidance of the racket trajectory actually hampered learning. This guidance contrasts with the time-shifted manipulation, where the stable solution was not imposed on the subjects. The subjects were still actively engaged in the learning process, which has proved to be critical for motor recovery (Lynch et al. 2005).

Furthermore, the manipulation provided minimal intervention as it only affected the critical instant, when the ball impacted the racket, as opposed to the entire racket trajectory. Prior studies on ball bouncing have also demonstrated the importance of ball-racket impact in behavior. For instance, in contrast to haptic guidance throughout the entire racket trajectory, haptic feedback at the ball-racket impact does improve performance and learning (Sternad et al. 2001; Ankarali et al. 2014). An intervention that focuses on physical contacts has also been shown a promising direction for robotic assistance in locomotor recovery. Ahn and Hogan (2012) have shown that periodic torque pulses to the ankle during gait can increase walking cadence. Similar to the ball bouncing task, exploiting the natural stability of walking trajectories may largely govern locomotion patterns, as suggested by research using passive-dynamic machines (Collins et al. 2005).

Conclusions and Outlook

With the time-shifted manipulation, the learned behavior persisted because removing the manipulation did not change the visuo-motor mapping and did not decrease task performance. These design principles should be considered when developing interventions for motor rehabilitation, where persistence and ultimately retention of the learned behaviors is critical. While this study could only make a case in point, it highlights that feedback to the learner can and should take on more than error information about the outcome. In learning real world skills, we are typically guided by parents, teachers, and coaches, and their guidance rarely comes in the form of an error score, or an explicit statement what the optimal solution is. Laymen and even coaches all too frequently do not have the vocabulary, let alone the quantitative measures to provide exact description of the target skill or their deviations. Instead, they provide instruction for how it should feel or look to the performer. In contrast, current research and virtual reality-based rehabilitation practices emphasize error- and reward-based learning, which is effective in many experimentally controlled tasks (Winstein and Schmidt 1990; Holden and Todorov 2002; Huber et al. 2010; Abe et al. 2011; Galea et al. 2015). However, activities of daily living are almost invariably redundant tasks that require additional and subtler guidance to help the learner or patient identify the mapping between execution and task outcome. It would be desirable to further develop principles of such implicit guidance that go beyond the error-based approach and develop manipulations that can be applied for rehabilitation through virtual environments.