A Fresh Perspective on Dissecting Action Into Discrete Submotions

Abstract

The hypothesis that reaching motions are constructed from discrete components has been explored since the earliest scientific investigations of human movement, although composition specifics have been contentious. We reinspect this process by analyzing the underlying motor intent (rather than actual motion) using our recently-developed intent determination technique. First, synthetic data analysis was used to determine our accuracy in detecting submotion events. Next, we evaluated this on healthy reaching movements and overcame the problem of indistinguishably blended submotions by exposing subjects to strong, abruptly changing forces, which lead to clear corrections identifiable using direction-based clustering. We were able to accurately recover submotion parameters and identify patterns in submotion count, peak kinetic energy, and peak-to-peak duration. These values were all exponentially distributed, which implies that selection of submotions may follow simple rules. This provides a novel opportunity to investigate human motor action using the tools of statistics.

I. INTRODUCTION

Since the outset of human reaching research, it has been speculated that healthy, adult reaching is composed of submotions [1]. These submotions are distinct and easily recognizable during infant development, but gradually blend together until their placement is mostly indistinguishable by adulthood [2]. Following a stroke, these submotions are again readily recognizable [3] and their degree of blending reveals information about the severity of stroke and progress towards recovery [4]. Neural correlates of submotions have also been identified [5]. Attempts to identify and characterize submotions in healthy adults are thwarted by the submotions’ shape: a sum of any variety of radial basis functions can be used to reproduce any shape of interest [6]. Attempts to identify submotions in undisturbed reaching are therefore “doomed to succeed” and cannot support a rigorous and falsifiable hypothesis.

We cannot prove that submotions exist, but we can decompose synthetic reaches of known composition to show that our method is accurate. This synthetic data is produced by first decomposing a subject’s movements and then reconstituting them using either the commonly used [6], [7] minimum jerk shape or a sinusoidal shape that has been shown to fit human reaching poorly [8]. As the true submotion shape is not known, it is important that our method be insensitive to the shape we assume.

Here, we leverage three key techniques to achieve accurate decomposition. First, we use very large forces to disturb the limb and allow for easy identification of distinct corrections. Second, we use a technique that can recover the desired trajectory of a reach despite the presence of these external forces – intent determination [9], [10]. This step allows us to examine the corrections the subject attempted to make rather than the path of their hand revealing how and when subjects modify their movement intent. Third, we identify submotions using the directional component of velocity rather than seeking to maximize a goodness of fit that might be more sensitive to submotions’ shape. To support the subunit composition hypothesis, we should recover either a consistent pattern of composition or a specific distribution underlying the stochastic composition we observe.

II. Methods

As it is not known when or how submotions are selected, it was not obvious what types of disturbances might produce especially unblended intents. We designed an unusually large disturbance consisting of a strong curl field followed by a pulse of force. We hoped that the combination of these unexpected, strong forces with an abrupt, time-dependent transition would interfere with learning [11] and instead cause subjects to adjust their intent. We then recovered that intent [9] and decomposed it into submotions. Next, we verified the accuracy of the decomposition. Finally, we examined the statistical properties of the submotions we recovered in order to make some inferences about how they are planned.

A. Intent Determination

Intent determination is a filtering algorithm that combines an arm model with measurements of the actual trajectory of the arm and the forces exerted on the arm to estimate the movement intent [9] (Fig. 1). First, the physical length of the upper arm and forearm are measured in situ and the subject reports their body mass. Second, these physical properties are placed into a model of the arm constructed from cadaver data [12], [13] and experimental observations [14]. Third, the arm model is solved for the intended acceleration and integrated numerically to find the intent trajectory: a time series of desired positions. These steps and parameters differ from our earlier work only in that we did not take extra steps in order to customize the stiffness model.

Fig. 1.

Using intent determination, an algorithm that recovers intent from force-disturbed movements [9], we were to expose subjects to strong, abrupt forces and observe abrupt corrections at harsh angles. This directional separation facilitated decomposition of those intents into submotions.

B. Submotion Modeling

We assume minimum-jerk submotions, with bell-shaped speed profiles as they are well-known and not significantly outperformed by any other shape with the same number or fewer parameters [8]. The intent trajectory, $\overrightarrow{y}$, is then a sum of scaled, translated versions of this profile κ,

$$\overrightarrow{y}(t)=\overrightarrow{ϵ}+\overrightarrow{y}(0)+\sum _n{\overrightarrow{L}}_n\kappa \left(t,C_n,S_n\right)$$ (1)

$$\kappa (t,C,S)=10{\tau }^3-15{\tau }^4+6{\tau }^5$$ (2)

$$\tau =\{\begin{array}{ll}0\hfill & \text{if }0>\frac{t-C}{S}+\frac{1}{2}\hfill \\ \frac{t-C}{S}+\frac{1}{2}\hfill & \text{if }0\le \frac{t-C}{S}+\frac{1}{2}\le 1\hfill \\ 1\hfill & \text{if }1<\frac{t-C}{S}+\frac{1}{2}\hfill \end{array}$$ (3)

where t is time, C_n and S_n are the center and duration in time of the nth subunit, ${\overrightarrow{L}}_n$ is the vector change in position due to the nth subunit, and $\overrightarrow{ϵ}$ combines all factors not explained by submotions such as the inaccuracies due to imperfect recording and intent determination. The cases used to define τ have the effect of limiting the contribution of a subunit such that it adds smoothly to the trajectory within its span and has a constant contribution thereafter. In terms of velocity,

$$\dot{\overrightarrow{y}}(t)=\dot{\overrightarrow{ϵ}}+\sum _n{\overrightarrow{L}}_n\dot{\kappa }\left(t,C_n,S_n\right)$$ (4)

$$\dot{\kappa }(t,C,S)=\left(30{\tau }^2-60{\tau }^3+30{\tau }^4\right)S^{-1}$$ (5)

the contribution of a subunit is dramatically simpler as it makes no contribution at all outside its span (velocity is strictly 0 outside the span). It is for this reason that we perform our decomposition in the velocity space.

C. Submotion Decomposition

A decomposition method must overcome many challenges. The κ component can itself be further decomposed into other radial basis functions. Worse, the decomposition can be dependent on the shape of the basis function used. Others have tried to overcome these challenges by invoking Occam’s Razor: minimizing the number of submotions used to decompose a movement also minimizes the number of free parameters. They optimize the parameters of n submotions and then increase n if a fit accuracy threshold has not yet been attained [6]. Alternatively, some groups have instead stopped at a point of diminishing returns [7]. This is not a falsifiable hypothesis as no threshold for n is obvious and quality of fit might be driven by factors other than an insufficient number of submotions. In particular, it is not known how many submotions might compose a typical motion or how small they might be. Instead of asking how many submotions we need to fit a movement, we instead examine how many directions are evident within a movement. Leveraging the distinctness of the submotions’ directions allows us to avoid concerns of spurious decomposition without explicitly minimizing any cost function.

A decomposition borne of subunit distinctness is straightforward: if submotions are distinct then their direction at peak speed is their direction. We need only to identify peak speeds, record their directions, and then use the dot product as an expedient means to identify when submotions start and end. This process is summarized graphically in figure 1. Stated as a stepwise algorithm that recovers a single subunit:

1. Identify the time at which the maximum speed occurs within a movement, C.

2. Note the movement direction at this time, ${\widehat{v}}_C=\frac{\overrightarrow{v}(C)}{|\overrightarrow{v}(C)|}$.

3. Construct a rate of change that depends only on direction, $\psi =\frac{d}{dt}\frac{{\widehat{v}}_C\cdot \overrightarrow{v}(t)}{|\overrightarrow{v}(t)|}$.

4. Take S as 2 times the distance from C to the nearest local maximum or minimum of ψ.

5. $\overrightarrow{L}={1.875}^{-1}S\overrightarrow{v}(C)$ because $\dot{\kappa }(t=C)=1.875S^{-1}$.

6. Subtract $\overrightarrow{L}\kappa (t,C,S)$ from the velocity trajectory.

7. Repeat the algorithm on the remainder to recover additional submotions as desired.

Our algorithm terminated once the largest remaining peak in speed did not exceed ten centimeters per second as the signal-to-noise ratio drops below 5:1 here. These steps rely on the fundamentally restrictive assumption that the direction observed at peaks in speed reflects a single subunit and not a sum of submotions. Naively, this assumption is quite poor as it presumes a structure to movement composition that is not founded in previous observations; however, our hypothesized subunit distinctness appears to be upheld.

D. Synthetic Data

One unique way to test our approach is to artificially construct a motion from known submovements, and then see if our approach can recover the original elements. As the submotion composition of movement is not known, we construct two best estimates: one which adheres to our assumptions regarding submotion shape and one which does not. Subject 1’s recovered movement intent was decomposed by the algorithm described above, without any refinement of S, and recomposed according to equation 1. This recomposition was our naive best estimate of the underlying subunit composition of healthy adult reaching. We additionally recompose movement using the sin function as an alternative basis in order to test the robustness of our extraction to our assumption that submovements are minimum jerk in shape. In this case $\dot{\kappa }=\frac{\pi }{2S}\text{sin}(\pi \tau )$. Fig. 4 (blue bars) demonstrates that this process does accurately recover subunit parameters from synthetic data, even when the sin

Fig. 4.

Error in recovered submotion properties was not dependent on the submotion shape used to compose synthetic data. Parameter recovery was accurate, despite a high variance, as evidenced by mean error not significantly differing from zero as detected by Student’s T-Test.

E. Experimental Design

Subjects made center-out reaching motions in three directions that were sometimes disturbed by a large and difficult-to-learn curl-kick force. This disturbance was time-varying,

$$\overrightarrow{F}=\{\begin{array}{ll}\pm \left[\begin{array}{cc}0& 50\\ -50& 0\end{array}\right]\overrightarrow{v}\hfill & \text{if }{t}_0\le t\le t_0+400ms\hfill \\ 15\frac{{\overrightarrow{x}}_f-\overrightarrow{x}}{\left|{\overrightarrow{x}}_f-\overrightarrow{x}\right|}\hfill & \text{if }{t}_0+400ms\le t\le t_0+800ms\hfill \\ 0\hfill & \text{elsewhere}\hfill \end{array}$$ (6)

and depended on velocity $\overrightarrow{v}$, position $\overrightarrow{x}$, and the position of the reach’s target ${\overrightarrow{x}}_f$. The direction of the curl field is pseudo-random, which makes it very difficult to correct for even if it is detected early. The kick portion was programmed to push the subject towards the target to induce replanning. The onset of movement, t₀, was detected as the moment the subject left the previous reach’s target. These forces were not present during the first 100 reaches so subjects could familiarize themselves with the robot and any adjustments could be made. After the hundredth reach, disturbances were presented pseudo-randomly such that each direction was disturbed fifteen times. A total of 810 reaches were performed.

F. Human Subjects

Our human reaching trajectories were produced by eight healthy adults, two females and six males, aged 21 to 34 years old who gave informed consent in accordance with Northwestern University Institutional Review Board, which specifically approved this study and follows the principles expressed in the Declaration of Helsinki. Subjects self-reported right-handedness, performed the reaches with their right arm, and were compensated for their time. Subjects’ arm segment lengths were directly measured in situ while body mass was self-reported.

G. Apparatus

Subjects held the handle of the planar manipulandum depicted in Fig. 3, which was programmed to compensate and minimize any friction or mass. The MATLAB XPC-TARGET package [15] was used to render the force environment described above and collect data at 1000 Hz. Visual feedback of target and hand locations was performed at 60 Hz using OpenGL. Force sensor drift was addressed using a linear re-zeroing procedure.

Fig. 3.

Subjects held the handle of a robotic device that rendered programmed forces and recorded position and force information. Position was indicated to the subject with a blue cursor and the target of a reach was indicated in red. The opaque, horizontal screen occluded subjects’ vision of their hand.

H. Statistics

In order to test for bias in our parameter recovery, we performed a Student’s T-Test to determine whether or not the mean of the difference of the known composition’s parameters and the decomposition’s parameters differed significantly from zero. Tests were conducted at the α = 0.05 significance level.

The cumulative distribution function of the exponential distribution is defined as the probability that the random variable A takes on a value less than or equal to a:P(A ≤ a) = 1 − e^−λa. Therefore we expect that if an empirical cumulative distribution, C(a), is generated by this distribution, ln(1 − C) should be linearly related to a as ln(1 − C) = −λa. We examine this linearity using the coefficient of determination, R².

III. Results

Parameter recovery from synthetic data was highly accurate. Student’s T-Test detected no significant bias (n = 1352, p > 0.49) between the parameters recovered and their known values. While peak location and position change had lower variance, duration and submotion count were detected with much higher variance. Decomposition accuracy did decrease somewhat when sin-based submotions were used to construct synthetic movement even though minimum jerk-based submotions were still used in the decomposition, but this decrease in accuracy was not significant (n = 1352, p > 0.09).

We failed to identify any structure or motifs in our submotion decompositions. Lacking evidence of a recurring structure, we instead sought to identify the statistical properties of submotions. The first pattern we noticed was that our residuals, which have units of velocity squared, appeared to be exponentially distributed (R² = 0.94, Fig. 5 upper-left plot). One explanation for this observation is that the largest remaining submotion accounts almost entirely for the residual and the submotions themselves are exponentially distributed in velocity squared. This hypothesis is supported as submotion peak kinetic energy, 1.875²L²S⁻², is indeed well-accounted for by an exponential distribution (R² = 0.95, Fig. 5 upper-right plot). Seeking other exponentially distributed quantities, we also discovered that the duration between submotion peaks and the number of submotions composing a movement also appear to be exponentially distributed (R² = 0.91 and R² = 0.93 respectively, Fig. 5 lower plots).

Fig. 5.

After discovering that the residuals of our decomposition appeared exponentially distributed in velocity squared (upper-left), we tested other quantities for this distribution and found that peak kinetic energy, submotion count, and the duration in between successive submotion peaks were also well-explained by an exponential distribution. Colors represent different subjects while dots represent individual submotions or reaches. Quantities were normalized by their mean, which causes all regressions to have the same slope, in order to facilitate comparison across subjects.

IV. Discussion

This study provided support for a novel approach to decompose measured actions into submotions in an improved manner that considers the underlying intent and does not depend on assumptions of shape or number of elements. By capitalizing on a new intent extraction technique [9] and exposing healthy adults to strong and lasting forces, midmovement replanning became obvious. We found that our method could accurately recover submotion parameters and that this accuracy was robust even when an incorrect submotion shape was assumed. This method was not necessarily precise, however. While this imprecision is problematic, future work might improve upon it it by using our unbiased estimate of subunit timing and duration to recover submotions’ shape empirically by averaging a sufficient number of submotions.

Our finding that submotion counts and peak kinetic energy were exponentially distributed means that submotion count will occasionally be very high and submovement magnitude will occasionally be very small. This contrasts with previous work that explicitly avoids finding small submotions [6], [7]. Cost functions are usually constructed such that they maximize quality of fit while minimizing the number of free parameters used to do so. Our work suggests that this is inappropriate as many of the submotions present may contribute little to the motion.

Finally, we found an interesting distribution of submotion parameters: most were exponentially distributed. While the source of this distribution is unclear, it suggests that there is a common underlying structure that can be understood and leveraged through further study. This may enable greater understanding of human movement and its building blocks, leading to a stronger comprehension of the way people plan and move in the world. Eventually, it may even help to correct planning and movements when this structure breaks down, such as following stroke.

Fig. 2.

To demonstrate our decomposition method, we begin with a recorded intent (black line, intent panel) and its speed trajectory (black line, speed panel). We have additionally marked pastel-colored submotions in the intent, speed, and remainder panels to illustrate typical properties. The largest submotion’s peak, C, is identified and marked with a red X. The time derivative of velocity’s dot product with the direction at C is computed as ψ. Submovement duration, S, is twice the distance in time from the red X to the red circle that marks a maximum in the rate of change of direction. The velocity at C and the submotion’s duration also define its magnitude L. We can then calculate the submotion’s contributions to position and velocity and subtract them from the intent trajectory.