Humans Use Similar Posture Sequences in a Whole-Body Tracing Task

Summary

Humans have elegant bodies that allow gymnastics, piano playing, and tool use, but understanding how they do this in detail is difficult because their musculoskeletal systems are extremely complicated. Previous studies have shown that common movements such as reaching for a coffee cup, cycling a bicycle, or playing the piano have common patterns across subjects. This paper shows that an arbitrary set of whole-body movements used to trace large closed curves have common patterns both in the trajectory of the body's limbs and in variations within those trajectories. The commonality of the result should spur the search for explanations for its generality. One such principle could be that humans choose trajectories that are economical in energetic cost. Another synergistic possibility is that common movements can be saved in segments that can be combined to facilitate the process of deployment.

Graphical Abstract

Highlights

• •Goal-oriented movements in a tracing task are highly stereotyped
• •Variations in repeated trials within an individual subject are correlated
• •Across different subjects, variations in posture sequences are correlated
• •The principle for similar posture changes is likely to be energy efficiency

Kinesiology; Bioinformatics; Computer Graphics; Computer Modeling

Introduction

In difficult movement tasks, such as retrieving an object from a cluttered environment or choosing balance positions for transporting a large unwieldy object, humans are inventive problem solvers, but at the other end of the movement spectrum in most everyday repetitive movements such as walking, sitting, and reaching humans exhibit large degrees of regularity (Bongers et al., 2012, Smeets et al., 2010, Flash and Henis, 1991, Flash and Hogan, 1985, Multon et al., 1999, Donelan et al., 2002). In the quest to understand the human movement system, it would be important to know if general movements have regularities across subjects as it would provide an important scaffold in the development of more detailed dynamic movement models.

There are at least two broad perspectives that suggest such use of regularity principles. One is the Bayesian perspective. Its adherents argue that this repeatability arises because such movements are committed to memory with precedence based on the probability of use (Wolpert and Ghahramani, 2000, Ingram et al., 2008, Körding, 2007). Such familiar movements even incorporate settings that anticipate perturbations. This repeatability has led to movements being subject to extensive analysis, but the focus has been on the exogenous constraints of the external task, rather than the much more complex endogenous constraints of the internal movement system that come into play more during large-scale movements.

The other perspective comes as a result of advances in models that can compute the joint torques in human-scale skeletal models. Early models attempted to model dynamics as an inverse problem that attempted to estimate the torques by modeling regularizing the dynamics equations as under-constrained systems proved cumbersome and prohibitively expensive. The newer models linearize the dynamic equations and use feedforward methods that are much better conditioned (Delp et al., 2007, Cooper and Ballard, 2012, Erez et al., 2015).

These methods show that the kinematics of a movement is directly related to its dynamics, thus raising the possibility that regularities in energetic cost of a movement may be indicative of regularities in the kinematics.

We are interested in the principles behind large-scale arbitrary movements, particularly with respect to variations between different subjects. Creating an experimental setting requires a way of measuring the kinematics of a movement. Models of human movement typically divide anatomical parts into discrete segments that have their own inertias and are interconnected to other segments by joints that are mostly rotary. Thus, a movement can be described as the time course of the coordinates of the joints. Our experimental setting uses an equivalent setting of 50 three-dimensional coordinates of a motion capture suit. The time course of these coordinates provides an equivalent representation of a movement's kinematics. To refer to the kinematics at a specific time we use the term posture. Classically, posture is used for particular poses such as sitting or standing, but we use it for all body orientations tested.

The posture formalism allows for a particularly straightforward method of testing the similarly between two postures: compute the Euclidean distance between every matching pair of markers and add them up. Although this method will not work for cases in which two postures are very contorted with respect to each other, it is fine for the situations measured in our experiment.

Thus, in this setting, the key question is now: for the larger movements, do the component posture changes also appear similar as in the case of common everyday movements, or are they very individualized across different subjects?

We sought a task that would test the extent of variation, especially in the case of whole-body movements governed by a common task, but one that allowed different subjects freedom in choosing movements to solve it. The task we chose had subjects tracing large-scale three-dimensional curves in virtual reality that required a series of whole-body movement sequences. Subjects could freely choose their starting posture and also were given no instructions as to how to comport themselves during the tracing process. Their postures were continuously recorded using the fifty-sensor motion-capture system. The central question was whether or not there would be any similarity in the postures used during tracing.

Body movements are challenging to study owing to their variation. Bernstein's famous well-known phrase characterizing repeated movements in terms of “repetition without repetition” emphasizes that repeated movements are never exactly the same (Berstein, 1966). However, repeated movement variations are never completely random. Informed by task goals, subjects can shape the variations in different parts of the body by co-contracting muscles to achieve desired dynamics in different sections of a trajectory (Latash et al., 2002). Thus, in looking for regularities in movements one has to deal with both that the trajectories will vary owing to muscle co-contraction and that the amount of co-contraction itself can be modulated throughout the movements.

Given these challenges, we developed specialized aggregation methods for data analysis that extracted similarities of posture sequences in the face of kinematic variations. Additionally, we analyzed the fine structure of the variations used by subjects during the tracing task. The interesting and unsuspected result was that both the movement's posture kinematics and kinematic variations showed striking commonalities across subjects, but in aggregation. Thus, such correspondences require refined methods for understanding and testing large-scale movement principles. Nonetheless, given the recent development of methods for extracting muscle torques from human dynamics models, an obvious and straightforward inference that can be drawn from our observed similarities in posture sequences is that similar movements may be selected to achieve low energetic costs.

Results

The main result is that, although the locations tracing data exhibit posture variations, both in repeat of a single subject and in trials by different subjects, the average postures show marked regularities in six aspects of the data that was subject to analysis:

• 1.The initial poses chosen by subjects grouped into a small set of preferred postures (see section Initial Posture Choices and Figure 1);
• 2.Stances in the specific points of tracing a square curve showed very small standard ellipsoids of all markers measured (see section Posture Matching during Tracing and Figure 2);
• 3.Analysis of data from throughout the traced curve showed that the average posture at every point on the curve was unique with respect to the averages at other points (see section Posture Matching during Tracing, Figure 3 and Figure 4);
• 4.A t test between a proximal relative posture and distal relative posture showed that the difference is significant at the 0.0001 level (see section Posture Matching during Tracing and Figure 5);
• 5.The variance of the markers scales according to their task relevance (see section Marker Variations during Tracing and Figure 6);
• 6.The variances in the subjects' postures were correlated. If at a point on the curve the variance of a trace calculated from a subject was relatively large, the average of the variance of all the repeated trials from all subjects would be relatively large also (see section Marker Variations during Tracing and Figure 7).

Figure 1.

Classifications of Starting Postures

The view is from the top of 3D space, and the black curves are the nine paths used in our experiment. The yellow dot on each curve indicates the starting position of the target sphere. The dots around each curve represent the average starting locations of head makers of the subjects while tracing this curve. The direction of each arrow is the direction from the head marker to the right index finger marker for each subject. Colors denote the different clusters classifications. For example, in Path 5, the square curve, only two starting postures were used. The distribution of the chosen initial postures for each path was shown in Table S1. For curves 2, 4, 5, and 7, one posture is overwhelmingly preferred over the others. The three-dimensional view of all nine curves were shown in Figure S2.

Figure 2.

Tracing Data Analysis for Path 5

(A) The skeleton clouds of 90 trials (18 subjects each with 5 repeats) when subjects' right index fingers reaching the two corners of the square path.

(B) Highly stereotyped postures generated by 90 trials. The spheres with different colors represent different markers. The central location and the size of a sphere indicate the mean position and the standard deviations of the corresponding marker, respectively.

Related to Figures S1 and S3.

Figure 3.

Posture Matching Results of Square Tracing at Frame 300

At the outset, for each frame, the mean posture at that frame is computed by averaging the postures at that frame. Next, the relative postures for frame 300 are compared with relative postures for other frames selected at 20-frame intervals. The colored dot represents the mean of Euclidean distances between checking relative postures at frame 300 and relative postures at other frames. The blue bar indicates the corresponding standard deviation. The colors of dots indicate the relative height from the laboratory floor with blue being the lowest and red the highest. In this example, the relative postures at frame 300 are the best match. The fact that all the other matches have higher distance measures indicates that the chosen match point is dissimilar to all the other points on the traced curve.

Figure 4.

Posture Matching Results for Nine Curves

The horizontal axis represents the checking frames, which were taken once every 20 frames. The vertical axis represents the frames of relative postures that were best matching with the postures at checking frames. Two red points and two yellow points indicate the matching results of the four fixed points in the virtual 3D. The two colors signify that these points were constrained to be at the same height. Naturally many subsets of blue points may have the same height, but this property was not checked. The net result is that almost every relative posture at each frame for each curve is unique. Given at least 50 samples per path and 9 paths, a total of more than 22,000 comparisons were made. This calculation makes the result that almost all of the 450 perfect matches are seen, even given that in many of the match pairings the tracing finger is at the same vertical height, remarkable.

Figure 5.

The Match Column Compares Matches of Postures at Their Original Location to the Mean Posture at that Location

The non-match column compares matches of postures to matches to the other mean postures on the curves. All nine curves are used in this comparison. This difference is obviously hugely significant, implying that the methodology is highly reproducible as postures that are at distal sites on the traced curve are very dissimilar. The t test of these data shows that the difference is significant at the 0.0001 level.

Figure 6.

Comparison of the Standard Deviation of Marker Positions

At each frame, the mean position of a marker was first computed, then we calculated the Euclidean distances between each of the positions of the marker and its mean position; finally, the standard deviation of the positions of the marker was computed. The corresponding average standard deviations averaged over all the frames of each curve are shown in Table S2.

Figure 7.

The Standard Deviations in Repeated Traces for Four Randomly Chosen Subjects Are Compared with the Average

To do this, for each ten-frame interval, the standard deviation of points is computed. Using these data the average standard deviation of these data is computed. Next, the individual variances are correlated with this average. Four colors signify four random subjects. The high R values indicate that subjects' variances, which indicate their co-contractions, are co-varying. All the R values for the four markers computed for all nine curves show positive correlations (see also Table S3).

Initial Posture Choices

Although the subjects could have chosen very different starting postures, they preferred one of a small number of specific groups. Small distributions in starting position and orientation can be explained if, at beginning of a trial, subjects roughly planned the sequences of tracing motions by visually tracking the path and its position on the target sphere en route. When they advanced to make contact with the target sphere, they placed their right index finger in a particular location on the path. As there were only so many ways for subjects to choose a comfortable place to start tracing, together with the constraint of the kinematic structure of their skeleton, the foot positions and body facing directions can be expected to show small distributions.

The postures at the first frame on each curve were taken as the participants' initial postures. Figure 1 illustrates the results of segmenting the initial postures into small groups. It showed that the paths with more distinct lobes had more clusters of distinct starting postures. Furthermore, it is obvious that the initial postures classification is mainly due to the standing locations, which roughly means participants' postures can be considered as exhibiting some similarity if they are standing in the same area and their fingers are attaching to the same point on the curve.

Posture Matching during Tracing

Once tracing has started, the postures of subjects can be compared at any point along the curve. Using one trace per participant, we calculated the three-dimensional standard deviation ellipsoid for each marker location. For example, tracing data for path 5 can be seen in Figure 2A, which illustrates all skeletons when subjects' index fingers contacted two corners while tracing a square path. Two common postures appeared corresponding to the two initial postures on different sides of the square curve (path 5) shown in Figure 1. The corresponding mean position of each marker and the standard deviation of marker positions are shown in Figure 2B. The ellipsoids' different colors represent different markers on the PhaseSpace suit. This comparison clearly indicates that subjects used similar postures at corners during tracing square path, a result that generalizes across positions and curves, as will be shown.

To more rigorously compare postures at all points along a path, the more sophisticated posture-matching method of comparison described in the tracing posture matching section (see Transparent Methods) was used. First, the mean posture of overall participants at each frame along the curve is computed by averaging the dataset using one trace from each subject. Thus, each frame has an associated mean posture. Next, each posture at a checking frame is matched against the mean postures for all the frames along the curve.

For instance, taking frame 300 as the checking frame, each of the postures at frame 300 was matched to all the mean postures in turn, and then the mean and the standard deviation of all matches at each frame were calculated. The results of such comparison are shown in Figure 3. The red dashed line illustrates the locus mean of the match, and the vertical bar in blue demonstrates the standard deviation. The match at frame 300 uses Equation 7, and all other matches use Equation 8 (see Transparent Methods).

As shown, the best match occurs at the frame 300, which means the best match occurs at the frame where the postures are taken from and its match is more inexact at other frames. It might be argued that the matches are different owing to the effect of the height of the curves above ground level. Different heights can make a difference, but there are large regions at the same heights with different matches. Figure 3 shows this by color coding the dots according to the heights of the curve points at the corresponding frames, with blue representing the lowest height and red the highest. It is readily seen that a large swath of points on the curve between frames 200 and 500 have very similar heights but their match costs are quite different. This format shows off the result that, although the matches may have considerable extents, their means are almost always very distinct.

This method can be extended for each point on every curve. Figure 4 summarizes the postures at each checking frame were the best matching with the mean posture at the same frame. For instance, as for path 2, the postures at frame 250 were best matched with the mean posture at frame 250. The plotted dots almost formed a line with the slope of 1, which is desired. Furthermore, the results for all nine curves show that the postures at each point are almost all unique.

The few outliers were generated because participants moved back to the initial positions at the end of tracing, which results in some similar postures at the very beginning and the very end of tracing. For example, the outlier of path 8 in Figure 4 implies the postures at frame 20 are best matching with the mean posture at frame 1,020. The study focused the results of the posture sequences during the tracing, which excluded the first 50 frames and last 50 frames. In this region, regardless of the subjects, the best match occurs at the appropriate location along the curve.

As mentioned earlier, one obvious reason to expect postures to be different is that the curves have many different heights that the tracing finger has to follow. However, there are many points along the curve at the same vertical height, including the four special points on each curve that were specifically chosen to be the same. The heights of the first two fixed points are 1.5 m and the heights of the remaining ones are 0.75 m. These two pairs are highlighted in red and yellow on Figure 4 and indicate distinct matches, even when the tracing figure heights are identical.

One final issue concerns the reproducibility of the method. How robust is the margin separating matches at correct positions and matches at arbitrary mismatched positions? To explore this issue, we averaged all matches at correct positions for all nine curves and compared this distribution to the corresponding calculation for incorrect matches. The result is shown in Figure 5, which shows that regardless of the curve, the posture at any point on a curve is easily distinguished from the postures at any other points on the curve. To demonstrate this result we used equal numbers of samples of the $Q_a$ and $Q_b$ calculated using Equation 7 and Equation 8, respectively (see Transparent Methods). The huge number of samples, together with the non-overlapping variances resulted t test level of significance greater than the 0.0001 level.

Marker Variations during Tracing

In concert with earlier observations (van Beers et al., 2012, Latash et al., 2002), the standard deviation of the task-irrelevant end-effectors was larger than that of task-relevant end-effectors, even when measured across subjects with different skeleton sizes. To show this relationship, we computed marker standard deviations using one trace per subject at four different distances from the tracing locus. Figure 6 shows the standard deviation of the task-relevant markers and task-irrelevant markers, across subjects with initial positions from the same cluster. As the most task-relevant end-effector, the right index finger had the smallest standard deviation, whereas as the most task-irrelevant end effector, the left index finger had the largest. The right shoulder marker and head marker, which are intermediate task-relevant markers, had moderate standard deviations along the entire tracing.

That distinctive patterns in the average of the variance data for all subjects have been shown leaves open the question of the issue of individual differences. Surprisingly it turns out that individual subjects all modulate their variations in the same way. To probe this relationship, we computed the standard deviations at small intervals at each frame of the five repeated trials for each subject using Equation 4 (see Transparent Methods) and sorted the intervals by standard deviation magnitude. Next, we computed the average of the standard deviations for each of the subjects. These calculations produced a series of tracing standard deviations for each subject as well as the standard deviation average across all subjects. This allowed the correlation of the standard deviation of each subject with the average of that of the group. A representative result is shown for the shoulder marker in Figure 7, which shows the mean and standard deviations of the tracing data sorted for increasing standard deviations. The important conclusion from this figure is that, although the local variance in tracing markers varies from point to point, it varies in a correlated way. If the variance is high at a point in tracing for one subject, it will also be relatively high for the average. The inset table in the figure shows the complete set of correlations for the four markers analyzed for each of the nine curves. The result is each of the 36 measurements is significantly positively correlated, with R values ranging from 0.40 to 0.87 and an average correlation 0.69. The implication is that subjects modulate their variance during tracing transit in the same way.

Discussion

The data analyses showed that, for large-scale movements of a constrained task, the observed movement kinematics are very similar across subjects, both in terms of mean posture values and their variances. At each point on any of the tested curves, the average posture is easily distinguished from all the other average postures by a metric that quantitatively compares posture differences.

Although there are local variations at points on the body, these turn out to be co-modulated also. When the average local standard deviation for all the traces increases, the standard deviation of the individual traces increases in a correlated way. This correlation may be a consequence of constraining the degrees of freedom of the task with muscle co-contraction, but further experiments with electromyography would be needed to settle this definitively.

One possible criticism of the experiment is that its main result might be expected as extensive data show that movement profiles are almost bell-shaped and there is a clear preference for certain comfortable postures, but those qualitative considerations fall short of accounting for the exactitude of the matches, given the extent of possible variations in the unconstrained nature of the task. Another factor in response is that the tracing target is continually moving at considerable speed, which obviates the use of quasi-static familiar posture segments.

Another potential criticism focuses on the fact that there are many situations in which humans have individual differences in their movements. Handwriting is one (Srihari et al., 2002, Said et al., 2000, Lacquaniti, 1989). However, this case might be accounted for as this situation makes extensive use of motor learning over a long period. Thus, humans can be expected to have differences in their local musculature that has trained in an area where variations have small differences in cost. Another possible point of contention is that humans have differences in gait that readily can be distinguished. However, it is the case that such gait differences are small compared with large variations in the postures at different tracing sites seen in our study. Yet another potential confounding issue is that differences in posture used to express emotion are easily detected (Roether et al., 2009, Troje, 2002, Montepare et al., 1987). The use of posture to express emotion has been given widespread attention, particularly in the graphics community. However, whether postures expressing different emotions could be confused could also benefit from a quantitative study. It could be the case that, in expressing motion, humans choose common postures also.

Given that the kinematics exhibited by tracing subjects is so regular, it is impossible to resist reflecting on why it might be so. In other words, why do they use these particular posture sequences? The question of human motion trajectories has been studied for several decades and tackled from two different perspectives. Researchers have for a long time made the distinction between the planning of a movement (Hollerbach and Atkeson, 1987) and its execution (Rosenbaum et al., 1995, Flash and Hogan, 1985), thinking that one had to choose between one or the other. However, the tracing data would suggest that these issues can be separated. The tracing task naturally separates planning and execution into separate phases. Subjects behave differently when choosing their starting posture, where different groups make different choices anticipating the whole traverse. Once they have made this choice, they engage in the act of tracing where they all agree on the postures taken.

Therefore, our preferred hypothesis is that, at least once the tracing starts, the posture sequences minimized metabolic energy. Our reasoning is driven by the fact that over the very recent past, a number of laboratories have built elaborate models of the human skeleton and its musculature and shown that the joint torques can be recovered by feedforward integration given the kinematics (Erez et al., 2015, Cooper and Ballard, 2012). Thus, given such a model, the kinematics is coextensive with the movement cost. Once the kinematics is given the movement cost is available. It should be emphasized that the assertion that common kinematics implies common movement cost is based on recent advances in complex three-dimensional Newtonian computer algorithms as well as earlier and simpler systems like that of (Alexander, 1997). Parenthetically it should be noted that, in anthropomorphic robot models of passive walking, the kinematics is correlated with the dynamics (Collins et al., 2001). Thus, although much work has to be done, the result could be suggesting that the subjects are choosing to follow low-cost trajectories.

Although the ability to integrate the dynamics equations given a model puts torques in register with kinematics, the reality of the ubiquitous use of vision in motor control planning allows for giving kinematics a causal status in motor control. This observation also impacts the possibilities for the brain's representation of movements. Common movement strategies argue the store of movement segments in the motor cortices for online transmission to the spinal cord (as opposed to real-time control). Studies by Churchland et al. (2010) show clear evidence of motor planning before movements. And the cortex is the only place where allocentric coordinates are converted to posture-centric coordinates.

Although the kinematics result reinforces the suggestion of a place of prominence in the motor plan, the representation of kinematics cannot simply be a stream of coordinates but has to have an associated grammar that breaks up this stream into “sentences” that reflect changes in set points mainly due to changes in physical contact with the world. An important concept for addressing these issues is that of the “uncontrolled manifold” (Latash et al., 2002), which formulates the control described in terms of task-relevant constraints. The idea of the uncontrolled manifold is also very sympathetic to an evolving view of motor representation in motor cortex. The original micro-stimulation experiments elicited localized body movements that were correlated with the organization of the topography of motor cortical area M1. However, stimulation with increased stimulus magnitude produced whole-body movements that could be interpreted as completely task-specific (Graziano and Aflalo, 2007) directed toward large-scale goals such as eating and defense. These task-orientated sequences suggest that the motor cortex's specific representations may include the longer sentences of a movement instead of the local responses that were initially used to define the sensory-motor homunculus. Our results have shown that the tracing finger's standard deviation was the smallest as expected, but the standard deviations of other components of the body were also affected by the task demands. In contrast, body segments that were less task-critical had larger standard deviations. In particular, the least task-relevant marker on the unused hand had the largest standard deviation. This result resonates with a number of previous results (van Beers et al., 2012, Latash et al., 2002) and suggests that the kinematics alone is not enough to code a movement but that it has to be augmented with additional parameters that shape its planned use.

The next step forward in the tracing task would be to attempt to make the connection between the kinematics and energetic cost. Since classical oxygen consumption methods are impractical and closed form analytically methods do not scale up, the full body forward integration approach is rapidly becoming the method of choice since they could compute the torques of participants directly from the motion capture data. Although the measurements of kinematics are not the same as joint torques, which are created by elaborate sequences of muscle contractions, there is considerable evidence showing that they are directly related. They are linked in computational models that can compute joint torques in a complex multi-joint human model (Delp et al., 2007). The implication of this recent computational capability is that one can think of kinematics as a motor plan that can be converted into an equivalent torque plan when the movement is executed. Fortunately, our laboratory developed a 50 degree of freedom dynamic model (Cooper and Ballard, 2012) that can calculate the torques of each joint given the motion capture data. We can create different movement trajectories, such as adding perturbations to the tracing trajectories of each body part, and see if calculated joint torques minimizing posture sequences for the dynamic human model agree with observed tracing posture sequences.

In summary, the overall result shows that movements themselves are highly stereotyped. This stereotypicality takes a special form. Although the movements vary, their mean postures across subjects and variations in repeated trials within an individual subject are highly correlated. Thus, given this methodology, the pattern of movements selected by different subjects was essentially the same, both in the average posture sequences and the variation in those sequences.

An important initial choice is the posture. Subjects could choose any starting posture, yet different subjects limited their choices to a small set, suggesting their tracing plan had a discrete number of solutions.

The tracing loci revealed that the standard deviation of task-relevant motion capture markers was observed to be smaller than that of markers that were not relevant to the task. This pattern is consistent with the uncontrolled manifold theory (Latash et al., 2002) of control in that the distal degrees of freedom must be programmed to orient the tracing finger's axial variance to be minimal.

Finally, the most important result of the experiment is the degree to which similar tracing postures suggest that there may be a principled objective function used by the subjects. Although many exigencies could impact any particular movement choice, posture changes that are saved for the long term are likely to be energy efficient. Future experiments will explore various metrics to see if the role of energy can be established definitively. If this turned out to be the case, this factor would impact almost every brain subsystem involved in motor control.

Limitations of the Study

There are three limitations of the study. First, the age scope of participants is within a limited range. Since the participants are all University students, their ages are between 18 and 28 years. The experimental results should apply to other age ranges, but this needs to be tested. Second, the movement of the subject is slightly hampered by the experimental system's need for cables. Currently, we have an accompanying person to manage the system's cables. Last, the tracing curve occupies 1 m × 1 m to 2 m × 2 m space. We would like to test the experimental protocol in a larger workspace that would allow even larger-scale movements.

Methods

All methods can be found in the accompanying Transparent Methods supplemental file.

Published: September 27, 2019