A controller for walking derived from how humans recover from perturbations

Abstract

Humans can walk without falling despite some external perturbations, but the control mechanisms by which this stability is achieved have not been fully characterized. While numerous walking simulations and robots have been constructed, no full-state walking controller for even a simple model of walking has been derived from human walking data. Here, to construct such a feedback controller, we applied thousands of unforeseen perturbations to subjects walking on a treadmill and collected data describing their recovery to normal walking. Using these data, we derived a linear controller to make the classical inverted pendulum model of walking respond to perturbations like a human. The walking model consists of a point-mass with two massless legs and can be controlled only through the appropriate placement of the foot and the push-off impulse applied along the trailing leg. We derived how this foot placement and push-off impulse are modulated in response to upper-body perturbations in various directions. This feedback-controlled biped recovers from perturbations in a manner qualitatively similar to human recovery. The biped can recover from perturbations over twenty times larger than deviations experienced during normal walking and the biped’s stability is robust to uncertainties, specifically, large changes in body and feedback parameters.

1. Introduction

Human walking is known to be energy efficient [1–4] and stable [5–7], but we do not know how these potentially conflicting properties are realized simultaneously. Specifically, we do not yet have a complete characterization of how humans control walking. One way to examine how humans control walking, or indeed any movement task, is to apply unforeseen perturbations and examine the transients back to steady state. While a number of such walking perturbation experiments have been performed [5–15], none of them have been used to derive a controller sufficient for a complete, even if simplified, simulation of walking. Instead, the perturbation experiments have only been used to obtain insights into limited aspects of walking control such as foot placement or centre of pressure modulation [5,7,15–18], ankle impedance [12,19], impulse response functions for some muscles [14,15] or some aspect of body motion [9]. Other authors have considered natural step-to-step variability in unperturbed steady-state locomotion [10,20–24] to obtain information about stability and control of walking and running, but again, we do not know of a complete synthesis of a walking controller from such measurements. Similarly, there have been many stable bipedal walking simulations [25–28], stable two-legged walking robots [29,30], and robotic prostheses and exoskeletons [31,32], but none of these simulations, robots or assistive devices have controllers that are quantitatively derived from human walking response to perturbations. Our study could be considered a partial analogue of [33] in which a running controller was obtained for a simple biped model by fitting model responses to human responses to natural step-to-step variability.

Here, we derive a complete walking controller for a minimal and classical model of walking, based on human responses to perturbation experiments. Specifically, we perform human subject experiments in which we apply unforeseen perturbations on humans as they walked on a treadmill, and then use data from these experiments to derive a controller that describes how foot-placement and push-off impulse are controlled for the inverted pendulum model of walking. This is a model used extensively in both biomechanics (to understand human movement [34]) and robotics (as a simple model for control and planning [35,36]). We show that the human-derived controller stabilizes inverted pendulum walking, is robust to large changes in body parameters, and prevents falls from large perturbations to the upper body.

2. Methods

2.1. Experimental methods

The experimental protocol was approved by the Ohio State University Institutional Review Board. Twelve subjects (N = 12, eight male, four female, age = 25.5 ± 4 years, leg length = 0.97 ± 0.06; mean ± s.d.) participated with informed consent. Subjects walked on a treadmill at 1.2 m s⁻¹, while discrete perturbations were applied to them by the experimenters, that is, pulls through a light stiff cable tied at the waist (figure 1). One subject walked at 1.1 m s⁻¹.

Figure 1.

Experimental set-up. Subjects walked with a loose safety harness on an instrumented treadmill, with randomly spaced unforeseen perturbations being applied by pulling on the perturbing cables. The subject wore passive noise-cancelling earmuffs so they could not hear the perturbation coming (though the perturbation was indeed very quiet due to human pulling). The subject also wore cardboard blinders that prevented them from seeing the perturbation through peripheral vision. The figure shows the set-up for sideways perturbation experiment; the backwards perturbation experiment is identical, except the two sideways perturbing cables are replaced by one perturbing cable able to pull backwards. The perturbing cables were attached to the person via magnets so that the cable disengages when a force threshold of roughly 100 N is crossed. The notation for the coordinate directions (X, Y, Z) is shown, but the origin of the system is the stance foot at mid-stance for each step. (Online version in colour.)

We performed two different types of perturbation experiments: (i) anterior–posterior (AP) perturbations, involving only backward pulls, and (ii) sideways or medio-lateral (ML) perturbations, involving either leftward or rightward pulls in random order. Nine subjects participated in AP trials and seven subjects participated in ML trials, and four subjects participated in both. For each type of perturbation experiment, subjects walked on the treadmill for 10 trials of 4 min each. Each such trial had a total of 10 perturbations. The pulls were exerted manually by the experimenters, rather than by using a robotic device (say, as in [7]). The perturbing cables were usually slack (negligible forces) and became taut to apply forces only during the brief perturbations. About 20% of these perturbations were ‘fake’, so that the experimenter goes through the motion of pulling on the subject without making the cables taut and applying substantial forces. Such fake pulls were designed to ensure that participants could not reliably predict the onset of perturbations and to rule out the possibility that subjects were responding to the perceived motion of the experimenter rather than the effect of the perturbing forces. Subjects wore earmuffs and blinders to block any visual or audio cues (figure 1) and walked with arms crossed because the sideways perturbing cables interfere with arm swing. The timing of the pulls was randomized, with 15–45 s between consecutive pulls. We performed this randomization by repeatedly generating a uniformly distributed random number s between 15 and 45, and alerting the experimenter to manually exert a pull when exactly s seconds have passed after the previous pull. The phase of the perturbations were not controlled, and were randomly distributed over the whole gait cycle (see electronic supplementary material, figure S1 for perturbation phase histograms). For ML pulls, in addition to randomizing the pull timing, we also randomized the sequence of left versus right pulls by having one experimenter on each side and alerting either the left or the right experimenter to apply the pull. At the beginning and end of each subject’s session, a 30 s unperturbed walking bout was performed to serve as a baseline.

Kinematic data for the feet and the pelvis were collected using a marker-based motion capture system (Vicon T20, eight cameras, 100 Hz, five pelvis markers and four markers on each foot). Ground reaction forces were captured through a pair of force plates within the treadmill (Bertec FIT split-belt instrumented treadmill), but not used in this study. Perturbing forces were also measured using light load cells (Phidgets 100 Kg S-type) in series with the perturbing cables. The data will be available without restrictions through an open database, Dryad. Figure 2 shows perturbing forces and impulses from two example trials, one each from the backward and sideways perturbation experiments. While the forces applied were not carefully controlled, typical maximum forces in each pull were in the 45 N to 75 N range. The pull directions were approximately along the treadmill (AP) or perpendicular to the treadmill (ML), enforced by having the experimenter pull from sufficiently far away from the subject (about 3 m) and appropriately positioned behind or on either side of the treadmill. Our analysis does not rely on the exact control of these perturbing force magnitudes or directions.

Figure 2.

Forces applied during the perturbation experiment. (a) Example of applied perturbation forces from one trial for one subject in the backwards perturbations experiment. (b) Histogram of applied impulses (normalized for each subject, pooled over all subjects) from all backwards perturbation trials for ‘highly perturbed’ steps. (c) Example of applied perturbation forces from one trial for one subject in the sideways perturbations experiment. (d) Histogram of applied impulses (normalized for each subject, pooled over all subjects) from all sideways perturbation trials for ‘highly perturbed’ steps. (Online version in colour.)

2.2. Identifying highly perturbed steps

First, we simplify the human data by representing the walking motion with three salient points (as in [21]), one marker for each foot (heel marker) and one for the pelvis (approximated by averaging the pelvis markers as in [21,23]), as a proxy for the centre of mass (CoM) [37]. These three representative points are hereafter referred to as the pelvis and feet. See figure 1 for the coordinate system used: Z is vertical (upward positive), Y is forward and X is rightward, so that the pelvis position is denoted (X_pelvis, Y_pelvis, Z_pelvis).

Mid-stance is defined as when the forward position of the averaged pelvis marker (Y_pelvis) equals the forward position of the representative foot marker currently in stance. The position of the stance foot at mid-stance is taken to be the origin (0, 0, 0) of coordinates for each step, so that mid-stance happens when Y_pelvis = 0, with the pelvis moving forward relative to the foot. Each walking bout was divided into two sequences of steps, steps starting at a left mid-stance and ending at a right mid-stance, and steps starting at a right mid-stance and ending at a left mid-stance. For each such step starting and ending at a mid-stance, the stance foot position at the initial mid-stance is the origin (0, 0, 0) and the foot position for the immediately subsequent contralateral stance foot in the same coordinate frame is (X_foot, Y_foot, 0).

The steps in which the applied external impulse was very small were labelled as unperturbed (impulse less than a small threshold, 2 Ns, designed to ignore noise and fake pulls). All subject data were then normalized using body mass m, leg length ℓ and acceleration due to gravity g; all analyses and results are presented in this non-dimensional form, for instance, distances normalized by ℓ, speeds normalized by $\sqrt{g\ell }$.

In the analysis below, we only use steps that were highly perturbed, corresponding to a state deviation greater than a threshold. To determine the set of all such perturbed steps, we used the sideways velocity and forward velocity $({\dot{X}}_{\mathrm{pelvis}},{\dot{Y}}_{\mathrm{pelvis}})$ of the CoM in the treadmill-belt frame. This velocity was compared to the mean mid-stance CoM velocity for the unperturbed steps. Steps where the difference between these two values exceeded twice the standard deviation of the unperturbed steps (characterizing the natural unperturbed variability) were labelled as ‘highly perturbed.’ This criterion implies that we ignore about 95% of state deviations that occur due to natural variability in the absence of perturbations, ensuring that we primarily capture state deviations due to externally applied perturbations.

The normalized perturbed step data from all subjects were pooled together, producing a total of about 4900 perturbed steps of data across the two experiments: 2442 steps for backward perturbations and 2445 steps for sideways perturbations. Figure 2 also shows a histogram of normalized applied impulses (normalized using individual subject weights, subject leg lengths and the acceleration due to gravity) from all trials in both experiments, looking only at impulses from these highly perturbed steps.

2.3. Three-dimensional inverted pendulum walking

We use the measured human recovery responses subsequent to these highly perturbed steps to derive a stabilizing controller for inverted pendulum walking, perhaps the simplest model of human walking [34]. Here, a biped with a point-mass body (figure 3a) vaults over a constant-length leg during each step, describing, in general, a three-dimensional (3D) inverted pendulum motion (figure 3b,c). The transition from one step to the next is accomplished by two collisional impulses, pushing off impulsively with the trailing leg, followed by an impulsive heel strike with the leading leg. The heel strike is modelled as a passive plastic collision with a nominally rigid leg. By changing the push-off impulse and where the foot is placed (X_foot, Y_foot), the inverted pendulum walker can achieve complex walking trajectories and also recover from perturbations [2,28]. See electronic supplementary material, S1 for mathematical details and the equations of motion.

Figure 3.

Inverted pendulum walking with a simple biped. (a) This biped consists of a point-mass and two mass-less legs. The inverted pendulum walking gait consists of single-stance inverted-pendulum motions separated by an instantaneous step-to-step transition consisting of a push-off and a heel-strike impulse along the trailing and leading legs. (b) The 3D inverted pendulum walking motion consists of 3D inverted pendulum phases, and the foot placements not in a line. A two-step periodic 3D inverted pendulum walking motion is shown. (c) The 3D inverted pendulum walking motion can be controlled by modulating the next target foot-position and the applied push-off impulse, allowing the biped to recover from perturbations. (Online version in colour.)

For this inverted pendulum model, first, we determine the unique two-step periodic motion that matches the mean step length, mean step width and mean walking speed of our subjects (figure 3b). See electronic supplementary material, S1 and S2 for the computational procedure for obtaining this best-fit periodic motion. We consider this to be ‘nominal motion’ for the walker in the absence of any perturbation. Deviations from this nominal motion are to be corrected by the feedback controller, and such deviations in state and control from their nominal values are represented by the prefix Δ. Specifically, we assume that the foot placement and the push-off impulse are changed in response to deviations in mid-stance state S of the biped, characterized by the three variables $(X_{\mathrm{CoM}},{\dot{X}}_{\mathrm{CoM}},{\dot{Y}}_{\mathrm{CoM}})$, in the belt-fixed frame with origin at the current stance foot. These three scalar variables form the complete state of the inverted pendulum walker: due to the choice of origin, we have Y_CoM = 0 at mid-stance; further, Z_CoM and ${\dot{Z}}_{\mathrm{CoM}}$ are simple functions of $(X_{\mathrm{CoM}},{\dot{X}}_{\mathrm{CoM}},{\dot{Y}}_{\mathrm{CoM}})$ due to the constancy of the leg length ℓ. Next, we describe how the feedback controller for inverted pendulum walking is derived so as to approximate the human perturbation responses. For this comparison, we use the averaged human pelvic marker state as a proxy for the CoM state.

2.4. Inferring step-to-step map directly from human data

First, from the highly perturbed steps in the human data, we determine a step-to-step map; this step-to-step map describes the state transition from one highly perturbed mid-stance, say, step n, to the next mid-stance, namely the (n + 1)^th step:

$$\mathrm{\Delta }{\left[\begin{array}{c}{X}_{\mathrm{pelvis}}(n+1)\\ {\dot{X}}_{\mathrm{pelvis}}(n+1)\\ {\dot{Y}}_{\mathrm{pelvis}}(n+1)\end{array}\right]}_{\mathrm{mid-stance}}=J_1\cdot \mathrm{\Delta }{\left[\begin{array}{c}{X}_{\mathrm{pelvis}}(n)\\ {\dot{X}}_{\mathrm{pelvis}}(n)\\ {\dot{Y}}_{\mathrm{pelvis}}(n)\end{array}\right]}_{\mathrm{mid-stance}},$$ 2.1

where Δ refers to changes in the states from the mean state across all mid-stances with the same stance foot (that is, all left mid-stances or all right mid-stances) and J₁ is a 3 × 3 matrix. Two such matrices J₁ are derived, both using ordinary least squares [21]: one for the transition from the left mid-stance to the right mid-stance and another for the transition from the right mid-stance to the left mid-stance. These step-to-step maps characterize the cumulative effect of the human walking controller over the mid-stance-to-mid-stance period.

2.5. Inferring foot placement dynamics directly from human data

The position of one stance relative to the previous stance foot is characterized by two numbers: a step-length Y_foot and a step-width X_foot, defined as the distance between the successive stance feet (in the belt frame) in the AP and the ML directions, respectively. We computed the mean step length and step width using unperturbed steps for each trial and deviations from these trial means were measured for all perturbed steps. Then, we computed the best linear mapping from mid-stance state deviations to step width and step length deviations corresponding to the next foot placement (as in [21]), given by:

$$\mathrm{\Delta }\left[\begin{array}{c}{X}_{\mathrm{foot}}(n+1)\\ Y_{\mathrm{foot}}(n+1)\end{array}\right]=K\cdot \mathrm{\Delta }{\left[\begin{array}{c}{X}_{\mathrm{pelvis}}(n)\\ {\dot{X}}_{\mathrm{pelvis}}(n)\\ {\dot{Y}}_{\mathrm{pelvis}}(n)\end{array}\right]}_{\mathrm{mid-stance}},$$ 2.2

where K is a 2 × 3 matrix quantifying the sensitivity of the foot placement to the mid-stance state deviations. Again, two such mapping were computed, one for left foot placements and another for right foot placements.

2.6. Fitting a controller to inverted pendulum walking

The foot placement and push-off impulse controller for inverted pendulum walking are derived so as to best approximate the step-to-step map and foot placement dynamics directly derived from human data (J₁ and K). A push-off impulse controller cannot be derived directly from human data since humans use forces spread out over the duration of a step rather than use a discrete impulse. In the simple inverted pendulum model of human walking, the complex modulation of ground reaction forces is captured simply by the control of the push-off impulse.

The biped has three scalar control variables per step, namely, the fore-aft foot placement, the sideways foot placement, and the push-off impulse. We assume that these control variables are determined by a once-per-step linear controller of the following form:

$$\mathrm{\Delta }\hspace{0.17em}\left[\begin{array}{c}{X}_{\mathrm{foot}}(n+1)\\ Y_{\mathrm{foot}}(n+1)\\ I_{\mathrm{push-off}}(n+1)\end{array}\right]=J_4\cdot \mathrm{\Delta }\hspace{0.17em}{\left[\begin{array}{c}{X}_{\mathrm{CoM}}(n)\\ {\dot{X}}_{\mathrm{CoM}}(n)\\ {\dot{Y}}_{\mathrm{CoM}}(n)\end{array}\right]}_{\mathrm{mid-stance}}.$$ 2.3

That is, the deviations in the mid-stance state on the nth step results in the modulation of the immediately following foot placement and push-off impulse. The elements of this matrix J₄ are the control gain unknowns we seek.

First, we compute the linearized step-to-step map for the inverted pendulum biped model, written as the sum of two terms, one due to the passive inverted pendulum dynamics and another due to the effect of the controller:

$$\begin{array}{c}\mathrm{\Delta }\hspace{0.17em}{\left[\begin{array}{c}{X}_{\mathrm{CoM}}(n+1)\hfill \\ {\dot{X}}_{\mathrm{CoM}}(n+1)\hfill \\ {\dot{Y}}_{\mathrm{CoM}}(n+1)\hfill \end{array}\right]}_{\mathrm{mid-stance}}=J_2\cdot \mathrm{\Delta }\hspace{0.17em}{\left[\begin{array}{c}{X}_{\mathrm{CoM}}(n)\hfill \\ {\dot{X}}_{\mathrm{CoM}}(n)\hfill \\ {\dot{Y}}_{\mathrm{CoM}}(n)\hfill \end{array}\right]}_{\mathrm{mid-stance}}\hfill \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+J_3\cdot \mathrm{\Delta }\hspace{0.17em}\left[\begin{array}{c}{X}_{\mathrm{foot}}(n+1)\hfill \\ y_{\mathrm{foot}}(n+1)\hfill \\ I_{\mathrm{push-off}}(n+1)\hfill \end{array}\right].\hfill \end{array}$$ 2.4

Here, J₂ and J₃ are properties of the inverted pendulum model, and can be derived directly from the inverted pendulum model by linearizing its dynamics about the non-planar periodic motion. The sensitivity matrix or Jacobian J₂ is derived by a finite difference approximation, by computing how a small perturbation in each state direction, namely $[X_{\mathrm{CoM}}(n),{\dot{X}}_{\mathrm{CoM}}(n),{\dot{Y}}_{\mathrm{CoM}}(n)]$ at one mid-stance, grows over one step for inverted pendulum walking in the absence of any feedback control, that is, Δ$[X_{\mathrm{foot}}(n+1),y_{\mathrm{foot}}(n+1),I_{\mathrm{push-off}}(n+1)]=0$. Thus, the J₂ matrix captures the ‘passive dynamics’ of the inverted pendulum walking model. The matrix J₃ is derived, again by a finite difference approximation, by computing the effect of small deviations in each of the control variables, namely $[X_{\mathrm{foot}}(n+1),y_{\mathrm{foot}}(n+1),I_{\mathrm{push-off}}(n+1)]$, in the absence of any initial state perturbations at the initial mid-stance, that is, $\text{Δ}[X_{\mathrm{CoM}}(n),{\dot{X}}_{\mathrm{CoM}}(n),{\dot{Y}}_{\mathrm{CoM}}(n)]=0$.

Substituting our linear controller definition (equation (2.3)) into this linearized inverted pendulum model dynamics (equation (2.4)), we write the step-to-step state transition map for the model as:

$$\mathrm{\Delta }\hspace{0.17em}{\left[\begin{array}{c}{X}_{\mathrm{CoM}}(n+1)\\ {\dot{X}}_{\mathrm{CoM}}(n+1)\\ {\dot{Y}}_{\mathrm{CoM}}(n+1)\end{array}\right]}_{\mathrm{mid-stance}}=(J_2+J_3\cdot J_4)\cdot \mathrm{\Delta }\hspace{0.17em}{\left[\begin{array}{c}{X}_{\mathrm{CoM}}(n)\\ {\dot{X}}_{\mathrm{CoM}}(n)\\ {\dot{Y}}_{\mathrm{CoM}}(n)\end{array}\right]}_{\mathrm{mid-stance}}.$$ 2.5

We solve for the control gain matrix J₄ by solving a nonlinear least-squares problem. Specifically, we minimize the weighted squared difference between the model and experimental versions of foot placement controller gains and the mid-stance-to-mid-stance state transition maps. That is, the matrix error terms we seek to minimize are: e₁ = J₄(1 : 2, : ) − K for the foot placement controller gains and e₂ = J₁ − (J₂ + J₃ . J₄) for the mid-stance-to-mid-stance state transition maps. This gives us 15 scalar error terms in the objective function, six from the foot-placement gain error e₁ and nine from the mid-stance-to-mid-stance map error e₂. The squares of these 15 scalar error terms are scaled by the error variances of the corresponding elements of K and J₁. The sum of the squares of these scaled error terms is minimized. A more formal representation of the optimization problem can be found in electronic supplementary material, S3.

2.7. Deriving a left-right symmetric controller

Most humans are approximately left-right mirror symmetric, but not exactly, at least as inferred from steady unperturbed walking dynamics [21,38]. That is, the regression coefficients in these linear models for left to right transitions are similar in magnitude to those for the right to left transitions, except for the signs of coefficients that couple sideways terms (i.e. step width and sideways CoM position or velocity) to forward terms (i.e. step length forward velocity and push-off impulse) or vice versa being reversed. The coefficients are mirrored with respect to the sagittal plane. To understand this mirroring, consider a rightward push when the right foot is in stance and a leftward push when the left foot is in stance. Both perturbations should both affect the magnitude of step width in the same way; however, a larger step width in left-stance means placing the right foot further right, and a larger step width in right stance means placing the left foot further left [21].

Here, for simplicity, we constrained the inferred controllers to be exactly mirror symmetric. To do this, we first produced a mean mirror symmetrized left-to-right version of K and J₁ by averaging the left-to-right regression and the mirrored version of the right-to-left regression. This new mirror symmetrized K and J₁ are then used in the optimization to derive the inverted pendulum controller. As an alternative to symmetrizing the gains by taking the mean, we could directly fit a single mirror-symmetric controller to all of the data, which gives approximately the same results.

3. Results

All variables and parameters listed in the results below are non-dimensional, normalized using body mass m, leg length ℓ and the acceleration due to gravity g. The ± and ∓ signs represent mirror symmetry in the control gain values, with the top sign representing the gains for left-to-right transitions and the bottom sign for the right-to-left transitions.

3.1. Nominal inverted pendulum walking with non-zero step width

The typical inverted pendulum walker has a zero step width planar motion [34,39]. Such planar inverted pendulum walkers cannot reasonably approximate the average human walking trajectory, which has non-zero step width and non-planar body motion [40]. Thus, here, to capture the 3D aspects of human walking, we use a non-planar 3D inverted pendulum walking with non-zero step width.

Specifically, we obtained the nominal inverted pendulum walking motion of the simulated biped as having the same average step length (0.66 ± 0.03), step width (0.23 ± 0.04), and forward speed (0.39 ± 0.02) as the human subjects, in the absence of external perturbations. Figure 4 shows that this nominal motion closely resembles the mean unperturbed motion for our human subjects, even though we did not fit the detailed motion.

Figure 4.

Comparing human data to simulated biped. (a) Data from a human subject in the medio-lateral perturbation experiment are shown on the left-panel, compared with analogous motions of the inverted pendulum biped on the right panel. The nominal trajectory of the pelvis and the position of the foot are shown in the transverse plane (top down view). Two perturbed trajectories and their respective foot positions are also shown. For the simulated biped (right panel), we show top-view walking trajectories with and without a rightward perturbation applied to the initial mid-stance state. The trajectory and corresponding foot positions can be seen. The two trajectories chosen had both a rightwards position and velocity deviation at mid-stance, so as to be more distinct from the unperturbed trajectory. (b) Data from a human subject in the anterior–posterior perturbation experiment are shown in the left panel, compared with motions from the inverted pendulum model in the right panel. The nominal variation of the pelvis forward velocity as a function of gait phase can be seen. Two perturbation recovery trajectories are also shown. For the simulated biped (right panel), we show fore-aft velocity trajectories for gaits with and without a backward perturbation applied to the initial mid-stance state. The variation of the CoM forward velocity as a function of gait phase can be seen. More perturbed trajectories are shown in electronic supplementary material, figure S3. (Online version in colour.)

Given that we fit the model to the data, it may seem that this qualitatively similarity is just a consistency check. However, while we fit the biped model to the data, we do not fit full shapes of trajectories. We just fit the values of three variables (speed, step length and step width) for the nominal gait. Thus, that the shape of the trajectories for the body is similar for the data and the model both for the nominal trajectory and for perturbation responses is not inevitable, and relies on the 3D inverted pendulum walking model being a reasonable model of walking, capturing the essential CoM mechanics. For instance, the more common planar inverted pendulum model does not capture the top view kinematics of the CoM.

3.2. Perturbations significantly change the motion of the human subjects

Perturbations applied through pulls at the waist successfully moved subjects away from their nominal motion. When a rightward pull is applied to the subject, their subsequent pelvis trajectory and foot position are shifted in the direction of the pull by a margin much greater than 1 s.d. of normal variability during unperturbed walking, as can be seen in figure 4a. When a backward pull is applied to the subject, the pelvis forward velocity slows down by a margin much greater than 1 s.d., as can be seen in figure 4c. This is a consistency check: our goal with the perturbation experiment was to generate state deviations sufficiently far away from nominal human motion variability and we find that these large deviations were actually achieved, giving about 4900 total steps across all subjects and experiments that satisfied the criteria for ‘highly perturbed’ steps outlined earlier. Figure 4a,c shows only two perturbed trajectories out of thousands; see electronic supplementary material, S4 for more such trajectories.

3.3. Foot placement control is well captured by a linear model

The foot placement control matrix K derived directly from regression on the human data is given by the linear equations:

$$\begin{array}{r}\mathrm{\Delta }\hspace{0.17em}{X}_{\mathrm{foot}}=\hspace{0.17em}\hspace{0.17em}1.86\cdot \mathrm{\Delta }\hspace{0.17em}{X}_{\mathrm{pelvis}}+1.96\cdot \mathrm{\Delta }\hspace{0.17em}{\dot{X}}_{\mathrm{pelvis}}\pm 0.07\cdot \mathrm{\Delta }\hspace{0.17em}{\dot{Y}}_{\mathrm{pelvis}},\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{with}\hspace{0.17em}{R}^2=0.85\end{array}$$

$$\begin{array}{rl}\text{and}\mathrm{\Delta }\hspace{0.17em}{Y}_{\mathrm{foot}}=& \mp 0.47\cdot \mathrm{\Delta }\hspace{0.17em}{X}_{\mathrm{pelvis}}\mp 1.04\cdot \mathrm{\Delta }\hspace{0.17em}{\dot{X}}_{\mathrm{pelvis}}\\ & +0.81\cdot \mathrm{\Delta }\hspace{0.17em}{\dot{Y}}_{\mathrm{pelvis}},\text{with}\hspace{0.17em}{R}^2=0.3,\end{array}$$

corresponding to $K=[1.86,\hspace{0.17em}1.96,\pm 0.07;\mp 0.47,\mp 1.04,\hspace{0.17em}0.81]$. See electronic supplementary material, S5 for the confidence intervals for these calculations.

These linear descriptions have high R² values (R² = 0.85) for sideways foot placement, suggesting that a much more complex model with more state variables or nonlinearities may not explain the foot placement much better. There is no a priori necessity for linear models to capture such a large fraction of the variability. By contrast, the fore-aft foot placement is less well explained by the linear model (R² = 0.3). This may be because in the fore-aft direction, humans use more within-foot modulation of centre of pressure to correct CoM state perturbations, so that the necessity of stepping-based foot placement control may be less [17]. Furthermore, we speculate that the lower explanatory power of the linear model may suggest the use of more complex models, having nonlinearities or more state variables, or more sensorimotor noise in that direction (given that motor noise may scale with forces, which are greater in the fore-aft than in the sideways direction).

3.4. Step in the direction of the perturbation

This foot placement controller suggests that humans step in the direction of the perturbation, as determined previously in natural variability and for smaller perturbations [7,21]. Specifically, a rightward push during left stance will result in the next right foot placement to be more rightward of its usual position; if the push is leftward, the right foot placement is left of its usual position. Similarly, when pushed backward, the human takes a shorter next step, and by extrapolation, takes a longer step when pushed forward. Furthermore, there is coupling between sideways foot placement and fore-aft motion, and vice versa. In each case, ‘how much’ the subject changes their foot-position in the next step depends on their pelvis-state deviation and the coefficients (gains) in the linear equations. While the remarks in this and the next paragraph refer to the foot placement control based on torso state directly derived from human data (equations (3.1)–(3.2)), we find that the same properties hold for the final foot placement controller for the inverted pendulum model, discussed later in equations (3.3)–(3.4).

3.5. Perturbation-derived foot placement gains were qualitatively similar to that for unperturbed walking

The foot placement controller is qualitatively similar to the foot-placement controller that was derived using the same methods but using natural variability in unperturbed walking (as in [21]). That is, the same terms had large coefficients and the signs of the coefficients were the same. Specifically, performing a linear regression to predict coefficients derived from unperturbed walking to those from perturbed walking [21], we obtain a 96.2% R² with a slope of 0.93, suggesting considerable correlation between the coefficients across the two conditions; the largest difference in coefficients is in the coefficient relating X_foot and ${\dot{X}}_{\mathrm{pelvis}}$ (see figure 7). The quantitative differences between the foot placement gains derived perhaps suggests a small nonlinearity or a more complex underlying controller for larger perturbations. The qualitative similarity supports the use of natural variability to infer details of the controller. One makes different underlying assumptions when using small natural variability versus large externally applied perturbations, and that they produce qualitatively similar controllers suggests that the results are robust to such different assumptions.

Figure 7.

Unperturbed walking vs perturbed walking. The foot placement gains inferred from the perturbation experiments herein (horizontal axis) are quantitatively similar to those derived from unperturbed walking (vertical axis) in a previous study [10]. The similarity two sets of gains is characterized by the best fit line y = 0.93x − 0.07, with an R² = 0.963. It appears that the largest relative difference in the gains is in the response of sideways foot placement X_foot to deviations in mid-stance $\dot{X}$_pelvis. Dropping this gain in the comparison gives an R² = 0.996 and a regression equation y = 1.05x − 0.03. Note that the gains from [10] have been non-dimensionalized here for direct comparison.

3.6. A full-state controller for the inverted pendulum walker

Fitting the controlled inverted pendulum walker to the human perturbation recoveries, we obtain the following relations for the posited feedback controller:

$$\mathrm{\Delta }\hspace{0.17em}{X}_{\mathrm{foot}}=2.08\cdot \mathrm{\Delta }\hspace{0.17em}{X}_{\mathrm{CoM}}+2.24\cdot \mathrm{\Delta }\hspace{0.17em}{\dot{X}}_{\mathrm{CoM}}\mp 0.3\cdot \mathrm{\Delta }\hspace{0.17em}{\dot{Y}}_{\mathrm{CoM}},$$ 3.3

$$\mathrm{\Delta }\hspace{0.17em}{Y}_{\mathrm{foot}}=\mp 0.55\cdot \mathrm{\Delta }\hspace{0.17em}{X}_{\mathrm{CoM}}\mp 1.07\cdot \mathrm{\Delta }\hspace{0.17em}{\dot{X}}_{\mathrm{CoM}}+0.99\cdot \mathrm{\Delta }\hspace{0.17em}{\dot{Y}}_{\mathrm{CoM}}$$ 3.4

$$\begin{array}{rl}\text{and}\hspace{0.17em}\mathrm{\Delta }\hspace{0.17em}{I}_{\mathrm{push-off}}=& \mp 0.05\cdot \mathrm{\Delta }\hspace{0.17em}{X}_{\mathrm{CoM}}\mp 0.60\cdot \mathrm{\Delta }\hspace{0.17em}{\dot{X}}_{\mathrm{CoM}}\\ & -0.13\cdot \mathrm{\Delta }\hspace{0.17em}{\dot{Y}}_{\mathrm{CoM}},\end{array}$$ 3.5

corresponding to a feedback gain matrix:

$$J_4=\left[\begin{array}{ccc}2.08& 2.24& \mp 0.3\\ \mp 0.55& \mp 1.07& +0.99\\ \mp 0.05& \mp 0.60& -0.13\end{array}\right].$$

The impulse controller has the intuitive property that when pushed forward, the push-off impulse decreases and when pushed backward, the push-off impulse increases. More significantly, the push-off impulse is coupled to the sideways perturbations. When on a left stance phase, a push to the right results in a smaller subsequent push off and a push to the left results in a larger push off. The foot-position controller has the same qualitative interpretation as described earlier, but is slightly different quantitatively from that derived directly from human data, a trade-off made by the optimization to better approximate the step-to-step map. Electronic supplementary material, S6 describes how well this controller approximates human behaviour.

3.7. The controlled biped simulation is stable and approximates human responses to perturbations

The derived controller stabilizes the simulated inverted pendulum biped walking with a human-like step length, step width and speed. That is, small perturbations decay exponentially and the biped approaches the nominal periodic motion.

The physical mechanisms by which the foot placement and the impulse controller achieve stability are intuitive: by stepping in the direction of the perturbation, the controller changes the leg force direction, thereby increasing the horizontal component of the leg force in a direction opposing the perturbation. This serves as a restoring force to counter the perturbation. The impulse controller acts in a similar way, increasing the impulse when the forward velocity is too low, thereby helping to move back toward the desired forward velocity.

More specifically, perturbations to this simulated inverted pendulum biped produce recovery trajectories that qualitatively resemble corresponding human recoveries in our experimental data (figure 4a,b). More quantitatively, pooled across all the highly perturbed steps, after one complete step, the linear step-to-step map J₁ approximates the sideways position with an average R² of 0.58, the sideways velocity with an average R² of 0.52, and the fore-aft velocity with an average R² of 0.27 (see electronic supplementary material, S5). Overall, these R² values suggest that the linearized model approximates the sideways trajectories better than the fore-aft trajectories. Additionally, as a consistency check, for the range of perturbations applied in the experiment, we found that the linearized mid-stance-to-mid-stance inverted pendulum dynamics in equation (2.4) is a good approximation of the full nonlinear inverted pendulum model (electronic supplementary material, figure S5).

Given that we fit the model to the data, it may seem that this qualitatively similarity is again just a consistency check. However, while we fit the biped model controller to the mid-stance to mid-stance map derived from human responses, we did not fit full shapes of the response trajectories. Thus, that the shape of the perturbation response trajectories for the body is similar for the data and the model suggests that the model, despite being simple and having only a small number of parameters to tune for the fits, has successfully captured the key elements of the response. In fact, even the linear stability of the simulation is not inevitable and relies on the ability of the controlled inverted pendulum model to approximate the step-to-step map of the human. Thus, a priori, all three of the following were possible for the controlled inverted pendulum model: (a) the simulation is unstable, (b) the simulation is stable but is not sufficiently similar to humans, and (c) the simulation is stable and is similar to humans in its nominal gait and recovery transients.

3.8. Controller can reject large disturbances

We computed an approximate ‘basin of attraction’ for the controlled biped (figure 5) by determining the set of perturbations that does not result in the biped falling down during the next 20 steps and converging asymptotically to the nominal periodic motion. Consider uniaxial perturbations, applied only along one of the three directions, x_CoM, ${\dot{x}}_{\mathrm{CoM}}$ or ${\dot{y}}_{\mathrm{CoM}}$, while the other two are unperturbed at mid-stance. In each direction, figure 5 shows that the controlled biped can handle 20–30 times higher state perturbations than the standard deviation of the natural variability we measured in normal ‘unperturbed’ steps (see electronic supplementary material, table S1) and about 7–10 times the standard deviations of the highly perturbed steps (see electronic supplementary material, S5). Compared to these large basins of attraction for the controlled biped, the uncontrolled 3D inverted pendulum biped is unstable, and thus has a zero basin of attraction.

Figure 5.

Basin of attraction. The biped is given an initial mid-stance disturbance in two out of three of the states and then simulated for 20 steps. Disturbance combinations where the biped does not fall down are marked for the (a) ${\dot{Y}}_{\mathrm{CoM}}=0$ plane, (b) ${\dot{X}}_{\mathrm{CoM}}=0$ plane, (c) X_CoM = 0 plane. All quantities plotted are non-dimensional. (Online version in colour.)

3.9. Controller is robust to small deviations in feedback gains

Varying parameters one at a time, we find the range for each of the nine gains for which the biped remains stable. The absolute magnitude of all the eigenvalues is used to determine if the biped is stable. As we vary each of the nine gains in the inverted pendulum controller one at a time, we find that small, and in some cases even large, deviations in these gains do not cause the biped to become unstable. The exact range of errors for each of the controller gains can be seen in figure 6a.

Figure 6.

Robustness to parametric uncertainty. The biped is given an initial mid-stance perturbation. (a) An error is introduced in the value of each of the nine gain parameters. The eigenvalues for the 3 × 3 state transition matrix are measured. The range of gain errors for which the maximum absolute eigenvalue is less than 1 are denoted by the blue bars. (b) An error is introduced in one of the body parameters. The biped is simulated for 20 steps, the range of body parameter errors for which the biped does not fall down, i.e. the CoM remains above the ground, are marked with the black bars. All quantities shown are non-dimensional, so the leg length, gravity and mass errors admissible (as shown in b) are deviations from their nominal non-dimensional unit values, namely unity. Similarly, the errors admissible in the feedback gain terms are deviations from their nominal non-dimensional values given in equations (3.3)–(3.5). (Online version in colour.)

3.10. Controller can accommodate large changes to body properties

The system remains stable despite changes in body mass from $-50\text{\%}$ to +100%, leg length from $-10\text{\%}$ to +120% and gravity from $-50\text{\%}$ to +50%. The range of these errors can be seen in figure 6b. Such robustness to parameter changes will allow the biped to walk stably despite uncertainty about such parameters; for instance, such robustness may allow humans to walk stably with the same controller while carrying an unknown additional mass.

4. Discussion

The foot placement controller described here uses the lateral state of the CoM to determine step width similar to previously reported models [5,21,23]. Kim & Collins [27] describe a controller that uses both foot placement and push-off impulse to stabilize a biped model with more degrees of freedom than the one described here. Their controller allows this more complicated biped to tolerate a sideways impulse disturbance of ±6.3 N · s for a biped of mass 74 kg which equals a non-dimensional sideways CoM velocity disturbance of ±0.027 compared to a range of −0.20 to +0.26 for our simpler controlled biped.

The goal of this paper was to use human data to describe a method for deriving general controllers for models of walking and apply that specifically to the simplest biped model. To the extent that this simple biped model captures something about human biomechanics, the controller derived may shed light on the human control of walking. Indeed, our controller model responds qualitatively similarly to a human. By following the same procedure with more realistic biomechanical models, potentially constrained by available information about the structure of neural feedback, we can obtain controllers for such more complex models, perhaps approaching that of the true controller.

Specifically, these methods can be applied equivalently to any other model of human walking and any valid controller formulation for the selected model. A natural next model would be one that still uses massless legs, but includes a 3D upper body (pelvis), so that the controller needs to manage the orientation dynamics as well. We could also consider models that can manipulate the ankle or hip joints, where each joint torque is dependent on the current or past state of the body. Fitting much higher degree-of-freedom models to the data may not be feasible, as such fitting may require a higher dimensionality of perturbations than we achieved. Approaches based on the neuromechanics of walking [41] might be more suitable in these cases, where we have a small set of neural feedback parameters that can be tuned to best fit the perturbation recovery data. However, this requires us to characterize, using perhaps neural recordings, the structure of the neural controller. We have made other similar simplifications in our assumptions here, for instance, related to linearity of the control mechanisms, which could be generalized to nonlinear terms, and focusing on mid-stance state, which can be generalized to state at any point during the stance phase.

Prior work [7,15,17,21] suggests that the perturbation phase (given a perturbation magnitude) may affect the subsequent recovery response, as we will examine in future work. This difference between perturbation phases may be due, in part, to how a perturbation at one gait phase grows or decays even due to feedforward control as the gait proceeds forward to another gait phase. This is certainly true for the inverted pendulum walking gait, where say, a rightward perturbation during the early part of a left stance phase may grow over the stance phase due to inverted pendulum dynamics; so, in this case, a given perturbation earlier in stance is equivalent to a larger perturbation later in the stance phase. Perturbation phase may also affect future foot placement due to there being considerable sensorimotor delays, between sensing and muscle force development via feedback control. Such delays imply that perturbations that occur just before the foot placement may not leave enough time for neurally mediated feedback control to occur.

Complete simulations of bipedal walking, such as those demonstrated here for the simple inverted pendulum biped, could allow us to devise measures of stability which extrapolate the information from our model-based controllers. For instance, we could repeat this procedure for subjects with movement disorders, amputees, or the elderly [5,42], potentially with much smaller perturbation sizes, to characterize how their walking controller differs from those of young healthy non-amputee individuals. We could then predict probability of falls due to specific perturbations or compare the basin of attraction for controllers derived from any individual with those derived from subjects with a low fall risk. First, however, these measures would need to be validated either by deriving similar controllers from individuals known to have a high fall risk or by conducting experiments with even larger perturbations to determine whether the basins of attraction accurately measure the limits of stability for healthy individuals. Even after validation, we would still need to compare the effectiveness of such measures relative to other stability measures suggested by others [43]. The notion of stability of an equilibrium or a periodic motion cannot be captured by one number, and different metrics of stability need not be correlated. For instance, a system with a large basin of attraction can have lower robustness or attraction to the stable periodic motion.

Obtaining the foot placement and mid-stance-to-mid-stance linear models for individual subjects indicates qualitative similarity across subjects with some quantitative differences in the coefficients (electronic supplementary material, figures S4 and S5). To meaningfully compare the inferred feedback gains from different subjects or subject populations or experiments, we would first need to accurately characterize the trial to trial variability of such gains for individual subjects: repeating over different days and repeating with different experimenters with similar marker-placement instructions. One potential way to reduce variability due to experimental practice or marker placement may be to obtain CoM position and velocity estimates by combining integrated ground reaction force data and marker data with an appropriate Kalman-like filter to avoid drift [33,44].

We had the subjects walk with their arms crossed to allow for applying sideways perturbations without being interrupted by the swinging arm. This restriction of arm movement prevents the subjects from using arms for balancing, which may potentially change how they control walking with their legs. If we repeated these trials with arm movement, we hypothesize that the qualitative features of control inferred here will be preserved but potentially with some quantitative differences in stepping and push-off strategies [45,46]. This hypothesis will be tested in future work, and is plausible on account of another study [21] allowing some arm swing (without perturbations) did infer similar strategies. Another simplification in the experimental protocol was that the fore-aft perturbations we applied were only backward, but not forward. This will not have an effect on the inferred controllers if we are still in the linear regime, whereas a nonlinear controller may show different response magnitudes for forward and backward perturbations, and indeed perturbations in other directions in the horizontal plane spanned by fore-aft and sideways directions. Finally, we only considered discrete perturbations to the pelvis and did not consider other kinds of perturbations or perturbations to other parts of the body. If such alternative perturbations eventually affect the CoM state, our inverted pendulum biped controller can make predictions for the subsequent control action, which can be tested in future studies.

All the human-derived results presented here are across pooled sideways and backward perturbation experiments. The presence of small nonlinearities may partly explain differences in R² values when foot placement linear models (as in equations (3.1)–(3.2)) were derived using just the sideways perturbation trials or just the backward perturbation trials. Specifically, the R² for fore-aft foot placement was higher (0.52) when derived from the backward perturbation experiments and lower (0.25) when derived from the sideways perturbation experiments, compared to 0.3 when derived from all trials pooled. Conversely, the R² for the sideways foot placement was lower (0.75) when derived from just the backward perturbation experiments, compared when all the data were used (0.85).

The controller we have derived is applicable directly to level ground constant speed walking. Of course, real human locomotion involves walking with turning, on stairs and slopes, with changing speeds, running, etc. While our controller does not apply directly to these situations, the methods used herein can be repeated in those circumstances to produce a more general or at least task-specific or context-specific controller, which we hope to explore in future work. Furthermore, we hypothesize that the controller, given its robustness, is able to walk stably at least on small slopes and slightly uneven terrain.

We have used the word ‘control’ throughout this article to characterize how the human subject and the mathematical model respond to perturbations, for instance, through the modulation of foot placement and the push-off impulse. To be clear, for the human, this control need not entirely be due to neurally mediated feedback control, but instead due to the interaction between the feedforward dynamics of the human body (given some fixed motor outputs to muscles, say) and sensor-driven feedback control, which could be spinal or cortical. Identifying the relative contributions of feedforward and feedback control may require experiments that measure motor outputs after a perturbations to see if they change, and measure degradation in locomotor performance after degrading sensory feedback. For the mathematical model itself, both the foot placement and the impulse controllers are entirely feedback controlled, as these quantities can be directly controlled in the model as inputs to the inverted pendulum dynamics.

Human-derived controllers, such as proposed here, can have other practical applications. For instance, they could be used to inform the development of better feedback controllers for robotic prostheses and exoskeletons [31,32]. Current robotic prostheses and exoskeleton controllers, when based on human walking, usually only use information regarding the average motion or torques during walking, but not how humans respond to perturbations. Implementing a human-derived controller into such devices may make them more ‘natural’ to the user, and perhaps reduce the duration needed to learn to use such devices effectively. More generally, such human walking controllers could inform the design of other biomechatronic assistive devices or even passive objects such as backpacks, so as to maximize stability while walking with such devices or objects. Analogously, one could implement controllers derived from human walking directly onto walking robots [29,30]. Such human-inspired robot controllers can make the robots more human-like, walking and respond to perturbations like humans do. Our methods could also be applied in the context of human learning, both children learning to walk or other humans learning to walk under a new circumstance (e.g. wearing a new device or on a new surface), which will allow us to track how the controller and stability properties change and (presumably) improve during learning [32,47].

To be clear, there is an infinity of possible controllers that would stabilize the inverted pendulum gait (e.g. [25–28,30]). For any given stable gait, generically, a nearby feedback-gain set will also be stable, unless the system is at the boundary of stability (which is not the case here). Indeed, figure 6 shows the set of feedback gains that stabilizes the biped. In addition to these feedback controllers, we could imagine controllers with other architectures, for instance, using feedback from many previous steps than just the previous or current one.

One general goal in robotics is to develop controllers that might result in bipedal walking robots that are more stable and more efficient than current bipeds with ad hoc controllers. To achieve stable and efficient locomotion, bipedal legged robot controllers need not necessarily track the human walking controller. However, it may be desirable for psychological or social reasons to control a robot similarly to how a human walks, that is, build robots that not just look like humans, but also behave in a similar way, up to the way they recover from perturbations. While biomimetism in robots may not always be optimal, it sometimes can be despite different cost functions for humans and robots (e.g. [1] showed such coincidence in simple models). It is an open question as to whether biomimetic control is reasonably efficient and robust, and one can only settle this question by implementing such controllers on actual hardware. This study represents a step towards, and provides a general method for, the development of such biomimetic controllers. We note that foot placement control with qualitative similarities with that obtained here is often used in complex robots, although not directly biomimetic [30,36].

Ethics

Experimental protocols were approved by the Ohio State University IRB and all subjects participated with informed consent.

Authors' contributions

V.J. and M.S. conceived the study. V.J. performed all the human subject experiments. V.J. performed all data analyses and computational simulations, partly in discussions with M.S. V.J. wrote the first draft and V.J. and M.S. edited further. All authors approve the final draft.

Competing interests

We declare no conflicts of interest.

Data accessibility

The data and code will be available without restrictions through the figshare database. Code is available at: https://doi.org/10.6084/m9.figshare.9337445.v1 (doi:10.6084/m9.figshare.9337445) and the human data is available at: https://doi.org/10.6084/m9.figshare.9273491.v1 (doi:10.6084/m9.figshare.9273491).

Funding

This work was supported by the National Science FoundationCMMI grant nos. 1538342 and 1254842. Thanks to Movement Laboratory members for helping apply sideways perturbations for a few subjects.