Passivity-Based Control with a Generalized Energy Storage Function for Robust Walking of Biped Robots

Abstract

This paper offers a novel generalization of a passivity-based, energy tracking controller for robust bipedal walking. Past work has shown that a biped limit cycle with a known, constant mechanical energy can be made robust to uneven terrains and disturbances by actively driving energy to that reference. However, the assumption of a known, constant mechanical energy has limited application of this passivity-based method to simple toy models (often passive walkers). The method presented in this paper allows the passivity-based controller to be used in combination with an arbitrary inner-loop control that creates a limit cycle with a constant generalized system energy. We also show that the proposed control method accommodates arbitrary degrees of underactuation. Simulations on a 7-link biped model demonstrate that the proposed control scheme enlarges the basin of attraction, increases the convergence rate to the limit cycle, and improves robustness to ground slopes.

I. Introduction

An idea that is key to biped locomotion is underactuation. During human gait, the foot rolls across the ground from heel to toe, which does not represent a controlled interaction with the environment [1]. This is characteristic of how most bipeds interact with the environment, and it introduces a degree of underactuation in the system. Some biped models (e.g., the compass-gait model [2]) shortcut this challenge by combining the stance foot and ankle together into a single point that “sticks” to the ground until the other foot makes contact with the ground. Other approaches embrace the underactuation and design a controller that creates a stable periodic orbit in the Hybrid Zero Dynamic (HZD) manifold as in [3]. This is done by enforcing a set of time-invariant trajectories that are obtained by an optimization procedure and using a feedback controller to enforce asymptotic convergence to the zero dynamics manifold. This methodology has enjoyed success on a wide range of systems as demonstrated in [4]. However, it lacks an inherent notion of utilizing the natural system dynamics.

A particularly well-known phenomena in biped locomotion is that of passive dynamic walking, as first reported by McGeer et al. in [5], where an uncontrolled biped is able to walk down a shallow slope simply under the power of gravity. In other words, a stable walking gait can naturally emerge from the mass/geometry properties of the biped and environment. During this process, the exchange of potential to kinetic energy over the duration of a step is exactly canceled by inelastic impacts with the ground. The mechanical energy of the system remains constant and is conserved from step to step. This stable walking can be characterized by a limit cycle in the phase space of the biped as reported in [6].

Goswami et al. were the first to exploit these natural dynamics and use passivity-based control (PBC) to explicitly drive the energy of the biped to a constant value along its natural limit cycle to induce stable walking [6]. Others have since built on these ideas and demonstrated more sophisticated examples of energy-tracking PBC that improve different properties of the limit cycle such as increasing the basin of attraction and increasing the convergence rate to the limit cycle [7]. Separate methodologies such as Energy Shaping [8]–[10], IDA-PBC [11], and the Control Lyapunov Function (CLF) method [12] have also utilized these connections between energy and limit cycles to create stable walking gaits. However, some of these methods historically require assumptions that limit their applicability. Energy shaping relies on the system meeting a matching criteria that restricts the degree of underactuation allowed [13]. Most passivity-based energy methods rely on the existence of a limit cycle with constant mechanical energy. This assumption prevents application on high-dimensional biped models, which typically require some external control action to inject and/or dissipate energy in order to create a limit cycle. While the CLF method has been shown to perform well on these systems [14], it is not passive. There is a gap to be filled in the realm of passivity-based control with applications to high dimensional underactuated systems.

This paper presents a passivity-based controller based on a generalized energy expression in the storage function, which defines a novel passive output that accounts for the energy stored and dissipated by an arbitrary inner-loop controller. It is assumed that this inner-loop controller generates a stable limit cycle for the biped on a given slope. The outer-loop PBC will then increase the basin of attraction, improve the robustness to the ground slope, and increase the rate of convergence back to the stable limit cycle. The control method is also shown to perform with an arbitrary degree of underactuation in the system. The rest of the paper subscribes to the following format: Section II introduces the dynamic model of the biped. Section III offers a brief review of passivity and derive a PBC from an energy based storage function. Finally, Section IV demonstrates simulation results on a 7-link biped model that utilizes a PD controller in the inner loop to create a stable limit cycle.

II. Modeling and Dynamics

In this paper, we consider a 2D biped model with torque only in the sagittal plane. For simplicity, we model the link between the two hip joints as a single joint and omit a torso link. Thus, together with a foot, shank, and thigh link for each leg we have a 7-link biped model. We model it as a kinematic chain with respect to an inertial reference frame (IRF) defined at either the stance heel or stance toe, depending on the phase of the single-support period (to be discussed in Section II-B). A diagram of the biped is shown in Fig. 1.

Fig. 1.

Kinematic model of the biped. COP denotes the Center of Pressure. The solid links denote the stance leg, the dashed links denote the swing leg.

The generalized coordinates of the biped model are defined as q = (p_x, p_y, ϕ, θ_a, θ_k, θ_h, θ_sk, θ_sa)^T ∈ ℝ^8×1, where p_x and p_y represent the Cartesian position of the stance heel in the inertial reference frame, and ϕ is the angle of the heel-to-ankle vector with respect to the vertical axis. The subscript i ∈ {a, k, h, sk, sa} (denoting the ankle, shank, hip, stance knee, stance ankle, respectively) is used to describe the angles θ_i between each link. The mass m_j, length l_j and inertia I_j of the links are indexed by the subscript j ∈ {f, s, t, h} which denotes the foot, shank, thigh, and hip, respectively.

A. Continuous Lagrangian Dynamics

The dynamics are derived using the Lagrangian formulation [15] to get the equation

$$M(q)\ddot{q}+C(q,\dot{q})\dot{q}+N(q)+A{(q)}^T\mathrm{\lambda }=\tau ,$$ (1)

where M(q) ∈ ℝ^8×8 is the inertia matrix, $C(q,\dot{q})\in {\mathrm{ℝ}}^{8\times 8}$ is the Coriolis/centrifugal matrix, and N(q) ∈ ℝ^8×1 is the gravity force vector. The term A(q)^T λ models the interaction between the biped’s foot and the ground, where the matrix A(q) ∈ ℝ^c×8 is defined as the gradient of the constraint functions, and c is the number of contact constraints that may change during different contact conditions. The Lagrange multiplier λ ∈ ℝ^c×1 is calculated using the method in [16], [17] and satisfies the assumption that the ground reaction forces do no work on the system. The torque vector is τ = B_uu+B_vv where B_u ∈ ℝ^8×d and B_v ∈ ℝ^8×m are mappings of the outer-loop PBC torques u and arbitrary inner-loop control torques v into the generalized coordinates, respectively. The number of control inputs d and m do not need to be the same and are addressed later.

B. Contact Constraints

Based on [18], [19], the single-support period can be broken down into three sub-phases: heel contact, flat foot, and toe contact, where holonomic contact constraints can be properly defined. Following the convention in [16], we express the holonomic contact constraints of the biped as relations between the position variables of the form

$$a(q_1,q_2,\dots ,q_c)=0_{c\times 1},$$ (2)

where q_j denotes the j-th element of the configuration vector q. There are c = 2 constraints for heel contact and toe contact whereas flat foot has c = 3 constraints. The constraint matrix can then be defined for all contact conditions as

$$A_{\mathit{sub}}=\frac{\partial a(q)}{\partial q}=[I_{c\times c}{0}_{c\times (8-c)}],$$ (3)

where sub ∈ {heel, flat, toe}. This form can be achieved by defining the IRF at the stance heel during heel contact and flat foot vs. the stance toe during toe contact.

C. Hybrid Dynamics

Biped locomotion can be modeled as a hybrid dynamical system which includes continuous and discrete dynamics [20]. The system follows a sequence of continuous dynamics and their discrete transitions, i.e., it cycles through different contact configurations defined in Sec. II-B during stance period and encounters impacts when the swing heel hits the ground or the flat foot slaps the ground. Following the same assumption in [20], our model only allows an instantaneous double-support phase and perfectly inelastic collisions. The velocity of the biped changes instantaneously after each impact while the position of the biped remains unchanged.

Based on the method in [16], the hybrid dynamics and impact maps during one step are computed in the following sequence:

$$\begin{array}{ccc}1.\hfill & M\ddot{q}+T(q,\dot{q})+A_{\text{heel}}^T\mathrm{\lambda }=\tau \hfill & \text{if}{a}_{\text{flat}}\ne 0,\hfill \\ 2.\hfill & {\dot{q}}^{+}=(I-X{(A_{\text{flat}}X)}^{-1}{A}_{\text{flat}}){\dot{q}}^{-}\hfill & \text{if}{a}_{\text{flat}}=0,\hfill \\ 3.\hfill & M\ddot{q}+T(q,\dot{q})+A_{\text{flat}}^T\mathrm{\lambda }=\tau \hfill & \text{if}\left|c_{\mathrm{p}}(q,\dot{q})\right|<l_{\mathrm{f}},\hfill \\ 4.\hfill & {\dot{q}}^{+}={\dot{q}}^{-},{(q{(1)}^{+},q{(2)}^{+})}^T=\mathcal{G}\hfill & \text{if}\left|c_{\mathrm{p}}(q,\dot{q})\right|=l_{\mathrm{f}},\hfill \\ 5.\hfill & M\ddot{q}+T(q,\dot{q})+A_{\text{toe}}^T\mathrm{\lambda }=\tau \hfill & \text{if}h(q)\ne 0,\hfill \\ 6.\hfill & (q^{+},{\dot{q}}^{+})=\mathrm{\Theta }(q^{-},{\dot{q}}^{-})\hfill & \text{if}h(q)=0,\hfill \end{array}$$

where the superscripts “−” and “+” indicate the pre-impact and the post-impact values, respectively. Also, $X=M^{-1}{A}_{\text{flat}}^T$, $\mathcal{G}={(l_{\mathrm{f}}\cos (\gamma ),l_{\mathrm{f}}\sin (\gamma ))}^T$, models the change in inertial reference frame, c_p is the trajectory of the COP, γ is the ground slope angle, and l_f is the foot length. The vector T groups the Coriolis/centrifugal terms and potential forces for brevity. The ground clearance of the swing heel is denoted by h(q), and Θ denotes the swing heel ground-strike impact map derived based on [20].

III. Energy and Passivity-Based Control

The passive compass-gait biped has no external force input during its continuous dynamics, thus the only work done on the system is by the discrete impacts with the ground. On a passive limit cycle, the kinetic energy of the biped is essentially reset after each impact, while the datum defining the potential energy is shifted to reset the potential energy. This gives rise to a constant generalized system energy [6]. A similar phenomena exists for an n-link biped with a controller that does work to cause the biped to follow a limit cycle. During the continuous dynamics, the work done by the controller exactly accounts for the change in the mechanical energy. If the work is reset to zero after each impact (which we can enforce by convention), then the generalized system energy is still constant on the limit cycle [14].

As shown in [14], we can define the generalized system energy as

$$E=K(q,\dot{q})+P(q)-W.$$ (4)

The mechanical energy of the system is the kinetic energy K plus the potential energy P of the biped, while the work done by the inner loop controller is

$$W={\displaystyle {\int }_0^{\stackrel{\sim }{t}}{\dot{q}}^T{B}_{v}vdt,}$$

which accounts for the energy stored, added, and dissipated over time $\stackrel{\sim }{t}$ by the controller torque v.

We can then consider the following storage function from [2] for the derivation of a passivity-based controller:

$$S=\frac{1}{2}{(E-E_{\mathit{ref}})}^2$$ (5)

where E_ref is the reference energy defined on a given limit cycle. Taking the time derivative of S, we obtain

$$\dot{S}=(E-E_{\mathit{ref}})(\dot{E}-{\dot{E}}_{\mathit{ref}}).$$

If we only consider Ė_ref ≡ 0, then

$$\dot{E}=\frac{d(K+P)}{dt}-\frac{dW}{dt}.$$

From the definition of W, the application of the fundamental theorem of calculus and the conservation of energy in a mechanical system yields

$$\dot{E}=({\dot{q}}^T{B}_{u}u+{\dot{q}}^T{B}_{v}v)-{\dot{q}}^T{B}_{v}v={\dot{q}}^T{B}_{u}u.$$

The time derivative of the storage function becomes

$$\dot{S}=(E-E_{\mathit{ref}}){\dot{q}}^T{B}_{u}u$$

with passive output

$$y^T=(E-E_{ref}){\dot{q}}^T{B}_u\in {ℝ}^{1\times d}.$$

If we feed back a scaled form of the passive output y, the outer loop control law that we arrive at is

$$u=-\mathrm{\Lambda }y=-\mathrm{\Lambda }(E-E_{\mathit{ref}})B_u^T\dot{q},$$ (6)

where Λ is a positive definite diagonal gain matrix. By substituting in the control law, we have

$$\dot{S}=-{(E-E_{\mathit{ref}})}^2{\dot{q}}^T{B}_u\mathrm{\Lambda }{B}_u^T\dot{q}=-2kS{\Vert \dot{q}\Vert }_{\mathrm{\Omega }}^2,$$

where ${\Vert \dot{q}\Vert }_{\mathrm{\Omega }}^2={\dot{q}}^T{B}_u\mathrm{\Omega }{B}_u^T\dot{q}$ is the square of a weighted norm, and Λ = kΩ. The variable k is a scaling factor which we can treat as a gain, while Ω is a diagonal positive semi-definite matrix that assigns relative weights in the norm. If we make the assumption that ${\Vert \dot{q}\Vert }_{\mathrm{\Omega }}^2\ge \eta$, then we arrive at a similar result to [2] with

$$S(t)\le S(0)e^{-2k\eta t},$$

which proves the exponential convergence to the limit cycle. By inspection we can see that the gain k provides a method of directly influencing the bound on the convergence rate of the storage function. The lower bound η on the norm is difficult to determine over the general phase space, analytically or computationally, since it varies with the system velocity.

The benefit of this formulation is that the PBC is capable of improving the storage function convergence regardless of the degree of underactuation, if the system operates far away from any stable equilibrium points in the actuated phase space. This requirement ensures that η = 0 is transient condition, and the storage function can be bounded by a new exponential function after passing through this state in the phase space. Any initial condition in the basin of attraction of a limit cycle satisfy’s this requirement.

One of the beneficial properties of PBC is that it is easy and natural to extend these results to the case of an actuator with saturation. If we consider a saturated version of the control

$$u=\mathit{sat}(-\mathrm{\Lambda }(E-E_{\mathit{ref}})B_u^T\dot{q})$$

such that

$$\dot{S}=-y\mathrm{\Lambda }\mathit{sat}(y)\le 0,$$

the resulting system is still passive because the function output preserves the sign of the input, similar to the results in [21].

Throughout this derivation, the form and properties of the inner loop control law v were left unspecified. This seems to indicate that an additional benefit of the proposed PBC approach is the potential to work with arbitrary inner loop controllers that generates a stable limit cycle, due to the form of the system energy (4) and storage function (5). In this paper we choose to use a PD controller to establish a stable limit cycle for the biped as in [16], [22], for simplicity. The control is defined as

$$v=-K_p(q_m-\delta )-K_d{\dot{q}}_m,$$

where q_m is the actuated coordinates vector, δ is the equilibrium vector, and the control gain matrices are denoted as K_p, K_d ∈ ℝ^5×5.

IV. Simulation Results and Discussion

To study the possible benefits of the proposed PBC, we conducted series of simulations on the 7-link biped model introduced in Section II, where the model parameters were chosen from [16, Table I]. We begin by finding a nominal limit cycle walking on a slope α = 0.095 rad under the influence of the PD control solely, which is used as the baseline comparison for the rest of this section. We then discuss the effects of the PBC on the rate of convergence and basin of attraction of the limit cycle. Afterwards, we move on to the results of varying the walking slope. Finally, we demonstrate the effects of PBC saturation and underactuation on the biped’s limit cycle.

The biped has three contact configurations and impacts, which causes the limit cycle to transition between 3 different constant system energies, E_ref1 → E_ref2 → E_ref3. This can be seen in the periodic, constant jumps in Fig. 2, which correspond to transitions in contact configurations (starting with heel contact). The y-axis is the generalized system energy E = K + P − W, and these constant values are used as the phase-specific reference energies in the PBC throughout the section.

Fig. 2.

System energy of the biped and PD controller while traversing the limit cycle. There are three constant energy levels with discrete jumps between them.

The simulations in Section IV-A,B are fully actuated in the outer loop during the flat foot phase and are underactuated with degree one during heel and toe contact. The constant parameters used are Ω = [0, 0, 0, 8, 3, 0.01, 3, 8]I_8×8. The first three entries in the diagonal vector that are zero correspond to (p_x, p_y, ϕ), which are contact constrained variables over the course of a step. The gains are chosen so that a joint does not experience an instantaneous switch in its control gain when switching from stance to swing. The numerical parameters for the biped and the PD controller can be found in [16].

A. Rate of Convergence and Basin of Attraction

Typically, the basins of attraction of passive biped limit cycles are quite small and sensitive to perturbations. The initial positions and velocities of the system must be close to the limit cycle, which can be difficult for a human to manually achieve by positioning and pushing a physical biped [5]. However, the basin of attraction can be significantly enhanced by the addition of the PBC in the outer loop.

When the system is solely under the influence of the PD controller, the storage function S and system energy E (mechanical energy minus the work done by the PD controller) remain constant during the continuous dynamics, and are only changed by the discrete impacts as demonstrated in Fig. 2 and 3. These impacts dissipate energy and cause the biped to converge toward the limit cycle. However, as shown in Fig. 3, implementing the PBC on top of the PD controller causes the storage function to decrease during the continuous dynamics as well. The convergence appears to be exponential, with different rates for each contact condition. Based on the control law derivation and storage function analysis from the previous section, we can conclude that the changes in convergence rate across impact events are due to the instantaneous changes in the norm of the joint velocities (i.e., the bound η changes).

Fig. 3.

A comparison of the perturbed system storage function with and without PBC. The PBC trajectory has k = 0.1.

We present phase portraits of the mechanical energy of the biped versus a phase variable that monotonically increases during each step [23]. This allows meaningful information to be conveyed using a two dimensional graph. The specific phase variable used in these plots is the global hip angle, which is defined from the vertical axis to the vector that connects the stance ankle to the hip. The mechanical energy over our phase variable represents a dimensionality reduction of the phase space onto a 2D plane.

Fig. 4 shows a comparison of the system behavior with and without the PBC when starting from an initial condition that is significantly distant from the limit cycle. With the PBC, the system converges back to the limit cycle in the left plot; the right plot without PBC does not converge and in fact falls over after just two steps. This comparison demonstrates that the basin of attraction of the limit cycle is increased by the PBC.

Fig. 4.

The biped’s phase portraits with (left) and without (right) the PBC under perturbations. Mechanical energy is plotted over a phase variable for the sake of visualizing the limit cycle in two dimensions.

B. Slope Robustness and New Limit Cycles

In addition to being sensitive to initial conditions, passive biped walkers are also sensitive to ground slope [24]. In [2], a passivity-based energy shaping approach was used to render the hybrid dynamics of a compass-gait robot invariant with respect to the ground slope, which, in effect, simply shifted the limit cycle in the phase plane. In contrast, the PBC in this paper causes a quite different effect.

In our simulation, the biped was initialized with its state on the nominal limit cycle for a slope on α = 0.095 rad. The initial slope was changed to α = 0.12 rad and was held constant while the biped converged with the PBC gain k = 1 to the new limit cycle, displayed in the left plot of Fig. 5. The PBC was then turned off, and as a result the biped fell over after 5 steps as shown in the right plot. This indicates that after applying the PBC, the biped has become robust to a wider range of slopes.

Fig. 5.

Biped limit cycle with slope α = 0.12 rad compared with the nominal slope α = 0.095 rad. The left figure shows the change in biped’s phase portrait when varying slope and gain using the PBC. The right figure shows the biped’s phase portrait without the PBC.

By inspection, we can see that the new limit cycle at k = 1 is not simply a shifted version of the nominal limit cycle. It is an emergent new limit cycle created by the PBC, which can be seen by comparing the limit cycles for k = 1 and k = 0.25 for α = 0.12 rad in Fig. 5. Here the biped was allowed to converge to the new limit cycle on the slope α = 0.12 rad with gain k = 1. The gain was then changed to k = 0.25, and as a result the biped converges to a new limit cycle. This suggests that the PBC is doing more than simply stabilizing an unstable limit cycle.

C. Underactuated PBC with Saturation

This section shows a simulation example that is reasonable to implement in a physical system. A minimal amount of motors is often desirable from the perspective of mechanical design, and real motors experience torque saturation. Since the ankle joint provides most of the power injection in bipedal systems [25], this motivates a case study that restricts the actuation to just these joints. For the purpose of maintaining symmetry, we have actuation at both ankles with Ω = [0, 0, 0, 1, 0, 0, 0, 1]I_8×8. Thus, the total degree of underactuation of the outer-loop due to the joint actuators and contact configuration is 6 during the heel and toe contact and 5 during flat foot.

The initial conditions and slope used for the saturated and underactuated simulation were the same as the full actuation case on the nominal slope. The control with ankle actuation alone is still capable of enhancing the basin of attraction of the limit cycle as indicated in Fig. 6. However, one can see that the number of cycles or steps necessary to reach the limit cycle has increased from three steps in the left plot in Fig. 4 to five steps in Fig. 6. This indicates that the convergence rate of the storage function has decreased due to the drop in the number of actuators.

Fig. 6.

Phase portrait of the biped with actuation at both ankle joints (saturated at 100 Nm).

V. Conclusion

This paper enhances the usefulness of passivity-based control for biped walkers and the stabilization of their limit cycles by generalizing both the expression used for the system energy and the control method to arbitrary degrees of underactuation. This underactuation can be enforced by physical properties of a system model such as rolling foot contact, or a result of actuator placement in the system. We show that with these changes the system still enjoys improved properties such as an increase in the basin of attraction, robustness to changes in slope, and increases in convergence rate. The immediate goals of the authors are to perform similar simulations with a variety of more complex inner loop controllers that are commonly applied on legged robots (e.g., Feedback Linearization [26] or HZD [4]). In addition, the ideas presented could have significant impact in the application on powered prostheses. Passivity-based methods are speculated to have good properties for human-machine interaction [16], and this paper specifically addresses the issues of underactuation and saturation that powered prosthetic devices inevitably face.