A Tube-based Model Predictive Control Method to Regulate a Knee Joint with Functional Electrical Stimulation and Electric Motor Assist

Abstract

A hybrid neuroprosthesis system is a promising rehabilitation technology to restore lower-limb function in persons with paraplegia. The technology combines functional electrical stimulation (FES) and a powered lower limb exoskeleton to produce movements for walking and standing. The main control challenge in the hybrid neuroprosthesis is to achieve an optimal coordination between FES and electric motors. Model-based optimal control methods have been suggested for the control of the hybrid neuroprosthesis. However, it is often difficult to effect robust control performance with model-based optimal control methods due to modeling uncertainties. A tube-based model predictive control (MPC) method is developed to obtain robust and optimal coordination between FES and an electric motor during a knee regulation task. An external feedback control is used to limit the error between the actual position and the MPC-computed nominal position. The tube-based MPC method is proven to have recursive feasibility, compliance to input constraints, and exponentially bounded stability. The experimental results obtained from an able-bodied participant and a participant with spinal cord injury validate the controller’s ability to allocate control inputs to FES and the electric motor as well as method’s robustness to modeling uncertainties.

I. Introduction

In the United States, around 117,000 people are reported to have incomplete or complete paraplegia that impairs lower limb motor function due to spinal cord injury (SCI) [1]. This type of injury limits ambulation and negatively impacts quality of life of the affected individuals. Functional electrical stimulation (FES) is often prescribed as a rehabilitation intervention to restore lower limb function. FES artificially stimulates motor units to activate paralyzed muscles. Reanimation of functional movements such as standing and walking have been shown by coordinated stimulation of multiple lower limb muscles [2], [3]. FES is usually applied through transcutaneous electrodes that recruit muscles in a non-selective and repeated manner. Due to the non-physiological nature of the recruitment, FES often causes a rapid onset of muscle fatigue [4], [5].

Alternatively, powered exoskeletons have also been developed to rehabilitate people with paraplegia [6], [7]. Unlike FES that deals with nonlinear and time-varying neuromuscular dynamics, exoskeleton actuators are reliable and generate predictable torques. However, the sole use of powered exoskeletons may require larger batteries for their operation, which may be cumbersome to use, and likely to reduce wearability [8].

Hybrid neuroprostheses that combine FES and powered exoskeleton or FES with electric motor assist have recently been proposed [9]–[17]. The technology is motivated to overcome the limitations of FES and the powered exoskeletons by using them in tandem. For example, the hybrid technology facilitates load sharing, which helps to alleviate FES-caused muscle fatigue and reduce electric motor power consumption. Potentially, lower power and torque requirements can reduce the size and weight of the powered exoskeletons in the hybrid system. The main challenge, however, is to optimally allocate control effort between FES and the electric motor. In our previous research, an offline optimization method was used to optimize a hybrid walking system that uses FES and a passive orthosis [18] and a hybrid leg extension neuroprosthesis [19]. Motivated to optimize FES control in real-time, in [20] a nonlinear model predictive controller (NMPC) to elicit knee extension was developed. This result was extended, in [15], to control knee extensions using a hybrid neuroprosthesis. The experiments were performed on a person with spinal cord injury and a person without disability. The results showed the ability of the hybrid neuroprosthesis to optimally coordinate FES and an electric motor. Further, torque savings were shown in the case of the hybrid neuroprosthesis compared to the electric motor only case. Moreover, the results showed a dynamic allocation of the torque to the electric motor as the muscle fatigued while regulating the knee. However, the NMPC method in [15] is not robust to modeling uncertainties that may hinder its ultimate use in therapy clinics. Modeling uncertainty can easily creep in because first it is often difficult to determine accurate parameters of the musculoskeletal model due to a tedious approach suggested in [15], [21] and secondly, it is well known that the model parameters vary day-to-day and person-to-person [22]. While numerous approaches have been suggested for robust and nonlinear control of FES [5], [23]–[27] and for FES with motor assistance [13], [28], but robust optimal control approaches for combined FES and motor assistance are amiss. Disturbances due to modeling uncertainties can affect control performance in practice and undermine stability of NMPC methods (invalidates the result obtained in a conventional stability analysis, like in [29]). Therefore, motivation exists to design robust NMPC methods for a hybrid neuroprosthesis or an FES system with electric motor assistance.

Various robust MPC approaches have been proposed for general dynamical systems [30]–[35]. Among them, the tube-based MPC is one of the promising robust methods for real-time combined human-machine control. The method does not add further constraints to the optimization problem and may keep computational load and time lower to enable its real-time implementation. The tube-based MPC was well studied for a constrained linear system in [32], where a constant gain feedback controller was used to reject the error between the actual and the nominal state. To deal with the nonlinear systems, another optimization solver outside of a nonlinear model predictive control (NMPC) was used to reject the disturbance [34], [36]. More importantly, a canonical scheme of stability proof, where the terminal state is confined to an invariant set to guarantee the recursive feasibility, was presented in [32]–[35]. Based on the scheme, authors in [37] derived a robust control invariant set approach for a class of Lipschitz nonlinear systems. The work in [38] derived a decentralized tube-based MPC method for multiple agent control. For the case in this paper, we recognized that the scheme can also guide us to develop a tube-based MPC for a hybrid neuroprosthesis.

However, for a system that demands high control frequency, the computational burden makes its real-time implementation a challenge. The attempts of realizing robust MPC in real-time applications was made recently. In [39], a real-time NMPC was developed to control a high-purity distillation column with a sampling time of 10s. The work in [40], [41] realized a tube-based NMPC on a tractor–trailer system (sampling time 0.2s), where a constant gain feedback controller (in the fashion of a linear controller) was used. Although these tube-based MPC methods showed a great success in terms of engineering practice, the theoretical system stability remained to be discovered. The work in [42] theoretically proved that the system state for a nonlinear ground vehicle system reaches an invariant set that is robust to disturbances under a linear feedback controller. In [43], a robust MPC method was developed to control a constrained unicycle robot. A nonlinear feedback law was designed while accounting for input constraints. The robust MPC method was simulated successfully with a sampling time 0.1s. However, the above mentioned tube-based NMPC method may not be generalizable to other nonlinear systems, and thus their application to a hybrid neuroprosthesis remains in question. Further, we were motivated to reduce the sampling time to the centi-second level in order to stabilize the hybrid neuroprosthetic system.

This paper aims to develop a robust NMPC that optimally allocates control between FES and an electric motor in real time with a high control frequency. The control objective is to regulate knee movement during a seated leg extension. A pure electric motor-driven feedback controller is devised with a Lyapunov stability approach to reject the disturbances in the hybrid neuroprosthesis dynamics. The feedback controller drives the actual knee angular position to the nominal position that is planned by the NMPC. A terminal state region and terminal controller is proposed to ensure recursively feasibility and asymptotic stability of the NMPC method. Overall, the tube-based NMPC methods keeps the error between the actual position and the nominal position constrained to a small region along the time horizon. During the experiments, a terminal cost method is used to replace the proposed terminal region for faster MPC implementation and to achieve a higher control frequency. By setting a 0.01s sampling time for the MPC, the experiments were performed on a participant with no disability and a participant with an SCI.

II. Dynamic Model

The dynamics of eliciting leg extension by using the hybrid leg extension neuroprosthesis system [15], illustrated in Fig. 1, is given as

$$M_m\ddot{\varphi }\left(t\right)+M_p\left(\varphi ,\dot{\varphi }\right)+M_g\left(\varphi \right)+\omega \left(t\right)={\tau }_m+{\tau }_{ke}$$ (1)

where ϕ, $\dot{\varphi }$, $\ddot{\varphi }\in ℝ$ denotes the anatomical knee joint position, velocity, and acceleration, respectively. From Fig. 1, it can be seen that $\varphi =\frac{\pi }{2}-\theta -{\theta }_{eq}$, where θ_eq is the angular position relative to vertical when the leg is completely relaxed. The nonlinear disturbance term is denoted as ω(t). The moment of inertia of the shank is $M_m\in {ℝ}^{+}$ and the gravitational term, $M_g\left(\varphi \right)\in {ℝ}^{+}$, is given as M_g (ϕ) = mgl_ccosϕ, where the variables m, g, $l_c\in {ℝ}^{+}$ are the mass of the shank, gravitational acceleration, and length from the knee joint to the center of mass of the shank, respectively. The torque $M_p\left(\varphi ,\dot{\varphi }\right)\in ℝ$ is the joint torque due to passive dynamics due to the musculoskeletal structure. ${\tau }_m\in ℝ$ is the torque of the motor, and ^˙ is the knee extension torque ${\tau }_{ke}\left(\varphi ,\dot{\varphi },a_{ke}\right)\in ℝ$ due to stimulation of the quadriceps muscles.

Fig. 1:

A representative figure depicting the hybrid leg extension neuroprosthesis, where the current, I, is the control input to FES of the quadriceps muscles and τ_m is the electric motor torque.

The passive joint torque can be expressed as a function of the anatomical knee joint angle, ϕ, as [44]

$$M_p\left(\varphi ,\dot{\varphi }\right)=d_1\left(\varphi -{\varphi }_0\right)+d_2\dot{\varphi }+d_3{\text{e}}^{d_4\varphi }-d_5{\text{e}}^{d_6\varphi }$$ (2)

where $d_i\in ℝ\hspace{0.17em}i=1,2,\dots ,6$ and ${\varphi }_0\in ℝ$ are subject specific parameters. The knee extension torque due to stimulation can be expressed as a function of the anatomical knee joint as

$${\tau }_{ke}=\left(c_2{\varphi }^2+c_1\varphi +c_0\right)\left(1+c_3\dot{\varphi }\right)a_{ke}\mu$$ (3)

where $c_j\in ℝ\hspace{0.17em}j=1,2,3$ are subject parameters. The muscle activation, a_ke ∈ [0, 1], in (3) is modeled as [15], [45]

$${\dot{a}}_{ke}=\frac{u_{ke}-a_{ke}}{T_a}$$ (4)

where $T_a\in {ℝ}^{+}$ is the muscle activation time constant. The normalized stimulation amplitude, u_ke ∈ [0, 1] in (4), can be determined from the input constraint characterized by the stimulation current amplitude, $I\in {ℝ}^{+}$, as

$$u_{ke}=\{\begin{array}{ll}0,\hfill & I<I_t\hfill \\ \frac{I-I_t}{I_s-I_t},\hfill & I_t\le I\le I_s\hfill \\ 1,\hfill & I_s<I\hfill \end{array}$$ (5)

where I_t, $I_s\in {ℝ}^{+}$ are the threshold current and saturation currents, respectively.

The muscle fatigue dynamics, µ ∈ [µ_min, 1] in (3), are given as [46]

$$\dot{\mu }=\frac{\left({\mu }_{min}-\mu \right)a_{ke}}{T_f}+\frac{\left(1-\mu \right)\left(1-a_{ke}\right)}{T_r}$$ (6)

where µ_min ∈ (0, 1) is the minimum fatigue level that the muscle can have, $T_f\in {ℝ}^{+}$ is the fatigue time constant, and $T_r\in {ℝ}^{+}$ is the recovery time constant.

Using equations (1), (4), and (6) the ideal dynamics of the hybrid neuroprosthesis system without a disturbance can be expressed in a state-space form as

$$\dot{x}=f\left(x,u\right)=\left[\begin{array}{c}{x}_2\\ \frac{1}{M_m}\left(u_1+{\tau }_{ke}-M_p-M_g\right)\\ \frac{u_2-x_3}{T_a}\\ \frac{\left({\mu }_{min}-x_4\right)x_3}{T_f}+\frac{\left(1-x_4\right)\left(1-x_3\right)}{T_r}\end{array}\right]$$ (7)

where $x={\left[x_1,x_2,x_3,x_4\right]}^T={\left[\varphi ,\dot{\varphi },a_{ke},\mu \right]}^T$ is the system state the states of the system and u = [u₁, u₂]^T = [τ_m, u_ke]^T are the inputs, where $u\in \mathcal{U}=\left\{u|u_{min}\le u\left(t\right)\le u_{max}\right\}$. u_ke is mapped to normalized muscle activation a_ke via (4), and then a_ke is mapped to τ_ke via (3).

Remark 1: The motor torque τ_m can be written as τ_m = k_mI where k_m is a motor constant and I is the current supplied to the motor. But, to simplify the derivation, τ_m is determined directly in this paper. For a multi-degree of freedom case, e.g. a lower limb hybrid exoskeleton, a control allocation matrix [47] maps voltage inputs to FES or electric motor current inputs to joint torques. An optimal control problem, then, can directly optimize these inputs.

The ideal system (7) with disturbance can be expressed as

$$\dot{x}\left(t\right)=f\left(x\left(t\right),u\left(t\right)\right)+w\left(t\right)$$ (8)

where $w\in {ℝ}^4$ is a nonlinear disturbance term in (1) that measures the nominal model state inaccuracy.

The nominal model can be expressed as

$$\dot{\overline{x}}=f\left(\overline{x},\overline{u}\right)$$ (9)

where $\overline{\cdot }$ represents the nominal variable, and this concept will also be used in the reminder of the paper. As the muscle activation and fatigue are not measurable, the estimated values are assumed to be the actual one. Therefore, the disturbance in (8) can be written as w = [w₁, w₂, 0, 0]^T. Further, the following assumptions for the system model are made.

Assumption 1: The inertia M_m is positive and is bounded by known constants as λ_m ≤ M_m ≤ λ_M.

Assumption 2: The term w is bounded, i.e., $w\in \mathcal{W}$, where $\mathcal{W}:=\left\{w\in {ℝ}^4|\Vert w\Vert \le \varpi ,\varpi >0,\forall t\ge 0\right\}$, and its incremental quantity is also bounded as $\Vert \dot{w}\Vert \in {\mathcal{L}}_{\infty }$.

III. Tube-based Model Predictive Control Formulation

A. Nominal Optimization Problem

The nominal optimal control problem is stated as

$$\begin{gathered} \underset{\overline{u}\left(t|t_k\right)}{\min }J\left(x_k,\overline{u}\left(t|t_k\right)\right)=V\left(\Delta \overline{x}\left(t_k+T\right)\right) \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\int }_{t_k}^{t_k+T}l\left(\Delta \overline{x}\left(t\right),\Delta \overline{u}\left(t|t_k\right)\right)dt. \end{gathered}$$ (10)

subject to

$$\dot{\overline{x}}=f\left(\overline{x},\overline{u}\right)$$ (a)

$$x_k\in \overline{x}\left(t_k|t_k\right)\oplus {\Omega }_{\zeta }$$ (b)

$$\overline{u}\in \overline{\mathcal{U}}$$ (c)

$$\Delta \overline{x}\left(t_{k+T}|t_k\right)\in {\Omega }_{\alpha }$$ (d)

In (10), time t satisfies t ∈ [t_k, t_k + T), which is the prediction horizon of MPC, and t_k is the real time and $k={\mathbb{I}}^{+}\cup \left\{0\right\}$. The cost function $J\in {ℝ}^{+}\cup \left\{0\right\}$ is formed by the stage cost function $l=l\left(\Delta \overline{x}\left(t\right),\Delta \overline{u}\left(t\right)\right)$, and the terminal cost penalty $V=V\left(\Delta \overline{x}\left(t_k+T\right)\right)$

$$l=\Delta \overline{x}{\left(t\right)}^{T}Q\Delta \overline{x}\left(t\right)+\Delta \overline{u}{\left(t\right)}^{T}R\Delta \overline{u}\left(t\right)$$

$$V=\Delta \overline{x}{\left(t_k+T\right)}^{T}P\Delta \overline{x}\left(t_k+T\right)$$

where P, $Q\in {ℝ}^{4\times 4}$ and $R\in {ℝ}^{2\times 2}$ are all positive definite and symmetric weight matrices so that l and V are positive definite (PD) and radially unbounded (RU). In the stage cost, $\Delta \overline{x}=x_d-\overline{x}$ where $x_d\in {ℝ}^4$ and $\overline{x}\in {ℝ}^4$ are the desired and nominal states, respectively, and $\Delta \overline{u}=u_d-\overline{u}$, where $u_d\in \overline{\mathcal{U}}$ and $\overline{u}\in \overline{\mathcal{U}}$ are the desired and nominal inputs, respectively, and $\overline{u}\left(t|t_k\right)$ is the nominal input trajectory over prediction horizon staring at time t_k, whose corresponding nominal state trajectory is given by $\overline{x}\left(t|t_k\right)$. The desired input vector u_d is computed from the nominal model to achieve a given desired state x_d. Thus, the pair (x_d, u_d) is the equilibrium point of the system. $\overline{\mathcal{U}}$ denotes the nominal input constraint, i.e., $\overline{\mathcal{U}}=\mathcal{U}\ominus {\mathcal{U}}_F$, where the set $\mathcal{U}$ denotes the actuator’s limits and ${\mathcal{U}}_F$ denotes the set of the input that is required from the feedback controller. The desired trajectory x_d(t), which includes desired angular position and desired angular velocity is designed such that x_d(t), $x_d^{\left(i\right)}\left(t\right)\in {\mathcal{L}}_{\infty }$, where $x_d^{\left(i\right)}\left(t\right)$ denotes the i^th time derivative $\forall i\in {\mathbb{I}}^{+}$. The terminal region, Ω_α is the terminal set, which is defined in the subsequent subsection 3.2. Ω_ζ is the tube region defined using the feedback controller defined in the subsequent subsection 3.3.

Defining the optimal control input trajectory as ${\overline{u}}^{\ast }\left(t|t_k\right)={\overline{u}}^{\ast }$. Afterwards, the first control input ${\overline{u}}^{\ast }\left(t|t:t_k\to t_{k+1}\right)$, where we define $△t=t_{k+1}-t_k$ as a very small value, is applied to the actual system [48], [49] (which will be combined with the feedback control in following section). The corresponding optimal state sequence is defined as ${\overline{x}}^{\ast }\left(t|t:t_k\to t_{k+1}\right)$. The terminal region, Ω_α is the terminal set, which is defined in the subsequent section.

Assumption 3: In (10), the function V, l, are continuous, f is twice continuously differentiable, $u\mapsto l\left(x,u\right)$ is coercive [35], and V (0) = 0, l(0, 0) = 0, f (0, 0) = 0. The set $\overline{\mathcal{U}}$ is compact, uniformly bounded and contains the origin.

Assumption 4: There exist ${\mathcal{K}}_{\infty }$ functions α_l and α_V, so that ${\alpha }_l\left(\Vert \Delta \overline{x}\Vert \right)\le l$, ${\alpha }_V\left(\Vert \Delta \overline{x}\Vert \right)\le V$.

Assumption 5: There exist an optimal input trajectory ${\overline{u}}^{\ast }\left(t|t_0\right)$ for (10) and a non-empty feasible region around (x^∗(t| t₀), u^∗(t| t₀)).

B. Recursive Feasibility and Stability of MPC

To ensure recursive feasibility and constraint compliance, the following steps are proposed [50]. First, the system in (7) can be linearized at the equilibrium point: (x_d, u_d), when the state is near the reference, as

$$\Delta \dot{x}=A\Delta x+B\Delta u$$ (11)

where

$$A={\frac{\partial f\left(x,u\right)}{\partial x}|}_{x=x_d,\hspace{0.17em}u=u_d}\hspace{1em},B={\frac{\partial f\left(x,u\right)}{\partial u}|}_{x=x_d,\hspace{0.17em}u=u_d}.$$

It can be confirmed he linearized dynamics in (11) is controllable. Thus, it meets the condition in [51] to guarantee NMPC stability. A controller gain K for the terminal controller Δu = KΔx such that A + BK is asymptotically stable is determined. Choose a constant $\kappa \in {ℝ}^{+}$ satisfying the inequality η < −λ_max(A + BK) and solve the following Lyapunov equation to determine a positive-definite and symmetric P

$$\begin{gathered} {\left(A+BK+\eta I\right)}^{T}P+P\left(A+BK+\eta I\right) \\ \hspace{0.17em}+Q+K^{T}RK\preccurlyeq 0. \end{gathered}$$ (12)

Find the largest possible α₁ such that $K\Delta x\in \overline{\mathcal{U}}\forall \Delta x\in {\mathrm{\Omega }}_{{\alpha }_1}$, where ${\mathrm{\Omega }}_{{\alpha }_1}:=\left\{\Delta x\in {ℝ}^n|\Delta x^{T}P\Delta x\le {\alpha }_1\right\}$ Then, find the largest possible α ∈ (0, α₁] such that

$$L_{\varphi }\le \frac{\kappa {\lambda }_{min}\left(P\right)}{\Vert P\Vert }$$

where $L_{\varphi }=sup\left\{\frac{\Vert \varphi \left(\Delta x\right)\Vert }{\Vert \Delta x\Vert }|\Delta x\in {\mathrm{\Omega }}_{\alpha },\Delta x\ne 0\right\}$ and ϕ(x) = f (Δx, KΔx) − (A + BK)Δx. Then, using Lemma 1 in [50], one can show that

$$\dot{V}+l\le 0$$ (13)

which implies that Δu = KΔx is invariant in Ω_α and satisfies the input constraints.

Assuming ${\overline{u}}^{\ast }\left(t|t_k\right)$ exists for t ∈ [t_k, t_k + T], the next feasible solution for t ∈ [t_k+1, t_k+1 + T] is constructed as

$$\widehat{u}\left(t|t_{k+1}\right)=\{\begin{array}{cc}{\overline{u}}^{\ast }\left(t|t_k\right),\hfill & t\in \left[t_{k+1},t_k+T\right)\hfill \\ {\kappa }_N\left(\Delta \overline{x}\left(t|t_{k+1}\right)\right),\hfill & t\in \left[t_k+T,t_{k+1}+T\right)\hfill \end{array}$$ (14)

where ${\kappa }_N\left(\Delta \overline{x}\left(t|t_{k+1}\right)\right)=u_d-K\Delta \overline{x}\left(t|t_{k+1}\right)$.

Theorem 1. The MPC algorithm is (i) recursively feasible and (ii) asymptotically stable under the actual optimal control sequence ${\overline{u}}^{\ast }$, if ${\overline{u}}^{\ast }\left(t|t_0\right)$ exists for t ∈ [t₀, t₀ + T].

Proof: (i) Recursive Feasibility: Let’s assume that ${\overline{u}}^{\ast }\left(t|t_k\right)$ exists for t ∈ [t_k, t_k + T]. When applying this sequence to the nominal system, the tracking error of the nominal system is driven into the terminal region Ω_α, i.e., $\Delta \overline{x}\left(t_k+T|t_k\right)\in {\Omega }_{\alpha }$. During the algorithm implementation, the first part of the optimal control sequence + the subsequently designed feedback controller is applied to the actual system over [t_k, t_k+1), and its state measurement at time t_k+1 satisfies $x_{k+1}\in \overline{x}\left(t_{k+1}|t_{k+1}\right)\oplus {\Omega }_{\zeta }$. This implies that $\overline{x}\left(t_{k+1}|t_k\right)$ is a feasible initial state for the optimization problem. Therefore, to solve the open-loop optimization problem at t_k+1with this initial condition, the next feasible control sequence for t ∈ [t_k+1, t_k+1 + T] is constructed as in (14). Due to the invariance of the terminal region Ω_αwith respect to the terminal controller, ${\kappa }_N\left(\Delta \overline{x}\left(t|t_k\right)\right)$, $\Delta \overline{x}\left(t_k+T|t_k\right)$ implies $\Delta \overline{x}\left(t_{k+1}+T|t_{k+1}\right)\in {\text{Ω}}_{\alpha }$. Then, combined with Assumption 5, the recursive feasibility can be achieved by induction.

(ii) Asymptotic Stability: According to (10), the cost under ${\overline{u}}^{\ast }\left(t|t_k\right)$ is

$$\begin{gathered} J\left(x_k,{\overline{u}}^{\ast }\left(t|t_k\right)\right)=V\left(\Delta \overline{x}\left(t_k+T\right)\right) \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\int }_{t_k}^{t_k+T}l(\Delta \overline{x}\left(t\right),{\overline{u}}^{\ast }\left(t|t_k\right)dt \end{gathered}$$ (15)

and the cost under $\widehat{u}\left(t|t_{k+1}\right)$ is

$$\begin{gathered} \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}J\left(x_{k+1},\widehat{u}\left(t|t_{k+1}\right)\right) \\ =V\left(\Delta \overline{x}\left(t_{k+1}+T\right)\right) \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\int }_{t_{k+1}}^{t_{k+1}+T}l\left(\Delta \overline{x}\left(t\right),\widehat{u}\left(t|t_{k+1}\right)\right)dt \end{gathered}$$ (16)

Subtracting (15) from (16) and using (14) yields

$$\begin{gathered} \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em} J\left(x_{k+1},\widehat{u}\left(t|t_{k+1}\right)\right)-J\left(x_k,{\overline{u}}^{\ast }\left(t|t_k\right)\right) \\ =V\left(\Delta \overline{x}\left(t_{k+1}+T\right)\right)-V\left(\Delta \overline{x}\left(t_k+T\right)\right) \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em} +{\int }_{t_k+T}^{t_{k+1}+T}l\left(\Delta \overline{x}\left(t\right),{\kappa }_N\left(\Delta \overline{x}\left(t|t_{k+1}\right)\right)\right)dt \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em} -{\int }_{t_k}^{t_{k+1}}l\left(\Delta \overline{x}\left(t\right),\Delta {\overline{u}}^{\ast }\left(t|t_k\right)\right)dt. \end{gathered}$$

After integrating (13), from [t_k+T t_k+T+1], it becomes

$$\begin{gathered} V\left(\Delta \overline{x}\left(t_{k+1}+T\right)\right)-V\left(\Delta \overline{x}\left(t_k+T\right)\right) \\ +{\int }_{t_k+T}^{t_{k+1}+T}l\left(\Delta \overline{x}\left(t\right),{\kappa }_N\left(\Delta \overline{x}\left(t|t_{k+1}\right)\right)\right)dt\le 0. \end{gathered}$$

Therefore,

$$\begin{gathered} \hspace{0.17em}\hspace{0.17em} J\left(x_{k+1},\widehat{u}\left(t|t_{k+1}\right)\right)-J\left(x_k,{\overline{u}}^{\ast }\left(t|t_k\right)\right) \\ \le -{\int }_{t_k}^{t_{k+1}}l\left(\Delta \overline{x}\left(t\right),\Delta {\overline{u}}^{\ast }\left(t|t_k\right)\right)dt\le 0. \end{gathered}$$ (17)

Because the optimal cost satisfies [52]

$$J\left(x_{k+1},{\overline{u}}^{\ast }\left(t|t_{k+1}\right)\right)\le J\left(x_{k+1},\widehat{u}\left(t|t_{k+1}\right)\right),$$

thus, from (17), we can conclude the non-increasing property of the cost

$$J\left(x_{k+1},{\overline{u}}^{\ast }\left(t|t_{k+1}\right)\right)\le J(x_k,{\overline{u}}^{\ast }\left(t|t_k\right).$$

By applying the Barbalat’s lemma, the error asymptotically approaches zero. ■

C. Feedback Controller

To eliminate the disturbance between the actual and the nominal state, a nonlinear feedback controller is designed. The nominal dynamics is expressed as

$$M_m\ddot{\overline{\varphi }}\left(t\right)+M_p\left(\overline{\varphi },\dot{\overline{\varphi }}\right)+M_g\left(\overline{\varphi }\right)-{\overline{\tau }}_{ke}={\overline{\tau }}_m$$ (18)

where $\overline{\varphi }$ denotes the nominal angle computed by MPC, i.e., ${\overline{x}}_1$, while ${\overline{\tau }}_m$ denotes the nominal (optimal) motor control input, i.e., $u_1^{\ast }$ and

$${\overline{\tau }}_{ke}\left(\overline{\varphi },\dot{\overline{\varphi }},{\overline{a}}_{ke},\overline{\mu }\right)=\left(c_2{\overline{\varphi }}^2+c_1\overline{\varphi }+c_0\right)\left(1+c_3\dot{\overline{\varphi }}\right){\overline{a}}_{ke}\overline{\mu }$$ (19)

denotes the nominal (optimal) FES control torque, where ${\overline{a}}_{ke}$ and $\overline{\mu }$ are constant (nominal) muscle activation and fatigue during the sampling period. The motor torque τ_m constitutes the nominal torque input from MPC and a feedback controller to reduce the nominal knee joint angle and the actual knee joint angle. Let the feedback control law is defined as $u_F={\tau }_m-{\overline{\tau }}_m$. The dynamics in (1) can be rewritten as

$$M_m\ddot{\varphi }\left(t\right)+M_p\left(\varphi ,\dot{\varphi }\right)+M_g\left(\varphi \right)+\omega -{\tau }_{ke}-{\overline{\tau }}_m=u_F.$$ (20)

Let us define the error, $e\in ℝ$, between the nominal knee joint angle and the actual knee joint angle

$$e=\overline{\varphi }-\varphi .$$

By introducing a constant $\alpha \in ℝ$, an auxiliary error, $r\in ℝ$, is defined as

$$r\triangleq \dot{e}+\alpha e.$$ (21)

On multiplying M_m with the time derivative of (21) and after substituting (20) and (18), we obtain

$$M_m\dot{r}=\tilde{N}+\omega -u_F$$ (22)

where $\tilde{N}$ is defined as

$$\begin{gathered} \tilde{N}=M_p\left(\varphi ,\dot{\varphi }\right)-M_p\left(\overline{\varphi },\dot{\overline{\varphi }}\right)+M_g\left(\varphi \right)-M_g\left(\overline{\varphi }\right) \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}+{\overline{\tau }}_{ke}\left(\overline{\varphi },\dot{\overline{\varphi }},{\overline{a}}_{ke},\overline{\mu }\right)-{\tau }_{ke}\left(\varphi ,\dot{\varphi },a_{ke},\mu \right)+\alpha M_{m}r-{\alpha }^2{M}_{m}e. \end{gathered}$$

Because, a_ke and µ in (4) and(6), respectively, are bounded normalized variables, then $\Vert {\overline{\tau }}_{ke}\left(\overline{\varphi },\dot{\overline{\varphi }},{\overline{a}}_{ke},\overline{\mu }\right)\Vert \le \Vert \left(c_2{\overline{\varphi }}^2+c_1\overline{\varphi }+c_0\right)\left(1+c_3\dot{\overline{\varphi }}\right)\Vert$, therefore, the term $\tilde{N}$ in (22) can be upper bounded as

$$\Vert \tilde{N}\Vert \le \rho \left(\Vert \sigma \Vert \right)\Vert \sigma \Vert$$ (23)

where $\rho \left(\Vert \sigma \Vert \right)\in ℝ$ is a positive monotonic bounded function, and $\sigma ={\left[\begin{array}{cc}{r}^T& e^T\end{array}\right]}^T$.

Based on the subsequent stability analysis a following control law is designed

$$u_F=\rho \left(\Vert \sigma \Vert \right)\Vert \sigma \Vert \text{sat}\left(\frac{r}{\epsilon }\right)+\varpi \text{sat}\left(\frac{r}{\epsilon }\right)+\kappa r$$ (24)

where $\text{sat}\left(\frac{r}{\epsilon }\right)$ is defined as

$$\text{sat}\left(\frac{r}{\epsilon }\right)=\{\begin{array}{cc}\frac{r}{\left|r\right|}& \left|r\right|\ge \epsilon \\ \frac{r}{\epsilon }& \left|r\right|<\epsilon \end{array}$$

and κ, $\epsilon \in ℝ$ are gains. On substituting (24) into (22),

$$M_m\dot{r}=\tilde{N}+\omega -\rho \left(\Vert \sigma \Vert \right)\Vert \sigma \Vert \text{sat}\left(\frac{r}{\epsilon }\right)-\varpi \text{sat}\left(\frac{r}{\epsilon }\right)-\kappa r.$$ (25)

Theorem 2. The feedback control law $u_F\in {\mathcal{U}}_F$ in (24) makes the closed loop system in (25) exponentially enters a region ${\Omega }_{\zeta }=\left\{\zeta |\Vert \zeta \Vert \le D=\epsilon \sqrt{\left(1+\frac{1}{{\alpha }^2{\gamma }^2}\right)\frac{{\lambda }_M}{{\lambda }_m}}+\epsilon \sqrt{\frac{{\lambda }_M}{{\lambda }_m{\gamma }^2}}\right\}$, where $\zeta =\overline{x}-x$, and ${\mathcal{U}}_F$ is bounded as ${\mathcal{U}}_F=\left\{u|\Vert u\Vert \le \rho \left({\varsigma }_0\varpi \right){\varsigma }_0\varpi +{\varsigma }_1\varpi \right\}$, where ς₀ and ${\varsigma }_1\in ℝ$ are defined as ${\varsigma }_0=\sqrt{{\left(1+\alpha \right)}^2+1}$, ${\varsigma }_1=\left(1+\kappa +\alpha \kappa \right)$. Thus the actual state is asymptotically ultimately bounded.

Proof: A positive definite Lyapunov function candidate $L_1\left(r\right):\mathbb{D}\to ℝ$ is defined as

$$L_1=\frac{1}{2}{M}_m{r}^2$$ (26)

where L₁, due to Assumption 1, satisfies

$$\frac{1}{2}{\lambda }_m{r}^2\le L_1\le \frac{1}{2}{\lambda }_M{r}^2.$$

The time derivative of (26) can be expressed as

$${\dot{L}}_1=M_m\dot{r}r$$ (27)

on substituting (25) to (27)

$${\dot{L}}_1=\tilde{N}r+\omega r-\rho \left(\Vert \sigma \Vert \right)\Vert \sigma \Vert \text{sat}\left(\frac{r}{\epsilon }\right)r-\varpi \text{sat}\left(\frac{r}{\epsilon }\right)r-\kappa r^2$$

according to (23)

$$\begin{gathered} {\dot{L}}_1\le \rho \left(\Vert \sigma \Vert \right)\Vert \sigma \Vert \left|r\right|+\left|\omega \right|\left|r\right| \\ \hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em} -\rho \left(\Vert \sigma \Vert \right)\Vert \sigma \Vert \text{sat}\left(\frac{r}{\epsilon }\right)r-\varpi \text{sat}\left(\frac{r}{\epsilon }\right)r-\kappa r^2. \end{gathered}$$

When |r| ≥ ε, we can obtain

$${\dot{L}}_1\le -\kappa r^2<0$$ (28)

so that the auxiliary error r would enter an invariant set in finite time [53]

$${\Omega }_r=\left\{r|\left|r\right|\le \epsilon \sqrt{\frac{{\lambda }_M}{{\lambda }_m}}\right\}.$$ (29)

In order to investigate behavior of e a positive definite Lyapunov function candidate $L_2\left(r\right):\mathbb{D}\to ℝ$ is defined

$$L_2=\frac{1}{2}{e}^2.$$ (30)

By using the definition in (21), its time derivative can be written as

$${\dot{L}}_2=-\alpha e^2+er.$$ (31)

Further using the definition in (29), (31) can be upper bounded by

$${\dot{L}}_2\le -\alpha {\left|e\right|}^2+\epsilon \sqrt{\frac{{\lambda }_M}{{\lambda }_m}}\left|e\right|.$$ (32)

For $\left|e\right|\ge \epsilon \sqrt{\frac{{\lambda }_M}{{\lambda }_m}}/\alpha \gamma$, where γ ∈ (0, 1), (31) can be written as

$${\dot{L}}_2\le -\left(1-\gamma \right)\alpha e^2-\left(\alpha \gamma \left|e\right|-\epsilon \sqrt{\frac{{\lambda }_M}{{\lambda }_m}}\right)\left|e\right|$$

which can be further bounded as

$${\dot{L}}_2\le -\left(1-\gamma \right)\alpha e^2<0$$

and e will finally enter an invariant set

$${\Omega }_e=\left\{e|\left|e\right|\le \epsilon \sqrt{\frac{{\lambda }_M}{{\lambda }_m}}/\alpha \gamma \right\}.$$ (33)

Because $\sigma ={\left[\begin{array}{cc}{r}^T& e^T\end{array}\right]}^T$, combining with Ω_r and Ω_e, it can be directly obtained that σ exponentially enters ${\Omega }_{\sigma }=\left\{\sigma \Vert \sigma \Vert \le \epsilon \sqrt{\left(1+\frac{1}{{\alpha }^2{\gamma }^2}\right)\frac{{\lambda }_M}{{\lambda }_m}}\right\}$. We can also see that $\zeta =\overline{x}-x={\left[e,\dot{e},0,0\right]}^T$, therefore, ζ exponentially enters ${\Omega }_{\zeta }=\left\{\zeta |\Vert \zeta \Vert \le D=\epsilon \sqrt{\left(1+\frac{1}{{\alpha }^2{\gamma }^2}\right)\left(\frac{{\lambda }_M}{{\lambda }_m}\right)}+2\epsilon \sqrt{\frac{{\lambda }_M}{{\lambda }_m{\gamma }^2}}\right\}$.

Below, we compute bounds on the controller to handle input constraints in the MPC formulation in (10)-d. After subtracting (8) from (9), and considering $u=\overline{u}$ and because of $\overline{x}\left(t_k\right)=x\left(t_k\right)$, we can obtain $\Vert {\overline{x}}_{k+1}-x_{k+1}\Vert \le \varpi$. Therefore, $\Vert \zeta \left(k\right)\Vert \le \varpi$, $\Vert r\left(k\right)\Vert \le \left(1+\alpha \right)\varpi$ and $\Vert \sigma \left(k\right)\Vert \le \sqrt{{\left(\varpi +\alpha \varpi \right)}^2+{\varpi }^2}$. Combining with the control law (24), we can write

$${\mathcal{U}}_F=\left\{u|\Vert u\Vert \le \rho \left({\varsigma }_0\varpi \right){\varsigma }_0\varpi +{\varsigma }_1\varpi \right\}.$$ (34)

where ς₀ and ${\varsigma }_1\in ℝ$ are defined as ${\varsigma }_0=\sqrt{{\left(1+\alpha \right)}^2+1}$, ς₁ =(1+ κ + ακ).

Using Theorem 1, the nominal state is asymptotically stable and using Theorem 2, ζ is ultimately bounded. Therefore, the actual state is asymptotically ultimately bounded. It is demonstrated in Fig. 2. ■

Fig. 2:

This figure shows the tube along real time horizon, which constrains the distance between the actual and optimal state. For simplicity it only shows two dimensions.

Remark 2: Usually, in any MPC problem there should be a feasible region around the nominal trajectory $\left\{{\overline{x}}^{\ast }\left(t_{k+1}|t_k\right)\right\}\forall k\in {\mathbb{I}}^{+}\cup \left\{0\right\}$. The aim of adding the feedback controller is to drive the actual state to enter that region. Also, we can conclude that the actual state is bounded around the nominal trajectory, i.e., $\left\{x\left(t_{k+1}|t_{k+1}\right))\right\}=\left\{{\overline{x}}^{\ast }\left(t_{k+1}|t_k\right)\right\}\oplus {\Omega }_{tube}\hspace{0.17em}\forall k\in {\mathbb{I}}^{+}\cup \left\{0\right\}$ where ${\Omega }_{tube}=\left\{w_d\in {ℝ}^4|\left|w_d\right|\le {\left[D,D,0,0\right]}^T\right\}$.

IV. Fast Optimization Algorithm Implementation

A. Terminal Constraint vs Terminal Cost Approach

In this paper, a terminal cost approach is suggested to be adopted for a real-time application case. In practice, a terminal constraint approach may cause a heavy computational load [48], [54]–[58]. Alternatively, the terminal cost approach, which is to select the terminal cost (P matrix in our case), can be used to make the terminal constraint hold implicitly [35].

Simulations that compares the two approaches was run using fmincon (MATLAB®) to validate that the terminal cost approach has a performance similar to the terminal constraint approach. In the terminal constraint approach, we computed α₁ so that in ${\Omega }_{{\alpha }_1}$ the input constraints are satisfied. Then we proceeded to compute α, but the value was too small [50] so it was hard to find a feasible solution to satisfy this harsh constraint. Therefore, instead of further computing α, the inequality in (13) is set to be a hard constraint during the optimization. In the second case, the terminal cost was solved from the Riccati equation [15], [57]. The results in Fig. 3 show the performance of the two approaches. It can be seen that the two cases have similar performances.

Fig. 3:

Validation of the MPC method with terminal cost.

B. Algorithm Implementation

A fast gradient-search algorithm [15], [57] is recruited as the optimization solver. Given the definition of cost function in (10) and dynamics in (7), the Hamiltonian in continuous version is

$$H\left(x,\lambda ,u\right)\triangleq l\left(x,u\right)+{\lambda }^{T}f\left(x,u\right)$$ (35)

where $\lambda \left(\tau \right)\in {ℝ}^4$ is the costate vector. With Gradient projection algorithm, the optimal solution at each time instant is obtained by minimizing the Hamiltonian with the necessary conditions [59].

With the algorithm, the initial input trajectory for every prediction horizon is updated along the dynamical constraint towards the direction of decreasing the running cost. A suboptimal solution is found when the search has found a local minimum, where the dynamical constraint is tangent to the cost function field projected on t. If the point, where the dynamical constraint and the cost function field are tangent, is not found, the search would stop on the boundary of the input authority until the optimum is found. The details of the algorithm are given in (I).

V. Experiments

A. Experimental Setup

The experiments were performed using the modified leg extension machine shown in Fig. 1. The modified leg extension machine has an encoder (Hengxiang, CN) with 1024 pulses per revolution resolution to measure the joint angle. A RehaStim 8-channel stimulator (Hasomed Inc., DE) was used to generate the biphasic pulse train, with 30Hz frequency and 400µs pulse-width, to stimulate the muscles via surface electrodes. A Harmonic Drive FHA-Mini-14C-100 electric motor (Harmonic Drive, US) combined with a gear system (gear ratio: 3:8) was used to generate torque that drives the knee angle.

A participant without a disability (S1) and a participant with Level T10 SCI (S2) participated in the study. Approval for the study was obtained from the Institutional Review Board of the University of Pittsburgh. Informed consent was obtained from both participants, who also passed the preliminary screening experiments for physical and mental demand. In the screening experiments, the participant’s muscle response to FES and pain perception to FES were verified. The participant would not move on to later experiments if there was no muscle response to FES (e.g., in some cases of SCI) or if she/he did not feel comfortable with FES. A participant who passed the screening can participate in system identification, fatigue parameter identification, and controller validation experiments. A set of 6 separate tests were performed for the dynamic model parameter identification, including the fatigue model, on a leg of each participant [15], [20], [44]. The details of these tests are provided in Appendix.

The dynamical system identification (without fatigue test) took as few as 40 minutes to up to 60 minutes per leg. This time varied a bit in some cases because of test preparation, retest of some tasks, parameter tuning, and other unanticipated events. Fatigue parameter estimation is a separate test in which the fatiguing protocol ran between 1–2 minutes to fatigue the muscle. The fatigue test can drive the recruited (superficial) muscles to their physical limits because the muscles cannot generate torques under the FES when they are completely fatigued. After the dynamical system identification, we allow the participant to rest for 15–30 minutes, before we repeat the procedure on the other leg. The controller validation experiments are performed on a separate day with at least a one day gap. This allowed the muscle to fully recover from fatigue after a day of rest and ready for controller validation experiments. In total, the time it took to perform the screening test, system identification, control validation, can be as few as 60 minutes to up to 90 minutes per leg of a participant. During all the tests, the participants were asked not to provide any volitional effort. These tests, although thorough, did not result in any dropout due to physical or mental demand required. The participants were allowed to pause the procedures, any time, when uncomfortable. All the consented participants completed all the procedures.

B. Control Performance

The sampling rate for the feedback controller was set to be 0.002s while the MPC was 0.01s. Runge-Kutta-4 method is used to integrate the discretized dynamical system for the controller. The control loop is shown in Fig. 4, and the controlled system is shown in Fig. 5.

Fig. 4:

Overview of the designed control system.

Fig. 5:

Experimental set-up consisted of a custom-made hybrid leg extension machine comprising an electric motor and a stimulator for FES

Two sets of experiments were performed: 1) regulate knee angle to 40 degrees solely by MPC and 2) regulate knee angle to 40 degrees by MPC with the assistance of the feedback controller. The weight matrices, Q, R, and P were kept same in both sets of experiments. Control performance shown in Fig. 6, demonstrates the effectiveness of the feedback controller. Fig. 7 shows how the actuators are allocated. It also shows the motor torque provided by the feedback controller.

Fig. 6:

This figure shows the control performance of MPC with and without the feedback controller.

Fig. 7:

The figures plot the control inputs.

VI. Conclusion

In this paper, a robust tube-based NMPC method is proposed to control a hybrid neuroprosthesis system to achieve a knee angle regulation task in the presence of disturbances and model uncertainties. In this method, MPC solves the optimal control sequence that balances control performance and effort, and allocate control efforts (an electric motor and FES). Experimental results on a participant without disability and a participant with SCI validate this method. This paper shows that a nonlinear feedback control method can be incorporated with an MPC method to overcome a model disturbance.

Fig. 13:

The results from parameter Test 5 for both participants. A sinusoidal stimulation was applied to the quadriceps and the resulting joint angle was measured. This plot shows the measured joint angle and the joint angle predicted by the model with the estimated parameters.

TABLE I:

Flow chart of the gradient projection algorithm.

1 Initialization: j = 0 (a) Choose initial control trajectory ${\overline{u}}_k^{\left(0\right)}\left(t\right)\in \mathcal{U}$, where t ∈ [tk, tk + T). (b) Solve for ${\overline{x}}_k^{\left(0\right)}\left(t\right)$ through the model where t ∈ [tk, tk + T).

2 Gradient Step: (d)Solve for the costates λ(j)(t) $\dot{\lambda }\left(t\right)={-\frac{\partial H\left(x,\lambda ,u\right)}{\partial x}|}_{x={\overline{x}}_k^{\left(j\right)},u={\overline{u}}_k^{\left(j\right)}}$ given ${\lambda }_k^{\left(j\right)}\left(t_k+T\right)={\frac{\partial V\left({\overline{x}}_k^{\left(j\right)}\left(t\right)\right)}{\partial \overline{x}\left(t\right)}|}_{t=t_k+T}.$ (e) Compute the search direction, $s_k^{\left(j\right)}\left(t\right)$, from the Hamiltonian $s_k^{\left(j\right)}\left(t\right)=-{\frac{\partial H\left(x,\lambda ,u\right)}{\partial u}|}_{x={\overline{x}}_k^{\left(j\right)},\hspace{0.17em}u={\overline{u}}_k^{\left(j\right)},\hspace{0.17em}\lambda ={\lambda }_k^{\left(j\right)}}.$ (f) Compute the step size, ${\alpha }_k^{\left(j\right)}$ ${\alpha }_k^{\left(j\right)}=\underset{\alpha >0}{\text{argmin}}\hspace{0.17em}J\left(x_k^{\left(j\right)},\psi \left({\overline{u}}_k^{\left(j\right)}+\alpha s_k^{\left(j\right)}\right)\right).$ (g) Compute the new control trajectory ${\overline{u}}_k^{\left(j+1\right)}\left(t\right)=\psi \left({\overline{u}}_k^{\left(j\right)}+{\alpha }_k^{\left(j\right)}{s}_k^{\left(j\right)}\right),$ ψ(·) is defined as the input constraint. (h) Solve for ${\overline{x}}_k^{\left(j+1\right)}\left(t\right)$ given ${\overline{u}}_k^{\left(j+1\right)}\left(t\right)$, (i) Evaluate the cost function $J\left(x_k^{\left(j+1\right)},{\overline{u}}_k^{\left(j+1\right)}\right)$. (j) Check Quit Conditions (j1) if $\left|J\left(x_k^{\left(j+1\right)},{\overline{u}}_k^{\left(j+1\right)}\right)-J\left(x_k^{\left(j\right)},{\overline{u}}_k^{\left(j\right)}\right)\right|$ is sufficient small or j exceeds the limitation iteration — quit (j2) set j = j + 1 — go back to (d).

Appendix

In this Appendix, the details of the model parameter identification are provided:

Test 1.

The saturation and threshold current amplitudes (I_s and I_t) were tested in this procedure. The current amplitude that produced the first significant contraction was set to be threshold, and the current amplitude that produced the last significant torque increase was defined as the current amplitude. The results are shown in Fig. 8.

Fig. 8:

The results from parameter Test 1 for both participants. Stimulation current amplitude ramp to determine the threshold current amplitude, I_t, and saturation current amplitude, I_s of the participant. The threshold is the current amplitude that causes the first significant torque measurement, and the saturation is the current amplitude that produced the last significant change in the torque measurement.

Test 2.

By holding the leg of the participant at different joint angles in the leg extension machine, the passive joint torque and gravitational torque can be measured by using the load cell. The results of these measurements can be used to estimate the passive stiffness (d_i for i = [1, 3, 4, 5, 6] and θ_eq) and mass parameters (m and l_c) of the participant. Each data point was measured at least twice and the average value was taken. A nonlinear, least-squares curve fitting algorithm was used to determine parameters that resulted in a best fit between the average data and the function of the passive knee torque, τ_p. The measured data and the resulting best fit determined by the nonlinear, least-squares curve fitting algorithm are shown in Fig. 9.

Fig. 9:

The results from parameter Test 2 for both participants. The exponential terms that model hyperextension and hyperflexion of the anatomical joint angles (ϕ) can be observed around 0° and 85°, respectively.

Test 3.

Isometric contraction torque was obtained by stimulating at the saturation level (from Test 1) for 2s at a number of different joint angles. The purpose of this test is to collect data that may be used to determine the torque-length parameters (c_i for i = [0 − 2]). Like Test 2, a nonlinear, least squares curve fitting algorithm was used to determine torque-length parameters that resulted in a best fit between the measured torque and joint angle at the different positions. The isometric contraction torques were measured at 7 different joint positions, and the best fit to the measured data are shown in Fig. 10.

Fig. 10:

The results from parameter Test 3 for both participants. Torques produced during isometric contraction tests at different anatomical joint angles (ϕ), and the best fit of the measured data to find the torque-angle characteristics.

One of the isometric contraction tests from Test 3 was used to determine the muscle activation time constant (T_a). Because the leg is fixed in an isometric configuration and the muscle was stimulated at the saturation level, which corresponds to a normalized stimulation of 1, the normalized joint torque is equivalent to the normalized muscle activation. The normalized joint torque was measured using the load cell for a step input of stimulation at the saturation level. Since the normalized joint torque measured by the load cell is equivalent to the normalized muscle activation under these conditions, the normalized load cell measurement was used as an approximate measurement of the first order muscle activation dynamics. The normalized muscle activation from an isometric contraction is shown in Fig. 11, where the stimulation begins at 1s.

Fig. 11:

Muscle activation time constant results from parameter Test 3 for both participants. The normalized load cell data were used to determine the muscle activation time constant. The first order system time constant was found by a response that best fits the normalized measured data.

Test 4.

Pendulum tests were run for determining the damping and inertial parameters of the system (d₂ and J). This was done by holding the leg at approximately full extension, then releasing it and letting it drop freely. An optical encoder mounted on the leg extension machine at the knee joint was used to measure the response of the leg. Then an optimization was used to determine the best fit damping and inertial parameters. The measured encoder data from the pendulum test and the response from the best fit model are shown together in Fig. 12. The discrepancy between the measured data and fit was also contributed by the previously determined parameters in Test 2.

Fig. 12:

The results from parameter Test 4 for both participants. Plot of the results of the pendulum test with the response of the model that best fits the measured response.

Test 5.

The purpose of this test is to estimate the force-velocity parameter, c₃. Movement of the knee joints was elicited by applying a sinusoidal stimulation, with a period of 8s, to the quadriceps muscles of the participant. This was measured by using the encoder mounted on the leg extension machine. The amplitude of the stimulation was selected such that for each participant the joint angle was between 10° and 70°. This ensured the muscles to be always in tension and sufficiently far from hyperextension/hyperflexion. The parameters estimated in Tests 1–5 were used to populate the model of knee extension, and then an optimization was performed to identify the force-velocity parameter that makes the model best match the measured data. Fig. 10 compares the measured knee joint angle to the knee joint angle of the model when given the same input.

Test 6.

The purpose of the following two part test is to estimate the parameters of the muscle fatigue dynamics (µ_min, T_f, and T_r). First a constant stimulation with an amplitude equal to the saturation is used to fatigue the muscles. Assuming that muscle activation reaches steady-state relatively quickly, and because the stimulation amplitude is equal to the saturation amplitude (i.e. a_ke ≈ u_ke = 1) the fatigue dynamics can be reduced to $\dot{\mu }.=\frac{1}{T_f}\left({\mu }_{min}-\mu \right)$. Assuming that the muscle is initially unfatigued at the start of the test (implies that µ(0) = 1) the solution to the reduced dynamics is $\mu \left(t\right)={\mu }_{min}-\left({\mu }_{min}-1\right)e^{-t/T_f}$.

After the muscles are fatigued, 0.5s pulse trains with an amplitude that is equal to the saturation were used every ten seconds to measure the rate at which the muscles were recovering. Assuming that the stimulation pulse trains were sufficiently short during recovery (a_ke ≈ 0) the muscle fatigue dynamics can be reduced to $\dot{\mu }=\frac{1}{T_r}\left(1-\mu \right)$, whose solution is $\mu \left(t\right)=1+\left({\mu }_r-1\right)e^{-t/T_r}$ where µ_r is the initial condition that is measured from the first contraction during the recovery period. A least-squares nonlinear curve fitting algorithm was used to solve for the parameters µ_min, T_f, and T_r that best fit the time responses of the muscle during fatigue and recovery to the normalized load cell measurements from the fatigue and recovery portions of the test. The normalized load cell data and the plot of the fatigue state that best fits the measured data during fatigue and recovery are shown in Fig. 14.

Fig. 14:

This figure demonstrates the results of the muscle fatigue and recovery parameter estimation test from Test 6. In 14b, recovery curves were fitted based on the peaks, which were extracted from the isometric torque measurements.

The parameters are shown in the Table II.

TABLE II:

Musculoskeletal parameters estimated for the right and left legs of each participant.

	S1	S2
Parameter	Value	Value
It [mA]	21.20	20.00
Is [mA]	60.00	70.00
m [kg]	4.68	2.59
lc [m]	0.37	0.16
J [kg m2]	0.17	0.24
θeq [rads]	1.19	1.24
d1 [Nm]	2.66×10−14	11.01
d2 [Nm]	1.69	2.01
d3 [Nm]	1.64	8.22×10−9
d 4	1.59	15.40
d5 [Nm]	0.73	0.99
d 6	−39.78	−21.27
ϕ0 [rads]	0.01	0.85
c0 [Nm]	78.78	30.37
c1 [Nm]	55.76	9.58
c2 [Nm]	−49.02	−17.49
c3	1.44	0.90
Ta [sec]	0.26	0.50
µ min	1.95×10−9	2.55×10−10
Tf [sec]	29.17	24.89
Tr [sec]	48.09	71.46